WO2011089884A1 - 画像分類装置、方法、プログラム、プログラムを記録する記録媒体及び集積回路 - Google Patents
画像分類装置、方法、プログラム、プログラムを記録する記録媒体及び集積回路 Download PDFInfo
- Publication number
- WO2011089884A1 WO2011089884A1 PCT/JP2011/000235 JP2011000235W WO2011089884A1 WO 2011089884 A1 WO2011089884 A1 WO 2011089884A1 JP 2011000235 W JP2011000235 W JP 2011000235W WO 2011089884 A1 WO2011089884 A1 WO 2011089884A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- face
- image group
- group
- information
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Definitions
- the present invention relates to an image classification device for classifying images.
- Digital image capturing devices such as digital still cameras and mobile phones with camera functions have become widespread, and recording media such as hard disks for recording captured images have been provided at low cost.
- a user of a digital image capturing device stores each captured image in a recording medium such as a large-capacity hard disk.
- each image is classified into several categories for the purpose of facilitating the user's search for images. There are things to do.
- Patent Document 1 As a technique for classifying an image, for example, as described in Patent Document 1 and Patent Document 2, a technique for extracting features of the image from one image and classifying the image using the extracted features
- Patent Document 3 As described in Patent Document 3, for example, a technique for classifying an image from information on the shooting time of the image is known.
- the image has different features from other images belonging to the image group including the image.
- the classification destination of the image may be a different category from the classification destination of other images belonging to the image group.
- a scene playing on the banks of the river was shot in an image group consisting of images taken when going to the picnic.
- an image only an image of a scene playing on the bank of the river is classified into a river fishing category, and other images are classified into a picnic category.
- An object of the present invention is to provide an image classification device capable of classifying images.
- the image classification device is based on image feature information indicating image characteristics of all or part of two or more images belonging to one image group. Based on the image group feature calculation unit for calculating the image group feature information indicating the feature of the image group, the image group feature information of one image group, and the reference information for classification, the image group is classified into a plurality of different ones. And an image group classification unit for classifying into any one of the classification destinations.
- An image classification apparatus having the above-described configuration classifies images in units of image groups based on image group feature information calculated based on image features that are not limited to information on image capturing times. Can do.
- the image classification device can classify images based on image characteristics other than the shooting time so that images belonging to the same image group are not classified into different categories.
- Hardware block diagram showing hardware configuration of image classification apparatus 100 Functional block diagram showing a functional configuration of the image classification device 100
- Directory structure diagram showing directory structure of image storage unit 231 Data structure diagram of image feature information stored in image feature information storage unit 232
- Data structure diagram of image group feature information stored in image group feature information storage unit 233 Data structure diagram of event feature information stored in event feature information storage unit 234
- Flowchart of image group classification operation performed by image classification apparatus 100 Diagram showing images belonging to an image group Data structure diagram of image group feature information stored in image group feature information storage unit 233
- Flowchart of event feature information generation operation performed by image classification apparatus 100 Data structure diagram of image group feature information stored in image group feature information storage unit 233
- Functional block diagram showing the functional configuration of the image classification device 1200 Data structure diagram of face correspondence table stored in face correspondence table storage unit 1201
- Data structure diagram of image group feature information stored in image group feature information storage unit 1233 Data structure diagram of event feature information recorded in event feature information storage unit 1234
- image group feature information indicating features of an image group is calculated for each image group including a plurality of images, the calculated image group feature information, and an image group
- An image classification device that classifies an image into one of different events in units of image groups on the basis of information indicating the characteristics of the event that is the classification destination will be described.
- the image group is a set of images made up of a plurality of images designated by the user. For example, a set of images taken at a picnic in Mt. Rokko in the early summer of 2009, for example, A collection of images taken when skiing in Shiga Kogen in the winter of 2008.
- an event represents the contents of an image group having a common feature.
- a set of image groups made up of images taken at a picnic is referred to as a picnic event.
- a set of image groups made up of images taken at the time of going to is classified as an event such as skiing.
- FIG. 1 is a hardware block diagram illustrating a main hardware configuration of the image classification apparatus 100.
- the image classification device 100 stores an image that is a digital photograph as data encoded by a JPEG (Joint Photographic Experts Group) method, and classifies the stored image.
- JPEG Joint Photographic Experts Group
- the image classification apparatus 100 is connected to a device that records images, represented by a digital still camera 192, via a detachable USB cable 195, and a display 193 for displaying images, and a monitor cable 196. Via the network 194, wirelessly communicate with a remote controller 197 that accepts an operation command from a user, and read and write data to an external recording medium represented by an SD memory card 191 or the like. .
- the image classification device 100 includes a system LSI (Large Scale Integrated Circuit) 110, a hard disk device 130, an external recording medium reading / writing device 140, a USB control device 150, an output device 160, an input device 170, and a communication device. 180.
- system LSI Large Scale Integrated Circuit
- a system LSI 110 includes a CPU 101, a ROM 102, a RAM 103, a hard disk device interface 104, an external recording medium reading / writing device interface 105, a USB (Universal Serial Bus) control device interface 106, an output device interface 107, and an input device.
- the CPU 101 is connected to the bus line 120 and executes a program stored in the ROM 102 or the RAM 103, thereby executing the ROM 102, the RAM 103, the hard disk device 130, the external recording medium reading / writing device 140, the USB control device 150, and the output device 160. , Controlling the input device 170, the communication device 180, and the decoder 111, various functions such as a function of reading and decoding the encoded image data from the hard disk device 130, and outputting the decoded image data to the display 193. Is realized.
- the ROM is connected to the bus line 120 and stores a program for defining the operation of the CPU 101 and data used by the CPU.
- the RAM 103 is connected to the bus line 120, temporarily stores data generated when the CPU 101 executes the program, and reads and writes data read from the hard disk device 130 and the external recording medium reading / writing device 140. Data, data received by the communication device 180, data to be transmitted, and the like are temporarily stored.
- the decoder 111 is a DSP (Digital Signal Processor) having a function of decoding encoded image data, is connected to the bus line 120, is controlled by the CPU 101, and has a JPEG decoding function.
- DSP Digital Signal Processor
- the hard disk device interface 104 is connected to the hard disk device 130 and the bus line 120, and mediates exchange of signals between the hard disk device 130 and the bus line 120.
- the external recording medium reading / writing device interface 105 is connected to the external recording medium reading / writing device 140 and the bus line 120, and mediates the exchange of signals between the external recording medium reading / writing device 140 and the bus line 120. is there.
- the USB control device interface 106 is connected to the USB control device 150 and the bus line 120, and mediates exchange of signals between the USB control device 150 and the bus line 120.
- the output device interface 107 is connected to the output device 160 and the bus line 120, and mediates exchange of signals between the output device 160 and the bus line 120.
- the input device interface 108 is connected to the input device 170 and the bus line 120, and mediates the exchange of signals between the input device 170 and the bus line 120.
- the communication device interface 109 is connected to the communication device 180 and the bus line 120, and mediates exchange of signals between the communication device 180 and the bus line 120.
- the hard disk device 130 is connected to the hard disk device interface 104 and is controlled by the CPU 101 to have a function of writing data to the built-in hard disk and a function of reading data written to the built-in hard disk.
- the external recording medium reading / writing device 140 is connected to the external recording medium reading / writing device interface 105 and controlled by the CPU 101 to write data to the external recording medium and to read data written to the external recording medium. And have.
- the external recording medium is a DVD (Digital Versatile Disc), DVD-R, DVD-RAM, BD (Blu-ray Disc), BD-R, BD-RE, SD memory card 191, etc.
- the recording medium reading / writing device 140 can read data from DVDs, BDs, etc., and write and read data to DVD-Rs, BD-Rs, BD-REs, SD memory cards, etc. it can.
- the USB control device 150 is connected to the USB control device interface 106 and controlled by the CPU 101, and has a function of writing data to an external device via a detachable USB cable 195, and a function of reading data written to the external device.
- the external device is a device that stores images, such as a digital still camera 192, a personal computer, a mobile phone with a camera function, and the like, and the USB control device 150 is connected to these external devices via the USB cable 195. Data can be written and read.
- the output device 160 is connected to the output device interface 107 and the monitor cable 196, is controlled by the CPU 101, and outputs data to be displayed on the display 193 via the monitor cable 196.
- the input device 170 is connected to the input device interface 108, is controlled by the CPU 101, and has a function of receiving an operation command from a user wirelessly transmitted from the remote controller 197 and transmitting the received operation command to the CPU 101.
- the communication device 180 is connected to the communication device interface 109 and the network 194, is controlled by the CPU 101, and has a function of transmitting / receiving data to / from an external communication device via the network 194.
- the network 194 is realized by an optical communication line, a telephone line, a wireless line, or the like, and is connected to an external communication device, the Internet, or the like.
- the external communication device is a device such as an external hard disk device that stores images, programs that define the operation of the CPU 101, and the like.
- the communication device 180 receives data from these external communication devices via the network 194. It can be read.
- the CPU 101 executes a program stored in the ROM 102 or the RAM 103, and the ROM 102, RAM 103, hard disk device 130, external recording medium reading / writing device 140, USB Various functions are realized by controlling the control device 150, the output device 160, the input device 170, the communication device 180, and the decoder 111.
- FIG. 2 is a functional block diagram showing a configuration of main functional blocks of the image classification device 100.
- the image classification device 100 includes an image group data receiving unit 201, an image writing / reading unit 202, an image feature information writing / reading unit 203, an image group feature information writing / reading unit 204, an image feature calculating unit 205, and an image group feature calculating unit.
- 206 event feature calculation unit 207, image group classification unit 208, event feature information write / read unit 209, classification result output unit 210, image group information reception unit 211, event information reception unit 212, image storage unit 231, image feature information
- the storage unit 232 includes an image group feature information storage unit 233 and an event feature information storage unit 234.
- the image feature calculation unit 205 further includes an image feature calculation control unit 221, a face feature amount extraction unit 222, a color feature amount extraction unit 223, and an object feature amount extraction unit 224.
- the image group data accepting unit 201 is connected to the image writing / reading unit 202 and accepts designation of an image of an image group 241 composed of two or more images, and the designated image group is included in one image group. It has a function to read as a group.
- the image group data receiving unit 201 When the image group data receiving unit 201 receives an image, the image group data receiving unit 201 receives an image from an external recording medium via the external recording medium reading / writing device 140, a case where an image is received from an external device via the USB control device 150, a communication An image may be received from an external communication device via the device 180.
- the image group data receiving unit 201 has a function of assigning an image ID for specifying the image when receiving the image.
- the image storage unit 231 is a storage area for storing a digital photograph as an image as image data encoded by the JPEG method, and is implemented as a partial area of the hard disk built in the hard disk device 130. ing.
- Each data stored in the image storage unit 231 is logically managed by a directory structure under the file system.
- FIG. 3 is a directory structure diagram showing the directory structure of the image storage unit 231.
- the directory structure of the image storage unit 231 is composed of a total of three hierarchies including a highest hierarchy 310, a first directory hierarchy 320, and a second directory hierarchy 330.
- the first directory hierarchy 320 includes a plurality of event directories such as a fireworks event directory 321, a picnic event directory 322, a ski event directory 323, and an actual data storage directory 324.
- the event directory is a directory having the same name as the event to which the image group is classified, and there is only one directory having the same name.
- the actual data storage directory 324 is a directory for storing images, and image data is stored only in the actual data storage directory 324.
- the second directory hierarchy 330 there are a plurality of image group directories such as a Y river fireworks display 2004 image group directory 331, a P company fireworks display 2005 image group directory 332, and Rokkosan 2009 early summer image group directory.
- image group directories such as a Y river fireworks display 2004 image group directory 331, a P company fireworks display 2005 image group directory 332, and Rokkosan 2009 early summer image group directory.
- the image group directory is a directory corresponding to an image group made up of images received by the image group data receiving unit 201, and all images belonging to the image group among the data held in the actual data storage directory 324. This is a directory in which the image data is linked by holding information indicating the address of the data.
- Each image group directory exists under the event directory corresponding to the event to which the corresponding image group is classified.
- the image writing / reading unit 202 is connected to the image feature calculation unit 205, the image feature calculation control unit 221, and the image group classification unit 208, and has a function of reading an image stored in the image storage unit 231; A function of writing an image in the unit 231; a function of changing a directory structure of the image storage unit 231; and a function of changing a link of image data.
- the face feature quantity extraction unit 222 is connected to the image feature calculation control unit 221, holds a predetermined face model indicating the features of a person's face, and refers to the held face model. Attempts to recognize the faces included in the image, the function to calculate the number of recognized faces and the area ratio of the area of the entire image to the area of the recognized face area as a facial feature amount, and for each recognized recognized face And a function of sequentially assigning a face ID for identifying the recognized face.
- the face model is, for example, information on luminance of parts forming the face such as eyes, nose, mouth, information on relative positional relationship, and the like.
- the color feature amount extraction unit 223 is connected to the image feature calculation control unit 221, and for each pixel included in the image, the color of the pixel is determined based on the color components constituting the pixel, for example, each luminance of Red, Green, and Blue. Is included in the image for the number of pixels specified for that color, for example, a function that specifies which color is black, blue, green, white, etc., and for each of the specified colors It has a function of calculating the ratio of the total number of pixels as a color feature amount.
- a certain pixel is, for example, black, for example, when the luminance of Red, the luminance of Green, and the luminance of Blue are all less than 10%
- a method of specifying a pixel as black There is a method of specifying a pixel as black.
- the object feature quantity extraction unit 224 is connected to the image feature calculation control unit 221, holds a predetermined object model indicating the feature of the object, and the name of the object corresponding to the model, and holds the object model The function of calculating the name of an object corresponding to the model of the recognized object as an object feature when attempting to recognize the object included in one image and successfully recognizing the object Have.
- the object model is, for example, a vehicle model, for example, information on brightness of parts forming a vehicle such as a windshield, a tire, and a headlight, information on a relative positional relationship, and the like.
- the image feature calculation control unit 221 includes an image writing / reading unit 202, an image feature information writing / reading unit 203, a face feature amount extracting unit 222, a color feature amount extracting unit 223, and an object feature amount extracting unit 224. Connecting.
- the image feature calculation control unit 221 reads one image from the image storage unit 231 via the image writing / reading unit 202, and for the one read image, the face feature amount extraction unit 222 and the color feature Using the quantity extraction unit 223 and the object feature quantity extraction unit 224, the image feature information is calculated.
- the image feature information storage unit 232 is a storage area for storing image feature information, and is mounted as a partial area of the hard disk built in the hard disk device 130.
- FIG. 4 is a diagram illustrating a data structure of image feature information stored in the image feature information storage unit 232.
- each image feature information stored in the image feature information storage unit 232 includes an image ID 401 indicating a corresponding image, a color feature amount 403 indicating a color feature of the corresponding image, and the like.
- the face feature amount 404 indicating the feature of the face recognized by the face feature amount extraction unit 222 included in the corresponding image (hereinafter referred to as a recognition face), and the object indicating the feature of the recognized object included in the corresponding image And feature quantity 405.
- the color feature amount 403 is composed of the ratio of the number of pixels of each color calculated by the color feature amount extraction unit 223, and indicates the feature of the color included in the image.
- the color feature amount 403 of the image feature information corresponding to the image having the image ID 401 of 01001 is an image in which black 431 is 10%, blue 432 is 20%, green 433 is 60%, and white 435 is 10%. Is shown.
- the face feature quantity 404 is calculated by the face feature quantity extraction unit 222, and includes the face ID of the recognized face, the area ratio of the recognized face area, and the coordinates of the recognized face area for each recognized face included in each image. , A number of faces 444 that is the number of recognized faces included in the image, and a maximum face area 445 that is an area ratio of the face having the highest area ratio among the recognized faces included in the image, It shows the characteristics of the recognized face included in the image.
- the coordinates of the recognized face area are a set of the coordinates of the upper left vertex and the lower right vertex of the minimum area rectangle among the rectangles surrounding the recognized face area by the face feature amount extraction unit 222. .
- the face feature amount 404 of the image feature information corresponding to the image having the image ID 401 of 01001 is a face such as a face 0001 having an area ratio of 30%, a face 0002 having an area ratio of 10%, and a face 0003 having an area ratio of 20%. This indicates that the number of recognized faces is five and the area ratio of the highest face among the recognized faces is 0.3.
- the object feature quantity 405 indicates the name of the recognized object included in each image calculated by the object feature quantity extraction unit 224.
- the object feature quantity 405 of the image feature information corresponding to the image with the image ID 401 of 01001 indicates that the image of the car is recognized in the image.
- the image feature information writing / reading unit 203 is connected to the image feature calculation control unit 221 and the image group feature calculation unit 206, and has a function of reading and writing image feature information to the image feature information storage unit 232. Have.
- the image group information reception unit 211 is connected to the image group feature calculation unit 206 and has a function of receiving an image group name.
- the image group feature calculating unit 206 is connected to the image group information receiving unit 211, the image feature information writing / reading unit 203, and the image group feature information writing / reading unit 204.
- the image group feature calculation unit 206 reads out image feature information corresponding to all images belonging to one image group from the image feature information storage unit 232 via the image feature information writing / reading unit 203, and reads out the image feature information.
- the image group feature information is calculated using the information and the name of the image group input from the image group information reception unit 211.
- the image group feature calculation unit 206 has a function of adding an image group ID for specifying the image group feature information to the calculated image group feature information.
- the image group feature information storage unit 233 is a storage area for storing image group feature information, and is mounted as a partial area of the hard disk built in the hard disk device 130.
- FIG. 5 is a diagram showing a data structure of each image group feature information stored in the image group feature information storage unit 233.
- each image group feature information stored in the image group feature information storage unit 233 includes an image group ID 501 indicating a corresponding image group and a color of an image belonging to the corresponding image group.
- Color feature amount average 502 indicating features
- face feature amount 503 indicating recognized face features of images belonging to the corresponding image group
- object feature amount 504 indicating names of objects of images belonging to the corresponding image group
- correspondence A total number 505 indicating the number of images belonging to the image group, a group name 506 indicating the name of the corresponding image group, and an event name 507 indicating the event name of the event in which the corresponding image group is classified.
- the color feature amount average 502 is an average value of the color feature amounts of images belonging to the corresponding image group, and indicates the color feature of the image group.
- the color feature amount average 502 corresponding to the image group whose image group ID is 0001 is an image group in which black 421 is 10%, blue 522 is 20%, green 523 is 40%, and white 524 is 30%. Is shown.
- the face feature amount 503 is the largest maximum face area 532 that is the largest of the face ID 531 including the ID of the recognized face included in the image belonging to the corresponding image group and the maximum face area 445 of the image belonging to the corresponding image group. And the total number of faces 533 which is the sum of the number of faces 444 of the images belonging to the corresponding image group, the number of faces 534 which is the number of images including the recognized faces, and the number of faces 444 of the images belonging to the corresponding image group.
- the largest face number 535 is the largest face and indicates the features of the recognized face included in the image group.
- the face feature quantity 503 corresponding to the image group whose image group ID is 0001 is such that faces 0001, 0002, 0003, etc. are recognized, the total maximum face area is 40%, and the total number of faces 533 is 7.
- the number of faces 534 is 2, and the maximum number of faces 535 is 5.
- the object feature quantity 504 indicates the feature of the recognized object included in the image belonging to the corresponding image group.
- the object feature quantity 504 corresponding to the image group whose image group ID is 0001 indicates that the recognized car and the recognized flower exist in the image belonging to the corresponding image group.
- the group name 506 indicates the name of the corresponding image group and is designated by the user.
- the directory name of the image group directory described above is determined by this group name 506.
- the event name 507 indicates the name of the event to which the corresponding image group is classified, and the event directory to which the directory of the image group directory name determined by the group name 506 belongs is determined by this event name 507. It is.
- the image group feature information writing / reading unit 204 is connected to the image group feature calculation unit 206, the event feature calculation unit 207, and the image group classification unit 208, and stores image group feature information in the image group feature information storage unit 233. It has a function of reading and writing.
- the event information receiving unit 212 is connected to the event feature information writing / reading unit 209 and has a function of receiving event feature information.
- the event feature information is information serving as a reference for classifying image groups into events, and details will be described later.
- the event feature information storage unit 234 is a storage region for storing event feature information, and is implemented as a partial region of the hard disk built in the hard disk device 130.
- FIG. 6 is a diagram showing a data structure of event feature information stored in the event feature information storage unit 234.
- the event feature information is a combination of reference information 601 indicating a criterion for classifying an image group and an event name 611 indicating a classification destination event classified by the reference information 601. It is constituted by.
- the standard of classifying the standard information of black 0.4 or more and 602 and the event name of fireworks into the event of fireworks indicates that black is 40% or more in the color feature amount average 502.
- the standard of classifying the standard information of blue 0.4 to 603 and the event name of scuba diving into the event of scuba diving indicates that blue is 40% or more in the color feature amount average 502.
- the standard of classifying the reference information of green 0.4 or more and 604 and the event name of picnic into the event of picnic indicates that green is 40% or more in the color feature amount average 502.
- the standard of classifying the event information called ski and the standard information of white 0.4 or more and 605 indicates that white is 40% or more in the average color feature amount 502.
- the standard of classifying the standard information of the number of people 5 or more 606 and the event name of the large number of people into the event of the large number of people indicates that the total number of faces 533 is 5 or more.
- the standard of classifying the standard information of 607 or less 607 and the event name of small number into an event of small number indicates that the total number of faces 533 is 4 or less.
- the event feature information writing / reading unit 209 is connected to the event information receiving unit 212, the event feature calculation unit 207, and the image group classification unit 208, and reads and writes event feature information to the event feature information storage unit 234. It has a function to perform.
- the image group classification unit 208 is connected to the image writing / reading unit 202, the image group feature information writing / reading unit 204, the event feature information writing / reading unit 209, and the classification result output unit 210.
- the image group classification unit 208 reads the image group feature information from the image group feature information storage unit 233 via the image group feature information write / read unit 204, and stores the read image group feature information and the event feature information storage unit 234. Based on the stored event feature information, the image group corresponding to the read image group feature information is classified into events.
- the event feature calculation unit 207 is connected to the image group feature information write / read unit 204 and the event feature information write / read unit 209.
- the event feature calculation unit 207 reads one or more pieces of image group feature information from the image group feature information storage unit 233 via the image group feature information writing / reading unit 204, and uses the read image group feature information to record event features. It has a function to create information.
- the classification result output unit 210 is connected to the image group classification unit 208 and has a function of displaying the classification result on the display 193 when the image group classification unit classifies the image group.
- the main operations performed by the image classification apparatus 100 are an image group classification operation in which an image belonging to an image group is input and the input image group is classified as an event, and two or more image groups are specified, and the specified image group There is an event feature information generating operation for newly generating event feature information by extracting common features.
- FIG. 7 is a flowchart of the image group classification operation performed by the image classification apparatus 100.
- the image classification device 100 starts the processing of the image group classification operation.
- the image group data receiving unit 201 starts reading an image of one image group, and the image group information receiving unit 211 starts receiving the name of the image group. (Step S700).
- the image group data reception unit 201 is connected to the external recording medium reading / writing device 140 from an external recording medium, from an external device via the USB cable 195 connected to the USB control device 150, or to the network 194. Images can be read from the connected communication device 180.
- the image group data receiving unit 201 reads images recorded on the SD memory card 191 one by one, sequentially assigns image IDs to the read images, and uses the image writing / reading unit to Write to the actual data storage directory 324 of the storage unit 231.
- the image group information receiving unit 211 receives the name of the image group by the operation of the remote controller 197 from the user.
- the image feature calculation control unit 221 uses the image writing / reading unit 202 to belong to the image group received by the image group data receiving unit 201. Images are read one by one (step S705).
- the face feature amount extraction unit 222 tries to recognize a face included in one image by referring to the face model held for one image read out by the image feature calculation control unit 221 and recognizes it.
- the number of detected faces and the ratio of the area of the recognized face area to the area of the entire image are calculated as face feature amounts (step S710).
- the color feature amount extraction unit 223 specifies the color of the pixel from each luminance of the color component of the pixel for each pixel included in the image, For each specified color, the ratio of the number of pixels specified for that color to the total number of pixels included in the image is calculated as a color feature amount (step S715).
- the object feature amount extraction unit 224 tries to recognize an object included in one image by referring to the model of the object to be held.
- the recognition is successful, the name of the recognized object is calculated as an object feature amount (step S720).
- the image feature calculation control unit 221 calculates image feature information from the calculated face feature amount, color feature amount, and object feature amount, and uses the image feature information writing / reading unit 203 to calculate the image feature information.
- the image feature information is written into the image feature information storage unit 232 (step S725).
- step S730 If the image feature calculation control unit 221 has not completed the calculation of the image feature information for all the images of one image group received by the image group data receiving unit 201 (step S730: No), it has not yet been performed. The process of step S705 is started again for the image for which the calculation of the image feature information has not been completed.
- the image group feature calculation unit 206 stores all image feature information corresponding to images belonging to the image group read by the image group data receiving unit 201 using the image feature information writing / reading unit 203.
- the image group feature calculation unit 206 When the image group feature calculation unit 206 reads all the image feature information corresponding to the images belonging to the image group, the image group feature calculation unit 206 determines the face ID including the IDs of all recognized faces of the images belonging to the image group and the maximum of the images belonging to the image group. The largest total face area among the face areas, the total number of faces that is the sum of the number of faces of images belonging to the image group, the number of faces that is the number of images including the recognized faces, and the images that belong to the image group The face feature amount of the image group consisting of the largest number of faces among the number of faces is calculated (step S735).
- the image group feature calculation unit 206 further calculates an average value for each color of the color feature amounts of the images belonging to the image group, and calculates an average color feature amount of the image group including the calculated average value for each color (step In step S740, the object feature amount of the image group including the names of all recognized objects included in the images belonging to the image group is calculated (step S745).
- the image group feature calculation unit 206 calculates the facial feature amount of the calculated image group, the average color feature amount of the calculated image group, the calculated object feature amount, and the image group received by the image group information reception unit 211.
- the image group feature information is calculated using the image name, and the calculated image group feature information is written into the image group feature information storage unit 233 using the image group feature information writing / reading unit 204 (step S750). .
- the image group classification unit 208 reads the image group feature information written earlier using the image group feature information write / read unit 204, and stores it in the event feature information storage unit 234 using the event feature information write / read unit 209. Read the recorded event feature information.
- the image group classification unit 208 compares the read image group feature information with the event feature information (step S755), and configures the event feature information in the constituent elements that configure the image group feature information.
- the event to which the image group corresponding to the read image group feature information should be classified is the event corresponding to the found reference information.
- the event in which the image group corresponding to the read image group feature information is to be classified is an event called another event. To do.
- the image group classification unit 208 uses the image writing / reading unit 202 to store the name of the image group under the event directory corresponding to the event to be classified in the image storage unit 231. By creating an image group directory with the same name and holding information indicating the addresses of all image data belonging to the image group under the image group directory, the data of all images belonging to the image group are stored. By setting the linked state, the image group is classified (step S760).
- the classification result output unit 210 causes the display 193 to display the event name of the event to be classified calculated by the image group classification unit 208 together with the name of the image group received by the image group information reception unit 211.
- the classification apparatus 100 ends the image group classification operation.
- FIG. 8A shows an image group 800 that includes images 801 to 804 that are photographs taken by the user.
- This image group 800 is, for example, an image group in which the name of the image group is Hakone 2008 Summer, and the image group ID is 0010.
- FIG. 8B shows an image group 820 that includes images 821 to 823 that are photographs taken by the user.
- This image group 820 is, for example, an image group whose name is Niseko 2009 winter and whose image group ID is 0011.
- the image group 820 includes a lot of white, which is the color of snow.
- FIG. 8C shows an image group 840 having images 841 and 842 that are photographs taken by the user as constituent elements.
- This image group 840 is, for example, an image group having the name of the image group as Miyakojima 2009 summer, and the image group ID is 0012.
- the color of the image included in the image group 840 includes a lot of blue which is the color of the sea water.
- FIG. 9 is a diagram illustrating a data structure of image group feature information corresponding to image groups having image group IDs 0010, 0011, and 0012, stored in the image group feature information storage unit 233.
- step S ⁇ b> 755 when the image group classification unit 208 reads out the image group feature information whose image group ID is 0010 from the image group feature information storage unit 233, the read image group feature is 0010.
- the information and the event feature information (see FIG. 6) read from the event feature information storage unit 234 are compared.
- the image group feature information of the image group ID 0010 is that the green feature 523 is 0.4 in the color feature amount average 502 and the total number of faces 533 is 7 in the face feature amount 503. It is determined that the image group feature information of ID0010 corresponds to the reference information 601 of green 0.4 or more and 604 and the reference information 601 of five or more people 606, and the image group of image group ID0010 is shown in FIG. It should be categorized as a picnic event and a large number of events.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named “Hakone 2008 Summer” under the event directory called the picnic event directory 322 of the image storage unit 231. Links to image data corresponding to the images 801 to 804 are created under the directory called Hakone 2008 Summer.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named “Hakone 2008 Summer” under the event directory of a large number of people (not shown) in the image storage unit 231. Links to image data corresponding to the images 801 to 804 are created under the created directory called Hakone 2008 Summer.
- step S ⁇ b> 755 when the image group classification unit 208 reads the image group feature information with the image group ID 0011 from the image group feature information storage unit 233, the read image group feature with the image group ID 0011 is used. The information is compared with the event feature information read from the event feature information storage unit 234.
- the white 524 is 0.5 in the color feature amount average 502, and the total number of faces 533 is 4 in the face feature amount 503. It is determined that the image group feature information of ID0010 corresponds to the reference information 601 of white 0.4 or more and 605 and the reference information 601 of 4 or less people 607, and the image group of image group ID0011 is an event called ski. , It should be classified as an event of small number of people.
- the image group classification unit 208 uses the image writing / reading unit 202 to create and create an image group directory named Niseko 2009 Winter under an event directory called ski event directory 323 in the image storage unit 231. Under the directory “Niseko 2009 Winter”, links to image data corresponding to the images 821 to 823 are created.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named “Niseko 2009 Winter” under the event directory of a small number of people (not shown) in the image storage unit 231. Links to the image data corresponding to the images 821 to 823 are created under the created directory Niseko 2009 Winter.
- step S755 when the image group classification unit 208 reads the image group feature information whose image group ID is 0012 from the image group feature information storage unit 233, the read image group ID is 0012. The information is compared with the event feature information read from the event feature information storage unit 234.
- the image group classification unit 208 is It is determined that the image group feature information that is ID0012 satisfies the standard information 601 of blue 0.4 to 603 and the standard information 601 (see FIG. 6) of 607 or less 607, and the image group that is the image group ID0012 is It is calculated that the event should be classified into an event called scuba diving and an event called small number of people.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named “Miyakojima 2009 Summer” under an event directory called “scuba diving event” (not shown) in the image storage unit 231. Then, links to the image data corresponding to the images 841 and 842 are created under the created directory called Miyakojima 2009 Summer.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named “Miyakojima 2009 Summer” under the small-numbered event directory (not shown) of the image storage unit 231. Links to the image data corresponding to the images 841 and 842 are created under the created directory called Miyakojima 2009 Summer.
- FIG. 10 is a flowchart of the event feature information generation operation performed by the image classification device 100.
- the remote control 197 starts the event feature information generation operation when receiving an operation for starting the event feature information generation operation from the user.
- the image group information receiving unit 211 receives designation of an image group ID from the user (step S1000), and the event information receiving unit 212 corresponds to newly created event feature information. Accept the event name.
- the image group feature calculation unit 206 uses the image group feature information writing / reading unit 204 to read out the image group feature information corresponding to the received image group ID from the image group feature information storage unit 233 and read the image group feature information. Information is sent to the event feature calculation unit 207.
- the event feature calculation unit 207 tries to calculate a common feature from the received image group feature information (step S1010).
- the common feature calculation performed by the event feature calculation unit 207 is, for example, that the value of a certain color in the color feature amount average 502 of the image group feature information is 0.4 or more in all image groups. In some cases, the color is a common feature.
- step S1010 in the image group feature information corresponding to all the designated image groups, out of the color feature amount average 502, the color of the ratio of 0.4 or more is the same color (for example, X color).
- step S1010: Yes new reference information of X color 0.4 or more is created as reference information 601 and the event name received by the event information receiving unit 212 is set as the corresponding event name.
- the event feature calculation unit 207 stores the created reference information in the event feature information storage unit 234 using the event feature information writing / reading unit 209 in association with the event name (step S1020).
- step S1010 when the common feature is not calculated (step S1010: No), and when the process of step S1020 is completed, the image classification device 100 ends the event feature information generation operation.
- FIG. 11 is a diagram showing a data structure of image group feature information corresponding to the image groups with image group IDs 0001 and 0010 stored in the image group feature information storage unit 233.
- step S1000 the image group information reception unit 211 receives designation of the image group IDs 0001 and 0010 from the user, and the event information reception unit 212 uses, for example, a picnic as an event name corresponding to newly created event feature information.
- the event feature calculation unit 207 receives the image group feature information of the image group IDs 0001 and 0010 stored in the image group feature information storage unit 233.
- step S1010 When the event feature calculation unit 207 tries to calculate a common feature from the received image group feature information (step S1010), the ratio of the green 523 of the color feature amount average 502 in the image group feature information with the image group ID 0001 is obtained. Since the ratio of green 523 of the color feature amount average 502 in the image group feature information with the image group ID 0010 is 0.4 (step S1010: Yes), the common feature is green 0.4.
- the above reference information is created.
- the event feature calculation unit 207 stores the event feature information storage unit 234 using the event feature information writing / reading unit 209 in association with the reference information of green 0.4 or more and the event name of picnic ( Step S1020).
- the image classification device 100 ends the event feature information generation operation.
- ⁇ Embodiment 2> a part of the image classification device 100 described in the first embodiment is transformed, and the recognized faces included in all the images stored are family, friends, and the like. A description will be given of the image classification device 1200 to which a function for determining any one of others is added.
- the image group feature information stored in the image classification device 1200 includes face feature amounts based on the determination of family, friend, and others.
- ⁇ Configuration> ⁇ Hardware Configuration of Image Classification Apparatus 1200>
- the hardware configuration of the image classification device 1200 is the same as the hardware configuration of the image classification device 100. Therefore, the description is omitted here.
- FIG. 12 is a functional block diagram showing the main functional block configuration of the image classification device 1200.
- a face correspondence table storage unit 1201 that is a storage area for storing a face correspondence table (described later) is newly added, and an image group feature information storage unit 233 is added.
- an image group feature information storage unit 1233 for storing image group feature information different from the image group feature information of the first embodiment and part of its constituent elements
- the event feature information storage unit 234 The event feature information of Embodiment 1 is transformed into an event feature information storage unit 1234 that stores event feature information that is partly different from its constituent elements, and the image group feature calculation unit 206 has a new function.
- the image group feature calculation unit 1206 is added (described later).
- the image group feature calculation unit 1206 newly adds a face classification function, a face group determination function, a face correspondence table update function, a face correspondence table read function, and the like to the image group feature calculation unit 206 in the first embodiment. It is modified as follows.
- the face classification function is to extract face features for the recognized faces indicated by all face IDs included in the image feature information recorded in the image feature information storage unit 232, and to extract the extracted face features. Based on this, the recognized faces are classified so that the recognized faces determined to be the same person are in the same set, and a label for identifying the set is given to each set of recognized face groups. It is a function.
- the facial features are, for example, the relative positional relationship of parts forming the face such as eyes, nose and mouth, and the area ratio of these parts.
- the face group determination function is a function for determining that a group of faces classified as the same person by executing the face classification function is one of a family member, a friend, and another person.
- the criterion for determining a recognized face classified as the same person as a family is when the recognized face classified as the same person exists in a plurality of image groups.
- the criterion for determining that a recognition face classified as the same person is a friend is that the recognition face classified as the same person exists only in a single image group and is recognized as the same person. This is a case where there are two or more faces.
- the criterion for determining that a recognized face classified as the same person is another person is a recognized face classified as the same person that is determined not to be a family and determined not to be a friend, that is, the same This is a case where there is only one recognized face classified as a person.
- the face correspondence table update function creates a face correspondence table (described later) based on the execution results of the face classification function and the face group determination function when the face classification function and the face group determination function are executed. This is a function for updating the face correspondence table stored in the face correspondence table storage unit 1201 with the face correspondence table.
- the face correspondence table reading function is a function for reading the face correspondence table stored in the face correspondence table storage unit 1201.
- the face correspondence table storage unit 1201 is a storage area for storing the face correspondence table, and is implemented as a partial area of the hard disk built in the hard disk device 130.
- FIG. 13 is a diagram showing the data structure of the face correspondence table stored in the face correspondence table storage unit 1201.
- the face correspondence table stored in the face correspondence table storage unit 1201 is the same person as the label 1301 for identifying a group of recognized faces classified as being the same person.
- a determination result 1302 indicating whether the group of classified recognition faces is determined to be one of a family member, a friend, or another person, and a recognition face belonging to the group of recognition faces classified as being the same person Face IDs 1303 to 1305 indicating the IDs of the faces.
- a group of recognized faces classified as label 1301 being A is classified as a face of a person determined to be a family, and is indicated by a face ID 0001, a face ID 0003, a face ID 0101, and the like. It shows that the face belongs.
- FIG. 14 is a diagram showing a data structure of each image group feature information stored in the image group feature information storage unit 1233.
- each image group feature information stored in the image group feature information storage unit 1233 includes an image group ID 1401 indicating a corresponding image group and an image belonging to the corresponding image group.
- the face feature quantity 1402 indicating the feature of the recognized face
- the total number 1403 indicating the number of images belonging to the corresponding image group
- the group name 1404 indicating the name of the corresponding image group
- the corresponding image group are classified.
- the event name 1405 indicating the event name of the event.
- the face feature value 1402 includes information related to a recognized face of a person determined to be a family (hereinafter referred to as a family face) and a recognized face of a person determined to be a friend (hereinafter referred to as a friend's face). And information related to the recognition face of a person determined to be another person (hereinafter referred to as the face of another person).
- the information related to the family face includes the family face ID 1421 which is the face ID of the family face included in the image belonging to the corresponding image group and the face area of the family face included in the image belonging to the corresponding image group.
- the largest family face area 1422 having a large rate, the largest family face number 1423 among the number of family faces included in one image, and the images belonging to the corresponding image group.
- the family face number ratio 1424 which is the ratio of the number of images including family faces to the number of sheets
- the family face number ratio 1425 which is the average number of family faces included in one image
- a family face area ratio 1426 is obtained by dividing a family maximum face area ratio having the largest face area ratio by the number of images including the family face.
- the family face ID 1421 is no corresponding ID, the maximum family face area 1422, the maximum family face count 1423, and the family face count ratio 1424.
- the family face number rate 1425 and the family face area rate 1426 are zero.
- the information related to the friend's face includes the friend face ID 1431 which is the face ID of the friend's face included in the image belonging to the corresponding image group, and the face area of the friend's face included in the image belonging to the corresponding image group.
- Friend maximum face area 1432 having a high rate
- the number of friends face number ratio 1434 which is the ratio of the number of images including the face of a friend to the number of sheets
- the friend face number ratio 1435 which is the average number of friend faces included in one image
- a friend face area ratio 1436 is obtained by dividing a friend maximum face area ratio having the largest face area ratio by the number of images including the face of the friend.
- the friend face ID 1431 is no corresponding ID, the friend maximum face area 1432, the friend maximum face count 1433, and the friend face count ratio 1434.
- the friend face number ratio 1435 and the friend face area ratio 1436 are zero.
- the information relating to the face of the other person includes the face area 1441 of the face of the other person included in the image belonging to the corresponding image group and the face area of the face of the other person included in the image belonging to the corresponding image group.
- Other person's maximum face area 1442 having a large rate, the maximum number of other persons' faces 1443 out of the number of other people's faces included in one image, and images belonging to the corresponding image group
- the ratio of the number of other person's faces 1444 that is the ratio of the number of images including the face of another person to the number of faces, the ratio of the number of other person's faces 1445 that is the average of the number of other people's faces included in one image, and the friend of each image
- the face area ratio 1446 is a face area ratio 1446 obtained by dividing the maximum face area ratio of others, which has the largest face area ratio, by the number of images including the faces of others.
- the other person's face ID 1441 does not have the corresponding ID, the other person's maximum face area 1442, the other person's maximum number of faces 1443, and the number of other person's faces 1444.
- the other person face ratio 1445 and the other person face area ratio 1446 are zero.
- the face feature quantity 1402 corresponding to the image group whose image group ID is 0020 recognizes the faces of family face IDs 0001 and 0002, the maximum family face area is 40%, and the maximum number of family faces is two.
- the family face number ratio is 100%, the family face number ratio is 1.5, and the family face area ratio is 30%, indicating that the face of a friend and the face of another person do not exist. .
- FIG. 15 is a diagram showing a data structure of event feature information recorded in the event feature information storage unit 1234.
- the event feature information is a set of reference information 1501 indicating a criterion for classifying an image group and an event name 1511 indicating an event to be classified classified by the criterion information 1501. It is constituted by.
- the standard of classifying the group information of the family face area ratio of 0.1 or higher and the event name of birthday party into the event of birthday party indicates that the family face area ratio 1426 is 10% or higher.
- This standard is based on the assumption that a birthday party often celebrates the birthday of the family and that the face of the family who has reached the birthday is increased.
- Non-family face count rate is less than 0.5
- non-family face rate is less than 1
- the maximum number of non-family faces is less than 3 and the event name of insect collection is classified as an event of insect collection. This indicates that the number of faces other than the family is less than 50%, the number of faces other than the family is less than 1, and the maximum number of faces other than the family is less than 3.
- insect collection often involves insects and plants that are the background, and there may be family members who accompany the subject. It is based on what is assumed to be unlikely.
- This criterion is that athletic meet is an event held at school, so family members may be included in the shooting, but at least a large number of people other than family members, such as school classmates, are taken. Is based on being.
- ⁇ Operation> As main operations performed by the image classification device 1200, in addition to the main operations performed by the image classification device 100 according to Embodiment 1, all the image feature information included in all the image feature information recorded in the image feature information storage unit 232 is used. For the recognized face indicated by the face ID, a facial feature is extracted, and based on the extracted facial feature, the same label is assigned to the face of the same person, and the face with the same label is assigned.
- a face correspondence table is generated by determining that the group is one of a family member, a friend, or another person, and the face correspondence table stored in the face correspondence table storage unit 1201 is generated by the generated face correspondence table. There is an operation for generating a face correspondence table for updating.
- the image group classification operation which is one of the main operations performed by the image classification device 1200, is obtained by changing a part of the operation performed by the image classification device 100 of the first embodiment.
- FIG. 16 is a flowchart of the face correspondence table generation operation performed by the image classification device 1200.
- the image read by the image group data reception unit 201 is recorded in the image storage unit 231 and the corresponding image feature information is recorded in the image feature information storage unit 232. Then, the face correspondence table generation operation is started.
- the image group feature calculation unit 1206 reads all image feature information stored in the image feature information storage unit 232 using the image feature information writing / reading unit 203.
- the image group feature calculation unit 1206 includes, from all the read image feature information, the face ID of the recognized face, the coordinates of the face area specified by the face ID, and the image corresponding to the image feature information including the face ID. ID is extracted (step S1600).
- the image group feature calculation unit 1206 reads an image specified by the image ID included in the image feature information including the face ID from the image storage unit 231 using the image writing / reading unit 202, and uses the extracted face ID as the extracted face ID. Facial features are extracted from all the face areas specified by the coordinates of the corresponding face area (step S1603).
- the image group feature calculation unit 1206 determines that a group of faces having the same facial features among the extracted facial features are the faces of the same person, and the recognized faces that are judged to be the same person are the same set. Then, the recognized faces are classified, and a label for specifying the set is given to the set of each recognized face group (step S1605).
- the image group feature calculation unit 1206 selects one image group (step S1610), and selects one of the labels attached to the recognition face included in the image group (step S1615). ), It is searched whether or not the recognized face to which the selected label is assigned exists in another image group (step S1620).
- step S1620 when the recognized face to which the selected label is assigned is present in another image group (step S1620: Yes), the image group feature calculation unit 1206 is assigned the selected label.
- the recognized face classified as the same person is determined to be a family (step S1625).
- step S1620 if the recognized face to which the selected label is assigned does not exist in another image group as a result of the search (step S1620: No), the image group feature calculation unit 1206 further selects the selected label. It is searched whether or not there are a plurality of recognized faces to which “” is assigned (step S1630).
- step S1630 when there are a plurality of recognition faces to which the selected label is assigned (step S1630: Yes), the image group feature calculation unit 1206 determines that the selected person is the same person to which the selected label is assigned. It is determined that the classified recognition face is a friend (step S1635).
- step S1630 when there is only one recognized face to which the selected label is assigned (step S1630: No), the image group feature calculation unit 1206 assigns the selected label. It is determined that the face is another person (step S1640).
- step S1625 When the process of step S1625 is completed, when the process of step S1635 is completed, or when the process of step S1640 is completed, the image group feature calculation unit 1206 does not select the label of the selected image group. Is checked (step S1645). If an unselected label remains (step S1645: No), the image classification device 1200 performs steps S1615 to S16 until there is no unselected label. S1645: The process of No is repeated.
- step S1645 If no unselected label remains in step S1645 (step S1645: Yes), the image group feature calculation unit 1206 checks whether or not an unselected image group remains (step S1650). When image groups remain (step S1650: No), the image classification device 1200 repeats the processes of steps S1610 to S1650: No until there are no unselected image groups.
- step S1640 If no unselected image group remains in step S1650 (step S1640: YES), the image feature information storage unit 232 displays, for each label, the label and a group of faces classified into the label are family members, A face correspondence table is created by associating a determination result indicating whether it is determined to be either a friend or another person and the face ID to which the label is assigned.
- the face correspondence table stored in the table storage unit 1201 is updated (step S1655).
- step S1655 the image classification device 1200 ends the face correspondence table generation operation.
- FIG. 17 is a diagram showing images belonging to the image group 1700, the image group 1720, and the image group 1740.
- the images 1701 to 1704 are images belonging to the image group 1700
- the images 1721 to 1724 are images belonging to the image group 1720
- the images 1741 and 1742 are images belonging to the image group 1740.
- a face 1711, a face 1712, a face 1715, a face 1732, a face 1739, and a face 1751 are recognized faces determined by the image group feature calculation unit 1206 to be the same person.
- the label E is given.
- the face 1713 and the face 1731 are recognized faces that are determined to be the same person by the image group feature calculation unit 1206, and, for example, have a label F attached thereto. To do.
- a face 1733, a face 1736, and a face 1737 are recognized faces that are determined to be the same person by the image group feature calculation unit 1206, and are, for example, given a label G Suppose that
- the face 1714, the face 1734, the face 1735, the face 1738, and the face 1752 are recognized faces that are classified as the same person by the image group feature calculation unit 1206, respectively. , Something else does not exist.
- the images stored in the image classification device 1200 are only the images 1701 to 1704, the images 1721 to 1724, the images 1741, and the images 1742.
- the image group feature calculation unit 1206 determines that it is a family.
- the image group feature calculation unit 1206 determines that it is a family.
- the recognized face to which the label G is assigned exists only in the image group 1720 and there are a plurality of faces classified as the same person. Therefore, the image group feature calculation unit 1206 determines that the face is a friend.
- the face 1714, the face 1734, the face 1735, the face 1738, and the face 1752 each have no other recognized face classified as the same person by the image group feature calculation unit 1206.
- the image group feature calculation unit 1206 determines that the person is another person.
- FIG. 18 is a flowchart of an image group classification operation performed by the image classification apparatus 1200.
- the image classification device 1200 starts the image group classification operation.
- Step S1800 to Step S1830: Yes are the same as the operations from Step S700 to Step S730: Yes (see FIG. 7) in the image group classification operation performed by the image classification apparatus 100 according to the first embodiment. Then, explanation is omitted.
- step S1830 when the calculation of the image feature information is completed for all the images received by the image group data receiving unit 201 (step S1830: Yes), the image classification device 1200 performs the face correspondence table generation operation described above. Is performed (step S1833).
- the image group feature calculation unit 1206 stores all the image features corresponding to the images belonging to the image group stored in the image feature information storage unit 232 and read by the image group data reception unit 201. Information is read using the image feature information writing / reading unit 203.
- the image group feature calculation unit 1206 reads all the image feature information corresponding to the images belonging to the image group, the image group feature calculation unit 1206 refers to the face correspondence table stored in the face correspondence table storage unit 1201 as the family face feature amount.
- the family face ID 1421 which is the face ID of the family face included in the image belonging to the corresponding image group and the family having the largest face area ratio among the family faces included in the image belonging to the corresponding image group Of the maximum face area 1422, the maximum number of family faces included in one image, the maximum number of family faces 1423, and the number of family faces for the number of images belonging to the corresponding image group.
- a family face number ratio 1424 which is a ratio of the number of images included
- a family face number ratio 1425 which is an average of the number of family faces included in one image
- the number of family faces in each image Chi most face area ratio family maximum face area ratio is large
- calculates the family face area ratio 1426 is obtained by dividing the number of images including the face of the family (step S1835).
- the image group feature calculation unit 1206 refers to the face correspondence table stored in the face correspondence table storage unit 1201, and uses the face of the friend included in the image belonging to the corresponding image group as the friend face feature amount.
- Friend face ID 1431 that is the face ID of the user, and the friend's maximum face area 1432 having the largest face area ratio among the faces of friends included in the images belonging to the corresponding image group, and included in one image
- the maximum number of friends 1433 that is the maximum number of friends' faces, and the ratio 1434 of the number of images that include the faces of friends relative to the number of images that belong to the corresponding image group 1434
- a friend face number ratio 1435 that is an average of the number of friend faces included in one image and a friend maximum face area ratio that has the largest face area ratio among friend faces in each image It calculates a friend face area ratio 1436 is obtained by dividing the number of images that include faces (step S 1840).
- the image group feature calculation unit 1206 refers to the face correspondence table stored in the face correspondence table storage unit 1201 and uses the face feature amount of the other person included in the image belonging to the corresponding image group as the face feature amount of the other person.
- Other face ID 1441 that is the face ID
- the maximum number of other people's faces 1443, and the number of other people's faces 1444 which is the ratio of the number of images including the faces of other people to the number of images belonging to the corresponding image group
- Other person face ratio 1445 which is the average of the number of faces of other people included in one image, and the maximum face area ratio of others having the largest face area ratio among the faces of friends in each image
- Calculating a person face area ratio 1446 is obtained by dividing the number of images containing the face (step S1845).
- the image group feature calculation unit 1206 calculates the calculated facial feature amount of the family, the calculated facial feature amount of the friend, the calculated facial feature amount of the other person, and the image group information received by the image group information receiving unit 211.
- the image group feature information is calculated using the name, and the calculated image group feature information is written into the image group feature information storage unit 1233 using the image group feature information writing / reading unit 204 (step S1850).
- the image group classification unit 208 reads the image group feature information written earlier using the image group feature information write / read unit 204, and stores it in the event feature information storage unit 234 using the event feature information write / read unit 209. Read the recorded event feature information.
- the image group classification unit 208 compares the read image group feature information with the event feature information (step S1855), and finds the corresponding event feature information in the image group feature information. An event in which an image group corresponding to the group feature information is to be classified is calculated.
- the image group classification unit 208 calculates that the image group corresponding to the image group feature information should be classified into an image group called another event when the corresponding event feature information cannot be found. To do.
- the image group classification unit 208 uses the image writing / reading unit 202 to store the name of the image group under the event directory corresponding to the event to be classified in the image storage unit 231. An image group directory with the same name is created, and a link to an image belonging to the image group is created under the image group directory, thereby classifying the image group (step S1860).
- the classification result output unit 210 displays the event name of the event to be classified calculated by the image group classification unit 208 on the display 193, and the image classification device 1200 ends the image group classification operation.
- FIG. 19 shows an image group 1900 having 16 images taken by the user as constituent elements.
- This image group 1900 is, for example, an image group in which the name of the image group is Koyama insect collection, and the image group ID is 0021.
- the image group feature information corresponding to the image group 1900 is, for example, image group feature information whose image group ID 1401 in FIG. 14 is 0021.
- the event name 1405 of the image group feature information whose image group ID 1401 is 0021 is insect collection, but this event name is blank until the event name is determined, and the event name is The event name decided for the first time after being decided is given.
- FIG. 20 shows an image group 2000 including images that are three photographs taken by the user as constituent elements.
- This image group 2000 is, for example, an image group whose name is T-gawa small athletic meet 2009, and the image group ID is 0022.
- the image group feature information corresponding to the image group 2000 is, for example, image group feature information whose image group ID 1401 in FIG. 14 is 0022.
- the event name 1405 of the image group feature information whose image group ID 1401 is 0022 is an athletic meet
- this event name is blank until the event name is determined, and the event name is determined.
- the event name determined for the first time after the event is given.
- FIG. 21 shows an image group 2100 that includes an image 2101 and an image 2103 that are photographs taken by the user.
- this image group 2100 is, for example, an image group whose name is the third birthday of a male A, and the image group ID is 0020.
- the recognition face 2102 and the recognition face 2104 are the same person A male, and the image 2101 and the image 2103 are images taken at the A male birthday party.
- the A man is determined to be a family by the image group feature calculation unit 1206.
- the image group feature information corresponding to the image group 2100 is, for example, image group feature information whose image group ID 1401 in FIG. 14 is 0020.
- the event name 1405 of the image group feature information whose image group ID 1401 is 0020 is the 3rd birthday of the A man, but this event name is blank until the event name is determined.
- the event name determined for the first time after the event name is determined is given.
- step S1855 when the image group classification unit 208 reads the image group feature information whose image group ID is 0021 from the image group feature information storage unit 233, the image group feature whose read image group ID is 0021. The information is compared with the event feature information read from the event feature information storage unit 1234.
- the image group feature information of the image group ID 0021 has a friend face number ratio of 0.44 as a non-family face number ratio and a friend face number ratio of 0.56 as a non-family face number ratio. Since the maximum number of friends other than the family is 2, the maximum number of friends is 2. Therefore, the image group feature information of the image group ID 0021 has a non-family face count rate of less than 0.5 and a non-family face count rate. Is less than 1 and the maximum number of non-family faces is less than 3, and the image group having the image group ID 0021 is determined to be classified as an insect collecting event. To do.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named Koyama insect collection under an event directory called an insect collection event directory (not shown) in the image storage unit 231. Then, a link to the image data of the image belonging to the image group ID 0021 is created under the created directory of collecting kosan insects.
- step S1855 when the image group classification unit 208 reads the image group feature information whose image group ID is 0022 from the image group feature information storage unit 233, the image group feature whose read image group ID is 0022. The information is compared with the event feature information read from the event feature information storage unit 1234.
- the image group feature information of the image group ID 0022 has a non-family face number rate of 1 and the non-family face number rate of 1 and the non-family face number rate of 3 As the number of faces, the maximum number of faces of others is 4, the image group feature information that is the image group ID 0022 has a non-family face count ratio of 0.8 or more and a non-family face number ratio of 1.5. As described above, it is determined that it corresponds to the reference information 1501 that the maximum number of faces other than the family is 3 or more, and it is determined that the image group having the image group ID 0022 should be classified as an athletic meet event.
- the image group classification unit 208 uses the image writing / reading unit 202 to create an image group directory named T-gawa small athletic meet 2009 under an unrepresented athletic event event directory in the image storage unit 231.
- a link to the image data of the image belonging to the image group ID 0022 is created under the created directory called T-gawa small athletic meet 2009.
- step S1855 when the image group classification unit 208 reads the image group feature information whose image group ID is 0020 from the image group feature information storage unit 233, the image group feature whose read image group ID is 0020. The information is compared with the event feature information read from the event feature information storage unit 1234.
- the image group classification unit 208 has a criterion that the image group feature information of the image group ID 0020 is a family face area ratio of 0.1 or more. It is determined that the information corresponds to the information 1501, and it is determined that the image group having the image group ID 0021 should be classified into an event called a birthday party.
- the image group classification unit 208 uses the image writing / reading unit 202 to store an image group named “A” 3rd birthday under an event directory (not shown) of the image storage unit 231 called a birthday event directory.
- a directory is created, and a link to the image data of the images belonging to the image group ID 0020 is created under the created directory of the third male A birthday.
- ⁇ Configuration> ⁇ Hardware Configuration of Image Classification Device 2200>
- the hardware configuration of the image classification device 2200 is the same as the hardware configuration of the image classification device 100. Therefore, the description is omitted here.
- FIG. 22 is a functional block diagram showing the main functional block configuration of the image classification device 2200.
- the differences from the image classification apparatus 100 according to the first embodiment are an event information receiving unit 212, an event feature calculation unit 207, an image group classification unit 208, an event feature information writing / reading unit 209, and an event feature information storage. And the clustering unit 2207 having a function of classifying image groups based only on the image group feature information is added.
- the clustering unit 2207 is connected to the image writing / reading unit 202, the image group feature information writing / reading unit 204, and the classification result output unit 210.
- the clustering unit 2207 uses the image group feature information writing / reading unit 204 to read a plurality of image group feature information from the image group feature information storage unit 233, and the color feature amount included in the read plurality of image group feature information.
- the main operations performed by the image classification device 2200 include a clustering operation for classifying an image group into events without using event feature information other than the operations described in the main operations performed by the image classification device 100 according to the first embodiment. is there.
- the clustering operation performed by the image classification device 2200 is that the clustering unit 2207 uses (1) image group groups as clusters so that image groups having similar values for one color in the average color feature amount become the same cluster. (2) For each classified cluster, perform the same classification as (1) for unselected colors, (3) repeat (2) until there are no unselected colors, and (4) image group group This is an operation of assigning an event name to each cluster that is a classification destination of the.
- FIG. 23 is a flowchart of the clustering operation performed by the image classification device 2200.
- the image classification device 2200 When the remote controller 197 receives from the user designation of a plurality of image group IDs indicating a plurality of image groups to be subjected to the clustering operation and an operation for starting the clustering operation, the image classification device 2200 performs the clustering operation. Start processing.
- the clustering unit 2207 uses the image group feature information writing / reading unit 204 to store the image group feature information corresponding to the image group ID specified by the user. (Step S2400).
- the clustering unit 2207 clusters the read image group feature information (hereinafter referred to as black clustering) using the black value of the color feature amount average among the read image group feature information (step S2405).
- This black clustering is: (1) Finding the minimum value of black values, and finding the minimum value + clustering value that is equal to or greater than the found minimum value (here, 0.25 is assumed, and hereinafter the clustering value is not written). Enter a numerical value of 0.25 directly into.) Classify the image groups with the following black values into the first cluster, and (2) find the maximum value of the black values and find the maximum value-0 Image group feature information having a black value that is greater than or equal to 25 and less than or equal to the maximum value is classified into a second cluster, and (3) an image group that is not classified into the first cluster or the second cluster Is classified into a third cluster.
- the clustering value is a reference value for determining whether or not the color feature value average color (here, black) values are similar to each other, and is stored in advance by the clustering unit 2207. It is.
- the clustering unit 2007 selects one of the clusters clustered by the black clustering (step S2410).
- the clustering unit 2007 uses the blue value of the color feature amount average among the image group feature information included in the selected cluster to cluster the image group feature information included in the selected cluster (hereinafter referred to as blue clustering). (Step S2415).
- This clustering by blue is the same as the clustering by black described above, in which black is replaced with blue.
- the clustering unit 2007 selects one of the clusters clustered by the blue clustering (step S2420).
- the clustering unit 2007 uses the green value of the color feature amount average among the image group feature information included in the selected cluster to cluster the image group feature information included in the selected cluster (hereinafter referred to as green clustering). (Step S2425).
- the clustering by green is the same as the clustering by black as described above, and the black is replaced with green.
- the clustering unit 2007 selects one of the clusters clustered by the clustering by green (step S2430).
- the clustering unit 2007 clusters image group feature information included in the selected cluster using the average white value of the color feature amount among the image group feature information included in the selected cluster (hereinafter referred to as white clustering). (Step S2435), and each classified cluster is set as a classified cluster that is not further clustered (step S2440).
- This white clustering is similar to the black clustering described above, in which black is replaced with white.
- the clustering unit 2207 searches for a cluster that has not been selected among the clusters by green clustering (step S2450), and if there is an unselected cluster, (Step S2450: Yes), one of the unselected clusters is selected, and the processes from Step S2430 to Step S2450 are repeatedly executed until there is no unselected cluster.
- step S2450 when there is no unselected cluster among the clusters by green clustering (step S2450: No), the clustering unit 2207 has an unselected cluster among the clusters by blue clustering. If there is an unselected cluster (step S2455: Yes), one of the unselected clusters is selected until there are no unselected clusters. The processes from step S2420 to step S2455 are repeatedly executed.
- step S2455 when there is no unselected cluster among the clusters by blue clustering (step S2455: No), the clustering unit 2207 has an unselected cluster among the clusters by black clustering. If there is an unselected cluster (step S2460: Yes), one of the unselected clusters is selected until there is no unselected cluster. The processes from step S2410 to step S2460 are repeatedly executed.
- step S2460 when there is no unselected cluster among the clusters by black clustering (step S2460: No), the image group to be classified is classified into one of the classification clusters. .
- the classification of the image group is completed in the processes from step S2400 to step S2650: No, but the clustering unit 2207 further performs the following process to determine the event name of each classification cluster.
- step S2460 when there is no unselected cluster among the clusters by black clustering (step S2460: No), the clustering unit 2207, for each classification cluster, the image group classified into the classification cluster. The average value of each color value is calculated (step S2465), and the color of the maximum average value among the calculated average values is extracted for each classification cluster (step S2470).
- the clustering unit 2207 determines the color of the maximum average value extracted for each classification cluster as the event name of the classification cluster (step S2475), and the image classification device 2200 ends the clustering operation.
- FIG. 24 is a schematic diagram of the clustering operation schematically showing the clustering operation performed by the image classification device 2200.
- the clustering unit 2207 uses the color feature amount average 2301 of the image group that is the target of the clustering operation among the image group feature information stored in the image group feature information storage unit, and the image that is the target of the clustering operation.
- the grouping of the group into a black event 2302, a blue event 2303, a green event 2304, and a white event 2305 is schematically shown.
- the clustering unit 2207 performs black clustering.
- the clustering unit 2207 Since the minimum value of the black values of the color feature amount average 2301 is 0.0 corresponding to the image group 1008, the clustering unit 2207 has a black value of 0.0 or more and 0.25 or less.
- Image group feature information corresponding to the image group ID 1001 hereinafter referred to as an image group 1001, and so on
- the image group 1008 is classified into a black first cluster.
- the clustering unit 2207 Since the maximum value of the black values of the color feature amount average 2301 is 0.6 corresponding to the image group 1002, the clustering unit 2207 has a black value of 0.35 or more and 0.6 or less. For example, the image group 1002 and the image group 1004 are classified into black second clusters.
- the clustering unit 2207 selects the first black cluster and performs clustering in blue.
- the clustering unit 2207 has 0.1 or more and 0.35 or less.
- the image group 1003, the image group 1006, the image group 1007, and the image group 1008 having a blue value of blue are classified into a blue first cluster.
- the clustering unit 2207 determines that. For example, the image group 1001 and the image group 1005 having a blue value of 45 or more and 0.7 or less are classified into a blue second cluster.
- the event name of the first classification cluster is a white event.
- the event name of the second classification cluster is green event.
- the event name of the third classification cluster is a blue event.
- the color of the maximum average value is 0.5 of black, so the event name of the fourth classification cluster is a black event.
- the image classification device 2200 selects one of the black event 2302, the blue event 2303, the green event 2304, and the white event 2305 as images corresponding to the image group ID 1001 to the image group ID 1008. Can be classified.
- ⁇ Supplement> an example of performing an image group classification operation, an event feature information generation operation, a face correspondence table generation operation, a clustering operation, and the like has been described for one embodiment of the image classification apparatus according to the present invention.
- the present invention can be modified, and the present invention is not limited to the image classification apparatus as shown in the above-described embodiment.
- the image stored in the image classification apparatus 100 is the data encoded by the JPEG method, but any image other than the JPEG method can be used as long as it can store a digital photograph as data. It may be encoded by an encoding method, for example, a PNG (Portable Network Graphics) method, a GIF (Graphics Interchange Format) method, or the like, or may be unencoded bitmap data.
- a PNG Portable Network Graphics
- GIF Graphics Interchange Format
- a digital photograph is shown as an example of content. However, as long as it is an image that can be stored as digital data, it may be data of a picture read by a scanner, for example.
- the interface 108, the communication device interface 109, the decoder 111, and the bus line 120 are integrated into the system LSI 110. However, the interface 108, the communication device interface 109, the decoder 111, and the bus line 120 are not necessarily integrated into one LSI. It may be realized.
- the decoder 111 is a DSP. However, the decoder 111 is not necessarily a DSP as long as it has a function of decoding encoded data. It may be a CPU different from the CPU 101, or may be a dedicated circuit composed of an ASIC or the like.
- the input device 170 has a function of accepting an operation command from a user wirelessly transmitted from the remote controller 197. However, if the input device 170 has a function of accepting an operation command from the user, For example, a configuration including a keyboard and a mouse and a function of receiving an operation command from a user via the keyboard and the mouse is not necessarily configured to receive an operation command transmitted wirelessly from the remote controller 197.
- the image group data receiving unit 201 receives designation of two or more images, and designates the designated image group as an image group included in one image group. If the association with the image group can be established, for example, the image group data receiving unit 201 receives the image data and the list of images belonging to the image group, and based on the received list, the image and the image group are combined.
- the image group data receiving unit 201 may include image data, information on the shooting time when the image data was shot, and information on the correspondence between the shooting time information and the image group. The image may be associated with the image group based on the received shooting time information. There.
- the image group data receiving unit 201 sequentially assigns an image ID to the read image.
- the image group data accepting unit 201 assigns an image ID corresponding to the image one-to-one. If possible, it is not always necessary to sequentially assign the image ID.
- black, blue, green, and white are specified as the colors specified by the color feature amount extraction unit 223. However, the colors are not limited to these, and may be red, yellow, and the like. It doesn't matter.
- the image feature calculation unit 205 calculates the color feature value after calculating the face feature value, and then calculates the object feature value. If the object feature amount can be calculated, it is not always necessary to start calculating the feature amounts in this order. For example, the calculation of the feature amount is started in the order of the color feature amount, the face feature amount, and the object feature amount. For example, the calculation of each feature amount may be started at the same time.
- the color feature amount extraction unit 223 calculates the color feature amount for all pixels included in the image. However, if the color feature amount can be calculated, the color feature amount extraction unit 223 does not necessarily include the image in the image. It is not necessary to calculate the color feature amount for all the included pixels. For example, after specifying a color for each pixel, pixels specified by the same color are adjacent to each other and there are more than the lower threshold number. For example, the color feature amount may be calculated for these pixels.
- FIG. 26 is a diagram showing images belonging to an image group.
- the image 2600 includes a pixel group 2601 to a pixel group 2609 composed of pixels less than the lower threshold number specified as black. There is no pixel specified as black other than these pixel groups.
- the image 2610 includes a pixel group 2611 composed of pixels equal to or more than the lower limit threshold number specified as black.
- black is not extracted as a color feature amount for the image 2600, but black is extracted as a color feature amount for the image 2610.
- an area where pixels specified by the same color are solidified to some extent is often a background area in an image such as the sky or the ground.
- the background region in the image includes the characteristics of the event that is the classification destination.
- the color feature amount extraction unit 223 having such a configuration can calculate the color feature amount based on the color included in the background in the image.
- the face model is, for example, information on the brightness of parts forming the face such as eyes, nose, mouth, and information on the relative positional relationship. Any other information may be used as long as the information can be used.
- the image group classification performed by the image group classification unit 208 is a classification in which, when there are a plurality of corresponding events, all of the corresponding events are to be classified, For example, by setting priorities to events, finding the corresponding events in order from the highest priority event, and setting only the corresponding event found first as the event to be classified, 1 There may be only one classification.
- the common feature calculation performed by the event feature calculation unit 207 is, for example, a color having a ratio of 0.4 or more in the color feature amount average 502 in each image group. If the common feature can be calculated, other methods such as logistic regression analysis and SVM (Support Vector Machine) method can be used. The common feature may be calculated by using a method such as the above.
- the event feature calculation unit 207 receives a set of an image group, an average of the color feature amount, and an event name of an event for which the image should be classified (hereinafter referred to as a correct event).
- the correct event is learned using the received average color feature amount, and the reference information corresponding to the correct event is calculated for each correct event.
- the reference information is classified into a green event with a probability of 80% if, for example, the green value of the color feature amount average is 0.4 or more.
- the facial features extracted by the image group feature calculating unit 1206 include, for example, the relative positional relationship of parts forming the face such as eyes, nose, mouth, and the area ratio of these parts. If it shows facial features that can classify the face, other facial features such as eye color, mole position, skin color, etc. It may be a thing, and it may be a combination of those representing a plurality of facial features.
- the criterion for determining the recognized face classified as the same person as a family is when the recognized face classified as the same person exists in a plurality of image groups,
- the criterion for determining a friend is a case where the recognized face classified as the same person exists only in a single image group, and there are two or more recognized faces classified as the same person.
- the criterion for determining that the person is another person is a recognition face that is determined not to be a family and is classified as the same person that is determined not to be a friend.
- the criterion for determining that a recognized face classified as the same person is a family is that the recognized face classified as the same person exists in three or more image groups and is a friend.
- the criterion for deciding is a case where the recognized face classified as the same person exists in two image groups, and the criterion for judging that the person is a different person is determined not to be a family and not to be a friend.
- the recognition faces may be classified as the same person.
- the names of family, friend, and others were used as the names of classification, it is not necessarily limited to names such as family, friends, and others.
- an image of a person's face that should be classified into a family in advance is used. It is possible to classify the family by a method such as classifying the person of the face having the same characteristics as the image of the person face to be classified into the registered family, and by visual observation by the user, A family may be divided by a method of dividing a person in an image into a family.
- the event feature information is the event feature information shown in FIG. 6, but the reference information other than the reference information shown in FIG. 6 or the event name shown in FIG. Event feature information including an event name may be used.
- the event feature information is the event feature information shown in FIG. 15, but the reference information other than the reference information shown in FIG. 15 and the event other than the event name shown in FIG. It may be event feature information including a name.
- FIG. 25 shows event feature information as an example of event feature information other than the event feature information shown in FIG. 6 and the event feature information shown in FIG.
- the event feature information shown in the figure is (a) when the criteria for classifying the event as a campfire is black 0.4 or more and 2501, and the condition of other people 5 or more 2511 is satisfied, (b ) The condition for classifying as an event of fireworks is a case where black is 0.4 or more and 2501 and the condition of less than 5 other people 2512 is satisfied, and (c) the criterion for classifying as an event of pool is blue 0. 4 or more and less than 0.8 2502, and the condition of 5 or more other people 2511 is satisfied. (D) The condition for classifying as an event of fishing is blue 0.4 or more and less than 0.8 2502.
- the criteria for classifying the event as a picnic is 0.4 to 2503 in green, and This is a case where the condition of five or more people 2511 is satisfied, and (f) the condition of classifying the event as insect collecting is a case of green 0.4 or more and 2503 and the condition that other people less than five people 2512 is satisfied.
- the criteria for classifying an event as ice skating is white 0.4 to 2504, and the condition that five or more other people 2511 are satisfied.
- the criteria for classifying the event as bathing is blue 0.8 or more and 2505, and This is a case where the condition of 5 or more of other people 2511 is satisfied
- the condition for classifying as an event of scuba diving is blue 0.8 or more and 2505
- the criteria for classifying the event as athletic meet is blue 0.4 to 2506 in the upper half area of the image, and 5 or more other people
- the condition to be classified as an event of roller skating is the condition of blue 0.4 to 2506 in the upper half area of the image and the condition of less than 512 2512 others.
- the criteria for classifying the event as kendo is a case where the upper half of the image is less than 0.44 in blue and satisfies the condition of 511 or more and 2511 others (n )
- the condition for classifying the event as a daily life of the house is the condition that the upper half of the image is less than blue 2507 and less than 2512 is 5 other people. This is a case where (16)
- the clustering operation performed using the color feature amount of the image group that is the classification target has been described as an example of the method of classifying the event performed by the clustering unit 2207, but the image group that is the classification target. Any classification method can be used as long as it can be classified.
- the clustering operation may be performed using a feature amount other than the color feature amount.
- an image group to be classified may be classified by using a method such as a K-means method.
- the clustering value is 0.25. However, if it is a criterion for determining whether or not the color values of the color feature amount average are similar to each other, For example, a value other than 0.25, such as 0.1, may be used.
- the clustering value is stored in advance by the clustering unit 2207, other configurations, for example, a configuration in which the clustering value is designated by the user may be used.
- the clustering unit 2207 has described an example in which each image group is classified into only one cluster. However, for example, one image group may be classified into two different clusters. I do not care.
- the image group classification operation or the like (see FIGS. 7, 10, 16, 18, and 23) shown in the first to third embodiments is performed by the CPU of the image classification apparatus and various CPUs connected to the CPU.
- a control program composed of program codes to be executed by a circuit can be recorded on a recording medium, or can be distributed and distributed via various communication paths.
- Examples of such a recording medium include an IC card, a hard disk, an optical disk, a flexible disk, and a ROM.
- the distributed and distributed control program is used by being stored in a memory or the like that can be read out by the CPU, and the CPU executes the control program to perform various functions as shown in each embodiment. It will be realized. A part of the control program is transmitted to a program executable device (CPU) separate from the image classification device via various communication paths and the like, and the part of the control program is executed in the separate program executable device. May be executed. (20) In the first embodiment, the face feature quantity extraction unit 222 sequentially assigns a face ID for specifying the recognized face to each recognized face, but it may be given avoiding duplication.
- the color feature amount indicates the feature of the entire image. However, if the color feature amount indicates the color feature of the image, for example, the upper half of the image It may be for a portion, or it may be for a plurality of portions, such as the left 10% portion of the image and the right 10% portion of the image.
- the event feature information is shown as an example composed of a combination of the reference information shown in FIG. 15 and the event name, but the event feature information shown in FIG. The information is not limited to information, and for example, various parameters of family members, friends, and others may be used as reference information.
- the clustering unit classifies the image group based on the clustering value determined by the color feature amount average value.
- the color feature amount is not necessarily limited. There is no need to classify based on the clustering value determined by the average value.
- the clustering unit may classify the image groups based on the clustering value determined so that the number of image groups classified into each of the classification destination categories is equal.
- the image classification device is based on image feature information indicating image features of all or a part of two or more images belonging to one image group. Based on the image group feature calculation unit for calculating the image group feature information indicating the feature of the image group, the image group feature information of one image group, and the reference information for classification, the image group is classified into a plurality of different ones. And an image group classification unit for classifying into any one of the classification destinations.
- this image classification device can classify images based on image characteristics other than the shooting time so that images belonging to the same image group are not classified into different categories. Has an effect.
- FIG. 27 is a functional block diagram showing a functional configuration of the image classification device 2700 in the above-described modification.
- the image classification device 2700 includes an image group feature calculation unit 2701 and an image group classification unit 2702.
- the image group feature calculation unit 2701 is connected to the image group classification unit 2702 and is based on image feature information indicating image features of all or a part of two or more images belonging to one image group.
- the image group feature information indicating the feature of the image group is calculated.
- the image group feature calculation unit 2701 is an image group feature information write / read unit 204, an image group feature calculation unit 206, an image group information reception unit 211, and an image group feature in the first embodiment (see FIG. 2).
- the information storage unit 233 is realized.
- the image group classification unit 2702 is connected to the image group feature calculation unit 2701, and based on the image group feature information of one image group and the reference information for classification, the image group is classified into a plurality of different classification destinations. It has a function of classifying any one of these categories.
- the image group classification unit 2702 includes an image group classification unit 208, an event feature information write / read unit 209, a classification result output unit 210, an event information reception unit 212, and an event in the first embodiment (see FIG. 2).
- the feature information storage unit 234 is realized.
- a reference information creation unit that creates the reference information for classification based on a plurality of pieces of image group feature information may be provided.
- An image feature calculation unit that calculates image feature information indicating the feature of the image from one image, and the image group feature calculation unit calculates the image group feature information as the image feature calculation. This may be performed based on the image feature information calculated by the unit.
- image feature information of images belonging to an image group can be created by itself, so that an image group can be classified without being given external image feature information. It has the effect of becoming
- the image feature calculation unit tries to detect a face included in the image by collating with a predetermined face model indicating the feature of the face, and is a face which is information related to the face included in the image.
- a face feature calculation unit that calculates information the image feature information calculated by the image feature calculation unit includes face information calculated by the face feature calculation unit, and the image group feature calculation unit calculates an image to be calculated
- the group feature information includes information related to a face
- the reference information for classification is information for determining which classification target is classified into different classification destinations. Including certain face reference information, wherein the image group classification unit classifies the image group into information relating to a face included in image group feature information of one image group and face reference information included in the reference information for classification. When It may be carried out on the basis of.
- Such a configuration has an effect that it is possible to classify an image group focused on a person who is a subject of an image belonging to the image group.
- the face feature calculation unit includes information related to the area of the detected face area included in the image in the face information to be calculated, and the image group feature calculation unit calculates the image group feature information to be calculated. Includes information related to the area of the face area detected by the face feature calculation unit included in the image belonging to the image group corresponding to the image group feature information, and the face reference information includes the face area information Information relating to the area includes face area reference information for determining which classification destination is classified into different classification destinations, and the image group classification unit assigns the classification of the image group to one The determination may be performed based on the information related to the area of the face area included in the image group feature information of the image group and the face area reference information included in the face reference information included in the reference information for classification.
- Photographers who shoot images of people tend to shoot such that the face of a person with high interest is larger and the face of a person with less interest is smaller.
- the area of the face region can be considered to reflect the degree of interest of the photographer for the photographer.
- Such a configuration has an effect that it is possible to classify an image group in which attention is paid to a person who is a subject of an image belonging to the image group.
- the information related to the area of the face area in which the face information is detected is the maximum face that is the face area having the largest area among the recognized face areas included in the image from which the face is detected.
- Information indicating the area of the area, and the information related to the area of the face area included in the image group feature information is a detection of a face in an image belonging to an image group corresponding to the image group feature information. In the image, it may be information indicating a value obtained by dividing the sum of the areas of the maximum face areas of the image in which the face is detected by the number of images in which the face is detected.
- the face in the maximum face area is considered to be the person who is the center of photographing, which is the person most focused on by the photographer among the group of persons who are the subjects of one image.
- the value obtained by dividing the total area of the maximum face area of the image from which the face is detected by the number of images from which the face has been detected is the person who is most interested in the image from which the face has been detected. It is thought that it is the average of the degree of interest in.
- Such a configuration has an effect that it is possible to classify an image group in which attention is paid to the average value of the interest level with respect to the central person who is the subject of the image belonging to the image group.
- an image storage unit for storing images belonging to a plurality of image groups is provided, and the face feature calculation unit attempts to detect a face for all images stored in the image storage unit, and The faces detected by the face feature calculation unit in all images stored in the image storage unit are classified into one of a plurality of face groups based on the detected face features.
- a face clustering unit and a face group in which two or more faces classified into the same face group by the face clustering unit exist in two or more image groups are classified into a first person group, and the face clustering unit converts the face group into the same face group.
- the image group feature calculating unit corresponds to the image group feature information to be calculated.
- the face reference information includes the face belonging to the first person group out of the area of the face area.
- Classified into Luke may include the face area reference information for determining.
- the person who is the subject is a first person group that exists in two or more image groups, a second person group that exists only in one image group, and there are a plurality of persons.
- the first person group and the third person group that do not belong to any of the first person group and the third person group are classified, and the image groups are classified by paying attention to the degree of interest of the persons for each of the divided person groups. It has the effect of becoming able to.
- the face feature calculation unit includes information related to the number of detected faces included in the image in the calculated face information
- the image group feature calculation unit includes in the image group feature information to be calculated
- the information related to the number of faces detected by the face feature calculation unit included in images belonging to the image group corresponding to the image group feature information is included, and the face reference information includes information related to the number of faces, It includes face number reference information for determining which of the different classification destinations is classified, and the image group classification unit determines the classification of the image group as an image group feature of one image group. This may be performed based on information related to the number of faces included in the information and face number reference information included in the face reference information included in the reference information for classification.
- Such a configuration has an effect that it is possible to classify image groups focusing on the number of persons who are subjects of images belonging to an image group.
- the information related to the number of detected faces of the face information is information indicating the number of detected faces included in an image from which a face is detected, and is included in the image group feature information
- the information relating to the number of faces to be displayed includes the number of images belonging to the image group corresponding to the image group feature information and the image including the detected face among images belonging to the image group corresponding to the image group feature information.
- the sum of the number of faces detected in the image including the detected face among the images belonging to the image group corresponding to the image group feature information corresponds to the image group feature information.
- Face number of information indicating the number of detected faces included in the largest image may be the at least one information.
- the ratio between the number of images belonging to the image group and the number of images including the detected face is the ratio of the images that include the detected face in the images belonging to the image group.
- the value obtained by dividing the total number of detected faces in the image including the detected face by the number of images including the detected face is detected in the image including the detected face. This is the average number of faces made.
- the ratio of the image that shows the detected face in the images that belong to the image group and the image that shows the detected face are detected.
- an image storage unit for storing images belonging to a plurality of image groups
- the face feature calculation unit tries to detect a face for all images stored in the image storage unit
- the faces detected by the face feature calculation unit in all images stored in the image storage unit are classified into one of a plurality of face groups based on the detected face features.
- a face clustering unit and a face group in which two or more faces classified into the same face group by the face clustering unit exist in two or more image groups are classified into a first person group, and the face clustering unit converts the face group into the same face group.
- the image group feature calculating unit corresponds to the image group feature information to be calculated.
- the face reference information belongs to the number of faces belonging to the first person group and the second person group among the number of faces.
- Number of faces for determining which classification destination is classified into different classification destinations for information relating to at least one of the number of faces and the number of faces belonging to the third person group Reference information may be included.
- the person who is the subject is a first person group that exists in two or more image groups, a second person group that exists only in one image group, and there are a plurality of persons, Classification into an image group focusing on the number of persons for each of the divided person groups by dividing into the third person group that does not belong to either the first person group or the second person group. Has the effect of being able to.
- the image feature calculation unit includes a color feature calculation unit that calculates color information that is information relating to a color included in the image, and the image feature information calculated by the image feature calculation unit is the color Including color information calculated by a feature calculation unit, the image group feature calculation unit includes information related to color in the image group feature information to be calculated, and the reference information for classification includes information related to color, Color reference information that is information for determining which of the different classification destinations is classified, and the image group classification unit determines the classification of the image group as an image of one image group The determination may be performed based on the color information included in the group feature information and the color reference information included in the classification reference information.
- Such a configuration has the effect that the image group can be classified based on the color characteristics of the images belonging to the image group.
- the image feature calculation unit limits the pixel value to be reflected in the color information to pixels constituting a pixel group including pixels adjacent to each other that are the same color or more than a predetermined number. Then, the calculation may be performed.
- pixels with the same color appear to a certain extent, for example, background colors such as the sky or the ground, pixels with the same color do not appear to a certain extent, for example, people It is possible to extract color feature amounts by distinguishing them from colors other than the background such as clothes.
- the image classification device may be configured to store an image group based on image feature information indicating image features of all or a part of two or more images belonging to the image group. Based on the image group feature calculation unit that calculates the image group feature information indicating the features and the image group feature information of the plurality of image groups, the image groups having similar features are classified into the same classification destination. An image group classification unit that classifies a plurality of image groups.
- the image classification device having the above-described configuration classifies images in units of image groups based on image group feature information calculated based on image features that are not limited to information on image shooting times. can do.
- the image classification device can classify images based on image characteristics other than the shooting time so that images belonging to the same image group are not classified into different categories.
- the image classification apparatus according to the present invention can be widely applied to devices having a function of storing a plurality of digital images.
- Image classification device 201 Image group data receiving unit 202 Image writing / reading unit 203 Image feature information writing / reading unit 204 Image group feature information writing / reading unit 205 Image feature calculating unit 206 Image group feature calculating unit 207 Event feature calculating unit 208 Image group classification unit 209 Event feature information write / read unit 210 Classification result output unit 211 Image group information reception unit 212 Event information reception unit 231 Image storage unit 232 Image feature information storage unit 233 Image group feature information storage unit 234 Event feature information storage Unit 221 image feature calculation control unit 222 face feature amount extraction unit 223 color feature amount extraction unit 224 object feature amount extraction unit
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Library & Information Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
Description
以下、本発明に係る画像分類装置の一実施形態として、複数枚の画像からなる画像グループ単位でその画像グループの特徴を示す画像グループ特徴情報を算出し、算出した画像グループ特徴情報と、画像グループの分類先であるイベントの特徴を示す情報とに基づいて、画像グループ単位で画像を互いに異なるイベントのうちのいずれかのイベントに分類する画像分類装置について説明する。
<画像分類装置100のハードウエア構成>
図1は、画像分類装置100の主要なハードウエア構成を示すハードウエアブロック図である。
図2は、画像分類装置100の主要な機能ブロックの構成を示す機能ブロック図である。
画像分類装置100の行う主な動作として、画像グループに属する画像を入力され、入力された画像グループをイベントに分類する画像グループ分類動作と、2以上の画像グループを指定され、指定された画像グループに共通の特徴を抽出することによって、新たにイベント特徴情報を生成するイベント特徴情報生成動作とがある。
図7は、画像分類装置100の行う画像グループ分類動作のフローチャートである。
以下、画像グループ分類動作についての具体例について、特にステップS755、ステップS760において、画像グループ分類部208が行う、画像グループを分類する処理について図面を用いて説明する。
この画像グループ800は、例えば、画像グループの名称を箱根2008夏とする画像グループであって、画像グループIDが0010であるものとする。
この画像グループ820は、例えば、画像グループの名称をニセコ2009冬とする画像グループであって、画像グループIDが0011であるものとする。
図10は、画像分類装置100の行うイベント特徴情報生成動作のフローチャートである。
以下、イベント特徴情報生成動作についての具体例について、図面を用いて説明する。
<実施の形態2>
以下、本発明に係る画像分類装置の一実施形態として、実施の形態1で説明した画像分類装置100の一部を変形し、記憶する全ての画像に含まれる認識される顔を、家族、友人、他人のいずれか1つに判定する機能が追加されている画像分類装置1200について説明する。
<画像分類装置1200のハードウエア構成>
画像分類装置1200のハードウエア構成は、画像分類装置100のハードウエア構成と同一である。従って、ここでは説明を省略する。
図12は、画像分類装置1200の主要な機能ブロック構成を示す機能ブロック図である。
画像分類装置1200の行う主な動作として、実施の形態1の画像分類装置100の行う主な動作に加えて、画像特徴情報記憶部232に記録されている全ての画像特徴情報に含まれる全ての顔IDによって示される認識顔に対して、顔の特徴を抽出し、抽出された顔の特徴に基づいて、同一人物である顔に同一のラベルを付与し、同一のラベルを付与された顔の集団が、家族、友人、又は、他人のうちのいずれかであると判定することで顔対応表を生成し、生成した顔対応表で、顔対応表記憶部1201で記憶されている顔対応表を更新する顔対応表生成動作がある。
図16は、画像分類装置1200が行う顔対応表生成動作のフローチャートである。
図18は、画像分類装置1200の行う、画像グループ分類動作のフローチャートである。
以下、画像グループ分類動作についての具体例について、特にステップS1855、ステップS1860において、画像グループ分類部208が行う、画像グループを分類する処理について図面を用いて説明する。
<実施の形態3>
以下、本発明に係る画像分類装置の一実施形態として、実施の形態1で説明した画像分類装置100の一部を変形し、イベント特徴情報を用いずに、画像グループを分類する画像分類装置2200について説明する。
<画像分類装置2200のハードウエア構成>
画像分類装置2200のハードウエア構成は、画像分類装置100のハードウエア構成と同一である。従って、ここでは説明を省略する。
図22は、画像分類装置2200の主要な機能ブロック構成を示す機能ブロック図である。
画像分類装置2200の行う主な動作として、実施の形態1の画像分類装置100の行う主な動作で説明した動作以外に、イベント特徴情報を用いずに、画像グループをイベントに分類するクラスタリング動作がある。
図23は、画像分類装置2200が行うクラスタリング動作のフローチャートである。
以下、クラスタリング動作についての具体例について、図面を用いて説明する。
<補足>
以上、本発明に係る画像分類装置の一実施形態について、画像グループ分類動作と、イベント特徴情報生成動作と、顔対応表生成動作と、クラスタリング動作等を行う例について説明したが、以下のように変形することも可能であり、本発明は上述した実施の形態に示した通りの画像分類装置に限られないことはもちろんである。
(1)実施の形態1において、画像分類装置100が記憶する画像として、JPEG方式で符号化されたデータとしたが、デジタル写真をデータとして記憶することができるものであれば、JPEG方式以外の符号化方式、例えばPNG(Portable Network Graphics)方式やGIF(Graphics Interchange Format)方式等で符号化されたものであっても構わないし、符号化されないビットマップ方式のデータであっても構わない。
(2)実施の形態1において、CPU101と、ROM102と、RAM103と、ハードディスク装置インターフェース104と、外部記録媒体読取書込装置インターフェース105と、USB制御装置インターフェース106と、出力装置インターフェース107と、入力装置インターフェース108と、通信装置インターフェース109と、デコーダ111と、バスライン120とが、システムLSI110に統合されているとしたが、必ずしも1つのLSIに統合されている必要はなく、複数の集積回路等で実現されていても構わない。
(3)実施の形態1において、デコーダ111は、DSPであるとしたが、符号化されたデータを復号する機能があれば、必ずしもDSPである必要はなく、例えば、CPU101が兼用する構成であっても構わないし、CPU101とは異なるCPUであっても構わないし、ASIC等で構成される専用回路であっても構わない。
(4)実施の形態1において、入力装置170は、リモコン197から無線で送信されるユーザからの操作コマンドを受け付ける機能を有する構成であるとしたが、ユーザからの操作コマンドを受け付ける機能があれば、必ずしもリモコン197から無線で送信される操作コマンドを受け付ける機能を有する構成でなくても、例えば、キーボードとマウスとを備え、キーボードとマウスとを介してユーザからの操作コマンドを受け付ける機能を有する構成であっても構わないし、ボタン群を備え、ボタン群を介してユーザからの操作コマンドを受け付ける機能を有する構成等であっても構わない。
(5)実施の形態1において、画像グループデータ受付部201が、2枚以上の画像の指定を受け付け、指定された画像群を、1つの画像グループに含まれる画像群とするとしたが、画像と画像グループとの対応付けを取ることができれば、例えば、画像グループデータ受付部201は、画像データと、画像グループに属する画像のリストとを受け取り、受け取ったリストに基づいて、画像と画像グループとを対応付けるといった構成であっても構わないし、例えば、画像グループデータ受付部201は、画像データと、その画像データが撮影された撮影時刻の情報と、撮影時刻の情報と画像グループとの対応関係の情報とを受け取り、受け取った撮影時刻の情報に基づいて、画像と画像グループとを対応付けるといった構成であっても構わない。
(6)実施の形態1において、画像グループデータ受付部201は、読み込んだ画像に対して、シーケンシャルに画像IDを付与するとしたが、画像に、画像と1対1に対応する画像IDを付与することができれば、必ずしもシーケンシャルに画像IDを付与しなくても構わない。
(7)実施の形態1において、色特徴量抽出部223が特定する色として、黒、青、緑、白としたが、これらの色に限られる必要はなく、例えば、赤、黄等であっても構わない。
(8)実施の形態1において、画像特徴算出部205は、顔特徴量を算出した後に色特徴量を算出し、その後、物体特徴量を算出するとしたが、顔特徴量と、色特徴量と、物体特徴量とを算出することができれば、必ずしもこの順番で各特徴量の算出を開始する必要はなく、例えば、色特徴量、顔特徴量、物体特徴量の順番で特徴量の算出を開始しても構わないし、例えば、同時に各特徴量の算出を開始するとしても構わない。
(9)実施の形態1において、色特徴量抽出部223は、画像に含まれる全ての画素を対象として色特徴量を算出するとしていたが、色特徴量を算出することができれば、必ずしも画像に含まれる全ての画素を対象として色特徴量を算出する必要はなく、例えば、各画素について色を特定した後、互いに同じ色に特定される画素が互いに隣接して下限閾値数以上存在する場合に限って、それらの画素を対象として、色特徴量を算出するとしても構わない。
(10)実施の形態1において、顔のモデルは、例えば、目、鼻、口等の顔を形成するパーツの輝度や、相対的な位置関係に関する情報等であるとしたが、顔を認識することができる情報であれば、これら以外の情報であっても構わない。
(11)実施の形態1において、画像グループ分類部208が行う画像グループの分類は、該当するイベントが複数ある場合には、該当するイベントの全てを分類すべきイベントとする分類であったが、例えば、イベントに優先順位を設けて、優先順位の高いイベントから順に該当するイベントを見つけていき、最初にみつかった該当するイベントのみを分類すべきイベントとするといったような、分類すべきイベントを1つのみとする分類であっても構わない。
(12)実施の形態1において、イベント特徴算出部207が行う、共通する特徴の算出とは、例えば、各画像グループにおける色特徴量平均502のうち、0.4以上の割合の色が同じ色である場合に、その色を共通特徴とすることであるとしたが、共通する特徴の算出をすることができれば、これ以外の方法、例えば、ロジスティック回帰分析法や、SVM(Support Vector Machine)法等の方法を用いることによって共通特徴を算出するとしても構わない。
(13)実施の形態2において、画像グループ特徴算出部1206が抽出する顔の特徴は、例えば、目、鼻、口等の顔を形成するパーツの相対的な位置関係や、これらパーツの面積比率等のことであるとしたが、顔を区分することができる顔の特徴を示すものであれば、これら以外、例えば、目の色や、ほくろの位置、肌の色等といった顔の特徴を示すものであっても構わないし、複数の顔の特徴を表すものの組み合わせであっても構わない。
(14)実施の形態2において、同一人物として区分された認識顔を家族であると判定する基準は、その同一人物として区分された認識顔が、複数の画像グループに存在する場合であって、友人であると判定する基準は、その同一人物として区分された認識顔が、単一の画像グループにのみ存在し、かつ、同一人物として区分された認識顔の数が2つ以上存在する場合であって、他人であると判定する基準は、家族でないと判定され、かつ、友人でないと判定された同一人物として区分された認識顔であるとしたが、必ずしもこのような判定基準でなくても、例えば、同一人物として区分された認識顔を家族であると判定する基準は、その同一人物として区分された認識顔が、3つ以上の画像グループに存在する場合であって、友人であると判断する基準は、その同一人物として区分された認識顔が、2つの画像グループに存在する場合であって、他人であると判定する基準は、家族でないと判定され、かつ、友人でないと判定された同一人物として区分された認識顔であるとしても構わない。
また、例えば、予め家族に区分されるべき人物の顔の画像を登録しておいて、この登録された家族に区分されるべき人物顔の画像と同じ特徴を持つ顔の人物を家族に区分するといった方法で家族を区分しても構わないし、ユーザよる目視によって、画像に写っている人物を家族に区分するといった方法で家族を区分しても構わない。
(15)実施の形態1において、イベント特徴情報が、図6で示されるイベント特徴情報であるとしたが、図6で示される基準情報以外の基準情報や、図6で示されるイベント名以外のイベント名を含んでいるイベント特徴情報であっても構わない。
(16)実施の形態3において、クラスタリング部2207が行うイベントに分類する方法の一例として、分類対象である画像グループの色特徴量を用いて行うクラスタリング動作について説明したが、分類対象である画像グループを分類することができれば、どのような分類方法であっても構わない。例えば、色特徴量以外の特徴量を用いて行うクラスタリング動作であっても構わないし、例えば、K-means法等の方法を用いることによって、分類対象である画像グループを分類するとしても構わない。
(17)実施の形態3において、クラスタリング値は0.25であるとしたが、色特徴量平均の色の値が互いに類似しているか否かを判断するための基準となるものであれば、例えば、0.1といったように、0.25以外の値であっても構わない。
(18)実施の形態3において、クラスタリング部2207は、各画像グループが、1つのクラスタにのみ分類する例について説明したが、例えば、1つの画像グループを、互いに異なる2つのクラスタに分類するとしても構わない。
(19)実施の形態1乃至3で示した、画像グループ分類動作等(図7、図10、図16、図18、図23参照)を画像分類装置のCPU、及びそのCPUに接続された各種回路に実行させるためのプログラムコードからなる制御プログラムを、記録媒体に記録すること、又は各種通信路等を介して流通させ頒布することもできる。このような記録媒体には、ICカード、ハードディスク、光ディスク、フレキシブルディスク、ROM等がある。流通、頒布された制御プログラムはCPUに読み出だされ得るメモリ等に格納されることにより利用に供され、そのCPUがその制御プログラムを実行することにより各実施形態で示したような各種機能が実現されるようになる。なお、制御プログラムの一部を画像分類装置とは別個のプログラム実行可能な装置(CPU)に各種通信路等を介して送信して、その別個のプログラム実行可能な装置においてその制御プログラムの一部を実行させることとしてもよい。
(20)実施の形態1において、顔特徴量抽出部222は、認識した顔それぞれに、その認識顔を特定するための顔IDを、シーケンシャルに付与するとしたが、重複を避けて付与することができれば、必ずしもシーケンシャルに付与しなくても構わない。
(21)実施の形態1において、色特徴量は、画像全体に対して特徴を示すものであるとしたが、画像の色の特徴を示すものであれば、例えば、画像の上半分といったような一部分に対すものであっても構わないし、画像の左10%の部分と画像の右10%の部分といったように複数の一部分に対するものであっても構わない。
(22)実施の形態2において、イベント特徴情報は、例として、図15に示されている基準情報と、イベント名との組によって構成されるものについて示したが、図15に示されるイベント特徴情報に限られるものではなく、例えば、家族や友人や他人の様々なパラメータを基準情報に用いたものであっても構わない。
(23)実施の形態3において、クラスタリング部は、色特徴量平均の値によって決定されるクラスタリング値に基づいて画像グループを分類するとしたが、画像グループを分類することができれば、必ずしも、色特徴量平均の値によって決定されるクラスタリング値に基づいて分類する必要はない。
(24)以下、さらに本発明の一実施形態に係る画像分類装置の構成及びその変形例と各効果について説明する。
201 画像グループデータ受付部
202 画像書込読出部
203 画像特徴情報書込読出部
204 画像グループ特徴情報書込読出部
205 画像特徴算出部
206 画像グループ特徴算出部
207 イベント特徴算出部
208 画像グループ分類部
209 イベント特徴情報書込読出部
210 分類結果出力部
211 画像グループ情報受付部
212 イベント情報受付部
231 画像記憶部
232 画像特徴情報記憶部
233 画像グループ特徴情報記憶部
234 イベント特徴情報記憶部
221 画像特徴算出制御部
222 顔特徴量抽出部
223 色特徴量抽出部
224 物体特徴量抽出部
Claims (17)
- 1つの画像グループに属する画像のうち全部又は一部の2枚以上の画像についての、画像の特徴を示す画像特徴情報に基づいて、画像グループの特徴を示す画像グループ特徴情報を算出する画像グループ特徴算出部と、
1つの画像グループの画像グループ特徴情報と、分類用の基準情報とに基づいて、当該画像グループを、互いに異なる複数の分類先のうちのいずれかの分類先に分類する画像グループ分類部とを備えることを特徴とする
画像分類装置。 - 複数の画像グループ特徴情報に基づいて、前記分類用の基準情報を作成する基準情報作成部を備えることを特徴とする
請求項1記載の画像分類装置。 - 1枚の画像から当該画像の特徴を示す画像特徴情報を算出する画像特徴算出部を備え、
前記画像グループ特徴算出部は、前記画像グループ特徴情報の算出を、前記画像特徴算出部によって算出された画像特徴情報に基づいて行うことを特徴とする
請求項2記載の画像分類装置。 - 前記画像特徴算出部は、顔の特徴を示す予め定められた顔のモデルと照合することで画像に含まれる顔の検出を試み、画像に含まれる顔に係る情報である顔情報を算出する顔特徴算出部を含み、
前記画像特徴算出部によって算出される画像特徴情報は、前記顔特徴算出部によって算出された顔情報を含み、
前記画像グループ特徴算出部は、算出する画像グループ特徴情報に、顔に係る情報を含ませ、
前記分類用の基準情報は、顔に係る情報を、互いに異なる分類先のうち、いずれの分類先に分類されるかを決定するための情報である顔基準情報を含み、
前記画像グループ分類部は、前記画像グループの分類を、1つの画像グループの画像グループ特徴情報に含まれる顔に係る情報と、分類用の基準情報に含まれる顔基準情報とに基づいて行うことを特徴とする
請求項3記載の画像分類装置。 - 前記顔特徴算出部は、算出する顔情報に、画像に含まれる検出した顔の領域の面積に係る情報を含ませ、
前記画像グループ特徴算出部は、算出する画像グループ特徴情報に、当該画像グループ特徴情報に対応する画像グループに属する画像に含まれる、前記顔特徴算出部によって検出された顔の領域の面積に係る情報を含ませ、
前記顔基準情報は、顔の領域の面積に係る情報を、互いに異なる分類先のうち、いずれの分類先に分類されるかを決定するための顔面積基準情報を含み、
前記画像グループ分類部は、前記画像グループの分類を、1つの画像グループの画像グループ特徴情報に含まれる顔の領域の面積に係る情報と、分類用の基準情報に含まれる顔基準情報に含まれる顔面積基準情報とに基づいて行うことを特徴とする
請求項4記載の画像分類装置。 - 前記顔情報の検出された顔の領域の面積に係る情報とは、顔を検出された画像に含まれる認識顔の領域のうち、面積が最大の顔の領域である最大顔領域の面積を示す情報であって、
前記画像グループ特徴情報に含まれる顔の領域の面積に係る情報とは、当該画像グループ特徴情報に対応する画像グループに属する画像のうちの顔を検出された画像において、当該顔を検出された画像の最大顔領域の面積の総和を、当該顔を検出された画像の枚数で除算することで得られる値を示す情報であることを特徴とする
請求項5記載の画像分類装置。 - 複数の画像グループに属する画像を記憶するための画像記憶部を備え、
前記顔特徴算出部は、前記画像記憶部に記憶されている全ての画像について顔の検出を試み、
前記画像記憶部に記憶されている全ての画像において前記顔特徴算出部によって検出された顔を、当該検出された顔の特徴に基づいて、複数の顔グループのうちのいずれかの顔グループに分類する顔クラスタリング部と、
前記顔クラスタリング部によって同一の顔グループに分類された顔が2つ以上の画像グループに存在する顔グループを第1人物群に区分し、前記顔クラスタリング部によって同一の顔グループにクラスタリングされた顔が1つの画像グループにのみ存在し、かつ、複数存在する顔グループを第2人物群に区分し、前記第1人物群と前記第2人物群とのいずれにも属さない顔グループを第3人物群に区分する人物群区分部とを備え、
前記画像グループ特徴算出部は、算出する画像グループ特徴情報に、当該画像グループ特徴情報に対応する画像グループに属する画像に含まれる、前記顔特徴算出部によって検出された顔の領域の面積のうち、前記第1人物群に属する顔の領域の面積、前記第2人物群に属する顔の領域面積、及び、前記第3人物群に属する顔の領域の面積のうち、少なくとも1つに係る情報を含ませ、
前記顔基準情報は、顔の領域の面積のうち、前記第1人物群に属する顔の領域の面積、前記第2人物群に属する顔の領域面積、及び、前記第3人物群に属する顔の領域の面積のうち、少なくとも1つに係る情報を、互いに異なる分類先のうち、いずれの分類先に分類されるかを決定するための顔面積基準情報を含むことを特徴とする
請求項5記載の画像分類装置。 - 前記顔特徴算出部は、算出する顔情報に、画像に含まれる検出した顔の数に係る情報を含ませ、
前記画像グループ特徴算出部は、算出する画像グループ特徴情報に、当該画像グループ特徴情報に対応する画像グループに属する画像に含まれる、前記顔特徴算出部によって検出された顔の数に係る情報を含ませ、
前記顔基準情報は、顔の数に係る情報を、互いに異なる分類先のうち、いずれの分類先に分類されるかを決定するための顔数基準情報を含み、
前記画像グループ分類部は、前記画像グループの分類を、1つの画像グループの画像グループ特徴情報に含まれる顔の数に係る情報と、分類用の基準情報に含まれる顔基準情報に含まれる顔数基準情報とに基づいて行うことを特徴とする
請求項4記載の画像分類装置。 - 前記顔情報の検出された顔の数に係る情報とは、顔を検出された画像に含まれる検出された顔の数を示す情報であって、
前記画像グループ特徴情報に含まれる顔の数に係る情報とは、当該画像グループ特徴情報に対応する画像グループに属する画像の枚数と、当該画像グループ特徴情報に対応する画像グループに属する画像のうち検出された顔を含む画像の枚数との比率を示す情報と、当該画像グループ特徴情報に対応する画像グループに属する画像のうち検出された顔を含む画像において検出された顔の数の総和を、当該画像グループ特徴情報に対応する画像グループに属する画像のうち検出された顔を含む画像の枚数で除算した値を示す情報と、当該画像グループ特徴情報に対応する画像グループに属する画像のうち検出された顔の数が最も多い画像に含まれる検出された顔の数を示す情報とのうちの、少なくとも1つの情報であることを特徴とする
請求項8記載の画像分類装置。 - 複数の画像グループに属する画像を記憶するための画像記憶部を備え、
前記顔特徴算出部は、前記画像記憶部に記憶されている全ての画像について顔の検出を試み、
前記画像記憶部に記憶されている全ての画像において前記顔特徴算出部によって検出された顔を、当該検出された顔の特徴に基づいて、複数の顔グループのうちのいずれかの顔グループに分類する顔クラスタリング部と、
前記顔クラスタリング部によって同一の顔グループに分類された顔が2つ以上の画像グループに存在する顔グループを第1人物群に区分し、前記顔クラスタリング部によって同一の顔グループにクラスタリングされた顔が1つの画像グループにのみ存在し、かつ、複数存在する顔グループを第2人物群に区分し、前記第1人物群と前記第2人物群とのいずれにも属さない顔グループを第3人物群に区分する人物群区分部とを備え、
前記画像グループ特徴算出部は、算出する画像グループ特徴情報に、当該画像グループ特徴情報に対応する画像グループに属する画像に含まれる、前記顔特徴算出部によって検出された顔数のうち、前記第1人物群に属する顔の数、前記第2人物群に属する顔の数、及び、前記第3人物群に属する顔の数のうち、少なくとも1つに係る情報を含ませ、
前記顔基準情報は、顔の数のうち、前記第1人物群に属する顔の数、前記第2人物群に属する顔の数、及び、前記第3人物群に属する顔の数のうち、少なくとも1つに係る情報を、互いに異なる分類先のうち、いずれの分類先に分類されるかを決定するための顔数基準情報を含むことを特徴とする
請求項8記載の画像分類装置。 - 前記画像特徴算出部は、画像に含まれる色に係る情報である色情報を算出する色特徴算出部を含み、
前記画像特徴算出部によって算出される画像特徴情報は、前記色特徴算出部によって算出された色情報を含み、
前記画像グループ特徴算出部は、算出する画像グループ特徴情報に、色に係る情報を含ませ、
前記分類用の基準情報は、色に係る情報を、互いに異なる分類先のうち、いずれの分類先に分類されるかを決定するための情報である色基準情報を含み、
前記画像グループ分類部は、前記画像グループの分類を、1つの画像グループの画像グループ特徴情報に含まれる色に係る情報と、分類用の基準情報に含まれる色基準情報とに基づいて行うことを特徴とする
請求項3記載の画像分類装置。 - 前記画像特徴算出部は、前記色情報へ反映させる画素値を、予め定められた数以上の、互いに同じ色である、互いに隣接する画素からなる画素群を構成する画素に限って、前記算出を行う
ことを特徴とする
請求項11記載の画像分類装置。 - 画像グループに属する全部又は一部の2枚以上の画像についての、画像の特徴を示す画像特徴情報に基づいて、画像グループの特徴を示す画像グループ特徴情報を算出する画像グループ特徴算出部と、
複数の画像グループの画像グループ特徴情報に基づいて、互いに類似する特徴を有する画像グループが同じ分類先に分類されるように、当該複数の画像グループを分類する画像グループ分類部とを備えることを特徴とする
画像分類装置 - 画像を分類する画像分類装置を用いて行う画像分類方法であって、
1つの画像グループに属する画像のうち全部又は一部の2枚以上の画像についての、画像の特徴を示す画像特徴情報に基づいて、画像グループの特徴を示す画像グループ特徴情報を算出する画像グループ特徴算出ステップと、
1つの画像グループの画像グループ特徴情報と、分類用の基準情報とに基づいて、当該画像グループを分類する画像グループ分類ステップとを備えることを特徴とする
画像分類方法。 - コンピュータに、画像を分類する画像分類装置として機能させるための画像分類プログラムであって、
コンピュータに、
1つの画像グループに属する画像のうち全部又は一部の2枚以上の画像についての、画像の特徴を示す画像特徴情報に基づいて、画像グループの特徴を示す画像グループ特徴情報を算出する画像グループ特徴算出部と、
1つの画像グループの画像グループ特徴情報と、分類用の基準情報とに基づいて、当該画像グループを分類する画像グループ分類部とを備えることを特徴とする画像分類装置として機能させることを特徴とする
画像分類プログラム。 - コンピュータに、画像を分類する画像分類装置として機能させるための画像分類プログラムを記録している記録媒体であって、
前記画像分類プログラムは、コンピュータに、
1つの画像グループに属する画像のうち全部又は一部の2枚以上の画像についての、画像の特徴を示す画像特徴情報に基づいて、画像グループの特徴を示す画像グループ特徴情報を算出する画像グループ特徴算出部と、
1つの画像グループの画像グループ特徴情報と、分類用の基準情報とに基づいて、当該画像グループを分類する画像グループ分類部とを備えることを特徴とする画像分類装置として機能させるプログラムであることを特徴とする
記録媒体。 - 1つの画像グループに属する画像のうち全部又は一部の2枚以上の画像についての、画像の特徴を示す画像特徴情報に基づいて、画像グループの特徴を示す画像グループ特徴情報を算出する画像グループ特徴算出部と、
1つの画像グループの画像グループ特徴情報と、分類用の基準情報とに基づいて、当該画像グループを分類する画像グループ分類部とを備えることを特徴とする
半導体集積回路。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011550843A JP5469181B2 (ja) | 2010-01-25 | 2011-01-19 | 画像分類装置、方法、プログラム、プログラムを記録する記録媒体及び集積回路 |
US13/520,398 US8712168B2 (en) | 2010-01-25 | 2011-01-19 | Image sorting device, method, program, and integrated circuit and storage medium storing said program |
CN201180006946.3A CN102725756B (zh) | 2010-01-25 | 2011-01-19 | 图像分类装置、方法、程序、记录程序的记录介质及集成电路 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010-012794 | 2010-01-25 | ||
JP2010012794 | 2010-01-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011089884A1 true WO2011089884A1 (ja) | 2011-07-28 |
Family
ID=44306686
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/000235 WO2011089884A1 (ja) | 2010-01-25 | 2011-01-19 | 画像分類装置、方法、プログラム、プログラムを記録する記録媒体及び集積回路 |
Country Status (4)
Country | Link |
---|---|
US (1) | US8712168B2 (ja) |
JP (1) | JP5469181B2 (ja) |
CN (1) | CN102725756B (ja) |
WO (1) | WO2011089884A1 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2014526738A (ja) * | 2011-09-07 | 2014-10-06 | インテレクチュアル ベンチャーズ ファンド 83 エルエルシー | 光源検出を利用したイベント分類方法 |
US9141856B2 (en) | 2011-07-13 | 2015-09-22 | Panasonic Intellectual Property Corporation Of America | Clothing image analysis apparatus, method, and integrated circuit for image event evaluation |
JP2016538567A (ja) * | 2013-09-19 | 2016-12-08 | ロレアル ソシエテ アノニム | 面の色及びスペクトルを測定しカテゴリ分類するためのシステム及び方法 |
JP2017162025A (ja) * | 2016-03-07 | 2017-09-14 | 株式会社東芝 | 分類ラベル付与装置、分類ラベル付与方法、およびプログラム |
JP2022541081A (ja) * | 2020-06-25 | 2022-09-22 | グーグル エルエルシー | 人々のグループおよび画像ベースの作成物の自動生成 |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8712168B2 (en) * | 2010-01-25 | 2014-04-29 | Panasonic Corporation | Image sorting device, method, program, and integrated circuit and storage medium storing said program |
US9465993B2 (en) * | 2010-03-01 | 2016-10-11 | Microsoft Technology Licensing, Llc | Ranking clusters based on facial image analysis |
JP2014229178A (ja) * | 2013-05-24 | 2014-12-08 | 株式会社東芝 | 電子機器および表示制御方法、プログラム |
US10043184B2 (en) * | 2014-05-30 | 2018-08-07 | Paypal, Inc. | Systems and methods for implementing transactions based on facial recognition |
JP6660119B2 (ja) | 2015-08-07 | 2020-03-04 | キヤノン株式会社 | 情報処理装置、情報処理方法、並びにプログラム |
JP6667224B2 (ja) | 2015-08-07 | 2020-03-18 | キヤノン株式会社 | 画像処理装置およびその制御方法、並びにプログラム |
US10860887B2 (en) * | 2015-11-16 | 2020-12-08 | Samsung Electronics Co., Ltd. | Method and apparatus for recognizing object, and method and apparatus for training recognition model |
US10237602B2 (en) * | 2016-11-30 | 2019-03-19 | Facebook, Inc. | Methods and systems for selecting content for a personalized video |
CN108256479B (zh) * | 2018-01-17 | 2023-08-01 | 百度在线网络技术(北京)有限公司 | 人脸跟踪方法和装置 |
CN108171207A (zh) * | 2018-01-17 | 2018-06-15 | 百度在线网络技术(北京)有限公司 | 基于视频序列的人脸识别方法和装置 |
CN109740664B (zh) * | 2018-12-28 | 2023-01-10 | 东莞中国科学院云计算产业技术创新与育成中心 | 柔性物体分类方法、装置、计算机设备和存储介质 |
CN113901981A (zh) * | 2021-08-27 | 2022-01-07 | 深圳云天励飞技术股份有限公司 | 设备聚类方法、装置、计算机设备及存储介质 |
CN114140655A (zh) * | 2022-01-29 | 2022-03-04 | 深圳市中讯网联科技有限公司 | 图像分类方法、装置、存储介质及电子设备 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005149323A (ja) * | 2003-11-18 | 2005-06-09 | Canon Inc | 画像処理システム及び画像処理装置並びに画像処理方法 |
JP2007129434A (ja) * | 2005-11-02 | 2007-05-24 | Sony Corp | 情報処理装置および方法、並びにプログラム |
JP2008071112A (ja) * | 2006-09-14 | 2008-03-27 | Casio Comput Co Ltd | 画像分類装置、画像分類方法及びプログラム |
JP2009265873A (ja) * | 2008-04-24 | 2009-11-12 | Nikon Corp | 画像群の標題付与装置、およびカメラ |
JP2009301119A (ja) * | 2008-06-10 | 2009-12-24 | Olympus Corp | 画像表示装置 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04232774A (ja) | 1990-12-28 | 1992-08-21 | Mitsubishi Paper Mills Ltd | 改ざん防止用感圧記録シート |
US5842194A (en) * | 1995-07-28 | 1998-11-24 | Mitsubishi Denki Kabushiki Kaisha | Method of recognizing images of faces or general images using fuzzy combination of multiple resolutions |
JP2001333352A (ja) | 2000-05-23 | 2001-11-30 | Fuji Photo Film Co Ltd | 画像ファイリング装置および画像ファイリング方法 |
US7912246B1 (en) * | 2002-10-28 | 2011-03-22 | Videomining Corporation | Method and system for determining the age category of people based on facial images |
JP2006350546A (ja) | 2005-06-14 | 2006-12-28 | Toshiba Corp | 情報処理装置、画像分類方法、および情報処理システム |
JP2007087253A (ja) * | 2005-09-26 | 2007-04-05 | Fujifilm Corp | 画像補正方法および装置 |
US7783085B2 (en) * | 2006-05-10 | 2010-08-24 | Aol Inc. | Using relevance feedback in face recognition |
CN101211341A (zh) * | 2006-12-29 | 2008-07-02 | 上海芯盛电子科技有限公司 | 图像智能模式识别搜索方法 |
US20100278396A1 (en) | 2008-01-18 | 2010-11-04 | Nikon Corporation | Image group title assigning device, image grouping device, representative image determination device for image group, image display device, camera, and image display program |
JP2011107997A (ja) * | 2009-11-18 | 2011-06-02 | Sony Corp | 情報処理装置、情報処理方法、およびプログラム |
US8712168B2 (en) * | 2010-01-25 | 2014-04-29 | Panasonic Corporation | Image sorting device, method, program, and integrated circuit and storage medium storing said program |
CN102549579B (zh) * | 2010-08-04 | 2016-06-08 | 松下电器(美国)知识产权公司 | 图像分类装置、方法以及集成电路 |
-
2011
- 2011-01-19 US US13/520,398 patent/US8712168B2/en active Active
- 2011-01-19 WO PCT/JP2011/000235 patent/WO2011089884A1/ja active Application Filing
- 2011-01-19 JP JP2011550843A patent/JP5469181B2/ja active Active
- 2011-01-19 CN CN201180006946.3A patent/CN102725756B/zh active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005149323A (ja) * | 2003-11-18 | 2005-06-09 | Canon Inc | 画像処理システム及び画像処理装置並びに画像処理方法 |
JP2007129434A (ja) * | 2005-11-02 | 2007-05-24 | Sony Corp | 情報処理装置および方法、並びにプログラム |
JP2008071112A (ja) * | 2006-09-14 | 2008-03-27 | Casio Comput Co Ltd | 画像分類装置、画像分類方法及びプログラム |
JP2009265873A (ja) * | 2008-04-24 | 2009-11-12 | Nikon Corp | 画像群の標題付与装置、およびカメラ |
JP2009301119A (ja) * | 2008-06-10 | 2009-12-24 | Olympus Corp | 画像表示装置 |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9141856B2 (en) | 2011-07-13 | 2015-09-22 | Panasonic Intellectual Property Corporation Of America | Clothing image analysis apparatus, method, and integrated circuit for image event evaluation |
JP2014526738A (ja) * | 2011-09-07 | 2014-10-06 | インテレクチュアル ベンチャーズ ファンド 83 エルエルシー | 光源検出を利用したイベント分類方法 |
JP2016538567A (ja) * | 2013-09-19 | 2016-12-08 | ロレアル ソシエテ アノニム | 面の色及びスペクトルを測定しカテゴリ分類するためのシステム及び方法 |
JP2017162025A (ja) * | 2016-03-07 | 2017-09-14 | 株式会社東芝 | 分類ラベル付与装置、分類ラベル付与方法、およびプログラム |
JP2022541081A (ja) * | 2020-06-25 | 2022-09-22 | グーグル エルエルシー | 人々のグループおよび画像ベースの作成物の自動生成 |
JP7167318B2 (ja) | 2020-06-25 | 2022-11-08 | グーグル エルエルシー | 人々のグループおよび画像ベースの作成物の自動生成 |
US11783521B2 (en) | 2020-06-25 | 2023-10-10 | Google Llc | Automatic generation of people groups and image-based creations |
Also Published As
Publication number | Publication date |
---|---|
JPWO2011089884A1 (ja) | 2013-05-23 |
CN102725756B (zh) | 2015-06-03 |
US8712168B2 (en) | 2014-04-29 |
US20120281887A1 (en) | 2012-11-08 |
CN102725756A (zh) | 2012-10-10 |
JP5469181B2 (ja) | 2014-04-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5469181B2 (ja) | 画像分類装置、方法、プログラム、プログラムを記録する記録媒体及び集積回路 | |
JP5739428B2 (ja) | 画像分類装置、方法、プログラム、プログラムを記録する記録媒体及び集積回路 | |
JP6023058B2 (ja) | 画像処理装置、画像処理方法、プログラム、集積回路 | |
JP5890325B2 (ja) | 画像データ処理装置、方法、プログラム及び集積回路 | |
JP5818799B2 (ja) | デジタル画像の美的品質の推定方法 | |
EP2567536B1 (en) | Generating a combined image from multiple images | |
US20180268205A1 (en) | Picture Ranking Method, and Terminal | |
US8526742B2 (en) | Image processing apparatus, method, and program that classifies data of images | |
US20110115937A1 (en) | Information processing apparatus, information processing method, and program | |
JP2006236218A (ja) | 電子アルバム表示システム、電子アルバム表示方法、及び電子アルバム表示プログラム | |
CN101262561B (zh) | 成像设备及其控制方法 | |
CN101753823A (zh) | 自动标记图像的装置及其方法 | |
JP2007513413A (ja) | 強調画像を選択するための内容認識 | |
JP2011096136A (ja) | オブジェクト識別装置及びオブジェクト識別方法 | |
JP2011109428A (ja) | 情報処理装置、情報処理方法、およびプログラム | |
JP4490214B2 (ja) | 電子アルバム表示システム、電子アルバム表示方法、及び電子アルバム表示プログラム | |
JP2006079460A (ja) | 電子アルバム表示システム、電子アルバム表示方法、電子アルバム表示プログラム、画像分類装置、画像分類方法、及び画像分類プログラム | |
JP2011049866A (ja) | 画像表示装置 | |
WO2009096524A1 (ja) | 画像向き判定方法と装置及びプログラム | |
KR20100026724A (ko) | 휴대단말의 촬영 데이터 관리 방법 및 장치 | |
JP2003208615A (ja) | 画像処理装置、及びプログラム | |
JP2010041503A (ja) | デジタルカメラ | |
JP2012043337A (ja) | 画像処理装置、撮像システム、画像処理方法、およびプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180006946.3 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11734491 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13520398 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011550843 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11734491 Country of ref document: EP Kind code of ref document: A1 |