JP4854957B2 - Representative image selection apparatus and method, and program - Google Patents

Description

  The present invention relates to a representative image selection device and method for selecting a representative image representing a plurality of images from a plurality of images, and a program for causing a computer to execute the representative image selection method.

  With the widespread use of digital cameras and scanners, anyone can easily acquire digital images. In addition, images are also organized on a personal computer. In this case, images are often organized by classifying images for each event that has occurred to the user. For example, when you go on a trip to Europe, images acquired in France, the United Kingdom, Spain, etc. are classified into categories for each country, and for images classified in the French category, going out in Paris, Nice, etc. Classify images into more detailed categories for each location. For images classified in Paris, the images are classified into finer categories such as the Eiffel Tower, Arc de Triomphe and Notre Dame. Furthermore, regarding Notre Dame, images may be classified into more detailed categories such as in front of the temple, in a remote place, and in the temple.

  If the images are hierarchized and classified into a plurality of categories in this manner, the images divided for each event can be referred to when referring to the images later, so that the images can be easily organized.

  On the other hand, the operation of classifying images into hierarchies is very troublesome for the user because it is necessary to perform the operation while looking at the image and following the user's memory.

  For this reason, various methods for automatically classifying images have been proposed. For example, the images are arranged on the time axis based on the shooting date / time information indicating the shooting date / time given to the images, and the images are classified into a plurality of categories according to the difference in shooting time difference between two images having adjacent shooting times. A classification method has been proposed (see Patent Document 1). Also, a method for classifying images into categories according to the user's schedule with reference to schedule information representing the user's schedule (see Patent Document 2), and calculating feature values from a plurality of images, A method of classifying images into a plurality of categories based on the relationship between them has also been proposed (see Patent Document 3).

  By using such a method, the user does not need to classify the image by himself / herself, so that the burden on the user when arranging the images can be reduced.

  Further, when organizing images, if representative images representing the image group classified into each category are displayed, it is possible to know at a glance what images are classified into each category. For this reason, there are a method for displaying a representative image of each category at the time of image search (see Patent Document 4) and a method for displaying a representative image in a folder when displaying a folder corresponding to each category (see Patent Document 5). Proposed.

In the methods described in Patent Documents 4 and 5, the representative image is selected by the user, but a method of automatically selecting the representative image has also been proposed. For example, a method of selecting an image having the maximum number of pixels among a plurality of images as a representative image (see Patent Document 6), and a method of recording an image reproduction time and selecting an image having a long reproduction time as a representative image (See Patent Document 7).
JP 2000-112997 A JP 2003-109773 A JP 2001-256244 A Japanese Patent Laid-Open No. 5-128166 JP 2003-296165 A JP 2000-244849 A Japanese Patent Laid-Open No. 2003-339013

  However, when taking a picture using a digital camera, the number of pixels of an image acquired by the digital camera is often constant because the number of pixels is not frequently changed in general. Under such circumstances, the representative image cannot be selected by the method described in Patent Document 6.

  Further, in the method of selecting an image having a long reproduction time as a representative image as in Patent Document 7, an image in which a user often takes an image from a digital camera and reproduces the image on the personal computer is a representative image. Selected as. However, the representative image cannot be selected immediately after the image is captured from the digital camera to the personal computer (that is, the image is not reproduced at all).

  The present invention has been made in view of the above circumstances, and in particular, an object of the present invention is to make it possible to reliably select a representative image important for a user from a plurality of images classified into categories as described above.

A first representative image selection device according to the present invention is a representative image selection device that selects a predetermined number of representative images from a plurality of images.
Candidate selection means for selecting a plurality of candidate images of the representative image from the category having the higher importance among the plurality of categories each having the importance set, obtained by classifying the plurality of images into a hierarchy. When,
Selection means for selecting the predetermined number of representative images from the selected plurality of candidate images.

  The “category with high importance” includes not only the category with the highest importance but also the top few categories when the categories are sorted in descending order of importance.

  In the first representative image selection device according to the present invention, the selection unit has processed each of the selected candidate images by performing image processing for improving the image quality of the candidate images. A representative value of a change amount of a pixel value between corresponding pixels of the image and the pre-processing image before the image processing is calculated, and the predetermined number of images from the smallest representative value in the ascending order of the representative value. It is good also as a means to select as a representative image.

  The “image processing” may be any processing that can improve the image quality, and specifically includes gradation correction processing, color correction processing, brightness correction processing, white balance correction processing, color processing, and so on. At least one of various processes such as a degree correction process, a sharpness process, a noise removal process, a red-eye correction process, and a beautiful skin process can be used.

  As the “representative value”, any value can be used as long as it is a value representative of the change amount of the pixel value in a part or the whole of the image. Specifically, the average value and the median value of the change amount of the pixel value Etc. can be used. As the “pixel value”, for example, when an image is composed of three colors of RGB, a luminance value calculated from each color and a pixel value of three colors themselves can be used. In this case, the amount of change in the pixel value includes the absolute value of the difference in luminance value between the image and the processed image, the distance in the RGB three-dimensional color space of the pixel value between the image and the processed image (that is, the square root of the square of the difference). ) Etc. can be used.

A second representative image selection device according to the present invention is a representative image selection device that selects a predetermined number of representative images from a plurality of images.
For each of the plurality of images, pixel values between corresponding pixels of a processed image obtained by performing image processing for improving the image quality of the image and an unprocessed image before the image processing is performed Change amount calculating means for calculating a representative value of the change amount of
Selecting means for selecting the predetermined number of images as the representative image from the smallest representative value in ascending order of the representative value.

  In the first and second representative image selection devices according to the present invention, the representative value may be a value obtained by normalizing an average value of the change amount of the pixel value.

  Further, the representative value may be a value obtained by correcting a value obtained by normalizing an average value of the change amount of the pixel value according to at least one information of the reproduction time of the image and the date and time of the last reproduction. Good.

  “Correction according to playback time” means that the longer the playback time is, the smaller the representative value is by subtracting the value corresponding to the playback time from the normalized value of the pixel value variation. means. “Correct according to the date and time of the last playback” means that the longer the elapsed time from the date and time of the last playback, the larger the value is added to the normalized value of the pixel value change This means that the representative value is increased.

A first representative image selection method according to the present invention is a representative image selection method for selecting a predetermined number of representative images from a plurality of images.
Of the plurality of categories obtained by classifying the plurality of images into hierarchies, each of which has a set importance level, a plurality of candidate images of the representative image are selected from the category with the high importance level,
The predetermined number of representative images are selected from the plurality of selected candidate images.

A second representative image selection method according to the present invention is a representative image selection method for selecting a predetermined number of representative images from a plurality of images.
For each of the plurality of images, pixel values between corresponding pixels of a processed image obtained by performing image processing for improving the image quality of the image and an unprocessed image before the image processing is performed Calculate the representative value of the amount of change in
The predetermined number of images are selected as the representative image from the smallest representative value in ascending order of the representative value.

  Note that the first and second representative image selection methods according to the present invention may be provided as programs for causing a computer to execute the methods.

  According to the present invention, a plurality of representative image candidate images are selected from a category having a high importance level among a plurality of categories having a set importance level, and a predetermined number of representative images are selected from the selected plurality of candidate images. The Here, since the candidate image is selected from a category having high importance, it is often important for the user. Therefore, it is possible to reliably select a representative image important for the user.

  In addition, since multiple candidate images are selected from categories with a high degree of importance among multiple categories, and representative images are selected from multiple candidate images, it is compared with the case where representative images are selected from the entire plurality of images. Thus, the processing time for selecting the representative image can be shortened.

  Here, the representative value of the change amount of the pixel value between corresponding pixels of the processed image obtained by performing image processing for improving the image quality and the pre-processed image before the image processing is small. This means that the change between the processed image and the pre-processed image is small, that is, a well-captured high-quality image with little need for image processing.

  Therefore, for each of the plurality of selected candidate images, the representative value of the change amount of the pixel value is calculated, and a predetermined number of images are selected as the representative image from the smallest representative value in ascending order of the representative value. An image having an image quality can be selected as a representative image.

  In addition, since the maximum value and the minimum value of the representative value are clarified by setting the representative value to a value obtained by normalizing the average value of the pixel value change amounts, the representative value can be easily understood.

  In this case, the representative image can be selected to reflect the user's preference by correcting the normalized value according to at least one information of the reproduction time of the image and the date and time of the last reproduction.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a representative image selection device according to the first embodiment of the present invention. As shown in FIG. 1, the representative image selection device 1 according to the first embodiment performs various controls such as image data recording control and display control, and also controls a CPU 12 and a CPU 12 that control each part of the device 1. A system memory 14 composed of a program to be operated, viewer software for viewing images, a ROM in which various constants are recorded, and a RAM serving as a work area when the CPU 12 executes processing, and various instructions. And an input unit 16 including a keyboard and a mouse for performing the above and a display unit 18 including a liquid crystal monitor for performing various displays.

  Further, the representative image selection device 1 reads the image data from the memory card 2 on which the image data is recorded and records the image data on the memory card 2, and the image data to JPEG or motion JPEG. A compression / decompression unit 22 that compresses or decompresses compressed image data, a hard disk 24 that records image data and various programs executed by the CPU 12, a system memory 14, a card slot 20, and a hard disk 24. And a memory control unit 26 for controlling.

  The representative image selection device 1 also includes an image classification unit 28 that classifies images stored in a folder desired by the user in the hard disk 24 into a plurality of categories, and an importance setting unit that sets the importance of each category. 30. Further, the representative image selection device 1 selects a representative image representing a plurality of images stored in a folder desired by the user from a display control unit 32 that controls the display of the display unit 18 and a category having a high degree of importance. A representative image selection unit 34.

  The image classification unit 28 classifies a plurality of images stored in the hard disk 24 into a plurality of categories by using the method described in Patent Document 1, for example. Hereinafter, image classification will be described. Here, a case will be described in which a plurality of images taken when a user goes on a trip to France is classified. FIG. 2 is a diagram showing a time table for one day of a user's trip to France. As shown in FIG. 2, it is assumed that the user visited Paris on a French trip and then moved to Nice by bus. In Paris, the Arc de Triomphe, the Eiffel Tower and Notre Dame are visited. Assume that the next day you visited the Burgundy region.

Here, if the user goes on a trip to France and then goes on a different trip, there will be a difference of one day or more between the French trip and the other trip. In addition, if you visit Paris and Nice on one day and visit the Burgundy region on the following day, you will have a difference of more than an hour between your visit to Paris and Nice and your visit to Burgundy. In the case of traveling, there will be no difference for 1 hour between visiting Paris, in the bus, and visiting Nice, but it will be more than 30 minutes. The Arc de Triomphe, Eiffel Tower and Notre Dame are visited in Paris, so the visit time will not be different for 30 minutes, but will be more than 10 minutes. Furthermore, in the Notre Dame temple, the user moves within the temple site in front of the temple, away from the temple, and inside the temple, but since the movement is within the same temple site, the visit time is 10%. The difference does not open for minutes, but the difference opens for more than 3 minutes.

  Therefore, when shooting is performed collectively at the place of visit, when the shooting time interval is less than 3 minutes and the number of images is plotted on the time axis between adjacent images, the shooting time interval is regarded as the same shooting time. The relationship is as shown in FIG. That is, as shown in FIG. 3, in the relationship between time and the number of images, it can be seen that a plurality of images (that is, image groups) are acquired at a certain time for each visited site.

  The image classification unit 28 uses this relationship to classify images into a plurality of categories. That is, referring to the relationship shown in FIG. 3, first, an image group having a shooting time interval of one day or more is classified as one category. In this case, all images acquired in the France trip are classified as one category. Next, for the classified images, an image group having an imaging time interval of 1 hour or more is classified as one category. In this case, the image group acquired in the Paris and Nice visits and the image group acquired in the Burgundy region visit are further classified as one category.

  Further, with respect to the classified images, an image group whose shooting time interval is 30 minutes or more and less than 1 hour is classified as one category. In this case, in the Paris and Nice visit categories, the image groups acquired in the Paris visit, in the bus, and in the Nice visit are further classified as one category. Furthermore, an image group whose shooting time interval is 10 minutes or more and less than 30 minutes is classified as one category. In this case, in the Paris visit category, the image groups acquired in the Arc de Triomphe visit, the Eiffel Tower visit, and the Notre Dame visit are further classified as one category. Further, an image group whose shooting time interval is 3 minutes or more and less than 10 minutes is classified as one category. In this case, in the category of visiting Notre Dame, the image groups acquired in front of the temple, in a place away from the temple, and in the temple are classified as one category.

  Then, one folder is created for each category, and the classified images are stored in folders corresponding to the respective categories, thereby completing the image classification.

  FIG. 4 is a diagram showing a folder structure after classification. Here, the folder structure when the images of the first day in France travel are classified will be described. As shown in FIG. 4, a folder F1 (French travel folder) corresponding to a French travel category for storing images acquired in a French travel is in the top layer, and is classified into a Paris category in the lower layer of the folder F1. A folder F1-1 corresponding to the Paris category and a folder F1-corresponding to the category moving on the bus for storing the images, images classified in the category moving on the bus, and images classified in the Nice category, respectively. Folder F1-3 corresponding to category 2 and Nice is created.

  Further, a folder F1 corresponding to the Arc de Triomphe category that stores images classified into the Arc de Triomphe category, images classified into the Eiffel Tower category, and images classified into the Notre Dame category under the Paris folder. 1-1, folder F1-1-2 corresponding to the category of Eiffel Tower and folder F1-1-3 corresponding to the category of Notre Dame are created.

  In addition, in the lower level of the Notre Dame folder, the temples store images classified in the previous category of the Notre Dame temple, images classified in the remote category, and images classified in the category of the temple, respectively. Folder F1-1-3-1 corresponding to the previous category, folder F1-1-3-2 corresponding to the category at a distant place, and folder F1-1-3-3 corresponding to the category in the temple Created.

  Although a name is given to the folder for explanation here, the name is not actually given to the folder, and a number corresponding to the hierarchy is simply given as shown in FIG. Only. For this reason, the user does not know which image is classified in which folder until the folder is opened.

  The importance level setting unit 30 sets the importance level of each category using the classification result obtained by the image classification unit 28.

  Here, when a plurality of images are classified into a plurality of categories, many people recognize that the category is more important as the number of images classified into each category is larger. In addition, in a category classified into hierarchies, there are many people who recognize that the category is more important as the number of images included in the category of the upper hierarchy of each category is larger. In addition, in a category classified into hierarchies, there are many people who recognize that the category is more important as the number of categories included in the same hierarchy and having a common upper hierarchy is larger. Moreover, in each category classified into hierarchies, many people recognize that the category is more important as the hierarchy is deeper (that is, the number of lower layers is larger). Furthermore, in a certain category classified into hierarchies, many people recognize that the category is more important as the number of hierarchies from the highest hierarchy to each category increases.

  In view of this point, the importance setting unit 30 regards the number of images included in each category, the number of images included in the higher-level category of each category, and the category related to each category for each category classified by the image classification unit 28. The importance of each category is set based on at least one of the following information: the number of subordinates, the number of layers below each category, and the number of layers from the highest layer to each category. Hereinafter, the setting of importance will be described.

  Here, as shown in FIG. 4, 100 images were acquired in a trip to France, of which 60 images were classified into the Paris category, 10 images classified into the category in the bus, Assume that there are 30 images classified in the Nice category. In addition, 20 images classified into the Arc de Triomphe category, 10 images classified into the Eiffel Tower category, and 30 images classified into the Notre Dame category in the lower level of the Paris category. To do. Furthermore, in the lower layer of the Notre Dame temple, 15 images classified in the previous category of the temple, 5 images classified in the category of the distant place, 10 images classified in the category in the temple Suppose that

  The importance setting unit 30 uses the following formula (1) to weight and add the number of images included in each category, the number of categories related to each category, and the number of layers below each category. Is calculated.

Importance = a1 × (number of images included in each category)
+ B1 × (the number of categories related to each category)
+ C1 × (the number of lower layers in each category) (1)
However, a1, b1, c1: weighting factors (in this embodiment, a1 = b1 = c1 = 1)
Here, the related category means a category that is included in the same hierarchy and is shared by one higher hierarchy among categories classified into hierarchies. Accordingly, in the present embodiment, the category of moving and nice on Paris and buses is a related category. The Arc de Triomphe, the Eiffel Tower, and Notre Dame are also related categories, and the front of the temple, the distant place and the inside of the temple are also related categories.

  For example, for the category of travel to France, the number of images included in each category is 100, the number of categories related to each category is 1, and the number of hierarchies below the category is 4, so the importance is 105. The importance calculated in this way is shown in FIG. Note that this importance calculation method is referred to as Method 1.

  Here, in Method 1, the weighting coefficient is set to a1 = b1 = c1 = 1, but importance is attached to any of the number of images included in each category, the number of categories related to each category, and the number of layers below each category. Of course, the weighting coefficient may be changed according to the above.

  The following formula (2) is used to weight and multiply the number of images included in each category, the number of categories related to each category, and the number of hierarchies from the highest hierarchy to each category to determine the importance of each category. It may be calculated.

Importance = a2 × (number of images included in each category)
× (Number of categories related to each category)
× (Number of layers from the upper level to each category) (2)
However, a2: Weight coefficient (a2 = 1 in this embodiment)
For example, for the category of travel to France, the number of images included in each category is 100, the number of categories related to each category is 1, and the number of layers from the highest layer to the category is 1, so the importance is 100. Become. The importance calculated in this way is shown in FIG. Note that this importance calculation method is referred to as method 2.

  Here, in method 2, the weighting factor is a2 = 1, but it is needless to say that the weighting factor may be changed.

  Also, the importance of each category may be calculated by weighted addition of the number of images included in each category and the number of images included in the category of the upper hierarchy of each category. For example, the number of images classified into the category in front of the temple is 15, the number of images classified into the category of Notre Dame, which is one higher, 30 and the number of images classified into the category of Paris, which is one higher than this. 60, and the number of images classified into the higher-level French travel category is 100. Therefore, the importance of the previous category of the temple is calculated as 15 + 30 + 60 + 100 = 205 when all the weighting factors are 1. In addition, the number of images classified into the Arc de Triomphe category is 20, the number of images classified into the Paris category that is one level higher than this, and the number of images classified into the category of higher-level France travel is 100. Therefore, the importance of the Arc de Triomphe category is 20 + 60 + 100 = 180 when all weighting factors are 1. The importance calculated in this way is shown in FIG. Note that this importance calculation method is Method 3.

  Here, in Method 3, all the weighting factors are set to 1, but it goes without saying that the weighting factors may be changed according to which hierarchy is emphasized.

  Furthermore, the importance may be calculated only for the category of the lowest hierarchy, and the calculated importance may be set as the importance of the category of the next higher hierarchy. Any of the above methods 1 to 3 may be used to calculate the importance for the category of the lowest hierarchy. Here, FIG. 8 shows the importance of each category when the importance of the category of the lowest hierarchy is calculated by the method 3. Note that this importance calculation method is referred to as Method 4.

  When the importance is calculated by the above methods 1 to 4, although the importance value is different, in the same hierarchy, when ranking the categories on the basis of the calculated importance, the importance level is determined by any method. The ranking is the same. For example, in the hierarchy one level below the category of Paris, the order of importance is the order of Notre Dame, Arc de Triomphe and Eiffel Tower. Also, in the hierarchy one level below Notre Dame, the order of importance is from the highest to the front of the temple, in the temple, and away from the temple.

  The representative image selection unit 34 selects a predetermined number (N) of representative images from the category having the highest importance among the categories set with the importance.

  FIG. 9 is a schematic block diagram showing the configuration of the representative image selection unit 34. As shown in FIG. 9, the representative image selection unit 34 includes a candidate image selection unit 50, an image processing unit 52, a change amount calculation unit 54, and a determination unit 56.

  The candidate image selection unit 50 selects a plurality of candidate images that are candidates for the representative image from the category having the high importance value calculated in the lowest hierarchy by the following processing. FIG. 10 is a flowchart showing candidate image selection processing performed by the candidate image selection unit 50. Here, at least M (M ≧ N) candidate images are selected.

  First, the number m of candidate images is set to 0 (step ST1), and the category having the highest importance among the unselected categories of candidate images in the category of the lowest hierarchy of the categories into which a plurality of images are classified. All images selected in the category are selected as candidate images (step ST2). Here, when m = 0, candidate images are selected from the category having the highest importance in the lowest hierarchy. For example, when the importance is calculated as described above, a candidate image is selected from a category in front of a temple in a lower hierarchy of the Notre Dame category.

  Subsequently, the number m of candidate images is set to the number m1 of candidate images selected in step ST2 (step ST3), and it is determined whether or not the number m of candidate images is M or more (step ST4). If step ST4 is positive, the candidate image selected in step ST2 is determined as the final candidate image (step ST5), and the process is terminated.

  If step ST4 is negative, the process returns to step ST2, and the processes after step ST2 are repeated.

  In the above processing, when the number of images classified into the most important category in the lowest hierarchy is less than M, the image classified into the next most important category is added to the candidate image, The processes after step ST2 are repeated until the number of candidate images becomes M or more.

  For example, in the case of calculating the importance as described above, when the number M of candidate images is 10, 15 images classified into the previous category of the temples in the lower hierarchy of the Notre Dame temple category are displayed. Select as a candidate image. In addition, when the number M of candidate images is 20, a total of 25 images including 15 images classified into the previous category of the temple and 10 images classified into the category within the temple. Select as a candidate image.

  The image processing unit 52 performs tone correction processing, color correction processing, brightness correction processing, white balance correction processing, saturation correction processing, sharpness processing, noise removal processing, red-eye correction processing, and skin softening according to a predetermined algorithm. Processed image data is obtained by performing image processing including at least one of various processes such as processes on all image data representing the selected candidate image.

  The change amount calculation unit 54 displays all representative values of pixel value change amounts between corresponding pixels of the pre-process image represented by the pre-process image data and the processed image represented by the processed image data. The image data is calculated. Note that the change amount of the pixel value is the change amount of the pixel value itself (for example, when the pixel value is composed of RGB colors, the difference ΔR, ΔG, ΔB between the RGB values in the corresponding pixels of the pre-processed image and the processed image) Or the absolute value of the difference between luminance values obtained from each color of RGB. Here, the average value of the change amount of the pixel value is used as the representative value. However, the present invention is not limited to this, and any value that represents the change amount of the pixel value such as the median value of the change amount of the pixel value may be used. Any value can be used.

  Note that the average value of the change amount of the pixel value may be normalized by the following equation (3), and the normalized value may be used as the representative value.

Representative value = (1−average value of change / maximum value of change) × 100 (3)
However, the maximum value of the change amount is the largest value among the change amounts calculated for all candidate images.

  The determination unit 56 sorts the images in ascending order of the representative value, and selects, from the minimum representative value to the Nth image, that is, the top N images with the small representative value as representative images.

  FIG. 11 is a flowchart showing representative image selection processing performed by the image processing unit 52, the change amount calculation unit 54, and the determination unit 56. First, the image data to be processed by the CPU 12 is set to j = 1 (j = 1 to M) so that the file name of the image data representing the candidate image is the first image data (step ST11). Next, the image processing unit 52 performs image processing on the image data to obtain processed image data (step ST12).

  Then, the change amount calculation unit 54 calculates the change amount of the pixel value between the unprocessed image and the processed image (step ST13). Further, the change amount calculation unit 54 calculates a representative value of the change amount of the pixel value (step ST14).

  Then, the CPU 12 determines whether or not the processing has been completed for the image data representing all candidate images (step ST15), and if step ST15 is negative, the file name is set to j = the next image data to be processed. j + 1 is set (step ST16), and the process returns to step ST12. If step ST15 is affirmed, images from the minimum representative value to the Nth image, that is, the top N images with small representative values are selected as representative images (step ST17), and the process ends.

  With the above processing, for example, when only images classified in the category in front of the temple are selected as candidate images, N representative images are selected from the images classified in the category in front of the temple. Further, when images classified into the category in front of the temple and the category in the temple are selected as candidate images, N representative images are selected from all the images in the category in front of the temple and the category in the temple. Selected. At this time, since the representative image is selected from at least one of the category in front of the temple and the category in the temple, there may be a category in which the representative image is not selected even if the candidate image is selected.

  When the CPU 12 starts up viewer software for browsing images stored in the hard disk 24, the display control unit 32 displays the importance setting unit 30 on the image list screen displayed on the display unit 18 by the viewer software. The set importance level is displayed on the display unit 18 so as to be visible, and the representative image selected by the representative image selection unit 34 is displayed.

  FIG. 12 shows an example of an image list screen. As shown in FIG. 12, the image list screen 40 is stored in the folder field 40A indicating the folder structure stored in the hard disk 24 and the folder selected in the folder field 40A (the folder surrounded by the rectangle A1 in FIG. 12). A list display field 40B for displaying a list of thumbnail images for the existing images is included.

  In the folder field 40A, the hierarchical structure of folders corresponding to the hierarchical categories as described above is displayed. The display control unit 32 sets ranks 1 to 4 in categories according to the importance of each category set by the importance setting unit 30. Then, the folders are color-coded according to the order, and the hierarchical structure of the folders is displayed in the folder field 40A. For example, the first folder is pink, the second folder is orange, the third folder is yellow, and the fourth folder is green. Here, in the folder field 40A of the image list screen 40 shown in FIG. 12, the hierarchy displayed at the top represents a folder in which the above-mentioned France travel images are classified.

  For example, when the importance is calculated by the method 4, the importance is set to 1st for the importance exceeding 200, the 2nd for the importance exceeding 150 and 2nd, and the order of importance exceeding 100 to 150 or less. The importance level may be 3rd, and the importance level 100 or less may be 4th.

  Here, not only the method of color-coding according to the importance, but also the importance ranking may be displayed as a numerical value as shown in FIG. Also, a mark or graph corresponding to the importance may be displayed.

  On the other hand, in the list display field 40B, when the user selects a folder at the lowest level, a list of thumbnail images for images stored in the selected folder is displayed. When a folder at a level is selected, a folder that is one level below the selected folder is reduced and displayed.

  FIG. 14 is a diagram showing an example of the image list screen 40 in which folders are reduced and displayed in the list display field 40B. In this example, it is assumed that the user has selected the Notre Dame folder (folder F1-1-3) in the French travel folder (ie, folder F1), and the representative image is the previous folder (folder F1-1-3-). It is assumed that three items from 1) and one item from the folder in the temple (folder F1-1-3-3) are selected.

  As shown in FIG. 14, the list display field 40B includes three folders F1-1-1-1, F1-1-3-2, and F1-1-3 which are one level lower than the selected folder F1-1-3. -3 is reduced. Three representative images are displayed as thumbnails in the folder F1-1-3-1 and one representative image is displayed in the F1-1-3-3. Since no representative image is selected from the folder F1-1-3-2, thumbnail display of the representative image is not performed in the reduced folder F1-1-3-2.

  Next, processing performed in the first embodiment will be described. FIG. 15 is a flowchart showing processing performed in the first embodiment. It is assumed that the image data is read from the memory card 2 and stored in a folder desired by the user of the hard disk 24. The viewer software is activated, and the processing is started when the user gives an instruction for classification from the input unit 16, and the image classification unit 28 stratifies the image data and classifies it into a plurality of categories as described above (step ST21).

  Then, the importance level setting unit 30 sets the importance level for the category (step ST22), and the candidate image selection unit 50 of the representative image selection unit 34 selects a candidate image of the representative image (step ST23). Subsequently, the image processing unit 52 performs image processing on the image data representing all selected candidate images to obtain processed image data (step ST24), and the change amount calculating unit 54 performs the pre-processing image and the processed image. The representative value of the change amount is calculated (step ST25). Then, the determination unit 56 determines a representative image (step ST26).

  Furthermore, the display control unit 32 makes the importance visible and displays the image list screen on the display unit 18 (step ST27), and the process is terminated.

  As described above, according to the first embodiment, a candidate image of a representative image is selected from a category having a high importance among a plurality of categories set with importance, and a representative image is selected from the plurality of selected candidate images. It is what you choose. Here, since the candidate image is selected from a category having high importance, it is often important for the user. Therefore, according to the present embodiment, it is possible to reliably select a representative image important for the user.

  In addition, since a candidate image is selected from a category having a high importance among a plurality of categories and a representative image is selected from the candidate images, the representative image is selected as compared with the case where the representative image is selected from the entire plurality of images. Processing time for selecting an image can be shortened.

  Here, the representative value of the change amount of the pixel value between corresponding pixels of the processed image obtained by performing image processing for improving the image quality and the pre-processed image before the image processing is small. This means that the change between the processed image and the pre-processed image is small, that is, a well-captured high-quality image with little need for image processing.

  For this reason, for each of the plurality of selected candidate images, a representative value of the change amount of the pixel value is calculated, and a predetermined number of images having a small representative value are selected as the representative images, thereby representing a high-quality image. It can be selected as an image.

  In addition, since the maximum value and the minimum value of the representative value are clarified by setting the representative value to a value obtained by normalizing the average value of the pixel value change amounts, the representative value can be easily understood.

  In the first embodiment, the top N images having a small representative value of the change amount of the pixel value are selected as the representative images. However, the file names when the selected candidate images are arranged in the order of the file names are selected. N images including the first image or the last image may be selected as representative images. Further, N images randomly selected from the candidate images may be used as representative images.

  In the first embodiment, images are hierarchized and classified into a plurality of categories based on the shooting date and time intervals. For example, a camera capable of acquiring GPS information indicating the latitude and longitude of a shooting location is used. When shooting is performed using this, GPS information is added to the image. In such a case, images may be hierarchized and classified for each shooting location using GPS information.

  In addition, a method of classifying images into categories according to the user's schedule with reference to schedule information representing the user's schedule as described in Patent Document 2, and a plurality of methods as described in Patent Document 3 The feature amount may be calculated from the image, and the image may be classified using a method of classifying the image into a plurality of categories based on the relationship between the feature amounts of the image.

  Of course, the user may manually classify the images according to his / her preference.

  In the first embodiment, the importance setting unit 30 sets the importance of each category using the methods 1 to 4, but the user manually sets the importance according to his / her preference. Of course, it may be set.

  Next, a second embodiment of the present invention will be described. FIG. 16 is a schematic block diagram showing the configuration of a representative image selection device according to the second embodiment of the present invention. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted here. The representative image selection device 1 ′ in the second embodiment does not include the image classification unit 28 and the importance setting unit 30 in the first embodiment, and a plurality of images stored in a folder desired by the user in the hard disk 24. This embodiment is different from the first embodiment in that a representative image selection unit 34 'for selecting a representative image from the plurality of images without classifying the images is classified into categories.

  FIG. 17 is a schematic block diagram showing the configuration of the representative image selection unit 34 ′. As shown in FIG. 17, the representative image selection unit 34 'corresponds to the image processing unit 52, the change amount calculation unit 54, and the determination unit 56 of the representative image selection unit 34 in the first embodiment, respectively. The change amount calculation unit 54 'and the determination unit 56' are provided.

  Next, processing performed in the second embodiment will be described. FIG. 18 is a flowchart showing processing performed by the representative image selection unit 34 ′ in the second embodiment. First, k = 1 (k is the number of image data stored in a folder desired by the user) is set so that the image data to be processed by the CPU 12 is the first image data having a file name (step ST31). ). Next, the image processing unit 28 'performs image processing on the image data to obtain processed image data (step ST32).

  Then, the change amount calculation unit 54 ′ calculates the change amount of the pixel value between the unprocessed image and the processed image (step ST33). Further, the change amount calculation unit 54 ′ calculates a representative value of the change amount of the pixel value (step ST34).

  Then, the CPU 12 determines whether or not the processing has been completed for all the images stored in the folder desired by the user (step ST35). If step ST35 is negative, the next image data whose file name is the processing target is determined. As much as possible, k = k + 1 is set (step ST36), and the process returns to step ST32. When step ST35 is affirmed, the images from the minimum representative value to the Nth image, that is, the top N images with small representative values are selected as representative images (step ST37), and the process is terminated.

  Thus, according to the second embodiment, a representative image of an image stored in a desired folder can be selected even when the image is not classified into a plurality of categories.

  In the first and second embodiments, the average value of the pixel value variation or the normalized value is used as the representative value, but the image stored in the folder desired by the user is stored several times. In the case of reproduction, the image reproduction time is recorded using the method described in Patent Document 7, and the normalized pixel value is reduced so that the representative value becomes smaller as the reproduction time becomes longer. You may correct the average value of variation. As a result, an image having a long reproduction time is more easily selected as a representative image.

  Also, the date and time when the image was last played back is recorded, and the normalized pixel value is set so that the representative value becomes larger as the elapsed time from this date and time to the date and time when the representative image selection process is performed is longer. You may correct the average value of variation. This makes it easier to select a recently reproduced image as a representative image. On the other hand, an image that has not been reproduced for a long time is less likely to be selected as a representative image.

  Thereby, a representative image can be selected reflecting a user's preference.

1 is a schematic block diagram showing the configuration of a representative image selection device according to a first embodiment of the present invention. The figure which shows the time table of one day with the user's trip to France The figure which shows the relation between the time axis and the number of images Figure showing the folder structure after classification The figure which shows the folder structure after classification with the importance which is calculated with method 1 The figure which shows the folder structure after classification with the importance which is calculated with method 2 The figure which shows the folder structure after classification with the importance which is calculated with method 3 The figure which shows the folder structure after classification with the importance which is calculated with method 4 Schematic block diagram showing a configuration of a representative image selection unit in the first embodiment Flowchart showing candidate image selection processing Flowchart showing representative image selection processing in the first embodiment The figure which shows the image list screen in 1st Embodiment (the 1) The figure which shows the image list screen in 1st Embodiment (the 2) The figure which shows the image list screen in 1st Embodiment (the 3) The flowchart which shows the process performed in 1st Embodiment. The schematic block diagram which shows the structure of the representative image selection apparatus by the 2nd Embodiment of this invention. The schematic block diagram which shows the structure of the representative image selection part in 2nd Embodiment. Flowchart showing representative image selection processing in the first embodiment

Explanation of symbols

1, 1 'representative image selection device 2 memory card 12 CPU
DESCRIPTION OF SYMBOLS 14 System memory 16 Input part 18 Display part 20 Card slot 22 Compression / decompression part 24 Hard disk 26 Memory control part 28 Image classification part 30 Importance setting part 32 Display control part 34,34 'Representative image selection part 40 Image list screen 50 Candidate image Selection unit 52, 52 ′ Image processing unit 54, 54 ′ Change amount calculation unit 56, 56 ′ determination unit

Claims (6)

In a representative image selection device that selects a predetermined number of representative images from a plurality of images,
Storage means for storing the plurality of images that are hierarchized in advance and classified into a plurality of categories, and importance is set for each of the plurality of categories;
Candidate selection means for selecting a plurality of candidate images of the representative image from the category having a high degree of importance among the plurality of categories for the plurality of images stored in the storage means;
A plurality of processed images are obtained by performing image processing to improve image quality for each of the selected plurality of candidate images, and for each of the candidate images, between corresponding pixels with the processed image Normalizing the average value of the pixel value change amount , and correcting the normalized value so that it decreases as the reproduction time of the candidate image increases, and the candidate image reproduces the normalized value last. The representative value of the change amount of the pixel value is calculated by performing at least one of the processes of correcting so as to increase as the elapsed time from the specified date and time increases , and from the smallest representative value in ascending order of the representative value A representative image selecting apparatus comprising: a selecting unit that selects the predetermined number of the candidate images as the representative image. In a representative image selection device that selects a predetermined number of representative images from a plurality of images,
Image processing means for obtaining a plurality of processed images by performing image processing for improving image quality for each of the plurality of images;
For each of the plurality of images, the average value of the amount of change in pixel value between pixels corresponding to the processed image is normalized so that the normalized value becomes smaller as the reproduction time of the image becomes longer. The amount of change in the pixel value by performing at least one of the process of correcting and the process of correcting the normalized value so that the elapsed time from the date and time when the image was last reproduced becomes longer a change amount calculating means for calculating a representative value,
A representative image selecting apparatus comprising: a selecting unit that selects the predetermined number of images as the representative image from the smallest representative value in order of decreasing the representative value. In a representative image selection method for selecting a predetermined number of representative images from a plurality of images,
Candidate selection means is hierarchized in advance and classified into a plurality of categories, and the plurality of images stored in the storage means for storing the plurality of images in which importance is set for each of the plurality of categories. Among the plurality of categories, select a plurality of candidate images of the representative image from the category with high importance,
The selection unit obtains a plurality of processed images by performing image processing for improving the image quality on each of the selected plurality of candidate images, and each of the candidate images is correlated with the processed image. A process of normalizing an average value of changes in pixel values between corresponding pixels and correcting the normalized value so that the reproduction time of the candidate image decreases, and the normalized value is converted into the candidate image. By performing at least one of the processes of correcting so that the elapsed time from the date and time when the image was last reproduced becomes longer, the representative value of the change amount of the pixel value is calculated, and the representative value is in ascending order. A representative image selection method, wherein the predetermined number of the candidate images are selected as the representative image from a minimum representative value. In a representative image selection method for selecting a predetermined number of representative images from a plurality of images,
The image processing means obtains a plurality of processed images by performing image processing for improving the image quality for each of the plurality of images.
A change amount calculation unit normalizes an average value of pixel value change amounts between pixels corresponding to the processed image for each of the plurality of images, and uses the normalized value as a reproduction time of the image. By performing at least one of a process of correcting to be smaller as the length is longer, and a process of correcting the normalized value to be larger as the elapsed time from the date and time when the image was last reproduced is longer, Calculating a representative value of the change amount of the pixel value ;
A representative image selecting method, wherein the selecting means selects the predetermined number of the images from the smallest representative value in ascending order of the representative value as the representative image. In a program for causing a computer to execute a representative image selection method for selecting a predetermined number of representative images from a plurality of images,
Among the plurality of categories for the plurality of images stored in a storage unit that stores a plurality of images that are previously hierarchized and classified into a plurality of categories, and importance levels are set for the respective categories. , A procedure for selecting a plurality of candidate images of the representative image from the category with high importance,
A plurality of processed images are obtained by performing image processing to improve image quality for each of the selected plurality of candidate images, and for each of the candidate images, between corresponding pixels with the processed image Normalizing the average value of the pixel value change amount , and correcting the normalized value so that it decreases as the reproduction time of the candidate image increases, and the candidate image reproduces the normalized value last. The representative value of the change amount of the pixel value is calculated by performing at least one of the processes of correcting so as to increase as the elapsed time from the specified date and time increases , and from the smallest representative value in ascending order of the representative value And a procedure for selecting the predetermined number of the candidate images as the representative image. In a program for causing a computer to execute a representative image selection method for selecting a predetermined number of representative images from a plurality of images,
Obtaining a plurality of processed images by performing image processing to improve image quality for each of the plurality of images;
For each of the plurality of images, the average value of the amount of change in pixel value between pixels corresponding to the processed image is normalized so that the normalized value becomes smaller as the reproduction time of the image becomes longer. The amount of change in the pixel value by performing at least one of the process of correcting and the process of correcting the normalized value so that the elapsed time from the date and time when the image was last reproduced becomes longer A procedure for calculating a representative value of
And a procedure for selecting the predetermined number of images as the representative image from the smallest representative value in ascending order of the representative value.

Images