JP2009267682A - Representative image determination device and camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce processing needed to determine a representative image. <P>SOLUTION: The representative image determination device 1 for image groups includes: a determination means 11 of determining an image representing an image group of a plurality of images from the image group; and a control means 11 of controlling the determining means 11 to determine the representative image based upon selection reference information for selecting the representative image which corresponds to a title given to the image group. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for determining a representative image from a plurality of images.

  A technique for selecting an image representing a group characteristic from among grouped images is known (see Patent Document 1). Evaluation values representing scene-likeness such as landscapes, still lifes, and persons are individually calculated from the images constituting the group. When there are the largest number of landscape scene images in the group, the image having the largest evaluation value for the landscape scene is selected as the representative image among the landscape scene images.

JP 2007-94990 A

  The prior art has a problem that the burden of calculation processing necessary for determining a representative image increases as the types of evaluation values to be calculated increase, such as landscapes, people, still life, events, and commemorative photos.

(1) A representative image determination device for an image group according to the present invention includes determination means for determining an image representative of an image group having a plurality of images from the image group, and selection criterion information for selecting a representative image. And control means for controlling the determining means so as to determine the representative image based on selection criterion information corresponding to the title assigned to the image group.
(2) The representative image determination apparatus according to claim 1 may further include a storage unit that stores in advance a plurality of selection criterion information respectively corresponding to a plurality of titles. In this case, the control means can read the selection criterion information corresponding to the title given to the image group from the storage means, and can control the determination means to determine the representative image based on the selection criterion information.
(3) In the representative image determination device according to claim 2, the selection criterion information may include information instructing processing to be performed according to the title. The determining means in this case can determine the representative image based on the processing result.
(4) In the representative image determination device according to the third aspect, the selection criterion information may include position information related to an area name included in the title. In this case, the determining means can determine the representative image based on the shooting position information of the individual images constituting the image group and the position information read from the storage means.
(5) In the representative image determination device according to the third aspect, the selection criterion information may include color information corresponding to the title. The determining means in this case can also determine the representative image based on the color area information in the individual images constituting the image group.
(6) In the representative image determination device according to the third aspect, the processing performed according to the title may include an instruction to acquire subject tracking information regarding each image constituting the image group. The determining means in this case can determine the representative image based on the acquired subject tracking information.
(7) A camera according to the present invention includes the representative image determination device according to any one of claims 1 to 6.

  According to the present invention, processing necessary for determining a representative image can be reduced.

The best mode for carrying out the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram for explaining a main configuration of an electronic camera 1 according to the first embodiment of the present invention. The electronic camera 1 is controlled by the main CPU 11.

  The photographing lens 21 forms a subject image on the imaging surface of the imaging element 22. The imaging element 22 is configured by a CCD image sensor or the like, captures a subject image on the imaging surface, and outputs an imaging signal to the imaging circuit 23. R (red), G (green), and B (blue) color filters are respectively provided on the imaging surface of the imaging element 22 so as to correspond to the pixel positions. Since the image sensor 22 captures a subject image through the color filter, the photoelectric conversion signal output from the image sensor 22 has RGB color system color information.

  The imaging circuit 23 performs analog processing (such as gain control) on the photoelectric conversion signal output from the imaging element 22, and converts the analog imaging signal into digital data by a built-in A / D conversion circuit.

  The main CPU 11 inputs a signal output from each block, performs a predetermined calculation, and outputs a control signal based on the calculation result to each block. The image processing circuit 12 is configured as an ASIC, for example, and performs image processing on the digital image signal input from the imaging circuit 23. The image processing includes, for example, contour enhancement, color temperature adjustment (white balance adjustment) processing, and format conversion processing for an image signal.

  The image compression circuit 13 performs image compression processing at a predetermined compression ratio on the image signal processed by the image processing circuit 12 by, for example, the JPEG method. The display image creation circuit 15 creates display data for displaying the captured image on the liquid crystal monitor 16.

  The buffer memory 14 temporarily stores data before and after image processing and during image processing, stores an image file before being recorded on the recording medium 30, and stores an image file read from the recording medium 30. Used for.

  The recording medium 30 includes a memory card that can be attached to and detached from the electronic camera 1. The recording medium 30 records captured image data and an image file including the information in accordance with an instruction from the main CPU 11. The image file recorded on the recording medium 30 can be read by an instruction from the main CPU 11.

  The flash memory 19 stores a program executed by the main CPU 11, data necessary for processing performed by the main CPU 11, and the like. The contents of programs and data stored in the flash memory 19 can be added and changed by an instruction from the main CPU 11.

  The operation member 17 includes various buttons and switches of the electronic camera 1, and outputs an operation signal corresponding to the operation content of each operation member, such as a release button pressing operation and a mode switching switch switching operation, to the main CPU 11.

  The GPS device 18 receives radio waves from GPS satellites in response to instructions from the main CPU 11 and outputs received signals to the main CPU 11. The main CPU 11 performs a predetermined calculation based on the received signal from the GPS device 18 and detects the positioning information (latitude, longitude, altitude) of the electronic camera 1.

  The electronic camera 1 performs predetermined image processing and compression processing on the image signal acquired by the image sensor 22 at the time of shooting, and adds additional information including positioning information and information about the shot image to the image data after the compression processing. It is configured to generate an added image file. Specifically, an image file in Exif format is generated in which image data in JPEG format is stored in the image data portion and additional information is stored in the additional information portion. The Exif format image file is obtained by embedding thumbnail images and additional information data in image data in the JPEG image format. The generated image file is stored in the recording medium 30.

  Further, the electronic camera 1 is configured to be able to switch between a shooting mode and a playback mode. The shooting mode is an operation mode in which a subject image is shot and data of the shot image is stored in the recording medium 30 as an image file. The reproduction mode is a mode in which the reproduced image based on the image data is displayed on the liquid crystal monitor 16 by reading the captured image data from the recording medium 30.

<Grouping of shot images>
The electronic camera 1 of this embodiment has a function of automatically grouping captured images. Specifically, the image files recorded on the recording medium 30 are grouped, and the image files are stored in a folder provided for each group. Further, the electronic camera 1 determines a title (for example, a shooting scene) representing the shooting target of the image group based on the image group included in each folder. Based on the title, an image file representing the image group of each folder (group) is selected for each folder (group). Even if the image file is not stored in a folder provided for each group, an image file management table may be created, and grouping information for the image file may be stored in the table.

  FIG. 2 is a flowchart for explaining the flow of the grouping process executed by the main CPU 11. When the operation signal instructing execution of the grouping process is input from the operation member 17, the main CPU 11 starts the process illustrated in FIG.

  In step S10 of FIG. 2, the main CPU 11 groups the image files recorded on the recording medium 30 by a clustering method, and proceeds to step S20. Details of the clustering process will be described later. In step S20, the main CPU 11 performs event determination for each image included in the cluster (a set of image files after grouping), and proceeds to step S30. “Event” represents an image shooting scene, and refers to, for example, “athletic meet”, “wedding”, “cherry blossom viewing”, “sea bathing”, “travel to Hokkaido”, and the like. Details of the event determination process for each image will be described later.

  In step S30, the main CPU 11 performs event determination for each cluster and proceeds to step S40. The point that “event” is the title is the same as in step S20 above, but the event determination in step S20 represents an individual image, whereas the event determination in step S30 represents a cluster. It differs in that “event” is determined. Details of the event determination process for each cluster will be described later.

  In step S40, the main CPU 11 integrates the clusters as necessary, and proceeds to step S50. Details of the cluster integration processing will be described later. In step S50, the main CPU 11 determines an image file representing the cluster and ends the processing in FIG.

<Clustering processing>
Details of the clustering process (S10) will be described with reference to the flowchart illustrated in FIG. The clustering process is performed using, for example, shooting date / time information. In step S11 of FIG. 3, the main CPU 11 extracts information indicating the shooting date and time recorded in the additional information portion of the image file for all image files, and proceeds to step S12.

  In step S12, the main CPU 11 determines whether or not information indicating the shooting date and time has been extracted from all the image files. When the necessary information has been extracted from all the image files, the main CPU 11 makes a positive determination in step S12 and proceeds to step S13. When the necessary information has not been extracted from all the image files, the main CPU 11 makes a negative determination in step S12. Return to step S11. When returning to step S11, the extraction process is repeated.

  In step S13, the main CPU 11 uses hierarchical clustering. Among them, for example, clustering is performed using the shortest distance method. Specifically, an image file is divided into a plurality of clusters (a set of image files with close shooting times) by repeating the process of sequentially merging clusters with close shooting times starting from one cluster per image. When the number of clusters has decreased to a predetermined number, the main CPU 11 proceeds to step S14. Alternatively, when the difference between the latest shooting time in one set and the earliest shooting time in the other set is greater than or equal to a predetermined time (for example, 3 hours) between adjacent clusters, the process proceeds to step S14. Note that methods other than the shortest distance method may be used in hierarchical clustering. Furthermore, a method other than hierarchical clustering (for example, a division optimization method) may be used.

  In step S14, the main CPU 11 generates a folder in the recording medium 30 corresponding to the cluster, and proceeds to step S15. In step S15, the main CPU 11 moves the corresponding image file into the generated folder and ends the process of FIG. As a result, the image file belonging to each cluster is stored in each folder corresponding to the cluster. Even if the image file is not stored in a folder provided for each group, an image file management table may be created, and grouping information for the image file may be stored in the table.

<Event determination process for each image>
Details of the event determination process (S20) in units of images will be described with reference to the flowchart illustrated in FIG. In step S21 of FIG. 4, the main CPU 11 identifies one cluster from the plurality of clusters and proceeds to step S22. The specific order is, for example, the order of shooting date and time (priority is given to the cluster having the image file with the earliest shooting time).

  In step S22, the main CPU 11 determines an event to be determined (referred to as an event candidate). The main CPU 11 refers to the event candidate table illustrated in FIG. 8 and selects an event corresponding to the month to which the shooting date of the image file constituting the cluster belongs. For example, if the image file was taken in May, “Hanami”, “Athletic meet”, and “Wedding” are the event candidates. The event candidate table describes events with a high occurrence frequency for each month, is created in advance based on the past event occurrence month, and is recorded in the flash memory 19.

  According to FIG. 8, an event having a strong correlation with the occurrence month or season is included only in the corresponding month, and an event having a weak correlation with the occurrence month such as “wedding” is included in a plurality of months. When the shooting dates of the image files constituting the cluster belong to a plurality of months, the main CPU 11 selects event candidates corresponding to the months to which more image files belong. After determining the event to be determined as described above, the main CPU 11 proceeds to step S23.

  In step S23, the main CPU 11 specifies one image file from the image files constituting the specified cluster, and proceeds to step S24. In step S24, the main CPU 11 calculates the feature amount of the image based on the image data included in the identified image file, and proceeds to step S25.

  The main CPU 11 calculates a feature amount of the image suitable for determining the event candidate determined in step S22. The relationship between the event candidate and the feature quantity to be calculated is tabulated in advance and recorded in the flash memory 19. The feature amount is, for example, color information, sharpness information, texture information, pattern information, brightness information and the like calculated based on pixel data constituting a predetermined area of the image. Further, image size and color histogram information may be handled as feature amounts. Since the feature amount calculation is a known technique, a detailed description of feature amount calculation is omitted in this description.

  In step S25 to step S27, the main CPU 11 calculates a probability representing the likelihood of each event using each discriminator corresponding to the event candidate determined in step S22. The discriminator is feature amount information calculated by machine learning using an SVM (Support Vector Machine) method based on a plurality of sample image data. For example, the classifier for “cherry-blossom viewing” is feature amount information calculated based on a plurality of “cherry-blossom viewing” and “non-cherry-blossom viewing” sample images. The classifier for “athletic day” is feature amount information calculated based on a plurality of sample images of “athletic day” and “non-athletic day”. The identifier for “wedding” is feature amount information calculated based on a plurality of “wedding” and “non-wedding” sample images. In the present embodiment, classifiers corresponding to each of a plurality of event candidates are generated in advance and recorded in the flash memory 19. In this article, the case where the number of event candidates is three is described as an example. The main CPU 11 calculates the event likelihood (probability) P listed as the event candidate for each image specified in step S23, and then proceeds to step S28.

  The probability P representing the likelihood of an event is a boundary that partitions the space in the feature amount space represented by the discriminator (for example, a boundary between the “athletic meet” region and the “non-athletic meet” region), and the feature amount calculated in step S24. Corresponds to the distance. The feature amount calculated from an image is located behind the feature amount region corresponding to “Athletic Meet” in the feature amount space represented by the classifier for “Athletic Meet”, up to the feature amount region corresponding to “Non-athletic Meet” If the distance is long, the probability of being “athletic meet” is high. On the other hand, if the feature amount calculated from the image is located at the end of the feature amount region corresponding to “athletic meet” and the distance to the feature amount region corresponding to “non-athletic meet” is short, the probability of being “athletic meet” is low . The main CPU 11 calculates the probability according to the distance.

  In step S28, the main CPU 11 determines whether or not the processing has been completed for all the image files in the identified cluster. When the main CPU 11 calculates the feature amount and the event likelihood (probability) P for all the images in the cluster, the main CPU 11 makes a positive determination in step S28 and proceeds to step S29. If the main CPU 11 has not performed the feature amount calculation and the event likelihood (probability) P for all the images in the cluster, the main CPU 11 makes a negative determination in step S28 and returns to step S23. After returning to step S23, the main CPU 11 identifies another image file among the image files constituting the identified cluster, and proceeds to step S24.

  In step S29, the main CPU 11 determines whether or not the processing has been completed for all clusters. When the main CPU 11 performs the feature amount calculation and the event likelihood (probability) P for all the clusters, the main CPU 11 makes an affirmative determination in step S29 and ends the processing in FIG. When the main CPU 11 has not performed the feature amount calculation and the event likelihood (probability) P for all the clusters, the main CPU 11 makes a negative determination in step S29 and returns to step S21. After returning to step S21, the main CPU 11 identifies another cluster and proceeds to step S22.

<Event determination processing for each cluster>
Details of the event determination process (S30) in units of clusters will be described with reference to the flowchart illustrated in FIG. In step S31 of FIG. 5, the main CPU 11 specifies one cluster from the plurality of clusters and proceeds to step S32. The specific order is, for example, the order of shooting date and time (priority is given to the cluster having the image file with the earliest shooting time).

  In step S32, the main CPU 11 specifies one event to be determined from the event candidates and proceeds to step S33. In step S33, the main CPU 11 specifies one image file from the image files constituting the cluster specified in S31, and proceeds to step S34. In step S34, the main CPU 11 determines whether or not the event likelihood (probability) P for the event specified in S32 is equal to or greater than a predetermined determination threshold value Pth. When the event likelihood (probability) P is equal to or greater than the determination threshold value Pth, the main CPU 11 makes a positive determination in step S34 and proceeds to step S35. When the event likelihood (probability) P is less than the determination threshold value Pth, the main CPU 11 makes a negative determination in step S34 and proceeds to step S36.

  In step S35, the main CPU 11 calculates Psum = Psum + P and proceeds to step S36. In step S36, the main CPU 11 determines whether or not the processing has been completed for all the image files in the specified cluster. When the main CPU 11 has processed all the images in the cluster, the main CPU 11 makes an affirmative decision in step S36 and proceeds to step S37. If the main CPU 11 has not processed all the images in the cluster, the main CPU 11 makes a negative determination in step S36 and returns to step S33. After returning to step S33, the main CPU 11 specifies another one of the image files constituting the specified cluster, and proceeds to step S34.

  In step S37, the main CPU 11 determines whether the number of event probabilities (probabilities) P added to Psum is equal to or greater than N percent of the total number of image files in the cluster. When the added number of P is N percent or more, the main CPU 11 makes a positive determination in step S37 and proceeds to step S38. When the added number of P is less than N percent, the main CPU 11 makes a negative determination in step S37 and proceeds to step S41. In step S41, the main CPU 11 sets Psum = 0 and proceeds to step S38.

  In step S38, the main CPU 11 determines whether or not the process of calculating Psum for all event candidates has been completed. When the calculation process is completed for all events, the main CPU 11 makes a positive determination in step S38 and proceeds to step S39. If the calculation process has not been completed for all events, the main CPU 11 makes a negative determination in step S38 and returns to step S32. After returning to step S32, the main CPU 11 specifies one other event to be determined from the event candidates and proceeds to step S33.

  In step S39, the main CPU 11 determines the event candidate corresponding to the maximum value among the Psum calculated for each event candidate as the title event of the cluster, and proceeds to step S42.

  In step S42, the main CPU 11 determines whether or not the calculation of Psum and the process for determining the title event have been completed for all clusters. When the process is finished for all clusters, the main CPU 11 makes a positive determination in step S42 and ends the process of FIG. If the processing has not been completed for all clusters, the main CPU 11 makes a negative determination in step S42 and returns to step S31. After returning to step S31, the main CPU 11 identifies another cluster and proceeds to step S32.

  9 to 11 are diagrams illustrating event determination for a cluster including a set of image files (number of images = 5). The event candidates are, for example, “athletic meet”, “wedding”, and “cherry blossom viewing”. For each of the images 1 to 5, the probability P of “athletic meet” likelihood, “wedding” likelihood, and “cherry blossom viewing” probability is set. It has been calculated. In this example, the determination threshold value Pth = 40% and the determination threshold value N = 40%.

  In the case of FIG. 9, the Psum regarding the “Athletic meet” likelihood is P for each of the images having the “Athletic meet” likelihood P equal to or greater than the determination threshold Pth (that is, the image 1 to the image 3 other than the image 4 and the image 5). Are added (step S35). In this case, Psum is 85 (image 1) +90 (image 2) +80 (image 3) +75 (image 5) = 330.

  In the case of FIG. 10, only P4 indicating “wedding” likelihood is equal to or greater than the determination threshold Pth, so only P calculated for image 4 is added to Psum regarding “wedding” likelihood. Target. Thereby, Psum = 45 (image 4) is obtained. Here, the image 4 (one image) corresponds to 20% of the number of images 5 and is therefore less than N (40%). Therefore, the main CPU 11 sets Psum in the case of FIG. 10 to 0 (step S41).

  In FIG. 11, Psum regarding “Hanami” -likeness is obtained for images (ie, image 1 to image 2 other than image 3 and image 4 to image 5) that are P or more indicating “hanami” -likeness, respectively. Of P is added (step S35). In this case, Psum is 60 (image 1) +70 (image 2) +65 (image 4) +75 (image 5) = 270.

  The main CPU 11 sets the “athletic meet” corresponding to the maximum Psum among the event candidates (“athletic meet”, “wedding”, and “cherry blossom viewing”) as the title event of the cluster.

<Cluster integration processing>
Details of the cluster integration processing (S40) will be described with reference to the flowchart illustrated in FIG. In step S51 of FIG. 6, the main CPU 11 determines whether or not the time difference between adjacent clusters among the plurality of clusters is equal to or less than the determination threshold T. For example, when the difference between the latest shooting time in one cluster and the earliest shooting time in the other cluster is equal to or less than T, the main CPU 11 makes an affirmative determination in step S51 and proceeds to step S52. When the time difference exceeds T, the main CPU 11 makes a negative determination in step S51 and ends the process of FIG. When a negative determination is made in step S51, cluster integration is not performed.

  In step S52, the main CPU 11 determines whether or not the title events of the adjacent clusters with the time difference T or less are the same. When the title event is the same, the main CPU 11 makes a positive determination in step S52 and proceeds to step S53. If the title events are not the same, the main CPU 11 makes a negative determination in step S52 and ends the process of FIG. When a negative determination is made in step S52, cluster integration is not performed.

  In step S53, the main CPU 11 integrates the two clusters into one cluster and ends the process shown in FIG. The main CPU 11 repeats cluster integration processing for all clusters. With the above processing, the number of clusters decreases.

<Determination of representative image for each cluster>
Details of the representative image determination processing in units of clusters will be described with reference to the flowchart illustrated in FIG. In step S61 of FIG. 7, the main CPU 11 specifies one cluster from the plurality of clusters, and proceeds to step S62. The specific order is, for example, the order of shooting date and time (priority is given to the cluster having the image file with the earliest shooting time).

  In step S62, the main CPU 11 reads the selection criterion information corresponding to the title event determined in step S39 from the flash memory 19, and proceeds to step S63. The selection criterion information is obtained by predetermining a representative image determination method for each title event, forming a table, and recording it in the flash memory 19.

  In step S63, the main CPU 11 selects a representative image from the image file group included in the cluster based on the selection criterion information, and proceeds to step S64. In step S64, the main CPU 11 determines whether representative images have been selected for all clusters. When the process is finished for all clusters, the main CPU 11 makes a positive determination in step S64 and ends the process of FIG. If the process has not been completed for all clusters, the main CPU 11 makes a negative determination in step S64 and returns to step S61. After returning to step S61, the main CPU 11 identifies another cluster and proceeds to step S62.

  The selection criterion information will be described. The main CPU 11 refers to the selection criterion information table illustrated in FIG. 12 and selects the selection criterion information corresponding to the cluster title event. The selection criterion information table is created in advance and recorded in the flash memory 19. For example, if the title event of a cluster is “Wedding”, “Hatsumode”, “Hinamatsuri”, “Entrance Ceremony”, “Graduation Ceremony”, the proportion of the facial area included in the image in the cluster Is selected as the representative image. Note that the face detection process for performing the face detection process based on the image data and detecting the “face” of the person included in the image is a known technique, and thus the description thereof is omitted.

  In addition, when the title event of the cluster is “sea bathing”, “diving”, “autumn leaf hunting”, “golf”, the ratio of the predetermined color area included in the image is the closest to the predetermined ratio among the images included in the cluster. Choose one as the representative image. The predetermined color is, for example, a blue region (sea bathing, diving), a red or yellow region (autumn leaves hunting), and a green region (golf).

  Further, among the images included in the cluster, an image having the maximum probability P indicating the likelihood of an event corresponding to the title event of the cluster may be selected as the representative image. As described above, the representative image determination condition is determined in advance for each title event, and an image representing the title event is determined based on the determination condition.

According to the embodiment described above, the following effects can be obtained.
(1) A plurality of event candidates (athletic meet, wedding ceremony, cherry-blossom viewing) are given to a cluster (a set of image files), and feature quantities suitable for determining each event candidate are calculated for each image in the cluster. An event representative of the cluster (a set of image files) is determined from event candidates based on the feature amount of each image. Unlike the prior art, the SVM method is not used for clusters, that is, the sum Psum for calculating the sum Psum of the event-likeness (probability) P for each image without the need to calculate the feature amount for the cluster. Since only the count of the number of added images, the Psum size comparison between the event candidates, and the size comparison between the determination threshold value Pth and the probability P are sufficient, processing can be performed in a shorter time compared to the case of calculating the feature amount for the image group. it can.

(2) Since the event candidate corresponding to the maximum value of the sum Psum of the event-likeness (probability) P for each image is determined, an event representing a set of image files can be appropriately determined.

(3) When the event likelihood (probability) P for each image is less than the determination threshold Pth, it is not added to the sum Psum, so that the degree of contribution of an image having a low probability of being an event candidate to the sum Psum is suppressed. Can do.

(4) When the number of images satisfying the determination threshold Pth (that is, the number of images obtained by adding P to the sum Psum) is less than a predetermined ratio with respect to the number of images in the cluster, Psum is set to 0 and the event candidate is excluded. I tried to do it. This makes it possible to appropriately determine an event that represents a set of image files.

(5) Clusters (a set of image files) are clustered according to the shooting date and time of images. Thereby, event candidates according to the shooting date and time can be given.

(6) Since a plurality of event candidates are given to the cluster, the most appropriate event can be determined from the plurality of event candidates.

(7) Among the plurality of clusters, when the time difference between adjacent clusters is equal to or less than the determination threshold T and the title events of the clusters are the same, both clusters are integrated into one. As a result, it is possible to avoid performing clustering more finely than necessary than in the case of clustering only by shooting date and time, and to conform to the intention when the images are classified and arranged.

(8) When the difference between the latest shooting time in one cluster and the earliest shooting time in the other cluster is equal to or less than T, the determination in step S51 is affirmative. Since the same event is often shot continuously without taking a shooting interval, it is possible to detect a case where there is a high possibility that the title events of the clusters are the same.

(9) The relationship between the event candidate and the feature quantity to be calculated is tabulated in advance and recorded in the flash memory 19. Thereby, it is possible to calculate the image feature amount suitable for the determination of the event candidate, and to appropriately determine the event representing the set of image files.

(10) A representative image is selected from images included in the cluster based on selection criterion information provided corresponding to the title event of the cluster. As a result, the representative image can be determined with the most suitable selection criterion for each cluster.

(11) Selection criteria information suitable for each title event is tabulated in advance and recorded in the flash memory 19. When the title event is different, the selection criterion is switched with reference to the selection criterion information table. By making a table, combinations of title events and selection criteria can be freely set.

(Modification 1)
The selection criterion information table may be configured to be changeable by a user operation. For example, the editing screen of the selection reference information table is displayed on the liquid crystal monitor 16, and the user changes the table contents by operating the operation member 17. The changed contents are recorded in the flash memory 19.

(Modification 2)
At the time of clustering, clustering may be performed on a shooting date basis.

(Modification 3)
When clustering, it may be performed according to the shooting position information, not according to the shooting date / time information. Specifically, in step S11 (FIG. 3), the main CPU 11 extracts the positioning information indicating the shooting position recorded in the additional information part of the image file for all the image files. In step S13 (FIG. 3), the main CPU 11 repeats the process of sequentially merging clusters with close shooting positions from one cluster for each image as a processing start point, so that the image file is divided into a plurality of clusters (close to shooting positions). Image files). The main CPU 11 proceeds to step S14 when the interval (distance) between the shooting positions between adjacent clusters is equal to or greater than a predetermined distance (for example, 1 km).

(Modification 4)
The event candidate table may be configured to be changeable by a user operation. For example, the event candidate table editing screen is displayed on the liquid crystal monitor 16, and the user changes the table contents by operating the operation member 17. The changed contents are recorded in the flash memory 19.

(Modification 5)
It is good also as a structure which abbreviate | omits step S22 of FIG. In this case, all event candidates possessed by the system are selected without referring to the event candidate table.

(Modification 6)
In the event candidate table in the third modification, for example, event candidates may be associated with each region including the shooting position. For example, if you are near the sea, choose events such as bathing, water splitting, surfing, diving and so on. Choose events like camping and hiking if you're near the mountains. In this case, the event candidate table describes events having a high occurrence frequency for each region. The event candidate table is created in advance based on past event occurrence locations and recorded in the flash memory 19.

(Modification 7)
The title event of a cluster (a set of image files) may be determined according to the number of images for which the event likelihood (probability) P exceeds the determination threshold value Pth. For example, among the event candidates (“athletic meet”, “wedding”, and “cherry blossom viewing”), the event candidate corresponding to the larger number of images exceeding the determination threshold Pth is set as the title event of the cluster.

(Modification 8)
Further, a combination of a determination method for determining an event candidate having a large number of images whose event likelihood (probability) P exceeds the determination threshold Pth as a title event and a determination method for determining an event candidate corresponding to the maximum Psum as a title event. Also good. In this case, when superiority or inferiority is not determined by any one of the determination methods, the determination is performed using the other determination method. For example, in the example of FIGS. 9 to 11, the event candidates “athletic meet” and “cherry-blossom viewing” have the same number of images whose event likelihood (probability) P exceeds the determination threshold value Pth and cannot be superior or inferior. Therefore, the event candidate “athletic meeting” corresponding to the maximum Psum is set as the title event of the cluster.

(Modification 9)
When determining whether or not the time difference between adjacent clusters is equal to or less than the determination threshold T, the determination may be made based on the time difference corresponding to the center of gravity of each cluster. In this case, the main CPU 11 performs a step when the difference between the average shooting time of the image group belonging to one cluster and the average shooting time of the image group belonging to the other cluster is equal to or less than T in step S51 (FIG. 6). Affirmative determination is made in S51 and the process proceeds to step S52.

(Modification 10)
In the case of Modification 3 and Modification 6, it is determined whether or not the distance between adjacent clusters is equal to or less than the determination threshold D. In this case, the main CPU 11 determines in step S51 (FIG. 6) that the shortest distance between the shooting position of the image group belonging to one cluster and the shooting position of the image group belonging to the other cluster is the above. If it is D or less, an affirmative decision is made in step S51 and the operation proceeds to step S52. Since the same event is often continuously shot in the same area, it is possible to detect a case where there is a high possibility that the title events of the clusters are the same.

  According to the tenth modification, when the distance difference between adjacent clusters is equal to or smaller than the determination threshold D and the title events of the clusters are the same, the two clusters are integrated into one. . As a result, it is possible to avoid clustering more finely than necessary than in the case of clustering only at shooting positions, and to conform to the intention when sorting and organizing images.

(Second embodiment)
Instead of the selection criterion information table illustrated in FIG. 12, a selection criterion information table illustrated in FIG. 14 may be referred to. The selection criterion information table shown in FIG. 14 is created in advance and recorded in the flash memory 19 in the same manner as the table according to the first embodiment. For example, when the title event of the cluster is “Travel in Hokkaido”, the main CPU 11 selects the representative image whose shooting position information is closest to the position information corresponding to the area (for example, a famous place in Hokkaido).

  Specifically, position information related to the name of the area included in the title event is recorded in the flash memory 19 in advance. For example, when “Hokkaido” is included in the title event, the main CPU 11 reads position information corresponding to a famous place of “Hokkaido” (for example, Sarobetsu Wilderness) from the flash memory 19. Further, the main CPU 11 determines whether or not the position information of the shooting point is recorded in the additional information data of each image file for all the image files included in the cluster, and the position information of the shooting point is recorded. If so, position information is acquired from the image file.

  Then, based on the position information read from the flash memory 19 and the position information acquired from the image file in the cluster, it is determined whether or not each image was taken at a famous place. As a result of comparison, the main CPU 11 selects the shortest distance between the two as the representative image of the cluster.

  If the title event of the cluster is “diving” or “sea bathing”, the main CPU 11 sets the ratio of “what the sea is reflected”, that is, the specific color area (in this case, the sea) in the image to a predetermined ratio. Select the closest image as the representative image. Specifically, color information related to the title event is recorded in the flash memory 19 in advance. For example, when the title event includes characters related to “sea”, the main CPU 11 reads information indicating a specific color (for example, cobalt blue) from the flash memory 19.

  The main CPU 11 further acquires color information from the image data of each image file for all the image files included in the cluster. For example, the image is divided into a predetermined number of areas, and it is determined whether or not the color of the pixel group located at the center of each divided area matches a specific color. Then, the main CPU 11 obtains a ratio between the predetermined number and the number of divided areas that match the specific color. Since the ratio of specific colors (ocean colors) in the image, such as shooting scenes on the ship, shooting scenes on the beach, and underwater shooting scenes, is different, the ratio information according to the scene you want to be the representative image in advance is flash memory 19 is recorded. As described above, the flash memory 19 also stores information indicating processing necessary for determining the representative image and information necessary for the processing.

  Furthermore, when the title event of the cluster is “athletic meet”, the main CPU 11 selects a representative image whose motion information of the main subject is larger than a predetermined value. Specifically, information instructing acquisition of subject tracking information is recorded in the flash memory 19 in advance. For example, when the title event includes characters related to “sports” such as “athletic meet”, the main CPU 11 reads information instructing acquisition of subject tracking information from the flash memory 19.

  The main CPU 11 further determines whether or not information related to the subject tracking result is recorded in the additional information data of each image file for all the image files included in the cluster, and information related to the subject tracking result is obtained. If it is recorded, information about the result of subject tracking is acquired from the image file.

  Then, the main CPU 11 compares time-series changes in the coordinate value of the subject position between the images based on the acquired tracking information. As a result of the comparison, the main CPU 11 selects the image having the largest change in the coordinate value of the subject position as the representative image of the cluster.

  When the main CPU 11 provides a folder structure corresponding to each cluster to the user, for example, a thumbnail image of a representative image is arranged on the folder icon and displayed on the liquid crystal monitor 16.

  That is, the main CPU 11 displays a thumbnail image of a representative image in which a famous place is photographed on the folder icon of the title event “Hokkaido Travel”. In addition, a thumbnail image of a representative image in which cobalt blue occupies a predetermined ratio of the shooting screen is displayed on the folder icon of the title event “Sea bathing”. Furthermore, a thumbnail image of a representative image in which a main subject with a large amount of movement is captured is displayed on the folder icon of the title event of “Athletic meet”.

According to the second embodiment described above, the following operational effects can be obtained.
(1) When the title event includes the name of the region XX as in XX trip, the image having the shooting position information closest to a predetermined point (such as a famous place) in the region is selected as the representative image. . In general, since a famous place is known to represent a region, an image suitable as a representative of a plurality of images constituting a cluster can be determined.

(2) When the title event is reminiscent of a specific color such as “sea bathing”, an image having a predetermined ratio of the subject area of that color in the image is selected as the representative image. In general, cobalt blue is known to represent the color of the sea. Therefore, if cobalt blue is associated with “sea bathing” in advance, a suitable image can be determined as a representative of a plurality of images constituting a cluster.

(3) When the title event includes a moving subject such as “athletic meet”, an image with a large change in coordinates indicating the subject position is selected as the representative image. Generally speaking, an athletic meet is known as an event with many moving scenes such as an athlete race, and therefore, a suitable image can be determined as a representative of a plurality of images constituting a cluster.

(Modification 11)
Based on the shooting position information, instead of selecting the image with the shortest distance from the sight as a representative image, the position information about a plurality of sights (for example, Sarobetsu Wilderness, Rishiri Island) is recorded in the flash memory 19 in advance, Priorities (for example, (1) Rishiri Island, (2) Sarobetsu Wilderness) may be assigned to these sights. If images taken at multiple different sights are included in the images that make up the cluster, the highest priority is added among images taken within a predetermined distance (eg 500 m) from the sights. An image having position information corresponding to a famous place (for example, Rishiri Island) is selected as a representative image. Thereby, when a picked-up image is included for a plurality of sights, a representative image according to the user's preference can be determined.

(Modification 12)
If the title event of the cluster is “diving” or “sea bathing”, a representative image may be selected when the proportion of the sea in the image is a predetermined proportion and the focal length of the photographing lens at the time of photographing is shorter than a predetermined value. . As the focal length information, information on photographing conditions stored as additional information data of the image file is used. Thereby, a representative image having a wide shooting angle of view can be selected.

(Modification 13)
The example in which images are grouped in the electronic camera 1 and the representative image is determined has been described. However, by causing the computer device 10 shown in FIG. 13 to execute a representative image determination program for performing the processing illustrated in FIGS. The representative image determination device may be configured. When the representative image determination program is loaded into the personal computer 10 and used, the program is loaded into the data storage device of the personal computer 10 and then executed as a representative image determination device for the image group. To do.

  The loading of the program to the personal computer 10 may be performed by setting the recording medium 104 such as a CD-ROM storing the program in the personal computer 10 or by a method via the communication line 101 such as a network. You may load. When passing through the communication line 101, the program is stored in the hard disk device 103 of the server (computer) 102 connected to the communication line 101. The representative image determination program can be supplied as various types of computer program products such as provision via the recording medium 104 or the communication line 101.

  Further, it may be executed on the server side in a form such as ASP (Application Service Provider).

(Modification 14)
The above-described shooting date / time information, shooting position information, shooting conditions, and provision of event candidates according to the presence / absence of a specific subject may be configured to include a multidimensional event candidate table by combining at least two. .

  The above description is merely an example, and is not limited to the configuration of the above embodiment.

It is a block diagram explaining the principal part structure of the electronic camera 1 by 1st embodiment of this invention. It is a flowchart explaining the flow of the grouping process which main CPU performs. It is a flowchart explaining a clustering process. It is a flowchart explaining the event determination process for every image. It is a flowchart explaining the event determination process for every cluster. It is a flowchart explaining cluster integration processing. It is a flowchart explaining the process which determines a representative image. It is a figure which illustrates an event candidate table. It is a figure which illustrates the event determination (athletic meet) with respect to a cluster. It is a figure which illustrates the event determination (wedding) with respect to a cluster. It is a figure which illustrates the event determination (cherry-blossom viewing) with respect to a cluster. It is a figure which illustrates a selection criteria information table. It is a figure which illustrates a computer apparatus. It is a figure which illustrates the selection reference | standard information table by 2nd embodiment.

Explanation of symbols

1 ... Electronic camera 11 ... Main CPU
DESCRIPTION OF SYMBOLS 14 ... Buffer memory 15 ... Display image creation circuit 16 ... Liquid crystal monitor 17 ... Operation member 18 ... GPS apparatus 19 ... Flash memory 30 ... Recording medium

Claims (7)

Determining means for determining an image representative of an image group having a plurality of images from the image group;
Control means for controlling the decision means to decide the representative image based on the selection criteria information according to the title given to the image group, the selection criteria information for selecting the representative image A representative image determination device for an image group. The representative image determination device according to claim 1,
Storage means for storing in advance a plurality of selection criterion information corresponding respectively to a plurality of titles;
The control means reads selection criterion information corresponding to a title assigned to the image group from the storage means, and controls the determination means to determine a representative image based on the selection criterion information. A representative image determination device for the image group. In the representative image determination device according to claim 2,
The selection criterion information includes information instructing processing to be performed according to the title,
The representative image determining apparatus for an image group, wherein the determining means determines the representative image based on the result of the processing. In the representative image determination device according to claim 3,
The selection criterion information includes location information related to the area name included in the title,
The determination unit determines the representative image based on shooting position information of individual images constituting the image group and position information read from the storage unit. Decision device. In the representative image determination device according to claim 3,
The selection criterion information includes color information corresponding to the title,
The representative image determining device for an image group, wherein the determining means determines the representative image based on area information of the color in individual images constituting the image group. In the representative image determination device according to claim 3,
The processing to be performed according to the title includes an instruction to acquire subject tracking information regarding each image constituting the image group,
The apparatus for determining a representative image of an image group, wherein the determining means determines the representative image based on the acquired subject tracking information.   A camera comprising the representative image determination device according to any one of claims 1 to 6.

Images