JP2009301119A - Image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device arranging and displaying photographic images such that a user can intuitively search for a desired photographic image. <P>SOLUTION: In this image display device, the photographic images are classified according to characteristics of photographic subjects and photographed dates of the photographic images recorded in a recording part of a server, and are arranged into folders. Bookshelf images corresponding to contents of the photographic images stored in the respective folders are created, and the corresponding folders 601, 602 are superposed on the created bookshelf images, and they are made to be displayed. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image display apparatus capable of organizing captured images and easily taking out and viewing the organized captured images.

  In recent years, digital cameras have become widespread and photographed images have been converted into digital data. Accompanying the digitization of photographed images, a large amount of photographed image data can be recorded on a large-capacity recording medium such as a hard disk. On the other hand, it has become difficult for a user to find a desired captured image from a large number of captured images recorded on a recording medium.

Patent Document 1 has been proposed as a proposal for organizing captured images so that a user can easily find a desired captured image from such a large number of captured images. In the proposal of Patent Document 1, the captured images are organized based on the supplementary information attached to the captured image data, the organized captured images are converted into an electronic album, and the electronic album is displayed side by side on the bookshelf image. I am doing so. As a result, the captured images are organized and displayed for each event, so that the user can easily infer which captured image is recorded in which electronic album.
JP 7-123307 A

  Here, in the method of Patent Document 1, the electronic albums are displayed side by side on the bookshelf image, but this arrangement method itself has no particular meaning. For this reason, it may be insufficient for the user to intuitively search for a desired captured image.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image display apparatus capable of organizing and displaying captured images so that a user can find a desired captured image more intuitively.

  In order to achieve the above object, the image display device according to the first aspect of the present invention is based on the shooting date and time attached to each of a plurality of captured images and the event information related to each of the captured images. The image processing apparatus includes a classification unit that classifies images into a plurality of groups, and a display unit that displays the groups according to the shooting date and time from the back to the front of a bookcase that virtually represents depth.

  ADVANTAGE OF THE INVENTION According to this invention, the image display apparatus which can arrange | position and display a picked-up image so that a user can find a desired picked-up image more intuitively can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of an image display system including an image display apparatus according to an embodiment of the present invention. The image display system shown in FIG. 1 is configured such that a camera 100, a scanner 200, and a personal computer 300 are communicably connected to a server 400 via a communication line 500. Here, the communication line 500 is a general-purpose communication line such as an Internet line.

  The camera 100 includes a control unit 101, an operation input unit 102, an imaging unit 103, a clock unit 104, a recording unit 105, a display unit 106, and a communication unit 107.

  The control unit 101 controls the overall operation of the camera 100. The control unit 101 controls various processes according to the operation state of the operation input unit 102 by the user. The operation input unit 102 is a power switch for turning on / off the power of the camera 100, a release button for giving an instruction to execute shooting of the camera 100, a cross key for performing various selection operations in the camera 100, and various decisions in the camera 100. Various operation members for the user to operate the camera 100 such as an OK button for operation are included.

  The imaging unit 103 captures a subject and acquires a captured image under the control of the control unit 101. The clock unit 104 detects the shooting date and time, that is, the acquisition date and time of the shot image. The shooting date and time detected by the clock unit 104 is attached to the captured image acquired by the imaging unit 103.

  The recording unit 105 is a memory card configured to be detachable from the camera 100, for example. In the recording unit 105, a captured image acquired through the imaging unit 103 is recorded. The display unit 106 displays various images such as images recorded in the recording unit 105 under the control unit 101.

  The communication unit 107 performs various processes related to communication with the server 400 under the control of the control unit 101.

  The scanner 200 takes a photograph or the like as digital data. The digital data of the image captured by the scanner 200 can be transmitted to the server 400. The personal computer (PC) 300 has a display unit such as a liquid crystal display and an operation input unit such as a mouse and a keyboard, and is installed with browser software for viewing Web page data transmitted from the server 400. ing. With this PC 300, it is possible to appreciate the captured image recorded in the server 400.

  The server 400 as an example of the image display apparatus according to the present embodiment includes a control unit 401, a recording unit 402, a subject detection unit 403, a date / time detection unit 404, a classification unit 405, and a communication unit 406. Yes.

  The control unit 401 controls various processes in the server 400. The recording unit 402 records a captured image transmitted from the camera 100 or the scanner 200 via the communication line 500. The recording unit 402 includes a hard disk having a larger recording capacity than the recording unit 105 of the camera 100.

  The subject detection unit 403 detects the whole or a part of the subject in the captured image recorded in the recording unit 402 using edge extraction processing or the like. The date / time detection unit 404 detects the shooting date / time of the captured image. If the captured image is transmitted from the camera 100, the information is extracted because the captured date and time is attached to the captured image. On the other hand, when the photographed image is transmitted from the scanner 200, for example, when the transmitted image includes character information indicating the photographing date and time, the character information is recognized as an image. Detect the shooting date.

  The classification unit 405 classifies the plurality of photographed images recorded in the recording unit 402 into a plurality of groups according to the photographing date / time detected by the date / time detection unit 404 and event information related to each photographed image. Details of this classification will be described later.

  The communication unit 406 performs various processes related to communication with the camera 100 under the control of the control unit 401.

  Hereinafter, the operation of the image display system shown in FIG. 1 will be described. FIG. 2 is a diagram showing an example of a screen displayed on the display unit of the PC 300 by the image display system shown in FIG. As described above, in this embodiment, the captured images recorded in the recording unit 402 are classified according to the shooting date and time and the contents of the event indicated by the captured image, and are grouped for each folder. Further, when a shelf image is generated according to the contents of the photographed image and the server 400 is accessed from the PC 300, the shelf image is displayed on the left and right sides of the screen as shown in FIG. Folders are superimposed and displayed on each of the bookshelf images.

  As shown in FIG. 2, the bookshelf image is displayed so that the depth is virtually represented by a perspective method, for example. Furthermore, in the present embodiment, as an example, a folder corresponding to a new group of shooting date and time (shooting age) is placed on the front shelf image of FIG. 2, and the shooting date and time (shooting date and time) is placed on the back shelf image of FIG. ) Place a folder corresponding to the old group. Of course, you may make it arrange | position the folder corresponding to a new group of imaging | photography date (photographing age) to the back shelf image.

  In FIG. 2, a folder 601 including photographed images classified by the classification unit 405 is arranged in the left bookshelf image (book shelf 1 and bookshelf 3), and the right bookcase image (book shelves 2 and 4) is arranged. The folder 602 including the captured images that are not classified by the classification unit 405 is arranged. Further, in the vicinity of each bookcase image, a label 603 for indicating the characteristics of the captured image in the folder arranged in each bookcase image is displayed.

  As shown in FIG. 2, the lower part (floor portion) of the screen may be used as a shelf image (shelf shelf 5), and the folder 604 may be arranged on this floor portion. Furthermore, the ceiling portion may be used as a bookshelf image.

  The user can operate the cursor 600 of FIG. When the user operates the cursor 600 to select any folder, thumbnail images of the captured images in that folder are displayed. Furthermore, the captured image selected by selecting the thumbnail image is enlarged and displayed on the display unit of the PC 300.

  2 is scrolled in the depth direction by operating a mouse or the like when the display of FIG. 2 is made, a folder in which past captured images are stored is further arranged than that shown in FIG. The displayed shelf image is displayed.

  Here, the control unit 401 counts the frequency of viewing the captured image by the user, for example, based on the number of folder selections. When the display of FIG. 2 is performed, the control unit 401 rearranges the folders according to the viewing frequency. In the example of FIG. 2, a folder storing captured images with a high viewing frequency is arranged in the upper part of the shelf image, and a folder storing captured images with a low viewing frequency is arranged in the lower part of the shelf image. Yes.

  FIG. 3 is a flowchart showing display control for performing the display of FIG. 3 is controlled by the control unit 401 of the server 400. In FIG. 3, only the concept of display control will be described, and details of each process will be described later.

  In FIG. 3, first, the control unit 401 uses the classification unit 405 to classify all the captured images recorded in the recording unit 402 into a plurality of groups, and group the classified captured images into folders for each group (step S1). ). After the classification process in step S1, the control unit 401 uses the classification unit 405 to select an event time at which the captured image in the folder is captured based on the shooting date and time of the captured image and the characteristics of the subject in the captured image. The estimated event time estimation process is performed (step S2). After the event time estimation process, the control unit 401 determines the conditions of the bookshelf image according to the event time (step S3). The name of the label 603 arranged in the vicinity of each bookshelf image is determined by the processing for determining the conditions of the bookshelf image. After determining the conditions of the bookshelf image, the control unit 401 performs a bookshelf organizing process for arranging each folder on each bookshelf image determined in step S3 (step S4). Next, the control unit 401 performs a book shelf creation process for increasing the number of steps in the book shelf image according to the appreciating state of the photographed image (step S5).

  Data for displaying the screen shown in FIG. 2 (Web page data such as Html data) is created by the processing from step S1 to step S5. When an access is made from the PC 300, the control unit 401 transmits the data created by the processing up to step S5 to the PC 300. As a result, the screen shown in FIG. 2 is displayed on the display unit of the PC 300 (step S6).

  FIG. 4 is a flowchart showing the classification process in step S1 of FIG. First, the classification unit 405 uses the date / time detection unit 404 to detect the shooting date / time of each captured image recorded in the recording unit 402. Then, the classification unit 405 determines whether or not a predetermined number of photographed images are photographed on the same day from the photographing date and time of each photographed image (step S11). If it is determined in step S11 that a predetermined number or more of the captured images are captured on the same day, the classification unit 405 collects the captured images captured on the same day in the same folder (step S12). For example, if it is an athletic meet or the like, there is a high possibility that a lot of pictures will be taken in one day. Therefore, when a large number of captured images are captured on the same day, it is assumed that there is some specific event and the captured images captured on that day are collected in the same folder. Note that the process of grouping the captured images into the same folder can be performed by changing the directory structure in the recording unit 402, for example. The folder name is automatically set, for example, by the classification unit 405 during the process of step S12. Of course, the user may be able to set by operating the PC 300.

  Further, in the determination in step S11, when a predetermined number or more of the captured images are not captured on the same day, the classification unit 405 has a period during which the constant imaging is performed within a predetermined period (a period during which the imaging frequency is high). It is determined whether or not it exists (step S13). If it is determined in step S13 that there is a period with high shooting frequency, the classification unit 405 collects all the shot images shot during the period with high shooting frequency in the same folder. For example, during a trip or the like, since shooting is performed at various places within a certain period, it is considered that the frequency of shooting within a predetermined period is high. Therefore, even when there is a period with high shooting frequency, if there is some specific event, the shot images shot within that period are collected in the same folder.

  Also, in the determination in step S13, when there is no period with high imaging frequency, the control unit 401 uses the same folder to store the captured images captured within a predetermined period, such as monthly, seasonal, or yearly. (Step S14).

  After step S12 or step S14, the classification unit 405 determines whether all captured images have been classified into folders (step S15). If it is determined in step S15 that classification of all captured images has not been completed, the process returns to step S11, and the classification unit 405 continues to classify captured images. On the other hand, in the determination of step S15, when the classification of all the captured images is completed, the classification unit 405 ends the process of FIG.

  FIG. 5 is a flowchart showing the event time determination process in step S2 of FIG. It is assumed that the date of birth of the user is registered in the recording unit 402 in advance prior to performing each process of FIG. Details of each process in FIG. 5 will be described later.

  In FIG. 5, the classification unit 405 performs a kindergarten determination process for all captured images (step S <b> 21). After this kindergarten determination process, the classification unit 405 performs a school determination process on a photographed image that is determined not to have been photographed in the kindergarten period (step S22). After this school determination process, a wedding determination process (step S23) is performed on a photographed image that has been determined not to have been photographed in school days (elementary school, junior high school, high school). After these processes, the classification unit 405 ends the process of FIG.

  FIG. 6 is a flowchart showing the flow of the kindergarten determination process. First, the classification unit 405 determines from the shooting date / time of the shot image whether there is a shot image (image shot in April of the year when the user is considered to enter the park) corresponding to the shooting season. Determination is made (step S31). If it is determined in step S31 that there is no photographed image whose photographing date and time corresponds to the entrance season, the classification unit 405 ends the process of FIG.

  In addition, in the determination in step S31, when there is a captured image whose shooting date / time corresponds to the entrance season, the classification unit 405 determines whether the number of captured images whose shooting date / time corresponds to the entrance season is equal to or greater than a predetermined number. That is, it is determined whether or not a plurality of captured images corresponding to the admission season are grouped in the same folder (step S32). If it is determined in step S32 that the number of photographed images corresponding to the entrance season is less than the predetermined number, the classification unit 405 ends the process of FIG.

  On the other hand, if it is determined in step S32 that the number of photographed images corresponding to the entrance season is equal to or greater than the predetermined number, the classification unit 405 uses the subject detection unit 403 to capture the photographed images corresponding to the entrance season. The face portion of the subject at is detected (step S33). After the face detection, the classification unit 405 determines whether or not there is a captured image in which a plurality of face portions are arranged in the captured image whose shooting date / time corresponds to the entrance season (step S34). The process of step S34 is a process for determining whether or not a group photo at the entrance ceremony has been taken. If it is determined in step S34 that there is no captured image in which a plurality of face portions are arranged, the classification unit 405 determines that a group photo has not been captured, and ends the processing in FIG.

  In addition, in the determination in step S34, when there is a photographed image in which a plurality of faces are arranged, the classification unit 405 causes the subject detection unit 403 to detect the body of the subject in the photographed image in which the plurality of faces are arranged. Part. Then, the classification unit 405 determines whether or not the plurality of subjects in the captured image are children (children) by determining whether the ratio of the face to the body (head and body) is large (step S35). ). If it is determined in step S35 that the plurality of subjects in the captured image are not children, the classification unit 405 ends the process in FIG.

  In the determination in step S35, if the subject in the captured image is a child, the classification unit 405 determines whether or not the plurality of subjects detected by the subject detection unit 403 are wearing substantially the same clothes (uniforms). Determination is made (step S36). This determination can be made, for example, by matching a body image between a plurality of subjects in a captured image. If it is determined in step S36 that a plurality of subjects do not wear the same clothes, the classification unit 405 ends the processing of FIG.

  In the determination in step S36, if a plurality of subjects are wearing the same uniform, it is estimated that a group photo in the entrance ceremony has been taken, that is, there was a kindergarten entrance. In this case, the classification unit 405 estimates that the photographed image corresponding to the entrance season determined in step S31 is a photographed image photographed in the entrance ceremony (step S37). Then, the recording unit 402 records the shooting year of these shot images as the year of the user's admission.

  Next, when thinking about the last event of kindergarten, for example, it may be possible to start a graduation ceremony two or three years after the year of admission. In order to identify the graduation type shoot image, the classification unit 405 determines whether there is a shoot image corresponding to the graduation season (an image taken in March two or three years after the year of admission). Is determined (step S38). If it is determined in step S38 that there is no photographed image whose photographing date and time corresponds to the graduation season, the classification unit 405 ends the processing in FIG.

  In addition, in the determination in step S38, if there is a captured image whose shooting date / time corresponds to the graduation season, the classification unit 405 has a predetermined number or more of shooting images whose shooting date / time corresponds to the graduation season. That is, it is determined whether or not a plurality of photographed images corresponding to the graduation season are grouped in the same folder (step S39). If it is determined in step S39 that the number of photographed images corresponding to the graduation season is less than the predetermined number, the classification unit 405 ends the process of FIG.

  On the other hand, if it is determined in step S39 that the number of photographed images corresponding to the graduation season is equal to or greater than the predetermined number, the classification unit 405 uses the subject detection unit 403 to indicate that the photographing date corresponds to the graduation season. The face portion of the subject in the captured image is detected (step S40). After the face detection, the classification unit 405 determines whether there is a captured image in which a plurality of face parts are arranged (step S41). If it is determined in step S41 that there is no captured image in which a plurality of face portions are arranged, the classification unit 405 ends the processing in FIG.

  If there is a photographed image in which a plurality of faces are arranged in the determination in step S41, it is estimated that a group photo at a graduation ceremony has been taken, that is, there was a kindergarten graduation. In this case, the classification unit 405 estimates that the captured image corresponding to the graduation season determined in step S38 is a captured image captured in a graduation ceremony (step S42). Then, the recording unit 402 records the shooting year of these shot images as the year of graduation of the user. Thereafter, the classification unit 405 ends the process of FIG.

  FIG. 7 is a flowchart showing the flow of the school determination process. First, the classification unit 405 takes a photographed image corresponding to the entrance season from the photographed date / time of the photographed image (an image photographed in April of the year when the user is considered to have entered elementary school, junior high school, or high school). Is determined (step S51). If it is determined in step S51 that there is no photographed image whose photographing date and time corresponds to the entrance season, the classification unit 405 ends the process of FIG.

  In addition, in the determination of step S51, if there is a captured image whose shooting date / time corresponds to the entrance season, the classification unit 405 determines whether the number of captured images whose shooting date / time corresponds to the entrance season is a predetermined number or more. That is, it is determined whether or not a plurality of photographed images corresponding to the entrance season are grouped in the same folder (step S52). If it is determined in step S52 that the number of photographed images corresponding to the entrance season is less than a predetermined number, the classification unit 405 ends the process of FIG. On the other hand, if it is determined in step S52 that the number of captured images corresponding to the entrance season is equal to or greater than the predetermined number, the classification unit 405 uses the subject detection unit 403 to capture the captured images corresponding to the entrance season. The subject at is detected. Then, the classification unit 405 determines whether or not there is a captured image in which a plurality of subjects are arranged in a captured image whose shooting date and time corresponds to the entrance season (step S53). If it is determined in step S53 that there is no photographed image in which a plurality of subjects are lined up, the grouping unit 405 performs the process of FIG. Terminate.

  If it is determined in step S53 that there is a captured image in which a plurality of subjects are arranged, a group photo in the entrance ceremony has been taken, that is, the classification unit 405 determines in step S51 that entrance has occurred. The photographed image corresponding to the determined entrance season is estimated to be a photographed image photographed in the entrance ceremony (step S54). Then, the recording unit 402 records the shooting year of these shot images as the entrance year.

  Next, a graduation ceremony image is determined. The classification unit 405 determines whether or not a photographed image corresponding to the graduation season (an image photographed in March three or six years after the entrance year) exists (step S55). If it is determined in step S55 that there is no photographed image whose photographing date and time corresponds to the graduation season, the classification unit 405 ends the process of FIG.

  In addition, in the determination of step S55, if there is a captured image whose shooting date / time corresponds to the graduation season, the classification unit 405 determines whether the number of captured images whose shooting date / time corresponds to the graduation season is a predetermined number or more. That is, it is determined whether or not a plurality of photographed images corresponding to the graduation season are collected in the same folder (step S56). If it is determined in step S56 that the number of captured images corresponding to the graduation season is less than a predetermined number, the classification unit 405 ends the processing in FIG. On the other hand, if it is determined in step S56 that the number of photographed images corresponding to the graduation season is equal to or greater than the predetermined number, the classification unit 405 causes the subject detection unit 403 to capture the photographed images corresponding to the graduation season. The subject at is detected. Then, the classification unit 405 determines whether or not there is a captured image in which a plurality of subjects are arranged in a captured image whose shooting date corresponds to the graduation season (step S57). If it is determined in step S57 that there is no captured image in which a plurality of subjects are arranged, the classification unit 405 ends the process of FIG.

  In the determination in step S57, if there is a captured image in which a plurality of subjects are arranged, it is estimated that a group photo at the graduation ceremony has been captured. Next, the classification unit 405 determines whether or not the shooting year of those shot images is three years after the year of entrance (step S58). If it is determined in step S58 that the shooting year is not three years after the entrance year, it is estimated that the shot image is an image of a graduation ceremony at an elementary school (step S59). Then, the recording unit 402 records the shooting year of the shot image corresponding to the entrance season determined in step S51 as the entrance year of the elementary school, and the shooting year of the shot image corresponding to the graduation season determined in step S55. The recording unit 402 records the year of graduation from the elementary school.

  If it is determined in step S58 that the shooting year is three years after the year of admission, those shot images are taken at a graduation ceremony for junior high school or high school. By subsequent processing, it is determined whether it is a junior high school graduation ceremony or a high school graduation ceremony. First, the classification unit 405 determines whether or not the graduation ceremony image immediately before the photographed image corresponding to the graduation season determined in step S55 is a graduation ceremony for private clothes, that is, a graduation ceremony image for an elementary school. (Step S60). In this determination, for example, it is determined whether or not a plurality of subjects are wearing different clothes in the previous graduation type photographed image. In the determination in step S60, if the previous graduation ceremony is a graduation ceremony for private clothes, the classification unit 405 captures a photographed image corresponding to the graduation ceremony season determined in step S55 as an image of a graduation ceremony for junior high school. Is estimated (step S61). Then, the recording unit 402 records the shooting year of the shot image corresponding to the entrance season determined in step S51 as the entrance year of the junior high school, and the shooting year of the shot image corresponding to the graduation season determined in step S55. Let the recording unit 402 record the year of graduation from junior high school.

  In addition, when the previous graduation ceremony is not a private graduation ceremony, the classification unit 405 determines the height of the subject in the captured image corresponding to the entrance season determined in step S51 and the graduation season determined in step S55. The height of the subject in the corresponding captured image is compared, and it is determined whether or not the change in height is smaller than a predetermined amount (step S62). If the change in height is not small in the determination in step S62, the classification unit 405 estimates that the captured image corresponding to the graduation season determined in step S55 is a high school graduation ceremony image (step S63). Then, the recording unit 402 records the shooting year of the shot image corresponding to the entrance season determined in step S51 as the year of high school entrance, and the shooting year of the shot image corresponding to the graduation season determined in step S55. The recording unit 402 records the year of graduation from high school. Further, in the determination in step S62, if the change in height is small, it cannot be determined whether it is a junior high school or a high school. In this case, the classification unit 405 estimates that the captured images determined in step S51 and step S55 are images of the school days (junior high school or high school) (step S64).

  After Step S59, Steps S61, S63, and S64, the control unit 401 determines whether or not school determination has been performed on all the captured images that are estimated not to be captured images in the kindergarten period (Step S65). In the determination in step S65, when there is a captured image that has not been determined yet, the classification unit 405 executes the processing after step S51 again. On the other hand, if it is determined in step S65 that all the captured images have been determined, the classification unit 405 ends the determination in FIG.

  FIG. 8 is a flowchart showing the flow of the wedding determination process. First, the classification unit 405 uses the subject detection unit 403 to detect the face portion of the subject in each captured image (step S71). After the face detection is completed, the classification unit 405 determines whether there is a captured image in which two men and women are lined up (step S72). If it is determined in step S72 that there is no photographed image in which two men and women are side by side, the classification unit 405 ends the process in FIG.

  Also, in the determination in step S72, when there is no photographed image in which two men and women are side by side, the classification unit 405 detects the body part of the subject in the photographed image by the subject detection unit 403. Then, the classification unit 405 determines whether or not the subject detected by the subject detection unit 403 is wedding clothes (step S73). This determination can be made, for example, by matching an image of the body part (clothing) of the subject in the photographed image with an image of a predetermined wedding dress.

  If it is determined in step S73 that the subject is wedding attire, it is basically considered to be a wedding photographed image. However, there may be situations where a wedding dress happens to happen at some event. In order to discriminate these, the classification unit 405 determines whether there is a photographed image that includes the same two persons in the photographed date and time after one year of the photographed date and time of the photographed image determined in step S73. It is determined whether or not (step S74). In the determination in step S74, if there is no captured image with the same two people in the captured date and time after one year determined in step S73, the classification unit 405 performs the process of FIG. Terminate.

  In addition, in the determination in step S74, if there is a captured image of the same two people in the captured date and time after one year determined in step S73, the classification unit 405 determines in step S73. The photographed image that has been determined is estimated as the photographed image photographed at the wedding (step S75). Then, the recording date and time of this captured image is recorded in the recording unit 402 as a wedding day. Further, the classification unit 405 causes the recording unit 402 to record the male and female face images in the captured image as a couple image (step S76). Thereafter, the classification unit 405 ends the process of FIG.

  Also, in the determination in step S73, if the subject is not wedding dress, the classification unit 405 is a captured image with a shooting date and time before the shooting date and time of the captured image determined in step S73. It is determined whether or not a photographed image is present (step S77). If it is determined in step S77 that there is no photographed image showing the same two people in the past, the classification unit 405 ends the process of FIG.

  In addition, in the determination in step S77, if there are captured images in which the same two people are present in the past, the classification unit 405 determines whether the number of captured images in which the same two people are captured is equal to or greater than a predetermined number. It is determined whether or not (step S78). If it is determined in step S78 that the number of captured images of the same two people is less than a predetermined number, the classification unit 405 ends the process of FIG.

  On the other hand, if it is determined in step S78 that the number of captured images of the same two people is greater than or equal to a predetermined number, it is assumed that the two people are married. Then, the classification unit 405 causes the recording unit 402 to record the photographing date and time of the photographed image in which the two of them are photographed as a temporary wedding date. Further, the classification unit 405 causes the recording unit 402 to record the face images of the men and women in the captured image. Thereafter, the classification unit 405 ends the process of FIG.

  FIG. 9 is a flowchart showing the book shelf arrangement process in step S3 of FIG. In FIG. 9, the control unit 401 determines whether or not there is a folder storing a photographed image corresponding to the kindergarten period (period from admission to graduation) (step S81). If it is determined in step S81 that there is a folder storing a photographed image corresponding to the kindergarten period, the control unit 401 places the folder storing the photographed image corresponding to the kindergarten period on the shelf image corresponding to “kindergarten”. Arrange (step S82). The bookshelf image corresponding to “Kindergarten” is arranged on the left side of the screen shown in FIG. On the other hand, if it is determined in step S81 that there is no folder storing the captured image corresponding to the kindergarten period, the control unit 401 skips the process in step S82.

  Next, the control unit 401 determines whether or not there is a folder storing a photographed image corresponding to the elementary school period (period from elementary school entrance to graduation) (step S83). If it is determined in step S83 that there is a folder storing a photographed image corresponding to the elementary school period, the control unit 401 places the folder storing the photographed image corresponding to the elementary school period on the shelf image corresponding to “elementary school”. Arrange (step S84). The shelf image corresponding to “elementary school” is arranged on the left side of the screen shown in FIG. 2 and in front of the book image of “kindergarten”. On the other hand, if it is determined in step S83 that there is no folder storing the captured image corresponding to the elementary school period, the control unit 401 skips the process of step S84.

  Next, the control unit 401 determines whether or not there is a folder storing a photographed image corresponding to a junior high school period (period from junior high school entrance to graduation) (step S85). If it is determined in step S85 that there is a folder storing a photographed image corresponding to the junior high school period, the control unit 401 sets the folder storing the photographed image corresponding to the junior high school period to the shelf image corresponding to “junior high school”. It arrange | positions above (step S86). The bookshelf image corresponding to “junior high school” is arranged on the left side of the screen shown in FIG. 2 and in front of the bookshelf image of “elementary school”. On the other hand, if it is determined in step S85 that there is no folder storing the captured image corresponding to the junior high school period, the control unit 401 skips the process of step S86.

  Next, the control unit 401 determines whether or not there is a folder storing a captured image corresponding to a high school period (a period from high school entrance to graduation) (step S87). If it is determined in step S87 that there is a folder storing the captured image corresponding to the high school period, the control unit 401 sets the folder storing the captured image corresponding to the high school period to the shelf image corresponding to “high school”. It arrange | positions above (step S88). The bookshelf image corresponding to “high school” is arranged on the left side of the screen shown in FIG. 2 and in front of the bookshelf image of “junior high school”. On the other hand, if it is determined in step S87 that there is no folder storing the captured image corresponding to the high school period, the control unit 401 skips the process of step S88.

  Here, if it is not possible to determine whether it is a junior high school or a high school in step S64 in FIG. 7 and there is a folder estimated to include a photographed image of the school days, the right shelf on the screen shown in FIG. Place on the image.

  Next, the control unit 401 determines whether or not there is a folder storing a captured image corresponding to a period before the kindergarten period (step S89). If it is determined in step S89 that there is a folder storing a photographed image corresponding to a period before the kindergarten period, the control unit 401 stores a photographed image corresponding to the period before the kindergarten period. In step S90, it is determined whether or not there is a folder in which many photos are stored. Whether or not the photographed image is a yes photograph is determined using pattern matching, for example. If it is determined in step S90 that there is a folder storing many yes photos, the control unit 401 places the folder storing many yes photos on the shelf image corresponding to “baby” ( Step S91). The shelf image corresponding to “baby” is arranged on the left side of the screen shown in FIG. 2 and in front of the book image of “kindergarten”. If it is determined in step S90 that there are no folders storing many yes photos, each folder corresponding to the period before the kindergarten period is arranged on the shelf image corresponding to “others” ( Step S92). The shelf image corresponding to “Others” is arranged on the right side of the screen shown in FIG. As a characteristic of the baby age, the time is roughly divided depending on whether walking or yes, so the determination in step S90 is performed.

  Next, the control unit 401 determines whether or not there is a folder storing a captured image corresponding to a period after the high school period (step S93). If it is determined in step S93 that there is a folder storing a captured image corresponding to a period after the high school period, the control unit 401 stores a captured image corresponding to a period after the high school period. It is determined whether or not a folder corresponding to the wedding exists in the selected folder (step S94). If it is determined in step S94 that there is a folder corresponding to the wedding, the control unit 401 further takes a captured image of the user's family in the folder storing the captured images corresponding to the period after the high school period. It is determined whether or not there is a folder in which is stored (step S95). Details of this family determination will be described later. If it is determined in step S95 that the folder storing the captured images of the user's family exists in the folder storing the captured images corresponding to the period after the high school period, the control unit 401 The folder containing the family's photographed images is placed on the bookshelf image corresponding to “family” (step S96). The bookshelf image corresponding to “family” is arranged at the frontmost position on the left side of the screen shown in FIG.

  Further, when there is no folder storing the photographed image corresponding to the period after the high school period in the determination in step S93, or when there is no folder storing the wedding photographed image in the determination in step S94. In the determination in step S95, if there is no folder storing the captured image of the user's family, the control unit 401 places the remaining folder on the shelf image corresponding to “others” (step S97).

  FIG. 10 is a flowchart showing the determination process in step S95 of FIG. First, the control unit 401 selects one captured image from a folder storing captured images corresponding to a period after the high school period. Then, the control unit 401 determines whether or not the photographed image is a photographed image having a photographing date and time after the wedding (step S101). If it is determined in step S101 that the selected photographed image is not a photographed image after marriage, it is estimated that the selected photographed image is not a family photographed image. In this case, the process proceeds to step S110.

  On the other hand, if it is determined in step S101 that the selected captured image is a captured image having a shooting date and time after the wedding, the control unit 401 uses the subject detection unit 403 to detect the face of the subject in the selected captured image. (Step S102). Next, the control unit 401 determines whether or not at least one of two men and women who are estimated to be married in the wedding determination process is included in the selected captured image (step S103). Note that there may be a case where the faces of two men and women are different from the faces recorded in the recording unit 402 as couple images due to changes in age or the like. Therefore, in step S103, it is desirable to make a determination using facial features that do not change much with age, such as eye spacing and nose position.

  In the determination in step S103, if neither of the two men and women who are estimated to be married in the wedding determination process are captured in the selected captured image, the child of the user is captured in the selected captured image. It is determined whether or not there is (step S104). This determination is performed by matching the image of the subject in the captured image with the image of the child of the user recorded in the recording unit 402 by a process described later. If it is determined in step S104 that the user's child is not shown in the selected photographed image, it is estimated that the selected photographed image is not a family photographed image. In this case, the process proceeds to step S110.

  In the determination in step S103, when the selected photographed image includes at least one of two men and women who are estimated to be married in the wedding determination process, the control unit 401 displays the two men and women. At the same time, it is determined whether or not there are a predetermined number or more of photographed images in which a child is photographed (step S105). In the determination in step S105, when there are a predetermined number or more of the photographed images showing the children together with the two men and women, the control unit 401 determines that the children are married in the wedding determination process. Assuming that there are two children, the child image is recorded in the recording unit 402 (step S106). Here, the recorded child image is used in the determination in step S104. On the other hand, if it is determined in step S105 that the number of photographed images showing children is less than the predetermined number, the control unit 401 skips the process of step S106.

  Next, the control unit 401 determines whether or not only one of the two men and women is shown in the selected captured image (step S107). Since the image of yourself is not the image of the family, for the captured image in which only one of the two men and women is shown, the captured image is the image of the family (user's spouse) or the user's own image. It is necessary to identify and classify it. For this reason, the process after step S107 is performed. When only one of the two men and women is shown in the determination in step S107, the control unit 401 is classified into a person and a student age (for example, high school age) shown in the selected photographed image. It is determined whether or not the person in the photographed image matches (actually, the facial feature matches) (step S108). If it is determined in step S108 that the person shown in the selected photographic image alone and the person shown in the photographic image classified as a student (for example, high school) match, It is determined that the person himself / herself is shown in the photographed image having the photographing date and time thereafter is the user himself / herself. In this case, it is estimated that the selected photographed image is not a family photographed image. Thereafter, the process proceeds to step S110.

  In the determination in step S104, if the selected captured image includes a captured image in which the child of the user is present, or in the determination in step S108, a person and a student who are captured in the selected captured image by one person. When the times do not match, the control unit 401 estimates the captured images having the shooting dates and times after the wedding ceremony as the captured images of the family (step S109).

  Thereafter, the control unit 401 determines whether or not determinations have been made for all photographed images after the wedding date (step S110). In the determination in step S110, if there is a captured image that has not been determined yet, after selecting another captured image, the control unit 401 executes the processing in step S101 and subsequent steps again. On the other hand, when it is determined in step S110 that all the captured images have been determined, the control unit 401 ends the determination in FIG.

  FIG. 11 is a flowchart showing the book shelf creation process. In FIG. 11, the control unit 401 compares the degree of appreciation of the captured image (actually the folder selection frequency) in each of the bookshelf images to determine whether there is a difference in the degree of appreciation of the captured image (step S <b> 111). ). If it is determined in step S111 that there is no difference in viewing frequency, the control unit 401 ends the process of FIG.

  On the other hand, when there is a difference in the viewing frequency in the determination in step S111, the control unit 401 adds another level to the bookshelf image (step S112). Next, the control unit 401 arranges a folder storing captured images with a low appreciation frequency (a folder with a small number of selections) in the lower stage created in step S112 (step S113), and then ends the process of FIG.

  As described above, according to the present embodiment, the photographed images are classified into a plurality of groups based on the photographing date and time of the photographed image and the characteristics of the subject reflected in the photographed image, and stored in a folder for each classified group. Each folder is arranged in a shelf image related to the contents of the folder. Thereby, the user can find out a desired captured image more intuitively.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention. For example, in the above-described embodiment, the bookshelf image is displayed on the display unit of the PC 300, but may be displayed on the display unit 106 of the camera 100.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

1 is a block diagram illustrating a configuration of an image display system including an image display device according to an embodiment of the present invention. It is the figure which showed the example of the screen displayed on the display part of PC by the image display system shown in FIG. It is a flowchart shown about the display control for performing the display of FIG. It is a flowchart shown about a classification process. It is a flowchart shown about an event time determination process. It is a flowchart which shows the flow of a kindergarten determination process. It is a flowchart which shows the flow of a school determination process. It is a flowchart which shows the flow of a wedding determination process. It is a flowchart shown about a bookshelf organization process. It is the flowchart shown about the determination process of the family image. It is a flowchart shown about book shelf creation processing.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Camera, 101 ... Control part, 102 ... Operation input part, 103 ... Imaging part, 104 ... Clock part, 105 ... Recording part, 106 ... Display part, 200 ... Scanner, 300 ... Personal computer (PC), 400 ... Server , 401 ... Control unit, 402 ... Recording unit, 403 ... Subject detection unit, 404 ... Date and time detection unit, 405 ... Classification unit, 406 ... Communication unit, 500 ... Communication line

Claims (5)

A classification unit for classifying the captured images into a plurality of groups based on the shooting date and time attached to each of the plurality of captured images and the event information relating to each of the captured images;
A display unit that displays the group according to the shooting date and time from the back of the bookshelf that virtually represents the depth to the front,
An image display device comprising:   The image display apparatus according to claim 1, wherein the event information is extracted based on characteristics of a subject in the captured image.   The image display device according to claim 1, wherein the display unit displays a name assigned to the feature of the group at a corresponding position on the bookshelf.   The image display device according to claim 1, wherein the display unit further displays the group in a direction different from the depth of the bookshelf according to a frequency with which the captured image is viewed.   The image display apparatus according to claim 1, wherein the display unit displays the groups from the back of the bookshelf toward the front in order of shooting date and time.

Images