JP5582453B2 - Image management apparatus, image management method, and program - Google Patents

Description

  The present invention relates to an image management apparatus, an image management method, and a program. More specifically, the present invention relates to an image management apparatus, an image management method, and a program that can share images captured by a digital camera or the like between users.

  An imaging device such as a digital camera that can record and save a subject image (hereinafter also referred to as a photograph) as electronic data (image file) can take a large number of photographs, and can reproduce and check captured photographs on the spot. In addition to the advantages, it has the advantage of being electronic data, for example, an image file can be attached to a mail and transmitted to a desired partner.

  However, such e-mail transmission is based on the intentional act of the sender (intentions such as “I want to show this photo to Mr. A”). , When the photo will arrive is unknown.

  Therefore, an image sharing service, that is, a service in which photographs taken with a digital camera or the like are uploaded to a server on the network and stored, and the photographs can be browsed in response to a request from each user terminal. Yes.

  For example, the following Patent Document 1 shows an example of such an image sharing service. However, since there is no particular description of image search in this document, for example, a large amount of data stored in a server is stored. There is a drawback that a desired photo cannot be found out of the photos.

  In this regard, for example, in the following Patent Document 2, shooting information such as “shooting position”, “shooting date and time”, and “shooting direction” of these photos is stored in the server together with the photos. A technique for searching for a photo using any one as a keyword is shown.

JP 2009-116500 A Japanese Patent Laid-Open No. 11-259502

  However, the technique described in Patent Document 2 has a problem that it is necessary to input a keyword one by one, which is troublesome and unusable.

  Accordingly, an object of the present invention is to provide an image management apparatus, an image management method, and a program capable of searching for an image without requiring keyword input.

In order to achieve the above object, an image management apparatus according to a first aspect of the present invention includes an image storage unit that stores a plurality of images of a plurality of users , and at least the image storage unit associated with each of the stored images. A shooting information storage unit that stores shooting location information of an image and user information indicating a user of the image, an acquisition unit that acquires a plurality of shooting location information, and a change in the plurality of shooting location information acquired by the acquisition unit. and change line generating means for generating a first variation curve showing a designation means for one of the plurality of user designates a plurality of images stored, among the plurality of images stored in the image storage unit , based from among a plurality of images designated by said designating means, in said shooting location information stored with the generated first change line on the photographing information storage means by the change curve generating means And a search means for searching the image.
An image management method according to a second aspect of the present invention includes an image storage step of storing a plurality of images of a plurality of users , and at least shooting location information of the image and the image in association with each stored image. A shooting information storage step for storing user information indicating a user, an acquisition step for acquiring a plurality of shooting location information, and a change line generation for generating a change line indicating a change in the plurality of shooting location information acquired by the acquisition step. A step of designating a plurality of images accumulated by one of the plurality of users, and the change among a plurality of images designated by the designation step among the plurality of accumulated images A search step for searching for an image based on the change line generated by the line generation step and the stored shooting location information.
A program according to a third aspect of the present invention includes an image storage unit that stores a plurality of images of a plurality of users in a server computer on a network, and at least captures the images in association with each of the stored images. Shooting information storage means for storing location information and user information indicating the user of the image , acquisition means for acquiring a plurality of shooting location information, and a change line indicating changes in the plurality of shooting location information acquired by the acquisition means Change line generating means, designation means for designating a plurality of images accumulated by one of the plurality of users, and a plurality of images designated by the designation means among the plurality of images accumulated in the image accumulation means from an image, an image based on said change line the shooting location information stored in the imaging information storage unit with the generated said change line by generating means Sa To realize the function of as Jiseul search means.

  According to the present invention, it is possible to provide an image management apparatus, an image management method, and a program capable of searching for an image without requiring keyword input.

1 is an overall configuration diagram of an embodiment. It is a block diagram of the server 2 and the terminals 3-5. It is a conceptual diagram of grouping information. It is the figure which looked down at the inside of box 30 from the upper surface. It is a file structure figure of an image. It is a structural diagram of an additional information table. It is a block diagram of the imaging devices 6-8. It is an operation | movement flowchart of the terminals 3-5. It is a figure which shows the designation | designated screen of the reference image for grouping. It is an operation | movement flowchart of the server 2. FIG. It is a figure which shows the operation | movement flow of an image reception process. It is a figure which shows the operation | movement flow of a keyword search process. It is a figure which shows the operation | movement flow of a grouping search process. It is explanatory drawing of grouping search.

Embodiments of the present invention will be described below with reference to the drawings.
First, the configuration will be described.
FIG. 1 is an overall configuration diagram of the embodiment. In this figure, a server 2 and terminals 3 to 5 of users 9 to 11 are connected to a network 1 such as the Internet. Here, only three terminals 3 to 5 are shown, but this is for convenience of explanation. Actually, there are a large number of units, and at least two units may be used.

  The server 2 accumulates image files arbitrarily uploaded from the respective terminals 3 to 5 (hereinafter sometimes simply referred to as images), and images accumulated in response to requests from the respective terminals 3 to 5 This is a so-called image storage server that permits browsing of the image. Note that “browsing” generally refers to an act of displaying an image on the screen of the terminals 3 to 5 and appreciating the image, but the embodiment is not limited to such a narrow interpretation. It refers to all actions that are downloaded to the requesting terminals 3 to 5 and “used”. That is, browsing is only one form of “use”, and includes, for example, an act of printing or transferring to another.

  Imaging devices 6 to 8 such as digital cameras can be appropriately connected to the terminals 3 to 5. For example, when the user 9 owns the terminal 3 and the imaging device 6, the user 10 owns the terminal 4 and the imaging device 7, and the user 11 owns the terminal 5 and the imaging device 8, the user 9 needs to Accordingly, the terminal 3 and the imaging device 6 can be connected and an image stored in the imaging device 6 can be taken into the terminal 3. Similarly, the user 10 can connect the terminal 4 and the imaging device 7 as necessary, and can capture an image stored in the imaging device 7 into the terminal 4, and the user 11 can input the terminal 5 as needed. And the image pickup device 8 can be connected and an image stored in the image pickup device 8 can be taken into the terminal 5.

  The “connection” between the terminals 3 to 5 and the imaging devices 6 to 8 may be either a cable connection or a wireless connection. In short, it is only necessary that an image stored in the imaging device 8 can be taken into the terminal 5. Alternatively, for example, when the storage medium (storage medium such as a flash memory) of the imaging devices 6 to 8 is detachable, the storage medium is removed from the imaging devices 6 to 8 and the storage medium is connected to the terminals 3 to 5 (media slots). Etc.) may be included in the interpretation of the above “connection”.

The users 9 to 11 can upload images captured in the terminals 3 to 5 to the server 2 as necessary. As a result, the server 2 can be used as an external image storage location, so that it is possible to prepare for troubles (disappearance of captured images) of the terminals 3 to 5, and to display images to other users on the network 1. It can also be made public.
Note that the terminals 3 to 5 are not necessarily owned by the users 9 to 11 themselves. It may be the property of another user, or may be a network terminal provided in a convenience store or an Internet cafe.
Alternatively, when a communication function can be added to the imaging devices 6 to 8, for example, when a communication card or the like is attached and the network can be accessed via the communication card, the users 9 to 11 do not go through the terminals 3 to 5. In addition, it is also possible to transmit an image directly from the imaging devices 6 to 8 to the server 2.
In any case, what is essential is that it is possible to identify which user has transmitted the image when the server 2 receives the image.
For this purpose, for example, information for identifying the user may be added to each image and transmitted, or transmission of a plurality of images between the server 2 and the terminals 3 to 5 or the imaging devices 6 to 8 is started. Information for identifying the user may be transmitted only once before and after.
The information for identifying the user may be a network address of the image transmission source or device identification information.
Further, when one user owns a plurality of cameras (imaging devices 6 to 8), for example, images taken by the same user with images taken by the respective cameras (imaging devices 6 to 8). Or may be determined as an image taken by another user.

  As described above, the server 2 is highly convenient. However, since the amount of stored images becomes enormous as the number of users (users) increases, it is necessary to devise an image search. A general “image storage server” can perform a keyword search as described in Patent Document 2 at the beginning. This keyword search is for searching additional information of an image. For example, in Patent Document 2 at the beginning, the shooting information such as “shooting position”, “shooting date” or “shooting direction” of an image is used as a keyword. I have to. Further, since the registrant information of the image is usually recorded as additional information, the registrant information can also be used as a keyword.

  Therefore, when searching for a desired image from a large amount of stored information, by using the above keywords (registrant information and shooting information) as appropriate, a search is performed using, for example, an image of a specific registrant. In addition, it is possible to perform a condition search such as an image shot in a specific period and an image shot in a specific area. This search condition can be expressed by a character string expression of [user ID] AND [2010/2/1 to 2010/2/31] AND [Kanto]. Here, [User ID] indicates unique information for designating a registrant, [2010/2/1 to 2010/2/31] indicates a specific period, and [Kanto] indicates a specific area. . “AND” is a conditional logic symbol corresponding to “and”.

  Such search condition formulas are frequently used in the field of databases, and are easy to understand for those who have knowledge in the field, but they are not familiar to those who are not and are simple. Absent. Also, for any person, it is troublesome to manually input such a search condition expression with the keyboard one by one, and if the number of characters is large, an erroneous input cannot be denied.

  Therefore, in this embodiment, in view of the drawbacks of keyword search (difficult and cumbersome operation, and erroneous input cannot be denied), a simple and easy-to-use search technique that does not have such disadvantages is proposed.

  FIG. 2 is a configuration diagram of the server 2 and the terminals 3 to 5. First, as shown to (a), the server 2 is the communication part 20 which exchanges data between the terminals 3-5 via the network 1, the control part 21 comprised with the computer, and its control part 21. A program storage unit 22 that stores a program describing the operation of the program in software, an operation unit 23 such as a keyboard and a mouse, a display unit 24 such as a liquid crystal display, and an image that accumulates images uploaded from the terminals 3 to 5 The storage unit 25 includes an additional information storage unit 26 that stores additional information (details will be described later) of the accumulated image, and a grouping information storage unit 27 that stores predetermined grouping information. In this figure, the image storage unit 25, the additional information storage unit 26, and the grouping information storage unit 27 are drawn as independent storage elements, but the present invention is not limited to this. For example, areas may be created in a storage unit such as a hard disk and the respective areas may be used as the image storage unit 25, the additional information storage unit 26, and the grouping information storage unit 27.

  Further, as shown in (b), the terminals 3 to 5 include a communication unit 60 that exchanges data with the server 2 via the network 1, a control unit 61 configured by a computer, and the control unit 61. Between a program storage unit 62 that stores a program that describes the operation of the program in software, an operation unit 63 such as a keyboard and a mouse, a display unit 64 such as a liquid crystal display, and external devices (such as the imaging devices 6 to 8). Stores various data such as an external interface unit 66 that transmits and receives data and an image that is arbitrarily captured from an external device (such as the imaging devices 6 to 8) or an image that is arbitrarily downloaded from the server 2 via the communication unit 60. And a data storage unit 65.

  Here, the grouping information classifies the images stored in the image storage unit 25 for each registrant (users 9 to 11), and a predetermined three-dimensional map according to the shooting date / time and shooting location information of each image. Plotting upward, connecting the plot points, drawing a change line indicating the change in time and place, and combining the change lines showing the predetermined similarity among the change lines for each registrant. Refers to (grouped) information.

  In short, the grouping information is information for grouping (grouping) images taken by a plurality of users who act together. This is easy to understand when, for example, it is assumed that the user 9 and the user 10 have their own imaging devices 6 and 7 and have traveled in the same process. This is because the user 9 and the user 10 take pictures at various places during the trip, and both pictures are taken at the same time and along the same route.

  By the way, even if a photo of another user 11 who did not travel together has been uploaded, the photo should not apply to the above conditions (same time and same route), so it is excluded from grouping. However, the possibility that a photograph of a user who does not travel together will be mixed in cannot be denied. This is because there is a risk of accidental behavior of “same time and same route”. To avoid this, the number of grouping samples (plot points) may be increased. That is, at the time of grouping, a plurality of photographs having a shooting date range (the range of the shooting date range) or an expansion range of the shooting location (the range of the shooting location range) are determined to be objects of determination. do it.

  FIG. 3 is a conceptual diagram of grouping information. In this figure, the box 30 in the figure has a vertical axis as a time axis (the top of the drawing is the past direction and the bottom is the current direction), and each slice plane of the box 30 (the slice direction is a direction orthogonal to the time axis). ) As the position coordinate plane, and the short side of the position coordinate plane is the Y axis (corresponding to the longitude on the map) and the long side is the X axis (corresponding to the longitude on the map). Therefore, the box 30 is a schematic representation of the “predetermined three-dimensional map”.

  As an example, several lines 31 to 35 are drawn inside the box 30. These lines 31 to 35 correspond to the above-described “change lines”. That is, the lines 31 to 35 classify the images stored in the image storage unit 25 for each registrant (users 9 to 11) and display them on the three-dimensional map according to the information of the shooting date and the shooting location of each image. Plot and connect the plot points.

  FIG. 4 is a view of the inside of the box 30 as viewed from above. In this figure, lines 31 to 35 show their own changes (changes in position and time), but when these changes are observed, similarity is recognized between specific lines. As described above, each of the lines 31 to 35 is obtained by classifying the images stored in the image storage unit 25 for each registrant (users 9 to 11). , Line 32 is the user B, line 33 is the user C, line 34 is the user D, line 35 is the user E, and the shooting location and time of the photos taken by those users are the same symbol (A When plotting in the figure using ~ E), in particular, the similarity that the lines approach or intersect at the plot points surrounded by the broken-line circular figures 36-41 is recognized.

  That is, in this example, the line 32 of the user B and the line 33 of the user C approach or intersect at four plot points (broken lines 36 to 39), and the line 35 of the user E, the user D, and the user D Each line 32-34 of B and user C approaches or intersects at three plot points (dashed circles 38, 40, 41).

  Therefore, from these, the user B and the user C can recognize the fact (first fact) that the user B and the user C existed at the same position at the same time in at least the above four plot points. In the three plot points, the fact that the user D, B, and C existed at the same position at the same time (second fact) can be recognized.

  And from these fact recognition, it can be said that the user B and the user C were acting together with considerable possibility (first evaluation), and the user E is the user D, B, It can be said that C was encountered (second evaluation).

  Of course, the first of these evaluations is not certain. This is because about four encounters often occur in daily life. In order to perform more reliable evaluation, it is only necessary to increase the number of the above-described encounter determination points (locations where lines approach or intersect).

  The above-mentioned “grouping information” refers to the evaluation result, and more specifically, for collecting (grouping) photographs taken appropriately during the action by a plurality of users who were acting together. This information. For example, according to the example of FIG. 4, it can be said that the user B and the user C were acting together with a considerable possibility from the above first evaluation. This information is used for grouping the photographs taken in (1), (B, B, B,..., And C, C, C,... In FIG. 4).

  Therefore, for example, if the user B is the user 9 in FIG. 1 and the user C is also the user 10, the grouping information storage unit 27 of the server 2 is photographed by the users 9 and 10 during the same action. Information for grouping all the photos is stored.

Such grouping processing can be referred to as “action locus matching determination processing based on a plurality of captured images”. This process can be performed, for example, according to the following flow.
(1) First, for a plurality of images photographed by the same imaging device, plotting is performed as a graph showing a change in photographing position with respect to photographing date and time.
(2) Next, continuous approximate curves (see lines 31 to 35 in FIG. 4) passing through the plurality of discretely plotted points (see symbols A to E in FIG. 4) are obtained.
(3) This approximate curve can be obtained by approximation using, for example, a spline function.
(4) Next, an approximate curve is obtained for each image pickup device in the same manner as in the above (1) to (3).
(5) Next, the similarity between the approximate curves is obtained.
(6) This similarity determination is performed as follows, for example.
(6-1) Obtain a difference value between two approximate curves every predetermined time (for example, every hour), calculate a mean square of the obtained plurality of difference values, and calculate the mean square value of two Assume similarity between approximate curves.
(6-2) For all combinations of the two approximate curves, the similarity is calculated by the above process (6-1).
(6-3) Two approximate curves having a similarity equal to or higher than a predetermined value are specified, and it is determined that the owner (user) of the imaging apparatus corresponding to each approximate curve has acted together.
The above processing may be calculated by considering the shooting position as a vector of longitude and latitude, but the approximate curve and the similarity are separately obtained when the shooting position is focused only on longitude and when only the latitude is focused. When the degree of similarity based on longitude and the degree of similarity based on latitude are both equal to or greater than a predetermined degree, the owner of the camera corresponding to each approximate curve should be determined to have acted together Also good.

  FIG. 5 is an image file structure diagram. In this figure, an image 42 is uploaded from any of the terminals 3 to 5 to the server 2 and stored and stored in the image storage unit 25 of the server 2. The file includes the shooting information unit 43 and the image data. It consists of two structural parts of the part 44. The image data part 44 is a part for storing the contents of the image as digital data, and the photographing information part 43 is a part for storing the name (file name, etc.) of the image and photographing information. Here, the shooting information includes information such as the device name (or model code, manufacturer code, etc.) of the imaging device that shot the image, the shooting date and time of the image, and the shooting location of the image. In addition, information such as an image size and an encoding method may be included.

  FIG. 6 is a structural diagram of the additional information table. In this figure, the additional information table 45 is a matrix-structured database table, and each row has a plurality of fields, in this example, an image ID field 46, a registered user ID field 47, a registered date field 48, a shooting information field 49, an image. In addition to a storage location field 49, a grouping information field 50, etc., the shooting information field 49 includes a file name field 52, a shooting device name field 53, a shooting date / time field 54, a shooting location field 55, etc. It consists of subfields.

Here, the image ID field 46 stores unique information (hereinafter referred to as image ID) for uniquely identifying the image stored in the image storage unit 25, and the registered user ID field 47 stores the image. Stores unique information (hereinafter referred to as a registered user ID) for uniquely identifying a registered user.
The registration date field 48 stores the registration date and time of the image (upload date and time of the image), and the shooting information field 49 (the shooting device name field 53, the shooting date and time field 54, and the shooting location field 55) Stores image shooting information (information of the shooting information unit 43 in FIG. 4).
The image storage location field 49 stores link information indicating the storage location of the image (position in the storage unit 25), and the grouping information field 50 stores grouping information about the image (in the grouping information storage unit 27). Link information to (stored information) is stored.

FIG. 7 is a configuration diagram of the imaging devices 6 to 8. As shown in this figure, the imaging devices 6 to 8 include an imaging unit 70 including a semiconductor image sensor such as a charge coupled device image sensor (CCD) and a complementary metal oxide semiconductor (CMOS), and a global positioning system (GPS). Positioning system) a position detection unit 71 that detects a shooting position based on information from a satellite, an operation unit 72 including a shutter button and various function keys, a control unit 73 configured by a computer, and the control unit 73 A program storage unit 74 that stores a program that describes the operation of the program in software, a display unit 75 such as a liquid crystal display, an image storage unit 76 that stores and saves captured images, and an external device (terminals 3 to 5 and the like) An external interface unit 77 that transmits and receives data to and from a power source unit 78 including a battery.
The imaging devices 6 to 8 are digital cameras, for example, but are not limited thereto. Any device including at least these components may be used. For example, a camera-equipped mobile phone may be used.

Next, the operation will be described.
FIG. 8 is an operation flowchart of the terminals 3 to 5. This operation flow shows the main part of the program stored in the program storage unit 62 of the terminals 3 to 5, and this operation flow is, for example, a predetermined key operation or display unit on the keyboard of the terminals 3 to 5 64 is started when a predetermined icon selection operation or the like on 64 is performed.

  When the operation flow is started, first, a predetermined menu is displayed on the display unit 64 (step S11). This menu has four menu items, “1. User registration”, “2. Image upload”, “3. Image search”, and “4. Exit”, and “1” to “4” on the keyboard. A menu item corresponding to the numeric key is selected by selectively operating the numeric key. When “4. End” is selected, the operation flow is ended.

  On the other hand, when “1. user registration” is selected, after executing the user registration process, that is, the process of registering the user of the terminal with respect to the server 2 (step S12), the menu selection screen is displayed again. Return. In this user registration process, for example, a user registration screen (not shown) is displayed to prompt the user to input required user information (information such as a user name and contact information and security information such as a password if necessary). After completing the input, the information may be transmitted to the server 2. In this case, the server 2 may perform new user registration processing according to the information.

  Until the user registration process is completed, menu items other than “1. User registration” and “4. Exit” (“2. Image upload” and “3. Image search”) cannot be used (cannot be selected). It is desirable to leave it in a state.

  Next, when “2. Upload image” is selected, after the image capture (step S13) and the image upload (step S14) are executed, the menu selection screen is displayed again. Note that image capturing typically refers to capturing images stored in the imaging devices 6 to 8 into the terminals 3 to 5, but is not limited thereto. Since the image to be “captured” is for uploading to the server 2 to the last, for example, an image that has already been stored in the data storage unit 65 may be used. Further, the capture destination may be other than the imaging devices 6 to 8 and the data storage unit 65 (for example, other imaging devices such as a mobile phone with a camera or an arbitrary server on the network 1).

  Next, when “3. Image search” is selected, a search mode selection menu is displayed on the display unit 64 (step S15). This menu has three menu items, “1. keyword search”, “2. grouping search”, and “3. return”, and the numeric keys “1” to “3” on the keyboard can be selectively selected. By operating, the menu item corresponding to the numeric key is selected. When “3. RETURN” is selected, the menu selection screen of step S11 is returned again, but when “1. keyword search” or “2. grouping search” is selected, processing corresponding to them is performed. Run.

<Keyword search>
In this search, first, the user is prompted to input a keyword (step S16), and the input keyword is transmitted to the server 2 (step S17). Then, it waits for a response from the server 2 (whether or not there is a keyword search hit) (step S18). If there is no hit, a warning message indicating that fact (for example, “No corresponding image was found”) is displayed. (Step S19), the screen returns to the search mode selection menu screen in Step S15 again.

  On the other hand, if there is a hit, the corresponding image (the keyword searched image) is downloaded from the server 2 (step S20), displayed on the display unit 64 (step S21), and then the search mode selection menu in step S15 again. Return to the screen.

  As the keyword, additional information of the image (information such as a registrant, shooting date and time, shooting location) can be used. For example, if a registrant is a keyword, all images of the registrant can be downloaded. Alternatively, if information such as the shooting date / time or shooting location is used as a keyword, only images having shooting information corresponding to the shooting date / time or shooting location can be downloaded. Alternatively, the shooting direction may be a keyword as in the prior art at the beginning. Moreover, you may use it combining the above keyword. For example, when a plurality of keywords are combined with an AND combination, an image satisfying all of the keywords can be downloaded, or when combined with an OR combination, an image satisfying any one of the keywords can be downloaded.

<Grouping search>
In this search, first, a grouping reference image is designated (step S22). The “grouping reference image” refers to a plurality of images serving as a reference for grouping images. Based on the reference image, the server 2 groups a required image from among a large amount of accumulated images, and transmits it to the terminal that has performed the search.

  FIG. 9 is a diagram showing a grouping reference image designation screen. In this figure, on the grouping reference image designation screen 80, an appropriate screen explanation (in the figure, "Please specify a photo as a grouping reference.") Is displayed, and below the screen explanation. Two command buttons (hereinafter referred to as a first command button 81 and a second command button 82) are provided.

  The title of the first command button 81 is “Upload a reference photo”, and the title of the second command button 82 is “Specify an uploaded photo”.

  “Uploading a reference photo” refers to uploading an image stored in the imaging devices 6 to 8 or an image captured in the data storage unit 65 to the server 2 as a grouping reference image. “Specify an uploaded photo” means that an image that has already been uploaded to the server 2 is designated as a grouping reference image.

  Therefore, when the first command button 81 is selected, image capture (step S25) and image upload (step S26) are executed, and when the second command button 82 is selected, the server The upload date is transmitted to 2 (step S27).

  The “upload date” in step S27 is an arbitrary date input by the user, but this date input can be omitted. When the date input is omitted, the date when the image was most recently uploaded by the user is regarded as the “upload date”.

As described above, in the grouping search, a button operation (the first command button 81, the second command) is used to select “upload a reference photo” or “specify an uploaded photo” as the grouping reference image. The command button 82) can be arbitrarily selected. For example, if a grouping search is to be performed based on an image that has not yet been uploaded, the first command button 81 may be pressed, or the server may already be If a grouping search is to be performed on the basis of the image registered in 2, the second command button 82 may be pressed.
For this reason, a grouping search can be performed promptly after shooting is completed, and a grouping search based on past uploaded images can be performed as desired.

  The server 2 groups and extracts images satisfying a predetermined condition from the images stored in the image storage unit 25 based on the grouping reference image, and retrieves these images as search terminals (any one of the terminals 3 to 5). ). The search terminal waits for a response (whether there is a grouping search hit) from the server 2 (step S23), and if there is no hit, a warning message indicating that fact (for example, “No corresponding image was found”, etc.) Is displayed (step S24), and the screen returns to the search mode selection menu screen of step S15 again.

  On the other hand, if there is a hit, the corresponding image (the image subjected to the cooling search) is downloaded from the server 2 (step S20), displayed on the display unit 64 (step S21), and then the search mode selection in step S15 is performed again. Return to the menu screen.

  FIG. 10 is an operation flowchart of the server 2. This operation flow shows the main part of the program stored in the program storage unit 22 of the server 2, and this operation flow is transmitted from the terminals 3 to 5 while the server 2 is powered on. Whether there is user registration (step S31), whether there is an image upload from terminals 3-5 (step S33), whether there is a keyword search request from terminals 3-5 (step S35), and terminals 3-5 Whether or not there is a grooving search request from (step S37) is repeatedly and continuously executed. When any of the determination results is YES, processing corresponding to the determination is executed.

  That is, when it is determined that there is user registration (YES in step S31), user registration processing is executed (step S32), and when it is determined that there is image reception (YES in step S33), image reception processing is executed ( If it is determined that there is a keyword search (YES in step S35), the keyword search process is executed (step S36). If it is determined that there is a grouping search (YES in step S37), the grouping search process is executed. (Step S38).

  The user registration process in step S32 is a process for registering necessary items (information such as a user name and contact information, and security information such as a password if necessary) in a user database (not shown).

  FIG. 11 is a diagram illustrating an operation flow of image reception processing. As shown in this figure, first, an image uploaded from the terminals 3 to 5 is received (step S41), and it is determined whether or not all images have been received (step S42). If reception of all images has not been completed, the process returns to step S41 again. If reception of all images has been completed, the received image is stored in the image storage unit 25 (step S43), and the additional information storage unit 26 is stored. After updating the information (step S44), the process returns to the operation flow of FIG.

  Here, the information update of the additional information storage unit 26 means that a new record corresponding to the number of received images is added to the additional information table 45 of FIG. 6 and necessary information is stored in each of these records. . Specifically, the image ID is stored in the image ID field 46, the registered user ID is stored in the registered user ID field 47, the upload date / time of the image is stored in the registration date / time field 48, and the photographing device name is stored in the shooting information field 49. Shooting information such as shooting date / time and shooting location, the registration location of each image (position of the image storage unit 25) is stored in the image storage location field 49, and the received information of the received images is stored in the grouping information field 50. This refers to storing link information to grouping information.

  FIG. 12 is a diagram showing an operation flow of the keyword search process. As shown in this figure, first, keywords from the terminals 3 to 5 are received (step S51), and then the additional information table 45 of the additional information storage unit 26 is searched based on the keywords (step S52). Then, it is determined whether or not the information corresponding to the keyword has been hit (step S53). If there is no hit, the terminal is notified (step S54). Is read from the image storage unit 25 and transmitted to the terminal (step S55), and then the operation flow returns to FIG.

  For example, when searching for a registered image of a specific user, the search may be performed using information (user name or the like) indicating the user as a keyword. When the server 2 receives the information (user name, etc.) indicating the user, the server 2 calculates the user ID from the information and searches the additional information table 45 of the additional information storage unit 26 using the user ID. As a result, a record in which the user ID corresponding to the registered user ID field 47 of the additional information table 45 is stored is extracted, and the image storage unit 49 uses the image ID field 46 and the image storage location field 49 of the record as a clue. The corresponding image is read from 25, and the image is transmitted to the terminal. Such a keyword search can be performed using information other than information indicating the user (such as a user name). For example, the keyword search can be performed using information such as a photographing device name, a photographing date and time, or a photographing location. An appropriate combination can also be performed.

  FIG. 13 is a diagram illustrating an operation flow of the grouping search process. As shown in this figure, first, information indicating a grouping reference image is received from the terminals 3 to 5 (step S61), and then the information in the grouping storage unit 27 is searched based on the information (step S62). . Then, it is determined whether or not the corresponding information has been hit (step S63). If there is no hit, the terminal is informed (step S64). On the other hand, if the hit is found, the grouped images are displayed. After reading from the image storage unit 25 and transmitting it to the terminal (step S65), the process returns to the operation flow of FIG.

  Here, the “information indicating the grouping reference image” in step S61 is an image corresponding to “upload a reference photo” specified in FIG. 9 or “uploaded”. This is information indicating whether the image corresponds to “specify a photo”. In the former case (when “Upload a reference photo” is selected), the reference image is uploaded to the server 2 immediately before (see step S26 in FIG. 9), and the image is stored in the image storage unit 25 of the server 2. Since it is registered, the image (the image uploaded immediately before) is set as a grouping reference image. In the latter case (when “Specify uploaded photo” is selected), among the images registered in the image storage unit 25 of the server 2, the registration date designated by the user (step S27 in FIG. 9). Image) is used as a grouping reference image.

  FIG. 14 is an explanatory diagram of grouping search. In this figure, 6 × 5 horizontally long rectangular figures arranged at equal intervals in the vertical and horizontal directions represent registered photographs (images stored in the image storage unit 25), and are described below each figure. The obtained character string illustrates “shooting date and time” and “shooting location”. Further, a two-digit character string consisting of one alphabet and one number described in each figure indicates the user who registered those photos and the registration order. For example, “A1” is 1 by the user A. "A2" is registered second by user A ... "B1" is registered first by user B, "B2" is second registered by user B “E1” indicates the first registered by the user E, and “E2” indicates the second registered by the user E....

  Assume that photographs B1 to B6 are designated as grouping reference images. In other words, it is assumed that the user B is performing a grouping search, and the photos B1 to B6 uploaded by the user B are designated as “grouping reference images”. In this figure, for convenience of explanation, these photographs B1 to B6 are painted black.

  According to the photography information of the photographs B1 to B6, these photographs B1 to B6 were photographed sequentially from “2010/3/1” to “2010/3/6”, and each photographing location is “ It can be seen that they moved while following the Tokaido west such as “Tokyo”, “Odawara”, “Shizuoka”, “Hamamatsu”, “Nagoya”, “Kyoto”.

  On the other hand, the photographs A1 to A6 of the user A were taken around the same time while visiting Hokkaido from “Wakkanai” → “Nayoro” → “Asahikawa” → “Furano” → “Iwamizawa” → “Sapporo”. Photographs D1 to D6 of D are taken at almost the same time on the route of “Fukushima” → “Niigata” → “Toyama” → “Kanazawa” → “Fukui” → “Maizuru”, and further, photos E1 of user E E6 was taken at almost the same time on the route of “Akita” → “Sakata” → “Niigata” → “Nagano” → “Nagoya” → “Ise”, and user C's photos C1 to C6 were “Tokyo”. It can be seen that the images were taken at almost the same time on the route “Odawara” → “Shizuoka” → “Hamamatsu” → “Nagoya” → “Kyoto”.

  Here, a specific place name (“Tokyo” ˜) is shown as a shooting place, but this is for convenience of explanation. The actual shooting location is, for example, latitude and longitude detected by the position detection unit 71 of the imaging devices 3 to 5.

  Now, when comparing the reference image (the photos B1 to B6 of the user B) with the photos of each user, the photos C1 to C6 of the user C almost coincide with the reference image in terms of the movement route and the shooting date and time. On the other hand, there is no great match between the photographs A1 to A6, D1 to D6, and E1 to E6 of the other users (user A, user D, and user E) and the reference image.

  Specifically, the shooting information of the photo B1 of the user B and the shooting information of the photo C1 of the user C are both coincided with “2010/3/1 15:00 Tokyo”, and the shooting information of the photo B2 and the photo C2 is “2010/3”. / 2 15:00 Odawara ”, the shooting information of the same photo B3 and C3 is“ 2010/3/3 15:00 Shizuoka ”, and the shooting information of the same photo B4 and C4 is“ 2010/3 ” / 4 15:00 Hamamatsu ”, the shooting information of the photo B5 and the photo C5 is“ 2010/3/5 15:00 Nagoya ”, and the shooting information of the photo B6 and the photo C6 is“ 2010/3 ”. / 6 15:00 Kyoto ”.

  On the other hand, none of the photos A1 to A6 of the user A match the shooting information of the reference image, and the photos D1 to D6 of the user D also do not match the shooting information of the reference image. Further, in the photos E1 to E6 of the user E, only one of the photos E6 matches the shooting information of a part of the reference image.

From the above, it can be said that the user B who took the reference images (photos B1 to B6) and the user C who took the photos C1 to C6 acted together with considerable possibility.
Moreover, since it is clear that the user A and the user D were in different places from the user B, it can be said that these users A and D are not acting together with the user B.

  Furthermore, user B and user E existed in a specific place (“Nagoya”) at a specific date and time (“2010/3/5 15:00”) based on the shooting information of photographs B5 and E5. The above-mentioned “presence” can be said to have been a coincidence, because the shooting information of photos B1 and E1, B2 and E2, B3 and E3, B4 and E4, and B6 and E6 do not match. .

  Therefore, in the case of the example of FIG. 14, the photos grouped as having a considerable similarity to the reference image (the photos B1 to B6 of the user B) are the photos B1 to B6 of the user B and the photos C1 of the user C. Accordingly, the user B's pictures B1 to B6 and the user C's pictures C1 to C6 are transmitted to the terminal of the user B who has performed the grouping search.

  As described above, in the grouping search of the embodiment, by simply specifying a photograph taken by itself as a reference image for grouping, an image of another user having considerable similarity to the reference image and its reference Images can be downloaded together to the search request terminal. At that time, since it is not necessary to perform troublesome keyword input, the operation is simple, and anyone can easily use it.

  In addition, the above grouping search is performed subsequent to image registration (image upload) to the server 2, or the grouping search is performed by designating an already registered image as a grouping reference image. Since it is possible to perform a keyword search, it is possible to select an appropriate search style according to the situation on the spot.

  In addition, in embodiment, although the terminals 3-5 and the imaging devices 6-8 are set as another structure, it is not limited to this. It may be considered as an integrated device (terminal-capable imaging device). Examples of such an integrated terminal-capable imaging apparatus include a mobile phone with a camera and an information processing terminal with a camera (such as a PDA, a personal computer, or a game terminal).

In the embodiment, in response to a grouping search request from a certain user (for example, user B), the user B is allowed to view the photos obtained by the grouping search. However, the present invention is not limited to this. Not. It may be allowed for other specific users. For example, considering a case in which a photograph of user C is grouped as a result of performing a grouping search in response to a request from user B, in this case, not only user B but also user C may be allowed to browse. This is because the user B and the user C took a picture while acting together, and it is natural to allow all the persons to browse. As a mechanism for allowing all relevant persons to browse, for example, it is conceivable to write the result of grouping search in the additional information table 45 of FIG. That is, a new browsing permission field may be created in the additional information table 45, and a user ID that allows browsing may be written in the browsing permission field.
As described above, the additional information table 45 has the shooting information field 49, and each information of “file name”, “shooting device name”, “shooting date / time”, and “shooting location” is stored in the shooting information field 49. Although stored, it is not limited to this. It is only necessary to store shooting information including at least “shooting date / time”, “shooting location”, and “shooting source” of each image. Here, the “photographing source” is information for clarifying the origin of the image and the requesting source of the grouping search. For example, “photographer”, “image owner”, “image It means any of the information indicating the “imaging imaging device”, “user who requested the search”, and “terminal that requested the search”.

1 network 2 server (image management device)
3 terminal 4 terminal 5 terminal 9 user 10 user 11 user 20 communication unit (image transmission means)
21 Control unit (photographing information retrieval means, image reading means)
25 Image storage unit (image storage means)
26 Additional information storage unit (imaging information storage means)

Claims (9)

Image storage means for storing a plurality of images of a plurality of users ;
Shooting information storage means for storing at least shooting location information of the image and user information indicating a user of the image in association with each accumulated image;
Acquisition means for acquiring a plurality of shooting location information;
Change line generating means for generating a first change line indicating changes in the plurality of shooting location information acquired by the acquiring means;
Designation means for designating a plurality of images accumulated by one of the plurality of users;
Of the plurality of images stored in the image storage means, the first change line generated by the change line generation means and the imaging information storage means from among the plurality of images specified by the specification means Search means for searching for an image based on the shooting location information stored in
An image management apparatus comprising: The change line generation means includes a plurality of images stored in the image storage means for each of the user information stored in the shooting information storage means and a plurality of the shooting locations associated with the user information. The image management apparatus according to claim 1 , wherein a second change line indicating a change in information is generated. The search means includes search means for searching for the second change line similar to the first change line generated by the change line generation means, and a plurality of images stored in the image storage means. The image management apparatus according to claim 2 , wherein a plurality of images belonging to the change line searched by the search unit are searched from inside. Image management apparatus according to any one of claims 1 to 3, further comprising a grouping unit for grouping the search image by the search means. Image management device according to the searched image in any of claims 1 to 4, further comprising a reading means for reading from said image storage means by the search means. The shooting information storage means further includes the shooting date and time of the image,
The change line generating means, an image management device according to any one of claims 1 to 5, characterized in that draw change line shown with a plurality of the photographing location information changes in the shooting date and time. A server including the image storage unit, the photographing information storage unit, the acquisition unit, the change line generation unit, and the search unit; and a terminal connected to the server via a network. The image management apparatus according to any one of claims 1 to 6 . An image storage step for storing a plurality of images of a plurality of users ;
A shooting information storage step for storing at least shooting location information of the image and user information indicating a user of the image in association with each accumulated image;
An acquisition process for acquiring multiple shooting location information;
A change line generation step of generating a change line indicating a change in the plurality of shooting location information acquired by the acquisition step;
A designation step for designating a plurality of images accumulated by one of the plurality of users;
Among the plurality of images stored, an image based on the change line generated by the change line generation step and the stored shooting location information is selected from the plurality of images specified by the specification step. A search process for searching;
An image management method comprising: On the server computer on the network,
Image storage means for storing a plurality of images of a plurality of users ;
Shooting information storage means for storing at least shooting location information of the image and user information indicating a user of the image in association with each of the accumulated images ;
Acquisition means for acquiring a plurality of shooting location information;
Change line generation means for generating a change line indicating a change in the plurality of shooting location information acquired by the acquisition means;
Designation means for designating a plurality of images accumulated by one of the plurality of users;
Of the plurality of images stored in the image storage unit, the change line generated by the change line generation unit and the imaging information storage unit are stored in the plurality of images specified by the specification unit. Search means for searching for an image based on the shooting location information,
A program to realize the functions as

Images