WO2012008511A1 - 画像処理装置及びプログラム - Google Patents

画像処理装置及びプログラム Download PDF

Info

Publication number
WO2012008511A1
WO2012008511A1 PCT/JP2011/066033 JP2011066033W WO2012008511A1 WO 2012008511 A1 WO2012008511 A1 WO 2012008511A1 JP 2011066033 W JP2011066033 W JP 2011066033W WO 2012008511 A1 WO2012008511 A1 WO 2012008511A1
Authority
WO
WIPO (PCT)
Prior art keywords
cluster
information
keyword
image
distribution state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2011/066033
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
尚之 宮下
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Olympus Corp
Original Assignee
Olympus Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Corp filed Critical Olympus Corp
Publication of WO2012008511A1 publication Critical patent/WO2012008511A1/ja
Priority to US13/740,521 priority Critical patent/US8983138B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location

Definitions

  • the present invention relates to an image processing apparatus, a program, and the like.
  • keywords are actively assigned to images for image retrieval.
  • the assigning methods are roughly classified into an approach that extracts image feature amounts from an image, recognizes a subject type by image recognition, and uses it as a keyword, and an approach that uses camera parameters (Exif, GPS, various sensors).
  • Japanese Patent Application Laid-Open No. H10-228561 describes a technique of assigning a keyword representing a place name prepared in advance using position information acquired together with image shooting.
  • a keyword is added using position information or a keyword is used using image recognition, the same keyword is always assigned if the same place or the same object is photographed, so the lifestyle and preferences of the user are not reflected.
  • an image processing device, a program, and the like that perform keyword assignment reflecting the user's lifestyle, preferences, and the like.
  • image processing that automatically assigns a keyword whether a place where a certain photograph is taken is an everyday place or an unusual place for a user Devices and programs can be provided.
  • One aspect of the present invention is an information acquisition unit that acquires a shooting position of a captured image or a position of an imaging device as coordinate information; a distribution state acquisition unit that acquires a distribution state of a plurality of coordinate information acquired as the coordinate information; And a keyword assigning unit that assigns a keyword corresponding to the acquired distribution state to the photographed image.
  • a keyword corresponding to the distribution state can be assigned to the captured image. Therefore, it is possible to assign keywords that reflect user preferences and the like.
  • the distribution state acquisition unit may acquire the distribution state of the coordinate information in a two-dimensional coordinate space representing a geographical position.
  • a geographical position for example, longitude / latitude
  • the information acquisition unit further acquires time information, which is information indicating time when the shooting position of the captured image or the position of the imaging device is acquired, and the distribution
  • the state acquisition unit acquires a distribution state of the coordinate information and the time information in a three-dimensional coordinate space having a third axis representing time in addition to the two-dimensional coordinates representing the geographical position, and assigns the keyword
  • the unit may add a keyword representing information related to time to the captured image.
  • the distribution state acquisition unit may include a cluster classification unit that classifies the coordinate information into a plurality of clusters including at least a first cluster and a second cluster.
  • the cluster classification unit classifies the first cluster and the second cluster into a reference cluster that is a reference cluster and a non-reference cluster that is a non-reference cluster. May be.
  • the cluster classification unit may classify a cluster having the largest number of captured images included in the cluster as the reference cluster among the plurality of clusters.
  • the cluster classification unit may set the reference cluster based on a user selection.
  • the cluster classification unit may set the reference cluster based on a time-series movement history of coordinate information in a two-dimensional coordinate space representing a geographical position.
  • the cluster classification unit may include the complexity of the distribution of coordinates represented by the coordinate information in the first cluster and the complexity of the distribution of coordinates represented by the coordinate information in the second cluster.
  • the reference cluster may be set based on the degree.
  • a map presenting unit that performs processing for presenting thumbnails of a plurality of images acquired as the photographed images on a map
  • the map presenting unit includes a distance between geographical photographing positions. For an image group smaller than a given threshold, one representative image included in the image group is presented as a thumbnail on the map, and in the case of a non-reference cluster, You may perform the process which sets the value of a threshold value large.
  • images that are widely distributed in the non-reference cluster are represented by one (or a small number) of thumbnails, and it is possible to appeal to the user sensuously that a series of images are a group of events. it can.
  • the keyword assigning unit adds a place name corresponding to a photographing position as the keyword to a plurality of images acquired as the photographed image.
  • a place name representing a narrow area may be given as compared with the captured image of the non-reference cluster.
  • the information acquisition unit further acquires time information, which is information indicating time when the shooting position of the captured image or the position of the imaging device is acquired, and the distribution
  • the state acquisition unit acquires a distribution state of the coordinate information and the time information photographing in a three-dimensional coordinate space having a third axis representing time in addition to the two-dimensional coordinates representing the geographical position, and the keyword
  • the assigning unit may switch a threshold for determining a temporal distance according to whether the reference cluster or the non-reference cluster is used when assigning a keyword representing information related to time to the captured image.
  • the information acquisition unit further acquires time information, which is information indicating time when the shooting position of the captured image or the position of the imaging device is acquired, and the distribution
  • the state acquisition unit acquires the coordinate information and the distribution state of the time information in a three-dimensional coordinate space having a third axis representing time in addition to the two-dimensional coordinates representing the geographical position, and the cluster classification
  • the unit may change the cluster set as the non-reference cluster to the reference cluster when a continuous shooting time exceeds a given threshold in the image group included in the non-reference cluster. Good.
  • the distribution state acquisition unit includes a distance information acquisition unit that acquires distance information between the plurality of clusters, and the keyword assigning unit adds the distance information acquisition unit to the captured image based on the distance information.
  • a keyword may be assigned.
  • the distribution state acquisition unit includes a distance information acquisition unit that acquires distance information between the plurality of clusters
  • the keyword assignment unit includes the reference cluster, the previous non-reference cluster, When the distance represented by the distance information between the two is large, a keyword indicating that the distance in the coordinate space is separated may be assigned to the captured image included in the non-reference cluster.
  • the distribution state acquisition unit includes a cluster classification unit that classifies the coordinate information into a plurality of clusters including at least a first cluster and a second cluster, and the cluster classification unit includes: One or more of the plurality of clusters may be classified as reference clusters, and the other clusters may be classified as non-reference clusters.
  • the distribution state acquisition unit includes a cluster classification unit that classifies the coordinate information into a plurality of clusters including at least a first cluster and a second cluster
  • the cluster classification unit includes: , Obtaining a representative position of each cluster of the plurality of clusters, regarding each cluster as a point corresponding to the representative position, and performing a clustering process again, so that the plurality of clusters are referred to as the reference cluster and the cluster You may classify into a non-reference
  • classification can be performed by a simple method called cluster reclustering processing.
  • the information acquisition unit may acquire the coordinate information based on a captured image captured by the same user.
  • the keyword assigning unit may perform a process of updating the keyword by assigning the keyword again to the captured image to which the keyword has been assigned.
  • an information acquisition unit that acquires, as coordinate information, a shooting position of a captured image or a position of an imaging device, and a distribution state that acquires a distribution state of a plurality of coordinate information acquired as the coordinate information
  • the present invention relates to a program that causes a computer to function as an acquisition unit and a keyword adding unit that adds a keyword corresponding to the acquired distribution state to the captured image.
  • FIG. 1 is a configuration example of an image processing apparatus according to the present embodiment.
  • FIG. 2 is a flowchart for explaining processing of the first embodiment.
  • 3A to 3C show examples of distribution of coordinate information.
  • FIG. 4 is a diagram for explaining a distance between clusters.
  • FIG. 5 is an example of screen data for displaying thumbnails on a map.
  • FIG. 6 is a data configuration example of image data, coordinate information, and keywords.
  • FIG. 7 is a diagram for explaining reference cluster determination processing based on time-series movement history of coordinate information.
  • 8A shows an example in which the complexity of the distribution of coordinate information is high
  • FIG. 8B shows an example in which the complexity of the distribution of coordinate information is low.
  • FIG. 9 is a flowchart for explaining processing of the second embodiment.
  • FIG. 10A and 10B show examples of screen data displayed when a reference cluster is determined by user selection.
  • FIG. 11 shows an example of a three-dimensional coordinate space composed of longitude, latitude, and time.
  • FIG. 12 shows an example of classification into three or more clusters.
  • FIG. 13 is a flowchart for explaining processing of the fourth embodiment.
  • the process which gives is shown.
  • a process in which a keyword “travel” is automatically given to a photograph when a user travels far away while taking a photograph on a daily basis is shown.
  • FIG. 1 shows a configuration diagram of an image processing apparatus according to this embodiment.
  • the image processing apparatus includes a processing unit 100, an input unit 200, a storage unit 300, and a display unit 400.
  • the processing unit 100, the input unit 200, the storage unit 300, and the display unit 400 are connected to each other.
  • the processing unit 100 performs various processes based on input information from the input unit 200 and information stored in the storage unit 300.
  • the function of the processing unit 100 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), a program, and the like.
  • the input unit 200 is used by the user to input various operation information of the image processing apparatus, and can be realized by various buttons, a GUI, and the like.
  • the storage unit 300 serves as a work area for the processing unit 100 and the like, and its function can be realized by a memory such as a RAM or an HDD (hard disk drive).
  • a memory such as a RAM or an HDD (hard disk drive).
  • the display unit 400 is for displaying various display screens, and can be realized by, for example, a liquid crystal display or an organic EL display.
  • the processing unit 100 includes an information acquisition unit 110, a distribution state acquisition unit 120, a keyword assignment unit 130, and a map presentation unit 150.
  • the information acquisition unit 110 acquires coordinate information. Specifically, for example, two-dimensional coordinate information (for example, X axis Y axis or latitude / longitude) representing a geographical position is acquired.
  • the coordinate information is acquired at the timing when the image is acquired, but is not limited thereto.
  • the coordinate information may be acquired even at the timing when the image is not acquired.
  • longitude / latitude information acquired by GPS or the like is acquired.
  • the coordinate information acquisition means is not limited to GPS.
  • the coordinate information may be acquired using a known method such as PlaceEngine. Further, not only longitude / latitude but also information related to altitude may be acquired.
  • the distribution state acquisition unit 120 acquires the distribution state of the coordinate information. Specifically, as described in the cluster classification unit 125 and the distance information acquisition unit 126, a plurality of coordinate information is clustered into two clusters, and a distribution state is acquired by obtaining distance information between the two clusters. .
  • the distribution state acquisition unit 120 includes a cluster classification unit 125 and a distance information acquisition unit 126.
  • the cluster classification unit 125 clusters a plurality of coordinate information. In the present embodiment, the first and second clusters are classified, and one is set as a reference cluster, and the other is set as a non-reference cluster.
  • the distance information acquisition unit 126 acquires an intercluster distance representing a distance between the reference cluster and the non-reference cluster.
  • the keyword assigning unit 130 assigns a keyword as metadata to the acquired image.
  • the assigned keyword is a keyword based on a geographical position, and may be one that reflects the user's environment and preferences. Further, as is conventional, a keyword that does not take into account the user's preference or the like (for example, a place name such as “Tokyo”) may be assigned.
  • the map presentation unit 150 displays images as thumbnails on a two-dimensional map. Specifically, an image as shown in FIG. 5 to be described later is generated and presented to the user.
  • coordinate information is acquired by the information acquisition unit 110 (S501).
  • longitude / latitude information is acquired using the functions of GPS and WiFi.
  • the cluster classification unit 125 of the distribution state acquisition unit 120 performs the clustering process on the vector group using the acquired coordinate information (longitude / latitude information) as a vector group in the two-dimensional coordinate space, and classifies the cluster into cluster 1 and cluster 2. (S502).
  • each cluster includes at least one coordinate information.
  • various general methods such as k-means can be used. Through the processing so far, FIG. 3A to FIG. 3B are acquired.
  • the cluster classification unit 125 classifies cluster 1 and cluster 2 into a reference cluster and a non-reference cluster (S503).
  • the number of images included in each of the cluster 1 and the cluster 2 is compared, and the cluster having the larger number of images is set as a reference cluster. Therefore, in FIG. 3C, the first cluster is a reference cluster, and the second cluster is a non-reference cluster.
  • the distance information acquisition unit 126 calculates the inter-cluster distance between the reference cluster and the non-reference cluster as shown in FIG. 4 (S504).
  • the Euclidean distance between the cluster centroids (average coordinates of the shooting position group of each cluster) is calculated using the following equations (1) to (3), but the present invention is not limited to this.
  • the Euclidean distance between the shooting position coordinates closest to each other between the clusters may be used, or the distance average of all the shooting position coordinates between the clusters may be used.
  • the Mahalanobis distance distance from the point to the distribution
  • the Mahalanobis distance from the centroid coordinate of the reference cluster to the non-reference cluster may be used.
  • Various general processes for calculating the distance may be used.
  • the distance information acquisition unit 126 may calculate the distance between the coordinate vector of each image included in each cluster and the opposite cluster, instead of the distance between clusters, and use it for keyword assignment described later.
  • the keyword assigning unit assigns a keyword to the image included in each cluster and stores it (S505). If the value of the distance between clusters is large, a keyword expressing that the geographical distance such as “far” or “travel” is long is assigned to the image included in the non-reference cluster. In addition, a keyword expressing that the geographical distance such as “near” or “neighborhood” is close to the image included in the reference cluster may be given.
  • a keyword that indicates that the geographical distance is close such as “close” or “neighborhood” to images included in either or both of the reference cluster and non-reference cluster, is added.
  • the magnitude of the distance is determined by a predetermined threshold value. For example, if a distance of 300 km or more on the map is set as a threshold value, a keyword such as “travel” is assigned to the image group of the non-reference cluster when the non-reference cluster is separated from the reference cluster by 300 km or more.
  • the keyword “travel” is automatically assigned only to a photograph taken in a trip to a distant place from a group of images taken by the user.
  • the image processing apparatus performs the process again at the timing when the shooting position increases, an arbitrary timing by the user, the passage of a certain time, etc., and updates the keyword of the image to which the keyword has been assigned. Therefore, it is possible to assign the latest keywords that reflect changes in user lifestyles and preferences.
  • the map presentation unit 150 displays an image as a thumbnail on the map
  • the processing can be changed depending on the reference cluster or the non-reference cluster.
  • FIG. it is assumed that an image group within a range equal to or less than a given threshold on the map is represented by one thumbnail.
  • the reference cluster is considered to correspond to a range in which the user behaves on a daily basis, there remains a problem in collecting the distributed image group in a large group.
  • Odaiba and Ikebukuro are far away from each other based on the daily life of people living in Tokyo. This is because it is assumed that these are different.
  • the non-reference cluster is an extraordinary place outside the range of action for the user, so it is difficult to assume frequent visits for different purposes. Therefore, in the example of FIG. 5, even if the image group distributed in the vicinity of Tohoku is distributed in the range between Odaiba and Ikebukuro in terms of distance, it is treated as the same “Tohoku trip” and one image. The problem is unlikely to occur even if represented by.
  • the image processing apparatus includes an information acquisition unit 110 that acquires coordinate information, a distribution state acquisition unit 120 that acquires a distribution state of a plurality of acquired coordinate information, And a keyword assigning unit 130 for assigning a keyword corresponding to the distribution state to the photographed image.
  • the coordinate information is information representing the position when the captured image is captured by the image capturing apparatus or the position of the image capturing apparatus acquired at an arbitrary timing regardless of the capturing of the image.
  • a keyword can be assigned to each captured image according to the distribution of coordinate information (for example, FIGS. 3A to 3C).
  • the distribution state is acquired, for example, in the example of FIGS. 3A to 3C, the shooting is mainly performed in Tokyo, and although it is a small number, the shooting is also performed in Tohoku. Information can be obtained.
  • the keywords that can be assigned are not limited to keywords such as “Tokyo” and “Tohoku” that do not change for any user. It is possible to assign keywords that reflect the user's place of residence, action range, and preferences, such as “close”, “neighborhood”, and “normal” to images in Tokyo, and “far”, “travel”, and “rare” in Tohoku images.
  • the image processing apparatus assigns a keyword to the image. Search and display can be performed efficiently by adding metadata to images.
  • the coordinate information is acquired at a timing other than the timing at which the image is captured, and therefore a keyword may be assigned to the coordinate information at that time.
  • the data structure is as shown in FIG. Normally, as shown by A1 in FIG. 6, coordinate information is associated with the image data as metadata, and a keyword is associated with the metadata as metadata.
  • a keyword may be associated with coordinate information (image data is NULL) acquired at a timing when an image is not captured. This is useful because a keyword can be used as metadata of coordinate information when there is a case where data processing is performed using coordinate information in some form.
  • the distribution state acquisition unit 120 acquires the distribution state of the coordinate information in the two-dimensional coordinate space representing the geographical position.
  • the geographical position can be acquired by using a function of GPS or WiFi, for example.
  • the distribution state acquisition unit 120 may include a cluster classification unit 125 that classifies coordinate information into a plurality of clusters including at least a first cluster and a second cluster.
  • a plurality of coordinate information can be classified into a plurality of clusters by the clustering process.
  • the first cluster and the second cluster are classified into two, but as in the fourth embodiment to be described later, there are three or more clusters.
  • the cluster classification unit 125 divides the first cluster and the second cluster into a reference cluster that is a reference cluster and a non-reference cluster that is a non-reference cluster, as shown in FIG. Classify.
  • the coordinate information classified into a plurality of clusters can be further classified from the viewpoint of whether it belongs to the reference.
  • classifying as standard / non-standard as mentioned above, it is possible to determine whether the area is a daily range or a non-daily range for the user, taking into account the user's preference etc. Keyword assignment.
  • the cluster classification unit 125 may classify a cluster having the largest number of captured images among a plurality of clusters as a reference cluster.
  • the cluster classification unit 125 may set a reference cluster based on a time-series movement history of coordinate information in a two-dimensional coordinate space representing a geographical position.
  • the time-series movement history of the coordinate information may specifically be, for example, one in which the coordinate information is connected by a straight line in time-series order.
  • the user's behavior it is assumed that the user moves from the home to the travel destination and returns to the home. That is, if the case of moving from destination to destination one after another is excluded from the assumption, the user's home is considered to be the center of a radial figure (in other words, “hub”) in FIG. . Therefore, a figure as shown in FIG. 7 may be created and a cluster serving as a hub may be set as a reference cluster.
  • a method of counting the number of times of entering (or the number of times of getting out or the sum of them) and making the most cluster as a reference cluster can be considered.
  • the cluster classification unit 125 may set a reference cluster based on the complexity of the distribution of coordinate information in the first cluster and the complexity of the distribution of coordinate information in the second cluster.
  • the complexity of the distribution of coordinate information becomes high when the coordinate information is distributed widely in a wide range as shown in FIG. 8A, and becomes narrow in a narrow range as shown in FIG. 8B. This information is low when it is dense or distributed on a straight line. An example of a specific calculation method will be described later.
  • the complexity calculation method is simply, for example, when each coordinate information is connected in time series, the sum of the absolute values of the line segment connecting the previous point and the current point, or the length of the line segment A method using the total sum of parameters as a parameter can be considered. If the sum of the absolute values of the angles is small, a linear movement is assumed as shown in FIG. 8B, and if the sum of the lengths of the line segments is small, it is assumed that the lines are densely packed in a narrow range. Both cases are classified as non-reference clusters.
  • the image processing apparatus may include a map presenting unit 150 that displays thumbnails of a plurality of images acquired as photographed images on a map.
  • the map presenting unit 150 displays one representative image included in the image group as a thumbnail for an image group in which the distance between geographical shooting positions is smaller than a given threshold.
  • the threshold value is set larger than that in the reference cluster.
  • FIG. 5 an image as shown in FIG. 5 can be presented to the user.
  • a reference cluster is set in the vicinity of Tokyo and a non-reference cluster is set in Tohoku as shown in FIG.
  • a method of representing a group of images within a given range with one thumbnail is used, and the threshold value at that time is made larger in the non-reference cluster than in the reference cluster. That is, a wide range of images is represented by one thumbnail in the non-reference cluster, but the range represented by one thumbnail is narrower than the non-reference cluster in the reference cluster. This is because, as described above, the reference cluster is a range in which daily action is performed, and thus it is considered that there are a plurality of points as points in a narrow range.
  • the keyword assigning unit 130 assigns a place name corresponding to the photographing position to the photographed image as a keyword, the place name representing a narrow area for the reference cluster image compared to the non-reference cluster image. May be given.
  • the place name representing a narrow area means, for example, that the area of the area indicated by the place name is narrower than the comparison target.
  • “Tokyo” for “Japan”, “MSNCTYSTAddressList13: Shibuya-ku, Tokyo; Address Shibuya-ku, Tokyo”, “MSNCTYSTAddressList13: Shibuya-ku, Tokyo; Shibuya-ku, Address” “Shibuya”, “Ebisu”, etc. This is almost the same as the classification of country, prefecture, city, street, etc. when deciding the address, but of course, an area name that is not used for the address may be used.
  • the place name representing a wide area and the place name representing a narrow area are in an inclusive relationship.
  • the present invention is not necessarily limited to this.
  • “Tohoku” representing a region and “Tokyo” representing a prefecture “Tokyo” represents a narrower region.
  • the distribution state acquisition unit 120 includes a distance information acquisition unit 126, and the keyword assignment unit 130 assigns a keyword to the captured image based on the distance information. Specifically, for example, when the distance between the reference cluster and the non-reference cluster is large, a keyword (for example, “far” or “travel” indicates that the distance is far from the captured image included in the non-reference cluster. ”,“ Exception ”,“ Rare ”, etc.).
  • a keyword for example, “far” or “travel” indicates that the distance is far from the captured image included in the non-reference cluster. ”,“ Exception ”,“ Rare ”, etc.
  • the threshold value of the distance information indicating that the non-reference cluster is far from the reference cluster can be freely determined. For example, a child with a narrow range of action is considered to be in an extraordinary environment even when moving about several tens of kilometers. On the other hand, an active adult may not recognize an unusual environment unless he / she travels several hundred kilometers or more. That is, the keyword should be given based on the distance information between the clusters, and it can be said that it is desirable that the threshold value for the determination is determined for each user.
  • the information acquisition unit 110 acquires coordinate information based on images captured by the same user.
  • the keyword assigning unit 130 may perform a process of updating the keyword by assigning the keyword again to the captured image to which the keyword has been assigned.
  • the distribution state of the coordinate information may be updated by the distribution state acquisition unit 120.
  • the reference cluster is updated in accordance with a case where the daily activity range is changed due to moving or the expansion of the activity range, and the keyword is updated accordingly.
  • the present embodiment described above also relates to a program that causes a computer to function as the information acquisition unit 110, the distribution state acquisition unit 120, and the keyword assignment unit 130.
  • the image processing apparatus can be realized by software without being limited to hardware means.
  • the image processing apparatus is not built in the image capturing apparatus, but the image capturing apparatus is used alone, and only the data obtained by the image capturing apparatus is input to an information processing apparatus such as a PC, and the image is programmed. It is conceivable to perform processing.
  • the program is recorded on an information storage medium.
  • various recording media that can be read by the information processing apparatus such as an optical disk such as a DVD or a CD, a magneto-optical disk, a hard disk (HDD), a memory such as a nonvolatile memory or a RAM, can be assumed.
  • an example is shown in which a user selects a reference cluster and assigns a keyword based on the selected cluster.
  • an arbitrary range preferred by the user can be used as the reference cluster, and although the time and effort of the user operation is increased, it is possible to assign keywords according to the preference.
  • FIG. 1 shows a configuration diagram of an image processing apparatus according to this embodiment.
  • the configuration of each part is the same as that of the first embodiment.
  • the present embodiment is different in that the cluster classification unit 125 sets the reference cluster and the non-reference cluster according to the input from the user.
  • the cluster classification unit 125 applies the information display device such as a display provided in the display unit to FIG.
  • the map information and the shooting position as shown in (A) are displayed, and the user selects a range on the map as a reference cluster by using a UI such as a mouse provided in the input unit (S602).
  • the selection range may be expanded in a circular shape by the user dragging the pointer, or a region surrounded by free hands may be set as the selection range.
  • Information regarding an address may be input, and a given range may be selected as a reference cluster around the input address. Further, although coordinate information is displayed in FIG. 10A, it may be selected from a complete white map.
  • the first cluster and the second cluster are set from the distribution of coordinate information (S603).
  • the cluster classification unit 125 displays a screen as shown in FIG. 10B, and selects which of the first cluster and the second cluster is the reference cluster (S604).
  • the acquisition of the inter-cluster distance by the distance information acquisition unit 126 (S605) is the same as in the first embodiment. Then, a keyword is assigned by the keyword assigning unit 130 according to the distance between clusters (S606).
  • the keyword assignment process is basically the same as that of the first embodiment, but the user may be allowed to select a threshold value for determining the distance.
  • keywords are assigned based on the range selected by the user, and the keywords are assigned according to personal preference.
  • the cluster classification unit 125 sets a reference cluster based on the user's selection, as shown in FIGS. 10 (A) and 10 (B).
  • a reference cluster that better reflects user preferences.
  • a range is selected or an address is input, or after clustering as shown in FIG. A method of selecting whether to use a reference cluster can be considered.
  • an address may be directly input after clustering, and a cluster closest to the input address may be used as a reference cluster.
  • FIG. 1 The block diagram is shown in FIG. 1, and the details of the processing are shown in FIG. As is clear from FIGS. 1 and 2, the outline of the configuration and processing is the same as that of the first embodiment. Hereinafter, different points will be described with reference to FIG.
  • coordinate information is acquired.
  • the information acquisition unit 110 acquires geographical shooting coordinate information of the image group. Specifically, longitude / latitude information acquired by GPS or the like is acquired as coordinate information. In the present embodiment, in addition to that, the time when the image was taken is acquired. Coordinate information is obtained as coordinates between a two-dimensional coordinate space of longitude and latitude and a one-dimensional coordinate space consisting of time (S501).
  • cluster 1 and cluster 2 are set based on the coordinate information.
  • geographical two-dimensional coordinate information is used, and one-dimensional coordinate information consisting of time is not used (S502).
  • S503 for classifying the reference cluster and the non-reference cluster is the same as in the first embodiment. Then, the distance between clusters is calculated (S504). Similar to the first embodiment, the distance in the two-dimensional coordinates of longitude and latitude is calculated. In addition, the distance in the one-dimensional coordinate space of the time axis is also calculated.
  • the keyword assigning unit 130 assigns a keyword to an image included in each cluster according to the inter-cluster distance and stores it. If the distance on the time axis is large, keywords such as “old” and “recent” if small are added. In addition, the distance in the two-dimensional coordinate space of longitude and latitude and the distance of the time axis are determined, respectively, and the position information such as “old” and “travel”, “recent” and “neighborhood” and the time axis are considered in consideration May be.
  • each shooting coordinate is obtained as a coordinate in a three-dimensional coordinate space composed of longitude, latitude, and time.
  • vectors in the three-dimensional coordinate space are also used.
  • the distance information acquisition unit 126 acquires distance information
  • a three-dimensional coordinate space is used.
  • the keyword assigning unit 130 assigns a keyword that represents the distance between the geography and the time axis, such as “far” if the distance in the three-dimensional coordinate space is large and “close” if the distance is small.
  • the image processing apparatus performs this processing again at the timing when the shooting position increases, an arbitrary timing by the user, a certain period of time, etc.
  • the image processing apparatus performs this processing again at the timing when the shooting position increases, an arbitrary timing by the user, a certain period of time, etc.
  • the information acquisition unit 110 further acquires time information in addition to the coordinate information. Then, the distribution state acquisition unit 120 acquires the distribution state of the coordinate information and time information in the three-dimensional coordinate space, and the keyword assigning unit 130 assigns a keyword representing information related to time.
  • the time information is information representing the time when information regarding the shooting position of the shot image or the position of the shot image is acquired.
  • keywords including time as well as longitude and latitude as shown in FIG.
  • keywords such as “recent” and “old”
  • keywords such as “recent trip” and “second trip” can be given together with longitude / latitude information.
  • the keyword assigning unit 130 may switch the threshold for determining the temporal distance according to whether the reference cluster or the non-reference cluster is used when assigning a time-related keyword to the captured image.
  • a threshold value that is smaller in the reference cluster than in the non-reference cluster may be set.
  • the keyword “recent” is assigned to the threshold one week ago
  • the keyword “recent” is assigned to the threshold one month ago.
  • the cluster classification unit 125 changes the cluster set as the non-reference cluster to the reference cluster when the continuous shooting time in the image group included in the non-reference cluster exceeds a given threshold. May be.
  • the continuous shooting time is defined as A when an image is shot at a non-reference cluster at a certain time A, and an image is shot at time B, which is the same cluster, without moving to another cluster. And the time between B and B.
  • the case where the continuous shooting time is equal to or greater than the given threshold is almost synonymous with the case where the non-reference cluster stays for a long time.
  • the configuration diagram is FIG. 1 as in the first embodiment.
  • FIG. 12 shows a schematic diagram of this example. As shown in FIG. 12, not only two clusters 1 and 2 are set, but three or more clusters (5 in the example of FIG. 12) are set. Each of the reference cluster and the non-reference cluster is composed of at least one cluster. In the example of FIG. 12, the reference cluster is composed of three clusters, and the non-reference cluster is composed of two clusters. .
  • a cluster group is set from the coordinate information distribution by the cluster classification unit 125 (S702).
  • various cluster group setting methods can be used. For example, in k-means, clustering may be performed by increasing the value of k. If it is the structure which enables a user's input like 2nd Embodiment, you may make a user select each coordinate information group made into a cluster. Thereby, as shown in FIG. 12, a plurality of clusters are set from the coordinate information distribution.
  • the cluster classification unit 125 classifies the cluster group into a reference cluster group and a non-reference cluster group (S703).
  • the distance information acquisition unit 126 calculates the inter-cluster distance between the reference cluster group and the non-reference cluster group (S704).
  • the calculation method is the same as in the first embodiment.
  • the center-of-gravity position of the reference cluster may be a position of the center of gravity based on all coordinate information in each cluster included in the reference cluster group.
  • a certain cluster included in the reference cluster group and a certain cluster included in the non-reference cluster group may be picked up, and the same calculation process as in the first embodiment may be performed.
  • the keyword assignment processing according to the inter-cluster distance in S705 is the same as in the first embodiment.
  • the distribution state acquisition unit 120 includes a cluster classification unit 125 that classifies coordinate information into a plurality of clusters including at least a first cluster and a second cluster. Then, the cluster classification unit 125 classifies one or more clusters among the plurality of clusters as reference clusters, and classifies the other clusters as non-reference clusters.
  • a representative position of each cluster of a plurality of clusters is obtained, and each cluster is regarded as a point corresponding to each representative position, and clustering processing is performed again.
  • the reference cluster and the non-reference cluster can each be composed of one or more clusters.
  • appropriate clustering I could not.
  • three clusters can be set for Tokyo, Tohoku, and Kansai, respectively, so that more appropriate clustering is possible.
  • 100 processing unit 110 information acquisition unit, 120 distribution state acquisition unit, 125 cluster classification unit, 126 distance information acquisition unit, 130 keyword assignment unit, 140 cluster classification unit, 150 map presentation unit, 160 distance information acquisition unit, 200 input unit, 300 storage unit, 400 display unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2011/066033 2010-07-15 2011-07-14 画像処理装置及びプログラム Ceased WO2012008511A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/740,521 US8983138B2 (en) 2010-07-15 2013-01-14 Image processing device, information storage device, and image processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2010160624A JP5495998B2 (ja) 2010-07-15 2010-07-15 画像処理装置及びプログラム
JP2010-160624 2010-07-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/740,521 Continuation US8983138B2 (en) 2010-07-15 2013-01-14 Image processing device, information storage device, and image processing method

Publications (1)

Publication Number Publication Date
WO2012008511A1 true WO2012008511A1 (ja) 2012-01-19

Family

ID=45469507

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/066033 Ceased WO2012008511A1 (ja) 2010-07-15 2011-07-14 画像処理装置及びプログラム

Country Status (3)

Country Link
US (1) US8983138B2 (enExample)
JP (1) JP5495998B2 (enExample)
WO (1) WO2012008511A1 (enExample)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013055569A (ja) * 2011-09-06 2013-03-21 Sony Corp 撮像装置、情報処理装置、それらの制御方法、および、プログラム
US9424280B2 (en) * 2012-12-28 2016-08-23 Animoto Inc. Organizing media items based on metadata similarities
KR101581917B1 (ko) * 2014-08-12 2016-01-04 네이버 주식회사 컨텐츠 표시 제어 장치, 컨텐츠 표시 제어 방법 및 이를 컴퓨터에서 실행하기 위한 컴퓨터 프로그램
KR101721114B1 (ko) * 2016-06-27 2017-03-30 서울대학교산학협력단 위치정보가 포함된 포인트 데이터를 다축척의 웹 지도상에 클러스터링하기 위하여 격자의 크기를 결정하는 방법

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006101095A (ja) * 2004-09-29 2006-04-13 Fuji Photo Film Co Ltd イベント画像公開方法及びそのシステム
JP2007257312A (ja) * 2006-03-23 2007-10-04 Fujifilm Corp アルバム作成システム、アルバム作成方法、及びプログラム
JP2008097147A (ja) * 2006-10-06 2008-04-24 Yafoo Japan Corp 主要なクラスタを判定する方法及びコンピュータ
JP2010140383A (ja) * 2008-12-15 2010-06-24 Sony Corp 情報処理装置および方法、並びにプログラム

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5164831A (en) * 1990-03-15 1992-11-17 Eastman Kodak Company Electronic still camera providing multi-format storage of full and reduced resolution images
JP4457660B2 (ja) * 2003-12-12 2010-04-28 パナソニック株式会社 画像分類装置、画像分類システム、画像分類に関するプログラム、およびそのプログラムを記録したコンピュータ読み取り可能な記録媒体
JP4388114B2 (ja) * 2004-07-13 2009-12-24 パナソニック株式会社 移動先表示装置および移動先表示方法
JP2006059090A (ja) * 2004-08-19 2006-03-02 Fujitsu Ltd データベース検索方法及びその装置
JP4781785B2 (ja) * 2004-11-09 2011-09-28 パナソニック株式会社 位置情報検出装置および位置情報検出方法
US7663671B2 (en) * 2005-11-22 2010-02-16 Eastman Kodak Company Location based image classification with map segmentation
JP4986225B2 (ja) 2007-03-26 2012-07-25 富士フイルム株式会社 登録制コミュニケーションサイトの提供装置及び方法
JP2009036594A (ja) * 2007-07-31 2009-02-19 Panasonic Corp 移動先予測装置および移動先予測方法
US8520979B2 (en) * 2008-08-19 2013-08-27 Digimarc Corporation Methods and systems for content processing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006101095A (ja) * 2004-09-29 2006-04-13 Fuji Photo Film Co Ltd イベント画像公開方法及びそのシステム
JP2007257312A (ja) * 2006-03-23 2007-10-04 Fujifilm Corp アルバム作成システム、アルバム作成方法、及びプログラム
JP2008097147A (ja) * 2006-10-06 2008-04-24 Yafoo Japan Corp 主要なクラスタを判定する方法及びコンピュータ
JP2010140383A (ja) * 2008-12-15 2010-06-24 Sony Corp 情報処理装置および方法、並びにプログラム

Also Published As

Publication number Publication date
JP5495998B2 (ja) 2014-05-21
JP2012022561A (ja) 2012-02-02
US20130129153A1 (en) 2013-05-23
US8983138B2 (en) 2015-03-17

Similar Documents

Publication Publication Date Title
US8611678B2 (en) Grouping digital media items based on shared features
JP5827445B2 (ja) 拡張現実インタラクションを実現する方法およびシステム
CN108369633B (zh) 相册的视觉表示
KR101557297B1 (ko) 장치에 내장된 3ｄ 콘텐츠 집계
CN105830092B (zh) 用于数字合成及/或检索的系统、方法及设备
CN104115180B (zh) 信息处理设备、信息处理方法和程序
KR101213857B1 (ko) 가상 세계
KR101213868B1 (ko) 가상 세계
JP2011215964A (ja) サーバ装置、クライアント装置、コンテンツ推薦方法及びプログラム
TW201508680A (zh) 來自社群和事件資料的即時活動建議
KR20150070358A (ko) 증강 현실의 존재성과 관련된 방법
KR20060128740A (ko) 정보 처리 장치, 정보 처리 방법, 및 프로그램
Zhang et al. Multi-video summary and skim generation of sensor-rich videos in geo-space
Adams et al. Extraction of social context and application to personal multimedia exploration
Schlieder et al. Photographing a city: An analysis of place concepts based on spatial choices
JP5495998B2 (ja) 画像処理装置及びプログラム
JP5960691B2 (ja) 興味区間特定装置、興味区間特定方法、興味区間特定プログラム
JP2014146248A (ja) 表示制御装置、表示制御方法およびプログラム
CN107079144B (zh) 一种实现媒体对象显示的方法、装置及系统
JP2015153157A (ja) 仮想情報管理システム
US20210073269A1 (en) Method for clustering at least two timestamped photographs
JP5351445B2 (ja) 画像検索システム
US8786752B2 (en) Digital device and method for controlling the same
JP7485893B2 (ja) 情報処理装置、制御プログラム、及び制御方法
US20210072869A1 (en) Method for retrieving at least two captured photographs

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11806841

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11806841

Country of ref document: EP

Kind code of ref document: A1