JP2004326193A - Image management system, image management method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To facilitate management of photographs by combining a photograph taken with an object imaged on the photograph. <P>SOLUTION: In a system for facilitating photographs retrieving, by combining what is imaged on the photograph with the photograph taken, when a photographer adds a new recognition element to recognized result of the photograph taken by the photographer, the new recognition element is inserted in a recognition element row of the photograph of a region photographed as an object, and at the same time, map information used for recognition is renewed, by making the recognition element act on the photograph newly photographed region. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image management system and an image management method for managing a large number of photographic images, and a computer program, and more particularly to an image management system and an image management method for managing a photographic image including one or more moving objects as subjects. , As well as computer programs.
[0002]
More specifically, the present invention relates to an image management system, an image management method, and a computer program for recognizing a subject in a photographed photograph and for facilitating the management of the photograph by combining the photograph and the subject. More particularly, the present invention relates to an image management system and an image management method for sharing information about a subject within a user group including a plurality of users, and a computer program.
[0003]
[Prior art]
In recent years, devices for outputting and reproducing captured images as digital contents, such as digital cameras, have become widespread. This type of photograph is stored on a magnetic tape, a magnetic disk, a semiconductor memory, or the like. The simple operation of the device and the easy output of the photos, combined with the simplicity of photographing, increase the number of photos. In such a case, from the viewpoint of effective use of the content, a suitable management method of the photograph is important.
[0004]
For example, a technique has been adopted in which predetermined meta information is added to an image, and the image is managed and searched based on the meta information. In this case, events and other situations at the time of photographing the photograph, episodes related to the photographing, information and impressions on the subject, and keywords related to these information are managed as meta information together with the image. However, depending on the user's manual input of the meta information, the work load is excessive and troublesome.
[0005]
In addition, there have been proposed some methods of automatically adding a shooting time, a shooting location detected using a GPS (Global Positioning System) or the like as meta information to an image body, and the like.
[0006]
Here, the present inventors think that it is possible to facilitate a photo search by combining what is in the photograph (object) with the photographed photograph.
[0007]
For example, as a recognition unit for recognizing a building or the like as a subject, a map on which each recognition unit is placed is prepared, a range to be photographed is compared with the map, and a matching recognition unit is determined as a subject included in the photograph. Can be recognized.
[0008]
By the way, map information is generally universal and is created for everyone. Here, when describing the map information on which the recognition unit is described, if it is set for everybody, the recognition unit necessary only for the individual user or the limited user group is omitted. Conversely, if it is attempted to cover the recognition units required by all users, the amount of map information becomes enormous, and it becomes difficult to handle the map information in storage and retrieval.
[0009]
For example, when using general map information for everyone, it is necessary for an individual or a user group to add recognition targets to all target photographs by hand input or the like.
[0010]
Further, even when taking a picture after this, it is necessary to add an index appropriately to the picture whose recognition element is to be photographed.
[0011]
[Problems to be solved by the invention]
An object of the present invention is to provide an excellent image management system, an excellent image management method, and a computer program that can appropriately manage a photographic image including one or more moving objects as subjects.
[0012]
A further object of the present invention is to provide an excellent image management system and image management method that can recognize a subject in a photographed photograph and can easily manage the photograph by combining the photograph and the subject, and an image management method. To provide a computer program.
[0013]
A further object of the present invention is to provide an excellent image management system and image management method, and a computer program, which can share information about a recognition unit recognized as a subject in a user group including a plurality of users. It is in.
[0014]
Means and Action for Solving the Problems
The present invention has been made in consideration of the above problems, and a first aspect of the present invention is an image management system that manages an image in combination with a subject included in a captured image,
First map information that describes a recognition target such as a building that is recognized as a subject at a corresponding location on the map;
Second map information describing a recognition target added by the user on a map,
Shooting state obtaining means for obtaining a shooting state when the user has shot an image,
A photographing space estimating means for calculating a photographing space to be photographed in a photographed image based on the photographing state;
A subject recognition unit that compares the shooting space calculated by the shooting space estimation unit with the first map information, and recognizes a recognition target existing in the shooting space on the first map information as a subject;
And a recognition target additional registration unit for additionally registering a new recognition target in a location corresponding to the shooting space on the second map information in response to a request for adding a recognition target from a user. Image management system.
[0015]
Here, the photographing state acquiring means acquires a camera position, a lens direction, a focal length, an angle of view, and an aperture value at the time of photographing as a photographing state. Then, the photographing space estimating means calculates a photographing target region including the focus plane and the depth of field as a photographing space based on the instruction values of the photographing states.
[0016]
Further, the image management system according to the present invention compares the second map information with another photographed image of the same user, or a photographing space for a photographed image of another user, and compares the second map information with the second map information. The image processing apparatus further includes a second object recognizing means for recognizing a recognition target existing in the imaging space as an object of a captured image.
[0017]
According to the present invention, the photographing space of a photograph photographed by the user is compared with general map information to recognize what the entity of the subject is on the photographed image. Further, the user can add a new recognition index to the recognition result of the photographed photograph. However, we respect that general map information is universal and created for everyone, and the recognition target to be additionally registered is the shooting space on the second map information prepared separately from general map information. It is described in the overlapping area.
[0018]
Further, with respect to another photographed image taken by the same user or a photographed image possessed by another user, the photographing space of each photographed image is collated with the second map information so as to collate with general map information. A new recognition index can be inserted into the recognition target list obtained by the processing. In addition, by newly registering a new recognition index in the second map information, the newly registered recognition index can be applied to a photograph taken of an area where the photographing space overlaps thereafter. .
[0019]
Because map information is generally universal and created for everyone, recognition units that are only needed within individual users or within a limited group of users are discarded. On the other hand, according to the present invention, by providing the second map information separately from the general map information, an additional recognition index not included in the general map information can be recognized in the specific user group. It can be used for
[0020]
In addition, the image management system according to the present invention receives a registration application for use of the second map information from another user, and registers or rejects the other user's use of the second map information. A second map information use management means may be further provided. In such a case, the second subject recognizing means performs the subject recognizing process based on the second map information only for a captured image of a user who has been registered for use in advance.
[0021]
Since the second map information has a different creation policy for each user and relates to the privacy of the user, the second map information cannot be disclosed to all users. Therefore, in order to use the second map information in which the added recognition index which is not included in the general map information is used, it is assumed that the user has registered the use of the map. For the devices that have rejected the map use registration, the map additional information including the added recognition index is not shared thereafter.
[0022]
According to a second aspect of the present invention, there is provided a computer program described in a computer-readable format so as to execute a process for managing an image in combination with a subject included in a captured image on a computer system. So,
A photographing state acquiring step of acquiring a photographing state when a user photographs an image, and a photographing space estimating step of calculating a photographing space to be photographed in a photographed image based on the photographing state;
The photographing space calculated in the photographing space estimation step is compared with first map information describing a recognition target to be recognized as a subject at a corresponding location on a map, and the photographing space in the photographing space is described on the first map information. A subject recognition step of recognizing a recognition target existing in the object as a subject;
In response to a user's request for adding a recognition target, a new recognition target is added and registered at a location corresponding to the shooting space on a second map information prepared separately from the first map information. A registration step;
A computer program characterized by comprising:
[0023]
The computer program according to the second aspect of the present invention defines a computer program described in a computer-readable format so as to realize a predetermined process on a computer system. In other words, by installing the computer program according to the second aspect of the present invention in a computer system, a cooperative action is exerted on the computer system, and the image management according to the first aspect of the present invention is performed. The same operation and effect as those of the system can be obtained.
[0024]
Further objects, features, and advantages of the present invention will become apparent from more detailed descriptions based on embodiments of the present invention described below and the accompanying drawings.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0026]
The present invention recognizes a subject in a photographed photograph and facilitates the management of the photograph by combining the photograph and the subject. More specifically, the present invention provides a new method for managing a photograph. When a special index is specified, the recognition unit is described in the map information, and the recognition target list attached to the already photographed photo is updated.
[0027]
According to the present invention, by adding a new index using a photograph taken at a certain photographing place, the new index is added to a photograph previously taken at the same photographing place. FIG. 1 illustrates a mechanism of a photographic image management method according to the present invention.
[0028]
In the illustrated example, an image is taken toward the building 31 at a certain camera position 11 on the map 3, and a photograph indicated by reference numeral 41 is taken. Further, a photograph is taken toward the building 31 at another camera position 12 on the map 3, and a photograph indicated by reference numeral 42 is taken. Then, subject recognition processing is performed on the photographs 41 and 42, and recognition target lists appearing in the photographs are created.
[0029]
Here, it is assumed that the user who has taken the photograph 41 gives the subject 31 in the recognition target list the name “Ryugujo” and registers it as a recognition index. In response to the additional registration by the user, the image management system describes the subject 31 as “Ryugujo” in the map information of the group to which the user belongs, and recognizes the subject 31 with another photograph 52 already taken. The recognition index of the subject 31 is also updated on the target list.
[0030]
FIG. 2 shows an external configuration of a photographing device according to the embodiment of the present invention. The illustrated device is, for example, a mobile phone with a camera function.
[0031]
The illustrated device includes a main body including a user operation unit such as a button, and a lid rotatably supported at a substantially rear edge of the main body. An antenna 112 for mobile phone communication and an antenna 121 for receiving a GPS signal are arranged at the tip of the lid, and a display device made up of a liquid crystal panel is incorporated on the front side 12 thereof. A camera lens 111 appears on the back surface 11 of the lid, and an image capturing process is activated by pressing a button 1241 assigned to a shutter function on the upper surface of the main body, and an object through the lens 111 is photographed. You.
[0032]
FIG. 3 shows the internal configuration of the device shown in FIG.
[0033]
A CPU (Central Processing Unit) 415 executes each program for realizing the mobile phone function and the camera function under the control of the operating system, so that the operation of the photographing apparatus 101 is controlled overall. The CPU 415 is interconnected to each unit via a bus 417.
[0034]
A RAM (Random Access Memory) 413 is configured by a readable / writable semiconductor memory, and loads an execution program code of the CPU 415 and temporarily stores work data at the time of activation of a mobile phone function or a camera function. Used for A ROM (Read Only Memory) 414 is configured by a read-only semiconductor memory, and permanently stores information written at the time of factory shipment, such as an execution program code of the CPU 415 and manufacturing information.
[0035]
The input unit 408 includes buttons that can be operated by the user, and is used for inputting a telephone number and other data. One of the operation buttons is assigned to the shutter 409 when the camera function is activated.
[0036]
The communication unit 401 performs a communication process with a base station on the mobile phone network, and further communicates with a server (described later).
[0037]
The position measuring unit 403 measures the current position of the device based on the GPS signal received by the antenna 121. The direction obtaining unit 404 includes a digital magnetic compass or the like, and obtains the orientation of the device or the direction of the camera / lens. The position measurement includes a position error based on the signal strength of the GPS signal and the spatial spread of the GPS satellite. In the present embodiment, the position measurement unit 403 estimates the position error and outputs the position error. The direction measuring unit 404 outputs a direction error that is a fixed value.
[0038]
The imaging unit 405 includes a camera / lens, an imaging element that captures an image on an image plane thereof, a signal processing module that processes an image signal, and the like. In the present embodiment, the imaging unit 405 outputs a shooting state such as a camera position, a lens direction, a focal length, an angle of view, and an aperture value.
[0039]
The display unit 406 outputs the processing result of the CPU 415 on the screen. For example, when the mobile phone function is activated, the input telephone number and the status of other devices during a call are displayed, and when the camera function is activated, a finder screen obtained through a camera / lens and an image after shooting are displayed on the screen. Is displayed.
[0040]
The output unit 407 includes an external output of an image signal, audio output from a speaker, vibration, and other devices that provide feedback to a user.
[0041]
The clock 416 measures real time and supplies a timer signal to the system. In the present embodiment, the clock 416 outputs the imaging time by the imaging unit 405 and the position measurement time by the position measurement unit 403.
[0042]
The photo storage unit 431 stores an image captured by the imaging unit 405. The photographing log storage unit 432 stores a photographing log including a photographing time, a photographing state, position measurement at the time of photographing, and error information included in direction acquisition for each photographed image.
[0043]
The device includes an ID holding unit 402 storing device identification information for device identification, and is transmitted from the communication unit 401 to the server together with the position information acquired by the position measuring unit 403. Further, the device IDs of other devices permitted to disclose their device positions are stored in the ID list 433.
[0044]
When a photograph is taken on the mobile phone shown in FIG. 3, the photographing unit 405 operates in synchronization with an input from a shutter 409 in the input unit 408 to take a photograph, and the photographed image is stored in the image storage unit 431. save. Along with taking a picture and storing an image, the time taken by the clock 416, the shooting state output from the imaging unit 405, the camera position obtained by the position measurement unit 403 and its error range, and the direction obtained by the direction acquisition unit 404. The lens direction and its error range are acquired and stored in the photographing log storage unit 432. In addition, even when photographing is not performed, the position of the device is grasped at regular intervals and recorded as a log together with the time measured by the clock 416.
[0045]
FIG. 4 schematically shows a configuration of a server which communicates with each device shown in FIGS. 2 to 3 and performs a subject recognition and a provision service of a recognition target list. This server receives information on the shooting state and the shooting time from the shooting side device, compares the shooting position with the position information of each building on predetermined map information, and sets a building or a person in the shooting space as a subject. Perform recognition processing.
[0046]
The CPU 515 executes each program for realizing the mobile phone function and the camera function under the control of the operating system, so that the overall operation of the server device is controlled in a comprehensive manner. The CPU 515 is interconnected to each unit via a bus 517.
[0047]
The RAM 513 is configured by a readable and writable semiconductor memory, and is used for loading an execution program code of the CPU 515 and temporarily storing work data. The ROM 513 is configured by a read-only semiconductor memory, and permanently stores information written at the time of factory shipment, such as an execution program code of the CPU 515 and manufacturing information.
[0048]
The communication unit 501 performs communication processing with a mobile phone owned by a user via a mobile phone network or another network.
[0049]
The map information storage unit 524 stores predetermined map information. The map information includes arrangement information on buildings and other objects existing at each location. Further, the map additional information storage unit 523 additionally registers and stores the recognition index additionally registered on the user's photographing device on the map information of the same user group.
[0050]
The entertainment calendar 525 manages, on a time axis, information on events and the like related to buildings and other objects arranged at each location in the map information.
[0051]
The shooting target range calculation unit 510 obtains shooting conditions such as a camera position, a lens direction, a focal length, an angle of view, and an aperture value from a shooting log attached to a shot image, and focuses on a focus plane based on the indicated values of the shooting conditions. And the depth of field are calculated as the permissible range to be photographed by the camera.
[0052]
The terminal position information storage unit 521 stores terminal position information transmitted from a device carried by each user. The ID disclosure information storage unit 522 stores the device ID of a device that is permitted to disclose its device position.
[0053]
The subject list acquisition unit 511 compares the shooting space calculated from the shooting log sent from the device on the shooting side with the position information of the subject sent from the device on the subject side, and determines the position in the shooting target range of the camera. Acquire a set of people as a subject list.
[0054]
The ranking point calculation unit 512 calculates an evaluation value according to the estimated situation in the image of the subject as a ranking point. The evaluation value referred to here is calculated based on the probability that the subject exists in the photographing space. However, since the captured image includes uncertain components such as an error in the camera position and an error in the lens direction, the likelihood that the subject exists in the imaging space is determined based on the imaging position error and the gaze direction error. Weighting is provided to give ranking points based on the accuracy of the information.
[0055]
In the present embodiment, a user who carries the device shown in FIGS. 2 and 3 recognizes a subject on a captured image by comparing the image capturing space of the photo captured by the user with the map information. Further, when a certain user adds a new recognition index to the recognition result in the same user group, the recognition index is also inserted into a recognition target list of a photograph of another user whose shooting space overlaps. In addition, by additionally registering a new recognition index in the map additional information of the user group, the recognition index is applied to a photograph of an area where the photographing space overlaps thereafter.
[0056]
As described above, in order to use an added recognition index not included in general map information for subject recognition within a specific user group, a server that provides a service for providing a subject recognition for a captured image and a recognition target list is provided. It is assumed that registration of map use is performed between users. FIG. 5 illustrates a processing procedure for making an application for map use registration to the server.
[0057]
First, a device 31 of a certain user makes a map use registration application to the center server 33 to apply for permission to use a map held by another user (T511). Next, the center server 33 confirms map use registration with respect to the device 32 of another user (T521).
[0058]
When the permission of map use is returned from the device 32 of another user (T531), the center server 33 updates the map use information and sends a map use registration change notification to the device 31 on the photographing side (T514).
[0059]
FIG. 6 shows a processing procedure when an application is rejected when a certain user makes an application for permitting use of a map by another user.
[0060]
First, a device 31 of a certain user makes a map use registration application to the center server 33 to apply for permission to use a map held by another user (T511). Next, the center server 33 confirms map use registration with respect to the device 32 of another user (T521).
[0061]
On the other hand, when a refusal to use the map is returned from the device 32 of another user (T532), the center server 33 sends a map use registration rejection notification to the device 31 of the requesting user (T513).
[0062]
Since the use of the map information has different policies for each user and is related to the privacy of the user, the map addition information including the added recognition index is not shared thereafter for the devices that have refused the map use registration.
[0063]
FIG. 7 illustrates a screen configuration example of the image management user interface. A photographed image (image) is displayed in an area indicated by reference numeral 2704. In the area indicated by reference numeral 2751, the shooting time is displayed, and in the area indicated by reference numeral 2754, the recognition type is displayed according to the priority order, and the value of each item is displayed and output to the right thereof.
[0064]
Reference numerals 2761 to 2763 denote a group of command buttons. When a selection operation such as clicking one of the buttons with a mouse is applied, the corresponding command processing is applied to the currently displayed photograph.
[0065]
In an area indicated by reference numeral 2764, thumbnailed photographs are listed in descending order of image points. The photo selected on the thumbnail list 2764 is displayed and output on the display area 2704. Using a jog dial, cursor keys, a mouse pointer, etc., a desired picture can be selected from the thumbnail list.
[0066]
FIG. 8 shows how the recognition target index is changed by operating the up and down buttons.
[0067]
Reference numeral 5201 denotes a field in which a user-designated recognition unit index is written. Reference numeral 5205 indicates a button for increasing the list order of the corresponding recognition unit by one, and reference numeral 5206 indicates a button for decreasing the list order of the corresponding recognition unit by one.
[0068]
In the illustrated example, the index list 5200 of the current recognition unit includes Heian Jingu Shrine 5202, Jingu Dori 5203, and Kyoto 5204. On the other hand, as shown by reference numeral 5207, when the recognition unit “Jingu-dori” is deleted, the recognition unit “Ryugujo” is added, and the recognition unit “Ryugujo” is ranked lower, the list rank becomes higher. The list order is changed to Heian Shrine 5212, Kyoto 5214, and Ryugu Castle 5218.
[0069]
FIG. 9 illustrates a processing procedure for updating a recognition index for a photograph that has already been taken.
[0070]
First, the photographing-side device 31 adds a new recognition index to the photographed photograph by the operation procedure as shown in FIG. 8 (T911), and transmits this to the center server 33 together with the photographing state of the photograph. By transmitting the request, a registration application for the map additional information is made (T912).
[0071]
The center server 33 calculates the shooting space for the photographic image to which the recognition index has been added (T921), and additionally registers it in the map additional information 523 as a recognition unit (T922).
[0072]
Then, after notifying the newly created recognition index to other users who have registered the use of the map addition information (T923), the ACC is returned to the device 31 that has added the recognition index (T924). .
[0073]
Next, the device 31 sends the photographing state of each of the other photographs to the center server 33 (T913), and requests a determination process to determine whether or not these photographs include the added recognition index. . The center server 33 performs subject recognition processing on these photographs again (T925), and returns the ranking point value of the recognized subject for each photograph to the device 31 (T926).
[0074]
Then, when the point value of the added recognition index becomes larger than 0 (T914), the device 31 additionally registers a new recognition index in the recognition index of the photograph (T915).
[0075]
Note that the device of the registered user who has received the notification in T923 performs the processes of T913 to T915 in the same manner, and performs the process of additionally registering a new recognition index in the recognition target list of already photographed photos.
[0076]
FIG. 10 shows, in the form of a flowchart, a processing procedure for updating map information by adding a recognition index performed at T922 in FIG.
[0077]
First, based on a shooting state including a camera position, a lens direction, a focal length, an angle of view, an aperture value, and the like at the time of shooting transmitted from the device 31 side in T912, shooting including a focus surface and a depth of field is performed. A space is calculated (step S1).
[0078]
Next, weight distribution is calculated for the photographing space according to the center angle and the distance from the focus plane (step S2). The reason why weighting is applied to the imaging space is to acquire a large number of recognition candidates according to the accuracy of the information and to present them to the user in a ranked form in a situation where the measurement accuracy is not sufficiently high.
[0079]
Next, when the added recognition target index overlaps the photographing space, the overlap amount is calculated (step S3). In the present embodiment, the number is equally divided by the number of overlaps.
[0080]
Then, the recognition target index is additionally registered in the map additional information database 423 (step S4), and the entire processing routine ends.
[0081]
FIG. 11 shows a photographing area. Each piece of photograph data includes a photographing position of the camera 1, a lens direction 4001, an angle of view 4003, a focus distance 4022, focal length information, and an aperture value. The photographing area 4002 is calculated using these photographing state parameter values. Here, an arc passing through each point represented by reference numerals 4012, 4022, and 4032 represents a focus plane. An arc passing through each point represented by reference numerals 4013, 4023, and 4033 represents the front depth of field. An arc passing through each point represented by reference numerals 4011, 4021, and 4031 represents a rear depth of field.
[0082]
As described above, the imaging space is weighted according to the central angle and the distance from the focus plane. FIG. 12 shows a state of the weight gradient in the photographing space. A graph indicated by reference numeral 4101 indicates a weight inclination in the camera direction, and a graph indicated by reference numeral 4102 indicates a weight inclination in the horizontal direction of the focus plane. In the present embodiment, as shown in the figure, the weight is reduced in the vertical direction (points 4023 and 4021) and in the horizontal direction (points 4012 and 4032) around the point 4022. It should be noted that when the map information is divided into cells (described later), it is standardized so that the integral of the photographing space becomes 1.
[0083]
FIG. 13 shows a state in which the photographing spaces of different users (or of different photographs) intersect. In the example shown in the figure, two photographing regions 1321 and 1322 intersect at a region indicated by reference numeral 1323.
[0084]
In the present embodiment, for convenience of calculation processing such as collation between the imaging space and the recognition target and collation between the imaging spaces, the image is divided into mesh cells and handled in cell units.
[0085]
FIG. 14 shows, in the form of a flowchart, a processing procedure for dividing a plurality of imaging spaces in units of cells. Here, a process of adding a shooting space as a recognition unit to data that has already been divided into cells is shown.
[0086]
Processing is performed in each cell. If there is no additional area (step S11), the next cell is processed. If there are no unprocessed cells, the entire processing routine ends.
[0087]
First, one additional area is taken out (step S12). If there is no additional area (step S13), the point value in the cell is normalized to 1 (step S14), written as cell information, and the next cell is processed.
[0088]
If there is additional information (step S13), an index is added to the cell (step S16), and the process returns to step S12.
[0089]
FIG. 15 shows a configuration example of a map information format used for operation of the subject list providing service.
[0090]
In the cell information section 7001, reference numeral 7111 indicates an area number when map information is divided. This value is 0 when the map is not divided. Reference numerals 7112 and 7113 in the figure indicate the number of cells in the horizontal and vertical positions when the map is divided into cells. Reference numerals 7121 to 7123 indicate the number of recognition units in the cell. If this value is 0, the subsequent recognition unit instruction 7123-1 is omitted.
[0091]
The number of recognition units in the cell (i, j) indicated by reference numeral 7123-5 is 5, and a recognition unit instruction 7123-1 is present. In this, a recognition unit number 7123-3 and a point 7123-2 are defined as a pair. The points are standardized so that they add to one.
[0092]
Next, in the recognition unit definition unit 7002, reference numeral 7021 represents the number of recognition units, which are described in order from the first. Here, for example, in the case of the 34th information, the information of the recognition unit number 34 is shown, and there are a recognition unit category 7221 and a recognition unit name 7222.
[0093]
FIG. 16 shows the relationship between the cell-divided map, the camera position, and the lens direction. In the example shown in the figure, the map is divided into six in the vertical direction and eight in the horizontal direction. In practice, some measures such as layering the cell division are performed, but are omitted in this specification for simplification of the description. Reference numeral 1 indicates a position of a photographing device such as a camera and a lens direction. In the example shown in the drawing, the image is standing near the intersection of Nijo-dori 1432 and Jingu-dori 1437 and is facing the Heian Jingu 1431 and is photographing.
[0094]
[Supplement]
The present invention has been described in detail with reference to the specific embodiments. However, it is obvious that those skilled in the art can modify or substitute the embodiment without departing from the scope of the present invention. That is, the present invention has been disclosed by way of example, and the contents described in this specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims described at the beginning should be considered.
[0095]
【The invention's effect】
As described above in detail, according to the present invention, an excellent image management system, an excellent image management method, and a computer program capable of suitably managing a photographic image including one or more moving objects as subjects are provided. can do.
[0096]
Further, according to the present invention, an excellent image management system and image management method capable of recognizing a subject appearing in a photographed photograph and facilitating the management of the photograph by combining the photograph and the subject, And a computer program.
[0097]
Further, according to the present invention, an excellent image management system, image management method, and computer program capable of sharing information on a recognition unit recognized as a subject within a specific user group including a plurality of users Can be provided.
[0098]
According to the present invention, when a new index is designated at the time of managing a photo, the recognition unit is described in the map information, and the recognition target list attached to the already taken photo is updated. Therefore, for the recognition index newly added by the user, the map information can be updated using the photographing space of the photo to which the user has added the recognition space as a recognition unit, and the recognition index for the photos taken so far can be updated.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining a mechanism of a photographic image management method according to the present invention.
FIG. 2 is a diagram illustrating an external configuration of a photographing device according to the embodiment of the present invention.
FIG. 3 is a diagram showing an internal configuration of the device shown in FIG. 2;
FIG. 4 is a diagram schematically illustrating a configuration of a server that provides a subject recognition and a recognition target list providing service.
FIG. 5 is an operation sequence diagram showing a processing procedure for making an application for map use registration to a server.
FIG. 6 is an operation sequence diagram showing a processing procedure when an application is rejected when a user makes an application for permission to use a map held by another user.
FIG. 7 is a diagram illustrating a screen configuration example of an image management user interface.
FIG. 8 is a diagram illustrating a state in which a recognition target index is changed by operating an up / down button.
FIG. 9 is an operation sequence diagram showing a processing procedure for updating a recognition index for a photograph that has already been taken.
FIG. 10 is a flowchart showing a processing procedure for updating a map information by adding a recognition index.
FIG. 11 is a diagram showing a shooting area.
FIG. 12 is a diagram illustrating a state of a weight gradient in a photographing space.
FIG. 13 is a diagram illustrating a state where shooting spaces owned by different users (or held by different photographs) intersect.
FIG. 14 is a flowchart showing a processing procedure for dividing a plurality of photographing spaces in cell units.
FIG. 15 is a diagram illustrating a configuration example of a map information format used for operation of a subject list providing service.
FIG. 16 is a diagram showing a relationship between a cell-divided map, a camera position, and a lens direction.
[Explanation of symbols]
401 communication unit 402 ID holding unit 403 position measurement unit 404 direction acquisition unit 405 imaging unit 406 display unit 407 output unit 408 input unit 409 shutter 413 RAM
414 ... ROM
415 ... CPU
416, clock 417, bus 431, photograph storage unit 432, photography log storage unit 433, ID list 501, communication unit 510, photography target range calculation unit 511, subject list acquisition unit 512, ranking point calculation unit 513, RAM
514 ... ROM
515 ... CPU
Reference numeral 521: terminal position information storage unit 522, ID public information storage unit 523, map additional information storage unit 524, map information storage unit 525, entertainment calendar storage unit

Claims (11)

An image management system that manages an image by combining with a subject included in a captured image,
First map information describing a recognition target to be recognized as a subject at a corresponding location on a map;
Second map information describing a recognition target added by the user on a map,
Shooting state obtaining means for obtaining a shooting state when the user has shot an image,
A photographing space estimating means for calculating a photographing space to be photographed in a photographed image based on the photographing state;
A subject recognition unit that compares the shooting space calculated by the shooting space estimation unit with the first map information, and recognizes a recognition target existing in the shooting space on the first map information as a subject;
And a recognition target additional registration unit for additionally registering a new recognition target in a location corresponding to the shooting space on the second map information in response to a request for adding a recognition target from a user. Image management system. The photographing state acquiring means acquires a camera position, a lens direction, a focal length, an angle of view, and an aperture value at the time of photographing as a photographing state,
The photographing space estimating means calculates a photographing space including a focus plane and a depth of field based on the instruction values of these photographing states.
The image management system according to claim 1, wherein: The second map information is compared with another photographed image of the same user or a photographing space of a photographed image of another user, and a recognition target existing in the photographing space is photographed on the second map information. A second object recognition unit that recognizes the object as an object of the image;
The image management system according to claim 1, wherein: Subject evaluation value calculation means for calculating an evaluation value according to a situation in a captured image of each subject recognized by the subject recognition means and / or the second subject recognition means;
Subject management means for managing subjects included in each image according to a priority order based on the evaluation value;
Image search means for searching for an image including the subject according to the priority order;
The image management system according to claim 1, further comprising: A second map information use management unit that accepts a registration application for use of the second map information from another user, and registers or rejects use of the second map information by the other user;
The second subject recognition means performs a subject recognition process based on the second map information only for a photographed image of a user whose use has been registered in advance;
The image management system according to claim 3, wherein: An image management method for managing an image in combination with a subject included in a captured image,
A photographing state acquiring step of acquiring a photographing state when a user photographs an image, and a photographing space estimating step of calculating a photographing space to be photographed in a photographed image based on the photographing state;
The photographing space calculated in the photographing space estimating step is compared with first map information describing a recognition target to be recognized as a subject at a corresponding location on a map, and the photographing space in the photographing space on the first map information A subject recognition step of recognizing a recognition target existing in the object as a subject;
In response to a user's request for adding a recognition target, a new recognition target is added and registered at a location corresponding to the shooting space on a second map information prepared separately from the first map information. A registration step;
An image management method, comprising: In the shooting state obtaining step, a camera position, a lens direction, a focal length, an angle of view, and an aperture value at the time of shooting are obtained as a shooting state,
In the photographing space estimation step, a photographing space including a focus plane and a depth of field is calculated based on the instruction values of the photographing states.
7. The image management method according to claim 6, wherein: The second map information is compared with another photographed image of the same user or a photographing space of a photographed image of another user, and a recognition target existing in the photographing space is photographed on the second map information. Further comprising a second subject recognition step of recognizing the subject as an image.
7. The image management method according to claim 6, wherein: A subject evaluation value calculating step of calculating an evaluation value according to a situation in a captured image of each subject recognized in the subject recognition step and / or the second subject recognition step;
A subject management step of managing subjects included in each image according to a priority order based on the evaluation value;
An image search step of searching for an image including the subject according to the priority order;
The image management method according to claim 6, further comprising: A second map information use management step of receiving a registration request for use of the second map information from another user and registering or rejecting the use of the second map information of the other user;
In the second subject recognition step, subject recognition processing based on the second map information is performed only for a captured image of a user who has been registered for use in advance.
The image management method according to claim 8, wherein: 11. A computer program written in a computer-readable format to execute a process for managing an image in combination with a subject included in a captured image on a computer system,
A photographing state acquiring step of acquiring a photographing state when a user photographs an image, and a photographing space estimating step of calculating a photographing space to be photographed in a photographed image based on the photographing state;
The photographing space calculated in the photographing space estimating step is compared with first map information describing a recognition target to be recognized as a subject at a corresponding location on a map, and the photographing space in the photographing space on the first map information A subject recognition step of recognizing a recognition target existing in the object as a subject;
In response to a user's request for adding a recognition target, a new recognition target is added and registered at a location corresponding to the shooting space on a second map information prepared separately from the first map information. A registration step;
A computer program comprising:

Images