JP5722177B2 - Image association apparatus, photographing apparatus, and image association program - Google Patents

Description

  The present invention relates to an image association apparatus, a photographing apparatus, and an image association program that display position information in association with each other using at least two image data.

  An imaging apparatus such as a digital camera has a technique of adding position information on a map where the captured image data is captured to captured image data, and combining and displaying the captured image data on a map. Such an imaging apparatus is disclosed in Patent Document 1, for example.

JP 2010-68212 A

However, the above technique allows the user to intuitively grasp the relationship between the image data and the map image data simply by combining and displaying the captured image data on the map image data using the position information obtained by capturing the captured image data. It was not such a composite display.
An object of the present invention is to provide an image association apparatus, a photographing apparatus, and an image association program capable of displaying such that the user can intuitively grasp the relationship between image data and map image data.

Image association device according to the main aspect of the invention, the place names and facility names, a database for recording in response to a first position information indicating the positions of these place names and facility names, from the character on the map An image capturing unit that captures first image data including character information and second image data having second position information indicating a shooting position, and the first image captured by the image capturing unit. A character reading unit that reads the character information from the image data, and searches the database for the place name or the facility name corresponding to the character information read by the character reading unit, and the searched place name or the facility name The position information acquisition unit for acquiring the first position information, the first position information acquired by the position information acquisition unit, and the second image data acquired by the image acquisition unit When correspondence with the second position information is obtained, an image association unit for associating the first image data with the second image data, and the first image data associated with the image association unit A display control unit for displaying the second image data.
A photographing apparatus according to a main aspect of the present invention includes the image association apparatus, photographs a subject, and acquires the second image data having the second position information indicating the photographing position.

An image association program according to a main aspect of the present invention includes: a first image data including character information including characters on a map; a second image data having second position information indicating a shooting position; An image capturing function for capturing the character information, a character reading function for reading the character information from the first image data captured by the image capturing function, and the character information read by the character reading function. place names or facility names is retrieved from the database, the retrieved the position information acquisition function for acquiring the first position information, the first position is acquired by the position information acquisition function of the place names or the facility names When the correspondence between the information and the second position information of the second image data captured by the image capturing unit can be obtained, the first image data and the previous image data And image association function to associate the second image data, thereby realizing the display control function of displaying the said image association said second image data and the first image data which gave related by function.

  According to the present invention, it is possible to provide an image association apparatus, a photographing apparatus, and an image association program capable of displaying such that the user can intuitively grasp the relationship between image data and map image data.

The block block diagram which shows one Embodiment of the imaging device provided with the image correlation apparatus which concerns on this invention. The figure which shows the imaging | photography condition when acquiring map image data with the same apparatus. The schematic diagram which shows the map image data acquired by the same apparatus. The figure which shows the imaging | photography condition at the time of image | photographing the target object by the same apparatus and acquiring captured image data. The schematic diagram which shows the picked-up image data acquired by the same apparatus. The schematic diagram which shows the database in the apparatus. The figure which shows the example of a display of the map image data linked | related by the image correlation part in the same apparatus, and picked-up image data. The figure which shows the other example of a display of the map image data linked | related by the image correlation part in the same apparatus, and picked-up image data. Explanatory drawing of correlation with the map image data and picked-up image data by the image correlation part in the same apparatus. The schematic diagram which shows the map image file recorded by the image recording part in the apparatus. FIG. 3 is a schematic diagram showing a captured image file recorded by an image recording unit in the apparatus. The figure which shows the imaging | photography condition when image | photographing the map described in the guidebook by the same apparatus and acquiring map image data. The figure which shows the example of a display of the map image data and imaging | photography image data which were made | formed by the image correlation part in the map recollection mode of the same apparatus. The camera control flowchart in the same apparatus. The reproduction mode flowchart in the same apparatus. The map recollection mode flowchart in the same apparatus. The figure which shows the modification of a display of the map image data and picked-up image data on the display in the same apparatus.

Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a block diagram of a photographing apparatus provided with an image association apparatus. The photographing apparatus 1 includes a signal processing control unit 2 including a computer, a photographing unit 3, a position detector 4, a recording unit 5, a display 6, an operation determination unit 7, and a program memory 8.
As shown in FIG. 2, the photographing unit 3 photographs a map M on which character information including characters is described as a subject, and obtains first image data, for example, map image data D1 shown in FIG. Further, the photographing unit 3 performs photographing including a main subject such as a person H as shown in FIG. 4 to obtain second image data, for example, photographed image data D2 shown in FIG.

The map image data D1 is acquired, for example, when the user Y photographs a map M posted on a signboard by the apparatus 1. In this map M, for example, character information composed of characters such as Δ □ temple, ○ × station, etc. is described.
The captured image data D2 is acquired, for example, when the user Y captures a subject including a main subject such as a person H by the apparatus 1.

The position detector 4 detects the position of the point where the photographing apparatus 1 exists, and is, for example, a wide area positioning system (GPS). This position detector 4 detects the position of the point where the photographing apparatus 1 exists based on the longitude and latitude.
The recording unit 5 includes a memory such as a RAM, and records, for example, one or more map image data D1 and one or more photographed image data D2. The recording unit 5 is formed with a database 5d for storing positions such as place names. FIG. 6 shows a schematic diagram of the database 5d. In the database 5d, place names, facility names, and the like, and first position information indicating the positions of these place names, facility names, and the like are recorded correspondingly. The place name is, for example, XX park, XX historical site, and the facility name is, for example, △ temple, XX station. Each position such as the place name and the facility name is represented by latitude and longitude. For example, the position of Δ □ temple is latitude XXX1 and longitude XXX2.

The display 6 is, for example, a liquid crystal display, and is provided on the back surface of the apparatus 1 and displays either one or both of the map image data D1 and the plurality of captured image data D2. A touch panel is provided on the display surface of the display 6. This touch panel outputs, for example, a touch position signal indicating an area pressed by a user's finger or the like.
The operation determination unit 7 receives the operation of the user Y with respect to the apparatus 1 and determines whether the content of the operation is, for example, a shooting mode or a playback mode, or whether the shutter is turned on and is a shooting operation. In addition, the operation determination unit 7 receives an operation of the user Y with respect to the apparatus 1 and determines whether the map image data D1 and the captured image data D2 are displayed on the display 6 in association with each other.

The program memory 8 stores an image association program in advance. This image association program captures map image data D1 including character information consisting of characters on a map and photographed image data D2 having second position information indicating a photographing position into a signal processing control unit 2 composed of a computer. An image capturing function to be read, a character reading function for reading character information from the map image data D1 captured by the image capturing function, and a place name or facility name corresponding to the character information read by the character reading function from the database 5d. A position information acquisition function for searching and acquiring the first position information of the searched place name or facility name, and the first position information acquired by the position information acquisition function and the captured image captured by the image capturing unit When the correspondence with the second position information of the data D2 is obtained, the image that associates the map image data D1 and the captured image data D2 And communicating with function, thereby realizing the display control function of displaying the the map image data D1 which gave associates the captured image data D2 by the image association function.
The image association function realizes that the shooting position of the shot image data D2 is indicated on the map in the map image data D1.

  The signal processing control unit 2 takes in one or a plurality of map image data D1 and one or a plurality of captured image data D2, and determines the latitude / longitude on the map image data D1 and the position where the captured image data D2 is captured. If the latitude / longitude indicating the position where the captured image data D2 is captured exists in the latitude / longitude on the map M captured in the map image data D1, the map image data D1 and the captured image are captured. The image data D2 is displayed in association with the image data D2. Specifically, the signal processing control unit 2 includes a computer, and includes an image capturing unit 10, a character reading unit 11, a position information acquisition unit 12, an image association unit 13, a display control unit 14, and an image recording unit 15. And a function of a notification unit 16, a position determination unit 17, and a reproduction image search unit 18.

  The image capturing unit 10 captures the map image data D1 and the captured image data D2 acquired by the capturing unit 3 from the capturing unit 3. The image capturing unit 10 captures not only the map image data D1 and the captured image data D2 from the capturing unit 3, but also the map image data D1 and the captured image data D2 from an external capturing device, an image processing device, or the like. But you can.

The character reading unit 11 reads character information from the map image data D <b> 1 captured by the image capturing unit 10, that is, character information such as Δ □ temple, ◯ × station, etc. described on the map M.
The position information acquisition unit 12 searches the database 5d for a place name or facility name corresponding to the character information such as △ □ temple, XX station, etc. read by the character reading unit 11, and the location name or facility name of the searched place name or facility name. If the latitude / longitude is the first position information, for example, Δ □ temple, the latitude XXX1 and longitude XXX2 representing the location of the Δ □ temple are acquired.

  The image association unit 13 includes the first position information acquired by the position information acquisition unit 12, for example, latitude XXX1 and longitude XXX2 representing the position of the temple, and the captured image data D2 captured by the image capturing unit 10. If there is a latitude and longitude corresponding to the latitude and longitude representing the imaged position, that is, the latitude and longitude representing the position where the captured image data D2 is captured in the latitude and longitude on the map M captured in the map image data D1, the map The image data D1 and the captured image data D2 are associated with each other. This association indicates a position where the photographed image data D2 is captured in the map image data D1.

The display control unit 14 associates the map image data D1 and the captured image data D2 associated by the image association unit 13, that is, the map image data D1 associated with the position where the captured image data D2 is captured in the map image data D1. And the captured image data D2 are displayed on the display 6.
FIG. 7 shows a display example of the association between the map image data D1 and the photographed image data D2 displayed on the display 6. The photographed image data D2 indicates by the tip of the arrow F that it has been acquired by photographing of the user Y at the position P on the map image data D1.
FIG. 8 shows another display example of the association between the map image data D1 and the captured image data D2. The map image data D1 shown in the figure is different from the map image data D1 shown in FIG. In the association shown in the figure, the captured image data D2 is superimposed on the position P where the captured image data D2 is captured on the map image data D1. Note that the captured image data D2 matches, for example, the center position of the image data D2 and the captured position P on the map image data D1.

FIG. 9 is a diagram for explaining the association between the map image data D1 and the captured image data D2. In this figure, the scale is almost accurate, and the direction in which each value of latitude and longitude (I, K) becomes larger corresponds to each direction of the xy coordinates. Further, the map image data D1 and the photographed image data D2 are examples obtained by photographing in an area represented by east longitude and north latitude on the earth, respectively. In the image in the figure, the place name / facility name □ × station is equivalent to the maximum latitude value and maximum longitude value , ○ × station is equivalent to the minimum latitude value and minimum longitude value , and these □ × station and ○ × station Is an example of an image having an intermediate latitude and longitude.

The figure shows the association between the map image data D1 and the photographed image data D2 shown in FIG. On the screen of the display 6, the lower left corner of the screen is coordinate (x, y) = (0, 0), and the upper right corner is coordinate (x, y) = (x10, y10). When the screen of the display 6 is represented by the latitude and longitude (I, K) of the map M, the lower left of the screen is latitude / longitude (I, K) = (I1, K1), and the upper right is the latitude / longitude (I, K). K) = (I2, K2).
On the screen of the display 6, the latitude difference between the lower left and upper right of the screen is (I2-I1), and the longitude difference is (K2-K1).
The coordinates (x3, y3) of the position P at which the captured image data D2 is displayed on the screen of the display 6 are obtained by the following equations (1) and (2) using the latitude and longitude (I3, K3) of the position P.
x3 = x10 · (K3−K1) / (K2−K1) (1)
y3 = y10. (I3-I1) / (I2-I1) (2)
The image recording unit 15 converts the map image data D1 and the captured image data D2 captured by the image capturing unit 10 into files and records them in the recording unit 5. The image recording unit 15 adds the latitude / longitude information on the map image data D1 to the map image data D1 to form a file, and the latitude / longitude information indicating the position where the captured image data D2 is captured in the captured image data D2. Add to file.

  FIG. 10 is a schematic diagram showing an example of a map image file MF obtained by converting the map image data D1 into a file. One map image file MF is created for one piece of map image data D1. The map image file MF includes, for example, the map image data D1, the shooting date and time of the map image data D1, the map image flag, and the four corners of the map image data D1 (for example, upper right, upper left, lower right, lower left on FIG. 3). ) And each latitude and longitude on the map M. The latitude longitude A1 in the upper right is (XXX11, XXX12), the latitude longitude in the upper left A2 is (XXX13, XXX14), the latitude longitude in the lower right A3 is (XXX15, XXX16), and the latitude longitude in the lower left A4 is (XXX17, XXX18). It is.

  FIG. 11 is a schematic diagram showing an example of a photographic image file TF obtained by making the photographic image data D2 into a file. One photographic image file TF is created for one photographic image data D2. The photographed image file TF includes, for example, photographed image data D2, a photographing date / time of the photographed image data D2, a map image flag, and latitude / longitude information indicating a position where the photographed image data D2 is captured.

  The notification unit 16 associates the latitude / longitude on the map image data D1 with the latitude / longitude indicating the position where the captured image data D2 is captured by the image association unit 13, that is, on the map M captured by the map image data D1. If there is no latitude / longitude indicating the position where the captured image data D2 is captured in the map image data D1, the map image data D1 and the captured image data D2 cannot be associated with each other. A warning is displayed on the screen of the display 6, or a warning is issued by voice through a speaker or the like. Since the map image data D1 has information on each latitude and longitude on the map M corresponding to the four corners, the image associating unit 13 includes information on each latitude and longitude at the four corners of the map image data D1 and the captured image data D2. By comparing with the latitude and longitude information of the shooting position, it can be determined whether or not the shooting position of the shot image data D2 exists in the map M of the map image data D1.

Next, the configuration of the map image data D1 in the playback mode will be described.
In the reproduction mode, the position determination unit 17 reads out the second position information included in the captured image data D2, that is, the latitude and longitude information indicating the position where the captured image data D2 is captured from the captured image file TF, and the captured image data D2. The shooting position obtained by shooting is determined.
The reproduction image search unit 18 is determined by the position determination unit 17 based on the place name and facility name recorded in the database 5d shown in FIG. 6 and each position (latitude and longitude information) such as the place name and the facility name. The map image data D1 including the latitude and longitude of the shooting position is retrieved from the plurality of map image data D1 recorded in the recording unit 5.
The display control unit 14 reproduces and displays the map image data D1 and the captured image data D2 searched by the reproduction image search unit 18 side by side on the screen of the display 6.

On the other hand, as shown in FIG. 12, the photographing unit 3 photographs map image data D1 including character information composed of characters on the map M, for example, a map M described in a guidebook B of a travel destination in each region. Then, the map image data D1 which is the first image data is acquired.
In the playback mode, the device 1 uses a plurality of photographed image data D2 already recorded in the recording unit 5 based on the longitude / latitude information of the map image data D1 such as the guide book B to map the map image such as the guide book B. At least one photographed image data D2 having a latitude and longitude in the vicinity including the longitude and latitude of the data D1 is retrieved from the recording unit 5, and the photographed image data D2 and the map image data D1 are reproduced and displayed on the display 6. In addition, this reproduction | regeneration mode reads the picked-up image data D2 already recorded on the recording part 5 based on map image data D1, such as guidebook B, and reproduces and displays it, and is henceforth called map recollection mode.

That is, the image capturing unit 10 captures the map image data D1 such as the guide book B acquired by the photographing unit 3 from the photographing unit 3. The map image data D1 is filed and recorded in the recording unit 5 in the same manner as the map image file MF shown in FIG.
In the map recollection mode, the character reading unit 11 uses characters such as XX temple as described above from the map image data D1 obtained by photographing the map M such as a guidebook captured by the image capturing unit 10. Read information.
The position information acquisition unit 12 searches the database 5d for a place name or facility name corresponding to the character information in the map image data D1 read by the character reading unit 11, and acquires the latitude and longitude of the searched place name or facility name. To do.

The image association unit 13 includes the latitude / longitude of the place name or facility name in the map image data D1 acquired by the position information acquisition unit 12, and the latitude / longitude of at least one photographed image data D2 already recorded in the recording unit 5. If this correspondence is obtained, the map image data D1 and the photographed image data D2 are associated with each other.
The display control unit 14 displays the map image data D1 and captured image data D2 associated by the image association unit 13 on the screen of the display 6 as shown in FIG. In the figure, the map image data D1 and the two photographed image data D2a and D2b are reproduced and displayed on the screen of the display 6, and the map image data D1 and the two photographed image data D2a and D2b are respectively indicated by arrows. They are related by F1 and F2. The tips of the arrows F1 and F2 indicate the shooting positions at which the captured image data D2a and D2b are acquired, respectively.

Next, the operation of the apparatus configured as described above will be described according to the camera control flowchart shown in FIG.
The operation determination unit 7 receives the operation of the user Y with respect to the apparatus 1 and determines whether the content of the operation is, for example, a shooting mode or a playback mode, or whether the shutter is turned on and is a shooting operation.
As shown in FIG. 2, the photographing unit 3 photographs a map M on which character information including characters is described as a subject, and obtains first image data, for example, map image data D1 shown in FIG. Alternatively, the photographing unit 3 performs photographing including a main subject such as a person H as shown in FIG. 4 to obtain second image data, for example, photographed image data D2 shown in FIG.

  In this state, in step S101, the signal processing control unit 2 determines whether or not the determination result of the operation determination unit 7 is the shooting mode. If the result of this determination is that the camera is in shooting mode, the signal processing control section 2 starts the shooting operation in step S102, and the captured image data acquired by the imaging section 3 is captured by the image capturing section 10, and this captured image is captured. Data is image-processed and displayed on the display 6 as a through image in step S103.

  In step S104, the signal processing control unit 2 determines whether or not the through image displayed on the display 6 is a map M as shown in FIG. Whether or not the map M is shot is determined, for example, by providing a button for a map shooting mode or the like in the operation determination unit 7 and determining that the map M is shot when this button is pressed.

If the result of this determination is that the map M is not photographed, the signal processing control section 2 judges in step S105 whether or not the release button has been pressed and photographed. When the release button is pressed, the signal processing control unit 2 performs a subject photographing operation in step S106. For example, the photographing unit 3 performs photographing including a main subject such as a person H as shown in FIG. 4 and obtains photographed image data D2 shown in FIG. 5, for example.
At the time of this photographing, the position detector 4 detects the position of the point where the photographing device 1 is present from the longitude and latitude, for example, by a wide-area positioning system (GPS).

  In step S107, the signal processing control unit 2 determines again whether or not the captured image is a map M as shown in FIG. Here, since normal photographing including a main subject such as the person H was performed as shown in FIG. 4, the signal processing control unit 2 performs, for example, the photographed image data D2 shown in FIG. For example, a photographed image file TF as shown in FIG. 11 is created using the longitude and latitude information of the point where the photographing apparatus 1 detected by 4 is present, that is, the longitude and latitude information of the position obtained by photographing the photographed image data D2. To do. The captured image file TF is recorded in the recording unit 5 by the signal processing control unit 2.

  On the other hand, as a result of the determination in step S104, if the through image displayed on the display 6 is a map M as shown in FIG. 2, for example, the signal processing control unit 2 uses the map M as the through image in step S111. As shown on the display 6. At the same time, the character reading unit 11 of the signal processing control unit 2 reads the character information from the map image data D1 captured by the image capturing unit 10, that is, for example, Δ □ temple, ○ × station, etc. described on the map M. Read character information.

In step S112, the position information acquisition unit 12 searches the database 5d for a place name or a facility name corresponding to character information such as Δ □ temple, XX station, etc. read by the character reading unit 11.
In step S113, the location information acquisition unit 12 determines the latitude and longitude as the first location information of the searched place name or facility name. For example, if it is a Δ □ temple, the latitude XXX1 indicating the location of the Δ □ temple, the longitude XXX2 is acquired.

  In step S <b> 114, the image association unit 13 captures the first position information acquired by the position information acquisition unit 12, for example, latitude XXX <b> 1 and longitude XXX <b> 2 representing the position of the temple and the image capturing unit 10. Correspondence with the latitude and longitude representing the position where the image data D2 was imaged, that is, the latitude and longitude representing the position where the captured image data D2 was imaged exists in the latitude and longitude on the map M captured in the map image data D1. Then, the map image data D1 and the captured image data D2 are associated with each other.

In step S115, the notification unit 16 captures the correspondence between the latitude / longitude on the map image data D1 and the latitude / longitude indicating the position where the captured image data D2 is captured by the image association unit 13, that is, the map image data D1. It is determined whether or not the latitude and longitude on the map M, that is, the latitude and longitude representing the position where the captured image data D2 is captured in the map image data D1 is present.
If the result of this determination is that there is a latitude / longitude representing the position at which the captured image data D2 was captured in the map image data D1, the signal processing control section 2 proceeds to step S105. On the other hand, if there is no latitude / longitude indicating the position where the captured image data D2 is captured in the map image data D1, the notification unit 16 associates the map image data D1 with the captured image data D2 in step S116. For example, a warning is displayed on the screen of the display 6, or a warning is issued by voice through a speaker or the like.

The display control unit 14 associates the map image data D1 and the captured image data D2 associated by the image association unit 13, that is, the map image data D1 associated with the position where the captured image data D2 is captured in the map image data D1. And the captured image data D2 are displayed on the display 6.
For example, as shown in the display example of the association between the map image data D1 and the photographed image data D2 shown in FIG. 7, the photographed image data D2 is acquired by photographing of the user Y at the position P on the map image data D1. This is indicated by the tip of F.
Further, as shown in FIG. 8, the captured image data D2 is superimposed on a position P where the captured image data D2 is captured on the map image data D1.

  The signal processing control unit 2 moves to step S105, for example, determines whether or not the release button is pressed to shoot, and when the release button is pressed, the map M shown in FIG. Perform the shooting operation. By this photographing operation, the photographing unit 3 acquires map image data D1 as shown in FIG. At the time of this photographing, the position detector 4 detects the position of the point where the photographing device 1 is present from the longitude and latitude, for example, by a wide-area positioning system (GPS).

In step S107, the signal processing control unit 2 determines again whether or not the captured image is a map M as shown in FIG. Here, since the map M was photographed, the image recording unit 15 converts the map image data D1 captured by the image capturing unit 10 into a file and records it in the recording unit 5 in step S109. For example, as shown in FIG. 10, the image recording unit 15 creates a map image file MF in which the latitude and longitude information on the map image data D1 detected by the position detector 4 is added to the map image data D1. The map image file MF is recorded in the recording unit 5 by the image recording unit 15.
Next, the operation of the apparatus in the playback mode will be described with reference to the playback mode flowchart shown in FIG.
If the result of determination in step S101 is not the shooting mode, the signal processing control unit 2 reads a plurality of captured image data D2 recorded, for example, in the recording unit 5 in step S121, and uses these captured image data D2 as a result. A list is displayed on the display 6 through the display control unit 14. For example, the display control unit 14 displays a predetermined number of photographed image data D2 among the plurality of photographed image data D2 on the display 6 as thumbnails.

In step S122, the signal processing control unit 2 determines whether, for example, any one of the plurality of photographed image data D2 displayed on the display 6 is selected and receives an instruction to enlarge and reproduce. Determine. Specifically, the signal processing control unit 2 selects the captured image data D2 to be enlarged and reproduced from among the plurality of captured image data D2 displayed on the display surface of the display 6. It is determined whether or not the touch panel area corresponding to the position of the display area D2 is touched by, for example, the user's finger.
When the touch panel area is touched by, for example, a user's finger, a touch position signal corresponding to the touch area is output from the touch panel, so that the signal processing control unit 2 outputs the touch position signal from the touch panel in step S123. By inputting the touch position signal, one selected photographed image data D2 is determined, and the photographed image data D2 is enlarged and displayed on the display surface of the display 6.

  In step S124, the signal processing control unit 2 determines whether or not the map image data D1 is displayed on the display surface of the display 6 in conjunction with the display of the captured image data D2. That is, the operation determination unit 7 of the signal processing control unit 2 receives the operation of the user Y with respect to the apparatus 1 and determines whether the map image data D1 and the captured image data D2 are displayed on the display 6 in association with each other.

As a result of the determination, if the map image data D1 and the captured image data D2 are displayed in association with each other on the display 6, the position determination unit 17 converts the captured image data D2 into a file as shown in FIG. 11 in step S125. The latitude I and longitude K information indicating the position where the captured image data D2 is captured is read from the captured image file TF.
In step S126, the reproduction image search unit 18 determines a position determination unit based on the place name and facility name recorded in the database 5d shown in FIG. 6 and each position (latitude and longitude information) such as the place name and the facility name. The map image data D1 including the latitude and longitude of the photographing position determined by 17 is searched from the plurality of map image data D1 recorded in the recording unit 5.
As a result of this search, in step S127, the signal processing control unit 2 includes information on the latitude I and longitude K representing the imaged position of the captured image data D2 in the plurality of map image data D1 stored in the recording unit 5. To determine whether there is map image data D1 including the latitude I and longitude K.
If there is map image data D1 including the latitude I and longitude K as a result of this determination, the display control unit 14 reads out the map image data D1 including the latitude I and longitude K from the recording unit 5 in step S128. The map image data D1 and the captured image data D2 are arranged in parallel and displayed on the display surface of the display 6 shown in FIG. The display control unit 14 may display the map image data D1 and the captured image data D2 on the display surface of the display 6 in association with each other as shown in FIG.
Thereafter, in step S129, the signal processing control unit 2 determines whether or not to end the parallel display of the map image data D1 and the captured image data D2 or the display in association with the map image data D1 and the captured image data D2. Determine. For example, when the user performs an end operation on the operation determination unit 7, the signal processing control unit 2 displays the map image data D1 and the captured image data D2 in parallel, or the map image data D1 and the captured image data D2. End the display of the association.
If there is no map image data D1 including latitude I and longitude K as a result of the determination in step S127, the notification unit 16 shown in FIG. 1 determines whether the map image data D1 and the captured image data D2 are in step S130. A notification that the association cannot be made, for example, a warning is displayed on the screen of the display 6, or a warning is issued by voice through a speaker or the like.

  As a result of the determination in step S122, when the photographed image data D2 to be enlarged and reproduced is not selected and an instruction not to enlarge and reproduce the photographed image data D2 enters the operation determination unit 7, the signal processing control unit 2 In step S122, for example, it is determined whether or not the operation determination unit 7 is operated to return to the immediately preceding process. If the process returns to the immediately preceding process, the signal processing control unit 2 returns to step S101. If not, the process proceeds to step S121.

Next, operation | movement of the apparatus in map recollection mode is demonstrated according to the map recollection mode flowchart shown in FIG. Note that description of steps S121 to S131 that are the same as the steps of the playback mode flowchart shown in FIG. 15 is omitted.
In this map recollection mode, a plurality of photographed image data D2 is recorded in the recording unit 5 in advance.
As a result of the determination in step S122, when the photographed image data D2 to be magnified and reproduced is not selected and an instruction not to perform magnified reproduction of the photographed image data D2 enters the operation determination unit 7, the signal processing control unit 2 Moving to S141, it is determined whether or not the map recollection mode is set.

  For example, when the user sets the map recollection mode for the operation determination unit 7, the signal processing control unit 2 acquires the map image data D1 in step S142. That is, as shown in FIG. 12, the photographing unit 3 photographs map image data D1 including character information composed of characters on the map M, for example, a map M described in a guidebook B of a travel destination in each region. Then, map image data D1 is acquired.

The image capturing unit 10 captures the map image data D <b> 1 acquired by the photographing unit 3 from the photographing unit 3. The map image data D1 is filed and recorded in the recording unit 5 in the same manner as the map image file MF shown in FIG. The map image file MF includes, for example, the map image data D1, the shooting date and time of the map image data D1, the map image flag, and the four corners of the map image data D1 (for example, upper right, upper left, lower right, lower left on FIG. 3). ) And each latitude and longitude on the map M.
In step S142, the character reading unit 11 reads character information including characters such as □ × station from the map image data D1 of the map M such as a guidebook acquired by the image acquisition unit 10.

  In step S143, the position information acquisition unit 12 obtains, from the database 5d, a place name or a facility name corresponding to character information such as a place name or a facility name such as □ × station in the map image data D1 read by the character reading unit 11. In step S144, the latitude and longitude of the searched place name or facility name are acquired.

  The image association unit 13 searches for a plurality of captured image data D2 already recorded in the recording unit 5, records the latitude and longitude of the place name or facility name in the map image data D1 acquired by the position information acquisition unit 12, and the recording It is determined whether or not correspondence with the latitude and longitude of at least one photographed image data D2 already recorded in the unit 5 can be obtained. As a result of the determination, if the correspondence between the latitude / longitude of the place name or facility name in the map image data D1 and the latitude / longitude of at least one photographed image data D2 already recorded in the recording unit 5 is obtained, the image association unit In step S145, 13 associates the map image data D1 with the captured image data D2.

  The display control unit 14 associates with the image associating unit 13, for example, as shown in FIG. The map image data D1 and the photographed image data D2 subjected to are displayed on the display 6. Further, as shown in FIG. 8, the display control unit 14 superimposes the captured image data D2 on the position P where the captured image data D2 is captured on the map image data D1. Further, as shown in FIG. 13, the display control unit 14 reproduces and displays the map image data D1 and the two photographed image data D2a and D2b on the screen of the display 6, and the two photographed image data D2a and D2b. Are associated with each other by arrows F1 and F2.

  As described above, according to the above-described embodiment, the map image data D1 and the captured image data D2 are captured, the character information is read from the map image data D1, and the place name or facility name corresponding to the character information is retrieved from the database 5d. If the first location information of the searched place name or facility name is acquired, and the correspondence between the first location information of the place name or facility name and the second location information of the photographed image data D2 is obtained, Since the map image data D1 and the photographed image data D2 are associated with each other, for example, an association indicating the photographing position of the photographed image data D2 is displayed on the map in the map image data D1, and the map displayed on the screen of the display 6 is displayed. If the user views the image data D1 and the captured image data D2, the relationship between the map image data D1 and the captured image data D2, for example, the map image data D Whether a photographed image data D2 captured anywhere position in the map M to be displayed can be the grasp intuitively by. For example, if a map M such as a signboard is photographed at a travel destination to acquire the map image data D1, and a subject is photographed at the travel destination to obtain a plurality of photographed image data D2, the latitude and longitude of the map image data D1 and the plural A plurality of photographed image data D2 can be associated with the map image data D1 based on the latitude and longitude of the photographed image data D2.

In the reproduction mode, a plurality of photographed image data D2 associated with the map image data D1 can be reproduced and displayed on the screen of the display 6.
Further, in the map recollection mode, for example, by taking a map M described in a guidebook B or the like of each destination in the area and acquiring the map image data D1, the map image data D1 is obtained in the vicinity of the latitude and longitude. A plurality of photographed image data D2 already photographed and acquired can be retrieved from the recording unit 5, and the plurality of photographed image data D2 and map image data D1 can be associated with each other and reproduced and displayed on the screen of the display 6.

The one embodiment may be modified as follows.
The display surface of the display 6 of the imaging device 1 is divided into a plurality of display areas, for example, four display areas E1 to E4 as shown in FIG. 17, and the captured image data captured in the normal imaging mode is displayed on the display area E1. Displayed, the captured image data D2 and the like are reproduced and displayed in the normal reproduction mode in the display area E2, the map image data D1 is displayed in the display area E3, and the captured image data D2 in the map recollection mode is displayed in the display area E4. You may do it.
In the above embodiment, the photographing apparatus 1 includes the image association apparatus. However, the image association apparatus may be used alone.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  1: imaging device, 2: signal processing control unit, 3: imaging unit, 4: position detector, 5: recording unit, 6: display, 7: operation determination unit, Y: user, M: map, H: person, 5d: Database, 10: Image capturing unit, 11: Character reading unit, 12: Position information acquisition unit, 13: Image association unit, 14: Display control unit, 15: Image recording unit, 16: Notification unit, 17: Position Determination unit, 18: reproduction image search unit, D1: map image data, D2: photographed image data.

Claims (16)

A database that records place names and facility names and first position information indicating the positions of these place names and facility names in correspondence with each other;
An image capturing unit that captures first image data including character information composed of characters on a map and second image data having second position information indicating a shooting position;
A character reading unit that reads the character information from the first image data captured by the image capturing unit;
Position information acquisition that searches the database for the place name or the facility name corresponding to the character information read by the character reading unit, and acquires the first position information of the searched place name or the facility name And
When the correspondence between the first position information acquired by the position information acquisition unit and the second position information of the second image data acquired by the image acquisition unit is obtained, the first image An image association unit for associating data with the second image data;
A display control unit for displaying the first image data and the second image data associated by the image association unit;
An image association apparatus comprising:   The image association apparatus according to claim 1, wherein the image association unit indicates the shooting position of the second image data on the map in the first image data.   The image association apparatus according to claim 2, wherein the image association unit indicates the shooting positions of a plurality of the second image data on the map in the first image data.   The display control unit includes at least one of the first image data and the at least one of the first image data associated with each other indicating the shooting position of the second image data on a map in the first image data by the image association unit. 4. The image association apparatus according to claim 3, wherein the second image data is displayed.   The image association apparatus according to claim 1, wherein the first and second image data are acquired by photographing.   6. The image association apparatus according to claim 5, wherein the first image data is acquired by photographing a map.   6. The image association apparatus according to claim 5, wherein the second image data is acquired by photographing a subject.   The image association apparatus according to any one of claims 1 to 7, wherein the first position information and the second position information are each expressed by latitude and longitude.   The image association apparatus according to claim 1, further comprising a position detection unit that detects the photographing position and acquires the second position information. A position determination unit configured to determine the shooting position from the second position information included in the second image data when set to the reproduction mode;
The first position information including the first position information corresponding to the photographing position determined by the position determination unit based on the place name or the facility name recorded in the database and the first position information. A playback image search unit for searching image data;
With
The display control unit displays the first image data and the second image data searched by the reproduction image search unit;
The image association apparatus according to claim 1, wherein the image association apparatus is an image associating apparatus. An image recording unit for recording the second image data;
In the map recollection mode, when the first image data including the character information composed of characters on the map is captured by the image capturing unit,
The character reading unit reads the character information from the first image data captured by the image capturing unit;
The position information acquisition unit searches the database for the place name or the facility name corresponding to the character information read by the character reading unit, and the first position of the searched place name or the facility name. Get information,
The image association unit searches the image recording unit for the second image data having the second position information that can correspond to the first position information acquired by the position information acquisition unit, and performs the search Associating the second image data and the first image data,
The display control unit displays the first image data and the second image data associated by the image association unit;
The image association apparatus according to claim 1.   In order to take correspondence between the first position information acquired by the position information acquisition unit and the second position information of the second image data captured by the image capturing unit, the first position information 2. The image association apparatus according to claim 1, wherein the image data is recorded in an image recording unit together with latitude and longitude information on the map corresponding to each corner of the first image data.   An informing unit that informs that the association between the first image data and the second image data is not possible unless the correspondence between the first position information and the second position information is obtained by the image association unit. The image association apparatus according to claim 1, further comprising: An image association apparatus according to claim 1,
The first image data is obtained by photographing a map on which character information consisting of characters is described as a subject, obtaining the first image data, and taking the subject, and having the second position information indicating the photographing position. 2 image data is acquired,
An imaging apparatus characterized by that. On the computer,
An image capturing function for capturing first image data including character information composed of characters on a map and second image data having second position information indicating a photographing position;
A character reading function for reading the character information from the first image data captured by the image capturing function;
A location information acquisition function that searches a database for a place name or facility name corresponding to the character information read by the character reading function, and acquires the first location information of the searched place name or facility name;
When the correspondence between the first position information acquired by the position information acquisition function and the second position information of the second image data acquired by the image acquisition unit is obtained, the first image An image association function for associating data with the second image data;
A display control function for displaying the first image data and the second image data associated by the image association function;
An image association program for realizing the above. The image association program according to claim 15 , wherein the image association function realizes that the shooting position of the second image data is indicated on the map in the first image data.

Images