JP6031915B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and a program.

  In recent years, means for acquiring geographic information, such as GPS (Global Positioning System), has become widespread, and there is an apparatus that includes information representing the shooting location of a photograph when the photograph is taken. For example, in a smartphone with a camera or the like, at the time of taking a photo, the current location is measured using GPS, and the positioning result is included in the image information of the taken photo and recorded.

Special table 2009-526302

  However, in devices such as a digital single-lens reflex camera, the GPS device is prepared as an option, but it is not always used. Accordingly, in many cases, image information of a photograph taken by a user of such a device does not include information on a shooting location.

  On the other hand, if the shooting location of a photo is recorded, it can be expected to have various effects such as helping the viewer to recall the memory associated with the photo. Therefore, there is a demand for a technique for adding information on a shooting location later to image information of a photograph in which shooting location information is not recorded.

  Instead of manually adding information here, the information about the position, time, and person is extracted from the photograph using OCR, the position information is extracted from the road sign, and the extracted information is added to the digital data as a tag. Patent Document 1 discloses that data management is performed. However, the place name represented by the photographed road sign board is not necessarily the place of photographing. This is because, for example, a road sign may be a display for providing information about a route to a destination, a point, and the like to a road user such as “20 km to Nihonbashi”.

  The present invention has been made in view of the above circumstances, and provides an image processing apparatus capable of adding information on a shooting location later to image information of a photo in which shooting location information is not recorded. One of its purposes.

  The present invention for solving the problems of the conventional example described above is an image processing apparatus, and means for acquiring image information to be processed and represents a place from the acquired image information to be processed A recognition processing means for recognizing a region including a character string and a character string representing a distance; and character recognition representing a character string representing a location and a character string representing a distance from the recognized region, and based on the character recognition result In addition, an estimation unit that estimates the shooting location of the image information that is the target of the processing and a unit that outputs information on the estimated shooting location are included.

  An image processing apparatus according to an aspect of the present invention includes a unit that acquires image information to be processed, a character string that represents a location, and a character string that represents a distance from the acquired image information to be processed. Recognition processing means for recognizing a region including a plurality of sets of characters, and character recognition of a character string representing a location and a character string representing a distance included in each set from the recognized region, and based on the character recognition result The image processing apparatus includes an estimation unit that narrows and estimates a shooting location range of the image information to be processed, and a unit that outputs the estimated shooting location information.

  In these, the recognition processing means further recognizes a character string representing a route from the acquired image information to be processed, and the estimation means recognizes a character recognition result of the character string representing the recognized route. May be used to estimate the shooting location of the image information to be processed.

  Further, the recognition processing means recognizes an image part including a columnar body from the image information to be processed as a candidate area including a character string representing a place and a character string representing a distance, and the recognized image part An area including a character string representing a place and a character string representing a distance may be searched from among the above.

  Further, the recognition processing means further recognizes the direction indicated by the image when the image information to be processed includes an image indicating the direction, and captures the image based on the direction recognized by the recognition processing means. It may further include means for estimating the direction.

  The image processing apparatus further includes means for referring to the shooting date and time of the image information to be processed and acquiring other image information captured in a predetermined time range including the shooting date and time. Recognizing at least one area including a character string representing a location and a character string representing a distance from other image information, the estimation means recognizes a character string in the recognized area, and the character recognition result And estimating the shooting location of the image information based on the image information, and further using the estimation result of the shooting location of the other image information to estimate the shooting location of the image information subject to the processing. Good.

  Furthermore, a program according to another aspect of the present invention provides a computer that obtains image information to be processed, a character string that represents a location, and a character that represents a distance from the acquired image information to be processed. A recognition processing means for recognizing an area including a column, and character recognition representing a place and a character string representing a distance from the recognized area, and based on the character recognition result, The estimation means for estimating the shooting location of the acquired image information and the means for outputting the information of the estimated shooting location are functioned.

  According to the present invention, information on a shooting location can be added later to the image information of a photo in which shooting location information is not recorded.

It is a block diagram showing the example of a structure of the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the content of the image database which the image processing apparatus which concerns on embodiment of this invention hold | maintains. It is a functional block diagram showing the example of the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the image part which the image processing apparatus which concerns on embodiment of this invention recognizes. It is a flowchart figure showing the example of operation of the image processing device concerning an embodiment of the invention. It is a flowchart showing the example of the process which takes out the area | region containing the character string showing the place name and distance by the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing explaining the example of the presumed operation | movement of the imaging location by the image processing apparatus which concerns on embodiment of this invention. It is another explanatory drawing explaining the example of the presumed operation | movement of the imaging location by the image processing apparatus which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the image processing apparatus 1 according to the embodiment of the present invention includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, a communication unit 15, and an input / output interface 16. It is configured. Here, the control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12.

  Specifically, in the present embodiment, the control unit 11 accepts image information to be processed via the input / output interface 16 and accumulates and stores it in the storage unit 12. The image information to be processed in the present embodiment is image information representing an image captured by a digital camera or the like, and includes metadata such as shooting date information and camera specifying information for specifying the captured camera. Here, the metadata may be so-called Exif (Exchangeable Image File Format) information.

  The control unit 11 according to the present embodiment acquires image information to be processed from the image information stored in the storage unit 12, and a character string representing a location from the acquired image information to be processed And an area including a character string representing a distance. Further, from this recognized area, a character string representing a place and a character string representing a distance are character-recognized, and a shooting location of image information to be processed is estimated based on the character recognition result. Details of the processing of the control unit 11 will be described later.

  The storage unit 12 stores a program executed by the control unit 11. The program may be provided by being stored in a computer-readable recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) and stored in the storage unit 12. The program may be distributed via a network or the like and stored in the storage unit 12. The storage unit 12 also operates as a work memory for the control unit 11.

  In the present embodiment, as illustrated in FIG. 2, the storage unit 12 stores image information and tag information in association with each other as an image database. The tag information as the metadata may include data extracted from the Exif data of the related image information.

  The operation unit 13 may be, for example, a mouse or a keyboard, or may be an input interface such as an infrared remote controller. In an example of the present embodiment, the operation unit 13 is an infrared input interface, and receives information representing the content of the user's instruction operation transmitted from the remote controller that has received the user's instruction operation. The operation unit 13 outputs information representing the contents of the received instruction operation to the control unit 11.

  The display unit 14 is an interface that outputs an image to an external display such as a built-in display or a home television device in accordance with an instruction input from the control unit 11. The communication unit 15 is a network interface, for example, and is connected to the network by wire or wirelessly and outputs information received via the network to the control unit 11. The communication unit 15 receives input of information to be transmitted via the network from the control unit 11 and transmits the information via the network.

  The input / output interface 16 is, for example, an SD card slot or a USB (Universal Serial Bus) interface. The input / output interface 16 reads image information from an SD card, a USB memory, a USB hard disk drive, or the like connected thereto according to an instruction input from the control unit 11 and outputs the image information to the control unit 11.

  Next, the content of the process of the control part 11 of this Embodiment is described. The control unit 11 executes the program stored in the storage unit 12 to functionally illustrate the image information acquisition unit 21, the recognition processing unit 22, the estimation unit 23, the information, as illustrated in FIG. It operates as including the output unit 24.

  Here, the image information acquisition unit 21 refers to, for example, tag information of the image information stored in the storage unit 12 and selects image information that does not include shooting location information as a candidate for processing. Then, the image information acquisition unit 21 acquires one of the selected pieces of image information as a processing target (reads out from the image database in the storage unit 12).

  The recognition processing unit 22 executes a process of recognizing an area including a character string representing a place and a character string representing a distance for the image information read from the image database. Specifically, this process is a process for recognizing a so-called road sign, and is performed as follows as an example.

In other words, as shown in FIG.
(A1) It is on a general road and represents the direction, direction and distance (no route display),
(A2) It is on a general road and represents the direction, direction and distance (with a route display),
(B) on a highway, indicating direction and distance,
(C) Notice of direction and direction,
(D) display direction or lane,
(E) A notice on the expressway on exits, service areas, tollgates,
(F) Some are famous or represent major points. For example, in Japan, these are stipulated in Appendix 2 of the instructions on road signs, lane markings, and road markings. In countries, for example, Manual on Uniform Traffic Control Devices (provided by the Federal Highway Administration in the United States) Some manuals (MUTCD) have similar specifications.

  As described in these regulations, the background color of the road sign is predetermined. Therefore, as shown in FIG. 5, the recognition processing unit 22 determines that the pixel value P represents a pixel value (a plurality of pixel values representing the predetermined color) from the pixels included in the image information acquired as the processing target. , Each of which is Q1, Q2,...), A pixel having a distance below a predetermined threshold in the color space is extracted (S1). This extraction result is, for example, as shown in (S1) of FIG. Here, the color space is a three-dimensional space that can be defined by each value of RGB (Red Green Blue), for example, and the distance between the pixel values may be defined by the Euclidean distance in this color space.

  Further, the recognition processing unit 22 performs edge detection image processing on the image information acquired as the processing target, and obtains binarized contour image information ((S2), (S2) in FIG. 6). . Further, the recognition processing unit 22 performs a labeling process on the pixel group in the region surrounded by the outline ((S3), (S3) in FIG. 6). Since this labeling process can use a widely known process for tracking the contour line and associating different identification information for each region surrounded by the contour line, detailed description thereof will be omitted here. The recognition processing unit 22 includes, for each region including the labeled pixel group, unique identification information for each region and information for specifying the region (image information having the same size as the image information to be processed, the labeled pixel group Is set to a significant pixel (for example, black), and a pixel other than the labeled pixel group is generated as a non-significant pixel (for example, mask image information) ((S4), (S4) in FIG. 6). ).

  The recognition processing unit 22 further executes a process of recognizing the outer shape for each region generated in the process (S4) (S5). As the process for recognizing the outer shape, for example, a widely known method such as a process for classifying and recognizing the outer shape using SOM (Self Organizing Maps) or the like can be adopted, and detailed description thereof is omitted here. The recognition processing unit 22 classifies the outer shape of each region into a rectangle, an inverted triangle, a hexagon, an arrow shape, and the like, and uses the classification result as a recognition result of the outer shape of the region. Then, as shown in FIG. 6 (S6), the recognition processing unit 22 stores the identification information unique to each region, the information for specifying the region, and the recognition result of the outer shape of the region in association with each other as a region database. Store in the unit 12 (S6). The recognition processing unit 22 finds the coordinate values of the pixels corresponding to the four corners of the region recognized as a rectangle, and obtains a homography matrix from this.

  Next, the recognition processing unit 22 counts the number of pixels extracted in the process (S1) among the pixels included in the area for each area found in the process (S4) (S7). That is, the number of pixels having a pixel value corresponding to the background color of the road sign in the region surrounded by the outline is examined. And the recognition process part 22 selects the area | region where this pixel number exceeds the predetermined threshold value (S8). The recognition processing unit 22 performs projective transformation on the pixel portion in the selected area of the image information to be processed using the homography matrix obtained from the outer shape of the area (S9).

  The recognition processing unit 22 obtains a part of the image information obtained as a result of the projective transformation (the image of the object imaged in the part that is the background color of the road sign in the image information that is the object of processing). , An image converted so as to be viewed from the front, hereinafter referred to as a partial image (S10) in FIG. 6), identification information of the area from which the partial image is extracted, and information indicating the outer shape of the area And stored in the storage unit 12 (S10). When the image information to be processed includes a plurality of regions including pixels corresponding to the background color of the road sign exceeding the threshold value, the partial image corresponding to each region is stored in the storage unit 12.

  A plurality of character strings representing directions, directions, and distances are indicated by white lines such as (a1) on a general road shown in FIG. 4 and indicating directions, directions, and distances (no route indications). In such a case, the image of each divided part is output (as shown in (S10) of FIG. 6).

  The estimation unit 23 performs a character recognition process (OCR process) on the partial image Pi generated by the recognition processing unit 22 (the partial image for each region i (i = 1, 2,...) Is Pi). Thereby, the character recognition result for every partial image is obtained. The estimation unit 23 checks whether or not the character recognition result Ci (i = 1, 2,...) For each partial image Pi includes a character string representing a distance. This may be done by checking whether the character string contains numbers.

The estimation unit 23 selects a partial image Pi in which a character string representing a place name (location) and a distance is included in the character recognition result Ci, and executes the following processing. First, the estimation unit 23 acquires a place name portion Li by using a character string Ci recognized from the selected partial image Pi and referring to a place name dictionary in which place name character strings are listed in advance. In addition, the estimation unit 23 extracts a numeric part from the character string Ci as a character string Di representing the distance. The estimation unit 23 estimates the shooting location based on the combination of the place name portion Li and the character string Di representing the distance.
Here, when referring to the place name dictionary, the place name portion Li may be extracted under the condition of partially matching the character string Ci recognized from the partial image Pi. For example, the place name “Hibiya” may be estimated and acquired as the place name portion Li by partial matching with “Hibiya” instead of the entire “Hibiya”.

  The estimation unit 23 refers to a database (referred to as a place name database) in which a character string Li representing a place name is associated with geographical coordinate information (referred to as a place name database), and is associated with the character string Li representing a place name. (Latitude / longitude information) Ti is acquired. In Japan, such a place name database includes location reference information provided by the Ministry of Land, Infrastructure, Transport and Tourism.

  The estimation unit 23 generates a virtual circle representing the range of the distance information Di on the map with the latitude and longitude Ti as the center. The estimation unit 23 refers to the place name database, and acquires character information of the place name associated with the latitude and longitude values within the overlapping range of the generated virtual circles. That is, the estimation unit 23 narrows down and estimates the range of the shooting location of the image information to be processed by using a plurality of virtual circles. Here, when a plurality of place name character information associated with the latitude and longitude values within the overlapping range of the generated virtual circles are found, the estimating unit 23 acquires the plurality of place name character information. To do. The place name represented by the character information acquired here is the estimation result of the shooting place. Or this estimation part 23 may output the value of the latitude longitude in the range which the said produced | generated virtual circle overlaps as an imaging | photography place estimation result as it is. For example, the value of the latitude and longitude at the center of gravity of the overlapping range of the generated virtual circle may be used as the shooting location estimation result.

  The information output unit 24 associates the character information of the place name obtained by the estimation unit 23 as tag information representing the shooting location of the image information to be processed, and associates it with the image information to be processed, in the storage unit 12. Record in the image database and update the image database.

  Further, when there is only one character string Li, Di representing the place name and distance extracted by the estimation unit 23, only one virtual circle is obtained, so there is no overlapping range of the virtual circles. In this case, the estimation part 23 should just acquire the character information of the place name linked | related with the value of the latitude longitude within the predetermined range from the circumference of a virtual circle with reference to a place name database. Alternatively, the estimation unit 23 uses the latitude / longitude values (may be the latitude / longitude values at a plurality of points on the circumference) existing on the circumference of the virtual circle as information representing the shooting location of the image information as it is. It may be output.

  Furthermore, the estimation unit 23 may ignore the partial image Pi that does not include the character string (number) representing the distance in the character recognition result Ci, without using it in the processing.

  Further, as illustrated in (a2) of FIG. 4, for a route display (R), the estimation unit 23 recognizes a character string representing the route included in the route display and represents this route. The shooting location of the image information may be estimated by further using the character recognition result of the character string.

  Specifically, when this processing is performed, the control unit 11 has a region whose outer shape is predetermined as a shape representing a route among regions included in the region database generated as the processing of the recognition processing unit 22 (for example, in Japan). If there is an inverted triangle or hexagon, select the area. And the recognition process part 22 counts the number of the pixels extracted by the process (S1) of FIG. 5 among the pixels contained in an area | region. That is, the number of pixels having a pixel value corresponding to the background color of the road sign in the region surrounded by the outline is examined. In the recognition processing unit 22, the estimation unit 23 performs the character recognition processing on a region where the number of pixels exceeds a predetermined threshold value (which may be different from the threshold value in the case of a rectangle). In Japan, national road numbers are numbers that are white in an inverted triangle and numbers are white, and numbers that are white in hexagons are numbers for local roads such as prefectural roads. In the case of a prefectural road, the prefectural name is also displayed.

Therefore, here, the result of the character recognition by the estimation unit 23 is
(1) a numeric character string recognized from an area associated with information representing an inverted triangular shape;
(2) a number and a character string of a prefecture name (region name) recognized from an area associated with information representing a hexagonal shape;
Either or both. That is, since there are sections where the national road and the local road overlap, if both of these (1) and (2) are recognized, the image is captured in the overlapping section, and the estimation unit 23 performs the overlap. The latitude / longitude information in the section is obtained, and the latitude / longitude information or the character string of the place name associated with the latitude / longitude information closest to the latitude / longitude information is obtained and used as the estimation result of the shooting location. According to this, the estimation accuracy of the shooting location can be further improved.

  The estimation unit 23 represents (1) a national road number for a character string of numbers recognized from an area associated with information representing an inverted triangle shape, and (2) a hexagonal shape. With respect to the number and the character string of the prefecture name recognized from the area associated with the information to be represented, the recognized number represents the number in the prefectural road represented by the character string of the name of the prefecture.

  Thereby, the estimation part 23 acquires the information which specifies a road from image information. Then, the shooting location is estimated on the assumption that the photographer was present on the road specified by this information. That is, as illustrated in FIG. 7, the estimation unit 23 uses the information on the place name and distance recognized from the image information, and recognizes the recognized national road or the cut off by the circumferences of a plurality of virtual circles generated on the map. Find the prefectural road segment (may be a curve). Of these line segments, the character information of the place names associated with the latitude and longitude values on the shortest line segment or within a predetermined distance range from the shortest line segment is obtained by referring to the place name database. To do. The reason why the shortest line segment is used is to shorten the distance to the center of each virtual circle as much as possible. Note that character information of a plurality of place names may be acquired. The place name represented by the character information acquired here is the estimation result of the shooting place. Specifically, in the example shown in FIG. 7, a virtual circle A having a radius of 10 km from “Nihonbashi” and a virtual circle B having a radius of 7 km from “Hibiya” are generated on the map. The road R identified as having a photographer is sandwiched between one side of the virtual circle A and one side of the virtual circle B when these virtual circles A and B partially overlap as shown in FIG. A portion r1 sandwiched between one side of the virtual circle B and the other side of the virtual circle A, and a portion r3 sandwiched between the other side of the virtual circle A and the other side of the virtual circle B. Here, since the shortest line segment is the part r3, the estimation unit 23 uses the point included in the part r3 as the estimation result of the shooting location. Further, as already described, the value of the latitude and longitude on this line segment (for example, the value of the latitude and longitude corresponding to the midpoint on the line segment) may be used as the estimation result of the shooting location.

  In addition, the control unit 11 further includes a pixel corresponding to the background color of the road sign in the region in which the image information to be processed includes an area whose outer shape is an image indicating a direction such as an arrow. When the number of pixels in the value exceeds a predetermined threshold (which may be different from the threshold for other external shapes), the direction indicated by the image, for example, the image In the case of an arrow, the direction of the arrow may be recognized, and the shooting direction may be estimated based on the direction (the direction of the arrow, etc.).

  Specifically, in this case, the recognition processing unit 22 of the control unit 11 includes a region whose outer shape is an arrow, and has a pixel value corresponding to the background color of the road sign in this region. Direction of the arrow indicated by the area (referred to as the target arrow area) when the number of pixels exceeds a predetermined threshold (which may be different from the threshold for other external shapes) Is defined as follows. That is, the recognition processing unit 22 matches the image information with the direction (rotation direction) at the time of shooting. For example, if the Exif information is included in the image information, a widely known process such as referring to the orientation information can be used.

  The recognition processing unit 22 adjusts the direction of the attention arrow region (if there are a plurality of attention arrow regions if there are a plurality of directions), for example, upward, left, right, lower four directions or upward, upper left, left It is classified into eight directions: left, lower left, lower, lower right, right, upper right. This classification can employ a widely known method such as performing learning.

  The recognition processing unit 22 generates a rectangle circumscribing the target arrow region, and the target arrow region in the direction classified into any one of the four directions of upper, left, right, and lower is classified into the rectangle. The side of the circumscribed rectangle on the direction side (for example, the upper side of the circumscribed rectangle if the target arrow region is classified upward), that is, the side in the direction of the arrow in the target arrow region The coordinates of the middle point are taken out as the target point coordinates. In addition, the recognition processing unit 22 regards the attention arrow region in the direction classified as one of the upper left, lower left, lower right, or upper right, the vertex of the circumscribed rectangle on the classified direction side of the rectangle, that is, The coordinates of the vertex in the direction of the arrow in the attention arrow area are extracted as the attention point coordinates.

  When the attention processing unit 22 extracts the attention arrow area and the attention point coordinates, the estimation unit 23 performs the character recognition processing on each area stored in the area database, and as a result, the barycentric coordinates of the obtained character string. Find the attention arrow area of the attention point coordinates closest to. Then, the character string is stored in the storage unit 12 in association with the orientation information classified for the found attention arrow region.

  The estimation unit 23 acquires, for example, the place name represented by the character string associated with the upward direction with reference to the place name database, and the photographer faces the direction of the acquired place name (the shooting direction is the acquired place name). Direction).

  In this case, the information output unit 24 uses the place name character information obtained by the estimation unit 23 as the tag information indicating the shooting location and the shooting direction of the image information to be processed as tag information indicating the shooting location and the shooting direction. The image information is recorded in the image database of the storage unit 12 in association with the image information, and the image database is updated.

The image processing apparatus 1 according to the present embodiment basically includes the above configuration and operates as follows. The image processing apparatus 1 selects image information of a photograph in which shooting location information is not recorded from among image information captured from an SD card or the like and stored in the storage unit 12, and uses the selected image information as a processing target. The process of recognizing the road sign is performed. When a road sign is recognized here, a character string representing a place name and a distance is acquired from the road sign by character recognition. Here, for example, when the road sign illustrated in FIG.
L1: “Kokubunji”, D1: “4”
L2: “Chofu”, D2: “5”
L3: “Tachikawa”, D3: “7”
Character strings Li and Di representing such place names and distances are acquired.

  The image processing apparatus 1 sets a virtual circular shape (virtual circle) with a radius D1: “4” kilometers around the latitude and longitude corresponding to L1: “Kokubunji” on the map with reference to the place name database. . In addition, when there are a plurality of “Kokubunji” in each place, a plurality of virtual circles may be set around the latitude and longitude corresponding to each. Similarly, the image processing apparatus 1 also applies to other recognition results, a virtual circle having a radius D2: “5” kilometers centered on the latitude and longitude corresponding to L2: “Chofu” on the map, and L3: “ A virtual circle with a radius D3: “7” kilometers centered on the latitude and longitude corresponding to “Tachikawa” is set.

  Then, the image processing apparatus 1 finds a region where a plurality of virtual circles overlap. That is, if there is a region that overlaps with another virtual circle in the region within each virtual circle, the region is found. Here, an area where virtual circles centering on the points of “Kokubunji”, “Chofu”, and “Tachikawa” intersect is found (FIG. 8). Here, for example, it is assumed that there are regions that intersect each other in the vicinity of “Fuchu” in “Tokyo”. In FIG. 8, place names registered in the place name database are respectively displayed. Therefore, the image processing apparatus 1 presumes that this image information was taken in “Tokyo, Fuchu”, and associates the information of the shooting location as a result of this estimation with the image information as tag information in the storage unit 12. Store.

  Note that the present embodiment is not limited to the examples described so far. First, in the description so far, the information on the shooting location is estimated for each piece of image information to be processed. However, for example, the shooting location for one of a plurality of pieces of image information taken in about 15 minutes is estimated. If this information can be estimated, it can be estimated that the other image information of the plurality of image information is also in the same shooting location (it can be assumed that the distance that can be moved in 15 minutes is not large). . Therefore, the control unit 11 of the image processing apparatus 1 according to the present embodiment may perform the following processing.

  That is, after selecting the image information to be processed, the control unit 11 refers to the shooting date / time information of the selected image information. And the other image information imaged in the predetermined time range including the said imaging | photography date is acquired from the memory | storage part 12. FIG. At this time, the control unit 11 is the same camera as the camera that captured the image information that is other image information captured in a predetermined time range including the shooting date and time and that has been processed. It is good also as acquiring the other image information image | photographed.

  For each of the image information to be processed and the acquired other image information, the control unit 11 recognizes a road sign and acquires a character string representing a place name and a distance from the recognized road sign by character recognition. The process to perform is performed. And the control part 11 sets the virtual circle centering on the latitude longitude corresponding to each place name on the map about the character string showing the place name and distance obtained from these several image information, and is made into several virtual Find areas where circles overlap. The control unit 11 obtains place name information associated with the latitude and longitude in the found area with reference to the place name database, and estimates that the obtained place name is a shooting place. The control unit 11 stores the information on the shooting location as a result of the estimation in the storage unit 12 in association with the image information to be processed as tag information. Similarly, the other image information acquired at this time may store the information on the shooting location as a result of the estimation in the storage unit 12 in association with the tag information.

  In addition, the control unit 11 performs processing for recognizing a road sign for each piece of image information to be processed and other acquired image information, and character recognition that represents a place name and a distance from the recognized road sign by character recognition. The process of acquiring may be performed, and the process of estimating the shooting location from the information indicating the place name and distance acquired for each piece of image information may be performed. In this case, a different shooting location may be estimated for each piece of image information. For example, the control unit 11 calculates an average value of latitude and longitude information of the shooting location estimated for each piece of image information, acquires information on a place name associated with the latitude and longitude closest to the average value from the place name database, and acquires the information. It may be estimated that the place name is the shooting place.

  In addition, the image processing apparatus 1 according to the present embodiment recognizes not only the road signs exemplified so far, but also, for example, utility poles, station name signs of stations, and the like, and recognizes place names, station names, telephone numbers, and the like included therein. However, it may be used for the process of estimating the shooting location.

  Furthermore, for example, some signs attached to utility poles have a distance clearly indicated along with a character string representing the place. An example is a character string such as “XX clinic, 30 meters away from here”. Therefore, the control unit 11 recognizes an image part including a columnar body for image information to be processed, and recognizes and acquires a character string representing a place and a character string representing a distance from the recognized image part. May be.

  Here, the character string representing the location may be, for example, a character string representing a facility such as “XX clinic” or a telephone number. The control unit 11 acquires the latitude and longitude information corresponding to the character strings representing these places with reference to a database used in a car navigation system, for example. The character string representing the distance is recognized by including the unit (meter “m” or kilometer “km”). Note that there may be a plurality of databases for each creation date. The control unit 11 refers to the shooting date and time of the image information to be processed, selects the database whose creation date and time is closest to the shooting date and time, refers to the selected database, and corresponds to the character string representing the place Get the latitude and longitude information. In this way, it is possible to perform processing in consideration of a case where the correspondence relationship between the name of the store to be a landmark and the latitude / longitude information corresponding thereto is changed.

  In the following, similarly to the case where the character string of the place name and the distance is recognized from the road sign, the control unit 11 uses the character string representing the distance centered on the latitude and longitude corresponding to the character string representing the character-recognized place. A virtual circle with a defined radius may be set on the map. In this case as well, when a plurality of virtual circles are obtained from the image information to be processed (or other image information captured in a predetermined time range including the shooting date and time of the image information to be processed) A region where a plurality of virtual circles overlap may be found. In this case, it is considered that the area is smaller than the latitude / longitude distribution of the place name in the place name database, so the place name registered in the place name database in relation to the latitude / longitude closest to the found area is estimated as the shooting location. You may acquire as a result. If multiple overlapping areas of virtual circles are found, select the area with the largest number of overlapping virtual circles and select the place name registered in the place name database in association with the latitude and longitude closest to the selected area. You may acquire as a presumed result of a photography place. If there is no overlapping area of virtual circles, each virtual circle is enlarged by a predetermined radius, and if overlapping areas are created, select the area with the largest number of overlapping virtual circles. A place name registered in the place name database in association with the latitude / longitude closest to the selected area may be acquired as an estimation result of the shooting place. In addition, if there is no area where the virtual circles overlap, the place name is associated with the latitude and longitude closest to the midpoint of the line connecting the centers of the virtual circles or the center of gravity of the polygon connecting the centers of the virtual circles. You may acquire the place name registered into the database as an estimation result of a photography place. Furthermore, in this case as well, the latitude and longitude information in the selected area, such as the center of gravity of the selected area, may be directly acquired as the estimation result of the shooting location, or the midpoint of the line segment that connects each center of each virtual circle Alternatively, in the case of using a polygonal center of gravity formed by connecting the centers of the virtual circles, information on the latitude and longitude of the position corresponding to the center of gravity may be acquired as the estimation result of the shooting location.

  When the control unit 11 obtains only one virtual circle from the image information to be processed (or other image information captured in a predetermined time range including the shooting date and time of the image information to be processed). From the circumference of this virtual circle, the place name registered in the place name database in association with the latitude and longitude within the predetermined range may be acquired as the estimation result of the shooting place.

  Further, in the above description, the processing target image information is a still image, but the processing target image information may be a moving image. Since the image information of the moving image is the same as the image information of the still image arranged in order of shooting time, the image information of the still image in the moving image is extracted at a predetermined time interval, and each is processed. It is only necessary to take out as information and obtain a shooting location estimation result for each piece of image information. Alternatively, each piece of image information of a still image in a moving image is processed, for example, a still image including a predetermined number or more of pixel values of colors close to the background color of a road sign is extracted and taken out as image information to be processed. It is also possible to obtain a shooting location estimation result for the extracted image information.

  In these cases, a logical sum set of the shooting location estimation results obtained for a plurality of image information extracted from the moving image is generated, and the shooting location information included in the generated logical sum set is associated with the image information of the moving image. And stored in the storage unit 12. The moving speed at the time of moving image shooting can also be estimated by using the information of each shooting date and time of a plurality of pieces of image information taken out as processing targets from the moving image. In this case, furthermore, it was not extracted from the moving image (estimated shooting location was found) between the multiple pieces of image information (estimated image information) that were extracted from the moving image as the processing target and the shooting location was estimated. For the image information (unestimated image), the estimated image information that is the closest shooting date and time before and after the shooting date and time of the unestimated image is extracted, and the two estimated image information items that have been extracted The estimation processing may be performed on the assumption that there is a shooting location of an unestimated image between the estimated shooting locations using the information of the shooting location estimated for each of the above.

として推定できる。 For example, the shooting date and time of the estimated image information that is the closest shooting date and time before the shooting date and time T of the unestimated image is T1, and the estimated latitude and longitude values are (LAT1, LON1). Also, assuming that the shooting date / time of the estimated image information that is the closest shooting date / time after the shooting date / time of the unestimated image is T2, and the estimated latitude / longitude value is (LAT2, LON2) ( T1 <T <T2), and the estimated shooting location for the non-estimated image information is

Can be estimated.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 11 Control part, 12 Storage part, 13 Operation part, 14 Display part, 15 Communication part, 16 Input / output interface, 21 Image information acquisition part, 22 Recognition processing part, 23 Estimation part, 24 Information output part

Claims (8)

Means for acquiring image information to be processed;
Recognition processing means for recognizing a region including a plurality of character strings representing a place and a character string representing a distance from the acquired image information to be processed;
A character string representing a place and a character string representing a distance included in each set are character-recognized from the recognized area, and based on the character recognition result, the range of the shooting location of the image information to be processed Estimating means for narrowing down and estimating,
Means for recording the estimated shooting location information in an image database in association with the image information to be processed;
Only including,
The estimating means sets on the map a plurality of virtual circles centered on a latitude and longitude corresponding to a character string representing a place included in each of the character string sets and having a radius determined by a character string representing a distance. An image processing apparatus that estimates the place name registered in the place name database in association with the latitude and longitude closest to the overlapping area of the plurality of virtual circles as shooting place information . The image processing apparatus according to claim 1 ,
The recognition processing means further recognizes a character string representing a route from the acquired image information to be processed,
The said estimation means is an image processing apparatus which estimates the imaging | photography location of the image information used as the said process object further using the character recognition result of the character string showing the recognized route. Means for acquiring image information to be processed;
Recognition processing means for recognizing a region including a character string representing a place and a character string representing a distance from the acquired image information to be processed;
An estimation means for recognizing a character string representing a place and a character string representing a distance from the recognized area, and estimating a shooting location of the image information to be processed based on the character recognition result; ,
Means for outputting information of the estimated shooting location,
The recognition processing means further recognizes a character string representing a route from the acquired image information to be processed,
The said estimation means is an image processing apparatus which estimates the imaging | photography location of the image information used as the said process object further using the character recognition result of the character string showing the recognized route. Means for acquiring image information to be processed;
Recognition processing means for recognizing a region including a plurality of character strings representing a place and a character string representing a distance from the acquired image information to be processed;
A character string representing a place and a character string representing a distance included in each set are character-recognized from the recognized area, and based on the character recognition result, the range of the shooting location of the image information to be processed Estimating means for narrowing down and estimating,
Means for outputting information of the estimated shooting location,
The recognition processing means further recognizes a character string representing a route from the acquired image information to be processed,
The said estimation means is an image processing apparatus which estimates the imaging | photography location of the image information used as the said process object further using the character recognition result of the character string showing the recognized route. The image processing apparatus according to any one of claims 1 to 4 , wherein:
The recognition processing means is a candidate for a region including a character string representing a location and a character string representing a distance,
Recognizing an image portion including a columnar body from the image information to be processed,
An image processing apparatus that searches a region including a character string representing a place and a character string representing a distance from the recognized image portion. An image processing apparatus according to any one of claims 1 to 5 ,
The recognition processing means further recognizes the direction indicated by the image when the image information to be processed includes an image indicating the direction,
An image processing apparatus further comprising means for estimating a shooting direction based on the direction recognized by the recognition processing means. The image processing apparatus according to any one of claims 1 to 6 ,
Means for acquiring other image information captured in a predetermined time range including the shooting date and time, referring to the shooting date and time of the image information to be processed;
The recognition processing means recognizes at least one area including a character string representing a place and a character string representing a distance from the other image information,
The estimation means character-recognizes a character string in the recognized area, estimates a shooting location of other image information based on the character recognition result, and calculates an estimation result of the shooting location of the other image information. And an image processing apparatus for further estimating the shooting location of the image information to be processed. Computer
Means for acquiring image information to be processed;
Recognition processing means for recognizing a region including a plurality of character strings representing a place and a character string representing a distance from the acquired image information to be processed;
A character string representing a place and a character string representing a distance included in each set are character-recognized from the recognized area, and based on the character recognition result, the range of the shooting location of the image information to be processed Estimating means for narrowing down and estimating,
Means for recording the estimated shooting location information in an image database in association with the image information to be processed;
To function as,
When functioning as the estimating means, a plurality of virtual circles having a radius defined by a character string representing a distance centered on a latitude and longitude corresponding to a character string representing a place included in each of the character string sets are mapped A program that is set above and estimates the place name registered in the place name database in association with the latitude / longitude closest to the overlapping area of the plurality of virtual circles as shooting place information .

Images