KR20080028698A - Method and system for mapping image objects in photo to geographic objects - Google Patents

Abstract

A method and a system for mapping an image object of a photo to a geographic object are provided to map the image object of the photo to the geographic object by using map and image information, and map knowledge for the object included or the geographic object in the photo to a map. An input unit(110) receives photo data. A database unit(120) stores/manages the photo and map data. A mapping unit(130) maps an image object included in the photo data to a geographic object included in the map data by analyzing the photo data and referring to the map data. The input unit includes an image capturing sensor(111), a location measuring sensor(112), a direction measuring sensor(113), an image encoder(117) encoding photo information by using measured location and direction, a photo information encoder(118), a tilt measuring sensor(114), a distance measuring sensor(115), and a metadata generator(119) generating metadata of the photo by using tilt or distance data. The database unit includes a photo database(121) storing/managing the photo data, the photo information, and the metadata, a map database(123), and a mapping database(122) storing/managing the photo data mapped to the map data.

Description

Method and system for mapping image objects in photo to geographic objects}

1 is a diagram illustrating a configuration of a system for mapping an image object in a photo to a geographic object according to an embodiment of the present invention.

2 is a flowchart illustrating a method of mapping an image object in a photo to a geographic object according to another exemplary embodiment of the present invention.

3 is a diagram illustrating an example of a process of classifying photographs based on contents according to the present invention.

4 is a diagram illustrating an example of inferring a candidate geographic object in a photograph according to the present invention.

5 is a diagram showing perspective transformation in the present invention.

6 is a diagram illustrating an example of a camera shape model in the present invention.

It is a figure which shows an example of calculating the angle of a visual field in this invention.

8 is a diagram illustrating another example of inferring candidate geographic objects in a photograph according to the present invention.

9 illustrates another example of inferring candidate geographic objects in a photograph according to the present invention.

FIG. 10 is a diagram for illustrating another example of inferring candidate geographic objects in a photograph according to the present invention. FIG.

11 is a diagram showing an example of inferring and mapping candidate geographic objects in a photograph according to the present invention.

12 is a diagram illustrating an example of a geographic object mapping procedure of a photograph according to the present invention.

It is a figure which shows an example which mapped the geographic object of a photograph in this invention.

14 is a diagram illustrating a mapping result of a photo image object and a geographic object according to the present invention.

FIG. 15 is a diagram for one example of mapping a building name to an image object of a photograph according to the present invention.

Fig. 16 is a diagram showing an example of image search according to information search in the present invention.

<Explanation of symbols for the main parts of the drawings>

100: geographic object mapping system of photographic image objects

110: input unit 120: database

130: mapping processing unit

The present invention relates to a method and a system for mapping an image object in a picture to a geographic object. More particularly, the present invention relates to analyzing a picture to determine a location and a photographing angle at which the picture is taken and to refer to the image object in the picture by referring to the geographic data. A method and system for mapping a geographic object of a map.

Recently, as multimedia devices such as digital cameras, camera phones, camcorders, and the like are developed, various services using multimedia data photographed through the multimedia devices are increasing.

In addition, various search services using the Internet are provided, and as the search query paradigm goes beyond the image, the user needs to retrieve and obtain the context information contained in the image itself. It is a trend that is becoming a pattern used as a search query.

To do this, we need to analyze the contents of an image and index contextual information about each object in the image. Various techniques can be used to analyze the contents of an image. For example, CBIR (Content-Based Image Retrieval) or Object Segmentation & Recognition (FB) and Face Detection & Recognition can be used. There is a person search. In addition, it is possible to analyze the additional information of the image, for example, by analyzing Exif (Exchangeable Image File Format) metadata from the cluster and using the same. In the case of photographs, indexing / search using temporal and spatial information recorded in Exif data has been attempted.

Among these techniques, techniques using only image analysis have limitations in that they do not provide satisfactory performance by the prior art. In this context, due to the recent development and universal use of GPS (Global Positioning System) devices, it is possible to record information by identifying the camera view point at the time of capturing a picture and the location of an object to be photographed. Later, applications to search / navigation / clustering using the same have been attempted.

Conventional photo analysis system performs a function of generating an annotation by resolving physical and cultural names of an object taken in the picture, but this provides only the location information where the picture is captured, but the object of the picture. There is no accurate mapping with the area. Therefore, the conventional photo analysis system can only estimate the list of geographic object names contained in the photograph by analyzing with interest the physical and cultural features contained in the photograph in order to annotate the photograph. There is a problem that implies many errors for.

The present invention has been made to solve the above problems of the prior art, and provides a method and system for mapping a video object in a photo with a geographic object using information on the geographic object in the photo using map information and image information. The purpose is to.

Another object of the present invention is to provide a method and a system for mapping an image object in a photo to a geographic object for providing a user by mapping a map with what is included in the photo and what is in a location in the photo.

Another object of the present invention is to map a building name / place name corresponding to the corresponding position of the picture to map the image object in the picture and the geographic object so that the user can easily see the picture and the building or location included in the picture And to provide the system.

It is still another object of the present invention to provide a method and a system for mapping an image object in a photo to a geographic object that enables additional search of the photos associated with the geographic name when the object region of the photo is designated and searched.

In order to achieve the above object and solve the problems of the prior art, the present invention analyzes the input unit for receiving photo data, a database for recording and maintaining the photo data and map data and the photo data, and referring to the map data. The present invention provides a system for mapping an image object in a photo to a geographic object including a mapping processor configured to map the image object of the photo data and the geographic object of the map data.

According to another aspect of the present invention, a method of mapping an image object in a photo to a geographic object includes receiving photo data, recording and maintaining the photo data and map data in a database, and storing the input photo data. Analyzing and mapping an image object of the photo data and a geographic object of the map data with reference to the map data.

In the present specification, a geographic object refers to an entity in the real world having a spatial property such as a location, a shape, a spatial relationship, and a non-spatial property such as a place name, a building name, and the like. An object (subject) refers to an object that is mapped to a photo.

Hereinafter, a method and a system for mapping an image object in a photo to a geographic object according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of a system for mapping an image object in a photo to a geographic object according to an embodiment of the present invention.

Referring to FIG. 1, the geographic object mapping system 100 of a photographic image object may include an input unit 110 to map information about a geographic object in a photograph with corresponding image object regions in the photograph using map information and image information. Database 120 and mapping processing unit 130 is included.

The input unit 110 receives photo data, photo information data, or meta data. That is, when the user takes a picture, the input unit 110 receives various input information associated with the taken picture from various sensors along with the image of the taken picture. The input information may include geographic location information, direction information, focal length, charge coupled device (CCD) size, camera tilt angle information, angle of view, subject distance, camera Various situation information generated as the picture is taken, such as zoom information. To this end, the input unit 110 includes an image capturing sensor 111, a position measuring sensor 112, a direction measuring sensor 113, a tilt measuring sensor 114, a distance measuring sensor 115, an image generator 116, and metadata. Generator 119.

The image capturing sensor 111 performs a sensing operation for capturing an image in the photographing space.

The position measuring sensor 112 measures the position of the photographing space. The position measuring sensor 112 may include a GPS sensor as a representative example, but is not limited to the GPS sensor but may include all means for measuring the position of the photographing space. The position measuring sensor 112 may determine the position of the photographing space by measuring, for example, longitude, latitude, and altitude of the photographing space as geographic location information of the photographing space. Can be.

The direction measuring sensor 113 measures the direction of the image photographed in the photographing space. That is, the direction measuring sensor 113 may determine whether the image of the photograph taken in the photographing space exists in which direction of east, west, north, and south by using a compass as a representative example.

The tilt measuring sensor 114 measures the tilt of the photographing angle of the image when the photograph is taken. That is, the tilt measurement sensor 114 is a representative example to measure the tilt of the camera when taking the picture using a gyro (Gyro), and outputs the measured tilt data.

The distance measuring sensor 115 measures the distance between the subject of the picture and the photographing means for taking the picture when the picture is taken. That is, the distance measuring sensor 115 is a representative example of measuring the distance between the subject of the picture and the image taking sensor 111 for taking the picture when the picture is taken by using a laser (laser), the measured distance Output the data.

The input unit 110 may or may not include the tilt measurement sensor 114 or the distance measurement sensor 115 as necessary.

The image generator 116 includes an image encoder 117 and a photo information encoder 118 for generating photo data composed of the encoded image and the photo information of the encoded image. Image encoder 117 encodes the sensed image. That is, the image encoder 117 may encode the sensed image in a JPG format which is an example of a representative image format. Photo information encoder 118 encodes photo information data using the measured position and the direction information. In addition, the picture information encoder 118 encodes picture information data such as camera zoom information, shutter information, illuminance information, etc. when the picture is taken. For example, Exif data including information of a photo taken by a digital camera may be a representative example of the photo information data.

The metadata generator 119 generates metadata of the photo using the gradient data or the distance data.

The database 120 records and maintains the photo data and the map data, and includes a photo database 121, a map database 122, and a mapping database 123.

Photo database 121 records and maintains image data of the photo, geographic information data, and the metadata of the photo, map database 1220 records and maintains the map data, and mapping database 123 maps the map. Record and maintain photo data mapped to data.

The mapping processor 130 analyzes the photo data and maps the image object of the photo data and the geographic object of the map data with reference to the map data. The mapping processor 130 includes a parser 131, an image decoder 132, a classifier 133, an extractor 134, an inference device 135, an area mapper 136, and an index generator 137.

A parser 131 parses the photo data and the metadata. That is, the parser 131 parses the image data of the photo, the photo information data, and the metadata from the photo data. The photo information data may include latitude, longitude, altitude, direction, focal length, or CCD size information, and the metadata may include tilt or subject distance information of a camera. The image decoder 132 decodes the image data of the picture by reflecting the parsing result.

The classifier 133 analyzes image data of the decoded picture and classifies it into a non-target picture or a target picture. The non-target picture is a picture that does not require geographic name mapping, such as a person or affix, and the target picture is a picture that requires geographic name mapping, such as an urban landscape. That is, the classifier 133 analyzes the photo information data and the image data of the decoded photo to classify the photograph into the target photograph when the photograph is an urban landscape, and when the photograph is an indoor photograph or a macro photograph, Filter by classifying as non-target picture.

The extractor 134 analyzes image data of the decoded picture based on content to extract a geographic area of the picture. That is, the extractor 134 extracts the image object region in the picture from the image data of the map through an image segmentation technique.

The reasoning unit 135 infers a candidate geographic object in the target photograph by referring to the map data.

4 illustrates an example of inferring candidate geographic objects in a picture, FIG. 5 illustrates a perspective transformation, and FIG. 6 illustrates an example of a camera shape model.

Referring to FIG. 4, reference numeral 410 is an example of a photograph of an urban landscape photographed in a city center, and includes a camera point, a direction, a tilt angle, and an angle of view. And distance information, and reference numeral 420 is three-dimensional map data corresponding to reference numeral 410. The reasoning unit 135 refers to the three-dimensional map data such as the reference numeral 420. The geographic object is photographed by the perspective transformation shown in FIG. 5 and the camera geometry model shown in FIG. 6. The candidate geographic object in the picture, such as the reference sign 410, is inferred to have a one-to-one mapping with. That is, the reasoning unit 135 considers the camera rotation, translation, and perspective transform shown in FIGS. 5 and 6 using geographic objects included in the image data of the photo using map information. Infer candidate geographic objects that are mapped in the picture using a camera geometry model.

When the reasoning unit 135 is all the camera origin, the direction, the tilt angle, the angle of view, the distance, three-dimensional map data of the photo data through the input unit 110 as shown in FIG. By using the camera geometry model, candidate geographic objects in the picture may be inferred so that a geographic object may be seamlessly mapped to the photo image object area. That is, the inference device 135 corresponds to one point of the picture by the camera geometry modeling when all information related to the picture is sensed and inputted by various sensors 111 to 115 of the input unit 110. Only geographic objects can be inferred.

It is a figure which shows an example of calculating the angle of a visual field in this invention.

Referring to FIG. 7, an angle of view is calculated as shown in Equation 1 using the focal length of the camera and the horizontal size of the camera CCD sensor recorded in the photo information data.

The actual size of the image sensor formed on the camera is recorded in the 'FocalPlaneXResolution' field when the photo information data is Exif information. In addition, the angle of the actual field of view can be accurately calculated in consideration of the amount of zoom of the camera.

8 is a diagram illustrating another example of inferring candidate geographic objects in a photograph according to the present invention.

Referring to FIG. 8, reference numeral 810 denotes an urban landscape photograph including a camera origin, a direction, an angle of view, and a camera tilt angle, and reference numeral 820 denotes a three-dimensional map corresponding to reference numeral 810. Represents two-dimensional map data rather than data. As shown in FIG. 8, when the 3D map information and the distance information do not exist, the reasoning unit 135 is arranged at a line of sight in each direction within an angle of view of the photographed view. Infer existing geographic names as candidate geographic names in each direction. In this case, several geographic objects can be candidates for a point in the picture. That is, the reasoning unit 135 is a candidate geographic name existing in the direction of the arrow in the picture, such as the reference sign 820, for example, due to the absence of the distance information of the object in the picture, such as 'city terminal, trade center'. By reasoning using a line of sight, one can infer all geographic objects that appear on a map at a viewing angle. In this case, a correction step is required for accurate mapping with the object area of the picture, which is performed by the area mapper 136.

9 illustrates another example of inferring candidate geographic objects in a photograph according to the present invention.

Referring to FIG. 9, reference numeral 910 includes a camera origin, a direction, and an angle of view, and represents an urban landscape photograph when the camera tilt angle is 20 degrees, and reference numeral 920 corresponds to reference numeral 910. Represents two-dimensional map data rather than three-dimensional map data. The reasoning unit 135 searches for geographic objects that may appear in the photograph area by reflecting the camera tilt angle of 20 degrees from the map data, and infers geographic object candidates based on the search results.

FIG. 10 is a diagram for illustrating another example of inferring candidate geographic objects in a photograph according to the present invention. FIG.

Referring to FIG. 10, reference numeral 1010 includes a camera origin, a direction, and an angle of view, and represents an urban landscape photograph having a camera tilt angle of 45 degrees, and reference numeral 1020 corresponds to reference numeral 1010. Represents two-dimensional map data rather than three-dimensional map data. The inference unit 135 searches for geographic objects that may appear in the photograph area by reflecting the camera tilt angle of 45 degrees from the map data, and infers geographic object candidates according to the search results.

On the other hand, if the camera tilt angle information does not exist, the reasoning unit 135 assumes that the camera has taken the picture in the horizontal direction with the ground, and searches for geographic objects that may appear in the picture area by searching the map data. Inferred as a candidate. In this case, a step of correcting for accurate mapping with the object area of the picture is required, which is performed by the area mapper 136.

11 is a diagram showing an example of inferring and mapping candidate geographic objects in a photograph according to the present invention.

Referring to FIG. 11, reference numeral 1110 denotes an urban landscape photograph including a camera origin, a direction, an angle of view, and a camera tilt angle, and the reference numeral 1120 is photographed at the same place as the reference numeral 1110. Unlike the photograph or reference numeral 1110, the photograph shows an urban landscape photograph that does not include the camera tilt angle. The reasoning unit 135 maps geographic objects that may appear in the picture area on the assumption that the camera taking the picture has taken the picture in a horizontal direction with the ground when no camera tilt angle exists. Search from the data and infer it as a geographic object candidate. Therefore, the picture of the reference 1120 is corrected because the object area mapping is inaccurate compared to the picture that can be accurately mapped, such as the picture of the reference 1110.

An area mapper 136 maps the inferred candidate geographic object and the geographic area of the extracted photo. That is, the area mapper 136 selects one of the candidates identified through the candidate geographic name inferred by the inference unit 135, and maps the selected candidate geographic object to the corresponding region of the extracted picture.

For example, when all input information is inputted by the input unit 110 as shown in the photograph of the reference numeral 1110 shown in FIG. 11, the area mapper 136 may perform accurate inference with the image object. The corresponding object name is mapped to the image object.

As another example, the area mapper 136 simply took the picture on a horizontal axis to solve the case where the camera tilt angle information does not exist as shown in the picture of reference numeral 1120 of FIG. 11. It is assumed that '0 degree' maps the geographic object to the image object in the picture. As such, when inferring multiple candidates without accurate mapping because no input information exists, the region mapper 136 utilizes image classification and image segmentation to generate the image object region. Corrects the mapping of geographic object names. Here, the image segmentation algorithm uses the level of performance of segmenting the approximate object region due to the limitation of analysis time and the accuracy of region extraction.

As described above, the geographic object mapping system 100 of the photographic image object according to the present invention is not inputted with all geographic object information when a picture is taken, but when it is cumbersome and difficult to measure the distance to the geographic object, the distance information is missing. Even a photograph may be corrected to map an exact geographic object name to the photograph.

In addition, the geographic object mapping system 100 of a photographic image object according to the present invention, when the camera tilt information or a sensor that is not universal for the sensing needs to be used, for example, a picture taken by a camera that does not have a sensor such as a gyro sensor Even if the input is corrected, the correct geographic object name can be mapped to the picture.

The index generator 137 generates an index of the photo based on the geographic area. That is, the index generator 137 generates an index by mapping an object name corresponding to the geographic object with the image object, and stores the generated index in the mapping database 123.

2 is a flowchart illustrating a method of mapping an image object in a photo to a geographic object according to another exemplary embodiment of the present invention.

Referring to FIG. 2, in step S200, the geographic object mapping system of the photo image object receives photo data, photo information data, or meta data through an input means. The input means has a photographing function such as a camcorder, a digital camera or a camera phone. The picture information data may include latitude, longitude, altitude, direction, focal length, or CCD size information, and the metadata may include tilt information of the input means or distance information between the input means and a subject when the picture is taken. Can be. That is, in step S200, the geographic object mapping system of the photographic image object performs image sensing for capturing a photograph through the input means, measures the position of a photographing space, and the direction of the photographed image in the photographing space. Measure an image, encode an image of the photographic data, and encode the photographic information data using the position or orientation. In operation S200, the geographic object mapping system of the photographic image object generates meta data of the photograph using the tilt information or the distance information through the input means and inputs the meta data. I can receive it.

The input photo data, photo information data or the meta data is recorded and maintained in a database. That is, the database records and maintains the photo data, the photo information data, the meta data, map data, and photo data mapped with the map data. The database may include a photo database in which the photo data and the photo information data are recorded, a map data in which the map data is recorded, and a mapping database in which the photo data mapped to the map data is recorded.

For example, in step S200, the object mapping system of the photographic image object may include an image of the photograph, location information at which the photograph was photographed, distance information at which the photograph was photographed, direction information at which the photograph was photographed, and photographing the photograph. All camera tilt information can be input.

As another example, in step S200, the object mapping system of the photographic image object may include an image of the photograph, location information at which the photograph was photographed, distance information between the input means at which the photograph was photographed, and the subject, and a direction in which the photograph was photographed. Information or tilt information of the input means may be selectively input when the picture is taken. Depending on the type of the input means, the distance information, the direction information, or the camera tilt information may not be input.

In operation S210, the geographic object mapping system of the photo image object parses the photo data, the photo information data, or the metadata.

In operation S220, the geographic object mapping system of the photo image object classifies the photo based on the contents of the photo data. That is, in step S220, the geographic object mapping system of the photo image object analyzes the photo data and classifies it as a non-target photo or a target photo. The non-target picture is a picture that does not require mapping of geographic names such as a person or affix, and the target picture is a picture requiring mapping of geographic names such as an urban landscape. Hereinafter, a procedure of classifying photographic data into non-target photographs or target photographs based on contents will be described in more detail with reference to FIG. 3.

3 is a diagram illustrating an example of a process of classifying photographs based on contents according to the present invention.

Referring to FIG. 3, in operation S310, the geographic object mapping system of the photo image object analyzes the photo information data and the decoded photo data and classifies the photo data as the target photograph when the photo data is an urban landscape.

In step S320, the geographic object mapping system of the photo image object analyzes the photo information data and the decoded photo data and classifies it as the non-target photo when the photo data is an indoor or macro photograph instead of the urban landscape. To filter.

As described above, the method for mapping a geographic object of a photographic image object according to the present invention analyzes the photographic information data and the photographic data and classifies the photograph as the target photograph when the photograph is an urban landscape, and the photograph is an indoor photograph or In the case of macro photography, only the final target picture to be mapped may be provided by filtering the non-target picture.

In operation S230, the geographic object mapping system of the photographic image object determines whether the classified photograph is the non-target photograph.

If the classified picture is the target picture rather than the non-target picture, in step S240, the geographic object mapping system of the photo image object analyzes the target picture based on the map data to analyze the geographic object in the picture. Infer the mapping candidate. That is, in step S240, the reasoning unit 135 of the geographic object mapping system of the photo image object uses the camera rotation shown in FIGS. 5 and 6 by using map information on the geographic objects included in the photo data. Candidate geographic objects that are mapped in the picture are inferred through a camera geometry model that takes into account translation and perspective transforms. The geographic object mapping system of the photographic image object may include the camera origin, the direction, the tilt angle, the angle of view, the distance, and the 3D map data of the photographic data as shown in FIG. 4. By using a camera geometry model, candidate geographic objects in the photo may be inferred so that the geographic objects may be seamlessly mapped one-to-one into the photo image object region. That is, in the method of mapping a geographic object of a photographic image object according to the present invention, for example, when all information related to the photograph is sensed and input by various sensors, one geograph corresponding to one point of the photograph by the camera geometry modeling is input. Only objects can be inferred.

In addition, a method of mapping a geographic object of a photographic image object according to the present invention is another example in which the 3D map information and the distance information do not exist as shown in FIG. 8 within an angle of view of the photographed view. Geographic names that exist in the line of sight in each direction may be inferred as candidate geographic names in each direction. In this case, several geographic objects can be candidates for a point in the picture.

Therefore, the geographic object mapping method of the photographic image object is a candidate geographic name existing in the direction of the arrow in the photograph as shown by reference numeral 820. For example, the distance information of the object in the photograph, such as 'city airport terminal and trade center', is used. Absence can lead to inferring all geographic objects that appear on the map at a viewing angle based on inference using a line of sight. In this case, the geographic object mapping method of the photographic image object according to the present invention further requires a correction step for accurate mapping with the object region of the photograph.

In operation S250, the geographic object mapping system of the photographic image object determines whether the inference result is correct inference.

If the inference result is not accurate inference, the geographic object mapping system of the photographic image object extracts an image object based on the content of the photographic data in step S260. That is, in operation S260, the geographic object mapping system of the photo image object may extract an image object region within the photo by image segmentation.

If the inference result is correct inference or the image object is extracted, the geographic object mapping system of the photo image object maps the inferred geographic object name to the image object region of the photo data in step S270. That is, in operation S270, the geographic object mapping system of the photographic image object may select any one of the inferred candidate geographic objects, and map the selected candidate geographic object to the image object region of the extracted photo. Hereinafter, a procedure of mapping a geographic object to an image object of a photo will be described in more detail with reference to FIG. 12.

12 is a diagram illustrating an example of a geographic object mapping procedure of a photograph according to the present invention.

Referring to FIG. 12, in operation S1210, the geographic object mapping system of a photographic image object classifies a category of an image object based on a segmentation region of the image data of the photograph. The category may include a building, a mountain, a river, a road, the sky, the sea, a beach, and the like. That is, in step S1210, the geographic object mapping system of the photographic image object divides the image data of the photograph, analyzes the divided image region, and classifies the category of the image object such as 'building, beach, sea', etc. do.

In operation S1220, the geographic object mapping system of the photo image object selects a geographic object name corresponding to the partition based on the candidate geographic object name.

For example, when the candidate geographic object name is 'West Sea, Daecheon Beach, Hanwha Condo', and the image object of the partitioned area is classified as 'Building', the geographic object mapping system of the photographic image object is determined in step S1220. Select 'Hanhwa Condo', the geographic name corresponding to the image object.

As another example, when the candidate geographic object name is 'West Sea, Daecheon Beach, Hanwha Condo', and the image object of the partitioned area is classified as 'Beach Beach', the geographic object mapping system of the photo image object in step S1220 A geographical name 'Daecheon Beach' corresponding to the image object may be selected.

As another example, when the candidate geographic object name is 'West Sea, Daecheon Beach, Hanwha Condo', and the image object of the partitioned area is classified as 'Ocean', the geographic object mapping system of the photo image object in step S1220 is A geographic name corresponding to the image object is selected as 'west sea'.

As another example, when there are several candidate geographic object names inferred, in operation S1230, the geographic object mapping system of the photo image object selects a geographic object name that appears closest to the position at which the photo was taken.

In operation S1230, the geographic object mapping system of the photo image object maps the selected geographic object name to each object region of the photo.

It is a figure which shows an example which mapped the geographic object of a photograph in this invention.

Referring to FIG. 13, reference numeral 1310 denotes a case in which geographic object name mapping information is 'Hanhwa Condo' for each object region of the picture, and reference numeral 1320 denotes geographic object name mapping information for each object region of the picture. Denotes a case of 'Daecheon Beach', and reference numeral 1330 denotes a case in which geographic object name mapping information is 'West Sea' for each object region of the picture.

As described above, the method for mapping a geographic object of a photographic image object according to the present invention accurately corrects a geographic name by overlaying the geographic name of the object on the center of an area so as to map a geographic name corresponding to an exact position in the picture. You can map geographic object names to video objects.

In operation S280, the geographic object mapping system of the photo image object generates a photo index based on the geographic object name of the image object region of the photo. That is, in step S280, the geographic object mapping system of the photo image object generates an index for the photo based on the geographic object name mapped with the image object of the photo, and stores the generated index in the mapping database. do.

For example, in the case of the reference numeral 1520 illustrated in FIG. 15, the method for mapping a geographic object of a photographic image object according to the present invention is a name of a geographic object mapped with the image object of the photograph, 'KEPCO, Cosmo Tower, and Chris Tower. , 'Trade tower, assem tower' can be generated as an index for the photo.

14 is a diagram illustrating a mapping result of a photo image object and a geographic object according to the present invention.

Referring to FIG. 14, reference numeral 1410 indicates that the geographic region mapped to the photo region in the arrow direction is a trade tower, and reference numeral 1420 denotes map data corresponding to the reference numeral 1410.

As described above, the method of mapping an image object in the image with a geographic object according to the present invention may provide a user by mapping a map with what the picture includes and what is in the location of the picture.

FIG. 15 is a diagram for one example of mapping a building name to an image object of a photograph according to the present invention.

Referring to FIG. 15, reference numeral 1510 represents an example of a query image in which a building name is mapped to an image object region of a photograph taken at Yeouido, and reference numeral 1520 is an image object region of a photograph taken at a Samsung station. An example of a query image in which the building names are mapped to each other is shown.

As described above, in the method of mapping an image object in a picture to a geographic object according to the present invention, a user can easily identify a building or a location included in the picture by mapping a building name / designation name corresponding to the corresponding location of the picture. .

Fig. 16 is a diagram showing an example of image search according to information search in the present invention.

Referring to FIG. 16, reference numeral 1610 denotes an information retrieval menu, and reference numeral 1620 is an example of a conventional query that is formatted according to the information retrieval menu. Represents a query. Reference numeral 1630 denotes an example of inputting a query by a photo, and indicates a geographic object name corresponding to an image object region selected by the user among the regions of the photo, and reference numeral 1640 denotes an image object region selected by the user. Shows the result of searching, navigating or clustering a building and a location corresponding to a geographic object name corresponding to.

As described above, the method of mapping an image object in a photo to a geographic object according to the present invention may additionally search for photos associated with a geographic name when specifying and searching an object region of the photo.

In addition, the method for mapping an image object in a photo to a geographic object according to the present invention includes a computer readable medium including program instructions for performing operations implemented by various computers. The computer readable medium may include a program command, a data file, a data structure, etc. alone or in combination. The medium or program instructions may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

According to the present invention, it is possible to provide a method and a system for mapping information about a geographic object in a picture to corresponding image object areas in the picture by using map information and image information.

In addition, according to the present invention, it is possible to provide a method and system for mapping an image object in a photo to a geographic object for providing a user by mapping a map with what is included in the photo and what is at a location in the photo. have.

In addition, according to the present invention, by mapping the building name / place name corresponding to the corresponding location of the picture to map the image object in the picture and the geographic object so that the user can easily see the picture or the building or location included in the picture and To provide that system.

Further, according to the present invention, a method and a system for mapping an image object in a photo to a geographic object for additionally searching for photos associated with a geographic name when specifying and searching an object region of the photo can be provided.

Claims (16)

An input unit for receiving photo data; A database for recording and maintaining the photo data and map data; And A mapping processor configured to analyze the photo data and to map an image object of the photo data and a geographic object of the map data with reference to the map data System for mapping the image object in the picture with a geographic object comprising a. The method of claim 1, The input unit, An image capturing sensor which performs sensing for capturing an image in the photographing space; A position measuring sensor measuring a position of the photographing space; A direction measuring sensor for measuring a direction of an image photographed in the photographing space; An image encoder for encoding the sensed image; And Photo information encoder for encoding photo information using the measured position and direction System for mapping the image object in the picture with a geographic object comprising a. The method of claim 2, A tilt measuring sensor measuring a tilt of a photographing angle of the image when the photograph is taken; or Distance measuring sensor for measuring the distance between the image and the input unit when the picture is taken The system for mapping the image object in the picture with a geographic object further comprising. The method of claim 3, A metadata generator for generating metadata of the photo using the gradient data or the distance data The system for mapping the image object in the picture with a geographic object further comprising. The method of claim 4, wherein The database, A photo database for recording and maintaining the photo data, the photo information data, and the metadata; A map database for recording and maintaining the map data; And Mapping database for recording and maintaining the photo data mapped with the map data System for mapping the image object in the picture with a geographic object comprising a. The method of claim 5, The mapping processing unit, A parser for parsing the photo information data and the metadata; An image decoder configured to decode an image of the photo data by reflecting a parsing result by the parser; A classifier that analyzes the parsing result and the image of the decoded photo data and classifies the image into a non-object photo or a target photo; An area extractor for extracting a geographic area of the picture by analyzing the image of the decoded picture data based on a content; An inferencing device that infers a candidate geographic object in the target photograph by referring to the map data; A mapper that maps the inferred candidate geographic object and the geographic area of the extracted photo; And Index generator to index photos based on geographic area System for mapping the image object in the picture with a geographic object comprising a. The method of claim 6, The classifier, The photograph information data and the decoded photograph data are analyzed to classify the photograph as the target photograph when the geographic information is needed, and the photograph is classified as the non-target photograph when the photograph does not need the geographic information. A system for mapping image objects to geographic objects. In the method of mapping an image object in a photo with a geographic object, Receiving photo data; Recording and maintaining the photo data and map data in a database; And Analyzing the input photo data and mapping an image object of the photo data and a geographic object of the map data with reference to the map data And mapping the image object in the photo to a geographic object. The method of claim 8, The step of inputting the photo data, Performing image sensing to take a picture; Measuring a position of a photographing space; Measuring a direction of an image photographed in the photographing space; Encoding the image; And Encoding photo information using the position and the direction And mapping the image object in the photo to a geographic object. The method of claim 8, Measuring an inclination of a photographing angle of the image or measuring a distance between an input means and a subject when capturing the image; And mapping the image object in the picture to a geographic object. The method of claim 10, Generating metadata of the photo using the slope or the distance And mapping the image object in the picture to a geographic object. The method of claim 11, The database, And recording and maintaining the photo data, the photo information data, the meta data, the map data, and the mapped photo data. The method of claim 12, The mapping of the image object of the photo data and the geographic object of the map data may include: Parsing the photo data, the photo information data and the meta data; Decoding the image of the photo data by reflecting the parsing result; Analyzing the picture data and classifying the picture data into a non-target picture or a target picture; Extracting a geographic region of a photo by analyzing the decoded image based on contents; Inferring a candidate geographic object in the target photograph with reference to the map data; Mapping geographic regions of the inferred candidate geographic object and the extracted photo; And Generating a photo index based on a geographic object name of the image object region of the photo; And mapping the image object in the photo to a geographic object. The method of claim 12, Analyzing the photo data and classifying it into a non-target photo or a target photo, Analyzing the picture information data and the decoded picture data to classify the picture as the target picture when the picture is a geographic name information, and classify the picture as the non-target picture when the picture is a picture that does not require geographic name information. And mapping a video object in a photo to a geographic object. The method of claim 13, The mapping of the inferred candidate geographic object and the geographic area of the extracted photo may include: Dividing the image data of the picture, analyzing the divided image area, and dividing an area of an image object; Selecting a geographic name corresponding to an area of the divided image object based on a candidate geographic object name; And Mapping the selected geographic object name to each image object region of the picture; And mapping the image object in the photo to a geographic object. A computer-readable recording medium for recording a program for executing the method of any one of claims 8 to 15 on a computer.

Images