JP2007293558A - Program and device for object recognition - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus that can recognize an object included in an image, with high accuracy. <P>SOLUTION: An environmental model is calculated, by acquiring a sensing image along with the information on the photographing time and position of the sensing image, using an environmental model defined by the time and position recorded, in advance, on the basis of the photographying time and the position acquired. An object image is generated from the object shape data registered in advance, by using the environmental model to search for an object from the sensing image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and program using a remote sensing image.

  Various objects (for example, an aircraft, a ship, a vehicle, and a train) exist in the remote sensing image. If these objects can be automatically extracted, identified, and searched from the image, various information can be obtained from the image in real time.

Here, extraction refers to specifying the position of an object included in an image. Identification is to specify the name of an object included in the image. Recognition is to identify the position of the object and the name of the object from the image by extraction processing and identification processing. The search is to specify the position of the object from the image using the name of the object as a keyword.
For object recognition, a dictionary storing the shape of the object is created in advance using an image database storing the image of the object to be recognized and its name, and the object included in the remote sensing image and the dictionary are included in the dictionary. This is done by checking the shape of the object to be checked.

  The shape of the object included in the image database is created from an image obtained by hollowing out the object included in the remote sensing image. In particular, if the time at which a remote sensing image is captured differs, the object image is also affected, and the shadow of the object and the image contrast vary. The luminance value of the object image also changes due to such fluctuations, which affects the object identification accuracy. Therefore, an object image database is created using objects images included in remote sensing images taken at various times and places in advance.

  On the other hand, since it is not efficient to collect an object image taken at an arbitrary shooting time from an actual remote sensing image, an object image is generated from a three-dimensional object model (object shape given as numerical data). (Rendering) techniques have been used. In general, in rendering, it is possible to perform hidden surface removal, shading, and image contrast adjustment in consideration of conditions such as the position of the viewpoint, the light source position, and the material.

  Also, the sun direction and solar altitude are calculated from the shooting date and time and shooting position information. There is also a method of estimating a shadow area from the sun direction, solar altitude, and position and height information of the recognition target object, and removing the shadow from the captured image (for example, see Patent Document 1).

  Also, the sun direction and solar altitude are calculated from the shooting date and time and shooting position information. There is also a method of removing shadows by performing frequency analysis of a remote sensing image and masking the frequency component corresponding to the sun direction and solar altitude (for example, see Patent Document 2).

Japanese Patent Laid-Open No. 10-269347 JP 2001-266162 A

  The above-described conventional method for creating a dictionary from an object image database created based on various aerial image attributes has a problem in that the capacity of the storage device for storing the dictionary increases and the processing time increases. It was.

  Further, in the method described in Patent Document 1, since the object in the captured image is corrected, it is required that the position and height information of the recognition target object is known, and when such information is not known in advance. Could not be applied. In addition, since the shadow area is removed in the method described in Patent Document 2, when a shadow is formed on the recognition target object, the recognition target object itself is removed as a shadow area. An object of the present invention is to provide an image processing apparatus that solves these problems.

In order to solve the above object, an outline of the invention disclosed in the present application is as follows.
A sensing image is acquired together with information on the shooting time and shooting position of the sensing image, and an environment model is calculated based on the acquired shooting time and shooting position using an environmental model defined by the time and position recorded in advance. After that, an object image is generated from object shape data registered in advance using an environmental model. A target recognition program and apparatus for performing a search from a sensing image using this object image and outputting a search result.

  According to the present invention, a dictionary for object identification can be created in consideration of variations that can be made in an object image due to different shooting dates and times and places, so that object recognition can be performed with high accuracy.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, in this application, “object” refers to features and moving objects, and “moving object” refers to aircraft, ships, vehicles, trains, and the like.

  In this embodiment, a remote sensing image obtained by measuring / recording radiation / reflected electromagnetic waves on the ground surface by a satellite or aircraft sensor will be described. The application target of the present application is not limited to a remote sensing image captured by a sensor mounted on a satellite or an aircraft, but the position of the sensor and the shooting time are known in advance outdoors, and there is a certain distance between the subject and the imaging unit. Any sensing image captured in various situations can be applied. In particular, it is useful when the spatial resolution is about 1 m and the influence of shadows on and around the object is large, which is a serious obstacle to recognizing the object on the sensing image.

  FIG. 1 is a diagram showing a positional relationship between an observation sensor, a recognition object, and the like in an embodiment of the present invention. An artificial satellite 101 equipped with an observation sensor observes an observation area 103. An observation range 105 is a range in which an image is captured by an observation sensor. Here, although the observation sensor is mounted on the artificial satellite, the captured image used in the present invention may be provided on a flying object (for example, an aircraft, a balloon) or a moving object such as a train, a vehicle, or a person.

  Sunlight 104 is irradiated from the sun 102 toward the observation area 103. Although the sun 102 is assumed as the light source, a light source other than the sun may be used. A plurality of light sources may be used. The sunlight 104 hits the recognition target object 106, and the portion where the light is blocked by the recognition target object moves as a shadow 107 on the observation range 105.

  FIG. 2 is a block diagram showing an embodiment of the image processing apparatus according to the present invention. The image processing device 203 is a device that obtains an observation signal from the observation sensor 201 and outputs an object recognition result to the recognition result display device 205. The observation sensor 201 images an observation target such as an urban area or a sea area, and outputs an observation image, the position of the observation sensor, the position of the observation target, and the observation date and time.

  The position of the observation sensor can be expressed by three sets of numerical values, for example, latitude, longitude, and altitude. There is a GPS (Global Positioning System) method for obtaining the position of the observation sensor. The position of the observation target is obtained by associating the latitude and longitude of the recognition target object from the range of the region captured by the observation sensor and the relative positional relationship where the recognition target object exists from the origin of the captured image. The range of the area photographed by the observation sensor can be expressed by a combination of four numerical values of northwest latitude, longitude, southeast latitude and longitude assuming a rectangular range, for example. The observation date and time can be realized by reading the time of a clock built in the apparatus in which the observation line 102 is mounted.

  Imaging may be based on a user's instruction or may be performed every predetermined period according to a predetermined rule. Further, the observation sensor 201 may be a passive sensor such as an optical system sensor or an active sensor such as a synthetic aperture radar. Moreover, you may make it the structure which performs an image generation process for every observation wavelength range using a hyper spectrum sensor.

  The environment information database 216 is a database that stores environment information that changes depending on the position of the observation sensor, the position of the observation target, and the observation date and time. The environmental information is, for example, solar altitude, solar orientation, weather conditions, and atmospheric conditions. The environment information generation unit 206 refers to the environment information database 210 based on the observation sensor position, the observation target position, and the observation date and time obtained by the observation sensor 202, and generates environment information about the target data.

  The object shape database 217 stores a three-dimensional model (object shape given as numerical data) of a recognition target object in association with an attribute of the object. The object image generation unit 207 generates an image of the recognition target object based on the environment information. For example, a rendering method (Mikio Takagi and Yoshihisa Shimoda “New Edition Image Analysis Handbook”, The University of Tokyo Press, September 10, 2004, pages 1758 to 1794) is used as an image generation method. The object shape database stores shapes of objects that can be recognized. When the user wants to limit the recognition target, the recognition target object input device 215 can input the name of the recognition target object, and the recognition target object selection unit 213 can limit the recognition target.

  When the object image generating unit 207 generates an object image, the object image generation control unit 209 can control the control parameters for object image generation. The control parameter includes, for example, the direction of the object to be generated and the color of the surface.

  Furthermore, the object image generated by the object image generating unit 207 can be superimposed on the background image stored in the background image database 218 in which various background images are stored in advance by using the background image superimposing unit 219. . When a plurality of background images are registered in the background image database 218, the user designates an image from the background image designation device 220, and the image designated from the background image database 218 by the background image selection means 221 is used. The image designation method can be realized by directly designating the image file name. A screen for previewing the background image may be provided so that the user can visually confirm the background image. By superimposing the background image, the luminance difference between the background image and the object image can also be used as a feature quantity for object identification. When the object feature learning unit 208 does not require a background image, the output of the object image generation unit 207 may be directly input to the object feature learning unit 208. In this case, the object image database 214 may store only images of objects that do not include the background.

  The object image generated by the background image superimposing means 219 can be visually confirmed on the screen by the object image display means 212. In that case, an actual object image that has been hollowed out from an actual remote sensing image in advance is stored in the object image database 214, and this can be referred to for visual comparison. As a result, when the user determines that the generated image is not appropriate, the object image is generated again using the object image generation control unit 209.

The object feature learning unit 208 extracts object feature knowledge effective for object recognition from the object image generated by the object image generation unit 207. The object feature knowledge is a set of a feature amount effective for classifying an object type and an object name. The feature amount includes the edge strength, frequency characteristics, luminance distribution, and the like of the object image.
The object recognition unit 204 scans the observation image to generate a feature amount, compares the feature amount with the feature amount of the object shape knowledge created by the object shape learning unit 208, and compares the feature amount with the most similar feature amount. The name is output as a recognition result. The recognition result display unit 205 displays the result recognized by the object recognition unit 204 on the screen.

  Each unit may be executed by reading a program into a computer, or may be performed in cooperation with a part of hardware. The recognition result display device 205 and the object image display device 212 may be the same display device, and 209, 215, and 220 can be combined with the same input device.

  FIG. 3 is a diagram showing an example of an object recognition processing flow in one embodiment of the present invention. First, an observation area is observed by the observation sensor 201, and an observation image, an observation sensor position, an observation target position, and an observation date and time are obtained (S301). Next, the environment information generation unit 206 generates environment information with reference to the environment information database 210 based on the observation sensor position, the observation target position, and the observation date and time (S302). Then, the user determines whether all objects are to be recognized (S303). When not all objects are to be recognized, the recognition target object name is input from the recognition target object input device 215, and only the shape of the recognition target object is selected by the recognition target object selection unit 213 from the object shape database 217. (S304). An image of the recognition target object is generated from the environmental information and the object shape database 217 by the object image generation means (207) (S305). Then, the background image superimposing means 219 superimposes the object image on the background image. As described above, when the object feature learning unit 208 does not require a background image, it can be skipped. Then, the object image superimposed on the background image or the object image is visually confirmed by the object image display means 212 (S306).

  If the user determines that the generated object image is not appropriate in S305, the control parameter is corrected (S306), and the object image is generated again using the corrected control parameter (S303). If the user determines that an appropriate object image has been generated in S305, object feature learning (S307) is performed. In the object feature learning (S307), the object feature is extracted from the object image to create object feature knowledge. In the object recognition (S310), the object recognition means (204) refers to the object shape knowledge, and the observation range. The object contained in the image in 205 is recognized, and the result is displayed (S311).

In the above description, it has been described that the object recognition unit 204 recognizes an object using the object shape knowledge. Naturally, the knowledge used for recognition is not limited to the shape, and image feature amounts such as spectrum, texture, and luminance distribution may be used. Moreover, you may use the full length and full width of an object.
FIG. 4 is a configuration example of a data structure used in the object shape database 210 according to the embodiment of the present invention. The object ID 401 stores a unique number for the object. The attribute 402 stores the type of the object. For example, “Airplane” for aircraft and “Ship” for ships. The object name 403 stores the name of the object. When the full length and full width of the object are effective for object recognition, an area for storing the full length 404 and the full width 405 is provided, and the full length and full width of the object are stored. When the object shape looks different depending on the direction of the recognition target object, a direction 406 is provided to store the object direction. “Left” for an airplane means that the nose is on the left side of the screen but is also oriented. The shape parameter 407 stores the shape of the object. As a method for expressing the shape of an object, a set of functions representing a curved surface can be used. Of course, other formats that can express a three-dimensional shape may be used. For example, a method may be used that represents a set of elevation points arranged in a grid pattern. In addition to the shape, the material, color, and electromagnetic wave reflection / absorption characteristics of the object surface can be included.

  FIG. 5 is a diagram showing an example of the processing flow S305 of the object image generation means 207 in one embodiment of the present invention. First, the recognition target object shape is converted into a representation format of a set of points (eg, latitude, longitude, height) in a three-dimensional space (S501). This conversion can be obtained by substituting the observation target position into the shape parameter 407, for example. When the spatial resolution of the shape parameter 407 and the spatial resolution obtained from the observation sensor 202 are different, the spatial resolution is matched with the observation sensor 202. An affine transformation can be used for this (Mikio Takagi and Yoshihisa Shimoda “New Edition Image Analysis Handbook”, The University of Tokyo Press, September 10, 2004, pages 1759 to 1764).

Next, in the projective transformation process (S502), the recognition target object expressed by the point set in the three-dimensional space is projected onto the two-dimensional plane to generate a projected image. Projective transformation can be performed by obtaining the intersection of the projection planes from the relationship between the viewpoint and the line of sight (Mikio Takagi and Yoshihisa Shimoda “New Image Analysis Handbook”, University of Tokyo Press, September 10, 2004, 1764 to 1766. Page).
In the shadow area calculation (S503), the solar altitude and direction are extracted from the environment information, and the shadow area formed on or around the recognition target object is specified. The length L of a shadow formed by irradiating an object having a height of h from the ground surface with sunlight having a solar altitude θ is L = h / tan θ, and the direction of the shadow is opposite to the solar direction δ. The solar altitude and direction at an arbitrary time at an arbitrary point can be calculated using values described in the calendar part of the scientific chronology.

  In the above, in order to model the shadow formed on or around the object, the case where the solar orientation and solar altitude are output as the output of the environmental model is assumed. In addition, a weather model can be considered as an environmental model. The weather model can be realized as a database system capable of searching for weather using the position and date / time as an index. The weather information of the observation target position is extracted from the weather model from the information of the observation target position and the observation date and time, and the image brightness of the observation image is predicted. For example, when the weather at the time of observation is clear, the cloud amount is regarded as 0 and the contrast of the observation image is increased. Conversely, when the weather is cloudy, the contrast of the observed image is lowered.

  An atmospheric model can also be considered as an environmental model. The observation image includes not only the reflection signal from the recognition object but also the influence of radiation and attenuation from the atmosphere between the observation sensor 202 and the recognition object. Processing for removing these influences is called atmospheric correction processing. The atmospheric model is a model that outputs an atmospheric correction amount when an observation date and time and an observation target position are input. For the calculation of the atmospheric correction, a method of subtracting a constant value from each pixel value of the observation image, a ratio calculation of each observation wavelength region, or the like can be used.

  Further, in the hidden surface erasing process (S504), a portion that cannot be seen from the observation sensor 202 is erased from the projection image. For the hidden surface removal, the Z sort method or the Z buffer method (Mikio Takagi, Yoshihisa Shimoda “New Edition Image Analysis Handbook”, The University of Tokyo Press, September 10, 2004, pages 1782 to 1788) can be used. The hidden surface can be easily deleted by assigning priorities according to the depth of each surface and displaying them in order from the back.

  Then, in the shading process (S505), a shade is added according to the degree of sunlight that hits the recognition target object. The Utah approximation method (Mikio Takagi and Yoshihisa Shimoda “New Edition Image Analysis Handbook”, The University of Tokyo Press, September 10, 2004, pages 1789 to 1792) can be used for shading.

  Finally, in the texture mapping process (S506), a two-dimensional pattern like a texture called a texture is pasted on the object image. Texture mapping can be performed by mapping the two-dimensional coordinates of the texture to each pixel of the recognition target object image using a mapping function (Mikio Takagi and Yoshihisa Shimoda “New Image Analysis Handbook”, The University of Tokyo Press, 2004. September 10, 1779-1794 pages). The texture can be set according to the material of the object surface. The material can be obtained from the reflection / absorption characteristics of the electromagnetic wave of the shape parameter 407, and information on the material may be stored in advance together with the shape parameter.

  FIG. 6 is a diagram showing an example of a processing screen displayed by the display device when processing is executed by the object image generating unit 207 according to the embodiment of the present invention. The processing screen 601 provides information related to processing to the user or inputs information from the user. The processing screen 601 includes a plurality of sub screens and provides a human interface. The sub screen 602 is an input parameter of the object image generation unit 207, and in this example, the observation sensor position, the observation target position, and the observation date and time are displayed. The sub screen 603 displays the solar orientation and solar altitude calculated from the input parameters. The sub screen 604 displays the name of the object currently targeted. By pressing a button on the right end of the sub screen 604, another name can be selected. The sub screen 605 displays the direction of the object that is currently targeted. Since the direction of the recognition target object is usually unknown in many cases, an object image in a plurality of directions is automatically generated. When the direction of the recognition target object is known, an object image can be generated by selecting only an arbitrary direction by pressing a button at the right end of the sub screen 605.

  On the sub-screen 606, a texture to be pasted on the object currently targeted is displayed. By pressing a button at the right end of the sub screen 606, it is possible to select and paste a texture (S506). The sub screen 607 is used by the user to visually confirm the object image generated by the object image generation unit 207, and displays the object image. While viewing this sub-screen 607, the brightness of an image generated by pressing the brightness fine adjustment button 610 can be finely adjusted. Further, a sub screen 609 may be provided to display the actual observation image. If the user grasps at least one target existing in the actual observation image at this stage, the object image 607 created by the object image generation unit 207 and the object in the actual observation image are displayed side by side. The naturalness of the created object image can be confirmed. As a result of the confirmation, if the user determines that the object image displayed on the sub screen 607 is unnatural compared to the actual observation image displayed on the sub screen 609, a button at the right end of the sub screen 606 is displayed. Press to select another texture and adjust to a similar image. Further, the brightness value can be finely adjusted by pressing the brightness adjustment button 610. Further, in addition to texture and brightness, color, reflection characteristics, and the like may be adjusted. If the result of the recognition result display S311 is displayed on the sub-screen 609 together with the actual observation image, and the object recognition S310 is performed again with the object image corrected with reference to it, a better result can be obtained. The sub screen 611 displays the background image name selected by the user. By pressing a button on the right end of the sub screen 611, another background image can be designated. The selected background image is displayed on the sub screen 612.

  FIG. 7 is a diagram illustrating an example of an object image generated by the object image generation unit 207 according to an embodiment of the present invention. The object image 701 is an image that is assumed to be captured in a mid-latitude region. The object image 702 is an image that is assumed to have been shot in a high latitude area. A diagram 710 shows the relationship between the observation position, the shadow of the object, and the sunlight in the middle latitude area on a vertical plane including the observation position and the sun position. Sunlight 704 hits object 703 at solar altitude θ1 (705), and a shadow of length L1 (705) is generated. Similarly, reference numeral 711 indicates the relationship between the observation position, the shadow of the object, and the sunlight in the low latitude area on a vertical plane including the observation position and the sun position. Sunlight 707 hits object 706 at solar altitude θ2 (708), and a shadow of length L2 (709) is generated. Thus, since the solar altitude changes with latitude, the size of the shadow around the object also changes. Similarly, the shadow formed on the object by the unevenness of the object itself varies depending on the latitude.

  FIG. 8 is a configuration example of a data structure of object shape knowledge generated by the object shape learning unit 208 according to an embodiment of the present invention. Here, learning is to extract information (for example, shape) effective for object identification from the object image, combine it with the attribute of the object shape database 210, and output it as object shape knowledge. Information described in 801-806 is the same as information 401-406 in FIG. The feature quantity 807 stores the feature quantity of the object generated by the object image generation means. That is, a value obtained by correcting the feature quantity 407 in consideration of imaging information is stored. In this example, the distribution of edge strength is used as the method for expressing the feature amount of the object. Of course, other representations can be used as long as the features are appropriate for identifying the object. For example, the luminance distribution, spectrum, texture, etc. of the object image may be used.

  FIG. 9 is a diagram showing an example of the processing flow S308 of the object recognition unit 204 in one embodiment of the present invention. The observation image has a size of vertical h pixels × horizontal w pixels and is represented by a two-dimensional array. Pixel (i, j) means a pixel in the i-th column and the j-th row of the two-dimensional array. First, the variable i is initialized to 1 (S901), and the variable j is initialized to 1. Next, a rectangular area A centered on the pixel (i, j) of the observation image is hollowed out (S903). The shape of the region A can be an arbitrary shape such as a circle or an ellipse. The size of the area A is set to the size of the largest object in the set of recognition target objects in advance. Then, the object shape feature B is extracted from the region A (S904). This feature amount is also generated using the same method as that used in the object shape learning unit 208. Subsequently, all feature quantities included in the object shape knowledge are compared with the object shape feature B, and the degree of similarity is calculated (S905). The name of the object having the most similar feature quantities is output as the identification result (S906). The value of the variable j is incremented by one (S907). If the value of the variable j is smaller than the observed image width, the process goes to S903, and if it is larger, the process goes to S909. In step S909, the value of the variable i is incremented by one. In step S910, if the value of the variable i is smaller than the observed image height, the process proceeds to step S902. Through the above processing, the recognition object included in the observation image can be found.

The degree of similarity calculated in S905 can be calculated using various distance scales such as a city block distance between feature quantities and an Euclidean distance.
In S906, when the degree of similarity is smaller than a preset threshold value, the identification result may not be output. By doing in this way, only a thing with a high degree of similarity can be shown to a user.

  In S906, a plurality of similar objects may be selected and a plurality of names corresponding to these objects may be output. In this way, the user can select the correct object name from a plurality of narrowed object name candidates while looking at the object image. Can be reduced. Further, the identification results are rearranged in order of the degree of similarity, and if the difference between the highest degree of similarity and the second highest degree of similarity is smaller than a preset threshold value, the identification result is not output. Also good. Thus, when an ambiguous recognition result is obtained, the recognition result can be prevented from being presented to the user.

  FIG. 10 is a configuration example of the data structure of the object recognition result generated by the object recognition unit 204 according to the embodiment of the present invention. The serial number of the recognized object is stored in 1001. The latitude and longitude of the recognized object position are stored in 1002 and 1003, respectively. The attribute 1004 stores the attribute of the recognized object. For example, “Airplane”, “Ship”, etc. The object name 1005 stores the name of the recognized object. Similarity degree 1006 stores the degree of similarity. In the object ID 1007, a numerical value corresponding to the object ID 401 is stored.

  FIG. 11 is a diagram showing an example of a processing result output screen when the object recognition unit 204 according to the embodiment of the present invention is executed. The output screen 1101 provides information regarding processing to the user. The output screen 1101 includes a plurality of sub screens. In the sub screen 1102, an object recognized in the observation image is highlighted and displayed. When there are a plurality of recognized objects, when one recognized object is selected with a pointing device such as a mouse, an image including the recognized object is displayed on the sub-screen 1103. In addition, the sub screen 1104 displays an object image corresponding to the object type determined to have the highest degree of similarity in S905. The sub screen 1105 displays the object name 1005 and the degree of similarity as numerical values. In addition, the sub screen 1106 displays the total length and width of the object on the sub screen 1104. When the range of the observation image is wide, the display image region can be moved to the region of interest of the user using the scroll bar 1107. Further, the observation image displayed on the sub screen 1102 can be enlarged and reduced by using an enlargement / reduction button 1108 in which the resolution of the observation image is low and it is difficult to confirm the object image.

  FIG. 12 is a block diagram showing an embodiment of the image processing apparatus according to the present invention. Only differences from the first embodiment will be described in detail below.

  In the observation information management DB 1218, at least one set of observation attribute information (a set of three numerical values of an observation sensor position, an observation target position, and an observation date) is registered. The object image generation unit 1211 obtains observation information one by one from the observation management DB 1218 and generates an object image based on the object shape database 1219. Then, object shape knowledge A1210, object shape knowledge B1211,... Are generated beforehand from these object images using object feature learning means 1209. The object shape knowledge selecting means 1206 selects object shape knowledge having the same observation attribute information from the object shape knowledge A1207, object shape knowledge B1208,... Using the observation attribute information as a key. Using the selected object feature knowledge, the object recognition unit 1204 recognizes the object from the image observed by the observation sensor.

  If the observation sensor is mounted on a flying object such as an aircraft or an artificial satellite, and the operational schedule (date / position pair) of the flying object is determined in advance, the observation information management database Observation attribute information can be created.

  In the above description, the object image generation unit 207 generates object images of all objects included in the object shape database 210. However, the present invention is not limited to this embodiment. For example, an object image of only an object designated by the user may be generated. In that case, an object image stored in the object shape database 210 is presented in advance on the display screen to receive a user's selection or an input from the user. Then, an object image is generated only for the object selected in S306.

  Further, although the object recognition unit 204 calculates the degree of similarity with all objects included in the object shape knowledge, the present invention is not limited to this embodiment. For example, the degree of similarity with only the object designated by the user may be calculated.

In addition, when a digital elevation model (DEM) to be observed is available by means other than the means described in the present invention, the coordinate conversion of the object image generation means 207 is performed using the elevation data of the DEM. S501 may be performed.
In the above-described embodiment, the description has been given for still images, but similar effects can be obtained for moving images.

  In the above-described examples, the explanation has been made on the assumption that the surface of the earth is observed. However, the present invention is not limited to this embodiment. For example, it can be applied to a spacecraft for planetary exploration.

  The light source is not limited to the sun. Any other star, artificial light source, reflected light, or scattered light may be used as long as the light source position and photographing time can be specified. Moreover, it is not limited to a single light source. A plurality of light sources capable of specifying the light source position and photographing time of each light source may be used.

It is a figure showing the positional relationship of an observation sensor, a recognition target object, etc. It is a block block diagram which shows one Embodiment of an image processing apparatus. It is the figure which showed an example of the object recognition processing flow. It is a structural example of the data structure used in the object shape database 210. FIG. 5 is a diagram illustrating an example of a processing flow of an object image generation unit 207. FIG. It is a figure which shows the example of the processing result display screen of an object image generation means. It is a figure which shows the example of the object image produced | generated by the object shape learning means 208. FIG. It is a figure which shows the structural example of the data structure of object shape knowledge. It is a figure which shows an example of the processing flow of the object recognition means. It is a figure which shows the structural example of the data structure of an object recognition result. It is a figure which shows the example of the process result display screen of an object recognition means. It is a block block diagram which shows one Embodiment of an image processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Object equipped with an observation sensor, 102 ... Sun, 103 ... Observation region, 104 ... Sunlight, 105 ... Observation range, 106 ... Recognition object, 107 ... Shadow, 201 ... Observation object, 202 ...
Observation sensor, 203 ... image processing apparatus, 204 ... object recognition means, 205 ... recognition result display means, 206 ... environment information generation means, 207 ... object image generation means, 208 ... object shape learning means, 209 ... object image generation control information Input device 212 ... Object image display device, 213 ... Recognition target object selection means, 214 ... Object image database, 215 ... Recognition target object input device, 216 ... Environment information database, 217 ... Object shape database, 218 ... Background image database, 219 ... Background image superimposing means, 220 ... Background image specifying device, 221 ... Background image selecting means, 401 ... Object ID, 402 ... Attribute, 403 ... Object name, 404 ... Full length, 405 ... Full width, 406 ... Direction, 407 ... Shape Parameters, 601 ... Processing screen, 602 ... Sub screen, 603 ... Sub screen, 604 ... Sub screen 605 ... Sub screen, 606 ... Sub screen, 607 Sub screen, 608 ... Sub screen, 609 ... Scroll bar, 610 ... Sub screen, 611 ... Sub screen, 612 ... Sub screen, 701 ... Object image, 702 ... Object image, 711 ... relationship between observation position and object shadow, sunbeam, 703 ... recognition target object, 704 ... sunbeam, 705 ... sun altitude, 706 ... shadow size, 712 ... relationship between observation position and object shadow, sunbeam, 707 ... Object to be recognized, 708 ... Sun rays, 709 ... Solar altitude, 710 ... Shadow size, 801 ... Object ID, 802 ... Attributes, 803 ... Object name, 804 ... Full length, 805 ... Full width, 806 ... Direction, 807 ... feature amount, 1001 ... recognized object number, 1002 ... latitude, 1003 ... longitude, 1004 ... attribute, 1005 ... object name, 1006 ... similarity, 1007 ... object ID, 1 01 ... Output screen 1102 ... Sub screen 1103 ... Sub screen 1104 ... Sub screen 1105 ... Sub screen 1106 ... Sub screen 1107 ... Scroll bar 1108 ... Enlarge / reduce button 1201 ... Observation target 1202 ... Observation Sensor 1203 Image processing device 1204 Object recognition means 1205 Recognition result display means 1206 Object feature knowledge selection means 1207 Object feature knowledge A 1208 Object feature knowledge B 1209 Object feature learning means , 1210 ... Object image generation control information input device, 1211 ... Object image generation means, 1212 ... Object image display device, 1213 ... Environment information generation means, 1214 ... Recognition target object selection means, 1215 ... Object image database, 1216 ... Recognition target Object input device, 1217 ... environmental information database, 1218 ... observation Broadcast management database, 1219 ... object shape database 1220 ... background image database, 1221 ... background image superimposing unit, 1222 ... background image selection unit, 1223 ... background image specifying means.

Claims (9)

Obtaining a sensing image together with information of the photographing time and photographing position of the sensing image;
Using the environmental model defined by the pre-recorded time and position, calculate the environmental model based on the acquired shooting time and shooting position,
Generate an object image from object shape data registered in advance using the calculated environmental model,
Perform a search using the generated object image from the sensing image,
A target recognition program that outputs the search result. The position and direction of the environmental model light source,
The target recognition program according to claim 1, wherein the object image is generated by generating a shadow of an object using the environmental model. The environmental model is a weather model or an atmospheric model,
The target recognition program according to claim 1 or 2, wherein the object image is generated by adjusting a pixel value based on the environmental model. The object shape data includes material information of the object,
The target object recognition program according to any one of claims 1 to 3, wherein the object image is generated using a texture determined based on the material information. An image background model,
Image superimposing means for superimposing the image background model and the object image;
The image processing apparatus according to claim 1, wherein an image generated by the image superimposing unit is input to the object shape learning unit. Calculate the environment model and generate the object image for a plurality of imaging times and shooting positions in advance.
When acquiring the sensing image and information of the shooting time and shooting position,
Read out the object image corresponding to the shooting time and shooting position from the previously generated object image,
6. The target recognition program according to claim 1, wherein the search is performed using the searched object image. When obtaining the remote sensing image, obtain only the shooting time of the remote sensing image,
7. The photographing position of the remote sensing image is obtained from the photographing time by reading information from a recording unit that records the photographing position of the sensing image in association with the photographing time in advance. The target recognition program described in Crab.   The target object recognition program according to any one of claims 1 to 7, wherein the generated object image and a search result in the sensing image are displayed side by side on a display unit. An acquisition unit for acquiring a sensing image together with information on a shooting time and a shooting position of the sensing image;
A recording unit for recording at least one environmental model and object shape data;
An object image generation unit that calculates an environment model at the acquired shooting time and shooting position using the environment model, and generates an object image from the object shape data using the calculated environment model;
An object recognition apparatus, comprising: an object search unit that performs an object search using the generated object image from the sensing image and outputs the search result.

Images