JP2009205553A - Image creating device and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image movement calculation method and a device therefor which allow to obtain more natural dynamic images by creating more exact intermediate images from the limited number of images. <P>SOLUTION: An image creating device divides a given image by any one of resolutions, raises the resolution while extracting corresponding areas by learning by a neural network, and extracts corresponding points. Therefore processing shall be carried out at first in the resolution of M×N, and two images are divided into a rectangle area where M×N does not overlap (S401). Although the area is divided here into a rectangle for convenience, dividing is not restricted to this but can be made in any shapes of area. A feature value which indicates a feature of the area is computed for each divided area. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image generation apparatus and method, and more particularly to an image generation apparatus and method for executing processing related to movement between a plurality of images.

  2. Description of the Related Art Conventionally, there has been a technology for generating a moving image in which the form of an image gradually changes by giving an initial image and a final image, such as morphing, and generating one or more images having a form between the images included in the first image It has been proposed (see, for example, Patent Document 1). Such morphing is achieved by, for example, obtaining an effect of distorting the image by moving a control point set on the image and smoothly deforming two types of figures from one to the other. Generally, it is not easy to set. That is, it is not easy to generate a new image without human intervention, such as an observer watching both screens to predict a change pattern and setting corresponding points.

  In certain fields or certain levels, there are some that automatically generate intermediate images. For example, in order to transform the characters displayed on the display device, the outline tree of the target figure is extracted and the outline tree is managed. A technique to be performed has been proposed (see, for example, Patent Document 2). As described above, various image movement calculation processing methods or systems for generating a moving image by generating an intermediate image from a plurality of limited images such as morphing have been proposed.

JP 2005-135046 A JP 2001-175881 A

  However, the conventional image generation method has a problem that it requires manpower when generating an intermediate image, and can handle only a limited number of types of images. Even if the intermediate image can be automatically generated, an unnatural image is generated that is not accurate compared to the original image, or it can be applied only to an image with a small amount of displacement. There is a problem of doing.

  The present invention has been made in view of such a problem, and an image generation apparatus and method for generating a more accurate intermediate image from a plurality of limited images and obtaining a more natural moving image. The purpose is to provide.

  In order to achieve such an object, the invention described in claim 1 of the present application is a method for generating an image by extracting corresponding points between two images by a neural network, each of the two images being A feature amount calculating step of calculating a feature amount vector in each of the divided regions by dividing into a plurality of regions that do not overlap with each other, and calculating a distance between position vectors for any one region of the first image A region search for searching for a region of the second image in which the value of the search function having the distance between the feature quantity vectors as an argument is minimum, a region searched for any one region, and a region before the search An extraction step of extracting corresponding points between the two images by repeating mapping information generation for generating correspondence mapping information between the two images due to a spatial difference between the two images; Characterized by comprising the steps of obtaining an intermediate image by calculating the amount of movement of each of the corresponding points in any of the image between the two images.

  According to a second aspect of the present invention, in the method of the first aspect, the feature quantity vector includes a luminance in the region.

  According to a third aspect of the present invention, in the method of the first or second aspect, the region is M × N rectangular regions.

  According to a fourth aspect of the present invention, in the method according to the third aspect, the extracting step includes a self-organizing map made up of M × N nodes each corresponding to M × N rectangular regions. And extracting corresponding points between two images.

  According to a fifth aspect of the present invention, in the method according to any one of the first to fourth aspects, the search function includes a distance between the position vectors and a weight for the position of the feature amount based on the calculated distance between the feature amount vectors. It is a function which adds the value which multiplied by.

  According to a sixth aspect of the present invention, in the method according to any one of the first to fifth aspects, the extracting step randomly selects any one region of the first image.

  According to a seventh aspect of the present invention, in the method according to any one of the first to sixth aspects, the mapping information generation is performed for a region of the searched second image and its vicinity with a corresponding region of the first image. This is executed by generating the relationship between the first image area and the corresponding second image area as mapping information by moving in the corresponding direction by a value obtained by multiplying the distance by the learning rate constant. And

  The invention according to claim 8 is characterized in that, in the method according to claim 7, the extraction step repeats the area search and the mapping information generation until the amount of movement moved in the mapping generation becomes a predetermined threshold value or less. To do.

  The invention according to claim 9 is an image generation apparatus that generates an image by extracting corresponding points between two images by a neural network, and divides the two images into a plurality of regions that do not overlap each other. , A feature quantity calculating means for calculating a feature quantity vector in each of the divided areas, and a distance between position vectors and a calculated distance between feature quantity vectors for any one area of the first image as arguments The correspondence between the two images due to the spatial difference between the area search for the area of the second image that minimizes the value of the search function and the area searched for any one area and the area before the search Extraction means for extracting corresponding points between two images by repeating mapping information generation for generating relationship mapping information a plurality of times, and each of arbitrary images between the two extracted images Characterized by comprising a means for obtaining an intermediate image by calculating the amount of movement of the corresponding points.

  The invention according to claim 10 is a program for causing a computer to execute a method of generating an image by extracting corresponding points between two images by a neural network, and the method includes: A feature amount calculating step of calculating a feature amount vector in each of the divided regions by dividing into a plurality of non-overlapping regions, and a distance between position vectors for any one region of the first image was calculated An area search for searching for an area of the second image in which the value of a search function having a distance between feature quantity vectors as an argument is minimum, an area searched for any one area, and an area before the search An extraction step for extracting corresponding points between two images by repeating mapping information generation for generating correspondence mapping information between two images due to a spatial difference. When, characterized by comprising the steps of obtaining an intermediate image by calculating the amount of movement of each of the corresponding points in any image between two images extracted.

  As described above, according to the present invention, the feature amount calculating step of dividing the two images into a plurality of regions that do not overlap each other and calculating a feature amount vector in each of the divided regions, An area search for searching for an area of the second image in which the value of the search function having the argument between the distance between the position vectors and the calculated distance between the feature quantity vectors is an arbitrary one area of the image of Correspondence between two images by repeating mapping information generation for generating correspondence mapping information between two images due to a spatial difference between a region searched for an arbitrary region and a region before the search. An extraction step of extracting points; and a step of acquiring an intermediate image by calculating a movement amount of each corresponding point in an arbitrary image between the two extracted images Since, it is possible to obtain a more natural image to generate a more accurate intermediate image from a plurality of limited images.

(First embodiment)
In the present embodiment, the corresponding points between a plurality of images are extracted using a neural network using the principle of the present invention, and the corresponding points are linearly complemented to generate an intermediate image at an arbitrary location. The principle of the present invention will be described. Here, since the corresponding point acquisition processing of the present invention is extremely effective, a moving image that is sufficiently appropriate and closer to the actual motion can be obtained by linear interpolation in generating the intermediate image. However, the present technology is not limited to this. Any interpolation process known in the field can be employed. For example, processing can be performed with appropriate weighting.

  More specifically, assuming the case where an intermediate image is generated based on two different images A and B (images 110 and 120 in FIG. 1) with reference to FIG. 1, according to the principle of the present invention, First, corresponding points between the image 110 and the image 120 are extracted. Here, the original images 110 and 120 may be any images, may be related to each other, or may not be related at all. In other words, in a typical example, there may be images before and after an actual motion of a certain subject, or a human-animal face that is completely unrelated. The present invention can extract more appropriate corresponding points regardless of the specific properties of the image. As can be understood intuitively by using the principle of the present invention, a corresponding point on the image 120 corresponding to an arbitrary point on the image 110 can be determined, and by applying the principle of the present invention, the corresponding point can be determined. The extraction is executed by automatically processing with a computer or the like.

  When the corresponding points of the image 110 and the image 120 are extracted in this way, for example, when the intermediate point on the line connecting the corresponding points is plotted, the intermediate image 130 in the middle is obtained. At this time, a point corresponding to each corresponding point of the images 110 and 120 on the intermediate image 130 is an intermediate point of a line segment connecting the two points. If the intermediate point is taken, the middle image is obtained, but an intermediate image at an arbitrary position between the images 110 and 120 is obtained by plotting the points having an arbitrary ratio of the line segments connecting the corresponding points. Can do.

(System configuration of this embodiment)
The system of the present embodiment is an image movement calculation processing device that supports program development using the above-described software program development technique. FIG. 2 is a diagram showing an overview of the system of this embodiment. In this system, a database 204 as storage means is connected to the control unit 201 of the image movement processing apparatus, and the original image data is stored. If an intermediate image is calculated, it is stored if necessary. The database 204 can also store a program for executing the processing of this embodiment. In this embodiment, the database 204 is shown as being connected to the outside of the control unit 201. However, the present invention is not limited to this. It can also be used by connecting to other databases.

  Note that the control unit 201 has normal functions necessary for performing various processes such as image processing, such as a CPU, a ROM, and a RAM, and a necessary interface drive. The image is displayed on a display 202 connected to the control unit 201. When a command is input, a menu is displayed or a processing result is displayed. Similarly, the input device is not limited to the keyboard 203, and various types of specifications known in the technical field such as a mouse and an input pad (not shown) can be used.

  The overall configuration of the present system is as described above. Under such hardware conditions, the execution of the individual processing of the present embodiment is performed by installing a software program on such hardware. . Each software can be shown in a module configuration as shown in FIG. 3, for example, but this is merely an example, and the functions of each module are further divided into several modules or the functions of several modules are integrated. It goes without saying that modules can be envisaged. Each module configuration will be described below. These modules are executed in cooperation with each other, and the processing of this embodiment described later is achieved.

  FIG. 3 is a diagram illustrating a module configuration of a program executed on the control unit 201 of the present embodiment. For example, the feature point extraction unit 301 divides two images into a plurality of rectangular regions, and calculates a feature amount vector in each region. The corresponding point extraction unit 302 creates a self-organizing map between the nodes by associating the divided regions with the nodes, thereby calculating, for example, the distance between the position vectors for any one region of the last image. The area of the last image in which the value of the search function having the argument of the distance between the feature quantity vectors as an argument is minimum is searched, and the spatial difference between the searched area and the area before the search is calculated. The searched node and its neighboring nodes are updated by a certain function. The corresponding point extraction unit 302 repeats the above search and update for a predetermined number of times, determines the correspondence at a stage where a certain convergence is obtained, and stores it in the database 204 or the like. In this embodiment, the intermediate image calculation unit 303 linearly processes the corresponding points after the corresponding points of the two images are determined for all the regions, and obtains the corresponding points of any intermediate image, and plots all the corresponding points. To calculate an intermediate image. The calculated intermediate image can be stored in the database as one frame of the moving image, and can be output to the display 202 or a printer (not shown) or transmitted to a remote device.

(Process of this embodiment)
The principle of the present invention and the configuration of the system used in the present embodiment have been described above. Next, two images, the first image A and the last image B, will be described as an example of processing that specifically applies the principle of the present invention. Explained. In the present embodiment, an intermediate image is calculated from the images A and B, and a moving image starting from the image A and ending with the image B is generated.

(1) Creation of Multi-resolution Feature Quantity Vector FIG. 4 is a flowchart showing the flow of image movement calculation processing of this embodiment. In this embodiment, first, a given image is divided at any resolution, and a corresponding area is extracted by learning using a neural network, and a corresponding point is extracted by increasing the resolution. For this reason, the processing is first advanced at the resolution of M × N, and the two images are divided into M × N non-overlapping rectangular regions (S401). Here, although the area | region divided | segmented for convenience was made into the rectangle, it is not restricted to this, It can divide | segment into the area | region of any form. For each divided region, a feature amount indicating the feature of the region is calculated. Here, as the feature amount, a value representing any feature that can be used in this technical field can be used. However, in the present embodiment, any value that mainly represents luminance or a property corresponding to luminance is used. Use one of numerical values such as average value, maximum value, minimum value, inflection point value, differential value, second-order differential value in the region, or a combination of some.

  In this embodiment, brightness and information related to brightness are used as the feature amount. However, depending on the image, color information may also strongly represent the feature. In such a case, the effect is obtained using the color difference and related information. In this case, the feature amount extraction method is different, and once the feature amount is acquired, the following processing can be used almost as it is. Similarly, if other attributes representing the characteristics of other images can be used, corresponding points can be extracted in the present embodiment by performing the following processing. Usually, since some of these numerical values, for example, k, are combined as the feature quantity, it becomes a k-dimensional vector, and a feature quantity vector is created.

In this way, by introducing the concept of feature quantity vector, each divided area is a position vector P _{i, j} indicating the position of the area on the image or a position X _{i, j of the} image A and an image B. It is represented by the position Y _{i, j} and the feature vector F _{i, j} (1 ≦ i ≦ M, 1 ≦ j ≦ N), and the corresponding points between the images are extracted using the information including the vector. Can continue.

(2) Extraction of feature regions using self-organizing maps Next, each region is replaced with a node of a neural network, and the nodes having feature vectors F _{i, j and the} like are used as attributes, thereby self-organizing these regions. An appropriate corresponding region is extracted by repeating the process of creating a map, searching for an appropriate node, and updating the node based on the result until the displacement due to the update falls within a certain range.

(I) Initialization of mapping First, in step S402 in FIG. 4, when a self-organizing mapping composed of M × N nodes is created so as to correspond to the area divided into M × N as shown in FIG. i, j has the feature vector F _{i, j, the} position vector position X _{i, j} and the position Y _{i, j} as attributes. Here, let X _{i, j} = Y _{i, j} be an initial state. That is, at the time of starting the processing, the two images have the same position as the corresponding point, and approach the point that actually corresponds depending on the state of the image. That is, in the initial stage, as shown in FIG. 5, the points on the image B corresponding to the image A are in the same place, but as a result of the processing of this embodiment, the points corresponding to the image A are 'Can be mapped on top.

(Ii) Selection of learning target region Next, a region to be processed is randomly selected from the rectangular regions divided by the image B (S404). Actually, by randomly selecting the entire image B, the corresponding node is selected, and the processing described later is executed for that node. After extracting corresponding points for the selected region, another region is selected to finally cover the previous region of the image B. In this embodiment, this selection needs to be made at random. . In other words, in this embodiment, if there is regularity in the selection of the region, depending on the region, there is a possibility of convergence to a point different from the original corresponding point by learning, so that a local solution may be selected. It is necessary to do. The random number to be used need not be completely random but may be a pseudo-random number as long as it has a certain randomness.

  For this selection, a point or part that is not evaluated as a corresponding point, for example, a part that does not exist in the first image but exists in the last image, specifically, a photograph in which the first image faces the person's side. Yes, when the last photo is a photo with the person facing the front, the ear that was initially hidden may not be able to be processed because there is no corresponding point after processing the area, There is a possibility of extracting a new point. If the processing of this embodiment is performed while leaving such an area as it is, unnecessary processing increases and processing capacity is also affected. Therefore, in this embodiment, a region to be post-processed is selected by removing such abnormal points using a Gaussian distribution.

  In this embodiment, as a result of dividing the image into M × N rectangles, the respective rectangular areas can be smoothly performed without overlapping each other. However, the principle of the present invention is not limited to this embodiment. Contrary to the processing example of the form, the corresponding point is the reference instead of the region. That is, since the present invention finds corresponding points between a plurality of images, it is only necessary to compare the points between the images, and it is possible to select an arbitrary point in the image and derive the feature amount. If possible, it is possible to select a point appropriately and perform processing based on the selected point. In the present embodiment, since luminance and color differences are used as feature amounts, it is often easier in processing using a computer to calculate feature amounts not for the points themselves but for a certain area around the points. As a result, in this embodiment, processing is performed for a region for convenience, and the center points of the region are determined as corresponding points, but the feature amount is calculated using an arbitrary region of the point while matching the points from the beginning. Can also be processed. In this case, unlike the present embodiment, the regions may overlap each other as well as the shape of the regions, or conversely there may be gaps.

(Iii) Search for Nodes In order to approach a plausible corresponding point, a node having a minimum value of a predetermined function of the feature amount for the selected node is searched (S405). That is, a function indicating a certain degree of similarity is set to search for corresponding points that are likely to be related in the two images based on the feature amount, and processing is performed so that the value becomes a certain value. Specifically, in the present embodiment, for example, the following values are used as the similarity function D _{i, j} (DP _{i, j} (b) _{i, j} , DF _{i, j} (b)) (similar function D _{i , j} is a function that takes a lower value as the degree of similarity increases, that is, the degree of similarity becomes higher). Here, DP _{i, j} indicates the distance between the position vectors of the current corresponding point and the node to be searched, that is, the distance in the geometric space. For example, L2 norm || Pb-Y _{i, j} || can do. DF _{i, j} indicates the distance between feature vectors of the current corresponding point and the node to be searched, for example, the distance in the luminance space or the color space.

It can be.

(I) D _{i, j} = DP _{i, j} (b) + βDF _{i, j} (b): β is a weight for the geometric space of the luminance space. (II) D _{i, j} = DP _{i, j} (b) * ( DF _{i, j} (b)) ^β : β is a weight for the geometric space of the luminance space. Either (I) or (II) can be arbitrarily selected as a similarity function, both of which are effective for implementing the present invention. Although (I) is linear, it is known that it is relatively easy to handle and highly effective.

(Iv) Update of adjacent node The node I, J = arg min D _{i, j} (b) searched for minimizing the value of the similarity function and its adjacent node are updated based on the following expression (S406).
Y _{i, j} (t) = Y _{i, j} (t-1) + αδ (X _{i, j} (t)-Y _{i, j} (t-1)) (2)

  Here, α (0 <α ≦ 1) is a learning rate coefficient. For example, a fixed value such as 0.01 or a monotonically decreasing function can be used. However, the present invention is not limited to this, and any appropriate value known in this technical field can be used. Any value can be used. In general, when searching for corresponding points between images with relatively little movement, a relatively small constant such as 0.01 can be used for stable convergence, but searching for corresponding points between images with large movements is possible. In this case, if a small α is used from the beginning, it may be an extreme solution and an exact corresponding point may not be found. Therefore, a function that takes a relatively large value at first and decreases monotonically as described later δ is used. Sometimes it is effective.

  For δ, assuming a function range (t) that monotonously decreases with respect to t and takes a value of 0 or more, δ = 1 when | Ii | ≦ range (t) and | Jj | ≦ range (t), Otherwise, assume a value of δ = 0. In the present embodiment, 8 Neighbors for performing processing on eight neighboring regions of the target point (region) 801 as shown in FIG. 6 is used. That is, the vicinity of the target area 801 is updated by the above equation (2) and converges to a certain correspondence. This is because a more appropriate corresponding point can be searched by incorporating the state of the image around the target point.

  In the present embodiment, 8 Neighbors are used as adjacent nodes as described above. However, the present invention is not limited to this, and various methods known in this technical field can be used. For example, 4 Neighbors can be used. In this case, δ described above is δ = 1 when | I−i | ≦ range (t) or | J−j | ≦ range (t), and δ = 0 otherwise.

  After updating in this way, the node that minimizes the similarity function is searched again, and the above processing is repeatedly executed (S407). That is, by repeating the processes (iii) and (iv) above a certain number of times, the corresponding area can be acquired for the area selected in (ii) above. For one region selected at random, the above processing is performed to obtain corresponding points, and thereafter the same processing is performed on the remaining regions at random to perform the entire image processing, and the corresponding points are extracted. (S403).

  When the corresponding points of M × N resolution are obtained in this way, the resolution is increased stepwise as shown in FIG. 7, and more accurate corresponding points are obtained with the resolution of M × N. Normally, the processing is performed in about three stages as shown in FIG. 7, but the processing is not limited to this, and the processing can be performed with a smaller number or a larger number according to the characteristics of the system and the image.

  Here, the fixed number of times varies depending on the system and also on the nature of the target image, but in this embodiment, for example, the processing of (iii) and (iv) is performed for each of the regions divided into M × N. If the calculation is performed from 16 times to 64 times, and M = N = 256 and calculation is performed under the condition of three stages of resolution, there are cases where the calculation is repeated about 3 to 12 million times. As the number of repetitions, a predetermined value can be determined as a result of experiments or the like, but for example, it can be used as a criterion for determining the number of repetitions that the displacement due to the update process (iv) is less than a certain value.

(Generate intermediate image)
If the corresponding points can be determined for the original plurality of images by the above processing, various image processing becomes possible based on this information. In the present embodiment, among the various application fields of the present invention, as the simplest, in particular, an intermediate image between images is calculated from two images. If an intermediate image is obtained, it can be output as a plurality of still images, and a fixed number of frames can be output every second to generate a moving image. For example, as described above, the corresponding points of the images A and B are extracted, and when the intermediate points on the line connecting the corresponding points are plotted, the corresponding points of the images A and B correspond to the intermediate image. The point to be used is the midpoint of the line segment connecting both points. If a point on the intermediate image such as the intermediate point is calculated and plotted, an intermediate image is generated. If the intermediate point is taken, the middle image is obtained, but an intermediate image at an arbitrary position between the images A and B is obtained by plotting points having an arbitrary ratio between the line segments connecting the corresponding points. Is possible.

  In the present embodiment, a relatively natural intermediate image can be obtained by linear processing as described above. However, the present invention is not limited to this, and the change becomes larger as the image B is approached except for linearity, that is, without using a fixed ratio. Can be weighted, or the change can be increased before and after the intermediate image. Thus, an intermediate image can be generated using any weighting function depending on the desired system purpose or any other method known in the art.

  Further, in this embodiment, an intermediate image is generated from two original images. However, various other points can be obtained by extracting corresponding points between a plurality of images using the principle of the present invention. It can be applied to image processing. For example, various application processes can be executed by an intermediate image extraction process similar to that performed for the above-described two images using three images. In other words, for example, consider preparing three facial images of the same person and combining them to generate various facial expressions. Generally, there are three types of facial expressions that express the most emotions: crying, laughing, and angry faces, and these are the original images. First, in step 1, the processing of the present embodiment is executed for each image pair to extract corresponding points of the three images. Next, in step 2, two image pairs are arbitrarily selected and a first intermediate image is generated at a predetermined ratio, and then another predetermined ratio is set for the remaining one image and the first intermediate image. To generate a second intermediate image. As described above, various facial expressions in which three kinds of original emotions are appropriately mixed can be obtained. By simply changing the predetermined ratio, it is possible to obtain facial images with surprisingly diverse expressions.

  One of various examples obtained in this way is an image as shown in FIG.

(Second Embodiment)
In the first embodiment described above, the present invention is applied to a plurality of images given in advance to generate an intermediate image or a moving image. In this embodiment, a client is connected to a server connected via the Internet. When the device accesses, an intermediate image is generated from a plurality of images in the server and provided to the client device.

  In this embodiment, the server provides, for example, product advertisements and product information to the client device. At that time, the image of the product is applied to the image of the product to perform image processing so as to attract more customers' interest. Can be. That is, when the customer selects a desired product from the client device and requests provision of information, the server applies the present invention to the image of the product stored in advance together with other information of the corresponding product, and creates a new image. Generate and send to client device. Specifically, an intermediate image is generated from a plurality of product images. For example, an image viewed obliquely based on a photograph from the front of a car and a photograph from a lateral direction can be generated and output, or It is possible to generate a moving image from a still image and explain the actual usage situation. By using the present invention, not only a natural intermediate image can be obtained by extracting appropriate corresponding points, but also the calculation can be made more efficient, so it is possible to cope with interactive systems via the Internet by performing high-speed processing. .

  As described above, by using the present invention, product information can be provided via the Internet.

It is a figure explaining extracting the point respectively corresponding between the image of one Embodiment concerning this invention, and an image. It is a block diagram which shows the system configuration | structure of one Embodiment concerning this invention. It is a figure which shows the structure of the functional module of this embodiment. It is a flowchart which shows an example of the process of this embodiment. It is a figure for demonstrating the search of the corresponding point of the process of this embodiment. It is a figure for demonstrating the vicinity of the process of this embodiment. It is a figure for demonstrating the resolution of the process of this embodiment.

Explanation of symbols

201 Computer 202 Screen 203 Input unit 204 Database 103 Network 104 Server 301 Feature amount extraction unit 302 Corresponding point extraction unit 303 Intermediate image extraction unit

Claims (10)

A method of generating an image by extracting corresponding points between two images by a neural network,
A feature amount calculating step of dividing the two images into a plurality of regions that do not overlap each other and calculating a feature amount vector in each of the divided regions;
For an arbitrary region of the first image, a region of the second image in which the value of the search function having the argument between the distance between the position vectors and the calculated distance between the feature vectors is minimized is searched. By repeating the area search, the mapping information generation for generating the correspondence mapping information between the two images based on the spatial difference between the area searched for the one arbitrary area and the area before the search, two times are obtained. An extraction step for extracting corresponding points between images;
Obtaining an intermediate image by calculating a moving amount of each corresponding point in an arbitrary image between the two extracted images.   The method according to claim 1, wherein the feature amount vector includes luminance in the region.   The method according to claim 1, wherein the area is M × N rectangular areas.   In the extracting step, corresponding points between the two images are extracted using a self-organizing map composed of M × N nodes respectively created corresponding to the M × N rectangular regions. The method according to claim 3.   The search function is a function for adding a distance between the position vectors and a value obtained by multiplying the calculated distance between the feature quantity vectors by a weight for the position of the feature quantity. 5. The method according to any one of 4.   The method according to claim 1, wherein the extracting step randomly selects an arbitrary region of the first image.   In the mapping information generation, the area of the searched second image and its vicinity are moved in the corresponding direction by a value obtained by multiplying the distance from the corresponding area of the first image by a learning rate constant, The method according to claim 1, wherein the method is executed by generating a relationship between a first image area and a corresponding second image area as mapping information.   The method according to claim 7, wherein the extraction step repeats the area search and mapping information generation until a movement amount moved by the mapping generation becomes a predetermined threshold value or less. An image generation apparatus for generating an image by extracting corresponding points between two images by a neural network,
A feature amount calculating means for dividing the two images into a plurality of regions that do not overlap each other and calculating a feature amount vector in each of the divided regions;
For an arbitrary region of the first image, a region of the second image in which the value of the search function having the argument between the distance between the position vectors and the calculated distance between the feature vectors is minimized is searched. By repeating the area search, the mapping information generation for generating the correspondence mapping information between the two images based on the spatial difference between the area searched for the one arbitrary area and the area before the search, two times are obtained. Extraction means for extracting corresponding points between images;
An image generation apparatus for calculating image movement, comprising: means for acquiring an intermediate image by calculating a movement amount of each corresponding point in an arbitrary image between the two extracted images. A program for causing a computer to execute a method of generating an image by extracting corresponding points between two images by a neural network, the method comprising:
A feature amount calculating step of dividing the two images into a plurality of regions that do not overlap each other and calculating a feature amount vector in each of the divided regions;
For an arbitrary region of the first image, a region of the second image in which the value of the search function having the argument between the distance between the position vectors and the calculated distance between the feature vectors is minimized is searched. By repeating the area search, the mapping information generation for generating the correspondence mapping information between the two images based on the spatial difference between the area searched for the one arbitrary area and the area before the search, two times are obtained. An extraction step for extracting corresponding points between images;
And a step of acquiring an intermediate image by calculating a movement amount of each corresponding point in an arbitrary image between the two extracted images.

Images