JP2012079004A - Feature point extraction device, image processing device, feature point extraction method, feature point extraction program and the like - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device, a method and the like for recognizing an object displayed in an image.SOLUTION: A feature point selection unit 8 specifies a candidate point to be a candidate of a feature point from pixel information input from a feature point score value selection unit 6. The feature point selection unit 8, when determining the newly specified candidate point to exist in the area of an already set feature point, determines the magnitudes of score values of the newly specified candidate point and the feature point corresponding to the area of the existing feature point where the candidate point exists, and when determining the score value of the newly specified candidate point to be higher, sets the newly specified candidate point as a new feature point.

Description

  The present invention relates to a technical field such as an apparatus and method for recognizing an object displayed in an image.

  In an image processing technique for recognizing an object displayed in an image and analyzing the movement of the object, for example, a known KLT (Kanade Lucas Tomasi) method, Moravec operator, SUSAN operator, or Harris operator is used. Feature points are extracted, and tracking processing is performed on the extracted feature points using the Newton-Raphson method.

  In addition, as a technique for extracting feature points from an image in a balanced manner, Patent Document 1 divides an input image into predetermined partial images, and extracts a predetermined number of characteristic points from each partial image. Is disclosed.

JP-A-2006-350897

  However, in the feature point extraction by the above-described methods, the periphery of a pixel (value having a high evaluation value) having a large luminance gradient (luminance difference) in the image is extracted as the feature point, and thus the evaluation value of the image is high. There was a problem that the values were concentrated locally.

  That is, in the above feature point extraction method, human clothes displayed as pixels having a large luminance gradient, edge portions of images, and the like (for example, contour portions of images and check patterns of clothes) are extracted as feature points. As a result, the extracted feature points are concentrated in one point, and feature points cannot be extracted in a well-balanced manner on the entire screen, and accurate operation analysis can be performed even if processing such as tracking is performed using the feature points. There was no problem.

  Accordingly, the present invention has been made in view of the above problems, and an example of the object thereof is to provide an apparatus and method for extracting accurate and available feature points from a captured image.

  In order to solve the above-described problem, the invention described in claim 1 is configured to configure the input image to track a feature point indicated in the image and obtain a motion vector of the feature point. A feature point extraction device that sequentially sets a predetermined quantity of the feature points according to a predetermined order from pixel information having predetermined position information, the candidate specifying candidate points that are candidate feature points from the pixel information A point specifying unit, a score value input unit that receives an input of a score value indicating the evaluation value of the candidate point, and a candidate point newly specified by the candidate point specifying unit are determined in advance from feature points that have already been specified. A region determination unit that determines whether or not the region is within the specified region, and a candidate point that is newly specified by the region determination unit exists within a region that is predetermined from the already specified feature point. A score value judging means for judging the magnitude of a score value inputted when the newly identified candidate point and the already identified feature point are identified as a candidate point when determined, And a feature point setting unit that sets the candidate point as a new feature point in the image when the score value determination unit determines that the score value of the candidate point is large.

  According to the present invention, it is possible to extract accurate and available feature points from a captured image.

  According to a second aspect of the present invention, there is provided the feature point extracting apparatus according to the first aspect, wherein a divided region setting unit that sets a divided region obtained by dividing the image into predetermined regions, and the candidate point specifying unit. Divided region determination means for determining whether or not a specified feature point already exists within a range of a predetermined divided region from the divided region as a reference, based on the divided region to which the newly identified candidate point belongs. And the score value determining means is an area in which a feature point already specified within a predetermined divided area from the divided area determined by the divided area determining means exists, If it is determined that a newly identified candidate point exists within a predetermined region from the already identified feature point, the newly identified candidate point and the already identified feature point Identified as a candidate point And comparing the input score value when it was.

  According to the present invention, it is possible to extract accurate and available feature points from a captured image while reducing the processing load of the feature point extraction device.

  The invention according to claim 3 is the feature point extraction device according to claim 1, wherein the newly specified candidate point is predetermined from the already specified feature point by the region determination unit. When it is determined that the image does not exist in the region, the feature point determination means sets the candidate point as a new feature point in the image.

  The invention according to claim 4 is the feature point extraction device according to claim 2, wherein the divided region determination unit sets the reference based on the divided region to which the newly identified candidate point belongs. If it is determined that there is no feature point already specified within the range of the predetermined divided region from the divided region, the feature point determining means sets the candidate point as a new feature point in the image It is characterized by that.

  The invention according to claim 5 is equipped with the feature point extraction device according to any one of claims 1 to 4 for tracking the feature points shown in the image and obtaining a motion vector of the feature points. It is characterized by.

  According to a sixth aspect of the present invention, in order to track a feature point indicated in an image and obtain a motion vector of the feature point, pixel information that constitutes the input image and has predetermined position information in the image A feature point extraction method in a feature point extraction device that sequentially sets a predetermined quantity of the feature points according to a predetermined order, wherein the feature point candidate specification specifies a candidate point that is a candidate for a feature point from the pixel information Newly specified candidate points are determined in advance from the already specified feature points by the step, the score value input step for receiving input of score values indicating the evaluation values of the candidate points, and the feature point candidate specifying step. A region determination step for determining whether or not a region exists in the region, and a candidate point newly identified by the region determination step is present in a region determined in advance from the already identified feature point A score value determining step for determining a magnitude of a score value input when the newly specified candidate point and the already specified feature point are specified as candidate points, A feature point setting step of setting the candidate point as a new feature point in the image when the score value determining step determines that the score value of the candidate point is large. To do.

  According to a seventh aspect of the present invention, in order to perform tracking processing of a feature point of a subject shown in an image and perform motion analysis of the subject, pixels that constitute the input image and have predetermined position information in the image A candidate point specifying means for specifying a candidate point that is a candidate for a feature point from the pixel information; a computer included in a feature point extracting device that sequentially sets a predetermined quantity of the feature points according to a predetermined order from the information; A score value input unit that receives an input of a score value indicating an evaluation value of a candidate point, and a candidate point newly specified by the candidate point specifying unit exists within a predetermined region from an already specified feature point The area determination means for determining whether or not the area determination means has determined that the newly specified candidate point is within a predetermined area from the already specified feature point In addition, the newly specified candidate point and the score value determining means for determining the magnitude of the score value input when the already specified feature point is specified as the candidate point, the score value determining means Thus, when the score value of the candidate point is determined to be large, the candidate point is made to function as a feature point setting unit that sets the candidate point as a new feature point in the image.

  As described above, according to the present invention, accurate and available feature points can be extracted from a captured image.

It is a block diagram which shows the structure of the image processing apparatus concerning this embodiment. It is a figure which shows the detail of a feature point table and a candidate point table. FIG. 6 is a schematic diagram schematically showing an outline of the operation of the feature point selection unit 8. FIG. 10 is a sequence diagram showing an operation of the feature point selection unit 8.

[1. Overview of image processing apparatus configuration and functions]
First, the best embodiment of the present application will be described with reference to the accompanying drawings.

  The embodiment described below is an embodiment when the feature point extraction device of the present application is applied to an image processing device.

  The image processing apparatus of the present application is an apparatus for recognizing an object displayed in a captured image and performing image processing for analyzing the movement of the object. Specifically, the image processing apparatus extracts a feature point from a captured image using a known KLT (Kanade Lucas Tomasi) method or the like, and applies a gradient method or a Newton-Raphson method to the extracted feature point. And performing a tracking process (that is, tracking a feature point shown in an image), and obtaining a motion vector of the feature point, thereby performing the image process.

  The feature point extraction device of the present application is incorporated in the image processing device and has a function of setting predetermined feature points to be described later in order to extract accurate and available feature points from the captured image.

  First, the configuration and functional overview of the image processing apparatus according to the present embodiment will be described with reference to FIG.

  FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to the present embodiment.

  The image processing apparatus S includes an optical system engine 1, an input image acquisition unit 2, a frame buffer 3, a feature point extraction unit 4, a differential image holding unit 5, a feature point score value selection unit 6, a feature point threshold storage unit 7, a feature point A selection unit 8 (an example of the feature point extraction device of the present application), a feature point information storage unit (region) 9 for each image, and a feature point motion vector extraction unit 10 are provided.

  In the optical system engine 1, the light taken in from the optical system lens 1a is adjusted by a not-shown diaphragm mechanism or the like and formed as an optical image (image) on the image sensor 1b. Such an image is converted into an electrical signal by the image sensor 1b and output to the input image acquisition unit 2 (camera I / F 2).

  The image sensor 1b converts the imaged optical image into an electrical signal proportional to the brightness of the optical image by a photoelectric exchange function using, for example, photodiodes arranged in a matrix, and the input image acquisition unit 2 (camera I / F). As an example of the image sensor 1b, for example, a charge coupled semiconductor (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like is applied.

  The input image acquisition unit 2 (camera I / F) processes the signal output from the image sensor 1b, and according to the resolution of the display, the color information (any primary color component of R, G, B) constituting the image Pixel (dot) information (raster data) having a charge etc. corresponding to 1), and a function of outputting the information to the frame buffer 3.

  As an example of this resolution, VGA (640 × 480 dots), SVGA (800 × 600 dots), XGA (1024 × 768 dots) or the like is applied, and pixel information is generated according to the resolution. That is, about 300,000 pieces of pixel information are generated by VGA, about 400,000 pieces of pixel information are generated by SVGA, and about 780,000 pieces of pixel information are generated by XGA.

  The pixel information is given predetermined position information (for example, coordinate information) in the image corresponding to the resolution. Therefore, for example, in the VGA described above, an image is formed by arranging 640 dot pixels arranged in the X direction (longitudinal direction of the image) with reference to a predetermined origin and further arranging 480 dots in the Y direction (vertical direction of the image). For this reason (that is, one grid of 640 × 480 dots can be grasped as one pixel), the position of a specific pixel constituting the image can be specified by XY coordinates. Hereinafter, the position information given to the pixel information is simply referred to as “pixel position”.

  As an example of the input image acquisition unit 2, for example, an interface corresponding to a known camera link that is a standard specification for connecting an industrial digital camera and an image input board is applied.

  The frame buffer 3 is a memory area for storing the input pixel information (pixel information, bitmap). For example, under the control of a program, the stored pixel information is stored outside (for example, feature points). A function of outputting the data to the extraction unit 4 and the like in a batch.

  The feature point extraction unit 4 identifies input pixel information and uses a known KLT method to select candidate points (hereinafter simply referred to as “candidate points”) as feature point candidates to be selected by the feature point selection unit 8. And a score value indicating the evaluation value.

  Here, the score value is calculated as an evaluation value of the candidate point, and is defined as a value indicating the magnitude of the luminance gradient of the pixel information in the KLT method described above. That is, when the score value is large, the luminance gradient of the pixel information is defined as large, and when the score value is small, the luminance gradient of the pixel information is defined as small. Therefore, the candidate points can be evaluated based on the score value.

  The score value calculation method is a well-known technique and thus will not be described in detail. However, the feature point extraction unit 4 calculates the eigenvalue G based on the pixel information calculated as the candidate point, and the eigenvalue G It is calculated as a score value. That is, in the known KLT method, an eigenvalue G is calculated as a luminance gradient, and pixels having a large value indicated by the eigenvalue G are calculated as candidate points for feature points.

  As will be described in detail later, the feature point selection unit 8 of the present application evaluates the value of the eigenvalue G calculated in this way (that is, compares the magnitude relationship of the values indicated by the eigenvalue G), and sets the candidate point as a new value. It is set as a feature point.

  The calculation method of the eigenvalue G is a well-known method and will not be described in detail, but an example will be described below. Note that the calculation method of the eigenvalue G is not limited to the following.

  As a method for calculating the eigenvalue G, first, the moment matrix of the spatial luminance gradient I at the position X = (x, y) of the image point is expressed by Equation (1).

また、モーメント行列Ｍを式（２）とした場合に、式（２）の固有値λは、Ｅを単位行列として、式（３）のλに対する解として求められる。

Further, when the moment matrix M is expressed by equation (2), the eigenvalue λ of equation (2) is obtained as a solution to λ of equation (3), where E is a unit matrix.

式（３）の解は、式（４）となる。

The solution of equation (3) is equation (4).

本実施例では、上記λのうち、式（５）を固有値Ｇとして算出する。

In this embodiment, among the above λ, equation (5) is calculated as the eigenvalue G.

このように算出された固有値Ｇの値を示すλを用いて、特徴点選択部８は、当該λが示す値の大小関係を比較し、上記特徴点を設定するようになっている。

Using the λ indicating the value of the eigenvalue G calculated in this way, the feature point selection unit 8 compares the magnitude relationships of the values indicated by the λ, and sets the feature point.

  In the present embodiment, the value λ calculated by the eigenvalue G using the KLT method is used as the score value. However, the present invention is not limited to this, and is quantified by, for example, known template matching or color matching. The value may be applied as a score value.

  In the following description, the score value indicates the eigenvalue G (λ) calculated based on the pixel information.

  Then, the feature point extraction unit 4 outputs the calculated score value to the differential image holding unit 5 and the feature point score value selection unit 6.

  The differential image holding unit 5 has a function of holding a differential image of the input image, and this differential image is used in, for example, calculation of a motion vector using a gradient method. 10 is output.

  The feature point score value selection unit 6 has a function of outputting pixel information corresponding to a score value exceeding a predetermined threshold from the score value input from the feature point extraction unit 4 to the feature point selection unit 8. The threshold value can be arbitrarily configured. For example, a predetermined score value is set as a threshold value (that is, only a value exceeding the predetermined threshold value is output to the feature point selection unit 8) or surrounded by a predetermined coordinate value. This range can also be used as a threshold value. This threshold value is stored in the feature point threshold value storage unit 7.

  By limiting the quantity of pixel information to be output to the feature point selection unit 8 in this way, the processing load on the feature point selection unit 8 and the entire image processing apparatus S can be reduced, and the processing speed can be increased.

  The feature point score value selection unit 6 may not be provided, and all pixel information corresponding to the score value input from the feature point extraction unit 4 may be output to the feature point selection unit 8.

  The feature point selection unit 8 functions as a candidate point specification unit, a score value input unit, a region determination unit, a score value determination unit, a feature point setting unit, a divided region setting unit, and a divided region determination unit of the present application. The candidate point input from the value selection unit 6 is identified, and the candidate point (newly identified candidate point) does not exist within a predetermined region from the already identified feature point (that is, close to it). The candidate points exist so as to have a distance greater than or equal to a predetermined distance), and a predetermined number of candidate points having a high score value are set as feature points.

  Note that “in a predetermined region from already set feature points” indicates a range that is a predetermined distance (for example, 10 × 10 dots) away from a pixel position with the feature point as a reference. Further, the area has an area corresponding to each set feature point. Therefore, the region exists for each feature point.

  Hereinafter, an area determined in advance from already set feature points is simply defined as “already set feature area”.

  In a general feature point extraction method, human clothes displayed as pixels with a large luminance gradient or edge portions of images are extracted as feature points. As a result, the extracted feature points are dense and accurate features are extracted. Points cannot be extracted, and accurate motion analysis cannot be performed even if processing such as tracking is performed using the feature points.

  However, according to each of the means included in the feature point selection unit 8, in order to identify the candidate point as a feature point when the newly identified candidate point does not exist in the already set feature point region, Accurate feature points can be extracted without dense feature points. Therefore, accurate and available feature points can be extracted from the captured image.

  Each means included in the feature point selection unit 8 will be described. First, the feature point selection unit 8 as the candidate point specification unit uses the pixel information input from the feature point score value selection unit 6 as a feature point candidate. Is identified as a candidate point.

  Further, the feature point selection unit 8 as a score value input unit accepts input of score values corresponding to the identified candidate points calculated by the feature point extraction unit 4. The candidate point specified by the feature point selection unit 8 is pixel information as described above. The pixel information has already been scored by the feature point extraction unit 4. The feature point selection unit 8 accepts input of the calculated score value.

  Further, the feature point selection unit 8 as the region determination unit determines whether or not the newly specified candidate point exists in the region of the already set feature point.

  In addition, when the feature point selection unit 8 as the score value determination unit determines that the newly specified candidate point exists in the already set feature point region, the newly specified candidate point When the already specified feature point is specified as a candidate point, the magnitude of the score value input from the feature point extraction unit 4 is determined.

  In addition, when the feature point selection unit 8 as the feature point setting unit determines that the score value of the newly specified candidate point is large, the feature point selection unit 8 sets the candidate point as a new feature point in the image.

  Details of the divided region setting unit and the divided region determining unit will be described later.

  The feature point information storage unit 9 for each image area stores information on the feature points specified by the feature point selection unit 8, and as an example thereof, a feature point table and a candidate point table are stored.

  Here, the feature point table and the candidate point table will be described with reference to FIG.

  FIG. 2A is a diagram illustrating details of the feature point table, and FIG. 2B is a diagram illustrating details of the candidate point table.

  As shown in FIG. 2A, the feature point table stores information about the feature points set by the feature point selection unit 8, and specifically, as shown in FIG. As described above, the X coordinate, the Y coordinate, the score value, the hit flag, and the registration flag are stored in association with each of the already set feature points (1) to (100).

  Each of the feature points (1) to (100) indicates a predetermined number of feature points set by the feature point selection unit 8. That is, the feature points (1) to (100) indicate feature points that have already been set. In the example shown in FIG. 2A, it is shown that 100 feature points have already been set as the predetermined quantity, but the present invention is not limited to this, and an arbitrary quantity of feature points can be set.

  In the X coordinate and Y coordinate, the position of a pixel already set as a feature point is stored. The score value stores the score value of a pixel that has already been set as a feature point. Note that the score value is a value calculated from luminance gradient information with peripheral pixels in one pixel on the image as described above, and indicates a higher value as the gradient is steeper.

  The hit flag is specified by the feature point selection unit 8 (candidate of the feature point input from the feature point score value selection unit 6), each of the candidate points (newly input candidate points) already set. This is a flag indicating whether or not the feature points (1) to (100) are present in the region (the region of each identified feature point).

  For example, if the candidate point specified by the feature point selection unit 8 exists in any of the already set feature points (1) to (100), the candidate point is already set. 1 is substituted for the value of the hit flag of each of the feature points (1) to (100) corresponding to the regions of the feature points (1) to (100).

  The registration flag is a flag indicating whether or not it is set as a feature point. As the value of the registration flag, 1 is substituted when it is set as a feature point, and 0 is substituted when it is not registered as a feature point.

  In the example shown in FIG. 2A, a state in which all of a predetermined number of feature points are set by the feature point selection unit 8 is shown. On the other hand, when the feature point selection unit 8 has set only feature points of a quantity less than a predetermined quantity, a blank location exists in the feature point column of the feature point table.

  The candidate point table shown in FIG. 2 (B) stores information on candidate points newly specified by the feature point selection unit 8, specifically, as shown in FIG. 2 (B). In addition, the X coordinate and Y coordinate of the newly identified candidate point and a deletion flag (not registered flag) are stored in association with each other.

  In the X coordinate and the Y coordinate, the pixel position of the newly specified candidate point is stored.

  The deletion flag is a flag indicating whether or not a candidate point newly specified by the feature point selection unit 8 is set as a feature point. As the value of the deletion flag, 0 is substituted when a candidate point newly specified by the feature point selection unit 8 is set as a feature point, and 1 is substituted when it is not set as a feature point.

  Note that information about feature points (including the feature point table and the candidate point table) is used when executing the tracking process, and thus two images (that is, consecutive frames) are held. Then, the feature point information storage unit 9 for each image region outputs information on the stored feature points to the feature point motion vector extraction unit 10.

  The feature point motion vector extraction unit 10 has a function of performing tracking processing on information about feature points input from the feature point information storage unit 9 for each image area using a known Newton-Raphson method or the like. . Since the tracking process is a known technique, a detailed description thereof will be omitted, but it refers to a process of analyzing the movement amount of the feature points in two consecutive images.

In order to implement a part or all of the functions of the frame buffer 3 to the feature point motion vector extraction unit 10, in order to execute each process on part or all of the frame buffer 3 to the feature point motion vector extraction unit 10. It is possible to apply specially designed hardware (so-called hardware acceleration). By implementing the feature point selection unit 4 and the like with dedicated hardware, it is possible to perform operations in parallel by making use of the parallelism of the circuit in the hardware, so that sequential instruction execution (for example, software by the CPU) This is because, for example, a higher processing speed is realized than in the case of (execution). Needless to say, a control device such as a CPU, a RAM, and a ROM can execute the functions of part or all of the frame buffer 3 to the feature point motion vector extraction unit 10 under the control of software. Absent.
[2. Operation of the feature point selection unit 8]
The feature point selection unit 8 in this operation example functions as a divided region setting unit and a divided region determination unit in addition to the above-described units, and identifies a feature point.

  In the following, an outline of the operation of the feature point selection unit 8 that functions as a divided region setting unit and a divided region determination unit in addition to the above-described units will be described in detail with reference to FIGS.

  FIG. 3 is a schematic diagram schematically showing an outline of the operation of the feature point selection unit 8, and FIG. 4 is a sequence diagram showing the operation of the feature point selection unit 8.

  First, the image used in this operation example is preliminarily divided into predetermined divided areas by the feature point selection unit 8 as divided area setting means.

  Specifically, as shown in FIG. 3, the feature point selection unit 8 serving as a divided region setting unit divides the image 21 into divided regions 1 to 30 (1 in FIG. 3) divided into predetermined divided regions. (Rectangular region to which the number of ~ 30) is assigned.

  Here, as described above, pixels according to the resolution are arranged in the image, and the image in this operation example has pixels of 640 dots in the X direction in the Y direction with respect to the origin (0, 0). A VGA in which 480 dot pixels are arranged is employed. That is, about 300,000 pixels (0, 0) to (639, 479) are arranged in the image. In each divided area, the number of pixels obtained by dividing the number of pixels of the image by each divided area (approximately 10,000 pixels are arranged in one divided area) is arranged. The feature point selection unit 8 in this operation example uses the origin as a reference in accordance with coordinate values (that is, (0, 0), (1, 0),... (639, 0), (0, 1 )...)), The new candidate points are sequentially specified. However, the present invention is not limited to this, and the candidate points can be specified in any order.

  Then, the feature point selection unit 8 specifies candidate points that are candidate feature points from the pixel information input from the feature point score value selection unit 6 (step S1 in FIG. 4). FIG. 3 schematically shows a candidate point 24 (double circle in FIG. 3) as the identified candidate point. Hereinafter, the pixel information input from the feature point score value selection unit 6 is referred to as “newly identified candidate points”.

  Then, the feature point selection unit 8 identifies in which divided area of the screen the candidate point is present from the pixel position of the identified candidate point (step S2 in FIG. 4). Then, the feature point selection unit 8 stores coordinate values (X coordinate and Y coordinate values) as pixel positions of the newly specified candidate points in the candidate point table.

  Next, the feature point selection unit 8 initializes the value of the hit flag in the feature point table and the value of the deletion flag in the candidate point table (that is, substitutes 0) (step S3 in FIG. 4).

  Next, the feature point selection unit 8 serving as a divided area determination unit is already set within a predetermined divided area from the divided area set as the reference with respect to the divided area to which the identified candidate point belongs. It is determined whether or not a feature point exists (step S4 in FIG. 4).

  The range of the predetermined divided area from the reference divided area is a range obtained by adding a predetermined number of divided areas to the divided area to which the identified candidate point belongs. Specifically, as shown in FIG. 3, the range of the predetermined divided area from the reference divided area is divided into five divided areas 8 as a predetermined quantity in the divided area 15 to which the identified candidate point 24 belongs. , 9, 10, 14, and 16. Hereinafter, the range of the predetermined divided area from the reference divided area is referred to as a “neighbor check area”.

  As the neighborhood check area, FIG. 3 shows a neighborhood obtained by adding the five divided areas 8, 9, 10, 14, and 16 as a predetermined quantity to the divided area 15 to which the identified candidate point 24 belongs. It will be shown as a check area 25.

  If the feature point selection unit 8 determines that the specified feature point already exists in the neighborhood check area (step S4 in FIG. 4: YES), the newly specified candidate point is already set. It is determined whether or not the feature point exists in the region (step S5 in FIG. 4).

  Specifically, as illustrated in FIG. 3, the feature point selection unit 8 determines that the newly specified candidate points 24 are regions 26 a to 31 a of already set feature points 26 to 31 of the already set feature points 26 to 31. It is determined whether or not it exists in a (rectangular region indicated by a broken line centering on a feature point).

  In the example shown in FIG. 3, the newly specified candidate point 24 does not exist in any of the already set feature point regions 26 a to 31 a (step of FIG. 4 described later). S5: In the case of NO).

  When the feature point selection unit 8 determines that the newly specified candidate point exists in the already set feature point region (step S5 in FIG. 4: YES), the feature point selection unit 8 refers to the feature point table. Then, 1 is assigned to the hit flag of the feature point corresponding to the already set feature point region where the candidate point exists (step S6 in FIG. 4).

  Then, the feature point selection unit 8 determines the magnitude of the score value of the feature point corresponding to the newly specified candidate point and the already set feature point region where the candidate point exists (see FIG. 4 step S7). The score value of the feature point corresponding to the already set feature point region may be the value input from the feature point extraction unit 4 when the feature point is specified as a candidate point. However, it may be newly calculated at this stage. Hereinafter, the score value of the feature point corresponding to the already set feature point region is simply referred to as “score value of the existing feature point”.

  When the feature point selection unit 8 determines that the score value of the existing feature point is large (step S7 in FIG. 4: YES), 1 is assigned to the deletion flag of the newly specified candidate point. (Step S8 in FIG. 4). In this case, in the following process, the newly specified candidate point is not set as a new feature point.

  On the other hand, when the feature point selection unit 8 determines that the score value of the existing feature point is small (step S7: NO in FIG. 4), any value is set as the deletion flag for the newly specified candidate point. Is also not substituted (ie, remains 0). In this case, in the following process, the newly specified candidate point is set as a new feature point.

  Then, the feature point selection unit 8 determines whether or not the above-described processing from step S4 to step S8 has been performed for all feature points existing in the neighborhood check area (step S9 in FIG. 4), and the processing is performed. (Step S9 in FIG. 4: YES), the process proceeds to step S10 in FIG. On the other hand, when the above process is not performed (step S9 in FIG. 4: NO), the process proceeds to step S4 in FIG. 4 to perform the process.

  In step S10 of FIG. 4, the feature point selection unit 8 determines the value of the newly specified candidate point deletion flag.

  If the value of the deletion flag is 0 (step S10: 0 in FIG. 4), the feature point selection unit 8 refers to the feature point table, and the feature point having a hit flag value of 1 is referred to. Is searched (step S11 in FIG. 4).

  When a feature point having a value of 1 in the hit flag exists, the feature point selection unit 8 sets the newly specified candidate point as a new feature point. Specifically, the feature point selection unit 8 newly specifies the feature point values (X coordinate, Y coordinate, and score value) that indicate a value of 1 in the hit flag stored in the feature point table. Rewrite the X coordinate, Y coordinate, and score value indicated by the candidate point (overwrite registration). Then, the feature point selection unit 8 ends the process.

  If there is no feature point having a value of 1 for the hit flag, the newly identified candidate point is set as a new feature point (new registration). In this case, the feature point selection unit 8 registers only feature points of a quantity less than a predetermined quantity, and there are blank places in the feature point column of the feature point table. Therefore, the feature point selection unit 8 stores the X coordinate, the Y coordinate, and the score value indicated by the newly specified candidate point in the blank portion of the feature point table. Then, the feature point selection unit 8 ends the process.

  On the other hand, when the value of the deletion flag is 1 (step S10: 1 in FIG. 4), the newly specified candidate point is not registered as a feature point, and the process is terminated.

  When the feature point selection unit 8 determines that there is no feature point already set in the neighborhood check area (step S4: NO in FIG. 4), the newly identified candidate point is a new one. It will be set as a feature point.

  As a specific process, the feature point selection unit 8 first proceeds to step S10. At this time, the value of the newly specified candidate point deletion flag indicates 0 because no value is substituted. Accordingly, the feature point selection unit 8 proceeds to the process of step S10: 0 in FIG. 4, and the newly specified candidate point is set as a new feature point through the process of step S12 or step S13. The Rukoto.

  In addition, embodiment described above does not limit the invention concerning a claim. And not all combinations of the configurations described in the above embodiments are essential means for solving the problems of the invention.

  As described above, in the present embodiment, the feature point selection unit 8 specifies candidate points that are candidate feature points from the pixel information input from the feature point score value selection unit 6, and is newly specified. If the candidate point is determined to exist in the already set feature point region, the newly identified candidate point and the feature point corresponding to the existing feature point region in which the candidate point exists Since it is configured to determine the magnitude of the score value and determine that the score value of the newly identified candidate point is large, the newly identified candidate point is set as a new feature point. It is possible to extract accurate and available feature points from the obtained images.

  In the above-described operation example, the divided areas 1 to 30 are obtained by dividing the image 21 into divided areas having a predetermined size. However, the present invention is not limited thereto, and the divided areas 1 to 30 may be divided into predetermined divided areas. For example, the image may be divided into a plurality of divided areas having different sizes.

  The neighborhood check area can be arbitrarily set. In the above-described operation example, the neighborhood check area is a size obtained by adding five divided areas from the divided areas set as the reference. However, when the area indicated by the already set feature point areas 26a to 31a is set as a narrow area, the neighborhood check area is set to a narrow area (for example, two divided areas are added from the reference divided area). The area can be set as a neighborhood check area.)

  In this case, the neighborhood check area is set in a narrower range than usual, but the feature point selection unit 8 can set the feature points without overlapping the feature points.

  In addition, when the area indicated by the already set feature point areas 26a to 31a is set as a wide area, the neighborhood check area is set to a wide area (for example, eight divided areas are added from the reference divided area). The area can be set as a neighborhood check area.)

  In this case, although the neighborhood check area is set in a wider range than usual, the feature point selection unit 8 can set the feature points without overlapping the feature points.

  Further, the local area and the neighborhood check area may not be set. In this case, the feature point selection unit 8 determines whether or not the newly specified candidate point exists in all the already set feature point regions in the image. Then, when the feature point selection unit 8 determines that the newly specified candidate point exists in the already set feature point region, the newly specified candidate point and the candidate point are The magnitude of the score value of the feature point corresponding to the existing feature point region is determined. When the feature point selection unit 8 determines that the score value of the newly specified candidate point is large, the feature point selection unit 8 sets the newly specified candidate point as a new feature point.

  In the above embodiment, an example in which the present application is applied to an image display device has been shown. However, for example, for a digital video camera, a digital camera, a personal computer, a home electronic device, etc. Is applicable.

  Furthermore, input device, action recognition, image blur correction device, approach / retreat detection, moving object detection, moving area detection, moving object state measurement, movement state estimation, movement ability evaluation, moving body speed measurement, rigidity evaluation / It is applicable to various uses such as estimation and three-dimensional measurement.

DESCRIPTION OF SYMBOLS 1 Optical system engine 2 Input image acquisition part 3 Frame buffer 4 Feature point extraction part 5 Differential image holding part 6 Feature point score value selection part 7 Feature point threshold value storage part 8 Feature point selection part 9 Feature point information storage part for every image area (region)
10 Feature point motion vector extraction unit S Image processing device

Claims (7)

In order to track the feature points shown in the image and acquire the motion vector of the feature points, a predetermined quantity is obtained from the pixel information constituting the input image and having predetermined position information in the image according to a predetermined order. A feature point extraction device for sequentially setting the feature points of
Candidate point specifying means for specifying candidate points as candidate feature points from the input pixel information;
Score value input means for receiving input of a score value indicating the evaluation value of the candidate point;
A region determination unit that determines whether or not the candidate point newly specified by the candidate point specifying unit exists within a predetermined region from the already specified feature point;
When the newly determined candidate point is determined to exist within a predetermined area from the already specified feature point by the area determining means, the newly specified candidate point and the already specified candidate point Score value determination means for determining the magnitude of the score value input when the feature point is identified as a candidate point;
A feature point setting unit that sets the candidate point as a new feature point in the image when the score value determining unit determines that the score value of the candidate point is large;
A feature point extracting apparatus comprising: The feature point extraction device according to claim 1,
Divided region setting means for setting a divided region obtained by dividing the image into predetermined regions;
Whether or not there is a feature point that has already been specified within the range of the predetermined divided region from the divided region that is the reference, based on the divided region to which the candidate point newly specified by the candidate point specifying unit belongs. A divided region determining means for determining,
The score value determining means is an area in which a feature point that has already been specified within the range of a predetermined divided area from the reference divided area by the divided area determining means, and is newly specified When it is determined that a candidate point exists within a predetermined region from the already specified feature point, the newly specified candidate point and the already specified feature point are specified as candidate points. A feature point extracting device that compares score values inputted when the feature points are input. The feature point extraction device according to claim 1,
When it is determined by the area determination means that the newly specified candidate point does not exist within a predetermined area from the already specified feature point, the feature point determination means Is set as a new feature point in the image. The feature point extraction device according to claim 2,
Based on the divided area to which the newly specified candidate point belongs, the divided area determining means determines that there is no feature point already specified within the range of the predetermined divided area from the reference divided area. If so, the feature point determination means sets the candidate point as a new feature point in the image.   An image processing apparatus for tracking a feature point shown in an image, wherein the feature point extraction apparatus according to claim 1 is mounted, and acquiring a motion vector of the feature point. In order to track the feature points shown in the image and acquire the motion vector of the feature points, a predetermined quantity is obtained from the pixel information constituting the input image and having predetermined position information in the image according to a predetermined order. A feature point extraction method in a feature point extraction apparatus for sequentially setting the feature points,
A feature point candidate specifying step for specifying a candidate point that is a candidate for a feature point from the input pixel information;
A score value input step for receiving an input of a score value indicating an evaluation value of the candidate point;
A region determination step for determining whether or not a candidate point newly specified by the feature point candidate specifying step is present in a predetermined region from the already specified feature point;
When it is determined by the region determination step that a newly identified candidate point exists within a predetermined region from an already identified feature point, the newly identified candidate point and the already identified candidate point A score value determination step for determining the magnitude of the score value input when the feature point that has been identified as a candidate point;
A feature point setting step of setting the candidate point as a new feature point in the image when the score value determining step determines that the score value of the candidate point is large;
A feature point extraction method characterized by comprising: In order to track the feature points shown in the image and acquire the motion vector of the feature points, a predetermined quantity is obtained from the pixel information constituting the input image and having predetermined position information in the image according to a predetermined order. A computer included in the feature point extraction device that sequentially sets the feature points of
Candidate point specifying means for specifying candidate points as candidate feature points from the input pixel information;
Score value input means for receiving input of a score value indicating the evaluation value of the candidate point;
A region determination unit for determining whether or not a candidate point newly specified by the candidate point specifying unit exists in a predetermined region from the already specified feature point;
When the newly determined candidate point is determined to exist within a predetermined area from the already specified feature point by the area determining means, the newly specified candidate point and the already specified candidate point Score value determining means for determining the magnitude of the score value input when the feature point is identified as a candidate point;
A feature point setting unit that sets the candidate point as a new feature point in the image when the score value determining unit determines that the score value of the candidate point is large;
Feature point extraction program characterized by functioning as

Images