JP2012068062A - Alignment device, alignment system and alignment program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an alignment device, an alignment system and an alignment program for aligning a photographic image of an object and design information of the object, based on a shape extracted from the photographic image of the object.SOLUTION: An alignment device 1 comprises: feature point selection accepting means 101 that selects a hole provided in an object 3 from photographic image information 111 photographing the object 3; feature point extract means 102 that extracts a feature point of the hole whose selection is accepted by the feature point selection accepting means 101; corresponding point selection means 103 that selects a corresponding point corresponding to the feature point to be a sample of the object 3 from object three-dimensional information 113; and alignment means 104 that performs coordinate conversion of the corresponding point so as to minimize a distance between a straight line passing through a view point and the feature point of the photographic image and the corresponding point and to align the photographic image information 111 of the object 3 and the object three-dimensional information 113 to be a sample of the object 3.

Description

  The present invention relates to an alignment apparatus, an alignment system, and an alignment program.

  Techniques for extracting portions such as holes and cylindrical end points from the shape of an object have been proposed.

  As a technique related to this, Patent Document 1 discloses a means for extracting point sequence information that can be a circle from edge information obtained from an image obtained by photographing an object, and measures the three-dimensional position of the point sequence. Means, a means for specifying a plane including the point sequence based on the three-dimensional position of the measured point sequence, and performing a circle approximation on the information of the point sequence on the specified plane, from among the approximated circles There is disclosed a three-dimensional position / orientation apparatus having means for selecting one that satisfies a predetermined condition and means for detecting a three-dimensional position / orientation with respect to a selected circle.

JP-A-10-326347

  An object of the present invention is to provide an alignment device, an alignment system, and an alignment program for aligning a captured image of an object and design information of the object based on a shape extracted from the captured image of the object. There is to do.

[1] Accepting means for accepting selection of a hole of the object from a captured image obtained by photographing the object;
Feature point extracting means for extracting feature points of the holes accepted by the accepting means;
Corresponding point selection means for selecting corresponding points corresponding to the feature points from design information indicating the design contents of the object;
An alignment apparatus that includes an alignment unit that aligns the photographed image and the design information by applying an operator that converts the coordinates of the corresponding point to the design information.

[2] The alignment device further includes:
Calculating means for calculating the operator so that the distance between the corresponding point and the straight line passing through the viewpoint and the feature point of the imaging means for imaging the object is minimized;
The alignment apparatus according to claim 1, further comprising:

[3] The calculation unit passes through the viewpoint and the feature point of the hole in accordance with a change rate of the shape of the hole in the captured image with respect to the shape of the hole that the reception unit has received the selection in plan view. The alignment apparatus according to claim 2, wherein a distance between the straight line and the corresponding point is weighted, and an operator that converts the coordinates of the corresponding point is calculated so that the weighted distance is minimized.

[4] The calculation means determines the viewpoint and the hole according to a change rate of the area of the shape of the hole in the photographed image with respect to the area of the shape of the hole when the selection is received by the reception means. 3. The alignment apparatus according to claim 2, wherein a distance between the straight line passing through the feature point and the corresponding point is weighted, and an operator for converting the coordinates of the corresponding point is calculated so that the weighted distance is minimized. .

[5] The calculation means determines the viewpoint of the photographed image according to at least one of a distance between the feature point and the viewpoint, a viewing angle of the feature point from the viewpoint, and an image quality of the feature point in the photographed image. 3. The position according to claim 2, wherein a distance between the straight line passing through the feature point and the corresponding point is weighted, and an operator for converting the coordinates of the corresponding point is calculated so that the weighted distance is minimized. Alignment device.

[6] The alignment apparatus further includes:
Calculating means for calculating an operator for converting the coordinates of the corresponding points so that a distance between a straight line passing through the viewpoint and the feature point of the photographing means for photographing the object and the corresponding point is minimized;
The alignment apparatus according to claim 1, further comprising:

[7] The calculation unit passes through the viewpoint and the feature point of the hole in accordance with a change rate of the shape of the hole in the captured image with respect to the shape of the hole that the reception unit has received the selection in plan view. The alignment apparatus according to claim 6, wherein the distance between the straight line and the straight line passing through the corresponding point is weighted, and an operator that converts the coordinates of the corresponding point is calculated so that the weighted distance is minimized.

[8] The calculation unit may determine the viewpoint and the hole according to a change rate of the area of the shape of the hole in the captured image with respect to the area of the shape in plan view of the hole received by the reception unit. The weight of the distance between a straight line passing through a feature point and the straight line passing through the corresponding point is weighted, and an operator that converts the coordinates of the corresponding point is calculated so that the weighted distance is minimized. Alignment device.

[9] The calculation means may determine the viewpoint of the photographed image according to at least one of a distance between the feature point and the viewpoint, a viewing angle of the feature point from the viewpoint, and an image quality of the feature point in the photographed image. And weighting a distance between a straight line passing through the feature point and a straight line passing through the corresponding point, and calculating an operator for converting the straight line passing through the corresponding point so that the weighted distance is minimized. The alignment device described in 1.

[10] The computer
Accepting means for accepting selection of a hole of the object from a captured image obtained by photographing the object;
Feature point extracting means for extracting feature points of the holes accepted by the accepting means;
Corresponding point selection means for selecting corresponding points corresponding to the feature points from design information indicating the design contents of the object;
An alignment program for applying an operator that converts the coordinates of the corresponding points to the design information, and causing the photographed image and the design information to function as alignment means.

[11] A storage device that stores a photographed image obtained by photographing the object from a predetermined viewpoint and design information serving as a sample of the object;
An accepting unit that accepts selection of a hole of the object from a captured image obtained by photographing the object, a feature point extracting unit that extracts a feature point of the hole accepted by the accepting unit, and the feature from the design information Corresponding point selecting means for selecting a corresponding point corresponding to a point, and alignment means for applying an operator that converts coordinates of the corresponding point to the design information and aligning the photographed image with the design information An alignment device;
An alignment system consisting of

  According to the invention which concerns on Claim 1, 10 or 11, based on the shape extracted from the picked-up image of a target object, the picked-up image of a target object and the design information of the said target object can be aligned.

  According to the invention which concerns on Claim 2, compared with the case where it does not have this structure, the precision of alignment with the picked-up image of a target object and the design information of the said target object can be improved.

  According to the invention of claim 3 or 7, as the rate of change of the shape of a part of the object in the captured image is larger, the weighting for alignment between the captured image of the object and the design information of the object is reduced. be able to.

  According to the invention according to claim 4 or 8, as the rate of change in the area of the shape of a part of the object in the captured image is larger, the weighting of the alignment between the captured image of the object and the design information of the object is increased. Can be small.

  According to the fifth or ninth aspect of the invention, the closer the distance, the smaller the viewing angle, and the better the state of the captured image, the greater the weighting of alignment between the captured image of the object and the design information of the object. can do.

  According to the invention of claim 6, based on the distance between the straight line passing through the viewpoint and the feature point of the captured image and the straight line passing through the corresponding point, the captured image of the target object and the design information of the target object are aligned. can do.

FIG. 1 is a schematic diagram illustrating a configuration example of an alignment system according to the present embodiment. FIG. 2 is a block diagram illustrating a configuration example of the alignment apparatus according to the present embodiment. FIG. 3A is a schematic diagram illustrating an example of the content of the captured image information, and FIG. 3B is a schematic diagram illustrating an example of the content of the captured information. FIG. 4 is a schematic diagram illustrating an example of a display image of the object three-dimensional information. 5A and 5B are schematic diagrams illustrating an example of a display image of photographed image information. FIG. 6 is a schematic diagram illustrating an example of a display image of target three-dimensional information. FIG. 7 is a schematic diagram illustrating a relationship between target 3D information and corresponding points. FIG. 8 is a schematic diagram illustrating an example of the configuration of correspondence information. FIG. 9 is a schematic diagram showing an operation example of the alignment calculation means. FIG. 10 is a flowchart illustrating an operation example of the alignment apparatus according to the present embodiment. FIG. 11 is a schematic diagram illustrating the relationship between the target three-dimensional information and the corresponding straight line. FIG. 12 is a schematic diagram showing another operation example of the alignment calculation means. FIG. 13 is a schematic diagram showing another operation example of the alignment calculation means. FIGS. 14A and 14B are schematic diagrams showing the shape of holes in the captured image information. FIGS. 15A and 15B are schematic diagrams illustrating an example of a display image of photographed image information. FIG. 16 is a schematic diagram illustrating an example of a display image of target three-dimensional information. FIG. 17 is a schematic diagram illustrating a relationship between target 3D information and corresponding points.

(Configuration of alignment system)
FIG. 1 is a schematic diagram illustrating a configuration example of an alignment system according to the present embodiment.

  The alignment system 5 is configured by connecting an alignment apparatus 1 and a camera 2 that captures an object 3 with a USB (Universal Serial Bus) cable or the like.

  The alignment device 1 is an information processing device that includes an electronic component such as a CPU (Central Processing Unit) having a function for processing information and a storage unit inside the main body, and processes image information captured by the camera 2. A display unit 12 such as a liquid crystal display for displaying an image; an operation unit 13 such as a keyboard or a mouse for outputting an operation signal corresponding to the operation content; The alignment apparatus 1 is, for example, a personal computer, and a PDA (Personal Digital Assistant), a mobile phone, or the like can also be used.

  The camera 2 shoots the object 3 from a plurality of shooting directions in addition to the shooting direction 20 a, records the shot shooting direction 20 a together with the shot image information, and transmits these pieces of information to the alignment apparatus 1.

  The object 3 is composed of a single surface or a plurality of surfaces, and has holes 31 to 35 through which screws or the like for fixing other components are passed through each surface. In addition, you may have the hole which does not penetrate the surface of the target object 3. FIG.

(Configuration of alignment device)
FIG. 2 is a block diagram illustrating a configuration example of the alignment apparatus 1 according to the present embodiment.

  The alignment apparatus 1 includes a CPU and the like. The control unit 10 controls each unit and executes various programs. A storage unit 11 is a storage medium such as an HDD (Hard Disk Drive) or a flash memory and stores information. A display unit 12 for displaying characters and images, such as a liquid crystal display, an operation unit 13 for outputting an operation signal corresponding to the operation content, such as a keyboard and a mouse, and the camera 2 for transmitting and receiving information. And a communication unit 14.

  The alignment device 1 is, for example, an electronic device such as a personal computer, a PDA, or a mobile phone. However, the alignment device 1 may be a server device that does not include the display unit 12 or the operation unit 13, and in this case, the network device or the like. The operation unit and display unit of the connected terminal device replace these functions.

  The control unit 10 executes a registration program 110, which will be described later, so that the captured image information reception unit 100, the feature point extraction unit 102, the feature point selection reception unit 101, the corresponding point selection unit 103, the registration unit 104, and the accuracy calculation. It functions as the means 105 or the like.

  The photographed image information accepting unit 100 accepts the photographed image information 111 photographed by the camera 2 via the communication unit 14 and the photographed information 112 including the contents relating to the photographing method, and stores them in the storage unit 11.

  The feature point selection accepting unit 101 uses coordinate points (hereinafter referred to as “feature points”) to be used for alignment according to the operation content of the user on the operation unit 13 from the photographed image information 111 accepted by the photographed image information accepting unit 100. .) Is extracted, the selection of the area corresponding to the hole in the display image of the captured image information 111 is accepted.

  The feature point extracting unit 102 extracts the center of gravity of the hole as a feature point from the region in the display image of the captured image information 111 received by the feature point selection receiving unit 101. For example, the center point of the screw hole is extracted as the feature point. Note that the shape of the opening provided on the surface is not limited to a perfect circle, and may be any shape that can uniquely determine the center regardless of the viewpoint. The shape of the opening is not limited to a closed curve.

  Corresponding point selection means 103 is a feature point extraction means 102 from object three-dimensional information 113, which is design information written in three dimensions to manufacture the object 3, according to the user's operation on the operation unit 13. A corresponding point corresponding to the extracted feature point is selected.

  The alignment unit 104 calculates and obtains a transformation matrix for the corresponding point that minimizes the difference in distance between the feature point and the corresponding point, and the captured image information 111 and the object three-dimensional information 113 are obtained by the transformation matrix. Align. A transformation matrix is a matrix that translates, rotates, and scales a three-dimensional position. Note that the alignment is not limited to a matrix, and any operator that translates, rotates, and enlarges or reduces a three-dimensional position may be used.

  The accuracy calculation unit 105 calculates the error in the shapes of the aligned captured image information 111 and the object three-dimensional information 113 as accuracy.

  The storage unit 11 includes an alignment program 110 that causes the control unit 10 to operate as each of the above-described units 100 to 105, captured image information 111 that is image information of the object 3 captured by the camera 2, and a camera when the image is captured Shooting information 112 including information such as the posture of the object 2, object three-dimensional information 113 which is design information described in three dimensions for manufacturing the object 3, and information on the correspondence between the feature points and the corresponding points. Correspondence information 114 and the like are stored.

  FIG. 3A is a schematic diagram illustrating an example of the content of the captured image information, and FIG. 3B is a schematic diagram illustrating an example of the content of the captured information.

  The photographed image information 111a shown in FIG. 3A is photographed image information obtained by photographing the object 3 from the photographing direction 20a by the camera 2 as shown in the photograph information 112 shown in FIG.

  The photographed image information 111b shown in FIG. 3B is photographed image information obtained by photographing the object 3 from the photographing direction 20b by the camera 2 as shown in the photograph information 112 shown in FIG.

  The shooting information 112 shown in FIG. 3C includes a shooting image information column in which an identifier of shooting image information generated by the camera 2 is recorded, a shooting direction column in which the shooting direction of the shooting image information is recorded, and the like.

  FIG. 4 is a schematic diagram illustrating an example of a display image of the target object three-dimensional information 113.

  The object three-dimensional display image 113 </ b> A is an image displayed on the display screen 120 by reproducing the object three-dimensional information 113 by the corresponding point selection unit 103. The target object three-dimensional display image 113 </ b> A may be an image whose display content is enlarged / reduced or rotated according to the user's operation content on the operation unit 13.

(Operation of alignment device)
Hereinafter, with reference to FIGS. 1 to 8, an example of the operation of the alignment apparatus 1 will be described in (1) basic operation, (2) feature point extraction operation, (3) corresponding point selection operation, and (4) alignment operation. Separately described.

(1) Basic operation First, the user photographs the object 3 from a plurality of photographing directions with the camera 2. The captured image information 111 and the associated shooting information 112 are stored in the storage unit of the camera 2.

  The user operates the operation unit 13 of the alignment apparatus 1 to instruct communication with the camera 2. The operation unit 13 outputs an operation signal according to the operation.

  When receiving the operation signal from the operation unit 13, the alignment apparatus 1 starts communication with the camera 2 via the communication unit 14. Next, the photographed image information accepting unit 100 accepts the photographed image information 111 and the photographed information 112 from the storage unit of the camera 2 and stores them in the storage unit 11. Note that the photographed image information accepting unit 100 may accept photographed image information 111 and photographed information 112 previously photographed from an external device other than the camera 2.

  Next, when the user requests alignment of the captured image information 111 and the object three-dimensional information 113 stored in the storage unit via the operation unit 13, the feature point selection receiving unit 101 of the alignment apparatus 1 First, the captured image information 111 is reproduced and a display image is displayed on the display unit 12.

  5A and 5B are schematic diagrams illustrating an example of a display image of photographed image information.

  As shown in FIG. 5A, the feature point selection accepting unit 101 includes a cursor 101a that operates in accordance with captured image information 111a captured from the capturing direction 20a on the display screen 120 and an operation signal output from the operation unit 13. Is displayed.

(2) Feature Point Selection Operation The user refers to the display screen 120 shown in FIG. 5A and drags the cursor 101a via the operation unit 13, and uses the selection frame 101b displayed by the drag operation. Select the hole (screw hole) to be used.

  FIG. 10 is a flowchart showing an operation example of the alignment apparatus 1 according to the embodiment of the present invention.

  The feature point selection receiving unit 101 receives the selection by the selection frame 101b shown in FIG. 5A selected by the above operation, and instructs the feature point extraction unit 102 to extract the feature points in the selection frame 101b.

  The feature point extraction unit 102 performs image analysis processing on the captured image information 111a in the selection frame 101b, and extracts the center of the image area corresponding to the screw hole as the feature point 31a (S1). The feature point is not limited to the center of the screw hole (hole) but may be the center of gravity of the shape of the screw hole (hole).

  The feature point selection accepting unit 101 writes the feature point 31 a extracted by the feature point extracting unit 102 in the correspondence information 114 of the storage unit 11.

(3) Corresponding point selection operation

  Next, the corresponding point selection unit 103 reproduces the object three-dimensional information 113 and displays the object three-dimensional display image 113 </ b> A on the display unit 12.

  FIG. 6 is a schematic diagram illustrating an example of the object three-dimensional display image 113A.

  As shown in FIG. 6, the corresponding point selection unit 103 displays a target three-dimensional display image 113 </ b> A and a cursor 103 a that operates according to an operation signal output from the operation unit 13 on the display screen 120.

  The user operates the operation unit 13 with reference to the display screen 120 shown in FIG. 6 so that the corresponding point corresponding to the feature point 31a selected in step S1 is selected by the cursor 103a.

  When the corresponding point selection unit 103 receives an operation signal corresponding to the operation content from the operation unit 13, the corresponding point selection unit 103 selects the corresponding point 41a defined in the object three-dimensional information 113 (S2). It is assumed that the corresponding point 41a is defined as shown below.

  FIG. 7 is a schematic diagram illustrating a relationship between target 3D information and corresponding points.

  Since the hole 41 is originally defined by a cylindrical shape in the object three-dimensional information 113, the corresponding point selection unit 103 defines the centers of the upper and lower surfaces of the cylinder as corresponding points 41a and 41b. When the corresponding point 41a or 41b is selected by the cursor 103a in FIG. 6, the corresponding point 41a is selected from the viewpoint on the display, and in the example of FIG.

  Next, correspondence information 114 is created from the feature points written in step S1 and the corresponding points written in S2.

  FIG. 8 is a schematic diagram illustrating an example of the configuration of the correspondence information 114.

  The correspondence information 114 includes a feature point ID column indicating an identifier for identifying a feature point, a shooting direction column indicating a shooting direction of the shooting image information 111 from which the feature point is selected, and a feature indicating the selected feature point (coordinates). It has a point field, a feature line field that indicates a feature line that passes through the viewpoint and feature points that define the shooting direction, and a corresponding point field that indicates corresponding points corresponding to the feature points.

  The shooting direction column and the feature point column are input by the feature point extracting unit 102 as described above, and the corresponding point column is input by the corresponding point selecting unit 103. The feature line in the feature line column is a straight line that passes through the viewpoint (viewpoint a or b) in the shooting direction in the shooting direction column and the feature point in the feature point column, and is calculated by the alignment unit 104 described below. The

  Next, the alignment means 104 confirms whether or not a sufficient number of pieces of information of the correspondence information 114 have been input for alignment, and if it is sufficient (S3; Yes), for example, in the present embodiment, it is a feature. When the combination of the point and the corresponding point is 3 or more, (4) the alignment operation is executed. Moreover, when the information of the correspondence information 114 is not sufficient (S3; No), steps S1 and S2 are repeated.

(4) Positioning Operation FIG. 9 is a schematic diagram showing an operation example of the positioning means 104.

The alignment means 104 obtains a transformation matrix T that minimizes the value of the following equation.

Here, p _i is the coordinates of the corresponding point, l _i passing through the viewpoint a or b and the feature point is a feature line, and f (x, y) is the distance between the point x and the line y.

  That is, in the correspondence information 114, the characteristic straight lines 51a, 52a, 55a, 52b, 55b... Shown in FIG. 9 and the corresponding corresponding points 41a, 42a, 45a, 42a, 45a. T is calculated such that the sum of the distances is minimized.

  Next, after calculating the transformation matrix T, the accuracy calculating means 105 compares the object 3 with the object 3D information 113 rotated by the transformation matrix T, and calculates the working accuracy of the shape of the object 3 and the like. To do. Further, the accuracy calculation unit 105 may calculate the positional accuracy of the holes 31 to 35 of the object 3 with respect to the object three-dimensional information 113 from the value of the above equation [1].

[Modification 1]
(4-1) Positioning Operation The positioning unit 104 calculates the equation [1] for all corresponding points in consideration of the user's selection mistake of the corresponding point 41a or 41b as shown in FIG. Also good.

That is, the transformation matrix T that minimizes the color value shown below is obtained.

Note that p _ij is the coordinates of the corresponding points. When j = 1, 2, they correspond to the corresponding points 41a and 41b shown in FIG.

[Modification 2]
FIG. 11 is a schematic diagram illustrating the relationship between the target three-dimensional information and the corresponding straight line.

  The corresponding point selection means 103 defines the hole 41 by the corresponding points 41a and 41b which are the centers of the upper surface and the lower surface of the cylinder forming the hole 41 from the object three-dimensional information 113 shown in FIG. 7 in the above embodiment. As a modification, it may be defined by a corresponding straight line 61 passing through two points, corresponding points 41a and 41b, so that selection mistakes of corresponding points 41a or 41b are eliminated.

  That is, in step S <b> 2 described above, when the corresponding point selection unit 103 receives the operation signal from the operation unit 13, the corresponding point selection unit 103 selects the corresponding straight line 61 defined in the object three-dimensional information 113.

(4-2) Positioning Operation The positioning means 104 performs the calculation described below using the selected corresponding straight line.

  FIG. 12 is a schematic diagram illustrating another example of the operation of the alignment unit 104.

The alignment means 104 obtains a transformation matrix T that minimizes the value of the following equation.

Note that h _i passing through two corresponding points defined in the hole is a corresponding line, l _i is a characteristic line, and g (x, y) is a distance between the line x and the line y.

  That is, among the correspondence information 114, the characteristic straight lines 51a, 52a, 55a, 52b, 55b... Shown in FIG. 12 and the corresponding straight lines 61, 62, 65, 62, 65. So that the sum of the distances is minimized.

  In the alignment calculation using Expression [3], whether or not the number of corresponding lines in step S3 is sufficient may be two or more when all the corresponding lines are not parallel. Further, when the corresponding straight lines are parallel, the number may be three or more. However, when the corresponding straight lines are parallel, there is a degree of freedom in the corresponding straight line direction as shown below, and the alignment cannot be performed.

(4-3) Positioning Operation FIG. 13 is a schematic diagram illustrating another example of the positioning unit 104.

  When the corresponding straight lines 66 to 68 are parallel to each other, the alignment means 104 is relatively relative to the alignment of the captured image information 111 and the object three-dimensional information 113 even if the value of the expression [3] is minimum. The degree of freedom is generated in the direction of the corresponding straight lines 66-68.

  Therefore, when there is another hole having a corresponding straight line that is not parallel to the corresponding straight lines 66 to 68, the positioning means 104 displays a message or the like on the display unit 12 to select another hole. To request.

  Further, in the object three-dimensional display image 113A, the object can be translated only in the directions of the corresponding straight lines 66 to 68 having a degree of freedom in accordance with the user's operation, and the user is required to align the position visually. May be.

Further, in addition to the corresponding straight line, the user may be further requested to specify a pair of feature points and corresponding points, and the variable a that minimizes the value of the following expression may be obtained.

  Here, a is a variable, and h is a vector representing the direction of parallel corresponding straight lines.

[Modification 3]
(4-4) Positioning Operation FIGS. 14A and 14B are schematic diagrams showing the shape of holes in the captured image information 111. FIG.

  The holes 31A and 31B are obtained by photographing the hole 31 of the object 3 from different photographing directions.

  The feature point extraction means 102 extracts feature points from the holes 31A and 31B. The closer to a perfect circle as in the hole 31A, the higher the feature point extraction accuracy, and the longer and shorter axes of the ellipse as in the hole 31B. The greater the difference, the lower the accuracy. In addition, as the distance from the viewpoint of the captured image information 111 to the holes 31A and 31B increases, the feature point extraction accuracy decreases. Furthermore, the extraction accuracy also decreases when the images are not in focus in the holes 31A and 31B and are unclear.

Therefore, the alignment means 104, the shape of the holes in the captured image information 111, the distance from the viewpoint, the coefficient w _i corresponding to the sharpness of the image is weighted by multiplying the equation [1], the minimum of the following formula A transformation matrix T is obtained.

[Modification 4]
In step S3 in FIG. 10, when the number of corresponding points is not sufficient (S3; No), steps S1 and S2 are repeated. However, when the number of holes in the object 3 is not sufficient or the number of holes is sufficient (S3; Yes) However, if the position of the hole is symmetric, a uniquely determined alignment calculation cannot be performed.

  Therefore, in addition to the feature points extracted from the holes and the corresponding points corresponding thereto, the alignment unit 104 selects the feature points extracted from the shape of the object and the corresponding points corresponding to the feature points to the user. Request.

(2-5) Feature Point Extraction Operation FIGS. 15A and 15B are schematic diagrams illustrating an example of a display image of photographed image information.

  The user refers to the display screen 120 shown in FIG. 15A, drags the cursor 101a via the operation unit 13, and selects a feature point to be used by the selection frame 101b displayed by the drag operation.

  When the feature point selection accepting unit 101 receives an operation signal corresponding to the above operation from the operation unit 13, the feature point selection accepting unit 101 instructs the feature point extracting unit 102 to extract a feature point within the selection frame 101b shown in FIG. .

  The feature point extraction unit 102 performs image analysis processing on the captured image information 111 c in the selection frame 101 b and extracts, for example, a corner shape or the like as the feature point 71.

(3-5) Corresponding point selection operation

  Next, the corresponding point selection unit 103 displays the target object three-dimensional display image on the display unit 12.

  FIG. 16 is a schematic diagram illustrating an example of an object three-dimensional display image.

  As illustrated in FIG. 16, the corresponding point selection unit 103 displays a target three-dimensional display image 113 </ b> B and a cursor 103 a that operates according to an operation signal output from the operation unit 13 on the display screen 120.

  The user operates the operation unit 13 with reference to the display screen 120 shown in FIG. 16 so that the corresponding point corresponding to the selected feature point 71 is selected by the cursor 103a.

When the corresponding point selection unit 103 receives an operation signal corresponding to the operation content from the operation unit 13, the corresponding point selection unit 103 selects the corresponding point 81 defined in the object three-dimensional information 113.
(4-5) Alignment Operation The alignment unit 104 uses the feature points 71 and the corresponding points 81 selected in the above (2-5) feature point selection operation and (3-5) corresponding point selection operation, as follows. A transformation matrix T that minimizes the value of the expression shown is obtained.

Here, q _k is the coordinates of the corresponding point 81 at the corner of the object 3, m _k is a feature line 71 passing through the viewpoint with the feature point 71 at the corner of the object 3, and w ₁ and w ₂ are weighting coefficients. It should be noted that it is desirable to increase w ₁ because the extraction accuracy is higher in the hole shape than in the square shape.

[Other embodiments]
The present invention is not limited to the above embodiment, and various modifications can be made without departing from the gist of the present invention. For example, as shown in FIG. 17, the feature point extracting means 102 may extract the outermost edge from the elliptical hole 39 and extract the center of gravity as the feature point 39a, or the center or center of gravity of the protrusion shape. May be extracted as feature points. In addition, when the hole of the object 3 is provided at an incorrect position, the alignment calculation means calculates the hole provided at the incorrect position using a method such as Thompson's rejection test. [6] may be excluded from p _i , l _i and h _i .

  Further, the alignment program 110 can be provided by being stored in a storage medium such as a CD-ROM, and can be downloaded from a server device connected to a network such as the Internet to a storage unit in the device. Good. The photographed image information receiving unit 100, the feature point extracting unit 102, the feature point selection receiving unit 101, the corresponding point selecting unit 103, the alignment unit 104, and the accuracy calculating unit 105 are partially or entirely realized by hardware such as an ASIC. May be. Note that each step shown in the operation description of the above embodiment can be changed in order, omitted or added.

DESCRIPTION OF SYMBOLS 1 ... Position alignment apparatus, 2 ... Camera, 3 ... Object, 5 ... Position alignment system, 10 ... Control part, 11 ... Memory | storage part, 12 ... Display part, 13 ... Operation part, 14 ... Communication part, 20a, 20b ... Shooting direction, 100 ... photographed image information receiving means, 101 ... feature point selection receiving means, 102 ... feature point extracting means, 103 ... corresponding point selecting means, 104 ... calculating means, 105 ... accuracy calculating means, 110 ... positioning program, 111: Shooting image information, 112: Shooting information, 113: Object three-dimensional information, 114: Corresponding information, 120 ... Display screen

Claims (11)

Accepting means for accepting selection of a hole of the object from a captured image obtained by photographing the object;
Feature point extracting means for extracting feature points of the holes accepted by the accepting means;
Corresponding point selection means for selecting corresponding points corresponding to the feature points from design information indicating the design contents of the object;
An alignment apparatus that includes an alignment unit that aligns the photographed image and the design information by applying an operator that converts the coordinates of the corresponding point to the design information. The alignment device further includes:
2. A calculation unit that calculates the operator so that a distance between a viewpoint of an imaging unit that images the object and a straight line passing through the feature point and the corresponding point is minimized. The alignment device described in 1.   The calculation means is a straight line that passes through the viewpoint and the feature point of the hole according to a change rate of the shape of the hole in the captured image with respect to the shape of the hole that the reception means has received a selection in plan view. The alignment apparatus according to claim 2, wherein the distance between the corresponding points is weighted, and an operator that converts the coordinates of the corresponding points is calculated so that the weighted distance is minimized.   The calculation means calculates the viewpoint and the feature point of the hole in accordance with a change rate of the area of the shape of the hole in the captured image with respect to the area of the shape of the hole that the reception means has received a selection in plan view. The alignment apparatus according to claim 2, wherein the distance between the passing line and the corresponding point is weighted, and an operator that converts the coordinates of the corresponding point is calculated so that the weighted distance is minimized.   The calculating means determines the viewpoint and the feature of the photographed image according to at least one of a distance between the feature point and the viewpoint, a viewing angle of the feature point from the viewpoint, and an image quality of the feature point in the photographed image. The alignment apparatus according to claim 2, wherein a distance between a straight line passing through a point and the corresponding point is weighted, and an operator that converts coordinates of the corresponding point is calculated so that the weighted distance is minimized. The alignment device further includes:
Calculating means for calculating an operator for converting the coordinates of the corresponding points so that a distance between a straight line passing through the viewpoint and the feature point of the photographing means for photographing the object and the corresponding point is minimized; The alignment apparatus according to claim 1, further comprising:   The calculation means is a straight line that passes through the viewpoint and the feature point of the hole according to a change rate of the shape of the hole in the captured image with respect to the shape of the hole that the reception means has received a selection in plan view. The alignment apparatus according to claim 6, wherein a distance to the straight line passing through the corresponding point is weighted, and an operator for converting the coordinates of the corresponding point is calculated so that the weighted distance is minimized.   The calculation means calculates the viewpoint and the feature point of the hole in accordance with a change rate of the area of the shape of the hole in the captured image with respect to the area of the shape of the hole that the reception means has received a selection in plan view. The alignment apparatus according to claim 6, wherein the distance between a straight line passing through and a straight line passing through the corresponding point is weighted, and an operator that converts the coordinates of the corresponding point is calculated so that the weighted distance is minimized. .   The calculating means determines the viewpoint and the feature of the photographed image according to at least one of a distance between the feature point and the viewpoint, a viewing angle of the feature point from the viewpoint, and an image quality of the feature point in the photographed image. The distance between a straight line passing through a point and a straight line passing through the corresponding point is weighted, and an operator for converting the straight line passing through the corresponding point is calculated so that the weighted distance is minimized. Alignment device. Computer
Accepting means for accepting selection of a hole of the object from a captured image obtained by photographing the object;
Feature point extracting means for extracting feature points of the holes accepted by the accepting means;
Corresponding point selection means for selecting corresponding points corresponding to the feature points from design information indicating the design contents of the object;
An alignment program for applying an operator that converts the coordinates of the corresponding points to the design information, and causing the photographed image and the design information to function as alignment means. A storage device for storing a photographed image obtained by photographing the object from a predetermined viewpoint and design information as a sample of the object;
An accepting unit that accepts selection of a hole of the object from a captured image obtained by photographing the object, a feature point extracting unit that extracts a feature point of the hole accepted by the accepting unit, and the feature from the design information Corresponding point selecting means for selecting a corresponding point corresponding to a point, and alignment means for applying an operator that converts coordinates of the corresponding point to the design information and aligning the photographed image with the design information An alignment system comprising an alignment device.

Images