JP2015114875A - Image processor, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor for providing an algorithm for estimating a graphic element with a simpler setting method for extracting the performance, and to provide an image processing method and an image processing program.SOLUTION: The image processor includes selection means for receiving the selection of a graphic element from a user to a setting image, setting value acquisition means for acquiring and storing a feature amount of the selected graphic element, and estimation means for specifying a graphic element of an estimation object included in a measuring image on the basis of the stored feature amount to output a parameter showing the specified graphic element.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for estimating a graphic element included in an input image.

  Conventionally, in the FA (Factory Automation) field, various image processing devices that perform processing such as defect inspection of workpieces to be measured, measurement of workpiece size and shape, and recognition of characters and figures on workpieces have been put into practical use. Has been. As an example, there are applications that perform positioning by measuring the position of positioning marks, etc., printed on the workpiece in advance by pattern matching, or measuring the position of the workpiece outline or edge in the workpiece. Are known.

  In order to apply the image processing result to the positioning application, it is necessary to make some settings in advance. Regarding such setting, in order to generate a registration pattern in a method of positioning by pattern matching of positioning marks, Japanese Patent Laid-Open No. 2006-252400 (Patent Document 1) discloses a two-stage fitting method. Since this algorithm performs the second fitting using all the points existing in the region with a certain width from the first fitting process, the result is not much different from the first fitting. One fitting is dominant. This remains a problem that has been pointed out as a problem in the generally known robust least squares method.

  In general, in such image processing, noise is inevitably mixed due to various disturbances. As algorithms for removing such noise and improving measurement accuracy, Japanese Patent Application Laid-Open No. 2007-249555 (Patent Document 2) and Japanese Patent Application Laid-Open No. 2009-186338 (Patent Document 3) are known.

  Japanese Patent Laying-Open No. 2007-249555 (Patent Document 2) discloses a technique for efficiently removing an outlier from coordinates included in a coordinate group to realize an accurate straight line. Further, JP 2009-186338 A (Patent Document 3) sets an appropriate reference line for a plurality of edge points even for an object having a complicated shape, and sets each edge point from the reference line. Disclosed is a technology that can accurately detect defects such as burrs and chips on the basis of the difference. That is, these prior art documents disclose a method for determining a linear element indicating the outer shape of a measurement object from an edge position or the like.

JP 2006-252400 A JP 2007-249555 A JP 2009-186338 A

  In general, there are a wide variety of applications that use graphic element estimation results, and it is not easy to set the estimation accuracy to be highest for each application. In particular, when a plurality of graphic elements are included in the input image to be processed, it is easy to select a graphic element intended in advance and automatically perform image processing according to the selected graphic element. It wasn't.

  Therefore, there is a demand for an image processing apparatus, an image processing method, and an image processing program that provide a simpler setting method for extracting the performance of an algorithm for estimating graphic elements.

  An image processing apparatus according to an aspect of the present invention includes: a selection unit that accepts selection of a graphic element from a user for a setting image; a setting value acquisition unit that acquires and stores a feature amount of the selected graphic element; Estimation means for specifying a graphic element to be estimated included in the measurement image based on the feature quantity and outputting a parameter indicating the specified graphic element.

  An image processing method according to another aspect of the present invention includes a step of accepting a selection of a graphic element from a user for a setting image, a step of acquiring and storing a feature amount of the selected graphic element, and a stored feature amount. And a step of specifying an estimation target graphic element included in the measurement image and outputting a parameter indicating the specified graphic element.

  An image processing program according to another aspect of the present invention includes: a step of accepting a selection of a graphic element from a user for a setting image; a step of acquiring and storing a feature amount of the selected graphic element; A step of specifying a graphic element to be estimated included in the measurement image based on the feature amount and outputting a parameter indicating the specified graphic element is executed.

  ADVANTAGE OF THE INVENTION According to this invention, the simpler setting method for extracting the performance can be provided with respect to the algorithm which estimates a graphic element.

1 is a schematic diagram illustrating an overall configuration of an image processing system including an image processing apparatus according to an embodiment. It is a figure which shows an example of an image processing result. It is a figure which shows an example of the edge point extracted from the input image. It is a figure which shows an example of the linear element determined from the edge point shown in FIG. It is a schematic diagram which shows the basic process sequence of the image processing apparatus of this Embodiment. It is a figure for demonstrating the decomposition | disassembly process to the graphic element of the image processing apparatus of this Embodiment. It is a figure which shows an example of the figure parameter of each figure element produced | generated by the decomposition | disassembly process shown in FIG. It is a figure which shows an example of the display screen for selecting the graphic element provided by the image processing apparatus of this Embodiment. It is a figure which shows the process example in case a user selects a linear element in the display screen shown in FIG. It is a figure which shows the process example in case a user selects a circle element in the display screen shown in FIG. It is a figure for demonstrating the usage example of the graphic parameter provided by the image processing apparatus of this Embodiment. It is a schematic diagram which shows the function structure for implement | achieving a measurement process in the image processing apparatus of this Embodiment. It is a flowchart for demonstrating the process sequence in the image processing apparatus of this Embodiment.

  Embodiments of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals and description thereof will not be repeated.

<A. Configuration of image processing system>
FIG. 1 is a schematic diagram illustrating an overall configuration of an image processing system 1 including an image processing apparatus 100 according to the present embodiment. The image processing apparatus 100 images the measurement object 2 (hereinafter also referred to as “work 2”) having a mark on the surface, and positions the work 2 based on the image processing result for the image obtained by the imaging. .

  Referring to FIG. 1, the image processing system 1 can communicate with an image processing apparatus 100, which is also called a visual sensor, an imaging unit 8 connected to the image processing apparatus 100, and the image processing apparatus 100 as main components. A PLC (Programmable Logic Controller) 5 and a movable stage 6 are included. FIG. 1 shows an image processing apparatus 100 configured integrally with the display unit 102 as an example.

  The image processing apparatus 100 is incorporated in a production line or the like, and specifies the position of the workpiece 2 by performing image processing as described later on a captured image generated by the imaging unit 8 imaging the workpiece 2. . In the following description, an image generated by imaging by the imaging unit 8 is also referred to as an “input image”. As will be described later, in the present embodiment, processing for determining image processing settings in advance (hereinafter also referred to as “setting processing”), and processing for actually imaging a measurement object and executing image processing (hereinafter referred to as “processing processing”). (Also referred to as “measurement process”). An image used for the setting process is referred to as a “setting image”, and an image used for the measurement process is referred to as a “measurement image”.

  Here, the “measurement image” typically means an image generated by imaging the workpiece 2 to be subjected to image processing. That is, the “measurement image” is an input image having the work 2 to be processed as a subject. “Setting image” means an image that includes the same graphic element as the graphic element estimated to be included in the “measurement image”, and typically captures a reference workpiece as a reference in image processing, etc. This means an image that is generated. That is, the “setting image” is an input image having a reference work as a subject. It should be noted that the image does not have to include the same graphic element as the estimated graphic element, and may be an image including substantially the same graphic element.

  The input image that can be processed by the image processing apparatus 100 is not limited to an image generated by imaging by the imaging unit 8, and may be an image acquired in advance by some imaging apparatus, for example. In particular, the setting image may be an image synthesized from a design drawing of the work 2 or the like.

  The image processing apparatus 100 outputs the specified position information of the workpiece 2 to the PLC 5. The PLC 5 outputs a command to the movable stage 6 based on the position information. As described above, the image processing apparatus 100 may include a unit that outputs an instruction to the drive device that changes the position of the work 2 that is the subject based on the estimated parameter indicating the target graphic element.

  The imaging unit 8 is means for generating an input image by imaging a subject. As an example, in addition to an optical system such as a lens, a plurality of devices such as a CCD (Coupled Charged Device) and a CMOS (Complementary Metal Oxide Semiconductor) sensor are used. The image sensor is divided into pixels. An input image generated by imaging by the imaging unit 8 is transmitted to the image processing apparatus 100. You may further provide the illuminating device which irradiates light with respect to the workpiece | work 2 imaged by the imaging part 8. FIG. Further, the image processing apparatus 100 may be configured so that a plurality of imaging units 8 can be connected.

  When a computer having a structure according to a general-purpose computer architecture is used as the image processing apparatus 100, an OS for providing basic functions of the computer in addition to an application for providing the functions of the present embodiment. (Operating System) may be installed. In this case, the control program according to the present embodiment may execute processing by calling necessary modules out of program modules provided as a part of the OS in a predetermined order and / or timing. Good. That is, the program itself according to the present embodiment does not include the module as described above, and the process may be executed in cooperation with the OS. Accordingly, the control program according to the present embodiment may not include such a part of modules.

  The control program of the present embodiment may be provided by being incorporated in a part of another program. Even in that case, the program itself does not include the modules included in the other programs to be combined as described above, and the processing is executed in cooperation with the other programs. That is, the control program of the present embodiment may be in a form incorporated in such another program.

  Alternatively, part or all of the functions provided by executing the control program may be implemented as a dedicated hardware circuit.

<B. Issues and Background Technology>
As a premise for explaining the processing and functions in the image processing apparatus, the image processing method, and the image processing program of the present embodiment, a problem to be addressed in the image processing and a background art will be described.

  FIG. 2 is a diagram illustrating an example of the image processing result. The image processing result shown in FIG. 2 is a diagram showing positioning marks added in advance on the workpiece. This positioning mark includes a linear element parallel to the vertical direction and the horizontal direction of the paper, and another linear element connecting these linear elements. That is, the input image generated by imaging the workpiece by the imaging unit 8 mainly includes three linear elements.

  As a typical application, an intersection 25 of approximate lines 22 and 24 of two linear elements parallel to the longitudinal direction and the lateral direction of the positioning mark is determined, and the coordinate value of the intersection 25 is used. Positioning control is performed. As a specific processing procedure, an edge search area 21 which is a measurement target area is set for determining an approximate line 22 for an input image, and an edge search area 23 for determining an approximate line 24 is set. Is set. The set positions and sizes of the edge search areas 21 and 23 are registered in advance. Edge points are extracted in each of the edge search areas 21 and 23. Then, an approximate straight line indicating each straight line element is determined from the edge point group extracted in each of the edge search areas 21 and 23. The “+” mark in FIG. 2 indicates the intersection 25 of the approximate straight line based on the extracted edge point.

  In the image processing shown in FIG. 2, it is understood that the edge search areas 21 and 23 include a linear element 26 that is not the estimation target in addition to the linear element intended as the estimation target. That is, the edge points extracted for the straight line element 26 are used when determining the approximate straight lines 22 and 24. As described above, when a graphic element that is not the estimation target is mixed in the measurement target area in addition to the graphic element intended as the estimation target, the estimation accuracy may be reduced. That is, if a plurality of graphic elements are mixed in the measurement target region, the accuracy is easily lowered. As described above, since there are a plurality of graphic elements in a single edge search region, the positions of the approximate line and the intersection cannot be determined correctly, and the positional deviation may not be corrected.

  FIG. 3 is a diagram illustrating an example of edge points extracted from the input image. FIG. 4 is a diagram illustrating an example of a linear element determined from the edge points illustrated in FIG.

  FIG. 3 shows edge point extraction results for an input image that mainly includes three graphic elements. That is, as shown in FIG. 4, the input image includes graphic elements 1, 2, and 3. For example, let us consider a case in which the graphic element 1 is an estimation target among these graphic elements. In this case, it is necessary to determine the approximate straight line 1 shown in FIG. However, if all the edge points extracted from each of the graphic elements 1, 2, and 3 are used, the approximate straight line 2 is determined. Therefore, it is necessary to determine an approximate straight line limited to the edge points for the graphic element to be estimated among the edge point extraction results as shown in FIGS. In other words, if the components of a plurality of graphic elements cannot be separated, it must be an average linear approximation for all extracted edge points, which is completely different from the line that is originally supposed to be estimated. There is a problem that a straight line is output.

  There are a wide variety of applications that use graphic element estimation results as shown in FIGS. 3 and 4, and it is not easy to set each application to have the highest estimation accuracy. In particular, when a plurality of graphic elements are included in the input image to be processed, it is easy to select a graphic element intended in advance and automatically perform image processing according to the selected graphic element. There is a problem that is not.

<C. Overview>
In response to the above-described problems, in the image processing method of the present embodiment, the user selects a graphic element to be subjected to graphic fitting processing, and extracts and processes the selected graphic element. To automatically learn the appropriate settings.

  FIG. 5 is a schematic diagram illustrating a basic processing procedure of the image processing apparatus 100 according to the present embodiment. With reference to FIG. 5, in the image processing apparatus 100, a setting process 200 for determining various parameters and a measurement process 300 for actually executing image processing on an input image are executed.

  First, the contents of the setting process 200 will be described. The setting process 200 includes a pre-process 210, a tuning process 220, and a setting storage process 230.

  In the preprocessing 210, the setting image 20 is received as an input image, and image features necessary for estimating the graphic elements are extracted from the setting image 20. In the tuning process 220, the setting image 20 is decomposed into one or a plurality of graphic elements, and parameters (hereinafter also referred to as “graphic parameters”) indicating graphic elements corresponding to the graphic elements designated by the user are output. In the setting storage process 230, the graphic parameters output in the tuning process 220 are sequentially stored.

  The pre-processing 210 includes an image feature extraction module 212 that extracts image features included in the input setting image 20. As the image feature to be extracted, any type of information may be used, but in the present embodiment, edge points are extracted as image features. As a configuration for extracting edge points, the image feature extraction module 212 includes a gradient calculation module 214 and an edge peak point calculation module 216.

  The gradient calculation module 214 calculates the gradient of the pixel value along the search direction arbitrarily set for the setting image 20. More specifically, the gradient calculation module 214 sequentially calculates changes in pixel values between adjacent pixels. The edge peak point calculation module 216 calculates an edge point from the gradient of the pixel value calculated by the gradient calculation module 214. More specifically, the edge peak point calculation module 216 extracts a peak point from a profile along a calculated search direction having a gradient, and determines an edge point from the extracted peak point. The edge image 218 including the image feature extracted by the preprocessing 210 is a processing target of the tuning processing 220.

  The functions of the tuning process 220 include a graphic recognition module 222 and a graphic selection module 224.

  The graphic recognition module 222 decomposes the setting image 20 into one or more graphic elements, and outputs each graphic element and roughly estimated graphic parameters. Then, the graphic recognition module 222 corresponds to a selection unit, and a graphic from the user with respect to the graphic element estimated to be included in the measurement image 30 or the setting image 20 including the same graphic element as the desired graphic element. Accept element selection. As a method for selecting a graphic element by the user, the setting image 20 may be displayed and the user may directly select a graphic element included in the setting image 20 or may be included in the setting image 20 1. Two or more graphic elements may be estimated and the user may select the estimated graphic elements as candidates. Details of the graphic element selection processing by the user will be described later.

  The graphic selection module 224 corresponds to a setting value acquisition unit, and stores graphic parameters corresponding to a graphic element selected from a plurality of graphic elements in the setting storage unit 232. Details of the graphic parameters to be determined will be described later. The graphic parameters determined by the graphic selection module 224 are stored in the setting storage unit 232 in the setting storage process 230.

  The graphic parameters stored in the setting storage unit 232 are used when performing image processing on the measurement image 30 in the measurement processing 300.

  Through such a series of processes, it is possible to more easily determine graphic parameters for extracting the performance of an algorithm for estimating graphic elements.

  A graphic fitting module 310 is included as a function of the measurement process 300. The figure fitting module 310 corresponds to an estimation unit, specifies a figure element to be estimated included in the measurement image 30 based on a feature amount stored in the setting storage unit 232, and indicates an estimation indicating the specified figure element Output the result. Details of functions and processing in the figure fitting module 310 will be described later.

<D. Figure Recognition Module>
Next, the graphic element disassembly process and selection process in the graphic recognition module 222 included in the tuning process 220 shown in FIG. 5 will be described in more detail.

  The graphic recognition module 222 decomposes the setting image 20 into one or a plurality of graphic elements. FIG. 6 is a diagram for explaining a decomposition process into graphic elements of the image processing apparatus according to the present embodiment. FIG. 7 is a diagram illustrating an example of a graphic parameter of each graphic element generated by the disassembly process illustrated in FIG.

  As an example, it is assumed that the setting image 20 includes an image as shown in FIG. The graphic recognition module 222 decomposes the setting image 20 into a plurality of graphic elements 31 to 34 as shown in FIG. That is, the graphic element 31 is a circular element, and the graphic elements 32 to 34 are linear elements. In the following description, it is assumed that the graphic element that the user wants to recognize is the graphic element 34. Here, it is assumed that the true graphic parameters of the graphic element 34 are inclination = −1.0 and intercept = 15.0.

  The graphic recognition module 222 calculates the graphic parameters of each decomposed graphic element. Each figure parameter calculated is shown in FIG. When the user selects the graphic element 34, a graphic parameter corresponding to the graphic element 34 is selected. That is, by selecting a graphic element estimated to be included in the measurement image 30 or selecting a graphic element from the user for the setting image 20 including the same graphic element as the desired graphic element, the corresponding graphic parameter is set. It is determined. That is, a graphic element desired by the user and a roughly estimated graphic parameter are determined.

  As described below, the user may select a graphic element by displaying a graphic element obtained by the disassembly process as a selection candidate and explicitly selecting it by the user. Alternatively, among the graphic elements obtained by the decomposition process, a graphic element that matches a predetermined selection criterion may be selected. For example, if the selection criterion is “linear element” and “slope <0”, the linear element of the graphic element 34 is selected.

  Hereinafter, a process for giving an instruction for selecting a desired graphic element by the user will be described with reference to FIGS.

  FIG. 8 is a diagram illustrating an example of a display screen 250 for selecting a graphic element provided by the image processing apparatus 100 according to the present embodiment. FIG. 9 is a diagram illustrating a processing example when the user selects a linear element on the display screen 250 illustrated in FIG. 8. FIG. 10 is a diagram showing a processing example when the user selects a circle element on the display screen 250 shown in FIG.

  In the display screen 250 shown in FIG. 8, the input setting image 20 is displayed (image display area 252), and a button for selecting the type of graphic element to be selected is displayed (straight element selection button). 254 and a circle element selection button 256). When the user selects the linear element selection button 254, the screen transitions to a display screen for selecting a linear element as shown in FIG. 9, and when the user selects the circular element selection button 256, the circular element as shown in FIG. Transitions to the display screen for selection.

  It is not always necessary to display buttons for selecting the type of graphic element (straight element selection button 254 and circle element selection button 256) so that the type is automatically determined from the graphic element selected by the user. May be.

  When the linear element is selected, a display screen 260 as shown in FIG. 9A is displayed. The display screen 260 includes an image display area 252 for displaying the input setting image 20 and a coordinate value setting area 266 for displaying information of the selected straight line element. As shown in FIG. 9A, the user designates a graphic element to be selected for the setting image 20 displayed in the image display area 252. For example, as shown in FIG. 9A, the start point 262 and the end point 264 of the target graphic element are designated by operating the mouse. By specifying the start point and end point, the coordinate value of each point is set. That is, in the coordinate value setting area 266, the coordinate values of the designated start point and end point are set and displayed.

  FIG. 9A shows an example of an operation of specifying the start point and end point of a graphic element by dragging with the mouse. However, the operation is not limited to this, and the operation of tracing the periphery of the specified graphic element with the mouse is shown. Alternatively, it may be specified by an operation of enclosing an area including several points representing the graphic element to be specified with dots, lines, ellipses, or the like.

  On the other hand, when a circle element is selected, a display screen 280 as shown in FIG. 10A is displayed. Display screen 280 includes an image display area 252 for displaying input setting image 20 and a coordinate value setting area 288 for displaying information on the selected circle element. As shown in FIG. 10A, the user designates a graphic element to be selected for the setting image 20 displayed in the image display area 252. For example, as shown in FIG. 10A, the center point 282, the radius 284 in the X direction from the center point 282, and the radius 286 in the Y direction from the center point 282 are specified by operating the mouse. By specifying the center point and the radius in each axis direction, the coordinates of the center point and the radius in each axis direction are set (coordinate value setting area 288).

  FIG. 10A shows an example of an operation for specifying the center point and radius of a graphic element by dragging with the mouse. However, the present invention is not limited to this, and the periphery of the graphic element to be specified is traced with the mouse. It may be specified by an operation or an operation that encloses an area including some points representing the graphic element to be specified by a point, a line, an ellipse, or the like.

  As shown in FIGS. 9A and 10A, the user can directly specify a graphic element desired to be acquired by the input device. That is, the graphic recognition module 222 (FIG. 5) is configured so that the graphic element that the user wants to acquire can be directly specified via the input device.

  It is preferable that the graphic elements that can be selected by the user are estimated in advance from the setting image 20. That is, the graphic recognition module 222 (FIG. 5) preferably has a function of estimating one or more graphic elements included in the measurement image 30 as selection candidates from the feature amounts included in the measurement image 30. The function and process for estimating the graphic element from the measurement image 30 are the same as the function and process provided by the graphic element estimation module 314 (FIG. 12), which will be described later, and will not be described in detail here.

  9 and 10 exemplify display screens when selecting a linear element and a circular element as an example, the present invention is not limited thereto, and a graphic element having an arbitrary shape may be selected.

  As described above, the image processing apparatus 100 according to the present embodiment uses the setting image 20 so that the user can select a graphic element that the user wants to estimate. That is, the figure recognition module 222 (FIG. 5) has a function of displaying the measurement image 30 and receiving a user operation on the displayed measurement image 30.

<E. Figure selection module>
Next, graphic parameter output processing by the graphic selection module 224 included in the tuning processing 220 shown in FIG. 5 will be described. In the measurement process 300, the graphic parameters are (1) selection of graphic elements (hereinafter also referred to as “representative graphic elements”) to be subjected to graphic fitting processing, (2) removal of abnormal points, (3) smoothing, (4) Used as information for optimizing processing such as parameter determination of graphic elements.

  More specifically, the graphic selection module 224 uses, as the feature amount of the graphic element selected by the user, for example, the angle of the graphic element, the length of the graphic element, the size of the graphic element, the area of the graphic element, the area of the graphic element, Information such as the standard deviation of noise, the number of outliers, the strength of lines constituting the graphic element, the relative position of the graphic element in the image, and the color of the graphic element are acquired. Then, based on the feature amounts of these graphic elements, representative graphic elements are selected.

  Also, there are settings for dynamically determining parameters used in the measurement processing 300, such as abnormal point removal, smoothing, and graphic element parameter determination, in association with the feature amount of the selected graphic element. Done. For example, in FIG. 9 (b) and FIG. 10 (b) described above, it is set based on what feature amount the parameter used for removing the abnormal point is determined according to the graphic element selected by the user. The display screen to be displayed is shown.

  The display screen 270 shown in FIG. 9B includes an abnormal point removal setting area 276 for setting parameters used for removing abnormal points when a linear element is selected. In the abnormal point removal setting area 276, three prioritized feature quantities can be set.

  Similarly, the display screen 290 shown in FIG. 10B includes an abnormal point removal setting area 296 for setting parameters used for removing abnormal points when a circular element is selected. In the abnormal point removal setting area 296, three feature values with priorities can be set.

  When the measurement process 300 is executed, parameters used for removing abnormal points are dynamically determined using the feature values selected in the abnormal point removal setting area 276 or 296 among the feature values of the selected representative graphic element. Is done. At this time, three feature amounts are prioritized, and an object to be removed as an abnormal point is determined for each feature amount according to the priorities. That is, the graphic recognition module 222 accepts a designation from the user regarding the type of feature amount used to determine the parameter of the operation of the removing means.

  As shown in FIG. 9B and FIG. 10B, the user can specify the feature amount of the graphic element to be acquired. That is, the graphic recognition module 222 (FIG. 5) accepts the designation of the feature amount of the graphic element that the user wants to acquire.

<F. Measurement processing>
Next, with reference to FIG. 11, a usage example in the measurement process of the graphic parameter output from the graphic selection module 224 will be described. FIG. 11 is a diagram for explaining an example of using graphic parameters provided by the image processing apparatus according to the present embodiment. That is, FIG. 11 shows the result of precise estimation of the figure parameters obtained by performing the precision estimation using the surrounding data points for the figure element selected by the selection process.

  Let us consider a case where the precise estimation of the graphic parameters of the graphic element 34 shown in FIG. 6 is performed. It is assumed that the true graphic parameters of the graphic element 34 are inclination = −1.0 and intercept = 15.0.

  Referring to FIG. 11A, if the graphic parameters as described above are not used, the graphic element 34 (straight line element) to be estimated and graphic elements that are not to be estimated are mixed, and the estimation error becomes large. It is assumed that the graphic parameters in this case are slope = −1.2 and intercept = 15.8. This is a value having an error with respect to a true graphic parameter.

  On the other hand, referring to FIG. 11 (b), the periphery of the graphic element 34 is likely to be a graphic element other than the graphic element 34 (straight line element). The estimation accuracy is improved by narrowing. As described above, the graphic parameter output by the tuning process 220 is used for the measurement process.

  FIG. 12 is a schematic diagram illustrating a functional configuration for realizing the measurement process 300 in the image processing apparatus 100 according to the present embodiment. 12, the figure fitting module 310 (FIG. 5) of the measurement process 300 includes an image feature extraction module 311, a figure element estimation module 314, a representative figure selection module 315, a noise removal module 316, and a figure parameter. And an estimation module 320.

  The image feature extraction module 311 executes preprocessing for the measurement image 30. More specifically, the image feature extraction module 311 extracts image features included in the input measurement image 30. As the image feature to be extracted, any type of information may be used, but in the present embodiment, edge points are extracted as image features. As a configuration for extracting edge points, the image feature extraction module 311 includes a gradient calculation module 312 and an edge peak point calculation module 313. Functions in image feature extraction module 311 are similar to those in image feature extraction module 212 including gradient calculation module 214 and edge peak point calculation module 216, and thus detailed description will not be repeated.

  The graphic element estimation module 314 estimates one or more graphic elements included in the measurement image 30 from the feature amounts included in the measurement image 30. More specifically, the graphic element estimation module 314 processes the edge image of the measurement image 30 generated by the image feature extraction module 311 and estimates the graphic element included in the edge image.

  The representative graphic selection module 315 is based on the graphic parameters (feature amount of the graphic element selected by the user) set in the setting process 200 from one or more graphic elements estimated by the graphic element estimation module 314. Select a representative graphic element.

  The noise removal module 316 corresponds to a noise removal unit, and removes disturbance information from the measurement image 30 in accordance with the graphic element to be estimated (selected representative graphic element). The noise removal module 316 typically includes an abnormal point removal module 317 and a data smoothing module 318. Here, the noise removal module 316 operates according to a parameter determined based on a feature amount associated with a graphic element to be estimated (selected representative graphic element). Both the abnormal point removal module 317 and the data smoothing module 318 operate according to parameters determined based on the feature amount, and an example of this processing will be described later.

  The abnormal point removal module 317 removes abnormal points that do not correspond to the selected representative graphic element. Typically, the abnormal point removal module 317 determines an edge point that is determined not to be related to the representative graphic element as an abnormal point with respect to the edge image of the measurement image 30, and is determined to be the abnormal point. Remove edge points.

  The data smoothing module 318 performs filtering according to the selected representative graphic element. Typically, the data smoothing module 318 applies various smoothing processes to remove / reduce random noise included in the edge image of the measurement image 30.

  As will be described later, the abnormal point removal module 317 and the data smoothing module 318 execute processing according to the graphic parameters determined according to the selected representative graphic element. That is, each process is optimized according to the selected representative graphic element.

  The graphic parameter estimation module 320 estimates a parameter indicating a target graphic element from information on a region in the measurement image 30 corresponding to the selected representative graphic element. More specifically, the graphic parameter estimation module 320 determines graphic element parameters by performing graphic fitting on the edge image after noise removal from information on the representative graphic element selected by the representative graphic selection module 315. To do.

  As the figure fitting process by the figure parameter estimation module 320, the Hough transform is effective when the number of parameters to be estimated is relatively small. However, the present invention is not limited to these, and it may be determined using the least square method, and when the figure to be estimated has linearity, an algorithm such as principal component analysis can be adopted. Also in the graphic fitting process by the graphic parameter estimation module 320, the algorithm-specific parameter setting to be executed is determined according to the selected representative graphic element. In other words, the parameter estimation process indicating the graphic element is optimized according to the selected representative graphic element.

  As a result of the figure fitting process by the figure parameter estimation module 320, an estimation result is output.

<G. Optimization of processing using graphic parameters>
Next, an example of process optimization using the graphic parameters set in the setting process 200 will be described.

(G1: representative graphic element selection process)
As described above, the feature amount of the representative graphic element selected by the user is stored in the setting process 200 which can be said to be a pre-process of the measurement process 300. This feature amount includes the angle of the graphic element, the length of the graphic element, the size of the graphic element, the area of the graphic element, the standard deviation of the noise of the graphic element, the number of outliers, and the strength of the lines that make up the graphic element , The relative position of the graphic element in the image, the color of the graphic element, and the like. A representative graphic element included in the measurement image 30 is selected on the basis of the feature amount of each graphic element.

  As an example, the representative graphic selection module 315 selects one of the graphic elements estimated by the graphic element estimation module 314 that is most suitable for one or a plurality of criteria (here, feature amounts of each graphic element). May be. A plurality of graphic elements may be selected as representative graphic elements. In this case, a plurality of graphic elements of the same type such as two straight lines may be selected, or different types of graphic elements such as one straight line and one circle may be selected, or both may be mixed. The selection method as described above may be adopted. Basically, feature quantities are learned separately from the graphic elements selected by the user, so that a plurality of graphic elements can be selected as representative graphic elements. Further, a target graphic element may be selected from one or more graphic elements according to the degree of conformity with a predetermined standard. For example, when only one specific graphic element is selected, it is only necessary to select the most suitable graphic element among the estimated graphic elements, or to select a single type of graphic element. When selecting a plurality, it is sufficient to select as many as necessary that have the highest degree of conformity to the reference.

  On the other hand, when selecting different types of graphic elements, a first reference for the first type of graphic element and a second reference for the second type of graphic element are prepared. The graphic element that best matches each of the first criterion and the second criterion may be selected.

  The criteria used for selecting the representative graphic element include, for example, the length, size, area, line strength, fitting error, and predesignated angle / position / size for the graphic element selected in advance by the user. A feature amount such as similarity, dissimilarity, and color with respect to the height / radius / center position is used.

  When a plurality of types of feature amounts (graphic parameters) are set, priorities between feature amounts may be further set. Alternatively, a plurality of types of feature amounts may be combined, and a weighting factor is given to each feature amount, and a selection target is determined according to a fitness degree calculated by reflecting these weighting factors. May be.

(G2: Optimization of abnormal point removal parameters)
The abnormal point removal module 317 shown in FIG. 12 determines a parameter for determining a target to be removed as an abnormal point according to the selected representative graphic element.

  As an example, as shown in FIGS. 9B and 10B, the types of feature amounts of graphic elements for determining parameters for removing abnormal points are set in advance, and the set feature amounts are used. The criterion for determining an abnormal point may be dynamically determined.

  Instead of or in addition to this reference, an abnormality occurs depending on the amount of deviation from the reference distance when the user selects in the setting process 200 for the graphic element corresponding to the selected representative graphic element. The number of points to be removed as points may be determined. That is, with reference to the position of the graphic element selected by the user in the setting process 200, the abnormality determination criterion may be determined according to the distance of the user input from this criterion. Typically, an edge point that is farther from the reference position of the graphic element selected by the user in the setting process 200 is more likely to be determined as an abnormal point.

(G3: Data smoothing parameter optimization)
The data smoothing module 318 shown in FIG. 12 determines a smoothing parameter according to the selected representative graphic element. That is, the data smoothing module 318 performs filtering according to the selected graphic element (representative graphic element). More specifically, the number of smoothing and the smoothing filter size are automatically determined according to the feature amount of the graphic element selected by the user. For example, if the graphic element selected as the representative graphic element is relatively large, the filter size may be relatively large, and the number of times of smoothing may be relatively large. On the contrary, if the intensity of the line constituting the graphic element selected as the representative graphic element is relatively low, the filter size needs to be relatively small. Thus, the data smoothing parameters are optimized according to the feature amount of the graphic element selected as the representative graphic element.

(G4: Parameter optimization for figure parameter estimation)
The graphic parameter estimation module 320 shown in FIG. 12 determines the algorithm parameters according to the selected representative graphic element. For the estimation of graphic parameters in the graphic parameter estimation module 320, an appropriate algorithm is selected from a plurality of algorithms according to the type of the selected representative graphic element. This optimizes the parameters specific to the selected algorithm. That is, the graphic parameter estimation module 320 is configured to be able to execute an algorithm according to the type of graphic element to be estimated among a plurality of algorithms. The operation parameters of each of the plurality of algorithms are determined based on the feature amount associated with the graphic element to be estimated.

  For example, when a RANSAC (Random Sampling Consensus) method, which is an example of a random sampling method, is used as a parameter estimation algorithm, a combination of points selected at random according to the feature amount of a graphic element selected by the user, convergence Conditions, employment conditions, etc. are automatically determined. As a parameter estimation algorithm, in addition to the RANSAC method, a robust estimation method, a Hough transform (linear element), a Hough transform (circle element), a least square method, a principal component analysis method, a local outlier voting method, and the like can be used. . For each algorithm, the parameters determined according to the feature amount of the selected representative graphic element are as follows.

(A) Robust estimation method: number of employed points, weight (b) Hough transform (linear element): number of employed points, threshold for accumulated value, middle point coordinates (c) Hough transform (circle element): center coordinates, search radius ( d) Least-squares method: number of adopted points, search range (midpoint, slope)
(E) Principal component analysis: Basis number at the time of calculating the cumulative contribution ratio used for axis calculation (f) Local outlier voting method: Point combination The graphic element parameter estimation algorithm is not limited to the above, and any algorithm can be used. Regardless of which algorithm is used, parameters necessary for execution can be optimized according to the feature amount of the selected representative graphic element.

<H. Processing procedure>
FIG. 13 is a flowchart for explaining a processing procedure in the image processing apparatus 100 of the present embodiment. The processing procedure shown in FIG. 13 is typically realized by the processor of the image processing apparatus 100 executing a control program. The control program includes an image processing program for estimating a graphic element included in the input image. When the control program is executed by the image processing apparatus 100, an image processing method for estimating the graphic element included in the input image is executed. The

  As described above, in the image processing apparatus 100, the setting process 200 for determining various graphic constant parameters and the measurement process 300 for actually executing image processing on the input image are executed. In FIG. 13, the setting process 200 corresponds to steps S100 to S108, and the measurement process 300 corresponds to steps S200 to S210.

With reference to FIG. 13, the setting image 20 is acquired (step S100).
Subsequently, the setting image 20 is displayed, and selection of a graphic element from the user for the setting image 20 is accepted (step S102). At this time, one or more graphic elements included in the measurement image 30 may be estimated from the feature amounts included in the measurement image 30 and then displayed as selection candidates. Subsequently, when the user selects a graphic element, the feature amount of the selected graphic element is acquired from the setting image 20 and stored (step S104).

  Then, it is determined whether another graphic element has been selected by the user for the setting image 20 (step S106). If another graphic element is selected from the user for setting image 20 (YES in step S106), the process of step S104 is repeated.

  On the other hand, when another graphic element is not selected from the user for the setting image 20 (NO in step S106), it is determined whether another setting image 20 is requested by the user. Judgment is made (step S108). If another setting image 20 is requested by the user (YES in step S108), the processes in and after step S100 are repeated.

  On the other hand, if another setting image 20 is not requested from the user (NO in step S108), the setting process 200 ends.

After the setting process 200 is completed, the measurement process 300 is started at an arbitrary timing.
In response to a periodic or some event, the measurement image 30 is acquired (step S200). As a method of acquiring the measurement image 30, in addition to a method of acquiring data generated when the imaging unit 8 connected to the image processing apparatus 100 images the subject, the subject is imaged using an external imaging device. Thus, the input image generated may be acquired via some medium.

  Subsequently, a feature amount included in the measurement image 30 is extracted (step S202). Specifically, an edge image indicating the edge points included in the measurement image 30 is generated by calculating a gradient and an edge peak point for the measurement image 30.

  Subsequently, the estimation target graphic element included in the measurement image 30 is specified based on the feature amount of each graphic element acquired and stored in the setting process 200, and a parameter indicating the specified graphic element is output. A graphic fitting process is executed on the measurement image 30.

  Based on the feature quantity of each graphic element stored in advance, a representative graphic element is selected from the measurement image 30 from the feature quantities included in the measurement image 30 (step S204). By applying a Hough transform, a clustering algorithm, various detection logics, and the like to the edge image of the measurement image 30, the graphic elements included in the image are estimated.

  Information that becomes a disturbance is removed according to the selected representative graphic element (step S206). Specifically, noise factors caused by graphic elements other than the representative graphic element are reduced according to the selected representative graphic element. As processing for reducing the noise factor, removal of abnormal points and data smoothing are used. At this time, parameters for removing abnormal points and smoothing the data are determined based on the feature amount associated with the selected representative graphic element.

  Subsequently, a figure fitting process is executed. That is, a parameter indicating the target representative graphic element is estimated (step S208).

  At this time, in accordance with the algorithm for estimating the graphic element, the operation parameter is determined from the feature quantity associated with the selected representative graphic element.

  Finally, the estimation result of the representative graphic element acquired in step S208 is output (step S210). Then, the process ends.

  According to the image processing apparatus of the present embodiment, in the setting process, information for optimally performing graphic element estimation processing is acquired, and an algorithm for estimating graphic elements based on the previously acquired information Therefore, it is possible to determine a more appropriate parameter for extracting the performance.

  In addition, since the information acquired in advance in the setting process is used in the selection of the representative graphic element to be estimated for the graphic element, even when a plurality of graphic elements are mixed in the measurement image, The estimation accuracy can be increased because it is possible to estimate the graphic element by focusing on the target graphic element.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  DESCRIPTION OF SYMBOLS 1 Image processing system, 2 Measurement object, 5 PLC, 6 Movable stage, 8 Imaging part, 20 Setting image, 30 Measurement image, 100 Image processing apparatus, 200 Setting process, 210 Preprocessing, 212,311 Image feature extraction Module, 214, 312 Gradient calculation module, 216, 313 Edge peak point calculation module, 220 Tuning processing, 222 Graphic recognition module, 224 Graphic selection module, 230 Setting storage processing, 232 Setting storage section, 300 Measurement processing, 310 Graphic fitting module 314 Graphic element estimation module, 315 Representative graphic selection module, 316 Noise removal module, 317 Abnormal point removal module, 318 Data smoothing module, 320 Graphic parameter estimation module.

Claims (13)

A selection means for accepting selection of a graphic element from the user for the setting image;
Setting value acquisition means for acquiring and storing the feature amount of the selected graphic element;
An estimation unit that identifies a graphic element to be estimated included in the measurement image based on the stored feature amount and outputs a parameter indicating the identified graphic element;
Image processing device.   The image processing apparatus according to claim 1, wherein the selection unit includes a unit that estimates one or more graphic elements included in the measurement image as selection candidates from a feature amount included in the measurement image.   The image processing apparatus according to claim 1, wherein the selection unit is configured to be able to directly specify a graphic element that the user wants to acquire via an input device.   The image processing apparatus according to claim 1, wherein the selection unit accepts designation of a feature amount of a graphic element that the user wants to acquire.   The image processing apparatus according to claim 1, wherein the selection unit includes a unit that displays the measurement image and receives a user operation on the displayed measurement image. Further comprising a removing means for removing information that becomes a disturbance according to the graphic element to be estimated from the measurement image;
The image processing apparatus according to claim 1, wherein the removing unit operates in accordance with a parameter determined based on a feature amount associated with the graphic element to be estimated.   The image processing apparatus according to claim 6, wherein the removing unit includes a unit that removes or weights abnormal points that do not correspond to the selected graphic element.   The image processing apparatus according to claim 6, wherein the removing unit includes a unit that performs filtering according to the selected graphic element or adds a weight based on a filtering result.   The image processing apparatus according to claim 6, wherein the selection unit receives a designation from a user regarding a type of feature amount used for determining a parameter related to an operation of the removal unit. The estimation means is configured to be able to execute an estimation algorithm according to the type of graphic element to be estimated among a plurality of estimation algorithms,
The image processing apparatus according to claim 1, wherein a parameter relating to an operation of each of the plurality of estimation algorithms is determined based on a feature amount associated with the graphic element to be estimated. Imaging means for generating the measurement image by imaging a subject;
The image processing apparatus according to claim 1, further comprising: an output unit that outputs an instruction to a driving device that changes the position of the subject based on a parameter indicating the estimated graphic element. . Receiving a selection of a graphic element from the user for the setting image;
Obtaining and storing a feature quantity of the selected graphic element;
An image processing method comprising: specifying a graphic element to be estimated included in the measurement image based on the stored feature amount and outputting a parameter indicating the specified graphic element. An image processing program, the image processing program receiving a selection of a graphic element from a user for a setting image on a computer;
Obtaining and storing a feature quantity of the selected graphic element;
An image processing program for executing a step of specifying a graphic element to be estimated included in a measurement image and outputting a parameter indicating the specified graphic element based on the stored feature amount.