EP3060894B1 - Axial force measurement method - Google Patents

Axial force measurement method Download PDF

Info

Publication number
EP3060894B1
EP3060894B1 EP14798934.7A EP14798934A EP3060894B1 EP 3060894 B1 EP3060894 B1 EP 3060894B1 EP 14798934 A EP14798934 A EP 14798934A EP 3060894 B1 EP3060894 B1 EP 3060894B1
Authority
EP
European Patent Office
Prior art keywords
head
axial force
measurement method
bolt
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP14798934.7A
Other languages
German (de)
French (fr)
Other versions
EP3060894A1 (en
Inventor
Hisanori Nakamura
Masahito Sakakibara
Tatsuro Mori
Yuichi Hirano
Yusuke Matsumoto
Takashi SAKUI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toyota Motor Corp
Original Assignee
Toyota Motor Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toyota Motor Corp filed Critical Toyota Motor Corp
Publication of EP3060894A1 publication Critical patent/EP3060894A1/en
Application granted granted Critical
Publication of EP3060894B1 publication Critical patent/EP3060894B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • G06T7/0006Industrial image inspection using a design-rule based approach
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01LMEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE
    • G01L5/00Apparatus for, or methods of, measuring force, work, mechanical power, or torque, specially adapted for specific purposes
    • G01L5/0028Force sensors associated with force applying means
    • G01L5/0038Force sensors associated with force applying means applying a pushing force
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/66Analysis of geometric attributes of image moments or centre of gravity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30164Workpiece; Machine component

Definitions

  • the invention relates to an axial force measurement method for measuring an axial force of a bolt fastened to an object to be fastened.
  • the axial force of a bolt fastened to an object to be fastened is measured to, for example, verify the fastened state of the bolt.
  • There is a correlation between the axial force and the displacement amount of the bolt head that is, the difference in the recession amount of the head between the states before and after fastening. Therefore, the axial force is typically measured by acquiring the correlation between the axial force and the displacement amount of the head in advance, measuring the displacement amount of the head when actually measuring the axial force, and then measuring the axial force on the basis of the correlation.
  • the invention provides an axial force measurement method by which an axial force can be measured accurately and stably.
  • an axial force measurement method for measuring an axial force of a bolt fastened to an object to be fastened includes: acquiring a distance image with a height as a pixel value by measuring a height of the head of the bolt with a distance sensor; calculating a gravity center position of the head of the bolt by subjecting the distance image to binarization processing to acquire a binarized image and extracting an image of the head of the bolt from the binarized image; calculating an average value of a height of pixels positioned inside a minimum circle as a minimum point by subjecting the distance image to mask processing and extracting only pixels positioned inside the minimum circle centered on the gravity center position of the head of the bolt; calculating an average value of the height of extracted pixels with a relatively large height as a maximum point by extracting pixels with the relatively large height from the pixels of the distance image on the basis of pixel value of the pixel; calculating a displacement amount of the head of the bolt on the basis of a difference between a calculation result on the minimum point and
  • the pixels with the relatively large height may be extracted on the basis of the pixel value of the extracted pixel by extracting pixels positioned on virtual straight line from the distance image, the virtual straight line extending at a constant angular interval from the gravity center position of the head of the bolt toward the outside of the bolt.
  • pixels may be extracted by subjecting the distance image to mask processing, the pixels to be extracted being positioned outside a first circle which is centered on the gravity center position of the head of the bolt, the first circle being larger than the minimum circle, and the pixels to be extracted also being positioned inside a second circle which is centered on the gravity center position of the head of the bolt, the second circle being larger than the first circle, and the pixels positioned on the virtual straight lines may be extracted from among the extracted pixels.
  • pixels may be extracted as pixels with the relatively large height from the pixels by subjecting the distance image to mask processing, the pixels to be extracted being positioned outside a first circle which is centered on the gravity center position of the head of the bolt, first circle being larger than the minimum circle, and also positioned inside a second circle which is centered on the gravity center position of the head of the bolt, the second circle being larger than the first circle and the second circle being smaller than the head of the bolt.
  • the minimum circle may be smaller than a recess of the head of the bolt, the recess being formed in the gravity center position of the head of the bolt.
  • the effect demonstrated according to the aspect of the invention is that the axial force can be measured accurately and stably.
  • the axial force measurement method serves to measure the axial force of a bolt 10 fastened to an object to be fastened W.
  • the bolt 10 of the first embodiment is assumed to be a hexagonal bolt in which a recess 11a is formed in the gravity center (center) of a head 11, and an outer end portion is formed as a flat top portion 11b. Further, in the bolt 10, a flange 12 is formed below the head 11. The recess 11a is positioned lower than the top portion 11b.
  • the axial force measurement method With the axial force measurement method, the displacement amount of the head 11 is measured, and the axial force is measured on the basis of the displacement amount measurement result and the correlation.
  • the axial force measurement method significantly differs from the related art in that the displacement amount of the head 11 is measured using image processing.
  • the bolt which is the measurement object of the axial force measurement method is not limited to the bolt 10 of the first embodiment and may be, for example, a bolt with a flat head.
  • the top portion to the head is a portion that is not recessed even after the fastening, that is, close to the outer edge portion of the head.
  • the bolt which is the measurement object of the axial force measurement method may or may not have a flange formed thereon.
  • the axial force is measured using a distance sensor 20 and a personal computer 30.
  • the distance sensor 20 is disposed above the bolt 10, that is, at one side of the head 11, and measures the distance to the head 11 of the bolt 10.
  • the distance sensor 20 thus measures the height of the head 11.
  • the distance sensor 20 is provided with a storage device that stores a program for executing the image processing (the below-described smoothing processing, and the like) with respect to the measurement result on the distance to the head 11, and a computational device for executing the program.
  • the distance sensor 20 is electrically connected to the personal computer 30 and inputs the measurement result into the personal computer 30.
  • the personal computer 30 is provided with a storage device that stores a program for executing the image processing (the below-described binarization processing and mask processing) and the computational processing for implementing the axial force measurement method, and also a computational device for executing the program.
  • the personal computer 30 stores the correlation of the axial force and the displacement amount of the head 11, that is, the relational expression of such a correlation which is presented in FIG. 13 , in the storage device.
  • the personal computer 30 also stores the recession amount of the recess 11a (see the recess 11 a depicted by a two-dot chain line in FIG. 1 ) before the fastening in the storage device.
  • a step of acquiring the distance image 21 of the head 11 of the bolt 10 fastened to the object to be fastened W is performed with the distance sensor 20 (step S10).
  • the height of the head 11 of the bolt 10 is measured, for example, by measuring the height of the head 11 of the bolt 10 while moving the distance sensor 20 in the horizontal direction.
  • the distance image 21 with a height in the form of a pixel value is acquired.
  • the height of the head 11 of the bolt 10, the flange 12, and the upper surface of the object to be fastened W is indicated as pixel values.
  • the reference symbol A in FIG. 3 denotes a portion in which the measured height is larger than the actual height under the effect of dirt or the like.
  • outliers are removed from the distance image 21 by the distance sensor 20 (S20).
  • the axial force measurement method for example, pixels of a predetermined range are extracted from the distance image 21 and a simple average of the height of the predetermined range is calculated on the basis of the pixel values of the extracted pixels. Further, in the axial force measurement method, pixel values that are at a large distance from the calculated result of the simple average are determined to be outliers and removed from the pixels of the predetermined range.
  • the distance image 21 depicted in FIG. 3 is described as an image from which such outliers have been removed.
  • the distance image 21 from which the outliers have been removed is subjected to smoothing processing with the distance sensor 20 (step S30).
  • the smoothing processing is performed to smooth out the pixel values.
  • the smoothing processing performed with respect to the distance image 21 is, for example, the so-called Gauss filter processing in which the pixel value of a pixel of interest and the weighted average value of the pixel values of the pixels surrounding the pixel of interest are taken as the pixel values after the filtering.
  • the smoothing processing may be also the so-called median filter processing in which the pixel value of a pixel of interest and the central value of the pixel values of the pixels surrounding the pixel of interest are taken as the pixel values after the filtering.
  • the distance image 21 is acquired from which the surface roughness (fine irregularity) of the head 11 and the measurement error are removed as noise components.
  • the distance sensor 20 inputs the distance image 21 subjected to the smoothing processing into the personal computer 30 (see an arrow 21 shown in FIG. 1 ).
  • steps S20 and S30 may be also performed in the personal computer.
  • the distance image 21 subjected to the smoothing processing is subjected to binarization processing by the personal computer 30 (step S40).
  • the distance image 21 is colored in two different colors by using a threshold of pixel values as a reference.
  • the threshold of pixel values is set such that the head 11 has one color (for example, a white color) and the flange 12 and the object to be fastened W have the other color (for example, a black color), and the distance image 21 subjected to the smoothing processing is subjected to the binarization processing.
  • Such threshold of pixel values is set on the basis of a test that is performed before the axial force is actually measured.
  • a binarized image 31 is acquired in which the head 11 and portions other than the head 11 are colored in two different colors.
  • the portion A representing the measurement result which is larger than the actual height due to the effect of dirt or the like has the same color as the head 11 after the binarization processing.
  • the binarized image 31 is subjected to labeling processing by the personal computer 30 (step S50).
  • the same numbers are allocated to the pixels with the same continuing pixel values, and the pixels are color-coded for each number (by the surface area).
  • such labeling processing performed with respect to the binarized image 31 produces the labeling image 32 in which the portion A representing the measurement result which is larger than the actual height due to the effect of dirt or the like, the head 11, and other portions are colored in three different colors.
  • the image of the head 11 is extracted from the labeling image 32 by the personal computer 30 (step S60).
  • the surface area of pixel groups which have been color-coded from the labeling image 32 is calculated and a pixel group of a color for which the calculation result is less than a predetermined surface area is removed.
  • the pixel group corresponding to the head 11 is determined on the basis of the surface area of the pixel groups and the position thereof.
  • the processing of changing the pixel value of the pixel group determined to be the head 11 to a white color (that is, 255) and the pixel values of other pixel groups to a black color (that is, 0) is performed with respect to the labeling image 32.
  • a head extraction image 33 is created in which the head 11 and other portions are thus colored in two different colors.
  • the image of the head 11 can be extracted by removing the portion to which the dirt or the like has adhered. Therefore, the image of the head 11 can be accurately extracted.
  • the height of the object to be fastened W can be reversed (becomes larger than the actual height).
  • the binarized image 31 can be created in which the head 11 and the object to be fastened W have the same color.
  • the head 11 and portions other than the head 11 can be colored differently by subjecting the binarized image 31 to the labeling processing. Therefore, with the axial force measurement method, the image of the head 11 can be accurately extracted even when the measurement range of the distance sensor 20 is exceeded between the flange 12 and the object to be fastened W.
  • a gravity center position C of the head 11 (that is, a pixel corresponding to the gravity center of the head 11) is calculated by the personal computer 30 from the head extraction image 33 (step S70).
  • a step of extracting the image of the head 11 of the bolt 10 from the head extraction image 33 and calculating the gravity center position C of the head 11 of the bolt 10 is performed.
  • the image of the head of the bolt may be extracted from the binarized image 31 created in step S40 or the head extraction image 33 created in step S60.
  • the head extraction image 33 be used when extracting the image of the head of the bolt, as in the first embodiment.
  • the gravity center position of the head can be calculated by removing the portion representing the measurement result which is larger than the actual height due to the effect of dirt or the like, and therefore the gravity center position can be calculated more accurately.
  • the distance image 21 subjected to the smoothing processing is subjected to the mask processing by the personal computer 30 (step S80).
  • the mask processing is the processing of displaying one part of an image and not displaying, that is, hiding, another part of the image.
  • a reference image K1 is created in which a minimum circle K10 centered on the gravity center position C of the head 11 is drawn.
  • the outside of the minimum circle K10 is blackened.
  • the reference image K1 in step S80 serves to hide the portion outside the minimum circle K10.
  • the minimum circle K10 of the reference image K1 is less than the recess 11a.
  • the black circle indicating the gravity center position C is presented for the sake of convenience in order to indicate the gravity center position C.
  • the actual reference image is the image obtained by removing the black circle indicating the gravity center position C from the reference image K1 depicted in FIG. 9 .
  • the reference image K1 is overlapped on the distance image 21 subjected to the smoothing processing and a mask image 34 is created in which only the pixels positioned inside the minimum circle K10 are displayed in the distance image 21 subjected to the smoothing processing.
  • a minimum point is calculated by the personal computer 30 from the pixel values of the pixels of the distance image 21 that has been subjected to the mask processing and extracted, that is, has been continuously displayed in the mask image 34 (step S90).
  • the minimum point is the height of the head 11 in the periphery of the gravity center, that is, the height of the most receding portion (located at a low position) in the head 11 at the time of fastening.
  • the minimum circle K10 of the reference image K1 is a circle centered on the gravity center position C of the head 11. Therefore, each pixel of the distance image 21 that has been continuously displayed in the mask image 34 corresponds to the measurement result of the height of the head 11 in the periphery of the gravity center.
  • a simple average of the height of the pixels of the distance image 21 that has been continuously displayed in the mask image 34, that is, the pixels positioned inside the minimum circle K10, is calculated.
  • the result of calculating such a simple average is taken as the minimum point.
  • the axial force measurement method even when the calculated gravity center position C shifts with respect to the actual gravity center position, the error in calculation of the minimum point can be reduced. Further, in the axial force measurement method, even when small irregularity is formed in the periphery of the gravity center of the head 11, a variation in the minimum point caused by the irregularity can be reduced. In other words, in the axial force measurement method, the height of the minimum point can be measured accurately and stably.
  • a step of calculating the average value of the height of the pixels positioned inside the minimum circle K10 as the minimum point is performed.
  • the size of the minimum circle in step S80 is not particularly limited, provided that it is sufficiently large not to exceeds the range of the head, but a circle disposed inside the recess, for example, a circle with a radius of about several millimeters is preferred. With such a size, in the axial force measurement method, the height of the gravity center position periphery can be calculated more accurately.
  • the personal computer 30 subjects the distance image 21 that has been subjected to the smoothing processing, to the mask processing that differs in the processing contents from the processing of step S80 (step S100).
  • a reference image K2 is created that includes a first circle K21 which is larger than the minimum circle K10 of the reference image K1 and centered on the gravity center position C of the head 11, and a second circle K22 which is larger than the first circle K21 and centered on the gravity center position C of the head 11.
  • the inside of the first circle K21 the outside of the second circle K22 are both blackened.
  • the reference image K2 in step S100 serves to hide the zone inside the first circle K21 and the zone outside the second circle K22.
  • the first circle K21 is less than the head 11. In other words, the first circle K21 is less than the entire head 11 by the portion thereof that is not blackened. It is particularly preferred that the size of such first circle K21 be about the size of the recess 11a.
  • the second circle K22 is larger than the head 11 and less than the flange 12. It is particularly preferred that the size of such second circle K22 be about the size of the circumscribed circle of the head 11.
  • a white circle indicating the gravity center position C is presented for the sake of convenience in order to indicate the gravity center position C.
  • the actual reference image is the image obtained by removing the white circle indicating the gravity center position C from the reference image K2 depicted in FIG. 10 .
  • the reference image K2 is overlapped on the distance image 21 subjected to the smoothing processing and a mask image 35 is created in which only the pixels positioned between the circles K21 and K22 are displayed in the distance image 21 subjected to the smoothing processing.
  • a maximum point is calculated by the personal computer 30 from the distance image 21 that has been subjected to the mask processing and extracted, that is, has been continuously displayed in the mask image 35 (step S110).
  • the maximum point is the height of the top portion 11b of the head 11, that is, the height of the portion that does not recede at the time of fastening.
  • the first circle K21 is greater than the minimum circle K10 in the mask image 34 and disposed inside the head 11.
  • the second circle K22 is disposed outside the head 11. Therefore, the pixels of the distance image 21 that has been continuously displayed in the mask image 35 correspond to the measurement results on the height of the outer edge of the recess 11 a, top portion 11b, and flange 12.
  • pixels positioned on a virtual straight line L extending outward from the gravity center position C are extracted from the pixels extracted by performing the mask processing. Then, in the axial force measurement method, a plurality (for example, 30) of pixels with a relatively large height is extracted on the basis of the pixel values of the extracted pixels (see the circle depicted below mask image 35 on the lower side in FIG. 11 ). In this case, from the pixels extracted on the virtual straight line L, a plurality of pixels, the heights of which are arranged in descending order from the largest height, can be extracted as the pixels with a large height.
  • the curve depicted in the lower end portion of FIG. 11 represents the height of the pixel values positioned on the virtual straight line L. The height is shown to increase in the upward direction in FIG. 11 .
  • pixels positioned on a plurality of virtual straight lines L extending at a constant angular interval are extracted and the aforementioned processing of "extracting a plurality of pixels with a relatively large height from the extracted pixels positioned on the virtual straight line L" is implemented with respect to each virtual straight line L.
  • the angular interval of the virtual straight lines L is 20°, so that the mask image 35 could be easily seen.
  • the pixels of the top portion 11b are extracted from the mask image 35 by extracting a plurality of pixels with a large height that are positioned on the virtual straight lines L, this processing being performed with respect to a plurality of virtual straight lines L extending at a constant angular interval.
  • the simple average of the height of the extracted pixels is calculated and the calculation result is taken as the maximum point. More specifically, the maximum point is calculated by calculating the simple average of the height of the pixels extracted on the virtual straight lines L (simple average of the pixels for each virtual straight line L), and further calculating the simple average of the simple average values relating to each virtual straight line L.
  • the entire periphery of the top portion 11b is taken as the maximum point calculation object, instead of calculating the height of very few locations, that is, one or two points of the top portion 11b.
  • the variation of the maximum point can be reduced even when the phase of the head 11 is different or when small irregularity has been formed on the top portion 11b.
  • the maximum point can be measured accurately and stably.
  • the axial force measurement method includes a step of extracting the pixels with a large height on the basis of the pixel values of the pixels of the distance image 21 and calculating the average value of the height of the extracted pixels with a large height as the maximum point.
  • the pixels positioned on the virtual straight lines L may be extracted from the distance image 21.
  • the pixels positioned on the virtual straight lines L be extracted from the mask image, as in the first embodiment.
  • the extraction of small protruding portions formed close to the outer edge of the recess at the time the maximum point is calculated can be prevented. Therefore, the maximum point can be calculated more accurately in this axial force measurement method. Therefore, in the axial force measurement method, in particular, when the axial force of a bolt with a flat head is measured, the height of the maximum point can be measured more accurately and stably.
  • a step of calculating the displacement amount of the head 11 is performed by the personal computer 30 on the basis of the difference between the calculation result on the minimum point and the calculation result on the maximum point (step S120).
  • the displacement amount of the head 11 is calculated by calculating the difference between the calculation result on the minimum point and the calculation result on the maximum point and by calculating the difference between the calculation result on the difference between the minimum point and the maximum point, and the recession amount of the head 11 before the fastening which has been calculated in advance (heightwise dimension from the top portion 11b to the bottom of the recess 11 a).
  • the displacement amount of the head 11 can be calculated on the basis of the minimum point and maximum point that are calculated accurately and stably. Therefore, in the axial force measurement method, the displacement amount of the head 11 can be calculated accurately and stably.
  • a step of calculating the axial force is performed by the personal computer 30 (step S130).
  • the axial force is calculated by substituting the calculation result on the displacement amount of the head 11 into a pre-calculated relational expression of an axial force and the displacement amount of the head 11.
  • the displacement amount of the head 11, which is calculated accurately and stably, is substituted into the relational expression of the, axial force and the displacement amount of the head 11. Therefore, with the axial force measurement method, even when small irregularity has been formed on the periphery of the gravity center and the top portion 11b of the head 11, a displacement amount which is significantly different from the actual displacement amount can be prevented from being substituted into the relational expression of the axial force and the displacement amount of the head 11. Therefore, with the axial force measurement method, the axial force can be measured accurately and stably.
  • the order of calculating the minimum point and maximum point is not limited to that of the first embodiment, and the minimum point may be calculated after calculating the maximum point.
  • the relational expression of the axial force and the displacement amount of the head 11 is calculated using the distance sensor 20, the personal computer 30, and a load cell 40.
  • the distance sensor 20 and the personal computer 30 are the distance sensor 20 and the personal computer 30 that are used when measuring the axial force.
  • the personal computer 30 stores in the storage device a program for executing the calculation processing for calculating the relational expression of the axial force and the displacement amount of the head 11.
  • the load cell 40 is interposed between the bolt 10 and the object to be fastened W and measures the axial force.
  • the load cell 40 is electrically connected to the personal computer 30 and inputs the measurement result on the axial force into the personal computer 30 (see the arrow indicating the.load cell 40 in FIG. 14 ).
  • the relational expression of the axial force and the displacement amount of the head is calculated at a timing at which the relational expression of the axial force and the displacement amount of the head of the bolt, which is the axial force measurement object, can change, for example, when the material and type of the bolt are changed.
  • the recession amount of the head 11 before the fastening is calculated by the personal computer 30 (step S210).
  • steps S10 to S110 which are preformed when the axial force is measured, are performed in a state in which the measurement result of the load cell 40 is 0 kN, that is, a state in which no axial force has occurred.
  • the minimum point and maximum point of the bolt 10 before the fastening are calculated.
  • the recession amount of the head 11 before the fastening is calculated by calculating the difference between the calculation result on the minimum point and the calculation result on the maximum point.
  • the calculation result on the recession amount of the head 11 before the fastening is stored in the storage device of the personal computer 30.
  • the recession amount of the head 11 before the fastening which is calculated in such step S210, is used as the recession amount of the head 11 before the fastening when actually calculating the displacement amount of the head 11 (step S130).
  • the displacement amount of the head 11 before the fastening can be calculated correctly and stably.
  • step S210 is performed, as appropriate, and the recession amount of the head 11 before the fastening is acquired at a timing at which the recession amount of the head 11 before the fastening of the bolt 10, which is the axial force measurement object, can change, for example, when the lot of bolts 10 is changed.
  • the displacement amount of the head 11 is calculated by the personal computer 30 (step S220).
  • steps S10 to S120 which are performed when the axial force is measured, are performed in a state in which the bolt 10 is fastened to the object to be fastened W and the axial force is generated.
  • the displacement amount of the head 11 is calculated.
  • the measurement result of the load cell 40 and the calculation result on the displacement amount of the head 11 are stored in association with each other in the storage device of the personal computer 30.
  • the relationship between the axial force and the displacement amount of the head 11 can be acquired accurately and stably.
  • the relational expression of the axial force and the displacement amount of the head 11 is calculated by the personal computer 30 (step S230).
  • the relationship between the axial force (that is, the measurement result of the load cell 40) and the displacement amount of the head 11, which has been acquired in step S220, (see the black circles in FIG. 13 ) is arranged on a graph in which the displacement amount of the head 11 is plotted against the abscissa and the axial force is plotted against the ordinate. Further, in the axial force measurement method, the relation between the axial force and the displacement amount of the head 11 is acquired by approximating the measurement results on the axial force and the displacement amount of the head 11, with a monomial.
  • the relational expression of an axial force and the displacement amount of the head 11 can be calculated accurately and stably. Therefore, in the axial force measurement method, the axial force can be measured accurately and stably.
  • the contents of the mask processing performed during the calculation of the maximum point and the calculation contents of the maximum point are different from those of the axial force measurement method of the first embodiment. Therefore, the step of implementing the mask processing and the step of calculating the maximum point (steps corresponding to steps S100 and S110 of the first embodiment) will be described hereinbelow in detail, and the explanation of other steps will be omitted.
  • a reference image K3 is created, which includes a first circle K31 which is identical to the first circle K21 of the first embodiment and centered on the gravity center position C of the head 11 of the bolt 10 and a second circle K32 which is less than the second circle K22 of the first embodiment and centered on the gravity center position C of the head 11 of the bolt 10.
  • the inside of the first circle K31 and the outside of the second circle K32 are both blackened.
  • the reference image K3 is an image for hiding a range which is wider than that of the axial force measurement method of the first embodiment (see FIG. 11 ).
  • the second circle K32 is larger than the first circle K31 and less than the head 11 extracted in step S60. It is preferred that the size of such second circle K32 be about that of the inscribed circle of the head 11.
  • the white circle indicating the gravity center position C is presented for the sake of convenience in order to indicate the gravity center position C.
  • the actual reference image is the image obtained by removing the white circle indicating the gravity center position C from the reference image K3 depicted in FIG. 16 .
  • the reference image K3 is overlapped on the distance image 21 subjected to the smoothing processing, and a mask image 135 is created in which only the pixels positioned between the circles K31 and K32 are displayed in the distance image 21 subjected to the smoothing processing.
  • the mask image 135 is created with a displayed range narrower than that in the first embodiment.
  • the first circle K31 is smaller than the head 11.
  • the second circle K32 is as small as the inscribed circle of the head 11.
  • the mask processing is thus implemented on the distance image 21, and the pixels positioned outside the first circle K31 and also positioned inside the second circle K32 are extracted as the pixels with a large height.
  • a simple average of the height of the pixels extracted by performing the mask processing that is, the pixels of the distance image 21 that has been continuously displayed in the mask image 135, is calculated by the personal computer 30 as the maximum point.
  • the average value of the height of all of the pixels close to the outer edge of the top portion 11b is taken as the height of the maximum point.
  • the variation of the maximum point can be reduced, regardless of the phase of the head 11, or the presence or absence of small irregularity formed on the top portion 11b.
  • the maximum point can be measured accurately and stably. Therefore, in the axial force measurement method, the axial force can be measured accurately and stably.
  • the maximum point can be acquired without using the virtual straight lines extending outward from the gravity center position C. Therefore, the processing for measuring the axial force can be simplified.
  • the recession amount of the head 11 before the fastening is acquired in advance by using the axial force measurement method of the second embodiment.
  • the relational expression of the axial force and the displacement amount of the head 11 is acquired in advance by using the axial force measurement method of the second embodiment.
  • the axial force can be measured using the displacement amount of the head 11 before the fastening that has been calculated accurately and stably and the relational expression of the axial force and the displacement amount of the head 11. Therefore, in the axial force measurement method, the axial force can be measured accurately and stably.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Geometry (AREA)
  • Quality & Reliability (AREA)
  • Force Measurement Appropriate To Specific Purposes (AREA)

Description

    BACKGROUND OF THE INVENTION 1. Field of the Invention
  • The invention relates to an axial force measurement method for measuring an axial force of a bolt fastened to an object to be fastened.
  • 2. Description of Related Art
  • The axial force of a bolt fastened to an object to be fastened is measured to, for example, verify the fastened state of the bolt. There is a correlation between the axial force and the displacement amount of the bolt head (that is, the difference in the recession amount of the head between the states before and after fastening). Therefore, the axial force is typically measured by acquiring the correlation between the axial force and the displacement amount of the head in advance, measuring the displacement amount of the head when actually measuring the axial force, and then measuring the axial force on the basis of the correlation.
  • In "A Measurement Method of Bolt Loads Based on Deformation of a Top Surface of a Bolt Head", Transactions of the Japan Society of Mechanical Engineers. C 73(733), 2612-2618, the surface shape of the head is measured and the displacement amount of the head is measured on the basis of the difference between the displacement amount of the center (of gravity) of the head and the displacement amount of both ends. In "A Measurement Method of Bolt Loads Based on Deformation of a Top Surface of a Bolt Head", Transactions of the Japan Society of Mechanical Engineers. C 73(733), 2612-2618, the measurement results on the surface shape of the head are approximated by a six-order polynomial, and the displacement amount of the head is calculated from the approximation result in order to suppress the error in calculation of the displacement amount of the head caused by irregularity resulting from surface roughness.
  • However, in this case, it is not unlikely that the measurement results on the displacement amount at both ends cannot be accurately approximated, that is, the end portions obtained by approximation cannot be accurate. Therefore, with the technique disclosed in "A Measurement Method of Bolt Loads Based on Deformation of a Top Surface of a Bolt Head", Transactions of the Japan Society of Mechanical Engineers. C 73(733), 2612-2618, the displacement amount of the head cannot be measured accurately. As a result, it is not unlikely that the axial force cannot be measured accurately.
  • Further, with the technique disclosed in "A Measurement Method of Bolt Loads Based on Deformation of a Top Surface of a Bolt Head", Transactions of the Japan Society of Mechanical Engineers. C 73(733), 2612-2618, the displacement amount of the head is measured from the displacement amount in three locations, namely, in the center and at both ends of the head. Therefore, with the technique disclosed in "A Measurement Method of Bolt Loads Based on Deformation of a Top Surface of a Bolt Head", Transactions of the Japan Society of Mechanical Engineers. C 73(733), 2612-2618, where the measurement locations at both ends of the head are changed, the measurement results for the displacement amount of the head can change under the effect of the surface roughness of the head or irregularity formed at the head, even when the axial force is the same. In other words, with the technique disclosed in "A Measurement Method of Bolt Loads Based on Deformation of a Top Surface of a Bolt Head", Transactions of the Japan Society of Mechanical Engineers. C 73(733), 2612-2618, it is not unlikely that the axial force cannot be measured stably.
  • Other axial force measuring methods are known from documents JP 2006 194662 and JP 2011 179952 .
  • SUMMARY OF THE INVENTION
  • The invention provides an axial force measurement method by which an axial force can be measured accurately and stably.
  • According to an aspect of the invention, an axial force measurement method for measuring an axial force of a bolt fastened to an object to be fastened includes: acquiring a distance image with a height as a pixel value by measuring a height of the head of the bolt with a distance sensor; calculating a gravity center position of the head of the bolt by subjecting the distance image to binarization processing to acquire a binarized image and extracting an image of the head of the bolt from the binarized image; calculating an average value of a height of pixels positioned inside a minimum circle as a minimum point by subjecting the distance image to mask processing and extracting only pixels positioned inside the minimum circle centered on the gravity center position of the head of the bolt; calculating an average value of the height of extracted pixels with a relatively large height as a maximum point by extracting pixels with the relatively large height from the pixels of the distance image on the basis of pixel value of the pixel; calculating a displacement amount of the head of the bolt on the basis of a difference between a calculation result on the minimum point and a calculation result on the maximum point; and calculating an axial force by substituting the calculation result on the displacement amount of the head of the bolt into a pre-calculated relational expression of an axial force and the displacement amount of the head of the bolt.
  • In the axial force measurement method, when the maximum point is calculated, the pixels with the relatively large height may be extracted on the basis of the pixel value of the extracted pixel by extracting pixels positioned on virtual straight line from the distance image, the virtual straight line extending at a constant angular interval from the gravity center position of the head of the bolt toward the outside of the bolt.
  • In the axial force measurement method, when the maximum point is calculated, pixels may be extracted by subjecting the distance image to mask processing, the pixels to be extracted being positioned outside a first circle which is centered on the gravity center position of the head of the bolt, the first circle being larger than the minimum circle, and the pixels to be extracted also being positioned inside a second circle which is centered on the gravity center position of the head of the bolt, the second circle being larger than the first circle, and the pixels positioned on the virtual straight lines may be extracted from among the extracted pixels.
  • In the axial force measurement method, when the maximum point is calculated, pixels may be extracted as pixels with the relatively large height from the pixels by subjecting the distance image to mask processing, the pixels to be extracted being positioned outside a first circle which is centered on the gravity center position of the head of the bolt, first circle being larger than the minimum circle, and also positioned inside a second circle which is centered on the gravity center position of the head of the bolt, the second circle being larger than the first circle and the second circle being smaller than the head of the bolt.
  • In the axial force measurement method, the minimum circle may be smaller than a recess of the head of the bolt, the recess being formed in the gravity center position of the head of the bolt.
  • The effect demonstrated according to the aspect of the invention is that the axial force can be measured accurately and stably.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Features, advantages, and technical and industrial significance of exemplary embodiments of the invention will be described below with reference to the accompanying drawings, in which like numerals denote like elements, and wherein:
    • FIG. 1 is an explanatory drawing illustrating the configuration of a device for implementing the axial force measurement method according to the first embodiment;
    • FIG. 2 illustrates the procedure of the axial force measurement method according to the first embodiment;
    • FIG. 3 illustrates a distance image;
    • FIG. 4 illustrates a binarized image:
    • FIG. 5 illustrates a labeling image;
    • FIG. 6 illustrates a head extraction image;
    • FIG. 7 illustrates the results of labeling processing performed with respect to the distance image obtained by reversing the measurement result of a distance sensor;
    • FIG. 8 illustrates the distance image for which the gravity center position of the head has been calculated;
    • FIG. 9 illustrates a mask image for calculating the minimum point;
    • FIG. 10 illustrates a mask image for calculating the maximum point;
    • FIG. 11 illustrates the extraction of pixels with a relatively large height among the pixels positioned on the virtual straight lines from the mask image;
    • FIG. 12 illustrates a plurality of virtual straight lines and a mask image;
    • FIG. 13 illustrates the relationship between the axial force and the displacement amount of the head;
    • FIG. 14 illustrates the configuration of a device for acquiring the relationship between the axial force and the displacement amount of the head;
    • FIG. 15 illustrates the procedure for acquiring the relationship between the axial force and the displacement amount of the head; and
    • FIG. 16 illustrates a mask image for calculating the maximum point in the second embodiment.
    DETAILED DESCRIPTION OF EMBODIMENTS
  • The axial force measurement method according to the first embodiment will be described below.
  • As depicted in FIG. 1, the axial force measurement method serves to measure the axial force of a bolt 10 fastened to an object to be fastened W.
  • The bolt 10 of the first embodiment is assumed to be a hexagonal bolt in which a recess 11a is formed in the gravity center (center) of a head 11, and an outer end portion is formed as a flat top portion 11b. Further, in the bolt 10, a flange 12 is formed below the head 11. The recess 11a is positioned lower than the top portion 11b.
  • When the bolt 10 is fastened, the recess 11a recedes with centering on the gravity center, and the displacement amount of the head 11 increases as the axial force increases (see the recess 11 a in a state before the fastening which is indicated by a two-dot chain line in FIG. 1). In other words, there is a correlation between the axial force and the displacement amount of the head 11 before and after the fastening (see FIG. 13).
  • With the axial force measurement method, the displacement amount of the head 11 is measured, and the axial force is measured on the basis of the displacement amount measurement result and the correlation. In this case, the axial force measurement method significantly differs from the related art in that the displacement amount of the head 11 is measured using image processing.
  • The bolt which is the measurement object of the axial force measurement method is not limited to the bolt 10 of the first embodiment and may be, for example, a bolt with a flat head. When the axial force of such bolt with a flat head is measured, the top portion to the head is a portion that is not recessed even after the fastening, that is, close to the outer edge portion of the head. Further, the bolt which is the measurement object of the axial force measurement method may or may not have a flange formed thereon.
  • First, the configuration of the device used for implementing the axial force measurement method will be described below.
  • In the axial force measurement method, the axial force is measured using a distance sensor 20 and a personal computer 30.
  • The distance sensor 20 is disposed above the bolt 10, that is, at one side of the head 11, and measures the distance to the head 11 of the bolt 10. The distance sensor 20 thus measures the height of the head 11. The distance sensor 20 is provided with a storage device that stores a program for executing the image processing (the below-described smoothing processing, and the like) with respect to the measurement result on the distance to the head 11, and a computational device for executing the program. The distance sensor 20 is electrically connected to the personal computer 30 and inputs the measurement result into the personal computer 30.
  • The personal computer 30 is provided with a storage device that stores a program for executing the image processing (the below-described binarization processing and mask processing) and the computational processing for implementing the axial force measurement method, and also a computational device for executing the program. The personal computer 30 stores the correlation of the axial force and the displacement amount of the head 11, that is, the relational expression of such a correlation which is presented in FIG. 13, in the storage device. The personal computer 30 also stores the recession amount of the recess 11a (see the recess 11 a depicted by a two-dot chain line in FIG. 1) before the fastening in the storage device.
  • The procedure of the axial force measurement method will be described below.
  • As depicted in FIGS. 1 and 2, in the axial force measurement method, initially, a step of acquiring the distance image 21 of the head 11 of the bolt 10 fastened to the object to be fastened W is performed with the distance sensor 20 (step S10). In this case, in the axial force measurement method, the height of the head 11 of the bolt 10 is measured, for example, by measuring the height of the head 11 of the bolt 10 while moving the distance sensor 20 in the horizontal direction.
  • As a result, as depicted in FIG. 3, in the axial force measurement method, the distance image 21 with a height in the form of a pixel value is acquired.
  • In the distance image 21, the height of the head 11 of the bolt 10, the flange 12, and the upper surface of the object to be fastened W is indicated as pixel values. The reference symbol A in FIG. 3 denotes a portion in which the measured height is larger than the actual height under the effect of dirt or the like.
  • As shown in FIGS. 2 and 3, after the distance image 21 has been acquired, in the axial force measurement method, outliers are removed from the distance image 21 by the distance sensor 20 (S20). In this case, in the axial force measurement method, for example, pixels of a predetermined range are extracted from the distance image 21 and a simple average of the height of the predetermined range is calculated on the basis of the pixel values of the extracted pixels. Further, in the axial force measurement method, pixel values that are at a large distance from the calculated result of the simple average are determined to be outliers and removed from the pixels of the predetermined range.
  • For the sake of simplicity of explanation, the distance image 21 depicted in FIG. 3 is described as an image from which such outliers have been removed.
  • After the outliers have been removed, in the axial force measurement method, the distance image 21 from which the outliers have been removed is subjected to smoothing processing with the distance sensor 20 (step S30). The smoothing processing is performed to smooth out the pixel values.
  • In the axial force measurement method, the smoothing processing performed with respect to the distance image 21 is, for example, the so-called Gauss filter processing in which the pixel value of a pixel of interest and the weighted average value of the pixel values of the pixels surrounding the pixel of interest are taken as the pixel values after the filtering.
  • In the axial force measurement method, the smoothing processing may be also the so-called median filter processing in which the pixel value of a pixel of interest and the central value of the pixel values of the pixels surrounding the pixel of interest are taken as the pixel values after the filtering.
  • As a result, in the axial force measurement method, the distance image 21 is acquired from which the surface roughness (fine irregularity) of the head 11 and the measurement error are removed as noise components. As shown in FIG. 1, the distance sensor 20 inputs the distance image 21 subjected to the smoothing processing into the personal computer 30 (see an arrow 21 shown in FIG. 1).
  • In the axial force measurement method, steps S20 and S30 may be also performed in the personal computer.
  • As indicated in FIGS. 2 and 4, after the smoothing processing has been performed with respect to the distance image 21, in the axial force measurement method, the distance image 21 subjected to the smoothing processing is subjected to binarization processing by the personal computer 30 (step S40). In the binarization processing, the distance image 21 is colored in two different colors by using a threshold of pixel values as a reference.
  • In the axial force measurement method, the threshold of pixel values is set such that the head 11 has one color (for example, a white color) and the flange 12 and the object to be fastened W have the other color (for example, a black color), and the distance image 21 subjected to the smoothing processing is subjected to the binarization processing. Such threshold of pixel values is set on the basis of a test that is performed before the axial force is actually measured.
  • As a result, in the axial force measurement method, a binarized image 31 is acquired in which the head 11 and portions other than the head 11 are colored in two different colors.
  • In the first embodiment, it is assumed that the portion A representing the measurement result which is larger than the actual height due to the effect of dirt or the like, has the same color as the head 11 after the binarization processing.
  • As shown in FIGS. 2 and 5, after the binarization processing, in the axial force measurement method, the binarized image 31 is subjected to labeling processing by the personal computer 30 (step S50). In the labeling processing, the same numbers are allocated to the pixels with the same continuing pixel values, and the pixels are color-coded for each number (by the surface area).
  • In the axial force measurement method, such labeling processing performed with respect to the binarized image 31 produces the labeling image 32 in which the portion A representing the measurement result which is larger than the actual height due to the effect of dirt or the like, the head 11, and other portions are colored in three different colors.
  • As depicted in FIGS. 2 and 6, after the labeling processing, in the axial force measurement method, the image of the head 11 is extracted from the labeling image 32 by the personal computer 30 (step S60).
  • In this case, in the axial force measurement method, the surface area of pixel groups which have been color-coded from the labeling image 32 is calculated and a pixel group of a color for which the calculation result is less than a predetermined surface area is removed. In the axial force measurement method, the pixel group corresponding to the head 11 is determined on the basis of the surface area of the pixel groups and the position thereof. In the axial force measurement method, the processing of changing the pixel value of the pixel group determined to be the head 11 to a white color (that is, 255) and the pixel values of other pixel groups to a black color (that is, 0) is performed with respect to the labeling image 32.
  • In the axial force measurement method, a head extraction image 33 is created in which the head 11 and other portions are thus colored in two different colors.
  • As a result, in the axial force measurement method, for example, even when dirt or the like has adhered to the head 11, the image of the head 11 can be extracted by removing the portion to which the dirt or the like has adhered. Therefore, the image of the head 11 can be accurately extracted.
  • In this case, as depicted in FIG. 7, when the measurement range of the distance sensor 20 is exceeded with respect to the flange 12 and the object to be fastened W in the height-direction, that is, when the height position of the flange 12 is within the measurement range of the distance sensor 20, but the height position of the object to be fastened W is outside the measurement range of the distance sensor 20, the height of the object to be fastened W can be reversed (becomes larger than the actual height). In the axial force measurement method, where such distance image 21 is subjected to the binarization processing, the binarized image 31 can be created in which the head 11 and the object to be fastened W have the same color.
  • With the axial force measurement method, even in such a case, the head 11 and portions other than the head 11 can be colored differently by subjecting the binarized image 31 to the labeling processing. Therefore, with the axial force measurement method, the image of the head 11 can be accurately extracted even when the measurement range of the distance sensor 20 is exceeded between the flange 12 and the object to be fastened W.
  • As depicted in FIGS. 2 and 8, after the image of the head 11 has been extracted, in the axial force measurement method, a gravity center position C of the head 11 (that is, a pixel corresponding to the gravity center of the head 11) is calculated by the personal computer 30 from the head extraction image 33 (step S70).
  • In this case, in the axial force measurement method, for example, three straight lines connecting the opposing apexes of the hexagonal shape of the head 11 are drawn in the head extraction image 33, and the crossing point of the straight lines is calculated as the gravity center position C of the head 11.
  • In the axial force measurement method, it is not always necessary to draw the three straight lines, and the crossing point of two straight lines connecting the opposing apexes may be also calculated as the gravity center of the head.
  • Thus, in the axial force measurement method, a step of extracting the image of the head 11 of the bolt 10 from the head extraction image 33 and calculating the gravity center position C of the head 11 of the bolt 10 is performed.
  • Further, in the axial force measurement method, it is not always necessary to subject the binarized image to the labeling processing. In other words, in the axial force measurement method, the image of the head of the bolt may be extracted from the binarized image 31 created in step S40 or the head extraction image 33 created in step S60.
  • However, in the axial force measurement method, it is preferred that the head extraction image 33 be used when extracting the image of the head of the bolt, as in the first embodiment. As a result, in the axial force measurement method, the gravity center position of the head can be calculated by removing the portion representing the measurement result which is larger than the actual height due to the effect of dirt or the like, and therefore the gravity center position can be calculated more accurately.
  • As depicted in FIGS. 2 and 9, after the gravity center position C has been calculated, in the axial force measurement method, the distance image 21 subjected to the smoothing processing is subjected to the mask processing by the personal computer 30 (step S80). The mask processing is the processing of displaying one part of an image and not displaying, that is, hiding, another part of the image.
  • In the axial force measurement method, a reference image K1 is created in which a minimum circle K10 centered on the gravity center position C of the head 11 is drawn. In the reference image K1 in step S80, the outside of the minimum circle K10 is blackened. Thus, the reference image K1 in step S80 serves to hide the portion outside the minimum circle K10.
  • The minimum circle K10 of the reference image K1 is less than the recess 11a.
  • In the reference image K1 depicted in FIG. 9, the black circle indicating the gravity center position C is presented for the sake of convenience in order to indicate the gravity center position C. In other words, the actual reference image is the image obtained by removing the black circle indicating the gravity center position C from the reference image K1 depicted in FIG. 9.
  • In the axial force measurement method, the reference image K1 is overlapped on the distance image 21 subjected to the smoothing processing and a mask image 34 is created in which only the pixels positioned inside the minimum circle K10 are displayed in the distance image 21 subjected to the smoothing processing.
  • As a result, in the axial force measurement method, only the pixels positioned inside the minimum circle K10 are extracted from the distance image 21 subjected to the smoothing processing.
  • After the mask processing has been performed, in the axial force measurement method, a minimum point is calculated by the personal computer 30 from the pixel values of the pixels of the distance image 21 that has been subjected to the mask processing and extracted, that is, has been continuously displayed in the mask image 34 (step S90).
  • In the first embodiment, the minimum point is the height of the head 11 in the periphery of the gravity center, that is, the height of the most receding portion (located at a low position) in the head 11 at the time of fastening.
  • As mentioned hereinabove, the minimum circle K10 of the reference image K1 is a circle centered on the gravity center position C of the head 11. Therefore, each pixel of the distance image 21 that has been continuously displayed in the mask image 34 corresponds to the measurement result of the height of the head 11 in the periphery of the gravity center.
  • Accordingly, in the axial force measurement method, a simple average of the height of the pixels of the distance image 21 that has been continuously displayed in the mask image 34, that is, the pixels positioned inside the minimum circle K10, is calculated. In the axial force measurement method, the result of calculating such a simple average is taken as the minimum point.
  • In other words, in the axial force measurement method, not only the height of the gravity center of the head 11, that is, the height of only one point, but also the height of the periphery of the gravity center of the head 11, is the minimum point calculation object.
  • As a result, in the axial force measurement method, even when the calculated gravity center position C shifts with respect to the actual gravity center position, the error in calculation of the minimum point can be reduced. Further, in the axial force measurement method, even when small irregularity is formed in the periphery of the gravity center of the head 11, a variation in the minimum point caused by the irregularity can be reduced. In other words, in the axial force measurement method, the height of the minimum point can be measured accurately and stably.
  • Thus, in the axial force measurement method, a step of calculating the average value of the height of the pixels positioned inside the minimum circle K10 as the minimum point is performed.
  • The size of the minimum circle in step S80 is not particularly limited, provided that it is sufficiently large not to exceeds the range of the head, but a circle disposed inside the recess, for example, a circle with a radius of about several millimeters is preferred. With such a size, in the axial force measurement method, the height of the gravity center position periphery can be calculated more accurately.
  • As depicted in FIGS. 2 and 10, after the minimum point has been calculated, in the axial force measurement method, the personal computer 30 subjects the distance image 21 that has been subjected to the smoothing processing, to the mask processing that differs in the processing contents from the processing of step S80 (step S100).
  • At this time, in the axial force measurement method, a reference image K2 is created that includes a first circle K21 which is larger than the minimum circle K10 of the reference image K1 and centered on the gravity center position C of the head 11, and a second circle K22 which is larger than the first circle K21 and centered on the gravity center position C of the head 11. In the reference image K2 in step S100, the inside of the first circle K21 the outside of the second circle K22 are both blackened. In other words, the reference image K2 in step S100 serves to hide the zone inside the first circle K21 and the zone outside the second circle K22.
  • The first circle K21 is less than the head 11. In other words, the first circle K21 is less than the entire head 11 by the portion thereof that is not blackened. It is particularly preferred that the size of such first circle K21 be about the size of the recess 11a.
  • The second circle K22 is larger than the head 11 and less than the flange 12. It is particularly preferred that the size of such second circle K22 be about the size of the circumscribed circle of the head 11.
  • In the reference image K2 depicted in FIG. 10, a white circle indicating the gravity center position C is presented for the sake of convenience in order to indicate the gravity center position C. In other words, the actual reference image is the image obtained by removing the white circle indicating the gravity center position C from the reference image K2 depicted in FIG. 10.
  • In the axial force measurement method, the reference image K2 is overlapped on the distance image 21 subjected to the smoothing processing and a mask image 35 is created in which only the pixels positioned between the circles K21 and K22 are displayed in the distance image 21 subjected to the smoothing processing.
  • As a result, in the axial force measurement method, only the pixels positioned outside the first circle K21 and inside the second circle K22 are extracted from the distance image 21 subjected to the smoothing processing.
  • After the mask processing has been performed, in the axial force measurement method, a maximum point is calculated by the personal computer 30 from the distance image 21 that has been subjected to the mask processing and extracted, that is, has been continuously displayed in the mask image 35 (step S110).
  • In the first embodiment, the maximum point is the height of the top portion 11b of the head 11, that is, the height of the portion that does not recede at the time of fastening.
  • As mentioned hereinabove, the first circle K21 is greater than the minimum circle K10 in the mask image 34 and disposed inside the head 11. The second circle K22 is disposed outside the head 11. Therefore, the pixels of the distance image 21 that has been continuously displayed in the mask image 35 correspond to the measurement results on the height of the outer edge of the recess 11 a, top portion 11b, and flange 12.
  • As depicted in FIG. 11, in the axial force measurement method, pixels positioned on a virtual straight line L extending outward from the gravity center position C are extracted from the pixels extracted by performing the mask processing. Then, in the axial force measurement method, a plurality (for example, 30) of pixels with a relatively large height is extracted on the basis of the pixel values of the extracted pixels (see the circle depicted below mask image 35 on the lower side in FIG. 11). In this case, from the pixels extracted on the virtual straight line L, a plurality of pixels, the heights of which are arranged in descending order from the largest height, can be extracted as the pixels with a large height.
  • The curve depicted in the lower end portion of FIG. 11 represents the height of the pixel values positioned on the virtual straight line L. The height is shown to increase in the upward direction in FIG. 11.
  • As indicated in FIG. 12, in the axial force measurement method, pixels positioned on a plurality of virtual straight lines L extending at a constant angular interval (for example, 1 °) are extracted and the aforementioned processing of "extracting a plurality of pixels with a relatively large height from the extracted pixels positioned on the virtual straight line L" is implemented with respect to each virtual straight line L.
  • In FIG. 12, the angular interval of the virtual straight lines L is 20°, so that the mask image 35 could be easily seen.
  • Thus, in the axial force measurement method, the pixels of the top portion 11b are extracted from the mask image 35 by extracting a plurality of pixels with a large height that are positioned on the virtual straight lines L, this processing being performed with respect to a plurality of virtual straight lines L extending at a constant angular interval. In the axial force measurement method, the simple average of the height of the extracted pixels is calculated and the calculation result is taken as the maximum point. More specifically, the maximum point is calculated by calculating the simple average of the height of the pixels extracted on the virtual straight lines L (simple average of the pixels for each virtual straight line L), and further calculating the simple average of the simple average values relating to each virtual straight line L.
  • In other words, in the axial force measurement method, the entire periphery of the top portion 11b is taken as the maximum point calculation object, instead of calculating the height of very few locations, that is, one or two points of the top portion 11b.
  • As a result, in the axial force measurement method, the variation of the maximum point can be reduced even when the phase of the head 11 is different or when small irregularity has been formed on the top portion 11b. In other words, in the axial force measurement method, the maximum point can be measured accurately and stably.
  • Thus, the axial force measurement method includes a step of extracting the pixels with a large height on the basis of the pixel values of the pixels of the distance image 21 and calculating the average value of the height of the extracted pixels with a large height as the maximum point.
  • Further, in the axial force measurement method, it is not always necessary to perform the mask processing with respect to the distance image 21 when the maximum point is calculated. In other words, in the axial force measurement method, the pixels positioned on the virtual straight lines L may be extracted from the distance image 21.
  • However, in the axial force measurement method, it is preferred that the pixels positioned on the virtual straight lines L be extracted from the mask image, as in the first embodiment. As a result, the extraction of small protruding portions formed close to the outer edge of the recess at the time the maximum point is calculated can be prevented. Therefore, the maximum point can be calculated more accurately in this axial force measurement method. Therefore, in the axial force measurement method, in particular, when the axial force of a bolt with a flat head is measured, the height of the maximum point can be measured more accurately and stably.
  • As shown in FIG. 2, after the maximum point has been calculated, in the axial force measurement method, a step of calculating the displacement amount of the head 11 is performed by the personal computer 30 on the basis of the difference between the calculation result on the minimum point and the calculation result on the maximum point (step S120).
  • At this time, in the axial force measurement method, the displacement amount of the head 11 is calculated by calculating the difference between the calculation result on the minimum point and the calculation result on the maximum point and by calculating the difference between the calculation result on the difference between the minimum point and the maximum point, and the recession amount of the head 11 before the fastening which has been calculated in advance (heightwise dimension from the top portion 11b to the bottom of the recess 11 a).
  • In the axial force measurement method, the displacement amount of the head 11 can be calculated on the basis of the minimum point and maximum point that are calculated accurately and stably. Therefore, in the axial force measurement method, the displacement amount of the head 11 can be calculated accurately and stably.
  • As depicted in FIGS. 2 and 13, after the displacement amount of the head 11 has been calculated, in the axial force measurement method, a step of calculating the axial force is performed by the personal computer 30 (step S130). In this case, in the axial force measurement method, the axial force is calculated by substituting the calculation result on the displacement amount of the head 11 into a pre-calculated relational expression of an axial force and the displacement amount of the head 11.
  • In the axial force measurement method, the displacement amount of the head 11, which is calculated accurately and stably, is substituted into the relational expression of the, axial force and the displacement amount of the head 11. Therefore, with the axial force measurement method, even when small irregularity has been formed on the periphery of the gravity center and the top portion 11b of the head 11, a displacement amount which is significantly different from the actual displacement amount can be prevented from being substituted into the relational expression of the axial force and the displacement amount of the head 11. Therefore, with the axial force measurement method, the axial force can be measured accurately and stably.
  • The order of calculating the minimum point and maximum point is not limited to that of the first embodiment, and the minimum point may be calculated after calculating the maximum point.
  • The procedure for calculating the relational expression of the axial force and the displacement amount of the head 11 will be explained hereinbelow. Initially, the configuration of the device to be used for calculating the relational expression of the axial force and the displacement amount of the head 11 will be explained.
  • As depicted in FIG. 14, in the axial force measurement method, the relational expression of the axial force and the displacement amount of the head 11 is calculated using the distance sensor 20, the personal computer 30, and a load cell 40. The distance sensor 20 and the personal computer 30 are the distance sensor 20 and the personal computer 30 that are used when measuring the axial force. The personal computer 30 stores in the storage device a program for executing the calculation processing for calculating the relational expression of the axial force and the displacement amount of the head 11.
  • The load cell 40 is interposed between the bolt 10 and the object to be fastened W and measures the axial force. The load cell 40 is electrically connected to the personal computer 30 and inputs the measurement result on the axial force into the personal computer 30 (see the arrow indicating the.load cell 40 in FIG. 14).
  • The procedure for calculating the relational expression of the axial force and the displacement amount of the head 11 will be explained hereinbelow.
  • In the axial force measurement method, the relational expression of the axial force and the displacement amount of the head is calculated at a timing at which the relational expression of the axial force and the displacement amount of the head of the bolt, which is the axial force measurement object, can change, for example, when the material and type of the bolt are changed.
  • As depicted in FIGS. 14 and 15, in the axial force measurement method, the recession amount of the head 11 before the fastening is calculated by the personal computer 30 (step S210).
  • At this time, in the axial force measurement method, steps S10 to S110, which are preformed when the axial force is measured, are performed in a state in which the measurement result of the load cell 40 is 0 kN, that is, a state in which no axial force has occurred. As a result, in the axial force measurement method, the minimum point and maximum point of the bolt 10 before the fastening are calculated. Further, in the axial force measurement method, the recession amount of the head 11 before the fastening is calculated by calculating the difference between the calculation result on the minimum point and the calculation result on the maximum point. In the axial force measurement method, the calculation result on the recession amount of the head 11 before the fastening is stored in the storage device of the personal computer 30.
  • The recession amount of the head 11 before the fastening, which is calculated in such step S210, is used as the recession amount of the head 11 before the fastening when actually calculating the displacement amount of the head 11 (step S130).
  • As a result, in the axial force measurement method, the displacement amount of the head 11 before the fastening can be calculated correctly and stably.
  • In the axial force measurement method, step S210 is performed, as appropriate, and the recession amount of the head 11 before the fastening is acquired at a timing at which the recession amount of the head 11 before the fastening of the bolt 10, which is the axial force measurement object, can change, for example, when the lot of bolts 10 is changed.
  • After the recession amount of the head 11 before the fastening has been calculated, in the axial force measurement method, the displacement amount of the head 11 is calculated by the personal computer 30 (step S220).
  • At this time, in the axial force measurement method, steps S10 to S120, which are performed when the axial force is measured, are performed in a state in which the bolt 10 is fastened to the object to be fastened W and the axial force is generated. As a result, in the axial force measurement method, the displacement amount of the head 11 is calculated.
  • In the axial force measurement method, the measurement result of the load cell 40 and the calculation result on the displacement amount of the head 11 are stored in association with each other in the storage device of the personal computer 30.
  • In the axial force measurement method, such calculations of the displacement amount of the head 11 are repeated a plurality of times while changing the axial force.
  • As a result, with the axial force measurement method, the relationship between the axial force and the displacement amount of the head 11 can be acquired accurately and stably.
  • After the displacement amount of the head 11 at the time of fastening has been calculated, in the axial force measurement method, the relational expression of the axial force and the displacement amount of the head 11 is calculated by the personal computer 30 (step S230).
  • At this time, in the axial force measurement method, the relationship between the axial force (that is, the measurement result of the load cell 40) and the displacement amount of the head 11, which has been acquired in step S220, (see the black circles in FIG. 13) is arranged on a graph in which the displacement amount of the head 11 is plotted against the abscissa and the axial force is plotted against the ordinate. Further, in the axial force measurement method, the relation between the axial force and the displacement amount of the head 11 is acquired by approximating the measurement results on the axial force and the displacement amount of the head 11, with a monomial.
  • As a result, in the axial force measurement method, the relational expression of an axial force and the displacement amount of the head 11 can be calculated accurately and stably. Therefore, in the axial force measurement method, the axial force can be measured accurately and stably.
  • The axial force measurement method of the second embodiment will be explained below.
  • In the axial force measurement method of the second embodiment, the contents of the mask processing performed during the calculation of the maximum point and the calculation contents of the maximum point are different from those of the axial force measurement method of the first embodiment. Therefore, the step of implementing the mask processing and the step of calculating the maximum point (steps corresponding to steps S100 and S110 of the first embodiment) will be described hereinbelow in detail, and the explanation of other steps will be omitted.
  • As depicted in FIG. 16, in the axial force measurement method of the second embodiment, a reference image K3 is created, which includes a first circle K31 which is identical to the first circle K21 of the first embodiment and centered on the gravity center position C of the head 11 of the bolt 10 and a second circle K32 which is less than the second circle K22 of the first embodiment and centered on the gravity center position C of the head 11 of the bolt 10. In the reference image K3, the inside of the first circle K31 and the outside of the second circle K32 are both blackened. In other words, the reference image K3 is an image for hiding a range which is wider than that of the axial force measurement method of the first embodiment (see FIG. 11).
  • The second circle K32 is larger than the first circle K31 and less than the head 11 extracted in step S60. It is preferred that the size of such second circle K32 be about that of the inscribed circle of the head 11.
  • In the reference image K3 depicted in FIG. 16, the white circle indicating the gravity center position C is presented for the sake of convenience in order to indicate the gravity center position C. In other words, the actual reference image is the image obtained by removing the white circle indicating the gravity center position C from the reference image K3 depicted in FIG. 16.
  • In the axial force measurement method, the reference image K3 is overlapped on the distance image 21 subjected to the smoothing processing, and a mask image 135 is created in which only the pixels positioned between the circles K31 and K32 are displayed in the distance image 21 subjected to the smoothing processing. In other words, in the axial force measurement method of the second embodiment, the mask image 135 is created with a displayed range narrower than that in the first embodiment.
  • The first circle K31 is smaller than the head 11. The second circle K32 is as small as the inscribed circle of the head 11. Thus, in the axial force measurement method of the second embodiment, only the pixels close to the outer edge of the top portion 11b are extracted.
  • In the axial force measurement method of the second embodiment, the mask processing is thus implemented on the distance image 21, and the pixels positioned outside the first circle K31 and also positioned inside the second circle K32 are extracted as the pixels with a large height.
  • After the mask processing has been performed, in the axial force measurement method, a simple average of the height of the pixels extracted by performing the mask processing, that is, the pixels of the distance image 21 that has been continuously displayed in the mask image 135, is calculated by the personal computer 30 as the maximum point.
  • In other words, in the axial force measurement method of the second embodiment, the average value of the height of all of the pixels close to the outer edge of the top portion 11b is taken as the height of the maximum point.
  • As a result, in the axial force measurement method, the variation of the maximum point can be reduced, regardless of the phase of the head 11, or the presence or absence of small irregularity formed on the top portion 11b. Thus, in the axial force measurement method, the maximum point can be measured accurately and stably. Therefore, in the axial force measurement method, the axial force can be measured accurately and stably.
  • Further, in the axial force measurement method, the maximum point can be acquired without using the virtual straight lines extending outward from the gravity center position C. Therefore, the processing for measuring the axial force can be simplified.
  • Further, in the axial force measurement method of the second embodiment, the recession amount of the head 11 before the fastening is acquired in advance by using the axial force measurement method of the second embodiment. Furthermore, in the axial force measurement method of the second embodiment, the relational expression of the axial force and the displacement amount of the head 11 is acquired in advance by using the axial force measurement method of the second embodiment.
  • As a result, in the axial force measurement method, the axial force can be measured using the displacement amount of the head 11 before the fastening that has been calculated accurately and stably and the relational expression of the axial force and the displacement amount of the head 11. Therefore, in the axial force measurement method, the axial force can be measured accurately and stably.

Claims (5)

  1. An axial force measurement method for measuring an axial force of a bolt (10) fastened to an object to be fastened, the method characterized in that it comprises:
    acquiring a distance image (21) with a height as a pixel value by measuring a height of a head (11) of the bolt (10) with a distance sensor (20);
    calculating a gravity center position (C) of the head (11) of the bolt (10) by subjecting the distance image (21) to binarization processing to acquire a binarized image (31) and extracting an image of the head (11) of the bolt (10) from the binarized image (31);
    calculating an average value of a height of pixels positioned inside a minimum circle (K10) as a minimum point by subjecting the distance image (21) to mask processing and extracting only pixels positioned inside the minimum circle (K10) centered on the gravity center position (C) of the head (11) of the bolt (10);
    calculating an average value of the height of extracted pixels with a large height as a maximum point by extracting pixels with the large height from the pixels of the distance image (21) on the basis of the pixel value of the pixel;
    calculating a displacement amount of the head (11) of the bolt (10) on the basis of a difference between a calculation result on the minimum point and a calculation result on the maximum point; and
    calculating an axial force by substituting the calculation result on the displacement amount of the head (11) of the bolt (10) into a pre-calculated relational expression of an axial force and the displacement amount of the head (11) of the bolt (10).
  2. The axial force measurement method according to claim 1, wherein
    when the maximum point is calculated,
    the pixels with the large height are extracted from the extracted pixels on the basis of the pixel value of the pixel by extracting pixels positioned on a virtual straight line from the distance image (21), the virtual straight line extending at a constant angular interval from the gravity center position (C) of the head (11) of the bolt (10) toward the outside of the bolt (10).
  3. The axial force measurement method according to claim 2, wherein
    when the maximum point is calculated,
    pixels are extracted by subjecting the distance image (21) to mask processing, the pixels to be extracted being positioned outside a first circle which is centered on the gravity center position (C) of the head (11) of the bolt (10), the first circle being larger than the minimum circle (K10), and the pixels to be extracted also being positioned inside a second circle which is centered on the gravity center position (C) of the head (11) of the bolt (10), the second circle being larger than the first circle, and
    the pixels positioned on the virtual straight line are extracted from the extracted pixels.
  4. The axial force measurement method according to claim 1, wherein
    when the maximum point is calculated,
    pixels are extracted as pixels with the large height from the pixels by subjecting the distance image (21) to mask processing, the pixels to be extracted being positioned outside a first circle which is centered on the gravity center position (C) of the head (11) of the bolt (10), the first circle being larger than the minimum circle (K10), and the pixels to be extracted also being positioned inside a second circle which is centered on the gravity center position (C) of the head (11) of the bolt (10), the second circle being larger than the first circle and the second circle being smaller than the head (11) of the bolt (10).
  5. The axial force measurement method according to claim 1, wherein
    the minimum circle (K10) is smaller than a recess of the head (11) of the bolt (10), the recess being formed in the gravity center position (C) of the head (11) of the bolt (10).
EP14798934.7A 2013-10-21 2014-10-16 Axial force measurement method Not-in-force EP3060894B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013218632A JP5835300B2 (en) 2013-10-21 2013-10-21 Axial force measurement method
PCT/IB2014/002129 WO2015059538A1 (en) 2013-10-21 2014-10-16 Axial force measurement method

Publications (2)

Publication Number Publication Date
EP3060894A1 EP3060894A1 (en) 2016-08-31
EP3060894B1 true EP3060894B1 (en) 2017-07-05

Family

ID=51900478

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14798934.7A Not-in-force EP3060894B1 (en) 2013-10-21 2014-10-16 Axial force measurement method

Country Status (5)

Country Link
US (1) US9805459B2 (en)
EP (1) EP3060894B1 (en)
JP (1) JP5835300B2 (en)
CN (1) CN105659060B (en)
WO (1) WO2015059538A1 (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6536550B2 (en) 2016-12-08 2019-07-03 トヨタ自動車株式会社 Bolt axial force measuring device and bolt axial force measuring method
EP3379222B1 (en) 2017-03-22 2020-12-30 Methode Electronics Malta Ltd. Magnetoelastic based sensor assembly
JP7003724B2 (en) * 2018-02-22 2022-01-21 トヨタ自動車株式会社 Bolt axial force measurement method, fastening judgment method, axial force measuring device, fastening judgment device
US11135882B2 (en) 2018-02-27 2021-10-05 Methode Electronics, Inc. Towing systems and methods using magnetic field sensing
US11084342B2 (en) 2018-02-27 2021-08-10 Methode Electronics, Inc. Towing systems and methods using magnetic field sensing
US11221262B2 (en) 2018-02-27 2022-01-11 Methode Electronics, Inc. Towing systems and methods using magnetic field sensing
US11014417B2 (en) 2018-02-27 2021-05-25 Methode Electronics, Inc. Towing systems and methods using magnetic field sensing
US11491832B2 (en) 2018-02-27 2022-11-08 Methode Electronics, Inc. Towing systems and methods using magnetic field sensing
DE18907724T1 (en) 2018-02-27 2021-03-25 Methode Electronics, Inc. Towing systems and methods using magnetic field measurement
KR102070993B1 (en) * 2018-04-18 2020-01-29 공주대학교 산학협력단 Axial Force Measuring Device and Method of Bolt
JP2020134401A (en) * 2019-02-22 2020-08-31 トヨタ自動車株式会社 Shaft center behavior measurement method of crank shaft
JP2021086163A (en) 2019-11-25 2021-06-03 株式会社NejiLaw Individual identification system
JP7431662B2 (en) 2020-05-13 2024-02-15 株式会社日立製作所 Axial force calculation device
JP7419216B2 (en) * 2020-11-19 2024-01-22 株式会社日立製作所 Condition analysis system and condition analysis method for fastening members
CN112539866B (en) * 2020-11-20 2023-01-24 北京艾法斯特科技发展有限公司 Bolt axial force monitoring system and monitoring method based on visual deep learning

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004093362A (en) 2002-08-30 2004-03-25 Isuzu Motors Ltd Axial force measuring instrument, axial force measuring method and tightening device for bolt
JP2005297133A (en) 2004-04-13 2005-10-27 Isuzu Motors Ltd Fastening tool, and method for measuring axial force of bolt
JP2006194662A (en) * 2005-01-12 2006-07-27 Sanwa Tekki Corp Measuring method and device of bolt axial force
JP4636146B2 (en) * 2008-09-05 2011-02-23 ソニー株式会社 Image processing method, image processing apparatus, program, and image processing system
JP5402209B2 (en) * 2009-04-23 2014-01-29 富士ゼロックス株式会社 Image writing apparatus and image recording apparatus using the same
JP2011064105A (en) * 2009-09-16 2011-03-31 Hitachi Automotive Systems Ltd Valve timing control apparatus for internal combustion engine
JP2011179952A (en) 2010-03-01 2011-09-15 Isuzu Motors Ltd Bolt axial force detection system and bolt axial force detection method
CN102988052B (en) * 2011-09-09 2014-09-17 上海银晨智能识别科技有限公司 Method and system for measuring foot length
JP2013113696A (en) * 2011-11-29 2013-06-10 Mitsubishi Electric Corp Displacement measuring method and displacement measuring apparatus
CN102589461A (en) 2012-01-17 2012-07-18 华中科技大学 Image-based snow depth measuring method

Also Published As

Publication number Publication date
JP2015081798A (en) 2015-04-27
US9805459B2 (en) 2017-10-31
EP3060894A1 (en) 2016-08-31
CN105659060A (en) 2016-06-08
WO2015059538A1 (en) 2015-04-30
JP5835300B2 (en) 2015-12-24
CN105659060B (en) 2018-01-23
US20160267645A1 (en) 2016-09-15

Similar Documents

Publication Publication Date Title
EP3060894B1 (en) Axial force measurement method
EP3384429B1 (en) Method for identification of candidate points as possible characteristic points of a calibration pattern within an image of the calibration pattern
US20160055392A1 (en) Estimating degree of deterioration of road surface
US10043090B2 (en) Information processing device, information processing method, computer-readable recording medium, and inspection system
US9230318B2 (en) Analysis of the digital image of the external surface of a tyre and processing of false measurement points
US20170330315A1 (en) Information processing apparatus, method for processing information, discriminator generating apparatus, method for generating discriminator, and program
CN103673872A (en) Measurement method and measurement system of liquid drop volume
JP2012016454A5 (en)
CN111161263B (en) Package flatness detection method and system, electronic equipment and storage medium
KR102027986B1 (en) Bead recognition apparatus using vision camera and method thereof
CN107730521B (en) Method for rapidly detecting ridge type edge in image
JP2012120799A5 (en)
CN103400140A (en) Method for processing ice coated on insulator on basis of improved image method
CN115345821A (en) Steel coil binding belt loosening abnormity detection and quantification method based on active visual imaging
EP3955207A1 (en) Object detection device
CN115690104B (en) Wafer crack detection method and device and storage medium
US20140297224A1 (en) Data processing method of tire shape inspecting device
WO2016092783A1 (en) Information processing apparatus, method for processing information, discriminator generating apparatus, method for generating discriminator, and program
JP5155938B2 (en) Pattern contour detection method
US20200211194A1 (en) Attached object detection apparatus
CN106558048A (en) Screw array dystopy fault detection method and system
US10679336B2 (en) Detecting method, detecting apparatus, and computer readable storage medium
JP6114559B2 (en) Automatic unevenness detector for flat panel display
CN116703912B (en) Mini-host network port integrity visual detection method
CN108053414A (en) A kind of machine vision edge detecting technology

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20160421

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIC1 Information provided on ipc code assigned before grant

Ipc: G01L 5/00 20060101AFI20161206BHEP

Ipc: G06T 7/60 20170101ALI20161206BHEP

Ipc: G06T 7/00 20170101ALI20161206BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

DAX Request for extension of the european patent (deleted)
INTG Intention to grant announced

Effective date: 20170124

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 906927

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170715

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602014011579

Country of ref document: DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602014011579

Country of ref document: DE

Representative=s name: TBK, DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20170705

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 906927

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170705

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171005

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171006

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171105

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171005

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602014011579

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20180406

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: DE

Ref legal event code: R084

Ref document number: 602014011579

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171016

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171031

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171031

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20171031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171031

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171016

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20181016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20141016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20210913

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20210929

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20210908

Year of fee payment: 8

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602014011579

Country of ref document: DE

REG Reference to a national code

Ref country code: SE

Ref legal event code: EUG

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221031

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230503

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221017