JP5303405B2 - Vehicle inspection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle inspection apparatus performing inspection by taking a photograph of the side face of a vehicle, in particular, a vehicle inspection device forming an inspection image of the entire circumference of each wheel suitable for inspection, from among a plurality of partial images by correction, based on the rotation angles of the wheels among a plurality of cameras through measurement of the amount of abrasion that differs for each wheel, and the radius of the wheels. <P>SOLUTION: The vehicle inspection apparatus includes an image acquisition device disposed along a railway track to photograph a wheel image of the side face of a wheel of a vehicle traveling along the railway track; an image-storing device for storing the image photographed by the image-photographing device; and an image-processing device for performing image processing. The image storing device preliminarily stores a facing image of a reference wheel without abrasion. The image-processing device generates a facing image that faces the wheel, by applying planar projective transformation to the wheel image obtained by the image acquisition device; estimates the position of an axle of the wheel in the facing image; and stores, in the image storage device, an image in which the estimated axle position is corrected to coincide with the axle position of the facing image of the reference wheel, without abrasion stored in the image storage means. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a vehicle inspection device that photographs a wheel side surface of a vehicle and inspects and records the state of the wheel side surface.

  Since the wear and deterioration of the wheels proceed according to the travel distance of the vehicle, recording the progress of the deterioration is important for knowing the replacement time of the parts. The vehicle is stopped in the pit and the parts are inspected. However, the wheels are shielded by various parts in the periphery, and the entire circumference of the wheels cannot be acquired and inspected by one photographing.

  Under such circumstances, as a method for inspecting a wheel, a technique for photographing a wheel with a camera or the like while the vehicle is traveling is disclosed. For example, a wheel tread surface defect inspection device described in Patent Document 1 (Japanese Patent Laid-Open No. 7-174672) continuously measures a wheel tread surface with a plurality of cameras installed beside the track, and measures a wheel tread surface defect from the continuous image. . At this time, dirt and defects on the wheel are detected as foreign matters and then classified based on the aspect ratio.

  The wheel inspection apparatus described in Patent Document 2 (Japanese Patent Laid-Open No. 5-126686) is also obtained by continuously shooting the tread surface of a wheel approaching by a camera beside the track in the shape of a long slit in the horizontal direction, and stacking it vertically. The wheel tread for one round of the wheel is formed and stored.

  Here, it is assumed that all wheels have the same radius, and images taken by 10 cameras arranged at equal intervals of 1/10 of the perimeter of the wheel are 10 minutes of 360 degrees per rotation. In other words, it is described that panorama composition is performed by stacking at a pitch of 36 degrees. However, in reality, the radius of each wheel differs due to wear, and therefore the rotational speed of each wheel of a vehicle traveling at a constant speed is different. Therefore, the divided images of the wheels by the ten cameras are not always at a 36-degree pitch, and therefore a panoramic image suitable for inspection cannot be obtained even if they are stacked at a 36-degree pitch.

... JP-A-7-174672 ... Japanese Patent Laid-Open No. 5-126686

  By the way, since not only the wheel tread but also the wheel side surface is scratched and cracked, it is necessary to inspect the wheel side surface. Conventionally, only the wheel tread has been used for the technique of photographing and inspecting the wheel. The wheel side was not considered.

  Problems in the wheel side surface inspection will be described in detail later. However, a number of problems that did not become a problem in the wheel tread surface inspection were caused by the wear of the wheel. This means, for example, that the wheel radius changes due to wheel wear, and the result is that the vertical and horizontal alignment of the axle position in the photographed image and the correction of the rotation angle are required.

Further, it is effective to display and inspect the state of the entire circumference of the wheel side surface as a list, but in this case as well, a device that is not included in the tread surface inspection is necessary.
The problem to be solved by the present invention is to provide a vehicle inspection device capable of photographing and inspecting a wheel side surface.

  Furthermore, by correcting the amount of wear that differs for each wheel and the rotation angle of each wheel through measurement of the wheel radius, an inspection image of the entire circumference of the wheel suitable for inspection can be obtained from a plurality of partial images. The problem is to form.

  In the present invention, an image acquisition device that acquires a wheel image of a wheel side surface of a vehicle that is installed along the track and passes through the track, an image storage device that stores an image acquired by the image acquisition device, and image processing are performed. An image processing device, the image storage device holds in advance a facing image of the reference wheel without wear, and the image processing device performs a projective transformation on the wheel image acquired by the image acquisition device to the wheel. A front facing image is generated, the wheel axle position is estimated for the facing image, and the estimated axle position is held in the image storage means. The image is corrected to match the image and stored in the image storage device.

  In addition, the image processing apparatus performs polar coordinate conversion using the axle position as an origin for the wheel image after matching the axle position, generates a polar coordinate image, and stores the polar coordinate image in the image storage device.

  Further, the image storage device holds a template image on the polar coordinate image of the wheel bolt in advance, and the image processing device detects the bolt position by template matching on the polar coordinate image obtained by performing polar coordinate conversion with the axle position as the origin. Confirm.

  Further, the image processing device performs processing for detecting the presence or absence of a defect with respect to the polar coordinate image, and stores it in the image storage device.

  Further, the image processing device performs processing for detecting the presence or absence of a defect in the polar coordinate image, and stores the defect position in the image storage device in relation to the bolt position determined by template matching.

  Also, in the present invention, an image acquisition device that acquires a plurality of wheel images over the entire circumference of a wheel side surface of a vehicle that is installed along the track and passes through the track, and an image acquired by the image acquisition device. The image storage device includes an image storage device that stores images and an image processing device that performs image processing, and the image storage unit holds a front-facing image of a reference wheel that is free from wear. Projective transformation is performed on each of the plurality of wheel images over a range of distances to generate a plurality of facing images that face the wheels, and the axle positions of the wheels are estimated for each of the plurality of facing images, and the estimated axle position is Then, the image is stored in the image storage device after being corrected to the image matched with the axle position of the facing image of the reference wheel without wear held in the image storage means.

  Further, the image processing device generates a plurality of polar coordinate images by performing a polar coordinate conversion with the axle position as an origin for a plurality of wheel images after matching the axle position, and connects the plurality of polar coordinate images to the wheel image. Is created and stored in the image storage device.

  Moreover, a display means is provided, and a partial rectangular area of the polar coordinate image over the entire circumference of the connected wheel image is displayed on the display means.

  Moreover, a display means is provided, and a part of the trapezoidal region of the polar coordinate image over the entire circumference of the connected wheel images is extracted and displayed on the display means in the wheel shape.

  Further, the image storage means holds a template image on the polar coordinate image of the wheel bolt in advance, and the image processing device performs the bolt position by template matching on the polar coordinate image obtained by performing polar coordinate conversion with the axle position as the origin. And a plurality of polar coordinate images are connected from the bolt position interval to obtain a polar coordinate image over the entire circumference of the wheel image.

  The image processing apparatus obtains the radius or diameter of the wheel, obtains the rotation angle between the two or more cameras of the image acquisition apparatus based on this value, and acquires two or more images based on the rotation angle. The polar coordinate conversion images generated by the apparatus are joined to generate a polar coordinate conversion image for the entire circumference of the wheel.

  In addition, the image processing apparatus obtains a rotation angle that rotates between two or more cameras of the image acquisition apparatus by measuring the movement amount of the wheel structure by pattern matching processing. The polar coordinate conversion images generated by the above image acquisition device are joined to generate a polar coordinate conversion image for the entire circumference of the wheel.

  Further, the wear amount of the wheel is determined based on the difference between the radius of the wheel and the radius of the reference wheel.

  ADVANTAGE OF THE INVENTION According to this invention, the vehicle inspection apparatus which can image | photograph and test | inspect a wheel side surface can be provided, the image of the side surface for every wheel is managed, and database of a desired wheel is managed by managing a database by wheel ID number. A side image can be referenced.

  Further, it is possible to detect feature points by image processing, and it is possible to grasp the state of deterioration of the wheels by comparing with the feature points of the same wheel in the past. Further, a panoramic image can be formed by correcting according to the rotation angle of the wheel, whereby a wheel all-round image suitable for inspection can be obtained.

The figure which shows the processing flow of the image processing apparatus which concerns on the Example of this invention. The figure which shows the positional relationship of a test | inspection unit installed in the track and the track side. 1 is an overall configuration diagram of a vehicle inspection device 1. FIG. The figure which shows the structure of an image acquisition apparatus. The figure for demonstrating the imaging | photography visual field of a camera. The figure which shows that a wear degree differs for every wheel. The figure which shows that the camera is not facing directly with respect to a wheel. The figure which shows producing the difference of a horizontal direction at the time of imaging | photography for every wheel. The figure which shows imaging | photography at the time of a reference | standard wheel. The figure which shows imaging | photography at the time of a wear wheel. The figure which shows description of a coordinate system. The figure which shows the relationship between the reference | standard wheel 70 and the image matrix 25 on a screen. The figure which shows a reference | standard wheel world coordinate. The figure for defining a facing image matrix. The figure explaining an axle-axis coordinate detection process. The figure which shows the rectangle of a facing image matrix. The figure which shows the reference | standard wheel polar coordinate after conversion. The figure which shows the wheel image made into conversion object. The figure which shows the polar coordinate image matrix after conversion. The figure which shows the template of a volt | bolt. The figure which shows a polar coordinate image matrix. The figure which shows a feature point. The figure which shows the positional specification and ambiguity of a feature point. The figure which shows the several image from a several camera. The figure which shows the joining image of a several image. The figure which shows the panorama formation of the polar coordinate image matrix which concerns on the wheel 20. FIG. The figure which expanded virtual memory the polar coordinate image matrix of the wheel perimeter. The figure which shows the example which displayed a part of polar coordinate image matrix. The figure which shows the example displayed with the wheel image. The figure which expanded virtual memory the polar coordinate image matrix of the wheel perimeter. The figure which shows the example which displayed a part of polar coordinate image matrix. The figure which shows the example displayed with the wheel image. The figure explaining the importance of conversion to a standardized facing image matrix.

  Hereinafter, embodiments of the vehicle inspection apparatus of the present invention will be described.

  FIG. 2 is a diagram showing a positional relationship between the track and the inspection unit installed beside the track. As shown in this figure, a plurality of inspection units 2 (four in this embodiment) are installed on both sides of the lines 3 and 4. These units are 2 (RO) for inspecting the outside of the right wheel with respect to the traveling direction indicated by the arrow of the vehicle 6, 2 (RI) for inspecting the inside of the right wheel, and in the traveling direction of the vehicle 6. On the other hand, the unit that inspects the outside of the left wheel is named 2 (LO), and the unit that inspects the inside of the left wheel is also named 2 (LO), so that the outside and inside of the left and right wheels of the vehicle 6 are inspected independently. It has become.

  In general, since the vehicle inspection is performed before the vehicle 6 enters the garage, the inspection unit 2 is often installed in front of the garage. One or more vehicles form a train, and the train is assigned a train number. When such a vehicle approaches in front of the garage, the train number is specified from the train operation table and the train passing time, and the inspection unit 2 starts to operate. The inspection unit 2 can obtain the train wheel configuration as information as well as specifying the composition number. For this reason, if the image file is encoded and stored in the image file name while recognizing the number of the wheel image from the beginning, the image file will be stored in what number from the beginning of which organization. It is possible to specify whether the image is inside or outside of the left or right wheel.

  FIG. 3 is an overall configuration diagram of the vehicle inspection apparatus 1. In this figure, the vehicle inspection device 1 is composed of an inspection unit 2, a database 8, a panorama forming device 10, and a user interface device 12, and can communicate with each other via a communication line 14. Each of the four inspection units 2 (RO), 2 (RI), 2 (LO), and 2 (LI) in FIG. 2 has image acquisition devices 16, 17, 18, and 19, and For example, four images are acquired according to the rotation position.

  Each image acquisition device operates independently with the same function. An arrow in FIG. 3 indicates a direction in which the wheel 20 that is one of a plurality of wheels (hereinafter simply referred to as a wheel) included in the train including the vehicle 6 rolls. Each inspection unit 2 transmits four wheel images obtained by four-division shooting of the wheels 20 acquired by the respective image acquisition devices 16, 17, 18, 19 to the database 8, and the database 8 stores them.

  The panorama forming apparatus 10 generates a wheel all-round image that integrates one wheel from a plurality of divided wheel images stored in the database 8 and transmits the generated image to the user interface device 12. The user interface device 12 includes a command input device such as a keyboard and a mouse, and a display device. The user interface device 12 accesses the database 8 and refers to images and measurement data related to wheels, and performs pattern matching processing of wheel feature points to be described later. Do. Further, wheel images for the entire circumference transmitted from the panorama forming apparatus 10 are displayed on the display device. The operations of the database 8, the panorama forming apparatus 10, and the user interface apparatus 12 will be described in detail later.

  FIG. 4 is a diagram illustrating the configuration of the image acquisition device, and illustrates 16 configurations of the plurality of image acquisition devices as representatives. Since other image acquisition devices have the same configuration, description thereof is omitted here. The image acquisition device 16 includes a vehicle detection sensor 21, an illumination device 22, a camera 23, a wheel counting device 24, an image storage device 27, and an image processing device 26.

  The camera 23 is installed along the track 4 with a predetermined interval from an adjacent camera (for example, a camera attached to the image acquisition device 17), and the rotation of the wheel 20 is set at 360 degrees with one rotation of the wheel 20 being 360 degrees. Then, a part including the vicinity in contact with the track 4 is photographed. In the case of the present embodiment, by providing one camera for each of the image acquisition devices 16 to 19, a total of four cameras shoot one wheel, so that four shots are taken at a pitch of approximately 90 degrees. .

  FIG. 5 is a diagram for explaining the photographing field of view of the camera 23, and the rectangle 25 is the photographing field of view of the camera 23 provided in the image acquisition means 16. The wheel 20 of this embodiment has an axle 210 and eight bolts 31 to 37 at the center. However, the wheel to be inspected according to the present invention is not limited to this, and it is only necessary to have an axle at the center. Although the rectangle 25 which is the visual field of view of the camera includes a part of the wheel 20, it is not essential that the contact point 211 with the axle 210 and the track 4 is included in the field of view, but for the camera calibration work described later. It is preferable that the contact 211 is included.

  When photographing the entire circumference of the wheel with four cameras, the field of view of the camera 23 must be at least 46 degrees on both sides of the contact point 211. On the other hand, if the field of view is excessively widened, the resolution is lowered. Therefore, within the rectangle 25, a region 28 indicated by a horizontal line is photographed in a range of 46 degrees on both sides of the contact point 211 (total of 92 degrees) and 60 degrees on both sides including the lattice area 29 (total of 120 degrees).

  Further, since the camera 23 has an angle of view looking up the wheel 20 from below, the wheel 20 that is essentially a perfect circle is photographed with distortion by contracting in the vertical direction. Note that the camera 23 is a digital camera capable of shooting with a short exposure of 20 nanoseconds or less. The illuminating device 22 needs a strong light quantity that allows the camera 23 to sufficiently shoot even with exposure of 20 nanoseconds or less, and can use a strobe-controlled LED.

  Returning to FIG. 4, other configurations will be described in detail. The wheel detection sensor 21 can be composed of, for example, a magnetic sensor attached to the track 4 or a photoelectric sensor installed in the vicinity of the track 4, and detects that the wheel 20 has reached a predetermined position with a delay of several tens of nanoseconds. And outputs a synchronization signal. Since the moving wheel 20 is photographed by the camera 23, a “flow” occurs in the image. The shutter speed is sufficient so that the size of the “flow” is less than half the width of the feature point to be detected. Adjust at high speed.

  Here, the configuration of other devices of the image acquisition device 16 will be described together with the operation of the image acquisition device 16. First, the wheel detection sensor 21 detects that the wheel 20 has come to a predetermined position, and outputs a synchronization signal to the camera 23, the illumination device 22, and the wheel counting device 24. The camera 23 photographs the wheel 20 in synchronization with the synchronization signal, and the photographed wheel image is stored in the image storage device 27 together with the photographing time. The illumination device 22 emits light in synchronization with the synchronization signal and illuminates a part of the wheel 20.

  The wheel counting device 24 can know the number of wheels of the vehicle 6 by counting the number of rising times of the synchronization signal of the wheel detection sensor 21 and outputs this order to the image storage device 27 as a wheel number. .

  The image storage device 27 associates and stores the wheel ID, the wheel image, and the passage time, with the wheel number assigned with the unit number of the inspection unit 2 described above as a wheel ID. The unit number of the inspection unit 2 can specify which wheel of the vehicle 6 was shot from where.

  Further, the wheel counting device 24 counts the number of rises of the synchronization signal to a predetermined count value, thereby knowing the timing when the passing of one train of the vehicle 6 is completed, and notifying the image processing device 26 of this. The count value is reset to prepare for the arrival of a new vehicle. The predetermined count value is the number of wheels on one side for one train including the vehicle 6. The image processing device 26 processes the wheel image stored in the image storage device 27 after the wheel counting device 24 receives the notification of the completion of passing the train including the vehicle 6.

  Next, before explaining the details of the processing flow of the image processing apparatus 26 in FIG. 4, when performing various analysis processes using the wheel side images that are actually captured, it is particularly considered because of the side inspection. Items to be described will be described with reference to FIGS. Steps S2 to S5 in the processing flow (FIG. 1) of the image processing apparatus 26 described later are processes provided to deal with this matter.

  The first point to be considered is that the degree of wear differs for each wheel as shown in FIG. One side (inner side or outer side) of one wheel (left wheel or right wheel) for one train passes all while rotating on the same plane. However, since the degree of wear differs for each wheel, the radius differs, and each axle passes on a different straight line from the track 4. In FIG. 6, the same plane coincides with the drawing sheet. Although the side surface of the wheel 20a with less wear and the axle 210a and the side surface of the wheel 20b with high wear and the axle 210b are on the same plane, the axles 210a and 210b move from the track 4 at different heights.

  The second point to consider is that the camera is not facing the wheel as shown in FIG. In FIG. 7, the same plane is perpendicular to the paper surface and is represented by a line segment 40. The image acquisition devices 16 to 19 have the cameras 23, but each camera does not face the plane 40, and the distance (distX), the height with respect to the line 4 (heightY), and the camera angle θ vary. is there. For this reason, it is necessary to obtain a facing image equivalent to that captured by the camera v23 virtually facing the wheel 20 by the camera parameter obtained in step S2 of FIG.

  A third point to be considered is that a difference in the horizontal direction also occurs at the time of photographing for each wheel as shown in FIG. When a photoelectric sensor is used as the wheel detection sensor 21, the moment when the wheel 20 blocks the photoelectric sensor beam is the wheel detection timing. 8 represents the beam of the photoelectric sensor. FIG. 8 shows the moment when the wheel 20a with less wear blocks the beam 50, and the lower part of FIG. 8 shows the beam 20b with a shorter radius due to wear. It is the moment of blocking. Comparing the axle positions 210a and 210b of each wheel, it can be seen that the axle 210b of the wheel 20b with advanced wear is shifted to the left.

  The fourth point to be considered is that images are taken with different rotation angles depending on the size of the wheel radius as shown in FIG. In FIG. 9, a beam 50 is a photoelectric sensor beam related to the wheel detection sensor 21 built in the image acquisition device 16, and a beam 51 is a photoelectric sensor beam related to the wheel detection sensor 21 built in the image acquisition device 17. In FIG. 9A, the wheel 20 a that has not been worn out blocks the beam 50 and then rotates 90 degrees to block the beam 51.

  In FIG. 9B, after the worn wheel 20 b blocks the beam 50, it further rotates by more than 90 degrees to block the beam 51. This is because the wheel radius is shortened due to wear, and the perimeter is also shortened accordingly, so that the wheel 50 and the beam 51 rotate more. Since the wheels are imaged by the image acquisition devices 16 and 17 at the moment when the beams 50 and 51 are shielded, it can be seen that the images are captured with different rotation angles depending on the size of the wheel radius.

  As shown above, when inspecting using the captured wheel side image, it is important to make the image face-to-face, match the vertical and horizontal axle positions in the captured image, and correct the rotation angle. It is.

  It is clear that many of these problems are caused by the degree of wear of the wheels, and solutions for matters that did not become a problem in the conventional wheel tread inspection are strongly required in the wheel side inspection.

  For this reason, in the present invention, five coordinate systems are introduced to correct these problems, thereby enabling evaluation on the same screen. Hereinafter, for convenience of explanation, the correlation between coordinates when five coordinate systems are introduced is summarized in FIG. The five coordinate systems are an on-screen image matrix, a reference wheel world coordinate, a face-to-face image matrix, a reference wheel polar coordinate, and a polar coordinate image matrix, which can be mutually converted by a conversion formula.

  The on-screen image matrix [x, y] in FIG. 10, which is the first coordinate system, is a digital image that covers the rectangle 25 in FIG. 5 taken by the camera 23 in FIG. This image can be expressed in matrix, and pixels are specified by the number of rows and the number of columns. Hereinafter, the number of rows and the number of columns are referred to as the coordinates of the on-screen image matrix. The upper left corner of the rectangle 25 in FIG. 5 is the base point [0, 0], and the size per pixel is millimeter / pixel.

  The reference wheel world coordinates (X, Y) of FIG. 10 which is the second coordinate system is the point of contact with the track 4 when the reference wheel 70 blocks the position detection beam 50 in FIG. This is a real coordinate system in which the origin is 71, the X axis is taken along the track 4, and the straight line connecting the contact 71 and the axle 710 of the reference wheel 70 is the Y axis. The unit can be determined in an MKSA unit system such as millimeters.

Here, the reference wheel 70 is the same as the worn actual wheel 20 (the wheel represented by the image matrix on the screen in FIG. 5) in the arrangement of the axle and bolt, but it is guaranteed that there is no wear of the wheel. Is an ideal or virtual wheel. The radius length of the reference wheel 70 is the maximum value of the wheel radius length, and is a known fixed value r _STD . Therefore, the reference axle world coordinate of the axle 710 is always (0, r _STD ).

  The facing image matrix [j, i] in FIG. 10 which is the third coordinate system is a facing digital image after projective transformation with camera parameters described later, which can be expressed in matrix and has the number of rows. The pixel is specified by the number of columns. Hereinafter, the number of rows and the number of columns are referred to as the coordinates of the facing image matrix. A range of a rectangle 90 in FIG. 13 is a face-to-face image matrix, and an upper left corner is a base point [0, 0]. The size per pixel is expressed in millimeters / pixel.

  The reference wheel polar coordinates (t, r) in FIG. 10 which is the fourth coordinate system is a coordinate system taking real values, and the reference wheel world coordinates (X, Y) in FIG. Is expressed in a coordinate system in which the distance r is the vertical axis and the angle t with the Y axis is the horizontal axis. The unit of the r-axis can be an MKSA unit system such as millimeters, and the t-axis can be radians or (degrees).

  The polar coordinate image matrix [q, p] of FIG. 10 which is the fifth coordinate system is a digital image of the reference wheel polar coordinate (t, r), and can be expressed in matrix, with the number of rows and the number of columns. Identify the pixel. Hereinafter, the number of rows and the number of columns are referred to as polar coordinate image matrix coordinates. A rectangle 2000 in FIG. 16B is an example of a polar coordinate image matrix, and the upper left corner is the base point (0, 0). The size per pixel can be radians / pixel or degrees / pixel in the horizontal direction and millimeters / pixel in the vertical direction.

  FIG. 10 shows the conversion relationship between the above five coordinate systems. As will be described in detail later, the conversion from the on-screen image matrix to the reference wheel world coordinates is performed using the equation (8). The inverse transformation is done with the number (5). Conversion from the reference wheel world coordinates to the directly-facing image matrix is possible by the number (10), and the inverse conversion is performed by the number (9). The conversion from the reference wheel world coordinates to the reference wheel polar coordinates is performed by the number (19), and the inverse conversion is performed by the number (20). Conversion from the reference wheel polar coordinates to the polar coordinate image matrix is performed by the number (22), and the inverse conversion is performed by the number (21).

  Next, the processing flow of the image processing apparatus 26 will be described with reference to FIG.

[Step 1: Wheel image input]
First, in step S1, a wheel image is input from the image storage device 27 of FIG.

  Wheel ID is used as the file name of the wheel image. It is assumed that the image storage device 27 also records the image shooting time. The image obtained at this time is the image shown in FIG. 5. For example, in the case of the leading wheel, four images from 16, 17, 18, and 19 are input as a set for the inner side of the right wheel. Therefore, for a set of wheels, the left and right and inside and outside are evaluated, so 16 photographs are input for convenience.

[Step 2: Generation of facing image]
Next, in step S2, the point where the wheel and the camera described with reference to FIG. 7 do not face each other, that is, the problem of camera calibration is taken.

  Here, camera calibration, which is a premise of the processing in step S2, will be described. If the camera calibration is performed at least once when the camera 23 is fixed, then the projective transformation can be performed using the camera parameters obtained here. FIG. 11 shows the relationship between the reference wheel 70 and the on-screen image matrix 25. Here, calibration marks 91 to 94 are attached to the side surface of the reference wheel 70, and the distance in the direction of the track 4 and the distance in the direction perpendicular to the track from the contact point 71 with the track 4 are measured. A suitable unit is millimeter.

  These values are the reference wheel world coordinates in FIG. For the calibration marks 91 to 94, a known structure such as an axle 710 or a bolt in the reference wheel 70 may be used. However, it must be included in the rectangle 25 that is the range of the on-screen image matrix. These calibration marks 91 to 94 can be set to known coordinates (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4) by actual measurement on the reference wheel world coordinates. .

  On the other hand, the positions of the calibration marks 91 to 94 in the on-screen image matrix are expressed by coordinates where the upper left corner of the rectangle 25 in FIG. 5 is (0, 0), and can be obtained on the digital image. These can also be known. Therefore, these are referred to as [cx1, cy1], [cx2, cy2], [cx3, cy3], and [cx4, cy4].

  Here, it is assumed that unknown camera parameters to be obtained are c11, c12, c13, c21, c22, c23, c31, and c32. However, although the reference wheel world coordinate is a real number, the on-screen image matrix is an integer value, so it does not exactly match, but the integer value closest to the corresponding real value is selected. In this way, the relationship of number (1) is established between [x, y] of the on-screen image matrix and (X, Y) of the reference wheel world coordinates.

  Here, when H is deleted and arranged, the number (2) is obtained.

  When this is expressed in a matrix, the number is (3).

  The numbers (3) (X, Y) and [x, y] should be established by substituting the reference wheel world coordinates of the calibration marks 91 to 94 and the coordinates of the image matrix on the screen. Get the number (4).

  Camera parameters c11, c12, c13, c21, c22, c23, c31, c32 can be obtained by multiplying both sides by the inverse matrix of the 8 × 8 matrix on the left side of the number (4).

  When the calibration mark is set to 5 or more, the matrix on the left side of the number (4) has more rows than columns. Even in this case, by multiplying both sides by the transposed matrix of the matrix on the left side, the matrix on the left side can be made into an 8 × 8 matrix, and the camera parameters c11, c12, c13, c21, c22, c23, c31, and c32 can be solved.

  If the camera parameters are known by the above-described method, the reference wheel world coordinates (X, Y) are converted from the reference wheel world coordinates (X, Y) to the on-screen image matrix [x, y] by the number (5) obtained by summarizing the number (2) for x and y Can be converted.

  Further, when the number (5) is summarized for the reference wheel world coordinates, the number (6) is obtained.

  When this is expressed in a matrix expression, the number is (7).

  The number (7) is transformed to obtain the number (8).

  -1 on the right side of the right side matrix of the number (8) represents an inverse matrix. The inverse matrix has various solutions, including Gaussian sweeping, and can be easily obtained. Conversion from the on-screen image matrix [x, y] to the reference wheel world coordinates (X, Y) can be performed by the equation (8).

  Here, the reference wheel 70 is used as the subject for camera calibration, but a plate with a grid-like scale on the plane 40 where the side surfaces of the reference wheel 70 and the wheel 20 exist may be used as the subject.

  Subsequently, with reference to FIG. 13, a facing image matrix is defined in detail as a digital array of facing images. The facing image matrix is represented as [j, i]. A rectangle 25 in FIG. 11 is an on-screen image matrix before facing, and a rectangle 90 in FIG. 13 is a directly-facing facing image matrix, and the upper left corner is a base point [0, 0]. Here, the reference wheel world coordinates of the four corners are respectively (−W / 2, H0) for the upper left A point, (W / 2, H0) for the upper right B point, and (W / 2, H0 + H) for the lower left C point. ), The lower right D point is (−W / 2, H0 + H). W, H0, and H are real values of existing values.

  Furthermore, dx and dy are defined such that dx = W / M representing the horizontal size of one pixel of the directly-facing image matrix [j, i], and dy = H / N representing the vertical size. M and N are natural numbers representing the number of pixels in the horizontal and vertical directions. Both units are millimeters / pixel. In this case, the facing image matrix is a matrix of N rows and M columns, and the facing image matrix at the four corners is [0, 0] for point A, [M-1, 0] for point B, and [0 for point C, respectively. , N−1] and point D are [M−1, N−1].

  From the above, the reference wheel world coordinates (X, Y) can be expressed by the number (9), and conversion from the directly-facing image matrix [j, i] to the reference wheel world coordinates (X, Y) is possible. Become.

  Further, the number (9) obtained by solving the number (9) for j and i enables the conversion from the reference wheel world coordinates (X, Y) to the directly-facing image matrix [j, i].

  In the number (10), j and i are generally not integers, but an integer closest to the value on the left side is selected. The points of the directly-facing image matrix [j, i] are converted into the points of the reference wheel world coordinates (X, Y) by the equation (9). Subsequently, the points of the reference wheel world coordinates (X, Y) are converted into the points of the on-screen image matrix [x, y] by the number (5). The value obtained by the number (5) is generally not an integer, but the nearest integer is selected as [x, y].

  Since the on-screen image matrix [x, y] is a photographed digital image before facing, the brightness of the corresponding point is assigned as the brightness of the facing image matrix [j, i]. The above processing is performed over the entire area of the directly-facing image matrix [j, i].

  That is, it is performed in the range of j = 0 to M−1 and over the range of i = 0 to N−1. If the corresponding point is outside the rectangle 25 that is the area of the on-screen image matrix [x, y], a fixed value such as zero or 255 is assigned. As a result, the directly-facing image matrix [j, i] is a partial image of the facing wheel.

  The above is performed in step S2 of the processing flow of FIG. That is, in step S2, camera parameters are calculated by performing camera calibration on the reference wheel 70, the on-screen image matrix of the reference wheel 70 or the wheel 20 is converted into reference wheel world coordinates using the camera parameter, and the on-screen image is calculated. A facing image matrix is generated from the on-screen image matrix captured by the camera 23 using the matrix, the relationship between the reference wheel world coordinates and the facing image matrix.

[Step 3: Axle position detection]
Next, the axle coordinate detection process in step S3 will be described with reference to FIG.

FIG. 14 is a diagram illustrating a method of calculating the reference wheel world coordinates (X _c , Y _c ) and the directly-facing image matrix [x _c , y _c ] of the axle 210 of the wheel 20. Since the wheel 20 expressed by the reference wheel world coordinates and the facing image matrix is a perfect circle, the center of the wheel, that is, the position of the axle can be obtained by regarding the contour of the wheel as a part of the arc.

  As described above, the facing image matrix [j. i] is a digital image of the area of rectangle 90 in FIG. An image obtained by applying a contour enhancement process such as a Sobel filter to the digital image and binarizing it with a predetermined threshold is shown as a rectangle 1000. Here, the larger edge is shown in black as the contour edge. A horizontal line segment 1001 appears as a part of the outline of the line 4. Since this always appears at the same place by fixing the camera 23, it can be excluded from the processing target in advance.

  On the other hand, the contour appearing in the range 1002 of the rectangle 1000 is the contour of the wheel 20 and a part of the arc. Here, let arbitrary two points on the outline which are a part of a circle be E and F, respectively. The coordinate value can be known from a rectangle 1000 which is a binarized image.

However, these are the facing image matrix [j. i], it can be converted into reference wheel world coordinates by the number (9). _Let E (e _x , e _y ) and F (f _x , f _y ) be the coordinates of E and F as the converted reference wheel world coordinates. A vertical bisector H of E and F passes through the axle 210. The vertical bisector H is represented by the number (11) when an arbitrary point on the bisector H is (x, y).

  Here, the parameter of the number (11) is replaced with α, β, γ to obtain the number (12). However, α, β, and γ are parameters determined by the coordinate system of E and F.

Here, an arbitrary two points on the contour of the wheel 20 in the range 1002 are k ₀ sets (k ₀ is a natural number), and these vertical bisectors are drawn k ₀ . For this purpose, the subscript k is added and the number (12) is rewritten to the number (13). However, k = 1, 2,. . . , K ₀ .

Since all the k ₀ bisectors intersect at the axle 210, the solution of the simultaneous equation of the number (13) becomes the reference wheel world coordinates of the axle 210. In order to solve the simultaneous equations, the number (13) is set as the number (14) of the matrix expression.

  Further, for simplification, the number is represented by the matrix U (15).

  The both sides of the number (15) are multiplied by the transpose matrix of U to obtain the number (16).

U ^T U is a 2 × 2 matrix, and usually an inverse matrix exists and can be solved for (x, y). This is the reference wheel world coordinates (X _c , Y _c ) of the axle 210. To solve the number (16), both sides may be multiplied by an inverse matrix, or Gaussian sweeping may be used.

  If the number of the perpendicular bisectors is three or more, an error occurs in the detected position of the contour of the wheel 20, so that they do not intersect at one point on the axle 210. However, the vehicle reference wheel world coordinates of the axle 210 are obtained as the optimum solution with the least square error by the above method.

Note that, when the reference wheel world coordinates (X _c , Y _c ) are converted into a face-to-face image matrix by the number (10), a negative value can be obtained, but this is because the axle 210 is not within the range of the rectangle 90 and the rectangle 1000. This shows no problem.

The above is step S3 of the processing flow. Here, the reference wheel world coordinates are substituted into the numbers (11) to (16), but the coordinates of the facing image matrix may be substituted instead. In this case, the coordinates are calculated as real numbers, and the solution of the number (16) is the coordinates [j _c , i _c ] in the facing image matrix of the axle 210. When this is converted by the number (9), the reference wheel world coordinates, that is, (X _c , Y _c ) are obtained.

[Step 4: Shift correction]
Next, in step S4, shift correction is performed for the correction to the reference wheel and the radial length of the wheel.

When the axle is obtained in the process of step S3, for the reference wheel 70, the reference wheel world coordinates of the axle 710 are (X _STD , Y _STD ), and the coordinates of the facing image matrix are [j _STD , i _STD ]. Can be written.

On the other hand, for the wheel 20, the reference wheel world coordinates of the axle 210 are (X _c , Y _c ), and the coordinates of the facing image matrix are [j _c , i _c ]. Due to the wear described above, (X _STD , Y _STD ) and (X _c , Y _c ) and [j _STD , i _STD ] and [j _c , i _c ] generally do not match.

  However, since a reference wheel polar coordinate conversion table, which will be described later, is created in conformity with the reference wheel 70, when the wheel 20 is used, it is necessary to match the position of the axle by translation.

  The relationship when the coordinates of the front image matrix before and after the correction are [j, i] and [j, i '], respectively, is expressed by the number (17). The process of converting [j, i] to [j, i '] by the equation (17) is step S4.

After this processing, the coordinates of the axles of the facing image matrix of all the wheels are [j _STD , i _STD ]. Further, when this is converted into the reference wheel world coordinates by the number (9), (X _STD , Y _STD ) is obtained.

Here, it is also calculated that the radius of the wheel 20 is Y _c millimeter and the amount of wear from the reference wheel 70 is Y _STD −Y _c millimeter. Further, the relationship with r _STD that has been described above is r _STD = Y _STD .

  The inverse transformation of number (17) is number (18). Here, if the pixel brightness in the facing image matrix corresponding to each pixel is assigned with reference to the entire standardized facing image matrix, the standardized facing image matrix of the wheel 20 is obtained.

[Step 5: Create polar coordinate image]
Next, in step S5, a reference wheel polar coordinate conversion table is created. With reference to FIG. 15, the reference wheel polar coordinate image creation in step 5 will be described. FIG. 15A shows a rectangle 90 of a facing image matrix. FIG. 15B is a reference wheel polar coordinate image obtained by converting the facing image matrix.

  The image processing apparatus 26 has a reference wheel polar coordinate conversion table in advance, and first, generation of the reference wheel polar coordinate conversion table will be described. The reference wheel polar coordinate (t, r) shown in FIG. 15B is a coordinate system taking real values, and the distance r from the axle 710 of the reference wheel 70 in FIG. An abscissa is a declination t from a line segment (that is, the Y axis of the reference wheel world coordinates) connecting the contact point 71 of the track 4 (that is, the origin of the reference wheel world coordinates) and the axle 710. The unit of r-axis is MKSA unit system such as millimeters, and t-axis is radians or degrees.

  In the case where four cameras have a field of view of about 120 degrees at a pitch of about 90 degrees as in this embodiment, the range of the horizontal axis is -π / 3 radians to π / 3 radians, that is, -60 degrees to 60 degrees. It is appropriate that The range of the vertical axis is from Ro to Rw including the contact point 70 (Ro and Rw are real numbers). Here, the reference wheel world coordinates (X, Y) are converted from the reference wheel world coordinates (t, r) by the number (19).

  Also, the reference wheel polar coordinate (t, r) is converted from the reference wheel world coordinate (X, Y) by the number (20).

  Next, a polar coordinate image matrix [q, p] as shown in FIG. 16B is defined as a digital array of reference wheel polar coordinate images. The polar coordinate image matrix [q, p] is a digital image based on the reference wheel polar coordinate system and can be expressed in a matrix, and the pixels are specified by the number of rows and the number of columns. Hereinafter, the number of rows and the number of columns are called coordinates. The polar coordinate image matrix [q, p] is a coordinate indicated by taking the upper left point of the rectangle 90 of the captured image of FIG.

  Here, dq = 2d / (3T) representing the size in the horizontal direction of one pixel of the polar coordinate image matrix [q, p], dq = dp = (Rw−Ro) / R representing the size in the vertical direction. (T and R are natural numbers). The unit is “radian / pixel” in the horizontal direction and “millimeter / pixel” in the vertical direction.

  The polar coordinate image matrix [q, p] in FIG. 16B has the upper left corner of the rectangle 2000 as the base point [0, 0], and has the luminance value of the (q−1) th in the horizontal direction and the (p−1) th in the vertical direction. The polar coordinate image matrix [q, p] is a matrix of R rows and T columns, and the coordinates of the polar coordinate image matrix at the four corners of the rectangle 2000 are O [0, 0], P [T-1, 0], Q, respectively. [0, R-1], R [T-1, R-1]. Conversion from the polar coordinate image matrix [q, p] to the wheel reference wheel polar coordinate (t, r) is possible by the number (21).

  Also, the wheel reference wheel polar coordinates (t, r) can be converted into the polar coordinate image matrix [q, p] by the number (22).

  Further, as is apparent with reference to the diagram showing the relationship between the case coordinates in FIG. 10, by converting in the order of the number (21) and the number (20), the rectangular shape of the polar coordinate image matrix [q, p] is obtained. 2000 pixels are converted to reference wheel world coordinates (X, Y).

  Furthermore, since the reference wheel world coordinates (X, Y) can be converted from the reference wheel world coordinates (X, Y) to the facing image matrix [j, i] by the number (10), the pixels of the polar coordinate image matrix [q, p] are converted to the facing image matrix [j, i]. Can be associated. Since the directly-facing image matrix [j, i] is the facing digital image generated in step S2, the luminance of the corresponding pixel is assigned to each pixel of the polar image matrix.

  By performing the above processing over the entire area of the rectangle 2000 in FIG. 16B, the correspondence between each pixel in the polar coordinate image matrix and the directly-facing image matrix is determined. By making this correspondence into a table, calculation by the number (21), the number (20), and the number (10) can be omitted thereafter. Since the coordinates [q, p] of the polar coordinate image matrix and the coordinates [x, y] of the on-screen image matrix must be integers, the integer closest to the calculation result of the number (19) is selected.

  As a result, the face-to-face image matrix [j, i] corresponding to the polar coordinate image matrix [q, p] and the original image, that is, the on-screen image matrix [x, y] are the rectangle 90 and the rectangle 25 in the shooting range. If it is out of the range, a fixed luminance value such as zero or 255 is assigned.

  By the way, the reference wheel polar coordinate conversion table obtained above is a reference wheel polar coordinate conversion table that is converted on the basis of the axle 710 of the reference wheel 70. The wheel 20 needs to be corrected. For this purpose, the brightness value of the standardized facing image matrix created in step S4 is referred to instead of the facing image matrix.

  The wheel image divided and photographed by the above processing is converted into a polar coordinate image matrix obtained by converting the reference wheel polar coordinate image into a digital image. For example, feature points 121 and 122 existing on concentric circles centering on the axle 210 in FIG. 16A are converted into horizontal feature points 121 and 122, respectively, as shown in FIG. 16B. Bolts 33, 34, and 35 are also converted. The feature points 121 and 122 are scratches on the wheel 20, for example. Further, the contour of the wheel 20 is converted into a horizontal line segment 123. The above is the process of step S5 in the process flow.

[Step 6: Bolt detection]
Next, the bolt detection and feature point detection in steps S6 and S7 will be described. First, the bolt detection process in step 6 will be described with reference to FIG.

  Since the polar coordinate image matrix is a digital image, it can be detected by pattern matching using normalized correlation. The bolt shape is known, and the shape of the bolt after the reference wheel polar coordinate conversion is also known, and this can be registered as a template as shown in FIG.

  FIG. 17B is an example of a polar coordinate image matrix, which shows bolts 33, 34, and 35 that have undergone reference wheel polar coordinate conversion. Although the phase of the wheel, that is, the rotation angle varies depending on the timing of photographing, the bolts 33, 34, and 35 move at the same distance from the axle, and therefore move on the horizontal line after the reference wheel polar coordinate conversion.

  Therefore, if the template shown in FIG. 17A is searched in the vicinity of the horizontal line, the bolts 33, 34, and 35 can be detected. However, since there is no difference in shape between the bolts, the bolt on the wheel cannot be specified. For example, when there are k bolts (k is a natural number) on the wheel, k is specified with ambiguity. Here, only a specific bolt has a different shape from the others, and when this is detected as a template, the bolt can be specified, so the rotation angle of the wheel is also uniquely specified. The above is the bolt detection process of step S6.

[Step 7: Feature Point Detection]
Next, the feature point detection processing in step S7 will be described with reference to FIG.

  FIG. 17B is an example of a reference wheel polar coordinate matrix of the wheel 20 having the feature points 131. For regions other than the bolt region detected in step S6, the polar coordinate image matrix is binarized with a predetermined threshold value after the Sobel filter processing, and the presence / absence of a feature point and the position of the end point if there is a feature point Can be detected.

At the same time, the displacement in the horizontal direction from the position (q _b , p _b ) of the bolt closest to the feature point 131, in the example of FIG. 17B, is calculated, and the bolt position and the displacement from the bolt are characterized. The position information of the point 131 is used. Thus, the position of the feature point 131 on the wheel is specified by the positional relationship with the nearest bolt. Further, the feature point 131 can grasp the start point and end point position of the feature point as shown in FIG.

  FIG. 18 is a diagram illustrating the position specification and ambiguity of the feature point when the number of bolts is eight. If the shape of the eight bolts is the same and the bolt 35 cannot be identified as one of the bolts in FIG. 18, the feature point 131 is recorded with eight ambiguities at positions 132 to 139.

  However, as described above, if any one of the eight bolts has a special shape and the position of the special bolt can be specified, the position of the feature point can be uniquely determined from the order of the other bolt positions. Further, it is possible to eliminate the ambiguity of the position-specific difference by imprinting a predetermined mark on a predetermined area of the wheel, detecting the mark, and using this as a reference. The above is the feature point detection processing in step S7.

[Step 8: Output]
In step S8, the image processing apparatus 26 in FIG. 2 outputs the various images and data obtained up to step S7 to the database 8.

  The image and data output to the database 8 are the wheel image, the wheel ID, the passing time, the facing image generated in step S2, the axle position obtained in step S3, and the axle position and the origin of the reference wheel world coordinates. The length of the wheel radius that can be calculated from the difference, the polar coordinate image matrix generated in step S5, the bolt position determined in step S6, and the position and length of the feature point determined in step S7. The above is a series of processing flows in the image processing apparatus 26 shown in FIG.

  The database 8 collects images and measured data from the image acquisition devices 16 to 19 for each wheel ID, and stores the passage time as an index. If the passing time is linked to the composition number via the train operation table, it is possible to uniquely identify the composition number, the wheel ID, and the index composed of the inspection unit number.

Next, with reference to FIGS. 17B and 17C, the feature point pattern matching processing performed in the polar coordinate image matrix will be described. FIG. 17C shows the luminance pattern of the feature points cut out from the polar coordinate image matrix stored in the database 8, which is the template pattern 130. In the past, the image of the feature point detected in step S7 is output in step S8, and the index is managed and stored in the database 8. Later, when the same wheel is inspected, it is read from the database 8 and used as the template pattern 130. However, the feature point image as a template can be cut out and managed by an operator manually using the display device and the pointing device of the user interface 12.

  The template pattern 130 is a circumscribed rectangle of the feature point, and is represented by Tmp [q, p], where the horizontal direction is the t axis, the vertical direction is the r axis, and the upper left corner of the circumscribed rectangle is the base point (0, 0). At this time, the height of the template, that is, the number of pixels in the vertical direction is h, and the width of the template, that is, the number of pixels in the horizontal direction is w. On the other hand, it is assumed that the luminance pattern of the polar coordinate image matrix of FIG. 13B is represented by Trg [q, p]. At this time, the similarity Sim [q, p] by pattern matching based on the normalized correlation between them is expressed by the number (23).

If the shape of the template pattern 130 generated from the polar coordinate image matrix acquired in the past and the feature point 131 on the polar coordinate image matrix newly acquired at a later date are not extremely changed, both are searched by translation to the q axis and the p axis. The corresponding position can be detected only by performing the operation. In addition, although q changes due to the difference in the rotation angle of the wheel at the start of photographing, the distance from the axle is not changed by the correction process in Step 4 described above, so the template pattern 130 is also acquired during the pattern matching process. The coordinates [q ₁₃₀ , p ₁₃₀ ] of the polar coordinate image matrix may be stored together, p ₁₃₀ may be fixed, and the similarity Sim [q, p ₁₃₀ ] may be searched only in the q-axis direction.

  Thereby, it is specified whether the feature points on the polar coordinate image matrix acquired in the past and the feature points on the newly acquired polar coordinate image matrix are the same feature points on the wheel 20, and the length of the feature point is determined. Now, observe the aging of the shape. The similarity of the shape of the feature point is expressed by the similarity in the line segment direction of each portion obtained by dividing the feature point into several line segments. These pattern matching processes may be performed by the image processing apparatus 26, but may be performed by an information processing apparatus built in the user interface 12 as an offline process.

  The panorama forming apparatus 10 of FIG. 3 forms a panoramic polar coordinate image matrix for the entire circumference of the wheel from the polar coordinate image matrices 151 to 154 output from the image acquisition devices 16 to 19. FIG. 19 is a diagram illustrating the formation of a panoramic polar coordinate image matrix of the reference wheel 70. Since the cameras 25 incorporated one by one in the image acquisition devices 16 to 19 in FIG. 5A are installed so as to be photographed every time the reference wheel 70 is rotated 90 degrees, each of the polar coordinate image matrices 151 to 154 is ± When the 45-degree range is joined in the horizontal direction, a panoramic polar image matrix shown in FIG. 19B is obtained.

  FIG. 20 is a diagram showing panorama formation of a polar coordinate image matrix related to the wheel 20. Since the wheels 20 rotate the photographing timing of the four cameras 25 at an L-degree pitch, ± L / 2 degrees from the center of the polar coordinate image matrices 161 to 164 obtained by the image acquisition devices 16 to 19 at an L-degree pitch. Join. Here, L is obtained by the number (24).

In number (24), Y _STD is a known value for the radius of the reference wheel. _Yc is the radius of the wheel 20 and is calculated in the process of step S4. Note that the radius of the wheel 20 is independently measured four times by performing the processing of step S4 in each of the image acquisition devices 16 to 19, and the average value or median value of these values is expressed by a number (24). Y _c can be used. Further, the rotation angle between the cameras can measure the movement of a periodic structure such as a bolt by the above-described pattern matching without using the radius of the wheel 20.

  When shooting with four cameras as in the present embodiment, when the bolts are arranged in multiples of 4, that is, 4, 8, 12, 16, and the pieces are equally spaced, when rotating at a pitch of 90 degrees Since there is a bolt at the same position in each captured image, the displacement angle from 90 degrees can be calculated by detecting the displacement of the bolt position by pattern matching.

Here, a more general panorama forming method will be described. The perimeter of the wheel with the longest perimeter (reference wheel in the previous embodiment) is L _max , the perimeter of the shortest perimeter is L _min, and the number of cameras used for imaging is N _cam. (The left example is 4 in the embodiment). In this case, N _cam cameras are arranged beside the track at equal intervals of L _max / N _cam , and each camera is ± (180 / N _cam ) × (L _max / L _min with the contact point of the wheel and the track as the center. ) Degree range, that is, a range of (360 / N _cam ) × (L _max / L _min ) degree needs to be installed.

With this setting, it is possible to shoot with the N _cam camera without omission, with the smallest radius wheel and the smallest radius wheel, and the least overlap. Further, in order to integrate these into a panorama, a range of ± (180 / N _cam ) × (L _max / L) is joined to each polar coordinate image matrix at a pitch of (360 / N _cam ) × (L _max / L). To do. Here, L is the perimeter of the wheel 20.

  The panorama forming apparatus 10 stores an image in which a polar coordinate image matrix is panorama formed in the database 8 under the index management.

  With reference to FIG. 21, a process of displaying a panoramic image related to the wheels in the display function of the user interface device 12 will be described. FIG. 21A shows a polar coordinate image matrix generated by the panorama forming apparatus 10 and stored in the database 8 and formed on the virtual memory related to the display function of the user interface apparatus 12. is there.

  By writing all or part of the image developed in the virtual memory into the display memory related to the display function of the user interface 12, all or part of the polar coordinate image matrix around the panoramic wheel is displayed on the display function. The Since the number of pixels that can be displayed by the display function is limited, when displaying the entire polar coordinate image matrix, it is necessary to reduce the display resolution by thinning out the display pixels.

In the case of partial display, a desired part can be displayed by controlling the display parameter t _DISP 1 shown in FIG. 21A by using a pointing device such as a mouse or a keyboard of the user interface device 12. . Here, the display parameter t _DISP 1 indicates the number of offset pixels from the left end base point of the panoramic polar image matrix, and pixels belonging to the range of the rectangle 1700 specified by the number of offset pixels are related to the display function. FIG. 21B shows an example in which a part of the polar coordinate image matrix is displayed in this manner written in the display memory. If other parameters are manipulated to change the number of vertical and horizontal pixels of the rectangle 1700, in FIG. 21B, polar coordinate image matrices in different ranges can be displayed at different resolutions.

As shown in FIG. 10, when the conversion is performed in the order of the numbers (9), (19), and (22), the polar coordinate image matrix corresponding to the pixel [j, i ′] of any standardized facing image matrix [Q, p] can be obtained. For the panoramic polar image matrix, [q + 3 × q _offset , p], [q + 2 × q _offset , p], [q + q _offset , p], [q] for the image acquisition devices 16, 17, 18, and 19, respectively. , P] and a predetermined offset q _offset can be obtained.

Here, q _offset is calculated by (number of pixels for 90 degree polar coordinate image matrix related to reference wheel 70) × (radius length of reference wheel 70) / (length of radius of wheel related to photographing). .

The relationship between the pixel [j, i ′] of the normalized face-to-face image matrix and the pixel [q, p] of the panoramic polar coordinate image matrix is tabulated, for example, the display parameter t _{DISP in} FIG. As shown in FIG. 21 (c), by accessing a predetermined pixel of the polar coordinate image matrix which is panorama formed with a reference as 2 and writing it in the display memory based on the conversion table, The whole can be displayed on the display function of the user interface 12.

  Since the vicinity of the axle is displayed over a wide angle, a region indicated by approximately 1701 broken rectangle is converted as a reference image matrix and displayed as FIG. Thus, as shown in FIG. 21C, the state of an arbitrary wheel position can be displayed as a wheel image.

In FIG. 21A, the horizontal direction of the polar coordinate image matrix of the entire wheel formed in the panorama represents an angle of 360 degrees, and the right end of the right end should be connected to the left end. Therefore, when the display parameter t _DISP 1 and the display parameter t _DISP 2 are increased and the rectangle 1700 and the dashed rectangle 1701 protrude from the right end as shown in FIG. 22A, the corresponding portion at the left end is accessed for display or display. By displaying after reverse conversion, any part of the wheel can be displayed as shown in FIGS. 22 (b) and 22 (c).

  Based on the axle position detected in step 3, conversion into a standardized facing image matrix is performed in step 4. With reference to FIG. 23, the importance of the conversion into the normalized face-to-face image matrix will be described.

  FIG. 23 left shows an example in which the reference wheel 70 is converted into a polar coordinate image matrix using the reference wheel polar coordinate conversion table 2020 created in step 5. A rectangle 90 represents the range of the facing image matrix.

  The center of FIG. 23 shows an example in which the worn wheel 20 is converted into a polar coordinate image matrix using the reference wheel polar coordinate conversion table 2020 without going through the process of step 4. Since there is a difference between the axle position 710b assumed in the reference wheel polar coordinate conversion table 2020 and the axle 210, the polar coordinate image matrix is distorted.

  The right side of FIG. 23 shows an example in which the worn wheel 20 is converted into a standardized facing image matrix in the process of step 4 and then converted into a polar coordinate image matrix using the reference wheel polar coordinate conversion table 2020. Since the assumed axle position 710b matches the axle 210, a polar coordinate image matrix without distortion is obtained.

  However, areas 2021 and 2022 indicated by dot patterns are areas that are out of the field of view of the camera 23 and are not included in the on-screen image matrix that is the original image. A region 2021 generated by a vertical shift caused by wear of the axle is a region related to the rail 4, and there is no problem in the vehicle inspection device of the present invention.

  However, the region 2022 caused by the horizontal shift due to the delay of the wheel detection timing is problematic because it may include a wheel side region. For this reason, the delay of the wheel detection timing should be kept small.

The present invention is useful as a vehicle inspection device because it captures train wheel side images and tracks the secular change of the flaws and allows browsing of past data stored in a database.

DESCRIPTION OF SYMBOLS 1 ... Vehicle inspection apparatus 2 ... Inspection unit 3, 4 ... Track 6 ... Vehicle 20 ... Wheel 210 ... Axle 16, 17, 18, 19 ... of wheel 20 ... Imaging process Means 8 ... Database 10 ... Panorama forming means 12 ... User interface device 20 ... Wheel 21 ... Wheel detection sensor 22 ... Illumination device 23 ... Camera 24 ... Wheel counting device 25 ... Rectangle 26 that is the range of the image matrix on the screen ... Image processing device 27 ... Image storage device 28 ... 90 degree field of view 29 ... 120 degree field of view 30-37 The bolts 40 of the wheels 20... Planes 50, 51 through which the wheel side faces pass. The photoelectric beam 70 related to the wheel detection sensor 21... The reference wheel 710. a, 93a, 94a ... points 91b, 92b, 93b, 94b on the on-screen image matrix of the calibration mark ... points 71 on the reference wheel world coordinates of the calibration mark ... with the reference wheel 70 Contact point 90 of the line 4 ... rectangle 710b indicating the range of the front image matrix ... axle position 2020 assumed in the reference wheel polar coordinate conversion table 2020 ... reference wheel polar coordinate conversion table 2020 created in step 5
2021 ... Missing image information caused by wheel wear 2022 ... Missing image information caused by delay of wheel detection timing 1000 ... Binarized image 1001 ... Part of contour 4 of line 4 1002 .. Ranges 121 and 122 where wheel contours exist. Feature points 130 existing on concentric circles. Template pattern 2000 of feature points. Rectangles 132 to 139 indicating ranges of polar coordinate image matrix. Upper characteristic points 151 to 154 ... Polar coordinate image matrix 161 to 164 of reference wheel 70 ... Polar coordinate image matrix 1700 of wheel 20 ... Range 1701 to be displayed by display function of user interface device 12 ... User interface Range displayed by the display function of the device 12

Claims (13)

An image acquisition device that acquires a wheel image of a wheel side surface of a vehicle that is installed along a track and passes through the track, an image storage device that stores an image acquired by the image acquisition device, and an image processing device that performs image processing And
The image storage device holds in advance a facing image of the reference wheel without wear,
The image processing device generates a facing image that faces the wheel by performing projective transformation on the wheel image acquired by the image acquisition device,
Estimating the wheel axle position for the facing image;
The estimated axle position is corrected to an image matched with the axle position of the facing image of the reference wheel without wear held in the image storage means and stored in the image storage device. Vehicle inspection device. The vehicle inspection apparatus according to claim 1,
The image processing device generates a polar coordinate image with respect to the wheel image after matching the axle position, generates a polar coordinate image with the axle position as an origin, and stores the polar coordinate image in the image storage device. A vehicle inspection device. The vehicle inspection apparatus according to claim 2,
The image storage device holds in advance a template image on a polar coordinate image of a wheel bolt,
The vehicle image inspection apparatus, wherein the image processing apparatus determines a bolt position by template matching for the polar coordinate image obtained by performing polar coordinate conversion with an axle position as an origin. The vehicle inspection apparatus according to claim 2,
The vehicle inspection apparatus, wherein the image processing apparatus performs a process of detecting the presence or absence of a defect on the polar coordinate image and stores the defect in the image storage apparatus. The vehicle inspection apparatus according to claim 3,
The image processing device performs processing for detecting the presence or absence of a defect in the polar coordinate image, and stores the defect position in the image storage device in relation to a bolt position determined by template matching. Inspection device. An image acquisition device that acquires a plurality of wheel images over the entire circumference of a wheel side surface of a vehicle that is installed along the track and passes through the track, and an image storage device that stores an image acquired by the image acquisition device And an image processing device that performs image processing,
The image storage means holds in advance a facing image of the reference wheel without wear,
The image processing device generates a plurality of facing images that face the wheels by performing projective transformation on each of the plurality of wheel images over the entire circumference of the wheel side surface,
Estimating the wheel axle position for each of the plurality of facing images,
The estimated axle position is corrected to an image matched with the axle position of the facing image of the reference wheel without wear held in the image storage means and stored in the image storage device. Vehicle inspection device. The vehicle inspection apparatus according to claim 6, wherein
The image processing device generates a plurality of polar coordinate images by performing polar coordinate conversion with the axle position as an origin for the plurality of wheel images after matching the axle position.
A vehicle inspection apparatus that connects the plurality of polar coordinate images to create a polar coordinate image over the entire circumference of the wheel image and stores the polar image in the image storage device. The vehicle inspection apparatus according to claim 7, further comprising display means.
A vehicle inspection apparatus that displays a partial rectangular region of a polar coordinate image over the entire circumference of the connected wheel image on a display means. The vehicle inspection apparatus according to claim 7, further comprising display means.
A vehicle inspection apparatus, wherein a part of a trapezoidal region of a polar coordinate image over the entire circumference of the connected wheel image is extracted and displayed on a display means in a wheel shape. The vehicle inspection apparatus according to claim 7,
The image storage means holds in advance a template image on a polar coordinate image of a wheel bolt,
The image processing apparatus determines a bolt position by template matching for the polar coordinate image obtained by performing polar coordinate conversion with an axle position as an origin,
A vehicle inspection apparatus characterized in that a plurality of polar coordinate images are connected from a bolt position interval to obtain a polar coordinate image over the entire circumference of a wheel image. The vehicle inspection apparatus according to claim 7,
The image processing apparatus obtains a radius or a diameter of a wheel, obtains a rotation angle of the image acquisition apparatus that rotates between two or more cameras based on this value, and acquires two or more images based on the rotation angle. A vehicle inspection apparatus characterized in that polar coordinate conversion images generated by the apparatus are joined to generate a polar coordinate conversion image for the entire circumference of the wheel. The vehicle inspection apparatus according to claim 7,
The image processing device obtains a rotation angle that rotates between two or more cameras of the image acquisition device by measuring a movement amount of a wheel structure by pattern matching processing, and based on the rotation angle, A vehicle inspection apparatus characterized in that polar coordinate conversion images generated by two or more image acquisition devices are joined to generate a polar coordinate conversion image for the entire circumference of a wheel. The image inspection apparatus according to claim 6,
A vehicle inspection apparatus that determines the amount of wear of a wheel based on a difference between a radius of the wheel and a radius of a reference wheel.

Images