JP7237671B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing device and an image processing method.

As an inspection device for inspecting damage to the wheel tread of a vehicle, an inspection device that detects the presence or absence of damage by analyzing images of the wheel tread of a running vehicle taken by a camera is generally known. there is

In Patent Document 1, an inspection device that detects the position of a wheel traveling on a rail, illuminates the wheel at the timing when the wheel passes, and inspects damage to the wheel tread based on an image of the wheel tread of the vehicle. disclosed. In this inspection device, by continuously photographing the wheels of the running vehicle by the image pickup means, the whole circumference image of the wheel tread is captured and the presence or absence of damage is calculated by image processing.

In the image processing, first, the inspection device designates the area of the wheel tread by external input, and obtains the edge from the density difference between the pixels of the damaged area and the undamaged area on the designated area. . Next, the above-described inspection apparatus detects an area having a size equal to or larger than a certain value among the labeled areas as damage, and displays processed image data emphasizing the damaged portion.

JP-A-7-174672

However, the method disclosed in Patent Document 1 has the following problems.

That is, when capturing an image of the entire circumference of the wheel, it is necessary to divide the image and capture the image, which increases the number of captured images. For example, in the case of a railway, each vehicle has N wheels, one trainset has M vehicles, and if the wheel tread of one wheel is photographed in L divisions, the number of photographed images is N×M×. There are L sheets (N, M, and L are natural numbers). Therefore, it takes a long time to visually check all the captured images.

An object of one aspect of the present invention is to output image data that facilitates visual recognition of wheel damage.

In order to solve the above problems, an image processing device according to an aspect of the present invention measures damage to wheels from a plurality of captured images continuously captured by an imaging device of treads of wheels traveling on a railroad track. a detection unit that detects a damaged area, a selection unit that selects one captured image based on the position of the damaged area in each captured image, and an output unit that outputs a representative image based on the one captured image and have.

Further, in order to solve the above problems, an image processing method according to an aspect of the present invention provides an image processing device, which is a plurality of captured images continuously captured by an imaging device of the tread surface of a wheel running on a railroad track. , detecting a damaged area corresponding to the damage of the wheel, an image processing device selecting one captured image based on the position of the damaged area in each captured image, and based on the one captured image, a representative Including outputting images.

According to one aspect of the present invention, it is possible to output image data in which wheel damage is easily visible.

1 is a functional block diagram showing the configuration of an image data generation device according to Embodiment 1 of the present invention; FIG. FIG. 4 is a diagram showing a state in which the camera and wheels are seen from above in the vertical direction when photographing according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing a state in which the camera and wheels are seen from the horizontal direction when photographing according to the first embodiment of the present invention; FIG. 2 is a diagram showing an example of an image captured by the camera according to Embodiment 1 of the present invention; FIG. FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing a state in which the camera and wheels are seen from the horizontal direction when photographing according to the first embodiment of the present invention; FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing a state in which the camera and wheels are viewed from the horizontal direction when the vertical angle of the camera according to Embodiment 1 of the present invention is 0; FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing a state in which the camera and wheels are seen from the horizontal direction when the angle of the camera in the vertical direction is increased according to the first embodiment of the present invention; FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; FIG. 4 is a diagram showing an example of a photographed image according to Embodiment 1 of the present invention; 4 is a flowchart showing an example of processing executed by the image data generation device according to Embodiment 1 of the present invention; 4 is a flowchart showing details of processing of the image data generation device according to the first embodiment of the present invention; It is a figure for demonstrating the predetermined area|region which concerns on embodiment of this invention. It is a figure for demonstrating the predetermined area|region which concerns on embodiment of this invention. 4 is a flowchart showing an example of processing executed by the image data generation device according to Embodiment 1 of the present invention; FIG. 10 is a diagram showing an example of a photographed image according to Embodiment 2 of the present invention; FIG. 10 is a diagram showing an example of a photographed image according to Embodiment 2 of the present invention; FIG. 5 is a functional block diagram showing the configuration of an image data generation device according to Embodiment 2 of the present invention; FIG. 9 is a diagram showing an example of an image in which distortion of a wheel tread area is corrected according to Embodiment 2 of the present invention; FIG. 10 is a diagram showing an example of a synthesized image according to Embodiment 2 of the present invention; It is a figure which shows an example of the omnidirectional image which concerns on Embodiment 2 of this invention. 9 is a flow chart showing an example of processing executed by an image data generation device according to Embodiment 2 of the present invention; 9 is a flowchart showing details of processing of the image data generation device according to the second embodiment of the present invention; FIG. 10 is a diagram showing an example of an output screen according to Embodiment 2 of the present invention;

Each embodiment of the present invention will be described in detail with reference to the drawings. The drawings merely illustrate specific embodiments consistent with the principles of the invention. These are merely prepared for the understanding of the present invention, and are not intended to limit the interpretation of the present invention. In addition, each element illustrated in each drawing is exaggerated for better understanding of the present invention, so the actual spacing and size of each element are different.

In the following description, if the same reference numerals as those attached to an element in one drawing are also attached to the same element in other drawings, the same element has the same configuration, function, etc. Therefore, detailed description of the same elements is omitted.

[Embodiment 1]
Embodiment 1 of the present invention will be described below based on the drawings.

In this embodiment, by selecting a representative image from a large number of captured images, image data in which wheel damage can be easily visually recognized is output. At this time, if there is an overlapping area in the images obtained by dividing the wheel tread, there is a possibility that the same damage is present in a plurality of images. The wheel tread corresponds to the side surface of a cylinder, and since distortion occurs depending on the position of the wheel tread, the visibility of the detected damage differs. Therefore, when the same damage exists in multiple images, it is necessary to select an appropriate representative image. The selection method will be described below.

Below, the case of a railway vehicle will be described.

(Device configuration)
FIG. 1 is a functional block diagram showing the configuration of an image data generation device (image processing device) 1 according to this embodiment. As shown in FIG. 1 , the image data generation device 1 includes a damaged area detection section (detection section) 2 , a representative image selection section (selection section) 3 , and an output image generation section (output section) 4 .

As shown in FIG. 1, the image data generation device 1 acquires a plurality of captured image data and camera arrangement information. The damaged area detection unit 2 calculates the damaged area of the wheel tread (hereinafter also referred to as the tread) on each photographed image. The representative image selection unit 3 selects photographed image data used to create representative image data based on the camera arrangement information and the position of each damaged area. The output image generator 4 generates representative image data.

Specifically, the damaged area detection unit 2 detects a damaged area corresponding to the damage to the wheel 11 from a plurality of captured images continuously captured by a camera (imaging device) 12 of the tread 15 of the wheel 11 traveling on the railroad track. To detect. The representative image selection unit 3 selects one captured image based on the position of the damaged area in each captured image. The output image generator 4 outputs a representative image based on one captured image. According to the above configuration, it is possible to present to the user, as a representative image, an image in which damage to the tread surface 15 of the wheel 11 can be easily confirmed.

Here, the camera arrangement information is information on the position and orientation of the imaging means with respect to the object to be photographed, and an example thereof will be described with reference to the drawings.

FIG. 2 is a diagram showing a state in which the camera 12 and the wheel 11 are seen from above in the vertical direction when photographing. Here, in FIG. 2, the position where the camera 12 is located is the origin, the z direction is the traveling direction of the wheels 11 (rail direction), and the x direction is the sleeper direction of the rail 10 (horizontal direction perpendicular to the rail direction). It is a coordinate system that is set up as it is. When the camera 12 photographs the wheels 11 moving on the rail 10, the camera 12 is installed at a certain distance from the rail 10 from the viewpoint of safety and maintenance convenience. After the camera 12 is installed, the orientation of the optical axis 14 of the camera 12 in the horizontal direction is set so that the wheel 11 is included in the horizontal angle of view 13 of the camera 12 . The angle between the direction of the optical axis 14 in the horizontal direction and the z-direction is defined as a horizontal angle α of the camera 12 . When the wheels 11 running on the rail 10 are to be photographed separately, the position and orientation of the camera 12 are adjusted so that the wheels 11 are included in the horizontal angle of view 13 during all photographing. Next, the vertical position and orientation of camera 12 are adjusted.

FIG. 3 is a diagram showing a state in which the camera 12 and the wheel 11 are seen from the horizontal direction when photographing. The origin and z-direction are the same as in FIG. The y-direction is perpendicular to the rail and sleeper directions. Here, for example, the camera 12 is installed on the upper surface of the rail 10, and the vertical direction of the camera 12 is tilted upward. The angle formed by the orientation of the optical axis 14 in the vertical direction and the z-direction is defined as the vertical angle β of the camera 12 .

Finally, the inclination of the camera 12 (rotational position about the optical axis 14, roll direction) is adjusted so that the rail 10 is horizontal in the photographed image, for example.

FIG. 4 is a diagram showing an example of an image (captured image) captured by the camera 12. As shown in FIG. As shown in FIG. 4, the wheel 11 on the rail 10 and the wheel tread 15, which is the side surface of the wheel 11, are photographed.

In addition, if the amount of light is insufficient at the time of photographing, a separate lighting device may be arranged.

When photographing the wheel 11, a sensor that detects an object may be used. As a result, it is possible to detect the passing of the wheel 11 and shoot at the timing when the wheel 11 to be shot exists within the shooting range. For example, a photoelectric sensor may be arranged on the side of the rail 10 so that when the wheel crosses the angle of view of the camera, the image is taken.

In addition, when the moving speed of the vehicle is high, the frame rate of the camera 12 is low, or the angle of view is narrow, a plurality of cameras 12 may be installed to photograph the entire circumference of the wheel tread 15. . In this case, in order to prevent the damaged area from being overlooked or to ensure the pixel resolution, a part of the same area of the wheel tread 15 is overlapped between two photographed images in which the wheel tread 15 is adjacent. Shoot at the right time.

(Damaged area detector 2)
A damaged area detection unit 2 detects a damaged area of a wheel tread. As for the method of detecting the damaged area, a general-purpose method using the edge of the image may be used. Edges in the wheel tread area are detected using an edge detection method such as the Sobel or canny method, and labeling is performed using a method such as 8-neighbor labeling. The position and size of the area on the captured image are output as the damaged area.

Here, when detecting the damaged area from the photographed image, if the wheel tread area in the photographed image is detected and the damaged area is detected in the detected wheel tread area, erroneous detection can be reduced.

To detect the wheel tread area, for example, in FIG. 5, hough transform is used to detect the upper line 16 of the rail 10 . An edge detection method such as the Sobel or canny method is used to detect the edges in the image, and the edges forming the arc 18 that touches the line 16 at the point of contact 17 are extracted. The wheel tread 15 on the side surface of the arc 18 on the traveling direction side of the wheel 11 is extracted using the luminance difference with the surroundings. Thereby, the wheel tread area can be detected. Note that the wheel tread area may be input by the user from the outside.

(Representative image selection unit 3)
A representative image selection unit 3 selects a representative image from the camera arrangement information and the position of the detected damaged area. The representative image selection unit 3 preferentially selects an image in which damage is present at a position close to the imaging surface of the camera 12 as a representative image.

Specifically, the representative image selection unit 3 selects the position of the damaged area in each captured image and the directly facing position where the normal direction of the tread surface 15 is substantially parallel to the optical axis direction of the camera 12, based on the distance. Select one captured image. The representative image selection unit 3 may use the optical axis direction of the camera 12 to specify the facing position. In other words, the representative image selection unit 3 may select one captured image based at least on the optical axis direction of the camera 12 .

FIG. 6 is a diagram showing a state in which the camera 12 and the wheel 11 are seen from the horizontal direction when photographing. FIG. 6 assumes the same camera arrangement as in FIG. When damaged areas are detected at a plurality of positions, the positions of the damaged areas in the circumferential direction of the wheel 11 are different. to select a representative image. Therefore, the representative image selection unit 3 treats the position where the normal line on the wheel tread 15 is parallel to the optical axis 14 of the camera 12 as the facing position.

As shown in FIG. 6, the wheel tread 15 has a curved shape, and the direction of the normal to each position of the wheel tread 15 in FIG. 6 differs depending on the position. Furthermore, when the position of the wheel 11 on the rail 10 changes, the direction of the normal changes even at the same vertical position (y-coordinate). Here, the greater the degree that the direction of the normal line on the wheel tread 15 differs from the direction of the optical axis 14 of the camera 12, the more obliquely the wheel tread 15 is photographed. distortion increases.

Therefore, the image facing the camera 12 that is least distorted and easy for the user to observe differs depending on the positions of the observed wheel 11 and wheel tread 15 . For example, when one moving wheel 11 is photographed multiple times with one camera 12, the same camera 12 and the same wheel 11 are photographed at different positions at different photographing timings. position changes.

Hereinafter, the directly facing position will be described as the directly facing position only in the vertical direction.

In FIG. 6 , the position on the wheel 11 that most faces the imaging surface of the camera 12 in the vertical direction is the position of the point 20 where the tangent line 19 is perpendicular to the optical axis 14 . Therefore, the representative image selection unit 3 selects the image showing the damage at the position closest to the position of the point 20 as the representative image.

Next, a method of calculating the coordinates of the point 20 will be described.

In the yz plane, the contour of the wheel 11 is considered circular, and the outer contour (wheel tread 15) of the wheel 11 can be expressed by Equation 1.

Here, a, b, and c are the distances from the origin to the center of the wheel 11 in the x-axis direction, y-axis direction, and z-axis direction, respectively, and r indicates the radius of the wheel 11 . If the camera 12 height is the top surface of the rail 10, then b will equal r. Further, c is equal to the point of contact between the wheel 11 and the rail 10, and when the position of the camera 12 is fixed, from the coordinates of the point of contact between the wheel 11 and the rail 10 in the photographed image, the distance between the camera 12 and the wheel 11 is uniquely determined. Therefore, the value of c can be calculated from the coordinates of the contact position on the image.

Next, the normal vector of the wheel tread 15 will be described on the yz plane. If the position of the point 20 on the contour of the wheel 11 on the yz plane is (p, q), its normal vector is (pb, qc).

Therefore, as the point 20 directly facing the camera 12, the direction of the yz plane of the optical axis 14 of the camera 12 and the normal vector are parallel to this, and a point on the contour of the left side of the wheel 11 is obtained. good.

For example, by obtaining a position (p, q) such that the angle between the normal vector and the horizontal line is equal to the tilt angle β in the vertical direction of the camera, and converting it into the camera coordinate system, the vertical direction on the image is The facing position can be obtained.

7 and 8 are diagrams showing examples of captured images according to the present embodiment. In FIG. 7, there is a damage 21 under a point 20 directly facing the imaging plane of the camera 12 . In FIG. 8, the damage 21 exists at the same height as the point 20 directly facing the camera 12 . In the image of FIG. 7, the damage 21 exists at a position away from the point 20 of the directly facing position compared to the image of FIG. , the damage 21 is photographed distorted. The damage 21 can be detected and confirmed in any image, but when the user visually confirms the damage, the image in FIG. 8 has less distortion of the damage than the image in FIG. 7 and is easier to confirm. Therefore, the representative image selection unit 3 selects the image shown in FIG. 8 instead of the image shown in FIG. 7 as the representative image.

FIG. 9 is a diagram showing a state in which the camera 12 and the wheels 11 are seen from the horizontal direction when the angle of the camera 12 in the vertical direction is 0. As shown in FIG. A point 22 indicates the facing position at this time.

10 and 11 are diagrams showing examples of images captured at this time. None of the locations of damage 23 in FIGS. 10 and 11 exist at point 22, which is the diametrically opposed location. However, comparing the damage location of FIG. 10 with the damage location of FIG. 11, the damage location of FIG. Therefore, the representative image selection unit 3 selects the image shown in FIG. 10 as the representative image instead of the image shown in FIG.

FIG. 12 is a diagram showing a state in which the camera 12 and the wheels 11 are seen from the horizontal direction when the angle of the camera 12 in the vertical direction is increased. The point indicating the facing position at this time is the point 24 below the optical axis 14 .

13 and 14 are diagrams showing an example of an image taken at this time. None of the damage locations of the damage 25 in FIGS. 13 and 14 exists at the point 24 which is the diametrically opposed position. However, comparing the damaged location of FIG. 13 with the damaged location of FIG. 14, the damaged location of FIG. 13 is closer to point 24 and more directly opposite. Therefore, the representative image selection unit 3 selects the image shown in FIG. 13 as the representative image instead of the image shown in FIG.

A representative image can be selected by such a method. Here, in the present embodiment, the facing position in the yz plane of the wheel tread 15 is calculated based on the input angle of the camera 12, but a representative image is selected based on the state of the wheel tread 15. good too. For example, when the position and angle of the camera 12 are fixed and the position of the wheel 11 to be photographed is fixed, the facing position on the yz plane is also fixed, so the optimum position is calculated in advance and tabulated. A representative image can then be determined based on the location of the detected damage. Moreover, although the case where the same damage is detected in two images has been described above, it can also be applied to three or more images. Furthermore, although the same damage has been described above, it is also possible to apply the method to different damages.

The selected representative image can be preferentially notified to the user when the inspection result of the wheel tread 15 is displayed. That is, when displaying only one of a plurality of captured images or when displaying only one of a plurality of captured images in an enlarged manner, a representative image with good visibility can be used.

(Output image generator 4)
The output image generation unit 4 uses the representative image selected by the representative image selection unit 3 to generate and output output image data. At this time, an image showing the position of the damage may be generated on the representative image. For example, an image may be generated in which marks are added around the damaged location, or the damaged location is colored differently. Alternatively, the representative image may be output as it is.

(Overall processing flow)
FIG. 15 is a flowchart showing an example of processing executed by the image data generation device 1 according to this embodiment. Here, the image data generation device 1 generates a representative image from images captured for each wheel 11 . Processing for a plurality of shot images of the same wheel 11 taken in division will be described below.

The image data generation device 1 starts processing when the captured image and the camera arrangement information are acquired.

In the image data generation device 1, first, the damaged area detection unit 2 detects damage by performing image processing such as analyzing edge information of the captured image (step S1).

Next, the representative image selection unit 3 calculates the directly facing position in the vertical direction based on the position of the detected damaged area and the camera arrangement information (step S2).

Next, the image data generation device 1 determines whether or not processing for detecting damage has been completed for all the captured images (step S3). If all the captured images have not been processed (NO in step S3), the image data generation device 1 executes the process of step S1 again. When all the captured images have been processed (YES in step S3), the image data generating device 1 executes the determination in step S4.

In step S<b>4 , the damaged area detection unit 2 determines whether or not there is a damaged image in all the photographed images of the target wheel 11 . Here, the damage image indicates a photographed image in which damage is detected. If there is damage (YES in step S4), the process of step S5 is executed. If there is no damage (NO in step S4), the process of step S6 is executed.

In step S5, the representative image selection unit 3 selects, as a representative image, the damaged image in which the damage exists at the position closest to the directly facing position based on the information on the damaged position and the directly facing position.

In step S6, the representative image selection section 3 selects a predetermined image as a representative image. The predetermined image may be, for example, the most recently photographed image of the target wheel, or may be the photographed image designated by the user.

The output image generation unit 4 generates output image data using the representative image selected in step S5 or S6, outputs the output image data, and ends the process (step S7).

Here, step S5 will be described in detail with reference to the drawings.

FIG. 16 is a flow chart showing details of the processing in step S5 of FIG.

First, the representative image selection unit 3 sets a second predetermined image as a representative image (step S100). Here, the second predetermined image may be, for example, the damage image captured the earliest.

Next, the representative image selection unit 3 determines whether the damage position of the current damage image is closer to the facing position than the damage position of the representative image (step S101).

At this time, if multiple types of damage are detected in one damage image, the position of the damage with the largest size may be used for determination as the damage position. The size at this time shall be the actual size of the damage. The size of the damage is based on simulation or actual measurement according to the position of the wheel 11 on the rail 10 on the captured image and the position on the wheel tread 15. Apply a preset pixel resolution on the wheel tread 15. can be obtained by

In this way, the output image generation unit 4 generates an image showing the largest size of damage as a representative image, so that an image with a larger degree of damage and less distortion is used as a representative image. Output. Thereby, the user can appropriately grasp the state of the wheel tread 15 .

If the damage position of the current damage image is closer to the facing position than the damage position of the representative image (YES in step S101), the representative image selection unit 3 selects the current damage image as the representative image (step S102). . If the damage position of the current damage image is not closer to the directly facing position than the damage position of the representative image (NO in step S101), the representative image selection unit 3 skips the processing of step S102.

Next, the representative image selection unit 3 determines whether or not all the detected damaged images have been processed (step S103). If all damage images have been processed (YES in step S103), the process ends. If the processing has not been performed on all damaged images (NO in step S103), the process returns to step S101.

According to the image data generation device 1 according to the present embodiment described above, an image closest to the position that is most directly facing the imaging surface of the camera 12 in the vertical direction is generated as a representative image. As a result, a damage image with less distortion is selected and generated as a representative image, so that the user can easily check the state of damage.

Further, in step S101, the representative image selection unit 3 selects a representative image based on the position of the damage and the facing position, but it may also refer to the size and brightness of the damage to select the representative image. That is, the representative image selection unit 3 may select one captured image based on the size of the damaged area, or may select one captured image based on the pixel values of the damaged area.

For example, if the damage in each damage image has the same size and the distance between the position of the damage and the directly facing position is the same, the damage position of the damage with the greater brightness of the damage may be used for determination. In general, damage to the wheel tread 15 has a high light reflectance and is photographed brightly. In particular, when the camera 12 and lighting are arranged in the same direction, the damaged portion is photographed brighter than other regions. By generating a large and bright damage image as a representative image in this way, the user can more accurately grasp the damage state of the wheel tread 15 .

As another example, the distance between the damaged position and the directly facing position, the size of the damage, and the brightness are parameters, and the smaller the distance between the damaged position and the directly facing position, the larger the damage. It is also possible to calculate an evaluation value in which the value increases as the brightness of the damage increases, and the position of the damage with the highest evaluation value may be used for determination. In this way, by comprehensively judging the three elements of distortion, size, and brightness and generating a representative image, the user can more accurately grasp the state of damage to the wheel tread 15. can. Also, when calculating the evaluation value, each parameter may be weighted differently. In this case, it is possible to select a representative image according to the parameter that the user places importance on.

Further, in the process of extracting the damage image described in step S4, a constraint condition may be set for the damage to be extracted.

For example, it is possible to use only damage images in which the damage is in a predetermined area. The predetermined area at this time will be described with reference to the drawings. The predetermined area is a wheel tread range to be inspected by the inspection device, and is determined by pixel resolution and shielding by peripheral parts. For example, when the difference between the imaging direction of the camera and the normal direction of the wheel tread becomes large, the wheel tread is photographed obliquely, resulting in lower pixel resolution than in the case of direct imaging. Therefore, when performing an inspection with a desired pixel resolution or higher, an area having a desired pixel resolution or higher in a certain photographed image is set as a predetermined area, and the wheel tread of the other area is changed as the wheel rotates. Another image taken at another position is to be inspected.

At this time, the representative image selection unit 3 may select one captured image based on the position of the damaged area corresponding to the same damage. That is, when the predetermined regions overlap in the photographed images of the tread 15 of the adjacent wheel 11, there is a possibility that the same damaged region is detected in a plurality of images. Therefore, the representative image selection unit 3 determines whether the damages are the same based on the positions and sizes of the damages in the overlapping area. You can leave it as a candidate for

As a result, when the same damage is present in each photographed image, an image at a position closer to the front with less distortion is selected, so that the user can correctly grasp the shape of the damage. When the photographing position of the wheel 11 is determined, when there are overlapping areas, it is uniquely determined which area is more directly facing, so it is possible to reduce the amount of processing involved in determination.

Also, if there are parts around the wheel 11, the wheel tread 15 may be blocked and cannot be confirmed in the image. Therefore, by setting the wheel tread 15 excluding the shielded area as a predetermined area, appropriate damage detection processing can be performed. In other words, erroneous detection of damage in the component area can be reduced, and the amount of processing can be reduced by the amount of damage detection processing in the component area.

17 and 18 are diagrams for explaining the predetermined area. For example, in FIG. 17, an area 26 having a desired pixel resolution is set as the predetermined area. Further, when the upper part of the area of the wheel tread 15 is hidden by the peripheral component 28, as shown in FIG. . In this case, it is assumed that the position and size of the peripheral component 28 are obtained in advance. Then, a damaged image having damage at a position closest to the directly facing position within the predetermined area is selected as a representative image. Here, the damage position may be, for example, the center coordinates of the circumscribing rectangle of the detected damage area, the top, bottom, left, or right coordinates of the boundary of the circumscribing rectangle, or the center of gravity of the damage area calculated from the detected damage area. may be used as the coordinates of

FIG. 19 is a flowchart showing an example of processing executed by the image data generation device 1 when selecting a representative image from damage images in which damage is in a predetermined area. This flowchart is obtained by inserting step S110 instead of step S4 in the flowchart of FIG.

In step S110, the representative image selection unit 3 selects an image in which the damage is in a predetermined area from the photographed images with damage as the damage image. After step S5, the representative image selection unit 3 similarly selects a representative image using the damage image selected in step S110.

As described above, a representative image is generated by excluding an image in which an area that does not satisfy the desired resolution and an image in which an area blocked by peripheral components is damaged. As a result, an image that has a desired resolution and allows easy visual recognition of the wheel tread 15 can be selected as the representative image. In particular, by using an area that considers the shielded area by the peripheral parts, the damage is displayed at the same position regardless of the presence or absence of the peripheral parts, so it is still useful when comparing the degree of damage to different wheels.

The image data generation device 1 according to the present embodiment can be connected to an external display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display, or to a storage device such as a flash memory or a hard disk. can. As a result, the image data generated by the image data generation device 1 can be displayed on a display device, stored in a storage medium as an image data file, or read from an image stored in the storage medium and displayed on a display device. can do.

Each process of the present embodiment can be realized by software processing by CPU (Central Processing Unit) or GPU (Graphics Processing Unit), and hardware processing by ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). .

A method for realizing the functions described in this embodiment can also be provided as a program. A program may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be loaded into a computer system or downloaded from the Internet and executed to perform the processing of each unit. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices.

The "computer system" also includes the home page providing environment (or display environment) if the WWW system is used.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

[Embodiment 2]
A second embodiment of the present invention will be described. For convenience of description, members having the same functions as the members described in the first embodiment are denoted by the same reference numerals, and description thereof will not be repeated.

When continuously photographing a moving wheel tread, if the damage is large in the two shot images taken at the nearest time, part of the damage is applied to the upper end or the lower end of the tread in each photographed image, and Any captured image may show only a portion of the damage. This case will be described with reference to the drawings.

20 and 21 are diagrams showing an example of images of the wheel tread 102 of a running railroad vehicle taken continuously. At this time, it is assumed that the vehicle is moving from right to left. As shown in FIG. 20, the peripheral part 103 is a wheel top cover. Peripheral component 104 is a brake. A wheel tread 102 of the wheel 101 on the rail 100 is shielded by a peripheral component 104 . An area below the dotted line 105 and above the dotted line 106 on the wheel tread 102 is an area that does not overlap with another captured image. In FIG. 20, the damage 107 is on the upper side and partly shielded by the peripheral component 104 . On the other hand, in FIG. 21 the wheel 101 has moved to the left and part of the damage 107 is blocked by the rail 100 .

In addition to such a difference in shielded position, the vertical position on the tread surface of the damage is different, so the degree of facing the camera 12 to the imaging plane is also different, and distortion occurs according to the degree. Even if it is damaged, the state will be photographed differently. In such a case, although there are a plurality of photographed images that can be presented, it is necessary to present an appropriate image that allows the user to easily confirm the state of damage.

Therefore, the image data generation device (image processing device) 108 according to the present embodiment detects whether or not there is a damage image that extends over each photographed image, and if such an image exists, based on the damage position on the wheel tread, , an image with less distortion is selected and presented to the user. For example, an image is taken each time the wheel tread 102 moves by the length of the non-overlapping region. Furthermore, at the same time, using all the photographed images for one wheel, a single all-around image of the tread is synthesized and presented to the user. This allows the user to grasp the state of the entire wheel tread.

(Device configuration)
FIG. 22 is a functional block diagram showing the configuration of the image data generation device 108 according to this embodiment. The image data generation device 108 according to the present embodiment includes a damaged area detection unit 2, an all-around image generation unit (generation unit) 109, a representative image selection unit (input unit, selection unit) 110, and an output image generation unit 4. I have. Since the damaged area detection unit 2 and the output image generation unit 4 have the same configuration as that of the image data generation device 1, the description thereof will be omitted, and the omnidirectional image generation unit 109 and the representative image selection unit 110 will be described below. .

(Circumference image generator 109)
The omnidirectional image generation unit 109 according to the present embodiment corrects the distortion of the wheel tread area of each photographed image to generate an omnidirectional image of the wheel tread. That is, the omnidirectional image generation unit 109 synthesizes a plurality of captured images to generate an omnidirectional image of the tread. Then, the omnidirectional image generator 109 preferentially synthesizes one or more picked-up images selected based on the position of the damaged area in each picked-up image.

FIG. 23 is a diagram showing an example of an image in which the distortion of the wheel tread area is corrected. FIG. 23(a) shows an image obtained by correcting the distortion of the wheel tread of the photographed image of FIG. FIG. 23(b) shows a planar image obtained by correcting the distortion of the wheel tread in the photographed image of FIG.

A planar image at this time will be described. When the camera 12 is fixed and the size of the wheel is determined, the three-dimensional position of the wheel tread can be geometrically obtained from the wheel position on the rail in the photographed image, and based on that information, the distortion of the wheel tread area can be calculated. Correct and unfold on a plane. In the unfolded planar image, the horizontal direction corresponds to the width of the wheel, and the vertical direction corresponds to the vertical length of the non-overlapping area. The distortion correction method at this time may be a general-purpose method. For example, when the error is small, a method of simply developing the tread area detected on the image into a plane image may be used.

Here, an area 111 in (a) of FIG. 23 and an area 112 in (b) of FIG. A region 114 in (b) of 23 is an overlapping region.

In a normal case, the omnidirectional image generator 109 synthesizes the non-overlapping regions to generate a synthesized image as shown in FIG. 24(a). However, when an overlapping area occurs, the omnidirectional image generation unit 109 synthesizes using the overlapping area 113 of the image in which the position of the damage is closer to the directly facing position, as shown in FIG. 24(b). In this way, when damage is reflected in adjacent photographed images and there is an overlapping area, the omnidirectional image generation unit 109 preferentially synthesizes an image closer to the directly facing position. This allows a composite image of the damage with high resolution and low distortion to be generated. Further, in the above description, whether or not the position is close to the directly facing position is determined for each damage, but may be determined for each pixel in the overlap region. This allows a composite image of the damage with higher resolution and less distortion to be generated.

Next, the omnidirectional image generation unit 109 performs similar synthesis processing on the omni-photographed images to generate an omnidirectional image of the wheel tread as shown in FIG. 25 . Here, two damages, damage 107 and damage 116, are present on the omnidirectional image 115. FIG. It should be noted that, when synthesizing the omnidirectional images, for an overlapping area in each of the photographed images, both images may be blended at a predetermined ratio. At this time, the ratio of the image closer to the directly facing position may be larger than the other, and if the shape of the same damage in both images is significantly different, the image closer to the directly facing position is combined. You can even increase the ratio.

(Representative image selection unit 110)
The representative image selection unit 110 according to the present embodiment receives input of a designated area on the omnidirectional image. Then, the representative image selection unit 110 selects one captured image from the captured images including the designated area.

That is, the representative image selection unit 110 presents the user with an image of the entire circumference of the wheel tread, receives an input of the specified area specified by the user, and uses the photographed image in which the specified area is directly facing the camera. Select a representative image.

The processing at this time will be described below.

(Overall processing flow)
FIG. 26 is a flowchart showing an example of processing executed by the image data generation device 108 according to this embodiment. Since the flowchart of FIG. 26 has steps S300 to S303 added after step S3 to the flowchart of FIG. 15, the added steps will be described.

The omnidirectional image generation unit 109 determines whether or not the same damage exists in a combination of adjacent damage images (step S300).

If the same damage exists (YES in step S300), as shown in FIG. For an image in which the damage is not located closer to the directly facing position, the composition position is set so that the region excluding the overlapping region is composited (step S301).

If the same damage does not exist (NO in step S300), as shown in (a) of FIG. (step S302).

Then, the omnidirectional image generation unit 109 determines whether or not all combining positions have been obtained (step S303).

When all the combining positions have been obtained (YES in step S303), the omnidirectional image generation unit 109 cuts out each photographed image using the compositing positions, develops them into planar images, arranges them in order of photography, and generates an omnidirectional image. is generated (step S304). If the same damage does not exist (NO in step S300), the omnidirectional image generator 109 returns to the determination in step S300.

Next, the representative image selection unit 110 receives an input of a user-specified region, and generates a representative image using the photographed image data in which the user-designated region is closest to the position directly facing the camera (step S305). Then, the process ends.

Details of the processing of the representative image selection unit 110 at this time will be described with reference to the drawings. FIG. 27 is a flow chart showing the details of the processing in step S305 of FIG.

The representative image selection unit 110 presents the user with the omnidirectional image of the wheel tread generated by the omnidirectional image generation unit 109, and receives an external input of the user's designated area to be displayed (step S306). For example, the user designates an area to be displayed in the presented image through an external input such as a mouse or flick. A device capable of displaying an image, such as a liquid crystal display built into the image data generation device 108 or externally attached, can be used to present the omnidirectional image.

Next, the representative image selection unit 110 refers to the damage detection result calculated by the damage area detection unit 2, and determines whether or not there is damage in the user-designated area (step S307). If damage exists in the designated area (YES in step S307), the representative image selection unit 110 generates a representative image using the image in which the damage is closest to the directly facing position (step S308). If there is no damage in the designated area (NO in step S307), the representative image selection unit 110 generates a representative image using an image in which the designated area is positioned most directly in place of the damage (step S309). Then, the process ends.

In this way, based on the omnidirectional image of the wheel tread created so as to minimize distortion, the user designates the position of the damage to be focused on, and the photographed image of the position closest to the designated damage position is displayed. By generating a representative image using , the user can accurately grasp the state of the focused area of the wheel tread.

In addition, since a full-circumference image of the wheel tread is presented, the user can grasp the state of damage in the entire area of the wheel tread. In particular, when there are multiple damages in the entire area of the wheel tread, it is possible to grasp the ratio of damages in the entire wheel and the positional relationship of each damage.

In addition, there are cases where the image is captured at a timing different from the original due to reasons such as the timing of the image being captured being delayed or the traveling speed of the vehicle being faster than expected. In such a case, when the omnidirectional image generation unit 109 generates the omnidirectional image of the wheel tread, the wheel position in the photographed image is compared with the position of the wheel at the assumed photographing timing. It is sufficient to adjust and generate by shifting the non-overlapping area of the photographed image set in advance. In addition, if the shooting timing is too late or the shooting is not possible, if there is an area that cannot be captured in any of the captured images, another image will be displayed on the all-around image to indicate that there is a shooting error. For example, a black-painted image or the like may be used to generate the omnidirectional image. As a result, when the omnidirectional image cannot be captured due to a shooting error or the like, the user can grasp the imaged area on the wheel tread and the other area. If the size is smaller than a predetermined size, it may be determined that there is no particular effect on determination of damage.

Alternatively, the user may directly specify the captured image, and the representative image selection unit 110 may generate the representative image based on the specification. At this time, an omnidirectional image in which a region corresponding to the representative image is highlighted may be generated at the same time. As a result, the user can accurately ascertain which scratch on the wheel tread is the photographed image that is currently being displayed.

Further, when presenting the omnidirectional image generated by the omnidirectional image generation unit 109, an image of a predetermined range of the omnidirectional image may be used. For example, the omnidirectional image may be divided in the direction of the short side, and the divided images may be arranged vertically and presented. As a result, it is possible to display the image in accordance with the shape of the display area of the display, and to enlarge the size of the omnidirectional image.

FIG. 28 is a diagram showing an example of an output screen according to this embodiment. A representative image is displayed on the right side of the output screen. A plurality of captured images are displayed on the left side of the output screen. A representative image is an image selected from a plurality of captured images.

[Example of realization by software]
The control blocks of the image data generation devices 1 and 108 (damaged area detection unit 2, representative image selection unit 3, output image generation unit 4, all-around image generation unit 109, representative image selection unit 110) are integrated circuits (IC chips). It may be implemented by a logic circuit (hardware) formed in the same manner as above, or may be implemented by software.

In the latter case, the image data generation device 1, 108 is provided with a computer that executes instructions of a program, which is software that implements each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium storing the program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. In addition, a RAM (Random Access Memory) for developing the above program may be further provided. Also, the program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be implemented in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

〔summary〕
An image processing device according to aspect 1 of the present invention includes a detection unit that detects a damaged area corresponding to damage to the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of the wheel running on the railroad track; , a selection unit that selects one captured image based on the position of the damaged area in each captured image, and an output unit that outputs a representative image based on the one captured image.

According to the above configuration, since the picked-up image of the damaged area is selected based on the position of the damaged area in each picked-up image, it is possible to output image data that makes it easy to visually recognize the damage of the wheel.

The image processing apparatus according to aspect 2 of the present invention is the image processing apparatus according to aspect 1, wherein the selection unit is positioned such that the position of the damaged area and the normal direction of the tread are substantially parallel to the optical axis direction of the imaging device. , and the one captured image may be selected based on the distance between .

According to the above configuration, since the picked-up image of the damaged area is selected based on the distance from the position directly facing the imaging device on the tread, it is possible to output image data that makes it easy to visually recognize the damage to the wheel.

In the image processing apparatus according to aspect 3 of the present invention, in the aspect 1 or 2, the selection unit may select the one captured image based on the optical axis direction of the imaging device.

According to the above configuration, since the picked-up image of the damaged area is selected based on the optical axis direction of the imaging device, it is possible to output image data that makes it easy to visually recognize the damage of the wheel.

In the image processing apparatus according to aspect 4 of the present invention, in the aspects 1 to 3, the selection unit may select the one captured image further based on the size of the damaged area.

According to the above configuration, since the picked-up image of the damaged area is selected further based on the size of the damaged area, it is possible to output image data that makes it easy to visually recognize the damage of the wheel.

In the image processing apparatus according to aspect 5 of the present invention, in any one of aspects 1 to 4, the selection unit may select the one captured image further based on pixel values of the damaged area.

According to the above configuration, since the picked-up image of the damaged area is selected further based on the pixel values of the damaged area, it is possible to output image data in which the damage of the wheel is easy to visually recognize.

In the image processing apparatus according to aspect 6 of the present invention, in the aspects 1 to 5, the selection unit may select the one captured image based on the position of the damaged area corresponding to the same damage.

According to the above configuration, since the picked-up image of the damaged area is selected based on the position of the damaged area corresponding to the same damage, it is possible to output image data in which the wheel damage is easily visible.

An image processing apparatus according to aspect 7 of the present invention is, in the aspects 1 to 6, further comprising a generation unit that synthesizes the plurality of captured images to generate an all-around image of the tread, wherein the generation unit includes each of the captured images. One or more captured images selected based on the position of the damaged area within the image may be preferentially combined.

According to the above configuration, it is possible to output image data including the entire circumference of the tread surface of the wheel so that the damage to the wheel can be easily visually recognized.

An image processing apparatus according to aspect 8 of the present invention, in aspect 7, further includes an input unit that receives an input of a specified region on the omnidirectional image, wherein the selection unit selects the It is also possible to select one captured image.

According to the above configuration, it is possible to output image data in which the wheel damage is easily visible according to the user-specified region.

In the image processing method according to aspect 9 of the present invention, the image processing device detects a damaged region corresponding to the damage to the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of a wheel running on a railroad track. detecting, an image processing device selecting one captured image based on the position of the damaged area in each captured image, and outputting a representative image based on the one captured image.

According to the above configuration, since the picked-up image of the damaged area is selected based on the position of the damaged area in each picked-up image, it is possible to output image data that makes it easy to visually recognize the damage of the wheel.

The image processing apparatus according to each aspect of the present invention may be implemented by a computer. In this case, the image processing apparatus is implemented by the computer by operating the computer as each part (software element) included in the image processing apparatus. A control program for an image processing apparatus realized by a computer and a computer-readable recording medium recording the program are also included in the scope of the present invention.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

1, 108 image data generation device (image processing device)
2 Damaged area detection unit (detection unit)
3 Representative image selection section (selection section)
4 Output image generator (output unit)
109 Circumference image generation unit (generation unit)
110 representative image selection unit (input unit, selection unit)

Claims (9)

A detection unit that detects a damaged area corresponding to the damage of the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of the wheel running on the railroad track;
a selection unit that selects one captured image based on the position of the damaged area in each captured image and information on the position and orientation of the imaging device with respect to the wheel ;
an output unit that outputs a representative image based on the one captured image;
An image processing device comprising: A detection unit that detects a damaged area corresponding to the damage of the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of the wheel running on the railroad track;
a selection unit that selects one captured image based on the position of the damaged area in each captured image;
an output unit that outputs a representative image based on the one captured image;
with
The selection unit selects the one captured image based on the distance between the position of the damaged area and a directly facing position where the normal direction of the tread surface is substantially parallel to the optical axis direction of the imaging device. An image processing apparatus characterized by: A detection unit that detects a damaged area corresponding to the damage of the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of the wheel running on the railroad track;
a selection unit that selects one captured image based on the position of the damaged area in each captured image;
an output unit that outputs a representative image based on the one captured image;
with
The image processing device, wherein the selection unit selects the one captured image based on an optical axis direction of the imaging device. The image processing apparatus according to any one of claims 1 to 3, wherein the selection unit selects the one captured image further based on the size of the damaged area. The image processing apparatus according to any one of claims 1 to 4, wherein the selection unit selects the one captured image further based on pixel values of the damaged area. A detection unit that detects a damaged area corresponding to the damage of the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of the wheel running on the railroad track;
a selection unit that selects one captured image based on the position of the damaged area in each captured image;
an output unit that outputs a representative image based on the one captured image;
with
The image processing device, wherein the selection unit selects the one captured image based on a position of a damaged area corresponding to the same damage. A detection unit that detects a damaged area corresponding to the damage of the wheel from a plurality of captured images continuously captured by an imaging device of the tread surface of the wheel running on the railroad track;
a selection unit that selects one captured image based on the position of the damaged area in each captured image;
an output unit that outputs a representative image based on the one captured image;
with
further comprising a generation unit that synthesizes the plurality of captured images to generate an all-around image of the tread,
The image processing apparatus, wherein the generation unit preferentially synthesizes one or more captured images selected based on the position of the damaged area in each captured image. further comprising an input unit that receives input of a specified area on the omnidirectional image,
8. The image processing apparatus according to claim 7, wherein the selection unit selects the one captured image from captured images including the designated area. An image processing device detects a damaged area corresponding to the damage of the wheel from a plurality of captured images in which the tread surface of the wheel running on the railroad track is continuously captured by the imaging device, and the image processing device detects the damage area in each captured image. selecting one captured image based on information on the position of the damaged area and the position and orientation of the imaging device with respect to the wheel , and outputting a representative image based on the one captured image. An image processing method characterized by:

Images