JP2016058086A - Optical route examination system and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an optical route examination system and method for examining routes traveled by vehicles and/or equipment disposed alongside the routes.SOLUTION: Optical route examination systems 100 and methods obtain image data of a field of view of a camera 106 (106a, 106b) disposed onboard a vehicle 102 as the vehicle 102 moves along a route 120, and autonomously examine the image data onboard the vehicle 102 to identify one or more of a feature of interest and a designated object. The feature of interest includes a standard gauge distance between two or more portions of the first route, and the designated object includes a sign and wayside equipment.SELECTED DRAWING: Figure 1

Description

  Embodiments of the presently-disclosed subject matter relate to investigating the route traveled by a vehicle and / or equipment located alongside the route.

  The route along which the vehicle travels can be damaged over time due to long-term use. For example, the track on which the railway vehicle moves may be misaligned due to a shift in the ballast material underneath or the left-right swing of the railway vehicle. The track may bend or move slightly from the original placement of the track. This misalignment can include a curvature of the track that changes the distance between the rails of the track (ie, the standard gauge). Alternatively, the distance may remain the same (eg, both lines are curved by the same or approximately the same amount). This would pose a risk to the safety of the rail car, its passengers and the people nearby and the building. For example, the risk of railcar derailment will increase if the track deviates.

  Some known systems and methods for inspecting tracks include illuminating visible markers on the track and optically monitoring these markers to determine if the track has shifted. These visible markers may be created using, for example, laser light. However, these systems and methods may require additional hardware in the form of a light source, such as a laser light source. This additional hardware increases the cost and complexity of the system and may require specialized rail vehicles that are not used to carry passengers or cargo. In addition, these systems and methods typically require a rail vehicle to travel slowly on the track so that visible markers can be investigated.

  Some rail vehicles include a collision avoidance system to warn the rail car driver that there is a foreign object on the track ahead of the rail vehicle. However, these systems may only include a camera that provides images to the boarding driver. The driver manually inspects the image for any foreign object, and when the foreign object is identified by the driver, the driver responds accordingly. These types of systems are prone to human error.

  A rail vehicle or other type of vehicle can operate according to an automatic safety system that stops or decelerates the operation of a vehicle at a location. These systems may rely on a database that associates various positions of the route traveled by the vehicle with various speed limits. If the vehicle travels beyond these limits, the system can send a signal to the vehicle to decelerate or stop the vehicle. Some known systems leave the creation and / or updating of the database to a human operator, which can cause errors. As a result, the system may not have accurate information and in some places will allow the vehicle to travel beyond the limits. This will pose a significant safety risk.

  These and other types of safety systems may include an intersection system that warns and / or prevents simultaneous intersections of vehicles through the intersections of the route. For example, a railway vehicle may travel on a track that crosses a route on which another vehicle such as an automobile travels. The safety system may include a gate, a signal, or the like at an intersection between the track and the route along which the automobile travels. Some of these systems have gates, signals, etc. that are moving properly to stop or warn other vehicles when a rail vehicle is approaching an intersection in some situation, such as during a power outage. If not, it may not be possible to judge.

U.S. Pat. No. 8,744,196

  In one embodiment, a method (e.g., for investigating a route) includes image data of a field of view of a camera mounted on a first vehicle, wherein the first vehicle is moving along the first route. Acquiring in the case and automatically inspecting the image data on the first vehicle to identify one or more of the target feature or specified object.

  In another embodiment, the system (eg, route survey system) is configured to have one or more images configured to be mounted on the first vehicle when the first vehicle moves along the first path. Includes analysis processor. The one or more image analysis processors also obtain image data of a view of a camera mounted on the first vehicle and identify the target feature or one or more of the designated objects on the first vehicle. Configured to automatically examine image data.

  In another embodiment, another method (eg, for investigating a route) includes examining image data of a track having a plurality of rails. The image data can be acquired from a camera mounted on a vehicle that moves along the track. The method also determines a standard rail distance of the track based on at least a portion of the image data, and identifies a segment of the track having one or more damaged rails based on a trend in the standard rail distance of the rail. Steps.

  Specific embodiments and further advantages of the present invention are described in more detail in the following description with reference to the accompanying drawings.

1 is a schematic diagram of an optical path survey system according to one embodiment. FIG. It is a figure which shows an example of the image which the camera acquired about the division | segmentation of the path | route shown in FIG. It is a figure which shows another example of the image of the path | route shown in FIG. It is a figure which shows another example of a reference | standard visual profile. FIG. 3 is a diagram illustrating a visual mapping diagram of the image shown in FIGS. 2A and 2B and the reference visual profile shown in FIGS. 3A and 3B, according to one embodiment. FIG. 3 is a schematic diagram of an intersection between two or more routes according to one embodiment. It is a flowchart about the method of investigating a path | route from a vehicle, when a vehicle moves along a path | route. FIG. 2 is a superimposed display of three images acquired by one or more of the cameras shown in FIG. 1 and superimposed on each other, according to one embodiment. It is a flowchart about the method of investigating a path | route from a vehicle, when a vehicle moves along a path | route. FIG. 4 is a camera acquired image with a reference visual profile of a path, according to one embodiment. FIG. 6 is an illustration of another camera acquired image with a reference visual profile of a path, according to one embodiment. It is a figure which shows an example of the standard gauge distance judged by the image analysis processor. It is a figure which shows another example of the standard gauge distance judged by the image analysis processor shown in FIG. It is a figure which shows another example of the standard gauge distance judged by the image analysis processor shown in FIG. FIG. 6 shows a flowchart for a method of identifying damage to a path, according to one embodiment. FIG. 6 shows a flowchart for a method of identifying damage to a path, according to one embodiment. FIG. 6 is a schematic diagram of an optical path survey system according to another embodiment. FIG. 17 is a diagram showing image data acquired by the route survey system shown in FIG. 16 according to an example. FIG. 18 is a schematic diagram of a memory structure created based on a survey of signs shown in the image data of FIG. 17 and at least a portion of the survey of signs, according to one embodiment. 4 is a flowchart for a method of identifying information indicated on a sign from image data, according to one embodiment. It is a figure which shows the image data showing an intersection part by an example. 6 is a flowchart illustrating a method for investigating equipment along a track using image data, according to one embodiment.

  Embodiments of a route survey system and method of operation are disclosed herein. This system can be mounted on a vehicle traveling along a route. While moving along the route, a camera mounted on the vehicle can acquire or generate image data of the route and / or the surrounding area of the route. This image data can be examined on the vehicle to identify features of interest and / or designated objects. For example, the feature of interest can include a standard gauge distance between two or more portions of the path. For rail vehicles, the feature of the object identified from the image data can include a standard gauge distance between the rails of the route. Designated objects can include equipment along the track, such as safety devices, signs, signals, switches, or inspection devices. The image data is automatically inspected by the route survey system to determine changes in target characteristics, missing specified objects, damaged or malfunctioning specified objects, and / or specified object positions Can be judged. This automatic inspection may be performed without operator intervention. Alternatively, the automatic inspection may be performed with the help of the operator and / or at the operator's request.

  One or more embodiments described herein include systems and methods for detecting damage to a path traveled by a vehicle. The system and method can detect damage to the path using analysis of images collected from cameras mounted on the vehicle. The system and method can detect a misalignment of a track on which a railway vehicle travels. The system and method can detect this misalignment using analysis of track images collected from cameras on rail vehicles. Based on the detected misalignment, the railway vehicle driver can be warned so that the driver can perform one or more response actions, such as by decelerating and / or stopping the railway vehicle. it can. While the description herein focuses on railcars and tracks with rails, one or more embodiments are designed or allowed to travel on other off-highway vehicles (eg, on public roads). The present invention may be applied to a vehicle other than a railway vehicle, such as a non-vehicle) or an automobile. Further, one or more embodiments may be applied to tracks having different numbers of rails or paths other than tracks for rail vehicles, such as roads.

  The route image can be acquired from a camera attached to a vehicle such as a locomotive. The camera can be directed to the track (eg, pointing to the track) in the direction of travel of the railway vehicle. The camera can periodically (or aperiodically) acquire an image of the track for analyzing the positional deviation. If the track deviates, the track can cause derailment of the railway vehicle. Some of the systems and methods described herein detect rail misalignment in advance (e.g., before the rail vehicle reaches the off track) and alert the rail car driver, Prevent derailment. Optionally, in an unmanned rail vehicle (eg, a rail vehicle that operates automatically), the system and method may automatically decelerate or stop movement of the rail vehicle depending on the identification of the off-track. Good.

  Additionally or alternatively, one or more other responses may be initiated when an off-line section of the line is identified. For example, an alarm signal can be communicated (e.g., transmitted or distributed) to one or more other rail vehicles to alert other vehicles of misalignment and the alarm signal is placed on that track, or The device along the track can communicate with one or more devices along the track, so that the device along the track can communicate alarm signals to one or more other rail vehicle systems; The alarm signal can be communicated to a non-installed facility that can be arranged for such purposes as repairing a misaligned section of the track and / or further investigation.

  If the track is not in the same position as before due to misalignment or movement of the track, the track may have shifted. For example, instead of breaking or corroding, in the track, a shift of the track in the lateral direction from the previous position, such as the track position when the track was installed or previously inspected. And / or may result from longitudinal movement of the track.

  System and method for using a device that generates light to inspect a path, such as a laser light source that generates laser light on a rail of a line and monitors the laser light to identify changes in the rail profile In contrast, one or more aspects of the systems and methods described herein rely on acquisition of image data without generating light or other energy on the path. As described below, one or more systems and methods described herein may capture still and / or moving images of a path and compare these images and / or moving images to reference image data. it can. In at least one embodiment, the path is marked or inspected without using light such as laser light.

  FIG. 1 is a schematic diagram of an optical path survey system 100 according to one embodiment. The system 100 is mounted on a vehicle 102, such as a railway vehicle. The vehicle 102 can connect with one or more other vehicles, such as one or more locomotives or railway vehicles, to form a train that travels along a path 120 such as a railroad. Alternatively, the vehicle 102 may be another type of off-highway vehicle (eg, a vehicle that is not designed or allowed to travel on public roads) or another type of vehicle, such as an automobile. It is good. In some configurations, the vehicle 102 can pull and / or push passengers and / or cargo, such as in a train or other system of vehicles.

  The system 100 includes one or more cameras 106 (eg, cameras 106 a, 106 b) that are mounted on or otherwise connected to the vehicle 102, with the camera 106 along with the vehicle 102 along the path 120. Moving. The camera 106 may be a forward facing camera 106 when the camera 106 is oriented toward the direction of travel or movement 104 of the vehicle 102. For example, the fields of view 108 and 110 of the camera 106 represent a space acquired by an image obtained by the camera 106. In the illustrated example, the camera 106 is forward-looking when the fields of view 108 and 110 acquire images and / or moving images of the space in front of the moving vehicle 102. The camera 106 can acquire static (eg, still) images and / or moving images (eg, video). Optionally, one or more cameras 106 may be located inside the vehicle 102. For example, the vehicle 102 may include a cab camera 106 disposed in the driver's cab of the vehicle 102. Such a camera 106 can obtain images and / or movies through the window of the vehicle 102 and is filed on August 12, 2014, US Pat. No. 14 / 457,353 “Vehicle Imaging System And Method”. ”(For such a camera 106), the entire disclosure of which is incorporated herein by reference.

  The camera 106 can obtain an image of the route 120 while the vehicle 102 is moving at a relatively high speed. For example, the image shows that the vehicle 102 is at the upper limit speed of the route 120 or the upper limit speed, such as the course speed of the route 120 when the maintenance is not performed on the route 120 or when it is not below the upper limit speed of the route 120. It may be acquired while moving in the vicinity.

  The camera 106 operates based on a signal received from the camera controller 112. The camera controller 112 includes one or more hardware circuits or circuit elements that include and / or are connected to one or more computer processors (eg, microprocessors) or other electronic logic-based devices. Or mean. The camera controller 112 operates the camera 106 and causes the camera 106 to acquire image data. This image data represents an image of the field of view 108, 110 of the camera 106, such as an image of one or more portions or sections of the route 120 disposed in front of the vehicle 102. The camera controller 112 can change the frame rate of the camera 106 (eg, the speed or frequency at which the camera 106 acquires images).

  One or more image analysis processors 116 of the system 100 examine images acquired by one or more cameras 106. Processor 116 includes one or more computer circuits (eg, microprocessors) or other electronic logic-based devices and / or includes one or more hardware circuits or circuit elements connected thereto, or Or it can mean. In one aspect, the processor 116 examines the image by identifying which portions of the image are images for the path 120 and comparing these portions to one or more reference images. Based on the similarities or differences between the one or more camera acquired images and the reference image, the processor 116 can determine whether the segment of the path 120 indicated by the camera image is misaligned. Alternatively, the image analysis processor 116 can convert the image data into wireframe model data or generate wireframe model data, which is described in US patent application Ser. No. 14 / 253,294, “ "Route Image Prediction System And Method" ('294 application), the entire disclosure of which is incorporated herein by reference. The wireframe model data can be used to identify the location or shape of the path 120 or the like.

  2A and 2B are diagrams illustrating an example of the camera acquired image 200 of the section of the route 120. FIG. As shown in FIGS. 2A and 2B, the image 200 may be a digital image formed from a number of pixels 202 that vary in color and / or intensity. As the intensity of the pixel 202 increases, the color can become brighter (eg, whiter), and as the intensity of the pixel 202 decreases, the color can become darker. In one aspect, the image analysis processor 116 (shown in FIG. 1) examines the intensity of the pixels 202 to determine which portion of the image 200 represents the path 120 (eg, the rails 204 of the track). For example, the processor 116 may select a pixel 202 having an intensity greater than a specified threshold, and the pixel 202 may be a display portion of some or all of the pixels 202 or path 120 in the image 200 (eg, a line Which has a greater intensity than the average or median of the pixels 202 different from the rails 204). Alternatively, the processor 116 may identify rails 204 in the image 200 using another technique.

  Returning to the description of the system 100 shown in FIG. 1, the image analysis processor 116 stores one or more reference visual profiles of several such profiles stored in a computer readable memory, such as the image memory 118. You can choose from. The memory 118 includes or represents one or more memory devices, such as a computer hard drive, CD-ROM, DVD ROM, removable flash memory card, or magnetic tape. The memory 118 may store an image 200 (shown in FIGS. 2A and 2B) acquired by the camera 106 and a reference visual profile associated with the progression of the vehicle 102.

  The reference visual profile represents a designated layout of the route 120 when the route 120 is in a different position. For example, the reference visual profile may represent the position, placement, and relative position of the rails in the path 120, such as when the rails are laid, repaired, or last passed inspection.

  In one aspect, the reference visual profile is the designated standard gauge of the path 120 (eg, the distance between the rails of the track). Alternatively, the reference visual profile can be a previous image of the path 120 at the selected location. In another example, the reference visual profile may be a definition of where the path 120 (eg, rails of a track) is expected to be located in the image of the path 120. For example, different reference visual profiles may represent different shapes of rails 204 (shown in FIGS. 2A and 2B) at different locations along the movement of the vehicle 102 from one location to another.

  The processor 116 can determine which reference visual profile to select in the memory 118 based on the position of the vehicle 102 when the image 200 is acquired. The vehicle controller 114 can be used to manually and / or automatically control the movement of the vehicle 102 to track where the vehicle 102 was located when the image 200 was acquired. For example, the vehicle controller 114 can include and / or connect to a positioning system, such as a global positioning system or a cellular triangulation system, where the vehicle 102 is located. It can be determined whether it is located. Optionally, the vehicle controller 114 may determine where the vehicle 102 is located based on the moving speed of the vehicle 102 and the speed traveled on the route 120, the distance traveled by the vehicle 102, and the known layout of the route 120. Can be determined. For example, the vehicle controller 114 can calculate how fast the vehicle 102 has moved from a known location (departure point or other location).

  The processor 116 can select a reference visual profile from the memory 118 associated with and representing the specified layout or arrangement of the route 120 at the position of the vehicle 102 when the image 200 was acquired. This designated layout or arrangement can represent the shape, spacing, or arrangement of the route 120 for safe driving of the vehicle 102. For example, the reference visual profile can represent the standard gauge and alignment of the rails 204 of the rails when the rails are laid or last inspected.

  In one aspect, the image analysis processor 116 can measure the standard gauge of the segment of the path 120 shown in the image 200 to determine whether the path 120 is offset. 3A and 3B are diagrams showing another example of the image 200 of the path 120 shown in FIG. Image analysis processor 116 can inspect image 200 and measure a standard gauge distance 500 between rails 204 in path 120. In one aspect, the analysis processor 116 has been identified as representing one or more pixels 202 identified as representing one rail 204 and another rail 204, as shown in FIGS. 3A and 3B. A linear distance or linear distance between one or more other pixels 202 can be measured. This distance represents the standard gauge distance 500 of the path 120. Alternatively, the distance between other pixels 202 may be measured. The processor 116 multiplies the number of pixels 202 by the known distance that the width of each pixel 202 represents in the image 200, using a known transformation element to calculate the number of pixels 202 within the standard gauge distance 500, Determine the standard gauge distance 500 by converting to a length (such as centimeters or metric units), by modifying the scale of the standard gauge distance 500 shown in the image 200 by magnification, or by other methods. be able to. In one aspect, the image analysis processor 116 may convert the image data into image data as wireframe model data or generate such image data, as described in the '294 application. it can. The standard gauge distance 500 may be measured between portions of the wireframe model data representing the rail.

  The measured standard gauge distance 500 can be compared to a specified standard gauge distance stored in memory 118 (or stored elsewhere) for the imaging portion of path 120. The specified standard gauge distance may be the reference visual profile of the path 120 if this distance represents a specified placement or spacing of the rails 204 of the path 120. If the measured standard gauge distance 500 is different from the specified standard gauge distance beyond a specified threshold or tolerance, the processor 116 determines that the segment of the path 120 shown in the image 200 has shifted. Can do. For example, the specified standard gauge distance may represent the distance of the path 120 or the standard gauge distance when the rail 204 is laid or last passed the inspection. If the specified standard gauge distance 500 deviates too much from the specified standard gauge distance, this deviation can represent a changed or modified standard gauge distance of the path 120.

  Optionally, the processor 116 may measure the standard gauge distance 500 several times when the vehicle 102 travels and monitor the measured standard gauge distance 500 for changes. If the standard gauge distance 500 changes beyond a specified amount, the processor 116 can identify the next segment of the path 120 as possibly misaligned. However, as will be described below, the measured change in the standard gauge distance 500 may alternatively represent a change in the route 120 on which the vehicle 102 travels.

  By measuring the standard gauge distance 500 of the path 120, the image analysis processor 116 determines that one or more of the rails 204 in the path 120 have shifted, even if the segment of the path 120 includes a curve. Can be possible. Since the standard gauge distance 500 should be constant or substantially constant (eg, within manufacturing tolerances), the standard gauge distance 500 is a curve or straight line segment of the path 120 as long as the path 120 does not deviate. Should not change significantly.

  In one embodiment, the image analysis processor 116 can monitor changes in the standard gauge distance 500 to determine if one or more rails 204 in the path 120 have shifted. The image analysis processor 116 can track the standard gauge distance 500 to determine if the standard gauge distance 500 exhibits a specified trend within a specified distance and / or amount of time. For example, the standard gauge distance 500 increases over at least a first specified period or distance and then decreases over a specified second period or decreases over at least a first specified period or distance. Then, if increased over at least a second specified period, the image analysis processor 116 may determine that the rail 204 has shifted. Optionally, the image analysis processor 116 may determine that the rail 204 has shifted in response to a standard gauge distance 500 that increases or decreases within a specified detection time or distance limit, as described above. Good.

  FIG. 12 is a diagram showing an example of the standard gauge distance 1200 determined by the image analysis processor 116 shown in FIG. The image analysis processor 116 can repeatedly determine the standard gauge distance 1200 from image data acquired while the vehicle 102 moves along the rail 204 of the route 120. If the rails 204 are not curved or bent with respect to each other, the standard gauge distance 1200 is constant or relatively constant (eg, a change in the standard gauge distance 1200 due to noise in the system rather than damage to the rails 204). Will remain. However, if one rail 204 is bent with respect to another rail 204, the standard gauge distance 1200 may change as the vehicle 102 moves over the bent rail 204. For example, if one rail 204 is bent away from another rail 204 (eg, has a convex curve), the bent rail 204 moves away from the other rail 204 and then moves toward the other rail 204. Therefore, the standard gauge distance 1200 may increase and then decrease. On the other hand, if one rail 204 is bent toward another rail 204 (eg, having a concave curvature), the bent rail 204 moves away from the other rail 204 and then away from the other rail 204. Due to movement, the standard gauge distance 1200 may decrease and then increase.

  In addition to or in addition to identifying standard gauge distance 1200 to one or more thresholds to identify damage to rail 204, image analysis processor 116 inspects standard gauge distance 1200. It may be determined whether the standard gauge distance 1200 is changing with time or distance according to one or more predetermined trends or patterns.

  In the illustrated example, the standard gauge distance 1200 is shown along a horizontal axis 1202 representing time or distance along the path 120 and a vertical axis 1204 representing different magnitudes (eg, lengths) of the standard gauge distance 1200. The image analysis processor 116 can examine the standard gauge distance 1200 to determine if the standard gauge distance 1200 has increased along the path 120 over at least one first specified period or distance 1206. The image analysis processor 116 determines that the standard gauge distance 1200 is determined by determining that the average, median, or moving average, or moving median, of the standard gauge distance 1200 has increased during the period or distance 1206. It can be determined that it has increased over a period or distance 1206. Alternatively, the image analysis processor 116 determines that the standard gauge distance 1200 is by determining that the best fit line, linear regression, or other trend or pattern 1208 at the standard gauge distance 1200 increases at least during the period or distance 1206. Can be determined to have increased over a period or distance 1206. The period or distance 1206 can be a variable that is adjusted to prevent noise in the system from being identified as an actual change in the standard gauge distance 1200. For example, noise may not increase (or decrease) the standard gauge distance 1200 over a period or distance that is at least as long as the period or distance 1206, while the actual change in the standard gauge distance 1200 is at least There is a possibility of increasing (or decreasing) over a period or distance of the same length as the period or distance 1206.

  In one embodiment, the image analysis processor 116 can identify sections of the path 120 that include the rails 204 that have an increasing trend or pattern over at least a period or distance 1206 if damaged. Alternatively, the image analysis processor 116 may continue to inspect the standard gauge distance 1200 for further changes. The standard gauge distance 1200 shown in FIG. 12 shows a decreasing trend or pattern 1210 after an increasing trend or pattern 1208. The image analysis processor 116 can identify that the trend or pattern 1210 is decreasing, as well as identifying that the trend or pattern 1208 is increasing. As with the increasing trend or pattern 1208, the image analysis processor 116 can determine whether the decreasing trend or pattern 1210 continues for at least as long as the second specified distance or period 1212. If the decreasing trend or pattern 1210 continues for at least the same length as the second distance or period 1212, the image analysis processor 116 indicates that the decreasing trend or pattern 1210 indicates a change in the standard gauge distance 1200, most or all. It can be determined that it is not caused by noise. Periods or distances 1206, 1212 may occur at different times or locations than those shown in FIG. For example, the period or distance 1206 may begin when the image analysis processor 116 determines that the standard gauge distance 1200 is increasing, while the period or distance 1212 is the image analysis processor when the standard gauge distance 1200 is decreasing. Once 116 determines, it may start. The periods or distances 1206, 1212 may be close to each other and may be separated by a longer time or distance than that shown in FIG.

  FIG. 13 is a diagram showing another example of the standard gauge distance 1300 determined by the image analysis processor 116 shown in FIG. The standard gauge distance 1300 is shown along the horizontal axis 1202 and the vertical axis 1204 described above. In contrast to the standard gauge distance 1200 shown in FIG. 12, the standard gauge distance 1300 does not include an increasing trend 1208 or a decreasing trend 1210 that lasts at least for a specified period or distance 1206, 1212. Instead, any increase or decrease in the standard gauge distance 1300 occurs over a shorter period or distance. As a result, the image analysis processor 116 may not identify the increasing trend 1208 and / or decreasing trend 1210 at the standard gauge distance 1300.

  The distance and / or length of the distance or period 1206, 1212 extends may vary based on the amount of noise in the system. For example, if the measured change in the standard gauge distance 1200, 1300 increases for reasons other than the actual change in the standard gauge distance 1200, 1300, the distance and / or the length of the distance that the period 1206, 1212 extends. Can be increased so that noise is not identified as the bent rail 204. As another example, if the measured change in the standard gauge distance 1200, 1300 decreases for reasons other than the actual change in the standard gauge distance 1200, 1300, the time and / or distance that the distance or period 1206, 1212 extends. The length of can be shortened.

  Returning to the description of the standard gauge distance 1200 shown in FIG. 12, the image analysis processor 116 indicates that the standard gauge distance 1200 has actually changed and that the rail 204 has an increasing trend or pattern 1208 followed by a decreasing trend or pattern. In response to identifying 1210, it can be determined that it has been bent or damaged. Optionally, the image analysis processor 116 is responsive to the fact that the standard gauge distance 1200 has actually changed and the rail 204 has identified a decreasing trend or pattern 1210 followed by an increasing trend or pattern 1208. Can be determined to be bent or damaged. In one embodiment, if an increasing trend 1208 is identified but is not followed by a decreasing trend 1210 (or if the decreasing trend 1210 is not followed by an increasing trend 1208), the image analysis processor 116 may cause the path 120 to be damaged. May not be identified.

  In one aspect, the image analysis processor 116 displays an increasing trend 1208 occurring within a specified time or distance limit 1214 followed by a decreasing trend 1210 (or decreasing trend 1210 followed by an increasing trend 1208). In response to the identification, the path 120 is identified as damaged. Since at least some bending in the rails 204 of the path 120 appears to have occurred over a relatively short distance (eg, a few feet or meters), the image analysis processor 116 may increase and decrease over a period or distance longer than the limit 1214. Subsequent decreasing patterns 1208, 1210 (or decreasing and subsequent increasing patterns 1210, 1208) may not be identified as representing the actual curvature of rail 204.

  FIG. 14 is a diagram illustrating another example of the standard gauge distance 1400 determined by the image analysis processor 116. A standard gauge distance 1400 is shown along the horizontal and vertical axes 1202 and 1204 described above. As shown in FIG. 14, the standard gauge distance 1400 shows an increasing trend or pattern 1408 followed by a decreasing trend or pattern 1410. The increasing trend 1408 lasts longer than the specified period 1206 and the decreasing trend 1410 continues longer than the specified period 1212. Period 1206 may begin when image analysis processor 116 identifies that standard gauge distance 1400 is increasing, and period 1212 is identified by image analysis processor 116 that standard gauge distance 1400 is decreasing. You can start with. Alternatively, the image analysis processor 116 can identify the start of the time period 1206, 1212 in another manner.

  However, neither the increasing trend nor the decreasing trend 1408, 1410 occurs within the specified time or distance limit 1214. For example, the period or distance at which each of trends 1408, 1410 occurs is the total time encompassed by periods 1206, 1212 (eg, the time or distance extending from the beginning of period 1206 to the end of period 1212). They are separated from each other in time or space by a sufficiently large time or distance that is longer than the limit 1214. Because the curvature or misalignment of the rail 204 may have occurred over a relatively short distance (eg, a few feet or a few meters), the magnitude of the time or distance limit 1214 is such that the limit 1214 is a curve or position in the rail 204. It can be set so as to cut off the change of the standard gauge distance 1400 that does not represent a deviation. For example, an increasing trend 1408 and a subsequent decreasing trend 1410 (or decreasing trend 1410 and subsequent increasing trend 1408) may be over a relatively large distance (eg, over several feet or meters, ie, over several feet or meters). Trends 1408, 1410 may not indicate a misalignment of rails 204 if they are separated in time or space by the amount of time encompassed by the traveling vehicle. Instead, trends 1408, 1410 may indicate drift in the system or other measurement problems.

  By way of example only, limit 1214 represents a spatial distance of 2 feet (eg, 0.6 meters), 4 feet (eg, 1.2 meters), 10 feet (eg, 3 meters), or another distance. be able to. Optionally, limit 1214 may represent the amount of time required for the vehicle to travel over a similar distance, depending on the speed at which the vehicle moves.

  Returning to the description of the system shown in FIG. 1, in one embodiment, as the camera faces forward along the direction of travel of the vehicle, the image analysis processor 116 measures the standard gauge distance and enters the next segment of the path 120. On the other hand, a bent or misaligned portion of the path 120 can be identified. If the image analysis processor 116 determines from inspection of one or more images 200 that the next segment of the path 120 toward which the vehicle 102 is heading, the image analysis processor 116 sends an alarm signal to the vehicle controller 114. Can communicate with. This alarm signal can indicate to the vehicle controller 114 that the next segment of the route 120 is shifted. In response to this alarm signal, the vehicle controller 114 can take one or more response actions. For example, the vehicle controller 114 can include an output device, such as a display or a speaker, that visually and / or audibly alerts the driver of the vehicle 102 about the next offset section of the path 120. In that case, the driver may proceed by decelerating or stopping the movement of the vehicle, or by communicating with a non-mounted repair or inspection facility and requesting further inspection and / or maintenance of the deviated segment of the route 120, etc. Can be determined. Optionally, the vehicle controller 114 automatically decelerates or stops the movement of the vehicle 102 and / or automatically communicates with non-mounted repair or inspection equipment to further offset the offset section of the path 120. The response operation may be performed automatically, for example, by requesting inspection and / or maintenance.

  FIG. 15 is a flowchart illustrating a method 1500 for identifying damage to a path, according to one embodiment. The method 1500 may be implemented in one aspect by the system 100 shown in FIG. The method 1500 is used to measure the standard gauge distance of the path 120 (shown in FIG. 1) and one of the plurality of rails 204 (shown in FIG. 2) bends relative to another rail 204 of the path 120. It is possible to determine whether the path 120 is damaged by being bent, curved, or misaligned. Method 1500 can represent an operation that can be encoded with instructions stored in a memory device (eg, memory 118 shown in FIG. 1, another memory accessible to processor 116 shown in FIG. 1, etc.), And / or wired to the image analysis processor 116 to configure the system 100 and perform the operations described herein.

  In 1502, image data representing a route along which the vehicle travels is acquired. As described above, the image data can be acquired by one or more cameras located inside and / or outside the vehicle. The image data can include a still image, a moving image, a wire frame model, or the like.

  At 1504, a standard gauge distance of the path is measured based on at least a portion of the image data. For example, the distance between the portions of the image data representing the rails of the path can be measured and adjusted to determine the lateral separation distance between the rails. The standard gauge distance can be tracked over time, and changes in the standard gauge distance can be identified at various locations and / or times while traveling along the route.

  At 1506, the measured standard gauge distance is examined to determine whether the standard gauge distance is increasing (eg, has an increasing tendency). For example, if the standard gauge distance is inspected and whether the standard gauge distance increases with movement of the vehicle along the route, separately from and / or in addition to noise in the standard gauge distance measurement Can be judged. If the standard gauge distance is increased, the increased spacing between the rails may indicate damage to the path (eg, bending or bending of at least one rail). As a result, the flow of method 1500 can proceed to 1508. On the other hand, if the standard gauge distance has not increased, the flow of method 1500 can proceed to 1518 and will be described below.

  At 1508, an increasing trend in standard gauge distance is examined to determine if the standard gauge distance has increased for at least a first specified period and / or distance. Increasing trends are compared to a specified period and / or distance to prevent temporary changes in standard gauge distance due to factors other than bent or damaged rails from being misidentified as bent or damaged rails. it can. The specified distance can be 6 inches (eg, 15 centimeters), 9 inches (eg, 23 centimeters, 1 foot (eg, 30 centimeters), or another distance. The time required for the vehicle to travel over a specified distance based on the speed of the vehicle.

  If the increasing trend in the standard gauge distance lasts at least as long as the first designated time and / or distance, the increasing trend can indicate damage to the path. As a result, the flow of method 1500 may proceed to 1510. On the other hand, if the increasing trend does not last at least as long as the first specified time and / or distance, the increasing trend may indicate another factor, such as noise in the system. As a result, the flow of method 1500 may return to 1502.

  At 1510, the measured standard gauge distance is inspected to determine whether the standard gauge distance is decreasing following an increasing trend (eg, having a decreasing trend). For example, by investigating the standard gauge distance, apart from and / or in addition to noise in the standard gauge distance measurement, the standard gauge distance tends to increase with the movement of the vehicle along the route. It can be determined whether or not the value decreases after being identified. If the standard gauge distance decreases after an increasing trend, the decrease interval between the rails will cause damage to the path (eg, at least one bend or curvature of the rail returning from a position associated with an increase in standard gauge distance). May show. As a result, the flow of method 1500 can proceed to 1512. On the other hand, if the standard gauge distance has not decreased, the flow of method 1500 can return to 1502.

  At 1512, the decreasing trend in the standard gauge distance is examined to determine if the standard gauge distance is decreasing for at least a second specified period and / or distance. The second designated period and / or distance may be the same as the first designated period and / or distance, or may be longer than the first designated period and / or distance, or It may be short. The decreasing trend is compared to a specified period and / or distance to prevent temporary changes in standard gauge distance due to factors other than bent or damaged rails from being erroneously identified as bent or damaged rails. it can. The specified distance can be 6 inches (eg, 15 centimeters), 9 inches (eg, 23 centimeters, 1 foot (eg, 30 centimeters), or another distance. The time required for the vehicle to travel over a specified distance based on the speed of the vehicle.

  If the decreasing trend in the standard gauge distance continues for at least the same length as the second specified time and / or distance, the decreasing trend can indicate damage to the path. As a result, the flow of method 1500 can proceed to 1514. On the other hand, if the decreasing trend does not last at least as long as the second specified time and / or distance, the decreasing trend may indicate another factor, such as noise in the system. As a result, the flow of method 1500 may return to 1502.

  At 1514, a determination is made as to whether the first and second specified time periods and / or distances (where increasing and decreasing trends occur) occurred within specified outer distances and / or time limits. Method 1500 can include this determination to ensure that the change in increase or decrease in standard gauge distance occurs relatively close to each other. If the increase / decrease changes do not occur relatively close to each other (eg, within specified limits), the increase / decrease changes may not indicate damage to the path. As a result, the flow of method 1500 may return to 1502. On the other hand, if the increase / decrease changes occur relatively close to each other (eg, within specified limits), the increase / decrease changes may indicate damage to the path. As a result, the flow of method 1500 can proceed to 1516.

  At 1516, the segment of the path that has been identified as having a change in standard gauge distance is identified as a damaged segment of that path. As described herein, one or more response or countermeasure actions are then taken to a non-mounted position that automatically decelerates or stops vehicle movement, requires inspection, repair, or maintenance on the route. It can be taken such as by communicating a signal or by communicating a warning signal to one or more other vehicles. Additional image data of the path can continue to be acquired and investigated to monitor the standard gauge distance and / or to identify the damage category of the path. For example, the method 1500 can return to 1502.

  Returning to the description of the method 1500 at 1506, if no increasing trend is found in the standard gauge distance, the flow of the method 1500 can proceed to 1518. At 1518, the measured standard gauge distance is inspected to determine if the standard gauge distance is decreasing (eg, has a decreasing tendency). For example, if it is determined that the standard gauge distance does not have an increasing trend (which may result from one rail bending away from the other rail), the method 1500 may be It can be investigated whether the rail has a decreasing tendency (which may result from bending towards the other rail).

  Investigate the standard gauge distance to determine whether the standard gauge distance decreases with the movement of the vehicle along the route, separately from and / or in addition to noise in the standard gauge distance measurement. can do. If the standard gauge distance is decreasing, decreasing the spacing between the rails may indicate damage to the path (eg, bending or bending of at least one rail). As a result, the flow of method 1500 can proceed to 1520. On the other hand, if the standard gauge distance has not decreased, the flow of method 1500 can return to 1502.

  At 1520, the decreasing trend in the standard gauge distance is examined to determine if the standard gauge distance is decreasing for at least a second specified period and / or distance. As described herein, the decreasing trend is compared to a specified period and / or distance, and a temporary change in the standard gauge distance due to factors other than the bent or damaged rail is false as a bent or damaged rail. Can be prevented from being identified. If the decreasing trend in the standard gauge distance continues for at least the same length as the second specified time and / or distance, the decreasing trend can indicate damage to the path. As a result, the flow of method 1500 can proceed to 1522. On the other hand, if the decreasing trend does not last at least as long as the second specified time and / or distance, the decreasing trend may indicate another factor, such as noise in the system. As a result, the flow of method 1500 may return to 1502.

  At 1522, the measured standard gauge distance is inspected to determine whether the standard gauge distance is increasing following a decreasing trend (eg, having an increasing trend). If the standard gauge distance increases after a decreasing trend, the increased spacing between the rails causes damage to the path (eg, at least one bend or curvature of the rail returning from a position associated with a decrease in the standard gauge distance). May show. As a result, the flow of method 1500 can proceed to 1524. On the other hand, if the standard gauge distance has not increased, the flow of method 1500 can return to 1502.

  At 1524, the increasing trend in the standard gauge distance is examined to determine whether the standard gauge distance has increased for at least the first specified period and / or distance, as described above. Increasing trends are compared to a specified period and / or distance to prevent temporary changes in standard gauge distance due to factors other than bent or damaged rails from being misidentified as bent or damaged rails. it can. If the increasing trend in the standard gauge distance lasts at least as long as the first designated time and / or distance, the increasing trend can indicate damage to the path. As a result, the flow of method 1500 can proceed to 1526. On the other hand, if the increasing trend does not last at least as long as the first specified time and / or distance, the increasing trend may indicate another factor, such as noise in the system. As a result, the flow of method 1500 may return to 1502.

  At 1526, a determination is made as to whether the first and second specified time periods and / or distances (where increasing trends and decreasing trends occur) occur within specified outer distances and / or time limits. If the increase / decrease changes do not occur relatively close to each other (eg, within specified limits), the increase / decrease changes may not indicate damage to the path. As a result, the flow of method 1500 may return to 1502. On the other hand, if the increase / decrease changes occur relatively close to each other (eg, within specified limits), the increase / decrease changes may indicate damage to the path. As a result, the flow of method 1500 can proceed to 1516, described above.

  FIG. 4 is a diagram illustrating an example of the reference visual profile 300. The reference visual profile 300 shows the path 120 (shown in FIG. 1), such as where the path 120 is expected to be in an image acquired by one or more of the cameras 106 (shown in FIG. 1). Represents the specified layout.

  In the illustrated example, the reference visual profile 300 includes two designated areas 302, 304 that represent designated positions of the rails of the track. The designated areas 302, 304 are the pixels 202 (shown in FIGS. 2A and 2B) of the image 200 (shown in FIGS. 2A and 2B) representing the rails 204 (shown in FIGS. 2A and 2B), and the rails 204 are appropriately When aligned, it can represent the location where it should be placed. For example, the designated areas 302, 304 can represent the expected position of the rail 204 before acquiring the image 200. Rails 204 should be properly aligned when rails 204 have been laid or have passed the inspection of the position of rails 204, or at least in the same position as when they are within specified tolerances. Can do. This specified tolerance may represent a range of positions where the rail 204 may appear in the image 200 due to a shake or other movement of the vehicle 102 (shown in FIG. 1).

  Optionally, reference visual profile 300 may represent a previous image of path 120 acquired by camera 106 on the same or different vehicle 102. The designated areas 302, 304 can represent the location of the pixel 202 in the previous image identified as representing the path 120 (eg, the rail 204).

  In one aspect, the image analysis processor 116 can map the pixels 202 representing the path 120 (eg, the rail 204) to the reference visual profile 300, or the designated areas 302, 304 of the reference visual profile 300 can be mapped to the path 120. Can be mapped to a pixel 202 representing. This mapping may include determining whether the position of the pixel 202 representing the path 120 (eg, rail 204) in the image 200 is in the same position as the designated areas 302, 304 of the reference visual profile 300.

  5A and 5B illustrate a visual mapping diagram 400 of an image 200 and a reference visual profile 300 according to an example of the inventive subject matter described herein. The mapping diagram 400 shows an example of a comparison between the image 200 and the reference visual profile 300 performed by the image analysis processor 116 (shown in FIG. 1). As shown in the mapping diagram 400, the designated areas 302 and 304 of the reference visual profile 300 can be superimposed on the image 200. The processor 116 can then identify the difference between the image 200 and the reference visual profile 300. For example, the processor 116 can determine whether the pixel 202 representing the path 120 (eg, representing the rail 204) is located outside the designated areas 302, 304. Optionally, the processor 116 may not locate the locations of the pixels 202 that represent the path 120 in the image 200 (eg, the coordinates of these pixels 202) within the designated regions 302, 304 (eg, the designated regions 302, 304). The coordinates are not within the outer boundary of

  If the image analysis processor 116 determines that at least the specified amount of pixels 202 representing the path 120 are outside the specified areas 302, 304, the processor 116 determines that the section of the path 120 shown in the image 200 is misaligned. Can be identified. For example, the processor 116 may identify a group 402, 404, 406 of pixels 202 that represents the path 120 (eg, the rail 204) as being outside the designated area 302, 304. The number, fraction, percentage, or measurement of the pixel 202 that represents the path 120 and is outside the specified area 302, 304 exceeds a specified threshold (eg, 10%, 20%, 30%, or another amount) If so, the segment of the path 120 shown in the image 200 is identified as being shifted. On the other hand, if the number, fraction, percentage, or measurement of the pixel 202 that represents the path 120 and is outside the designated area 302, 304 does not exceed the threshold, the segment of the path 120 shown in the image 200 is shifted. Not identified.

  While the vehicle 102 is traveling across various sections of the route 120, the vehicle 102 may encounter (eg, approach) an intersection between a section of the traveling route 120 and another route section. There is sex. For rail vehicles, such an intersection can include a switch between two or more routes 120. Due to the arrangement of the rails 204 at the switching unit, the image analysis processor 116 may apply an examination of the image 200 to determine whether the rails 204 are displaced.

  FIG. 6 is a schematic diagram of an intersection (eg, a switching unit) 600 between two or more paths 602, 604 in accordance with an example of the inventive subject matter described herein. One or more of the paths 602, 604, or each may be the same as or similar to the path 120 shown in FIG.

  When the image analysis processor 116 measures the standard gauge distance 500 (shown in FIGS. 3A and 3B) and determines that the rails 204 of the paths 602 and 604 are displaced, the image analysis processor 116 indicates that the vehicle 102 is the switching unit. As the value approaches 600, a reduced standard gauge distance 500 can be identified. For example, when the vehicle 102 is traveling toward the switching unit 600 on the route 602 along the first traveling direction 606, or the vehicle 102 is traveling along the second traveling direction 608 on the route 604. When the vehicle 102 is traveling toward the switching unit 600 on the route 602 along the third traveling direction 610, the image analysis processor 116 determines the measured standard gauge distance. It can be determined that 500 is decreasing from the distance 500a to the short distance 500b, or to another distance or the like.

  If it is not known that the vehicle 102 is approaching the switching unit 600, the image analysis processor 116 can erroneously identify that the rail 204 is misaligned based on this decrease in the measured standard gauge distance 500. There is sex. However, in one aspect, the vehicle controller 114 is positioned at the position of the vehicle 102 as determined by the known position of the controller 114 and the switch 600 (eg, from a map or track database that provides the switch position). Based on this, the case where the vehicle 102 is approaching the switching unit 600 can be determined and notified to the image analysis processor 116. In that case, the image analysis processor 116 may wait until the vehicle 102 passes or passes through the switching portion 600, such as by not performing one or more of the response actions described above in response to the reduced measured standard gauge distance 500. The reduced standard gauge distance 500 may be ignored.

  The image analysis processor 116 may obtain one or more reference visual profiles from the memory 118 (shown in FIG. 1) that represent a path at or near the switching unit 600. Instead of representing parallel rails 204, these reference visual profiles can represent the placement of rails 204 at switching unit 600. In that case, the image analysis processor 116 compares the image of the route approaching the switching unit 600 with the reference visual profile, and determines whether the route at or near the switching unit 600 is shifted. Can do.

  Optionally, the image analysis processor 116 may determine that the vehicle 102 is approaching the switching unit 600 based on an acquired image of a route approaching the switching unit 600. For example, the distance between the rails 204 of different paths 602 and 604 approaching the switching unit 600 (for example, the standard gauge distance 500b) can be stored in the memory 118 as the reference visual profile. If the image analysis processor 116 determines that the standard gauge distance 500 measured from the image of the path 602 or 604 is the same as or equivalent to the stored standard gauge distance, the image analysis processor 116 switches the vehicle 102. It can be determined that the unit 600 is approaching. The image analysis processor 116 is used to determine when the vehicle 102 has approached the switching unit 600, confirm the position of the vehicle 102 as determined by the vehicle controller 114, and the controller 114 determines the position of the vehicle 102. When it is impossible to determine, it is possible to assist in specifying the position of the vehicle 102.

  In one aspect, the image analysis processor 116 can create a reference visual profile from image data obtained from a camera. For example, it is contemplated that the image analysis processor 116 may not have access to the reference visual profile, the segment of the path being investigated may not be associated with the reference visual profile, and so forth. The image analysis processor 116 may use the image data to create a reference visual profile “on the fly”, such as by creating a reference visual profile when the image data is acquired. In that case, the reference visual profile can be used to examine the image data from which the reference visual profile was created to identify problems in the path.

  FIG. 10 shows a camera acquired image 1000 with a reference visual profile 1002, 1004 of path 120, according to another example of the inventive subject matter described herein. The reference visual profiles 1002, 1004 are created by the image analysis processor 116 (shown in FIG. 1) from the image data used to create the image 1000. For example, the image analysis processor 116 can examine the intensity of the pixels to determine the location of the path 120 as described above. Within the location of the path 120, the image analysis processor 116 may find two or more pixels that have the same or equivalent intensity (eg, within a specified range of each other). Optionally, the image analysis processor 116 can identify more pixels that have the same or equivalent intensity.

  In that case, the image analysis processor 116 determines the relationship between these pixels. For example, the image analysis processor 116 can identify lines between pixels in the image 1000 for each rail 204. These lines represent the reference visual profiles 1002, 1004. The image analysis processor 116 then selects other pixels representing the rails 204 of the path 120 on the reference visual profile 1002, 1004 or within the reference visual profile 1002, 1004 (eg, the reference visual profile 1002, 1004 is designated). Or whether these pixels are outside the reference visual profile 1002, 1004. In the illustrated example, most or all of the pixels representing the rails 204 of the path 120 are on or within the reference visual profile 1002, 1004.

  FIG. 11 shows another camera acquired image 1100 with a reference visual profile 1102, 1104 of path 120 according to another example of the inventive subject matter described herein. Reference visual profiles 1102, 1104 may be created using image data used to form image 1100, as described above in connection with FIG. However, in contrast to the image 1000 shown in FIG. 10, the section 1106 of the path 120 does not fall on or within the reference visual profile 1104. This section 1106 is bent away from the reference visual profile 1104. The image analysis processor 116 can identify this section 1106 because the pixel having the intensity representing the rail 204 is no longer on or in the reference visual profile 1104. Therefore, the image analysis processor 116 can identify the section 1106 as the position shift section of the path 120.

  In one aspect, the image analysis processor 116 can use a combination of the techniques described herein to investigate the path. For example, if both rails 202, 204 of path 120 are bent or misaligned from their previous positions, but are still parallel or substantially parallel to each other, the standard between rails 202, 204 The gauge distance may be the same or substantially the same and / or may not be substantially different from the specified standard gauge distance 500 of the path 120. As a result, simply looking at the standard gauge distance in the image data may prevent the image analysis processor 116 from identifying damage (eg, bends) to the rails 202,204. To avoid this situation, the image analysis processor 116 further uses the image data to generate a reference visual profile 1102, 1104, which can be used as described above in connection with FIGS. It can be compared with the image data of the rail. In that case, a bend or other misalignment of the rails 202, 204 can be identified if the bend in the rails 202, 204 deviates from the reference visual profile created from the image data.

  FIG. 7 is a flowchart of a method 700 for investigating a route from a vehicle when the vehicle moves along the route. The method 700 may be performed by one or more embodiments of the route survey system 100 (shown in FIG. 1). At 702, a route image is obtained from one or more cameras of the vehicle. The image can be acquired for a segment of a route ahead of the vehicle (for example, the vehicle is moving toward the segment to be imaged) along the traveling direction of the vehicle.

  At 704, a reference visual profile of the route is selected based on the location of the imaged route segment. As noted above, the reference visual profile represents the specified standard gauge distance of the route, a previous image of the route, or a spatial representation of the location where the route is expected or previously located. Can do.

  At 706, the image is compared to a reference visual profile. For example, the standard gauge distance of the rail in the path image can be measured and compared to the standard gauge distance of the reference visual profile. Optionally, the position of the rail in the image may be determined and compared to the position of the rail in the previous image of the path. In one aspect, the position of the rail in the image is determined and compared to a designated region of the reference visual profile.

  At 708, it is determined whether there is a difference between the image of the route and the reference visual profile. For example, it can be determined whether the standard gauge distance measured from the image is different from the specified standard gauge distance of the reference visual profile. Additionally or alternatively, it can be determined whether the position of the rail in the image is different from the position of the rail in the previous image of the path. Optionally, it may be determined whether the position of the rail in the image is outside the designated area in the reference visual profile. If one or more of these differences are identified, the difference is that the path (eg, one or more rails) has shifted due to bending, movement towards the ground or the underlying ballast material, or breaking. There is a possibility to show that.

  If one or more differences between the image and the reference visual profile are identified, the path may have deviated from the previous or specified position. As a result, the flow of method 700 can proceed to 710. On the other hand, if no difference is identified, or if the difference is relatively small or minor, the path is still in the same position as the previous or specified position (or by a relatively small amount) Moved). As a result, the vehicle can continue to travel along the next segment of the route and the method 700 can return to 702.

  At 710, the path segment in the image is identified as being shifted. At 712, an alarm signal is communicated to one or more other rail vehicles to alert other vehicles of misalignment, and a device along the track communicates the alarm signal to one or more other rail vehicle systems. Communication of alarm signals to one or more devices along the track that are installed on or near the track, communicate alarm signals to non-equipped equipment, One or more response actions can be performed, such as by automatically decelerating or stopping, or notifying the boarding driver about the misalignment. Depending on whether the vehicle can continue to move along the route, the flow of method 700 can return to 702.

  In another aspect of the inventive subject matter described herein, an optical path survey system and method are mounted in front of a vehicle and are arranged in a direction (e.g., opposite) in which the path travels. Can use the camera. The camera captures images at a relatively high (eg, fast) frame rate, resulting in a static stable image of the path. Using multiple acquired images, the images are analyzed to identify and / or enhance obstacles (eg, pedestrians, cars, trees, etc.). The system and method can alert a vehicle driver about an obstacle or give instructions to trigger a braking action (manually or automatically). If the driver does not take any action to decelerate or step on the vehicle brake, the brake may be automatically activated without the driver's intervention.

  The camera can capture images at a relatively high frame rate (eg, at a relatively fast frequency), resulting in a static and stable image of the next position in the path of travel. There may be a time delay, i.e., lag (e.g., a few milliseconds) between capture times for images acquired with different cameras. In one aspect, images captured from different cameras in the same time frame (eg, within the same relatively short time frame) to identify foreign objects on or near the next segment of the path Compared to Feature detection algorithms can be used to identify important features on the image, such as people, birds, cars, and other vehicles (eg, locomotives). In one aspect, the image can be analyzed and used to identify the depth of the foreign object, estimate the size of the foreign object, and / or identify the foreign object. Different techniques can be used to remove or ignore non-stable obstacles such as snow, rain, and pebbles. Major obstacles such as cars and pedestrians on the track can be identified or highlighted and used to alert the vehicle driver about the presence of the major obstacles.

  Currently, train drivers may not receive warnings or identification results quickly enough for obstacles on the next segment of the track in various weather conditions. Even if the driver can see an obstacle, it may not have found it so quickly that the driver steps on the brakes and stops the train (or other vehicle) before colliding with the obstacle. is there. Collisions with obstacles can be avoided if the advanced image capture analysis techniques described herein can detect obstacles far enough away.

  Returning to the description of the route survey system 100 shown in FIG. 1, the one or more cameras 106 acquire several images 200 of the next segment of the route 120 while the vehicle 102 is moving along the route 120. can do. The following description focuses on the case where the image 200 is acquired by two or more cameras 106, but may optionally be acquired by only one camera 106. The image analysis processor 116 controls the camera 106 to at least 300 images per second for each camera, 120 images per second for each camera, 72 images for each camera, 48 images for each camera, 48 images for each camera, 24 images for each camera, Alternatively, the image 200 can be acquired at a relatively fast frame rate, such as by acquiring images at other rates.

  The image analysis processor 116 then compares the images acquired with one or more cameras 106 to identify differences in the images. These differences can represent temporary or permanent foreign objects on or near the section of the route 120 on which the vehicle 102 is traveling. A temporary foreign object is an object that is moving fast enough that the object does not interfere with or collide with the vehicle 102 when the vehicle 102 reaches the foreign object. A permanent foreign object is an object that is stationary or moving sufficiently slowly so that when the vehicle 102 reaches the foreign object, the vehicle 102 collides with the foreign object.

  FIG. 8 is a superimposed display 800 of three images acquired by one or more of the cameras 106 and superimposed on each other according to an example of the inventive subject matter described herein. The superimposed display 800 represents three images of the same section of the path 120 taken at different times by one or more cameras 106 and combined with each other. The image analysis processor 116 may or may not generate such a superimposed display when examining an image for a foreign object.

  As shown in the display 800, the path 120 is a permanent object in that the path 120 stays at the same or substantially the same position in images acquired at different times. This is because when the vehicle 102 (shown in FIG. 1) travels along the route 120, the route 120 does not move laterally with respect to the traveling direction of the vehicle 102. The image analysis processor 116 can identify the path 120 by examining the intensity of the pixels in the image as described above or using another technique.

  In addition, as shown in the display 800, the foreign object 802 is displayed in the image. The image analysis processor 116 examines the intensity of the pixels in the image (or uses another technique) and one of the pixels having the same or equivalent intensity (eg, within a specified range). The foreign object 802 can be identified by determining that the above groups are displayed at the positions of images close to each other. Optionally, image analysis processor 116 compares one or more of the images acquired by one or more cameras 106 and compares the image to one or more reference visual profiles, as described above. Can do. If differences between the image and the reference visual image are identified, the image analysis processor 116 may identify these differences as representing the foreign object 802. For example, if the reference visual profile represents only the rail 204, but the rail 204 and another object are displayed in the image, the image analysis processor 116 may identify the other object as a foreign object 802. In one aspect, the image analysis processor 116 can distinguish between the path 120 (eg, the rail 204) and the foreign object 802 by different shapes and / or sizes of the path 120 and the foreign object 802.

  When the foreign object 802 is identified, the image analysis processor 116 can instruct one or more of the cameras 106 to display the foreign object 802 in an enlarged manner and obtain one or more enlarged images. For example, the initial identification of the foreign object 802 can be confirmed by the image analysis processor 116 instructing the camera 106 to enlarge the field of view of the camera 106 and obtain an enlarged image of the foreign object 802. The image analysis processor 116 can check the enlarged image again to confirm the presence of the foreign object 802 or determine that the foreign object 802 does not exist.

  The image analysis processor 116 examines a series of two or more images (eg, an enlarged image or an image acquired prior to enlargement) and the foreign object 802 is a permanent object or a temporary object. You may judge whether there is. In one aspect, a foreign object 802 is identified by the processor 116 as a permanent object if the foreign object 802 is displayed in at least a specified number of images and identified by the processor 116 within a specified time period. The fact that the foreign object 802 is seen in the specified number of images (or more images) in at least the specified period means that the foreign object 802 is located on or near the next segment of the path 120. , And / or likely to stay on or near path 120.

  For example, birds flying on the route 120 and rainfall on the route 120 may be displayed in one or more images acquired by the camera 106. Since these foreign objects 802 tend to move at a considerably high speed, there is little possibility that these foreign objects 802 are displayed in an image exceeding a specified number of images for a specified period. As a result, the image analysis processor 116 does not identify these types of foreign objects 802 as permanent objects, but instead ignores these foreign objects or identifies them as temporary objects.

  As another example, a person standing or walking on the route 120 and a car that is still on the route 120 or moving slowly, such as a flying bird or a longer period of time than the rain 106 may be displayed in the acquired image. As a result, a person or car may be displayed in at least a specified number of images for at least a specified period of time. The image analysis processor 116 identifies such foreign objects as permanent objects.

  In response to identifying the foreign object as a permanent object, the image analysis processor 116 can perform one or more mitigation actions. For example, the image analysis processor 116 can generate an alarm signal (shown in FIG. 1) that is communicated to the vehicle controller 114. This alarm signal can generate one or more alarm devices, such as internal and / or external sirens, to generate an audible warning or alarm that the vehicle 102 is approaching a permanent object. Optionally, the warning signal may generate a visual or other warning to the driver of the vehicle 102 to notify the driver about a permanent object. Additionally or alternatively, a warning signal can cause the vehicle controller 114 to automatically activate the brakes of the vehicle 102. In one aspect, the alarm signal allows the vehicle controller 114 to communicate a signal to a switching unit or other device along the track that controls the switching unit, where the switching unit is automatically changed and the vehicle 102 is Leave the current traveling route 120 (a permanent object has been detected) and move to another, different route to avoid collision with the permanent object.

  In one example of the inventive subject matter described herein, the image analysis processor 116 can determine a permanent object movement speed and, if present, determine which mitigation actions should be performed. . In the example shown in FIG. 8, the foreign object 802 is displayed at various positions on the image with respect to the path 120. For example, in the first image, the foreign object 802 is displayed at the first position 804, and in the subsequent second image, the foreign object 802 is displayed at a different second position 806, and in the subsequent third image. The foreign object 802 is displayed at a different third position 808.

  The image analysis processor 116 can identify the change position of the foreign object 802 and estimate the moving speed of the foreign object 802. For example, the image analysis processor 116 can control the frame rate of the camera 106, and thus know the length of time that successive images were acquired. Image analysis processor 116 measures changes in the position of foreign object 802 between different positions 804, 806, 808, etc., and scales the change in these positions to an estimated distance that foreign object 802 has moved between the images. be able to. For example, the image analysis processor 116 can estimate the distance in the same way as measuring the standard gauge distance 500 shown in FIGS. 3A and 3B. However, instead of measuring the distance between the rails 204, the image analysis processor 116 estimates the travel distance of the foreign object 802.

  The image analysis processor 116 can estimate the moving speed at which the foreign object 802 is moving, using the change in position divided by the period in which images indicating various positions of the foreign object 802 are acquired. When the foreign object 802 is moving slower than the designated speed, the image analysis processor 116 can determine that the foreign object 802 may block the route 120 before the vehicle 102 reaches the foreign object 802. As a result, the image analysis processor 116 may respond more quickly, such as by immediately actuating the brake of the vehicle 102 (eg, to slow down and stop the movement of the vehicle 102 completely or sufficiently). Can generate an alarm signal for the vehicle controller 114. When the foreign object 802 is moving at least at the same speed as the designated speed, the image analysis processor 116 determines that there is a high possibility that the foreign object 802 has been removed from the route 120 before the vehicle 102 reaches the foreign object 802. Can do. As a result, the image analysis processor 116 automatically decelerates (but does not stop) the vehicle 102 by activating the alarm siren, automatically suppressing the throttle level, and / or activating the brake. For example, an alarm signal for the vehicle controller 114 may be generated that requests a response that does not require an emergency.

  In one embodiment, the image analysis processor 116 uses images acquired by two or more cameras 106 to confirm or deny potential identification of permanent objects on or near the path 120. be able to. For example, the processor 116 examines a first set of images from one camera 106a, examines a second set of images from another camera 106b, and the permanent object becomes a first image set. It can be determined whether both the set and the second set of images have been identified. If a permanent object is detected from both sets of images, the image analysis processor 116 can determine which mitigation action to perform as described above.

  The image analysis processor 116 can examine images acquired by two or more cameras 106 to estimate the depth of the foreign object 802. For example, images acquired by different, spaced cameras 106 at the same time or approximately the same time can result in a stereoscopic view of the foreign object 802. Because of the slightly different field of view of the camera 106, images acquired at the same or approximately the same time may have slight differences in the relative position of the foreign object 802, even when the foreign object 802 is stationary. is there. For example, the foreign object 802 may be slightly displayed on one side of an image acquired by one camera 106a as compared to an image acquired by another camera 106b. The image analysis processor 116 measures these differences (eg, by measuring the distance between common pixels or positions of the foreign object 802) and determines the depth of the foreign object 802 (eg, parallel or coaxial with the travel direction of the vehicle 102). The distance between the opposite sides of the foreign object 802 can be estimated along the direction. For example, if these differences are larger, a greater depth may be estimated than if the differences are small.

  The image analysis processor 116 can use the estimated depth to determine which mitigation action should be performed. For example, when the estimated depth is relatively large, the image analysis processor 116 can determine that the size of the foreign object 802 is larger than when the estimated depth is relatively small. The image analysis processor 116 may request a more strict mitigation action if the estimated depth is relatively large, and may require a less strict mitigation action if the estimated depth is relatively small. Can do.

  Additionally or alternatively, the image analysis processor 116 may examine the two-dimensional dimensions of the identified foreign object 802 in one or more images to determine which mitigation action should be performed. For example, the image analysis processor 116 can measure the surface area of the image representing the foreign object 802 in the image. The image analysis processor 116 can determine the size index of the foreign object 802 by combining this two-dimensional dimension of the foreign object 802 in the image and the estimated depth of the foreign object 802. The size index represents the size of the foreign material 802. Optionally, the size index may be based on the two-dimensional size of the imaged foreign object 802 and may not be based on the estimated depth of the foreign object 802.

  The image analysis processor 116 can use the size index to determine which mitigation actions should be performed. The image analysis processor 116 can request a more strict mitigation action when the size index is relatively large, and can request a less strict mitigation action when the size index is relatively small.

  The image analysis processor 116 can compare the two-dimensional region and / or estimated depth of the foreign object 802 with one or more object templates to identify the foreign object 802. The object template can be the same as the designated areas 302 and 304 shown in the reference visual image 300 in FIGS. 5A and 5B. As described above, the designated areas 302, 304 represent places where properly aligned rails 204 are expected to be placed in the image. Similar designated areas can represent the shape of other objects, such as pedestrians, cars, or livestock. Image analysis processor 116 may determine the size and / or shape of foreign object 802 in one or more images, the size and / or size of one or more designated areas (eg, object templates) that represent one or more different foreign objects. It can be compared with the shape. If the size and / or shape of the foreign object 802 is the same or equivalent (eg, within a specified tolerance), the image analysis processor 116 represented the foreign object 802 in the image represented by the object template. It can be identified as the same foreign object.

  The image analysis processor 116 can use the identification result of the foreign object 802 to determine which mitigation action should be performed. For example, if the foreign object 802 is identified as a car or pedestrian, the image analysis processor 116 may request a mitigation action more strictly than if the foreign object 802 is identified as something else, such as livestock.

  In one aspect, the image analysis processor 116 stores one or more of the images in the memory 118 and / or communicates the images to an unloaded location. Images are retrieved from the memory 118 and / or from non-mounted locations and the same segment of the path 120 obtained by the same vehicle 102 at different times and / or by one or more other vehicles 102 at other times. May be compared with one or more images. Using changes in the image of the path 120, such as the change in the path 120 over time, as identified in the image, due to wear and tear in the path 120, or spilling ballast material under the path 120, etc. The degradation of the path 120 can be identified.

  FIG. 9 shows a flowchart for a method 900 for investigating a route from a vehicle as the vehicle moves along the route. The method 900 may be performed by one or more embodiments of the route survey system 100 (shown in FIG. 1). At 902, multiple images of the route are obtained from one or more cameras of the vehicle. The image can be acquired for a segment of a route ahead of the vehicle (for example, the vehicle is moving toward the segment to be imaged) along the traveling direction of the vehicle.

  At 904, the image is examined to determine whether foreign material is present in one or more of the images. For example, the intensity of the pixels in the image can be examined to determine whether there is a foreign object on or near the segment of the path that the vehicle is approaching.

  At 906, it is determined whether a foreign object has been identified in the image. For example, if a foreign object is compared to a previous image or other reference visual profile and the shape of the object is in the current image but not in the previous image or other reference visual profile, the object may represent a foreign object There is sex. As a result, foreign objects are identified in the image and the flow of method 900 can proceed to 908. On the other hand, if no foreign material is identified in the image, the flow of method 900 will return to 902.

  In one aspect, the presence of a foreign object may be determined by examining a first set of images acquired by the first camera and a second set of images acquired by the second camera. If the foreign object is identified in the first set of images and the foreign object is identified in the second set of images, the flow of method 900 may proceed to 908. Otherwise, the flow of method 900 may return to 902.

  In one aspect, the presence of a foreign object may be determined by examining different images acquired at different magnification levels. For example, if a foreign object is identified by one or more images acquired at a first magnification level, the camera zooms in on the foreign object and acquires one or more images at the enlarged second magnification level. Also good. The image at the magnification level can be examined to determine whether there is a foreign object in the image. If the foreign object is identified in the magnified second magnification level image, the flow of method 900 can proceed to 908. Otherwise, the flow of method 900 may return to 902.

  In 910, it is determined whether the foreign object is a permanent object or a temporary object. As described above, two or more consecutive images of the path can be examined to determine whether foreign matter is present in the image. If a foreign object is displayed in at least a specified number of images in at least a specified period, the foreign object may be identified as a permanent object as described above. As a result, one or more mitigation actions may need to be taken to avoid collision with the foreign object, and the flow of method 900 may proceed to 912.

  On the other hand, if no foreign object is displayed in at least the specified number of images for at least the specified period, the foreign object may be a temporary object as described above and may not be identified as a permanent object. There is. As a result, when the vehicle reaches the position of a foreign object, there may be no foreign object and there may be no need to take one or more mitigation actions. In that case, the flow of method 900 may return to 902.

  At 912, one or more mitigation actions can be taken. For example, a vehicle driver can be alerted to the presence of a foreign object, can activate an audible and / or visual alarm, can automatically apply a vehicle brake, or can turn on a vehicle throttle. It can be reduced. As described above, the size, depth, and / or identification information of a foreign object can be determined and used to select which mitigation action to perform.

  In one example of the inventive subject matter described herein, a method (e.g., for optically exploring a path, such as a track), includes a railroad vehicle moving along the track, and a reference visual profile of that track segment. Acquiring (by one or more computer processors) one or more images of a section of the track from a camera attached to the railway vehicle. The reference visual profile represents the designated layout of the track. The method also compares (at one or more computer processors) one or more images of the track segment with the reference visual profile of the track, and between the one or more images and the reference visual profile. Identifying one or more differences as line misregistration sections (with one or more computer processors) may be included.

  In one aspect, the one or more images of the track segment map one or more image pixels to corresponding locations in the reference visual profile, and the one or more image pixels representing the track are within the reference visual profile. Is compared to the reference visual profile by determining whether they are in a common position as the track.

  In one aspect, the method identifies the one or more image portions representing a line by measuring the intensity of pixels in the one or more images, and the one or more image portions representing the line. Distinguishing from other portions of one or more images based on pixel intensity.

  In one aspect, the reference visual profile visually represents the location where the track was before acquiring one or more images.

  In one aspect, the method includes measuring a distance between rails of the track by determining the number of pixels disposed between the rails in one or more images.

  In one aspect, the method also includes comparing the distance to a specified distance and identifying a standard gauge change that has occurred in the section of the track.

  In one aspect, the method also includes identifying a switch in that section of the line by identifying changes in the number of pixels disposed between the rails in the one or more images.

  In one aspect, the method also includes creating a reference visual profile from at least one of the one or more images compared to the reference visual profile and identifying one or more differences.

  In one aspect, the method also includes one or more images of the track segment and one or more additional images of the track segment acquired by one or more other rail vehicles at one or more other times. And identifying the degradation of the section of the line.

  In one aspect, one or more images of the track segment are acquired while the railway vehicle is traveling at the upper limit speed (eg, track speed) of the track segment.

  In another example of the inventive subject matter described herein, a system (eg, an optical path survey system) includes a camera and one or more computer processors. The camera is attached to the railway vehicle and is configured to acquire one or more images of a section of the track while the rail vehicle is moving along the track. The one or more computer processors are configured to select a reference visual profile for the segment of the track that represents the designated layout of the track. The one or more computer processors also compare one or more images of the section of the track with the reference visual profile of the track, and determines one or more differences between the one or more images and the reference visual profile, It is configured to be identified as a line misalignment section.

  In one aspect, the one or more computer processors map one or more image pixels to corresponding positions in the reference visual profile, and the one or more image pixels representing the line are as lines in the reference visual profile. It is configured to compare one or more images of a section of the track with a reference visual profile by determining whether they are in a common location.

  In one aspect, the one or more computer processors identify a portion of the one or more images representing the line by measuring the intensity of pixels in the one or more images, and the one or more images representing the line A portion is configured to be distinguished from other portions of one or more images based on pixel intensity.

  In one aspect, the reference visual profile visually represents the location where the track was before acquiring one or more images.

  In one aspect, the one or more computer processors are also configured to measure the distance between the rails of the track by determining the number of pixels disposed between the rails in the one or more images.

  In one aspect, the one or more computer processors are configured to compare the distance to a specified distance and identify a standard gauge change that has occurred in the section of the track.

  In one aspect, the one or more computer processors are configured to identify a switch in that section of the line by identifying a change in the number of pixels disposed between the rails in the one or more images. Is done.

  In one aspect, the one or more computer processors are configured to create a reference visual profile from at least one of the one or more images compared to the reference visual profile and identify one or more differences.

  In one aspect, the one or more computer processors may include one or more images of the track segments and one or more of the track segments acquired by one or more other rail vehicles at one or more other times. Is compared to the additional image and is configured to identify degradation of the segment of the line.

  In one aspect, the camera is configured to acquire one or more images of the track segment, and the one or more computer processors are configured to detect track misalignment while the vehicle is traveling at the upper limit speed of the track segment. Configured to identify a segment.

  In another example of the inventive subject matter described herein, a method (eg, an optical path survey method) uses the one or more cameras on a vehicle moving along the path to determine the next segment of the path. Comparing a plurality of first images, and examining the first image using one or more computer processors to identify foreign objects on or near the next segment of the path; Identifying one or more differences between the first images using one or more processors and determining whether the foreign object is a temporary object or permanent based on the differences between the identified first images. And determining whether the foreign object is a temporary object or a permanent object, and performing one or more mitigation actions.

  In one aspect, the method also includes increasing the magnification level of the one or more cameras, zooming in on the foreign object, and acquiring one or more second images of the foreign object. The foreign object can be determined to be a permanent object according to a comparison between the first image and the one or more second images.

  In one aspect, the first image is acquired at different times and the step of performing one or more mitigation actions is based on the difference in the first images obtained at different times. Including the step of prioritizing.

  In one aspect, the method also includes calculating the depth of the foreign object and the distance from the vehicle to the foreign object based on a comparison of the first image and the second image.

  In one aspect, performing the one or more mitigation actions includes determining whether the foreign object is a permanent object or a temporary object and one or more from the difference between the first image. This is based on the depth of the foreign object calculated by the computer processor and the distance from the vehicle to the foreign object calculated by one or more computer processors from the difference between the first images.

  In one aspect, the method also includes estimating the moving speed of the foreign object from the difference between the first images using one or more computer processors.

  In one aspect, the one or more cameras acquire a first image at a first frame rate and an additional second image at a different second frame rate. The method also includes modifying at least one of the first frame rate or the second frame rate based on a change in the moving speed of the vehicle.

  In one aspect, the method also includes comparing the first image to a plurality of additional images of a route acquired by a plurality of other vehicles at one or more other times to identify a degradation of the route. Including.

  In another example of the inventive subject matter described herein, a system (eg, an optical path survey system) is attached to a vehicle and a plurality of next sections of the path while the vehicle is moving along the path. Including one or more cameras configured to acquire the first image of the first. The system also compares the first images with each other, identifies differences between the first images, and on or near the next segment of the path based on the differences between the identified first images. Whether the foreign object is a temporary object by identifying the foreign object, determining whether the foreign object is a temporary object or a permanent object based on the difference between the identified first images One or more computer processors configured to perform one or more mitigation actions in response to determining whether the object is a permanent object.

  In one aspect, the one or more computer processors also direct one or more cameras to increase the magnification level of the one or more cameras, zoom in on the foreign object, and obtain one or more second images of the foreign object. Configured to do. The foreign object can be determined to be a permanent object by one or more computer processors in response to a comparison between the first image and the one or more second images.

  In one aspect, one or more computer processors direct one or more cameras to acquire a first image at different times, and further, one or more computer processors acquire first images acquired at different times. One or more mitigation actions are configured to be performed by prioritizing one or more mitigation actions based on differences in the images.

  In one aspect, the one or more computer processors are also configured to calculate a depth of the foreign object and a distance from the vehicle to the foreign object based on the comparison of the first image.

  In one aspect, the one or more computer processors are based on the difference between whether the foreign object is a permanent object or a temporary object and the first image. Performing one or more mitigation actions based on the depth of the foreign object calculated by, and the distance from the vehicle to the foreign object calculated by one or more computer processors based on the difference between the first images. It is configured as follows.

  In one aspect, the one or more computer processors are also configured to estimate the moving speed of the foreign object from the difference between the first images.

  In one aspect, the one or more cameras acquire a first image at a first frame rate and an additional second image at a different second frame rate. The one or more computer processors can also be configured to modify at least one of the first frame rate or the second frame rate based on a change in the moving speed of the vehicle.

  In one aspect, the one or more computer processors also compare the first image with a plurality of additional images of a route acquired by a plurality of other vehicles at one or more other times to determine the degradation of the route. Configured to identify.

  In another embodiment, the optical path survey system examines image data and uses a vehicle-mounted camera to detect a sign beside the path. Certain signs (eg, distance markers) can be detected and stored in a memory structure such as a database or list. Image analysis (eg, optical character recognition) can be used to determine information about the sign (eg, characters, numbers, symbols, etc.). The memory structure can be constructed or created to include an image of the sign, information about the sign, and / or the position of the sign. In that case, the memory structure can be used and / or updated for various purposes, for example for automatic control of the vehicle and the like. For example, a positive train control (PTC) system or an onboard safety system may use information in a memory structure to determine when to decelerate movement of a vehicle within an area, when to allow the vehicle to accelerate, or It is possible to determine when the brakes should be automatically activated.

  FIG. 16 is a schematic diagram of an optical path survey system 1600 according to another embodiment. System 1600 is mounted on a vehicle 1602, such as a railcar. The vehicle 1602 may be the same as or different from the vehicle 102 shown in FIG. For example, the vehicle 1602 may indicate the vehicle 102. The vehicle 1602 can connect to one or more other vehicles, such as one or more locomotives or railway vehicles, to form a train that travels along a path 1620 such as a railroad. Alternatively, vehicle 1602 may be another type of off-highway vehicle (eg, a vehicle that is not designed or allowed to travel on public roads), or another type of vehicle, such as an automobile. It is good. In some configurations, the vehicle 1602 can pull and / or push passengers and / or cargo, such as in a train or other system of vehicles.

  The system 1600 includes one or more cameras 1606, which may be one or more of the cameras 106 shown in FIG. The camera 1606 can acquire static (eg, still) images and / or moving images (eg, video). Optionally, camera 1606 may be located inside vehicle 1602. The camera 1606 can acquire an image and / or video of the route 1620 and / or a sign placed beside the route 1620 while the vehicle 1602 is moving at a relatively high speed. For example, the image shows that the vehicle 1602 is at the upper limit speed of the path 1620 or the upper limit speed, such as the course speed of the path 1620 when maintenance is not performed on the path 1620 or when the speed is not lower than the upper limit speed of the path 1620. It may be acquired while moving in the vicinity.

  The system 1600 includes a camera controller 1612, which may be the camera controller 112 shown in FIG. The camera controller 1612 can control the operation of the camera 1606 in the same manner as described above in connection with the camera controller 112. System 1600 may also include one or more image analysis processors 1616, which may represent one or more image analysis processors 116 shown in FIG. The image memory 1618 of the system 1600 can represent the image memory 118 shown in FIG. The vehicle controller 1614 can represent the vehicle controller 114 shown in FIG. As described above, the vehicle controller 114 (and thus the vehicle controller 1614 in one embodiment) can include a positioning system that determines the position of the vehicles 102, 1602 along the path 120, 1620. Optionally, positioning system 1622 is separate from controller 1614 but may be operatively connected to controller 1612 and / or 1614 (eg, by one or more wired and / or wireless connections) As a result, positioning system 1622 can communicate data indicating the position of vehicle 1602 to controllers 1612 and / or 1614. Examples of positioning system 1622 include a global positioning system, a cellular triangulation system, a radio frequency identification (RFID) interrogator or reader (eg, reading a roadside transponder to determine position), or a previous location, vehicle There may be a computer microprocessor that calculates the position based on the speed of 1602 and / or the elapsed time from the layout of the path 1620.

  System 1600 can include a communication device 1624 that represents a transceiver circuit and associated hardware (eg, antenna 1626) that can wirelessly communicate information to and / or from vehicle 1602. In one aspect, the communication device 1624 communicates information between the vehicle 1602 and another vehicle mechanically coupled to the vehicle 1602 (eg, directly or by one or more other vehicles) The connection is made using one or more wirings, cables, buses, or the like (eg, a plurality of unit cables, electric wires, etc.).

  With continued reference to the system 1600 shown in FIG. 16, FIG. 17 shows image data 1700 acquired by the system 1600 by way of example. The camera 1606 can acquire or generate image data 1700 as the vehicle 1602 moves along the route 1620. Alternatively, the image data 1700 can be acquired or created while the vehicle 1602 is stopped. The target portion 1702 of the image data 1700 includes a sign 1704 that is located beside or near the path 1620 (eg, within the field of view of the camera 1606, within 10 feet of the path 1620, ie, within 3 meters, or another distance). Can be represented. The image analysis processor 1616 can examine the image data 1700 to identify the target portion 1702 that includes the sign 1704 as the vehicle 1602 moves and / or the target portion 1702 if the vehicle 1602 is stopped. Can be identified.

  In one aspect, the image analysis processor 1616 detects the sign 1704, such as based on the intensity of pixels in the image data 1700, or based on wireframe model data generated based on the image data 1700. Can do. For example, the pixels representing sign 1704 may be more similar to each other in terms of intensity or color than other pixels. Image analysis processor 1616 can identify indicia 1704 in image data 1700 and store target portion 1702 that includes indicia data 1700 and / or indicia 1704 in image memory 1618. The image analysis processor 1616 can examine the target portion 1702 of the image data 1700 and determine what information is represented by the sign 1704. For example, the image analysis processor 1616 can identify characters, numbers, symbols, etc. included in the sign 1704 using optical character recognition. While the indicator 1704 is shown as a printed indicator having a fixed value, the indicator 1704 may instead change the displayed characters, numbers, symbols, etc. over time. For example, the sign 1704 may be a display that can change displayed information, and the sign 1704 may include a placeholder that can change a character, a numerical value, a symbol, or the like. Alternatively, the image analysis processor 1616 can examine the target portion 1702 without first storing the image data 1700 and / or the target portion 1702 in the memory 1618.

  FIG. 18 is a schematic diagram of a memory structure 1800 created based on a survey of the sign 1704 shown in the image data 1700 of FIG. 17 and at least a portion of the survey of that sign 1704, according to one embodiment. The image analysis processor 1616 shown in FIG. 16 can identify information conveyed by the sign 1704 (eg, shown on the sign 1704) using optical character recognition or another technique. In the illustrated example, the image analysis processor 1616 can examine the target portion 1702 of the image data 1700 and determine that the indicator 1704 includes the number “225”. The image analysis processor 1616 can communicate with the positioning system 1622 shown in FIG. 16 to determine the position of the vehicle 1602 when the image data 1700 indicating the sign 1704 is acquired.

  Image analysis processor 1616 may store information shown in sign 1704 and the position of vehicle 1602 in memory structure 1800, as determined by positioning system 1622. Optionally, target portion 1702 and / or image data 1700 may be stored in memory structure 1800. Memory structure 1800 represents an organized list, table, database, or the like of different types of information associated with each other. For example, the memory structure 1800 can store a variety of different locations 1802 of different signs 1704 and information 1804 shown in different signs 1704.

  Memory structure 1800 may be stored locally in memory 1618 and / or remotely stored in a memory device that is not installed in vehicle 1602. The information shown in the sign 1704 and the position of the sign 1704 can be updated by the system 1600 of some vehicles 1602. For example, the communication device 1624 of the plurality of vehicles 1602 may store the information shown on the sign 1704 and the position of the sign 1704 on another vehicle or a memory device at another location, such as a transmission facility or another location (eg, image memory). 1618). The information and position of the sign 1704 can be updated and / or verified when a plurality of vehicles 1602 travel near the sign 1704.

  The information and location of the sign 1704 can be used by the system 1600 to determine if the sign 1704 has been damaged or obscured. The image analysis processor 1616 examines the image data 1700 and either does not identify the sign 1704 in the image data 1700 where the sign 1704 should be, or identifies the same information written on the sign 1704 to be written on the sign. If not, the image analysis processor 1616 can determine that the sign 1704 has been lost, damaged, or unreadable. For example, the image analysis processor 1616 can examine the memory structure 1800 and determine that the sign 1704 has been previously identified at a particular location. The image analysis processor 1616 examines the image data 1700 acquired at the same location, and whether the sign 1704 is shown in the image data 1700 and / or the information in the sign 1704 is the same as the information stored in the memory structure 1800. It can be determined whether or not. If the sign 1704 is not identified from the image data, the image analysis processor 1616 can determine that the sign 1704 has been removed. If the image analysis processor 1616 is unable to identify the information printed on the sign 1704, the image analysis processor 1616 may indicate that the sign 1704 has been damaged or at least partially out of view (eg, due to condensation, ice, or vegetation). It can be determined that it has been blocked. If the information shown on the sign 1704 does not match the information stored in the memory structure 1800 associated with the position of the sign 1704, the image analysis processor 1616 may indicate that the sign has been damaged and the sign 1704 is at least partially out of view. It can be determined that the information that is obstructed and / or stored in the memory structure 1800 and / or shown in the indicator 1704 is inaccurate.

  In response to identifying one or more of these problems with sign 1704 and / or memory structure 1800, image analysis processor 1616 may communicate one or more alert signals. These signals are communicated to another vehicle, requesting that the system 1600 mounted on the other vehicle confirm the image data of the sign 1704, or to the non-equipped facility, the sign 1704 and / or the memory structure 1800. It is possible to request inspection, repair, or maintenance of information recorded in the file.

  In one embodiment, information stored in memory structure 1800 can be used by vehicle controller 1614 to control the operation of vehicle 1602. For example, some signs 1704 may display a speed limit for the path 1620, and some signs 1704 indicate that the worker is working on or near the path 1620. Or some signs 1704 can instruct the vehicle 1602 to stop operating. Information read from sign 1704 by system 1600 and stored in memory structure 1800 can be used to automatically control the operation of vehicle 1602. The vehicle controller 1614 can monitor the position of the vehicle 1602 based on the data communicated from the positioning system 1622. In response to vehicle 1602 approaching or reaching a position associated with sign 1704 in memory structure 1800 (eg, has come within a specified distance of sign 1704), vehicle controller 1614 may have memory structure 1800. Can be determined to determine what information is shown on the sign 1704. If the information indicates a speed limit or a stop command, etc., the vehicle controller 1614 can automatically change the speed, stop the vehicle 1602 in response to an instruction displayed on the sign 1704, and / or An instruction can be displayed to the driver to change the speed or to stop the vehicle 1602. Optionally, the memory structure 1800 can include information used as a positive train control system to automatically control the movement of the vehicle 1602.

  FIG. 19 shows a flowchart for a method 1900 for identifying information indicated on a sign from image data, according to one embodiment. The method 1900 may be performed by one or more embodiments of the route survey system described herein. At 1902, the image data of the route is acquired while the vehicle is moving along the route. At 1904, it is determined whether the image data includes a sign. For example, the pixel intensity in the image data can be examined to determine whether a sign is present. If the sign is shown in the image data, the flow of method 1900 can proceed to 1906. If no sign can be seen in the image data, the flow of method 1900 can return to 1902, in which case additional image data can be obtained.

  At 1906, the portion of the image data representing the sign is examined to determine what information is shown on the sign. For example, optical character recognition or another technique (eg, manual inspection) is performed on the image data or a portion of the image data representing the sign so that any characters, numbers, symbols, etc. are indicated on the sign. You may judge whether it is done.

  At 1908, the position of the sign is determined. The position of the sign can be determined by determining the position of the vehicle when image data indicating the sign is obtained. Alternatively, the position of the sign may be manually input by the operator. In 1910, the location of the sign and the information indicated by the sign are recorded in a memory structure or the like. In that case, as described above, this memory structure is used to later confirm the status or state of the sign, to automatically control the operation of the vehicle, or to instruct the driver how to control the operation of the vehicle. Can be done. The flow of method 1900 may return to 1902 in which case additional image data is obtained.

  Returning to the description of the route survey system 1600 shown in FIG. 16, the system 1600 optionally examines the image data and the safety equipment on the route 1620 is functioning as intended or as designed. Can be guaranteed. For example, the image analysis processor 1616 can analyze image data that indicates equipment that intersects. The image analysis processor 1616 examines this data and the intersection facility crosses other vehicles at the intersection (eg, intersection between the route 1620 and another route such as a road for a car). It can be determined whether or not it is functioning to notify about the passage of the vehicle 1602 passing through the section.

  FIG. 20 shows image data 2000 representing an intersection 2002, according to an example. The image data 2000 can be acquired or generated by the camera 1606 (shown in FIG. 16) when the vehicle 1602 is moving toward the intersection 2002. Intersection 2002 represents an intersection between path 1620 and another path 2004, such as a road for a car. The image analysis processor 1616 (shown in FIG. 16) of the route survey system 1600 (shown in FIG. 16) mounted on the vehicle 1602 allows the image data acquired or generated by the camera 1606 to cross based on the position of the vehicle 1602. The period including or showing the part 2002 can be determined. For example, image analysis processor 1616 communicates with positioning system 1622 (shown in FIG. 16) and vehicle 1602 is at or near intersection 2002 (eg, 1/4 mile, ie 0.4 kilometers, or The case (within a specified distance from the intersection 2002), such as another distance. The location of intersection 2002 may be programmed into image analysis processor 1616 (eg, by hardwired to the hardware circuitry of processor 1616), stored in memory 1618 (shown in FIG. 16), or The processor 1616 may be accessible.

  In response to determining that the vehicle 1602 is at or approaching the intersection 2002, the image analysis processor 1616 obtains image data acquired or generated during the period when the vehicle 1602 is at or near the intersection 2002. Can be investigated. The processor 1616 examines the image data and determines whether a safety facility 2006 (eg, facilities 2006A-2006C) is present at or near the intersection 2002 (eg, 50 feet, ie 15 meters, or another It can be determined whether it is within a specified distance of the intersection 2002, such as a distance), and / or whether the safety facility 2006 is operating.

  In the illustrated example, the safety equipment 2006A shows a crossing sign. Similarly to the sign 1704 shown in FIG. 17, the safety equipment 2006A can display characters, numerical values, symbols, or the like and warn the driver of the vehicle about the intersection 2002. Safety facility 2006B indicates an electrical signal. The electrical signal may be, for example, within the specified distance (eg, 1/4 mile, ie, 0.4 kilometers, or other distance) of the vehicle traveling along the path 1620 toward the intersection 2002 and / or the intersection 2002. Such as a light source that operates to generate light in response to coming in. These light sources can be constant light (eg, blinking, ie, a light source that does not repeat ON and OFF), blinking light (eg, a light source that repeats ON and OFF alternately), or a combination thereof. Safety facility 2006C represents a circuit breaker, such as a gate, that operates to move (eg, lower) a vehicle on path 2004 that passes across path 1620 through intersection 2002. Safety facility 2006C may allow a vehicle on route 1620 to travel on route 1620 toward intersection 2002 and / or a specified distance of intersection 2002 (eg, 1/4 mile, ie 0.4 kilometers, or other It can operate (eg, fall) in response to being within a distance.

  The image analysis processor 1616 can examine the image data 2000 to ensure that the safety facility 2006 exists, is not damaged, and / or operates properly. The processor 1616 searches the entire image data 2000 and a group of pixels having the same or equivalent intensities (eg, within a specified range of each other, such as 1%, 5%, or 10%) corresponds. It can be determined whether the safety facility 2006 is at or near the position in the image data 2000. In one aspect, the processor 1616 compares reference image data with image data 2000, such as an object template similar to that described above in connection with FIGS. Whether it exists in 2000 can be determined. Alternatively, other techniques may be used. With respect to safety facility 2006B, processor 1616 can examine image data acquired or generated at different times to determine whether the light source of safety facility 2006B is activated and / or blinking.

  If the image analysis processor 1616 determines that one or more of the safety equipments 2006 has been lost, damaged, or inoperable based on at least a portion of the inspection of the image data, the image analysis processor 1616 may include one or more Alarm signals can be generated. These signals may be sent to the driver of the vehicle 1602 (eg, by displaying on a display screen or other output device of the vehicle 1602 or the controller 114, 1614), for repair, inspection, and / or further of the safety equipment 2006. To warn non-equipped equipment to request an investigation, or to alert other vehicles that the safety equipment 2006 may be malfunctioning or does not exist (eg, path 1620 and / or It is possible to communicate with other vehicles traveling on the route 2004.

  Optionally, the safety facility 2006 may be installed at a place other than the intersection 2002. The image analysis processor 1616 can examine the image data acquired or generated when the vehicle 1602 is positioned such that the field of view of the camera 1606 includes the safety facility 2006. The image analysis processor 1616 can examine this image data in a manner similar to that described above to determine whether the safety facility exists, is damaged, or is not functioning properly.

  Additionally or alternatively, equipment other than safety equipment 2006 can be examined by image analysis processor 1616. Image analysis processor 1616 may examine image data representing equipment along the track, such as safety equipment or other equipment located along path 1620. Equipment along the track may include equipment that is within a specified distance of the path 1620, such as 50 feet, ie 15 meters, or another distance.

  FIG. 21 illustrates a flowchart of a method 2100 for investigating equipment along a track using image data, according to one embodiment. The method 2100 may be performed by one or more embodiments of the route survey system 100, 1600 described herein. At 2102, the position of the vehicle is determined while the vehicle moves along the route. In 2104, it is determined whether the position of the vehicle is equipment along the track or in the vicinity thereof. For example, the location of the vehicle is compared to a memory structure (eg, list, table, or database) that contains the location of the equipment along the various stored tracks (eg, safety equipment, indicating equipment, or route switchers) can do. If the position of the vehicle is within a specified distance (eg, 50 feet, ie, 15 meters, or another distance) from the equipment along the track, the flow of method 2100 can proceed to 2106. Alternatively, if the vehicle is not at or near equipment along the track, the flow of method 2100 can return to 2102. For example, the further location of the vehicle can be identified and investigated to determine when the vehicle approaches equipment along the track.

  In 2106, image data acquired or generated by a camera mounted on the vehicle is examined. For example, if the vehicle is at or near equipment along a railroad, the on-board camera view may include equipment along the railroad. Image data acquired or generated by the camera can be examined during at least a portion of the period during which the field of view includes equipment along the track. At 2108, it is determined whether the image data indicates that equipment along the track has been damaged, lost, and / or not operating properly. For example, if the equipment along the track is not displayed in the image data, the equipment along the track may have been lost. If the equipment along the track is not displayed as well as the object template or the previous image, the equipment along the track may be damaged and / or malfunctioning. If the image data indicates that the equipment has been lost, damaged, and / or malfunctioned, the flow of method 2100 can proceed to 2110. Otherwise, the flow of method 2100 can return to 2102, where further image data can be examined elsewhere and equipment along other tracks can be examined.

  At 2110, one or more alarm signals are generated. For example, a signal may be generated and / or communicated to a display or monitor, etc., to alert the driver in the vehicle that equipment along the track has been lost, damaged, and / or malfunctioning. can do. As another example, signals may be generated and / or communicated to non-installed equipment and may require inspection, repair, and / or replacement of equipment along the track. Optionally, the signal may be communicated to one or more other vehicles to warn that equipment along the track has failed, lost, and / or malfunctioned.

  In one or more embodiments described herein, the image data can be examined by an image analysis processor when the vehicle is moving and / or when the image data is output from a camera. For example, instead of acquiring image data and storing the image data for a long period of time (eg, until the vehicle moves so that the camera's field of view is not included in any part of the image data), the image analysis processor The image may be examined while the same object or route segment or the like is in the camera's field of view.

  In one embodiment, a method (e.g., for investigating a route) includes image data of a field of view of a camera mounted on a first vehicle, wherein the first vehicle is moving along the first route. Acquiring in the case and automatically inspecting the image data on the first vehicle to identify one or more of the target feature or specified object.

  In one aspect, the feature of interest is a standard gauge distance between two or more portions of the first path, and the step of automatically examining the image data determines one or more changes in the standard gauge distance. Including the steps of:

  In one aspect, the method is responsive to one or more changes in the standard gauge distance indicating one or more of an increasing trend and a decreasing and / or decreasing trend after the increasing trend and an increasing trend after the decreasing trend. Identifying the section of the first path as damaged.

  In one aspect, the segment of the first route includes an increasing trend that occurs over at least one of the first designated time or the first designated distance and at least one or more of the second designated time or the second designated distance. Is identified as damaged in response to a decreasing trend that occurred over time.

  In one aspect, the first path segment is one or more of a first specified time or distance and one of a second specified time or distance that is within at least one of an outer specified time limit or an outer specified distance limit. Based on the above, it is identified as damaged.

  In one aspect, the designated object is a sign and the method also includes determining the position of the sign and automatically examining the image data to determine information displayed on the sign.

  In one aspect, the method uses the position of the sign and the information displayed on the sign by at least one of the first vehicle or the one or more second vehicles to be used by the first vehicle or the one or more second. Storing in a memory structure configured to automatically control operation of at least one of the vehicles.

  In one aspect, the designated object is a facility along the track, and the step of automatically examining the image data is a damaged or missing facility along the track based on at least a portion of the image data. Or determining one or more of the malfunctioning states.

  In one aspect, the designated object is a safety facility located at an intersection between the first route and the second route on which the first vehicle travels, and the step of automatically examining the image data includes The equipment gate did not move to block the movement of one or more second vehicles through the intersection along the second path, and / or the safety equipment light signal was not activated, and And / or determining that the safety equipment sign is one or more of missing or damaged.

  In another embodiment, the system (eg, route survey system) is configured to have one or more images configured to be mounted on the first vehicle when the first vehicle moves along the first path. Includes analysis processor. The one or more image analysis processors also obtain image data of a view of a camera mounted on the first vehicle and identify the target feature or one or more of the designated objects on the first vehicle. Configured to automatically examine image data.

  In one aspect, the feature of interest is a standard gauge distance between two or more portions of the first path, and the one or more image analysis processors automatically account for one or more changes in the standard gauge distance. Configured to judge.

  In one aspect, the one or more image analysis processors include one or more standard gauge distances indicating one or more of an increasing trend and a decreasing and / or decreasing trend after the increasing trend and an increasing trend after the decreasing trend. In response to the change, the first path segment is configured to be identified as damaged.

  In one aspect, the one or more image analysis processors may cause the first path segment to have an increasing trend and a second specified time or second time that have occurred over at least one of the first specified time or the first specified distance. It is configured to identify as damaged in response to a decreasing trend that occurs over at least one of the two specified distances.

  In one aspect, the one or more image analysis processors may include one or more of the first specified time or distance and the first specified path segment being within at least one of an outer specified time limit or an outer specified distance limit. It is configured to identify as damaged according to one or more of the two specified times or distances.

  In one aspect, the designated object is a sign and the one or more image analysis processors determine the position of the sign, automatically examine the image data to determine the information displayed on the sign, and the sign And the information displayed on the sign are used by at least one of the first vehicle or the one or more second vehicles to at least the first vehicle or the one or more second vehicles. One of the operations is configured to be stored in a memory structure configured to automatically control.

  In one aspect, the designated object is a facility along the track, and the one or more image analysis processors are based on at least a portion of the image data, the facility along the track is damaged, missing, or It is configured to automatically determine that one or more of the malfunctioning states are present.

  In one aspect, the designated object is a safety facility located at an intersection between the first route and the second route on which the first vehicle travels, and the one or more image analysis processors are provided in the safety facility. The gate did not move to block the movement of one or more second vehicles through the intersection along the second path, or the safety equipment light signal was not activated, or the safety equipment sign Is automatically determined to be one or more of being at least one of lost or damaged.

  In another embodiment, another method (eg, for investigating a route) includes examining image data of a track having a plurality of rails. The image data can be acquired from a camera mounted on a vehicle that moves along the track. The method also determines a standard rail distance of the track based on at least a portion of the image data, and identifies a segment of the track having one or more damaged rails based on a trend in the standard rail distance of the rail. Steps.

  In one aspect, identifying the section of the track as one or more damaged rails identifies a first trend in standard gauge distance and an opposite second trend in standard gauge distance following the first trend. Includes steps.

  In one aspect, identifying the section of the track as one or more damaged rails determines that each of the first trend and the second trend occurs over at least one or more of a specified time or specified distance. Depending on.

  The components of the system described herein include a hardware circuit or circuit component that includes or is connected to one or more processors, such as one or more computer microprocessors. Or can be represented. The operations of the method described herein and the system are sufficiently complex that the operations cannot be performed in the mind by an average person or a person skilled in the art within a commercially reasonable period of time. Can be. For example, image data exploration can take into account a large amount of information and a human completes the image data exploration within a commercially reasonable period of time to control the vehicle based on the image data exploration. It can be based on relatively complicated calculations that cannot be performed. The hardware circuitry and / or processor of the system described herein can be used to significantly reduce the time required to acquire and examine image data, and can be secure and / or commercially Within a reasonable period of time, image data can be examined and damaged portions of the path can be identified.

  As used herein, a structure, restriction, or element that is “configured” to perform a task or operation is specifically formed, configured, programmed, or adapted to correspond to that task or operation. The For clarity and to avoid misunderstandings, an object that can be modified to simply perform a task or action is not “configured” to perform the task or action as used herein. Instead, the use of “configured as” used herein to perform a corresponding number or action in a manner different from a “commercial” structure or element that is not programmed to perform a task or action. Means a structural application or characteristic of a particular structure or element, programming, and / or any structure, restriction, or structural requirement of an element that is described as “configured to” perform a task or action.

  It will be understood that the above description is illustrative and not restrictive. For example, the above-described embodiments (and / or aspects thereof) may be used in combination with each other. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the inventive subject matter without departing from the scope of the invention. Although the dimensions and types of materials described herein are intended to define the parameters of the present inventive subject matter, they are illustrative rather than limiting. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. Accordingly, the scope of the present subject matter should be determined with reference to the appended items and the full scope of equivalents of such items. In the accompanying items, the terms “including” and “in which” are used as plain English equivalents of the terms “comprising” and “where”, respectively. Further, in the appended items, terms such as “first”, “second” and “third” are used merely as symbols and are not intended to impose numerical requirements on those objects. Furthermore, the limitations of the attached items are and unless such items explicitly use the phrase “means for” followed by a statement of functional void of further structure. Until now, it is not described in means-plus-function format and is not intended to be construed under 35 USC 112 (f).

  The description herein discloses several embodiments of the present subject matter, and further includes those skilled in the art, including making and using any device or system, and performing any embedded method. Examples are used to enable implementation of embodiments of the inventive subject matter. The patentable scope of the inventive subject matter can include other examples that occur to those skilled in the art. Such other embodiments are within the scope of an item if they have structural elements that do not differ from the wording of the item or contain equivalent structural elements that do not differ substantially from the wording of the item.

  The foregoing description of certain embodiments of the present inventive subject matter will be better understood when read in conjunction with the appended drawings. As long as the drawings illustrate functional block diagrams of various embodiments, functional blocks do not necessarily indicate a partition between hardware circuits. Thus, for example, one or more functional blocks (eg, a processor or memory) may be implemented with a single piece of hardware (eg, a general purpose signal processor, a microcontroller, a random access memory, and a hard disk). Similarly, the program may be a stand-alone program, may be incorporated as a subroutine in the operating system, and may be a function within an installed software package. The various embodiments are not limited to the arrangements and instrumentality shown in the figures.

  As used herein, an element or step written in the singular and an element or step preceded by the word “a” or “an” are plural unless otherwise stated to the contrary. It should be understood that this possibility is not excluded. Furthermore, references to “one embodiment” of the inventive subject matter should not be construed as excluding the existence of additional embodiments that include the recited features. Further, unless explicitly stated to the contrary, embodiments that “comprise”, “include” or “have” one or more elements with the specified characteristics do not have such characteristics May contain elements.

  Since any changes may be made to the system and method described above without departing from the spirit and scope of the inventive subject matter herein, all of the inventive subject matter in the above description and accompanying drawings is hereby incorporated by reference. The concept of the invention is to be construed as merely illustrative, and not as limiting the invention.

DESCRIPTION OF SYMBOLS 100 Optical path | route inspection system 102 Vehicle 104 Direction of travel or movement 106 Camera 106a Camera 106b Camera 108 Field of view 110 Field of view 112 Camera controller 114 Vehicle controller 116 Image analysis processor 118 Image memory 120 Path 200 Camera acquisition image 202 Pixel, rail 204 Rail 300 Reference visual profile 302 Designated area 304 Designated area 400 Visual mapping diagram 402, 404, 406 Group 500 Standard gauge distance 500a Distance 500b Distance 600 Intersection (switching part)
602 Path 604 Path 606 First travel direction 608 Second travel direction 610 Third travel direction 700 Method 800 Superimposed display 802 Foreign object 804 First position 806 Second position 808 Third position 900 Method 1000 Camera acquisition image 1002 Reference visual profile 1004 Reference visual profile 1100 Camera acquired image 1102 Reference visual profile 1104 Reference visual profile 1106 Section 1200 Standard gauge distance 1202 Horizontal axis 1204 Vertical axis 1206, 1212 Distance or period 1208 Increasing trend or pattern 1210 Decreasing trend or pattern 1214 Time Or distance limit 1300 standard gauge distance 1408 increasing trend or pattern 1410 decreasing trend or pattern 1500 method 1600 optical path survey system 1602 vehicle 1606 camera 16 2 Camera controller 1614 Vehicle controller 1616 Image analysis processor 1618 Image memory 1620 Path 1622 Positioning system 1624 Communication device 1626 Antenna 1700 Image data 1702 Target part 1704 Sign 1800 Memory structure 1802 Location 1804 Information 2000 Image data 2002 Crossing 2004 Path 2006 Safety Equipment 2006A Safety equipment 2006B Safety equipment 2006C Safety equipment

Claims (20)

The image data of the field of view (108, 110) of the camera (106, 1606) mounted on the first vehicle (102, 1602) is displayed on the first route (120, 602) by the first vehicle (102, 1602). 604) and acquiring as it moves along
Automatically inspecting the image data mounted on the first vehicle (102, 1602) to identify one or more of a target feature or designated object.   The feature of the object is a standard gauge distance (500, 1200, 1300) between two or more parts of the first path (120, 602, 604), and automatically inspecting the image data The method of claim 1, comprising determining one or more changes in the standard gauge distance (500, 1200, 1300). The section of the first route (120, 602, 604) is
Uptrend (1208) and downtrend (1210) following the uptrend (1208), or downtrend (1210) and uptrend (1208) following the downtrend (1210)
The method of claim 2, further comprising identifying as damaged in response to the one or more changes in the standard gauge distance (500, 1200, 1300) indicating one or more of the following.   The segment of the first path (120, 602, 604) includes the increasing trend (1208) and the second designation that have occurred over at least one of a first designated time or a first designated distance (1206). 4. The method of claim 3, wherein the method is identified as damaged in response to the decreasing trend (1210) occurring over at least one or more of time or a second specified distance (1212).   The segment of the first path (120, 602, 604) is the first specified time or distance (1206) of the first specified time or distance (1206) within at least one of an outer specified time limit or an outer specified distance limit (1214). 5. The method of claim 4, wherein one or more and the one or more of the second specified time or distance (1212) are identified as damaged. The designated object is a sign (1704),
Determining the position (1802) of the sign (1704);
The method of claim 1, further comprising: automatically examining the image data to determine information (1804) displayed on the sign (1704).   The position (1802) of the sign (1704) and the information (1804) displayed on the sign (1704) may be used as the first vehicle (102, 1602) or one or more second vehicles (102, 102). 1602) and configured to automatically control the operation of the at least one of the first vehicle (102, 1602) or the one or more second vehicles (102, 1602). The method of claim 6, further comprising storing in a memory structure (1800).   The designated object is a facility along a track, and the step of automatically examining the image data is based on at least a portion of the image data, the facility along the track is damaged or missing. Or the method of claim 1, comprising determining one or more of a malfunctioning state.   The designated object is an intersection (600) between the first route (120, 602, 604) on which the first vehicle (102, 1602) travels and the second route (120, 602, 604). , 2002), and the step of automatically examining the image data includes the step of the gate of the safety facility (2006) being along the second path (120, 602, 604) That one or more second vehicles (102, 1602) passing through the intersection (600, 2002) did not move to block movement, or that the light signal of the safety facility (2006) was not activated, Or the method of claim 1, comprising determining that the safety equipment (2006) sign (1704) is at least one of missing or damaged. One or more images configured to be arranged on a first vehicle (102, 1602) such that the first vehicle (102, 1602) moves along a first route (1620, 2004) An analysis processor (116, 1616), wherein the one or more image analysis processors (116, 1616) are also images of the field of view of the camera (106, 1606) mounted on the first vehicle (102, 1602). Configured to obtain data (1700) and automatically inspect the image data (1700) on the first vehicle (102, 1602) to identify one or more of the feature or specified object of interest. The
system.   The feature of the object is a standard gauge distance (500, 1200, 1300) between two or more portions of the first path (1620, 2004), and the one or more image analysis processors (116, 1616). 11) is configured to automatically determine one or more changes in the standard gauge distance (500, 1200, 1300). The one or more image analysis processors (116, 1616) are:
Uptrend (1408) and downtrend (1410) after the uptrend (1408), or downtrend (1410) and uptrend (1408) after the downtrend (1410)
Configured to identify that the section of the first path (1620, 2004) is damaged in response to the one or more changes in the standard gauge distance (500, 1200, 1300) indicating one or more of 12. The system of claim 11, wherein:   The one or more image analysis processors (116, 1616) may be configured such that the segment of the first path (1620, 2004) is at least one of a first specified time or a first specified distance (1206). Identified as damaged in response to the increasing trend (1408) occurring and the decreasing trend (1410) occurring over at least one of a second specified time or second specified distance (1212) 13. The system of claim 12, wherein:   The one or more image analysis processors (116, 1616) may be configured such that the segment of the first path (1620, 2004) is within at least one of an outer specified time limit or an outer specified distance limit (1214). 14. The device of claim 13, configured to identify as damaged in response to one or more of a specified time or distance (1206) and one or more of the second specified time or distance (1212). System.   The designated object is a sign (1704), and the one or more image analysis processors (116, 1616) determine the position (1802) of the sign (1704) and automatically use the image data (1700). To determine the information (1804) displayed on the sign (1704), the position (1802) of the sign (1704), and the information (1804) displayed on the sign (1704) Is used by said at least one of said first vehicle (102, 1602) or one or more second vehicles (102, 1602), said first vehicle (102, 1602) or said one or more 11. A memory structure (1800) configured to automatically control said at least one operation of said second vehicle (102, 1602). System described.   The designated object is a facility along the track, and the one or more image analysis processors (116, 1616) may cause the facility along the track to be damaged based on at least a portion of the image data (1700). The system of claim 10, wherein the system is configured to automatically determine that there is one or more of being present, missing, or malfunctioning.   The designated object is at an intersection (600, 2002) between the first route (1620, 2004) and the second route (1620, 2004) on which the first vehicle (102, 1602) travels. A safety facility located (2006), wherein the one or more image analysis processors (116, 1616) are arranged such that a gate of the safety facility (2006) is crossed along the second path (1620, 2004). Failure to move to block movement of one or more second vehicles (102, 1602) through (600, 2002), or the light signal of the safety facility (2006) is not activated, or the safety 11. The facility (2006) configured to automatically determine that the sign (1704) of the facility (2006) is one or more of being lost or damaged. System. A step of examining image data (1700) of a track having a plurality of rails (204), wherein the image data (1700) is mounted on a vehicle (102, 1602) that moves along the track (106). , 1606),
Determining a standard gauge distance (500, 1200, 1300) of the track based on at least a part of the image data (1700);
Identifying the section of the track as having one or more damaged rails based on a trend in the standard gauge distance (500, 1200, 1300) of the track.   Identifying the section of the track as having one or more damaged rails includes a first trend in the standard gauge distance (500, 1200, 1300) and the standard gauge distance following the first trend ( The method of claim 18 comprising identifying a reverse second trend at 500, 1200, 1300).   The step of identifying the section of the track as having one or more damaged rails is such that each of the first trend and the second trend is at least one of a specified time or a specified distance (1206, 1212). 20. The method of claim 19, wherein the method occurs in response to determining what happens over time.