JP6697797B2 - Optical path survey system and method - Google Patents

Description

  Embodiments of the subject matter disclosed herein relate to investigating a path traveled by a vehicle and/or equipment located near the path.

  The path traveled by a vehicle can be damaged over time due to long-term use. For example, a railroad track on which a railroad car moves may be displaced due to a ballast material below the railroad track, a lateral swing of the railroad car, or the like. The track may bend or move slightly from the original arrangement of the track. This misalignment can include track bending, which changes the distance between rails of the track (ie, standard gauge). Alternatively, this distance may remain the same (eg, both lines bend the same or about the same amount). This would jeopardize the safety of the rail car, its passengers, and the people nearby and the building. For example, the risk of railroad vehicle derailment would increase if the railroads are misaligned.

  Some known systems and methods for inspecting lines include illuminating visible markers on the lines and optically monitoring these markers to determine if the lines have become misaligned. These visible markers may be created using laser light, for example. However, these systems and methods may require additional hardware in the form of light emitting devices, such as laser light sources. This additional hardware adds to the cost and complexity of the system and may require dedicated rail vehicles that are not used to carry passengers or cargo. Moreover, these systems and methods typically require that the railroad vehicle travel slowly on the track so that visible markers can be surveyed.

  Some rail vehicles include a collision avoidance system to alert the rail vehicle driver of a foreign object on the track in front of the rail vehicle. However, these systems may only include cameras that provide an image to the boarding driver. The driver manually inspects the image for any foreign material, and when the foreign material is identified by the driver, the driver responds accordingly. These types of systems are prone to human error.

  Rail vehicles or other types of vehicles may operate according to an automatic safety system that either deactivates or slows down the vehicle in a position. These systems may rely on databases that associate different locations on the route traveled by the vehicle with different speed limits. If the vehicle exceeds these limits, the system can send a signal to the vehicle to slow or stop the vehicle. Some known systems rely on human operators to create and/or update the database, which can cause errors. As a result, the system may not have accurate information and, in some locations, will allow the vehicle to drive beyond the limits. This would pose a significant safety risk.

  These and other types of safety systems may include intersection systems that warn and/or prevent simultaneous intersections of vehicles through intersections of routes. For example, a railway vehicle may travel on a track that crosses a route along which another vehicle, such as an automobile, travels. The safety system can include gates, signals, etc. at the intersection between the track and the route that the vehicle travels. Some of these systems have gates, traffic lights, etc. working properly to stop or warn other vehicles when they are approaching an intersection in some circumstances, such as during a power outage. If not, it may not be possible to make a decision.

U.S. Pat. No. 8,744,196

  In one embodiment, a method (eg, for surveying a route) includes image data of a field of view of a camera mounted on the first vehicle as the first vehicle travels along the first route. Optionally obtaining and automatically inspecting the image data on the first vehicle to identify one or more of the features of interest or the specified object.

  In another embodiment, the system (eg, a route survey system) has one or more images configured to be mounted on the first vehicle when the first vehicle travels along the first path. It includes an analysis processor. One or more image analysis processors are also provided on the first vehicle to acquire image data of a field of view of a camera mounted on the first vehicle and identify one or more of the features or designated objects of interest. It is configured to automatically inspect image data.

  In another embodiment, another method (eg, for surveying a route) comprises surveying image data for a track having multiple rails. The image data can be acquired from a camera mounted on a vehicle moving along the railroad track. The method also includes determining a standard track distance for the track based on at least a portion of the image data, and identifying a section of the track having one or more damaged rails based on the trend in the standard track distance for the track. And steps.

  Specific embodiments and further advantages of the present invention will be described in more detail in the following description with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of an optical path survey system according to one embodiment. It is a figure which shows an example of the image which the camera acquired about the division|segmentation of the route shown in FIG. It is a figure which shows another example of the image of the path|route shown in FIG. It is a figure which shows another example of a reference|standard visual profile. FIG. 3B is a visual mapping diagram of the images shown in FIGS. 2A and 2B and the reference visual profile shown in FIGS. 3A and 3B, according to one embodiment. FIG. 6 is a schematic diagram of an intersection between two or more routes, according to one embodiment. 3 is a flowchart of a method of investigating a route from a vehicle when the vehicle moves along the route. FIG. 3 is a diagram of a superimposed display of three images acquired by one or more of the cameras shown in FIG. 1 and superimposed on one another according to one embodiment. 3 is a flowchart of a method of investigating a route from a vehicle when the vehicle moves along the route. FIG. 6 is a diagram of a camera acquired image with a reference visual profile of a path, according to one embodiment. FIG. 6 is a diagram of another camera acquired image with a reference visual profile of a path, according to one embodiment. It is a figure which shows an example of the standard gauge distance judged by the image analysis processor. It is a figure which shows another example of the standard gauge distance judged by the image analysis processor shown in FIG. It is a figure which shows another example of the standard gauge distance judged by the image analysis processor shown in FIG. FIG. 6 illustrates a flowchart of a method for identifying damage to a path, according to one embodiment. FIG. 6 illustrates a flowchart of a method for identifying damage to a path, according to one embodiment. FIG. 6 is a schematic diagram of an optical path survey system according to another embodiment. FIG. 17 is a diagram showing image data acquired by the route survey system shown in FIG. 16 according to an example. FIG. 18 is a schematic diagram of a memory structure created based on at least a portion of the inspection of the sign and the inspection of the sign shown in the image data of FIG. 17, according to one embodiment. 6 is a flow chart of a method of identifying information indicated on a sign from image data according to one embodiment. FIG. 6 is a diagram showing image data representing an intersection portion according to an example. 6 is a flow chart illustrating a method of surveying equipment along a railroad track using image data, according to one embodiment.

  Embodiments of a route probing system and method of operation are disclosed herein. The system can be installed in a vehicle traveling along a route. While traveling along the route, a camera mounted on the vehicle may acquire or generate image data of the route and/or the area surrounding the route. This image data can be inspected on the vehicle to identify features of interest and/or designated objects. For example, the feature of interest can include a standard track distance between two or more portions of the route. For railway vehicles, the features of interest identified from the image data can include standard gauge distances between rails of the route. Designated objects may include trackside equipment such as safety devices, signs, signals, switches, or inspection devices. The image data is automatically inspected by the path-finding system to determine changes in the characteristics of the object, to identify missing or damaged or malfunctioning designated objects, and/or the location of designated objects. Can be judged. This automatic inspection may be performed without operator intervention. Alternatively, the automated inspection may be performed with the assistance of the operator and/or at the operator's request.

  One or more embodiments described herein include systems and methods for detecting damage to a path taken by a vehicle. The system and method can use analysis of images collected from cameras mounted on the vehicle to detect damage to the path. The present system and method can detect misalignment of railroad tracks on which rail vehicles run. The system and method can detect this misalignment using analysis of railroad track images collected from cameras on the rail vehicle. Based on the detected misalignment, the driver of the rail vehicle can be alerted, so that the driver can take one or more responsive actions, such as by slowing and/or stopping the rail vehicle. it can. Although the description herein focuses on rail vehicles and railroad tracks, one or more embodiments may be designed or licensed for driving other off-highway vehicles (eg, public roads). Vehicles other than railway vehicles, such as non-rail vehicles) or automobiles. Further, one or more embodiments may apply to railroad tracks having a different number of rails or routes other than railroad tracks for rail vehicles, such as roadways.

  The image of the route can be obtained from a camera attached to a vehicle such as a locomotive. The camera can be directed (eg, pointing at the track) in the direction of travel of the rail car. The camera can periodically (or non-periodically) acquire an image of the railroad track for analyzing the displacement. If the tracks are misaligned, the tracks can cause derailment of the rail car. Some of the systems and methods described herein detect rail track misalignment in advance (eg, before the rail vehicle reaches the railroad track) and alert the rail vehicle driver by: Prevent derailment. Optionally, for unmanned railcars (eg, railcars that operate automatically), the systems and methods may automatically slow or stop the movement of the railcar in response to identifying the displaced track. Good.

  Additionally, or alternatively, when a staggered section of the line is identified, it may initiate one or more other responses. For example, an alert signal may be communicated (eg, transmitted or distributed) to one or more other rail vehicles to alert other vehicles of misalignment, the alert signal being located on the track, or A device along one or more tracks near the track can be communicated, so that the device along the track can communicate an alert signal to one or more other rail vehicle systems, The alert signal may be communicated to off-board equipment that may be located for repair and/or further investigation of the out-of-track section.

  If the track is not in the same position as before because of the track's misalignment or movement, the track may be misaligned. For example, instead of breaking or corroding, in a track, the track's lateral movement from the previous position, such as the position of the track when it was installed or previously inspected. And/or may result from longitudinal movement of the track.

  A system and method for using a device for generating light to inspect a path, such as a laser light source for generating laser light on a rail of a track and monitoring the laser light to identify changes in the profile of the rail. In contrast, one or more aspects of the systems and methods described herein rely on the acquisition of image data without producing light or other energy on the path. As described below, one or more systems and methods described herein may capture still and/or moving images of the path and compare these images and/or moving images with reference image data. it can. In at least one embodiment, the path is marked or probed without the use of light, such as laser light.

  FIG. 1 is a schematic diagram of an optical path survey system 100 according to one embodiment. System 100 is mounted on a vehicle 102, such as a rail vehicle. The vehicle 102 may be connected to one or more other vehicles, such as one or more locomotives or rail vehicles, to form a train that travels along a path 120, such as a track. Alternatively, vehicle 102 may be another type of off-highway vehicle (eg, a vehicle that is not designed to travel on public roads, or is not allowed to travel on public roads), or another type of vehicle, such as an automobile. May be In one configuration, the vehicle 102 may pull and/or push passengers and/or cargo, such as in a train or other system of vehicles.

  The system 100 includes one or more cameras 106 (eg, cameras 106 a, 106 b) mounted on or otherwise connected to a vehicle 102, the cameras 106 along with a vehicle 120 along a path 120. Moving. The camera 106 may be a front-facing camera 106 when the camera 106 is oriented toward the direction 104 of travel or movement of the vehicle 102. For example, the fields of view 108, 110 of the camera 106 represent the space captured in the images obtained by the camera 106. In the illustrated example, the camera 106 is forward facing when the views 108, 110 capture images and/or videos of the space in front of the moving vehicle 102. The camera 106 can capture static (eg, still) images and/or video (eg, video). Optionally, one or more cameras 106 may be located inside vehicle 102. For example, vehicle 102 may include a cab camera 106 that is located within the cab of vehicle 102. Such a camera 106 can obtain images and/or video through a window of the vehicle 102 and is filed on Aug. 12, 2014, US Patent No. 14/457,353 entitled "Vehicle Imaging System And Method". (For such camera 106 ), the entire disclosure of which is incorporated herein by reference.

  The camera 106 can capture an image of the route 120 while the vehicle 102 is moving at a relatively high speed. For example, the image may indicate that the vehicle 102 is at the upper speed limit of the route 120, or the upper speed limit of the route 120, such as the route speed of the route 120 if maintenance is not being performed on the route 120 or is not below the upper speed limit of the route 120. It may be acquired while moving in the vicinity.

  The camera 106 operates based on the signal received from the camera controller 112. The camera controller 112 includes one or more computer processors (eg, microprocessors) or other electronic logic based devices and/or one or more hardware circuits or circuit elements connected thereto. Or means. The camera controller 112 operates the camera 106 to cause the camera 106 to acquire image data. This image data represents an image of the field of view 108, 110 of the camera 106, such as an image of one or more portions or sections of the route 120 located in front of the vehicle 102. The camera controller 112 can change the frame rate of the camera 106 (eg, the speed or frequency at which the camera 106 captures images).

  The one or more image analysis processors 116 of the system 100 examine the images acquired by the one or more cameras 106. Processor 116 includes one or more computer processors (eg, microprocessors) or other electronic logic-based devices, and/or one or more hardware circuits or circuit elements connected thereto, or Or it can mean. In one aspect, the processor 116 examines the image by identifying which portions of the image are the images for the path 120 and comparing those portions to one or more reference images. Based on the similarities or differences between the one or more camera-captured images and the reference image, the processor 116 can determine if the segments of the path 120 indicated by the camera images are misaligned. Alternatively, the image analysis processor 116 can transform the image data into wireframe model data or generate wireframe model data, which is described in US Pat. No. 14/253,294. Route Damage Prediction System And Method" ('294 application), the entire disclosure of which is incorporated herein by reference. The wireframe model data can be used to identify the location, shape, etc. of the path 120.

  2A and 2B are diagrams showing an example of the camera-acquired image 200 in the section of the route 120. As shown in FIGS. 2A and 2B, the image 200 can be a digital image formed from a number of pixels 202 of varying color and/or intensity. The higher the intensity of the pixel 202, the lighter the color may be (eg, whiter), and the lower the intensity of the pixel 202, the darker the color may be. In one aspect, image analysis processor 116 (shown in FIG. 1) examines the intensities of pixels 202 to determine which portion of image 200 represents path 120 (eg, rail 204 of the track). For example, the processor 116 may select pixels 202 that have an intensity greater than a specified threshold, such that the pixels 202 may include some or all of the pixels 202 in the image 200 or a display portion of a path 120 (eg, a line). Has a greater intensity than the mean or median of the pixels 202 that are different from the rails 204). Alternatively, processor 116 may use another technique to identify rails 204 in image 200.

  Returning to the description of the system 100 shown in FIG. 1, the image analysis processor 116 stores one or more reference visual profiles of a number of such profiles stored in a computer readable memory, such as the image memory 118. You can choose from. Memory 118 includes or represents one or more memory devices, such as computer hard drives, CD-ROMs, DVD ROMs, removable flash memory cards, or magnetic tape. The memory 118 may store the image 200 captured by the camera 106 (shown in FIGS. 2A and 2B) and a reference visual profile associated with the travel of the vehicle 102.

  The reference visual profile represents a specified layout of the path 120 when the path 120 is in different positions. For example, the reference visual profile may represent the position, placement, relative position of the rails in path 120, such as when the rails were laid, repaired, or last passed an inspection.

  In one aspect, the reference visual profile is a designated standard gauge of the path 120 (eg, the distance between rails of a track). Alternatively, the reference visual profile can be a previous image of the path 120 at the selected location. In another example, the reference visual profile can be a definition of where the path 120 (eg, rail of a track) is expected to be located in the image of the path 120. For example, different reference visual profiles can represent different shapes of rails 204 (shown in FIGS. 2A and 2B) of railroad tracks at different locations along the movement of vehicle 102 from one location to another.

  Processor 116, when acquiring image 200, can determine which reference visual profile to select in memory 118 based on the position of vehicle 102. The vehicle controller 114 can be used to manually and/or automatically control the movement of the vehicle 102 and track where the vehicle 102 was located when the image 200 was acquired. For example, the vehicle controller 114 may include and/or be coupled to a positioning system, such as a positioning system such as a global positioning system or a cellular triangulation system, where the vehicle 102 is. You can determine if you are located. Optionally, vehicle controller 114 determines where vehicle 102 is located based on the speed of travel of vehicle 102 and the speed traveled on path 120, the distance traveled by vehicle 102, and the known layout of path 120. You can judge whether. For example, the vehicle controller 114 can calculate how fast the vehicle 102 has moved from a known location (starting point or other location).

  The processor 116 can select a reference visual profile from the memory 118 that is associated with and represents the specified layout or arrangement of the route 120 at the location of the vehicle 102 when the image 200 was acquired. This designated layout or arrangement can represent the shape, spacing, arrangement, etc. of the route 120 for safe driving of the vehicle 102. For example, the reference visual profile can represent a standard track and alignment of rails 204 of the track when the track was laid or last inspected.

  In one aspect, the image analysis processor 116 can measure the standard gauge of the section of the path 120 shown in the image 200 to determine if the path 120 is offset. 3A and 3B are diagrams showing another example of the image 200 of the route 120 shown in FIG. 1. The image analysis processor 116 can inspect the image 200 and measure a standard gauge distance 500 between the rails 204 of the path 120. In one aspect, the analytic processor 116 has been identified as representing one or more pixels 202 that represent one rail 204 and another rail 204, as shown in FIGS. 3A and 3B. A linear or linear distance to one or more other pixels 202 can be measured. This distance represents the standard gauge distance 500 of the route 120. Alternatively, the distance between other pixels 202 may be measured. Processor 116 uses a known transform element to calculate the number of pixels 202 within standard gauge distance 500 by multiplying the number of pixels 202 by the known distance that the width of each pixel 202 represents in image 200. Determine standard gauge distance 500 by converting to length (such as centimeters or meters), by modifying the scale of standard gauge distance 500 shown in image 200 by magnification, or by other methods. be able to. In one aspect, the image analysis processor 116 can convert the image data into image data as wireframe model data, or generate such image data, as described in the '294 application. it can. The standard gauge distance 500 may be measured between portions of the wireframe model data representing rails.

  The measured standard gauge distance may be compared to a designated standard gauge distance stored in memory 118 (or stored elsewhere) for the imaged portion of path 120. The specified standard gauge distance may be the reference visual profile of the path 120 if this distance represents a specified placement or spacing of the rails 204 of the path 120. If the measured standard gauge distance 500 differs from the designated standard gauge distance by more than a specified threshold or tolerance, the processor 116 determines that the segment of the path 120 shown in the image 200 has been misaligned. You can For example, the specified standard gauge may represent the distance or standard gauge of the path 120 when the rail 204 was laid or last passed the inspection. If the designated standard gauge distance 500 deviates too much from the designated standard gauge distance, the deviation may represent a modified or modified standard gauge distance of the route 120.

  Optionally, the processor 116 may measure the standard gauge distance 500 several times as the vehicle 102 travels, and monitor the measured standard gauge distance 500 for changes. If the standard gauge distance 500 changes by more than a specified amount, the processor 116 can identify the next segment of the path 120 as potentially misaligned. However, as described below, a change in the measured standard gauge distance 500, or alternatively, may represent a switch in the route 120 along which the vehicle 102 travels.

  By measuring the standard gauge distance 500 of the path 120, the image analysis processor 116 may determine if one or more of the rails 204 in the path 120 are offset, even if the segment of the path 120 includes a curve. You can enable it. Because the standard gauge distance 500 should be constant or substantially constant (eg, within manufacturing tolerances), the standard gauge distance 500 may be a curve or straight section of the path 120 unless the path 120 is offset. Should not change significantly.

  In one embodiment, the image analysis processor 116 can monitor changes in the standard gauge distance 500 to determine if one or more rails 204 of the path 120 have slipped. The image analysis processor 116 can track the standard gauge distance 500 to determine if the standard gauge distance 500 exhibits a specified trend within a specified distance and/or amount of time. For example, the standard gauge distance 500 increases over at least a first specified time period or distance and then decreases over a second specified time period, or decreases over at least a first specified time period or distance. , Then the image analysis processor 116 may determine that the rails 204 have been misaligned if they increase over at least a second specified time period. Optionally, image analysis processor 116 may also determine that rails 204 have deviated in response to increasing or decreasing standard gauge distances 500 within specified detection times or distance limits, as described above. Good.

  FIG. 12 is a diagram showing an example of the standard gauge distance 1200 determined by the image analysis processor 116 shown in FIG. The image analysis processor 116 can repeatedly determine the standard gauge distance 1200 from the image data acquired while the vehicle 102 is moving along the rails 204 of the route 120. If the rails 204 are not curved or bent with respect to each other, the standard gauge distance 1200 may be constant or relatively constant (eg, change in the standard gauge distance 1200 due to noise in the system rather than damage to the rails 204). Will remain. However, if one rail 204 is bent relative to another rail 204, the standard gauge distance 1200 may change as the vehicle 102 travels on the bent rail 204. For example, if one rail 204 is bent away from another rail 204 (eg, has a convex curve), the bent rail 204 moves away from the other rail 204 and then moves towards the other rail 204. Therefore, the standard gauge distance 1200 may increase and then decrease. On the other hand, if one rail 204 bends toward another rail 204 (eg, has a concave curve), the bend rail 204 moves away from the other rail 204 and then away from the other rail 204. Due to movement, the standard gauge distance 1200 may decrease and then increase.

  In addition to, or in addition to, identifying damage to the rails 204 by comparing the standard gauge distance 1200 to one or more thresholds, the image analysis processor 116 examines the standard gauge distance 1200. , The standard gauge distance 1200 may be determined according to one or more predetermined trends or patterns over time or distance.

  In the illustrated example, the standard gauge distance 1200 is shown along a horizontal axis 1202 representing time or distance along the path 120 and a vertical axis 1204 representing different magnitudes (eg, lengths) of the standard gauge distance 1200. The image analysis processor 116 may inspect the standard gauge distance 1200 and determine if the standard gauge distance 1200 is increasing along the path 120 for at least one first specified period or distance 1206. The image analysis processor 116 determines that the standard gauge distance 1200 is determined by determining that the average value, median value, or moving average value, or moving median value of the standard gauge distance 1200 has increased during a period or distance 1206. It can be determined that it is increasing over a period or distance 1206. Alternatively, the image analysis processor 116 determines that the standard gauge distance 1200 is increased by determining that the best fit line, linear regression, or other trend or pattern 1208 at the standard gauge distance 1200 increases during at least the period or distance 1206. , Can be determined to have increased over a period or distance 1206. The time period or distance 1206 can be a variable adjusted to prevent noise in the system from being identified as an actual change in the standard gauge distance 1200. For example, noise may not increase (or decrease) the standard gauge distance 1200 over at least the same length of time or distance 1206 as the duration or distance, while the actual change in standard gauge distance 1200 is at least It may increase (or decrease) over a period or distance of the same length as the period or distance 1206.

  In one embodiment, image analysis processor 116 can identify sections of path 120 that include rails 204 that have an increasing trend or pattern over at least a period or distance 1206 if damaged. Alternatively, the image analysis processor 116 may continue to check the standard gauge distance 1200 for further changes. The standard gauge distance 1200 shown in FIG. 12 shows an increasing trend or pattern 1208 followed by a decreasing trend or pattern 1210. Image analysis processor 116 may identify trends or patterns 1210 as decreasing, as well as identify trends or patterns 1208 as increasing. As with the increasing trend or pattern 1208, the image analysis processor 116 can determine if the decreasing trend or pattern 1210 continues for at least as long as the second specified distance or period 1212. If the decreasing trend or pattern 1210 continues for at least the same length as the second distance or period 1212, the image analysis processor 116 indicates that the decreasing trend or pattern 1210 exhibits a change in the standard gauge distance 1200, for the most part or all. It can be determined that it is not due to noise. The time periods or distances 1206, 1212 may occur at different times or locations than those shown in FIG. For example, the period or distance 1206 may start when the image analysis processor 116 determines that the standard gauge distance 1200 is increasing, and the period or distance 1212 is the image analyzing processor when the standard gauge distance 1200 is decreasing. It may start when 116 determines. The time periods or distances 1206, 1212 may be in close proximity to each other or separated by a longer time or distance than that shown in FIG.

  FIG. 13 is a diagram showing another example of the standard gauge distance 1300 determined by the image analysis processor 116 shown in FIG. The standard gauge distance 1300 is shown along the horizontal axis 1202 and vertical axis 1204 described above. In contrast to the standard gauge distance 1200 shown in FIG. 12, the standard gauge distance 1300 does not include an increasing trend 1208 or a decreasing trend 1210 that continues for at least a specified period or distance 1206, 1212. Instead, any increasing or decreasing trend in standard gauge distance 1300 occurs over a shorter period or distance. As a result, the image analysis processor 116 may not identify an increasing trend 1208 and/or a decreasing trend 1210 in the standard gauge distance 1300.

  The length of time and/or distance the distances or periods 1206, 1212 extend may vary based on the amount of noise in the system. For example, the measured change in standard gauge distances 1200, 1300 may increase as the cause other than the actual change in standard gauge distances 1200, 1300 is increased by a distance or period 1206, 1212 and/or the length of the distance. Can be increased so that noise is not identified as bent rails 204. As another example, if the measured change in standard gauge distance 1200, 1300 decreases for reasons other than the actual change in standard gauge distance 1200, 1300, the time and/or distance over which distance or period 1206, 1212 extends. Can be shorter in length.

  Returning to the description of the standard gauge distance 1200 shown in FIG. 12, the image analysis processor 116 indicates that the standard gauge distance 1200 has actually changed, and that the rail 204 has an increasing tendency or pattern 1208 followed by a decreasing tendency or pattern. Depending on what has been identified as 1210, it can be determined to bend or be damaged. Optionally, image analysis processor 116 is responsive to the actual gauge distance 1200 actually changing and that rail 204 has identified a decreasing trend or pattern 1210 followed by an increasing trend or pattern 1208. It can be judged that it has been bent or damaged. In one embodiment, if the uptrend 1208 is identified but not followed by the downtrend 1210 (or if the downtrend 1210 is not followed by the uptrend 1208), the image analysis processor 116 causes the path 120 to be damaged. May not identify as done.

  In one aspect, the image analysis processor 116 determines an increasing trend 1208 occurring within a specified time or distance limit 1214 and a subsequent decreasing trend 1210 (or a decreasing trend 1210 and a subsequent increasing trend 1208). Depending on the identification, the path 120 is identified as damaged. Because at least some of the bends in rails 204 of path 120 may have occurred over a relatively short distance (eg, a few feet or meters), image analysis processor 116 may increase the increase over a period or distance above limit 1214 and Subsequent decrease patterns 1208, 1210 (or decrease and subsequent increase patterns 1210, 1208) may not be identified as representing the actual curvature of rail 204.

  FIG. 14 is a diagram showing another example of the standard gauge distance 1400 determined by the image analysis processor 116. The standard gauge distance 1400 is shown along the horizontal and vertical axes 1202, 1204 described above. As shown in FIG. 14, the standard gauge distance 1400 shows an increasing trend or pattern 1408 followed by a decreasing trend or pattern 1410. Increasing trend 1408 lasts longer than specified time period 1206 and decreasing trend 1410 lasts longer than specified time period 1212. Period 1206 can begin when image analysis processor 116 identifies that standard gauge distance 1400 has increased and period 1212 identifies that image gauge processor 116 has reduced standard gauge distance 1400. If you can start. Alternatively, image analysis processor 116 may otherwise identify the beginning of periods 1206, 1212.

  However, both increasing and decreasing trends 1408, 1410 do not occur within the specified time or distance limits 1214. For example, the time period or distance at which each of the trends 1408, 1410 occurs is the time or distance that the total time encompassed by the time period 1206, 1212 (eg, the time or distance extending from the beginning of the time period 1206 to the end of the time period 1212). Separated from each other in time or space by a sufficiently large time or distance that is longer than the limit 1214. The size of the time or distance limit 1214 is such that the limit 1214 does not bend or position on the rail 204 because the bending or misalignment of the rail 204 may have occurred over a relatively short distance (eg, a few feet or meters). It can be set so as to block a change in the standard gauge distance 1400 that does not represent a deviation. For example, increasing trend 1408 followed by decreasing trend 1410 (or decreasing trend 1410 followed by increasing trend 1408) may be a relatively large distance (eg, over a few feet or meters, ie, over a few feet or meters). Trends 1408, 1410 may not exhibit misalignment of the rails 204 when separated in time or space by the amount of time covered by the traveling vehicle). Instead, trends 1408, 1410 may indicate drift or other measurement problems in the system.

  By way of example only, the limit 1214 represents a spatial distance of 2 feet (eg, 0.6 meters), 4 feet (eg, 1.2 meters), 10 feet (eg, 3 meters), or another distance. be able to. Optionally, limit 1214 may represent an amount of time required for the vehicle to travel a similar distance, depending on the speed at which the vehicle travels.

  Returning to the description of the system shown in FIG. 1, in one embodiment, the camera is directed forward along the direction of travel of the vehicle, so the image analysis processor 116 measures the standard gauge distance and determines the next segment of the path 120. On the other hand, it is possible to identify the bend or the position shift position of the path 120. If the image analysis processor 116 determines from inspection of the one or more images 200 that the next segment of the path 120 on which the vehicle 102 is heading has deviated, the image analysis processor 116 issues an alert signal to the vehicle controller 114. Can be communicated to. This alert signal can indicate to the vehicle controller 114 that the next segment of the path 120 is off. In response to the alert signal, the vehicle controller 114 can take one or more responsive actions. For example, the vehicle controller 114 can include an output device such as a display or speaker to visually and/or audibly alert the driver of the vehicle 102 about the next offset section of the path 120. In that case, the driver may proceed, such as by slowing or stopping the movement of the vehicle, or by communicating with unloaded repair or inspection equipment to request further inspection and/or maintenance of the misaligned sections of the path 120. Can be determined. Optionally, vehicle controller 114 automatically slows or stops the movement of vehicle 102 and/or automatically communicates with off-board repair or inspection equipment to further deviate the segment of path 120. The response operation may be performed automatically, such as by requesting inspection and/or maintenance.

  FIG. 15 illustrates a flow chart for a method 1500 of identifying damage to a path, according to one embodiment. Method 1500, in one aspect, can be performed by system 100 shown in FIG. The method 1500 is used to measure the standard track distance of the path 120 (shown in FIG. 1) such that one of the rails 204 (shown in FIG. 2) bends relative to another rail 204 of the path 120. It is possible to determine whether the path 120 is damaged due to the fact that it is bent, curved, misaligned, or the like. Method 1500 can represent operations that can be encoded with instructions stored in a memory device (eg, memory 118 shown in FIG. 1, another memory accessible to processor 116 shown in FIG. 1, etc.), Wired connection to image analysis processor 116 and/or to configure system 100 to perform the operations described herein.

  At 1502, image data representing a route along which the vehicle travels is acquired. As mentioned above, the image data may be acquired by one or more cameras located inside and/or outside the vehicle. The image data can include a still image, a moving image, a wireframe model, or the like.

  At 1504, a standard track distance for the route is measured based on at least a portion of the image data. For example, the distance between the portions of the image data representing the rails of the path can be measured and adjusted to determine the lateral separation distance between the rails. The standard gauge distance can be tracked over time, and changes in the standard gauge distance can be identified at various positions and/or times while traveling the route.

  At 1506, the measured standard gauge distance is investigated to determine if the standard gauge distance is increasing (eg, has an increasing trend). For example, inspecting standard gauge distances and, in addition to and/or in addition to noise in the measurement of standard gauge distances, whether the standard gauge distance is increasing with the movement of the vehicle along the route. Can be judged. If the standard gauge distance is increased, the increased spacing between the rails may indicate damage to the path (eg, bending or bending of at least one rail). As a result, the method 1500 flow may proceed to 1508. On the other hand, if the standard gauge distance has not increased, then the method 1500 flow may proceed to 1518, which is described below.

  At 1508, the increasing trend in standard gauge distance is investigated to determine if the standard gauge distance is increasing for at least a first designated time period and/or distance. The increasing trend is compared to a specified period and/or distance to prevent temporary changes in standard gauge distances due to factors other than flexed or damaged rails from being falsely identified as flexed or damaged rails. it can. The specified distance can be 6 inches (eg, 15 centimeters), 9 inches (eg, 23 centimeters, 1 foot (eg, 30 centimeters), or another distance. , The time required for the vehicle to travel a specified distance based on the speed of the vehicle.

  If the increasing trend in standard gauge distance lasts at least as long as the first specified time and/or distance, the increasing trend may indicate damage to the route. Consequently, the method 1500 flow may proceed to 1510. On the other hand, if the increasing trend does not last for at least as long as the first specified time and/or distance, the increasing trend may indicate another factor such as noise in the system. Consequently, the method 1500 flow may return to 1502.

  At 1510, the measured standard gauge distance is examined to determine if the standard gauge distance is decreasing (e.g., has a decreasing trend) following an increasing trend. For example, by examining standard gauge distances, apart from and/or in addition to noise in the measurement of standard gauge distances, standard gauge distances tend to increase with vehicle movement along the route. After being identified, it can be determined if it is decreasing. If the standard gauge is decreasing after an increasing trend, the decreasing distance between the rails will cause damage to the path (eg, at least one bend or bend in the rail returning from a position associated with the increase in standard gauge). May indicate. As a result, the flow of method 1500 may proceed to 1512. On the other hand, if the standard gauge distance has not decreased, then the method 1500 flow may return to 1502.

  At 1512, a decreasing trend in standard gauge distance is investigated to determine if the standard gauge distance has been decreasing for at least a second designated period and/or distance. The second specified time period and/or distance may be the same as the first specified time period and/or distance, or is longer than the first specified time period and/or distance, or May be short. The decreasing trend is compared to a specified period and/or distance to prevent temporary changes in standard gauge distances due to factors other than flexed or damaged rails from being falsely identified as flexed or damaged rails. it can. The specified distance can be 6 inches (eg, 15 centimeters), 9 inches (eg, 23 centimeters, 1 foot (eg, 30 centimeters), or another distance. , The time required for the vehicle to travel a specified distance based on the speed of the vehicle.

  If the decreasing trend in standard gauge distance lasts for at least as long as the second specified time and/or distance, the decreasing trend may indicate damage to the path. As a result, the flow of method 1500 may proceed to 1514. On the other hand, if the decreasing trend does not last at least as long as the second specified time and/or distance, the decreasing trend may indicate another factor such as noise in the system. Consequently, the method 1500 flow may return to 1502.

  At 1514, a determination is made as to whether the first and second specified time periods and/or distances (where increasing and decreasing trends occur) occurred within specified outside distances and/or time limits. Method 1500 can include this determination to ensure that changes in increases or decreases in standard gauge distance occur relatively close to one another. If the up/down changes do not occur relatively close to each other (eg, within specified limits), the up/down changes may not indicate damage to the path. Consequently, the method 1500 flow may return to 1502. On the other hand, if the up/down changes occur relatively close to each other (eg, within specified limits), the up/down changes may indicate damage to the path. As a result, the method 1500 flow may proceed to 1516.

  At 1516, the segment of the route for which a change in standard gauge distance is identified is identified as the damage segment for that route. As described herein, one or more response or remedial actions may then be taken to unmounted positions requiring automatic deceleration or stopping of vehicle movement, inspection, repair, or maintenance on the path. Signals may be communicated, or alert signals may be communicated to one or more other vehicles, etc. Additional image data for the track may continue to be acquired and investigated to monitor standard gauge distances and/or to identify damage segments on the track. For example, method 1500 can return to 1502.

  Returning to the description of method 1500 at 1506, if an increasing trend is not found in standard gauge distances, the flow of method 1500 may proceed to 1518. At 1518, the measured standard gauge distance is examined to determine if the standard gauge distance is decreasing (eg, has a decreasing trend). For example, if the method 1500 determines that the standard gauge distance does not have an increasing trend (which may result from one rail bending away from another rail), the method 1500 determines that the standard gauge distance is ( It can be investigated whether the rails have a decreasing tendency (which can result from bending towards other rails).

  Examine the standard gauge distance to determine if the standard gauge distance is reduced by vehicle movement along the route, in addition to and/or in addition to noise in the standard gauge distance measurements. can do. If the standard gauge distance is reduced, the reduced spacing between the rails may indicate damage to the path (eg, bending or bending of at least one rail). As a result, the method 1500 flow may proceed to 1520. On the other hand, if the standard gauge distance has not decreased, then the method 1500 flow may return to 1502.

  At 1520, the decreasing trend in standard gauge distance is investigated to determine if the standard gauge distance has been decreasing for at least a second designated period and/or distance. As described herein, the decreasing trend is compared to a specified time period and/or distance, and a temporary change in standard gauge distance due to factors other than a bend or damage rail is erroneous as a bend or damage rail. Can be prevented from being identified. If the decreasing trend in standard gauge distance lasts for at least as long as the second specified time and/or distance, the decreasing trend may indicate damage to the path. As a result, the flow of method 1500 may proceed to 1522. On the other hand, if the decreasing trend does not last at least as long as the second specified time and/or distance, the decreasing trend may indicate another factor such as noise in the system. Consequently, the method 1500 flow may return to 1502.

  At 1522, the measured standard gauge distance is examined to determine if the standard gauge distance is increasing (eg, has an increasing trend) following a decreasing trend. If the standard track distance increases after a decreasing trend, the increased spacing between the rails will cause damage to the path (eg, at least one bend or bend in the rail returning from the position associated with the decrease in standard track distance). May indicate. As a result, the method 1500 flow may proceed to 1524. On the other hand, if the standard gauge distance has not increased, then the method 1500 flow may return to 1502.

  At 1524, the increasing trend in standard gauge distance is examined to determine if the standard gauge distance is increasing for at least a first designated time period and/or distance, as above. The increasing trend is compared to a specified period and/or distance to prevent temporary changes in standard gauge distances due to factors other than flexed or damaged rails from being falsely identified as flexed or damaged rails. it can. If the increasing trend in standard gauge distance lasts at least as long as the first specified time and/or distance, the increasing trend may indicate damage to the route. As a result, the flow of method 1500 may proceed to 1526. On the other hand, if the increasing trend does not last for at least as long as the first specified time and/or distance, the increasing trend may indicate another factor such as noise in the system. Consequently, the method 1500 flow may return to 1502.

  At 1526, a determination is made as to whether the first and second specified time periods and/or distances (where increasing and decreasing trends occur) occurred within specified outside distances and/or time limits. If the up/down changes do not occur relatively close to each other (eg, within specified limits), the up/down changes may not indicate damage to the path. Consequently, the method 1500 flow may return to 1502. On the other hand, if the up/down changes occur relatively close to each other (eg, within specified limits), the up/down changes may indicate damage to the path. As a result, the flow of method 1500 may proceed to 1516, described above.

  FIG. 4 is a diagram showing an example of the reference visual profile 300. The reference visual profile 300 is for the path 120 (shown in FIG. 1), such as where the path 120 is expected to be in an image acquired by one or more of the cameras 106 (shown in FIG. 1). Represents the specified layout.

  In the illustrated example, the reference visual profile 300 includes two designated areas 302, 304 that represent designated locations of rails on a track. The designated areas 302, 304 are such that the pixels 202 (shown in FIGS. 2A and 2B) of the image 200 (shown in FIGS. 2A and 2B) representing the rails 204 (shown in FIGS. 2A and 2B) are properly aligned with the rails 204. When aligned, it can represent where it should be placed. For example, the designated areas 302, 304 can represent the expected position of the rail 204 prior to capturing the image 200. If the rail 204 is in the same position as when it was laid or last passed the inspection of the position of the rail 204, or at least in the same position as it is within specified tolerances, the rails 204 should be properly aligned. You can This specified tolerance range may represent a range of positions where rails 204 may appear in image 200 due to swaying or other movements of vehicle 102 (shown in FIG. 1).

  Optionally, reference visual profile 300 may represent a previous image of path 120 captured by camera 106 on the same or different vehicle 102. The designated areas 302, 304 may represent the location of the pixel 202 in the previous image identified as representing the path 120 (eg, rail 204).

  In one aspect, the image analysis processor 116 can map the pixels 202 representing the path 120 (eg, the rails 204) to the reference visual profile 300, or the designated areas 302, 304 of the reference visual profile 300 to the path 120. Can be mapped to pixel 202 representing This mapping may include determining whether the location of the pixel 202 representing the path 120 (eg, rail 204) in the image 200 is in the same location as the designated area 302, 304 of the reference visual profile 300.

  5A and 5B show a visual mapping diagram 400 of an image 200 and a reference visual profile 300, according to an example of the inventive subject matter described herein. Mapping diagram 400 illustrates an example of a comparison between image 200 and reference visual profile 300 performed by image analysis processor 116 (shown in FIG. 1). As shown in mapping diagram 400, designated areas 302, 304 of reference visual profile 300 can be overlaid on image 200. Processor 116 can then identify the differences between image 200 and reference visual profile 300. For example, the processor 116 can determine whether the pixel 202 representing the path 120 (eg, representing the rail 204) is located outside the designated area 302, 304. Optionally, processor 116 causes the positions of pixels 202 representing path 120 in image 200 (eg, the coordinates of those pixels 202) not to lie within designated areas 302, 304 (eg, designated areas 302, 304). It is not a coordinate located within the outer bounds of.

  If the image analysis processor 116 determines that at least a specified amount of pixels 202 representing the path 120 are outside the specified areas 302, 304, the processor 116 determines that the section of the path 120 shown in the image 200 is misaligned. Can be identified. For example, the processor 116 may identify a group 402, 404, 406 of pixels 202 that represents the path 120 (eg, rail 204) as outside the designated area 302, 304. The number, fraction, percentage, or measurement of pixels 202 that represent path 120 and that are outside specified areas 302, 304 exceeds a specified threshold (eg, 10%, 20%, 30%, or another amount). If so, the segment of the path 120 shown in image 200 is identified as offset. On the other hand, if the number, fraction, percentage, or measurement of the pixels 202 that represent the path 120 and are outside the designated area 302, 304 does not exceed a threshold, then the section of the path 120 shown in the image 200 will be misaligned. Not identified if present.

  While the vehicle 102 is traveling over various sections of the route 120, the vehicle 102 may encounter (eg, approach) an intersection between a section of the route 120 being traveled and another section of the route. There is a nature. For rail vehicles, such intersections may include transitions between two or more paths 120. Due to the placement of the rails 204 at the switch, the image analysis processor 116 may apply a survey of the image 200 to determine if the rails 204 are misaligned.

  FIG. 6 is a schematic diagram of an intersection (eg, switch) 600 between two or more paths 602, 604, according to an example of the inventive subject matter described herein. One or more of paths 602, 604, or each, can be the same as or similar to path 120 shown in FIG.

  When the image analysis processor 116 measures the standard gauge distance 500 (shown in FIGS. 3A and 3B) and determines that the rails 204 of the routes 602 and 604 are deviated, the image analysis processor 116 causes the vehicle 102 to switch the switching unit. As approaching 600, the reduced standard gauge distance 500 can be identified. For example, when the vehicle 102 is traveling along the first traveling direction 606 toward the switching unit 600 on the route 602, or the vehicle 102 is traveling along the second traveling direction 608 on the switching unit 600 on the route 604. If the vehicle 102 is traveling toward the switching unit 600 on the route 602 along the third traveling direction 610, the image analysis processor 116 determines that the measured standard gauge distance is It can be determined that 500 is decreasing from distance 500a to a shorter distance 500b, to another distance, or the like.

  If the vehicle 102 is not aware that it is approaching the switching unit 600, the image analysis processor 116 may erroneously identify the rail 204 as misaligned based on this decrease in the measured standard gauge distance 500. There is a nature. However, in one aspect, the vehicle controller 114 controls the position of the vehicle 102 as determined by known positions of the controller 114 and the switch 600 (eg, from a map or track database that provides the switch position, etc.). It may determine (based on) the vehicle 102 approaching the switch 600 and notify the image analysis processor 116. In that case, the image analysis processor 116 does not perform one or more of the response actions described above in response to the reduced measured standard gauge distance 500, such as until the vehicle 102 passes or passes the switch 600. , The reduced standard gauge distance 500 may be ignored.

  The image analysis processor 116 may obtain one or more reference visual profiles from the memory 118 (shown in FIG. 1) that represent paths at or near the switch 600. Instead of representing the parallel rails 204, these reference visual profiles can represent the placement of the rails 204 at the switch 600. In that case, the image analysis processor 116 compares the image of the route approaching the switching unit 600 with the reference visual profile to determine whether the route at or near the switching unit 600 is deviated. You can

  Optionally, the image analysis processor 116 may determine that the vehicle 102 is approaching the switching unit 600 based on the acquired image of the route approaching the switching unit 600. For example, the distance between the rails 204 of the different routes 602 and 604 approaching the switching unit 600 (for example, the standard gauge distance 500b) can be stored in the memory 118 as a reference visual profile. If the image analysis processor 116 determines that the standard gauge distance 500 measured from the image of the route 602 or 604 is the same as or equivalent to the stored standard gauge distance, the image analysis processor 116 causes the vehicle 102 to switch. It can be determined that the unit 600 is approaching. The image analysis processor 116 is used to determine when the vehicle 102 approaches the switching unit 600, confirm the position of the vehicle 102 as determined by the vehicle controller 114, and the controller 114 determines the position of the vehicle 102. When it is not possible to make a judgment, it is possible to assist in specifying the position of the vehicle 102.

  In one aspect, the image analysis processor 116 can create a reference visual profile from the image data acquired from the camera. For example, it is contemplated that the image analysis processor 116 may not have access to the reference visual profile, the segment of the path being investigated may not be associated with the reference visual profile, and so on. The image analysis processor 116 can use the image data to create a reference visual profile “on the fly”, such as by creating a reference visual profile when the image data was acquired. In that case, the reference visual profile can be used to examine the image data for which the reference visual profile was created to identify problems in the path.

  FIG. 10 illustrates a camera acquired image 1000 with reference visual profiles 1002, 1004 of path 120, according to another example of the inventive subject matter described herein. Reference visual profiles 1002, 1004 are created by image analysis processor 116 (shown in FIG. 1) from the image data used to create image 1000. For example, the image analysis processor 116 can examine the intensity of the pixels to determine the location of the path 120, as described above. Within the location of path 120, image analysis processor 116 may find two or more pixels that have the same or similar intensities (eg, within a specified range of each other). Optionally, image analysis processor 116 can identify more pixels with the same or similar intensities.

  In that case, the image analysis processor 116 determines the relationship between these pixels. For example, image analysis processor 116 can identify lines between pixels in image 1000 for each rail 204. These lines represent reference visual profiles 1002, 1004. The image analysis processor 116 then determines that other pixels representing the rails 204 of the path 120 are on or within the reference visual profile 1002, 1004 (eg, the reference visual profile 1002, 1004 has been designated). Within a certain distance) or whether these pixels are outside the reference visual profile 1002, 1004. In the illustrated example, most or all of the pixels representing the rails 204 of the path 120 are on or within the reference visual profile 1002, 1004.

  FIG. 11 illustrates another camera-captured image 1100 with reference visual profiles 1102, 1104 of path 120, according to another example of the inventive subject matter described herein. Reference visual profiles 1102, 1104 can be created using the image data used to form image 1100, as described above in connection with FIG. However, in contrast to image 1000 shown in FIG. 10, section 1106 of path 120 does not fall on or within reference visual profile 1104. This section 1106 bends away from the reference visual profile 1104. The image analysis processor 116 can identify this section 1106 because the pixel having an intensity representative of the rail 204 is no longer on or within the reference visual profile 1104. Therefore, the image analysis processor 116 can identify the segment 1106 as the displacement segment of the path 120.

  In one aspect, the image analysis processor 116 can use a combination of the techniques described herein to probe a path. For example, if both rails 202, 204 of path 120 are bent or misaligned from their previous position but still parallel or substantially parallel to each other, a standard between rails 202, 204. The track gauges may remain the same, or substantially the same, and/or may not be substantially different from the designated standard track gauge 500 of the route 120. As a result, just looking at the standard gauge distance in the image data may not allow the image analysis processor 116 to identify damage (eg, bends) to the rails 202, 204. To avoid this situation, the image analysis processor 116 also uses the image data to generate reference visual profiles 1102, 1104, which are then profiled as described above in connection with FIGS. It can be compared with rail image data. In that case, a bend or other misalignment of the rails 202, 204 can be identified if the bend in the rails 202, 204 deviates from the reference visual profile created from the image data.

  FIG. 7 is a flowchart of a method 700 for surveying a route from a vehicle as the vehicle travels along the route. Method 700 may be performed by one or more embodiments of route survey system 100 (shown in FIG. 1). At 702, an image of the route is acquired from one or more cameras on the vehicle. The image can be acquired for a section of the route in front of the vehicle along the traveling direction of the vehicle (eg, the vehicle is moving toward the section being imaged).

  At 704, a reference visual profile of the path is selected based on the location of the imaged section of the path. As mentioned above, the reference visual profile shall represent a specified standard gauge distance of the route, a previous image of the route, or a spatial representation of where the route is expected to be or was previously located. You can

  At 706, the image is compared to a reference visual profile. For example, the standard gauge of the rail in the route image can be measured and compared to the standard gauge of the reference visual profile. Optionally, the position of the rail in the image may be determined and compared to the position of the rail in previous images of the route. In one aspect, the position of the rail in the image is determined and compared to a designated area of the reference visual profile.

  At 708, it is determined if there is a difference between the path image and the reference visual profile. For example, it can be determined whether the standard gauge distance measured from the image is different from the designated standard gauge distance of the reference visual profile. Additionally, or alternatively, it may be determined whether the position of the rail in the image differs from the position of the rail in the previous image of the route. Optionally, it may be determined whether the position of the rail in the image is outside the designated area in the reference visual profile. If one or more of these differences are identified, the difference is that the path (eg, one or more rails) has been displaced, such as by bending, moving toward ground or underlying ballast material, or breaking. May indicate that.

  If one or more differences between the image and the reference visual profile are identified, the path may have deviated from its previous or specified location. As a result, the flow of method 700 may proceed to 710. On the other hand, if no difference is identified, or if the difference is relatively small or minor, then the route is still in the same position as the previous or designated position (or only a relatively small amount). Moved). As a result, the vehicle can continue traveling along the next segment of the route and method 700 can return to 702.

  At 710, the partition of the path in the image is identified as offset. At 712, an alert signal is communicated to one or more other rail vehicles to alert the other vehicles of misalignment, and a device along the track communicates the alert signal to one or more other rail vehicle systems. To communicate alarm signals to one or more equipment along the track that is installed on or near the track so that it can One or more response actions can be performed, such as by automatically decelerating or stopping, or by notifying the boarding driver of the misalignment. Depending on whether the vehicle can continue traveling along the route, the method 700 flow may return to 702.

  In another aspect of the inventive subject matter described herein, an optical path survey system and method includes a plurality of optical path survey systems mounted in front of a vehicle and arranged in a direction in which the path travels (eg, facing each other). Cameras can be used. The camera captures the image at a relatively high (eg, fast) frame rate, resulting in a static, stable image of the path. Using multiple acquired images, the images are analyzed to identify and/or highlight obstacles (eg, pedestrians, cars, trees, etc.). The system and method can alert the vehicle driver of an obstacle or provide an instruction to trigger a braking action (either manually or automatically). If the driver does not take a decelerating action or depresses the vehicle brakes, the brakes may be activated automatically without driver intervention.

  The camera can capture images at a relatively high frame rate (eg, at relatively fast frequencies), resulting in a static, stable image of the next position on the path traveled. There may be a time delay or lag (eg, a few milliseconds) between the acquisition times for images captured with different cameras. In one aspect, images captured from different cameras in the same time frame (eg, within the same relatively short time frame) identify foreign objects on or near the next section of the path. Compared to. Feature detection algorithms can be used to identify important features on the image such as people, birds, cars, and other vehicles (eg, locomotives). In one aspect, the images can be analyzed and used to identify foreign object depth, estimate foreign object size, and/or identify foreign objects. Different techniques can be used to remove or ignore unstable obstacles such as snow, rain, and pebbles. Major obstacles such as cars and pedestrians on the track can be identified or highlighted and used to alert the vehicle driver of the presence of major obstacles.

  Currently, train drivers may not be able to receive an alert or identification result early enough for obstacles on the next section of the track in various weather conditions. Even if the driver can see the obstacle, it may not find it fast enough to stop the train (or other vehicle) before he hits the obstacle before colliding with it. is there. If the advanced image capture analysis techniques described herein are able to detect distant obstacles fast enough, collisions with obstacles can be avoided.

  Returning to the description of the route survey system 100 shown in FIG. 1, the one or more cameras 106 capture several images 200 of the next segment of the route 120 while the vehicle 102 is moving along the route 120. can do. Although the following description focuses on capturing the image 200 with two or more cameras 106, the image 200 may optionally be captured with only one camera 106. The image analysis processor 116 controls the cameras 106 to at least 300 images per second for each camera, 120 images per second for each camera, 72 images per second for each camera, 48 images per second for each camera, 24 images per second for each camera, Alternatively, the image 200 can be acquired at a relatively high frame rate, such as by acquiring the image at another rate.

  The image analysis processor 116 then compares the images captured by the one or more cameras 106 to identify differences within the images. These differences may represent a temporary or permanent foreign object on or near a segment of the route 120 on which the vehicle 102 is traveling. A temporary foreign object is an object that is moving fast enough so that the object does not interfere with or collide with the vehicle 102 when the vehicle 102 reaches the foreign object. A permanent foreign object is an object that is stationary or is moving slowly enough that the vehicle 102 will collide with the foreign object when the vehicle 102 reaches the foreign object.

  FIG. 8 is a superimposed display 800 of three images acquired by one or more of cameras 106 and superimposed on one another, according to an example of the inventive subject matter described herein. Overlaid display 800 represents three images of the same segment of path 120, taken by one or more cameras 106 at different times and combined together. The image analysis processor 116 may or may not generate such overlays when inspecting the image for foreign objects.

  As shown in the display 800, the path 120 is a permanent object in that the path 120 remains in the same or substantially the same position in images acquired at different times. This is because when the vehicle 102 (shown in FIG. 1) travels along the route 120, the route 120 does not move laterally with respect to the traveling direction of the vehicle 102. Image analysis processor 116 may identify path 120 by examining the intensities of pixels in the image, as described above, or using another technique.

  Further, as shown in the display 800, the foreign matter 802 is displayed in the image. The image analysis processor 116 examines the intensities of pixels in the image (or uses another technique) and one of the pixels having the same or equivalent intensities (eg, within a specified range). The foreign matter 802 can be identified by determining that the above groups are displayed at the positions of the images that are close to each other. Optionally, image analysis processor 116 compares one or more of the images captured by one or more cameras 106 and compares the images with one or more reference visual profiles, similar to that described above. You can If differences between the image and the reference visual image are identified, image analysis processor 116 may identify those differences as representative of foreign object 802. For example, if the reference visual profile represents only the rail 204, but the rail 204 and another object are displayed in the image, the image analysis processor 116 may identify the other object as a foreign object 802. In one aspect, the image analysis processor 116 can distinguish the path 120 (eg, the rail 204) and the foreign object 802 by the path 120 and the foreign object 802 having different shapes and/or sizes.

  Once the foreign object 802 is identified, the image analysis processor 116 can direct one or more of the cameras 106 to magnify the foreign object 802 and obtain one or more magnified images. For example, the initial identification of the foreign matter 802 can be confirmed by the image analysis processor 116 instructing the camera 106 to expand the field of view of the camera 106 and acquiring an enlarged image of the foreign matter 802. The image analysis processor 116 can again inspect the magnified image and confirm the presence of the foreign object 802 or determine that the foreign object 802 is absent.

  The image analysis processor 116 examines a series of two or more images (eg, a magnified image or images acquired prior to magnification) to determine whether the foreign object 802 is a permanent object or a temporary object. You may decide whether there is. In one aspect, the foreign object 802 is identified by the processor 116 as a permanent object if the foreign object 802 is displayed in at least the specified number of images within the specified time period and identified by the processor 116. The presence of the foreign matter 802 in the designated number of images (or more images) at least during the designated period means that the foreign matter 802 is located on or near the next section of the route 120. , And/or likely to remain on or near path 120.

  For example, birds flying on path 120, rainfall on path 120, etc. may be displayed in one or more images captured by camera 106. Since these foreign matters 802 tend to move at a considerably high speed, it is unlikely that these foreign matters 802 will be displayed in an image that exceeds a designated number of images for a designated period. Consequently, the image analysis processor 116 does not identify these types of foreign objects 802 as permanent objects, but instead ignores them or identifies them as temporary objects.

  As another example, a person standing or walking on path 120, and a car stopped or moving slowly on path 120 may have a camera over a period longer than flying birds or rainfall. It may be displayed in the image captured by 106. As a result, the person or vehicle may be displayed in at least the specified number of images for at least the specified time period. Image analysis processor 116 identifies such foreign objects as permanent objects.

  In response to identifying the foreign object as a permanent object, image analysis processor 116 may perform one or more mitigation actions. For example, the image analysis processor 116 can generate an alert signal (shown in FIG. 1) that is communicated to the vehicle controller 114. The alert signal may cause one or more alerting devices, such as internal and/or external sirens, to sound and generate an audible alert or alert that the vehicle 102 is approaching a permanent object. Optionally, the alert signal may generate a visual or other alert to the driver of vehicle 102 to inform the driver about a permanent object. Additionally or alternatively, the alert signal may cause vehicle controller 114 to automatically activate the brakes on vehicle 102. In one aspect, the alert signal allows the vehicle controller 114 to communicate the signal to a switch or other track-side device that controls the switch, the switch is automatically changed, and the vehicle 102: Leave the currently running route 120 (where a permanent object was detected) and move to another, different route to avoid collision with a permanent object.

  In one example of the inventive subject matter described herein, the image analysis processor 116 can determine the speed of movement of a permanent object and determine which mitigation actions, if any, should be performed. .. In the example shown in FIG. 8, the foreign matter 802 is displayed at various positions in the image with respect to the path 120. For example, in the first image, the foreign material 802 is displayed in the first position 804, in the subsequent second image, the foreign material 802 is displayed in a different second position 806, and in the subsequent third image. The foreign matter 802 is displayed at a different third position 808.

  The image analysis processor 116 can identify the changed position of the foreign matter 802 and estimate the moving speed of the foreign matter 802. For example, the image analysis processor 116 can control the frame rate of the camera 106 and thus know the length of time that successive images were acquired. Image analysis processor 116 measures changes in the position of foreign object 802 between different positions 804, 806, 808, etc. and scales these changes in position to an estimated distance that foreign object 802 has moved between the images. be able to. For example, the image analysis processor 116 can estimate the distance in a manner similar to measuring the standard gauge distance 500 shown in FIGS. 3A and 3B. However, instead of measuring the distance between the rails 204, the image analysis processor 116 estimates the distance traveled by the foreign object 802.

  The image analysis processor 116 can estimate the moving speed at which the foreign object 802 is moving by using the change in the position divided by the period in which the images showing the various positions of the foreign object 802 are acquired. If the foreign object 802 is moving slower than the specified speed, the image analysis processor 116 can determine that the foreign object 802 may be blocking the route 120 before the vehicle 102 reaches the foreign object 802. As a result, the image analysis processor 116 responds more quickly, such as by immediately actuating the brakes on the vehicle 102 (eg, to fully or sufficiently slow and stop the movement of the vehicle 102). An alarm signal can be generated to the vehicle controller 114 requesting If the foreign object 802 is moving at at least the same speed as the specified speed, the image analysis processor 116 determines that the foreign object 802 is likely to have been removed from the route 120 before the vehicle 102 reaches the foreign object 802. You can As a result, the image analysis processor 116 automatically decelerates (but does not stop) the vehicle 102 by activating an alarm siren, automatically reducing throttle levels, and/or activating a brake. By way of example, an alarm signal can be generated to the vehicle controller 114 requesting a non-urgent response.

  In one embodiment, the image analysis processor 116 uses the images captured by the two or more cameras 106 to confirm or deny potential identification of permanent objects on or near the path 120. be able to. For example, processor 116 may inspect a first set of images from one camera 106a and a second set of images from another camera 106b to determine that a permanent object is a first image of the first image. It can be determined whether it has been identified in both the set and the second set of images. If a permanent object is detected in both sets of images, the image analysis processor 116 can determine which mitigation action to perform, as described above.

  The image analysis processor 116 can examine the images acquired by the two or more cameras 106 to estimate the depth of the foreign object 802. For example, images acquired by different, spaced cameras 106 at or about the same time may result in a stereoscopic view of the foreign object 802. Due to the slightly different views of the camera 106, images acquired at the same time or at about the same time may have slight differences in the relative position of the foreign object 802, even when the foreign object 802 is stationary. is there. For example, foreign object 802 may be slightly displayed on one side of the image captured by one camera 106a compared to the image captured by another camera 106b. The image analysis processor 116 measures these differences (eg, by measuring the distance between the common pixels or locations of the foreign object 802) and determines the depth of the foreign object 802 (eg, parallel or coaxial with the direction of travel of the vehicle 102). The distance between the opposite sides of the foreign object 802 can be estimated along a certain direction. For example, if these differences are larger, greater depth may be estimated than if the differences are small.

  The image analysis processor 116 can use the estimated depth to determine which mitigation action to perform. For example, if the estimated depth is relatively large, the image analysis processor 116 may determine that the foreign object 802 is larger in size than if the estimated depth is relatively small. The image analysis processor 116 may request a more stringent mitigation operation if the estimated depth is relatively large, and request a less stringent mitigation operation if the estimated depth is relatively small. You can

  Additionally or alternatively, the image analysis processor 116 may examine the two-dimensional dimensions of the identified foreign object 802 in one or more images to determine which mitigation action should be performed. For example, image analysis processor 116 can measure the surface area of the image that represents foreign object 802 within the image. The image analysis processor 116 can combine this two-dimensional dimension of the foreign material 802 in the image with the estimated depth of the foreign material 802 to determine the size index of the foreign material 802. The size index represents the size of the foreign matter 802. Optionally, the size index may be based on the two-dimensional size of the imaged foreign object 802 and not the estimated depth of the foreign object 802.

  The image analysis processor 116 can use the size index to determine which mitigation action to perform. The image analysis processor 116 may request a stricter mitigation operation when the size index is relatively large, and may request a less stringent mitigation operation when the size index is relatively small.

  The image analysis processor 116 can compare the two-dimensional region and/or estimated depth of the foreign object 802 with one or more object templates to identify the foreign object 802. The object template can be similar to the designated areas 302, 304 shown in the reference visual image 300 in FIGS. 5A and 5B. As noted above, the designated areas 302, 304 represent where the properly aligned rails 204 are expected to be placed in the image. Similar designated areas may represent the shape of other objects such as pedestrians, cars, or livestock. The image analysis processor 116 may determine the size and/or shape of the foreign object 802 in one or more images by determining the size and/or shape of one or more designated regions (eg, object templates) that represent one or more different foreign objects. It can be compared with the shape. If the size and/or shape of the foreign material 802 is the same or equivalent (eg, within a specified tolerance), the image analysis processor 116 represents the foreign material 802 in the image as represented by the object template. They can be identified as the same foreign matter.

  The image analysis processor 116 can use the identification result of the foreign matter 802 to determine which mitigation operation should be performed. For example, if the foreign object 802 is identified as a car or a pedestrian, the image analysis processor 116 may request a more stringent mitigation action than if the foreign object 802 were identified as something else, such as livestock.

  In one aspect, the image analysis processor 116 stores one or more of the images in the memory 118 and/or communicates the images to an unloaded location. The image is retrieved from memory 118 and/or from an unloaded location and acquired by the same vehicle 102 at different times and/or by one or more other vehicles 102 at other times and in the same segment of path 120. May be compared to one or more images of Changes in the image of path 120 may be used to identify changes in path 120 over time as identified in the image, such as by identifying wear and tears in path 120, or spilling ballast material under path 120. Degradation of path 120 can be identified.

  FIG. 9 shows a flow chart for a method 900 of surveying a route from a vehicle as the vehicle travels along the route. Method 900 can be performed by one or more embodiments of route survey system 100 (shown in FIG. 1). At 902, multiple images of the route are acquired from one or more cameras of the vehicle. The image can be acquired for a section of a route in front of the vehicle along the traveling direction of the vehicle (eg, the vehicle is moving toward the section being imaged).

  At 904, the image is examined to determine if a foreign object is present in one or more of the images. For example, the intensities of pixels in the image can be examined to determine if there is a foreign object on or near the segment of the route that the vehicle is approaching.

  At 906, it is determined whether a foreign object has been identified in the image. For example, if an object is compared to a previous image or other reference visual profile and the shape of the object is in the current image but not in the previous image or other reference visual profile, the object may represent the object. There is a nature. As a result, the foreign material is identified in the image and the method 900 flow can proceed to 908. On the other hand, if no foreign material is identified in the image, the flow of method 900 will return to 902.

  In one aspect, the presence of a foreign object may be determined by examining a first set of images captured by the first camera and a second set of images captured by the second camera. If the foreign object is identified in the first set of images and the foreign object is identified in the second set of images, then the method 900 flow may proceed to 908. Otherwise, the method 900 flow may return to 902.

  In one aspect, the presence of foreign material may be determined by examining different images acquired at different magnification levels. For example, if a foreign object is identified in one or more images acquired at the first magnification level, the camera zooms in on the foreign object and acquires one or more images at the magnified second magnification level. Good. The image at the magnification level can be examined to determine if there are any foreign objects in the image. If the foreign object is identified in the magnified second magnification level image, the method 900 flow may proceed to 908. Otherwise, the method 900 flow may return to 902.

  At 910, it is determined whether the foreign object is a permanent object or a temporary object. As described above, two or more consecutive images of the path can be examined to determine if a foreign object is present in the image. If the foreign object is displayed in at least the specified number of images for at least the specified time period, the foreign object may be identified as a permanent object, as described above. As a result, one or more mitigation actions may need to be taken to avoid collision with the foreign object and the method 900 flow may proceed to 912.

  On the other hand, if the foreign object is not displayed in at least the specified number of images for at least the specified time period, the foreign object may be a temporary object and may not be identified as a permanent object, as described above. There is. As a result, when the vehicle reaches the location of a foreign object, the foreign object may not be present and may not need to take one or more mitigation actions. In that case, the flow of method 900 may return to 902.

  At 912, one or more mitigation actions can be taken. For example, a vehicle driver may be alerted to the presence of a foreign object, an audible and/or visual alarm may be activated, the vehicle may be automatically braked, or the vehicle throttle may be turned on. Can be reduced. As described above, the size, depth, and/or identification information of the foreign material can be determined and used to select which mitigation action to perform.

  In one example of the inventive subject matter described herein, a method (e.g., for optically surveying a path, such as a track) is a railroad vehicle traveling along a track and a reference visual profile of a section of the track. While selecting (with one or more computer processors), acquiring one or more images of a section of track from a camera mounted on a rail vehicle. The reference visual profile represents the designated layout of the track. The method also includes comparing (at one or more computer processors) one or more images of the section of the track with a reference visual profile of the track, and between the one or more images and the reference visual profile. There may be the step of identifying (at one or more computer processors) one or more differences as misaligned sections of the track.

  In one aspect, the one or more images of the section of the track map the pixels of the one or more images to corresponding locations in the reference visual profile, and the pixels of the one or more images representing the track are within the reference visual profile. It is compared to the reference visual profile by determining if it is in a common position as the tracks of the.

  In one aspect, the method identifies the one or more portions of the image representing the track by measuring the intensities of pixels in the one or more images, and determining the portion of the one or more images representing the track. , Distinguishing from other parts of the one or more images based on pixel intensities.

  In one aspect, the reference visual profile visually represents where the railroad track was before the acquisition of the one or more images.

  In one aspect, the method includes measuring the distance between rails of a track by determining the number of pixels placed between the rails in one or more images.

  In one aspect, the method also includes comparing the distance to a specified distance to identify the changed standard gauge for that section of the track.

  In one aspect, the method also includes identifying a switch in that section of the track by identifying a change in the number of pixels located between the rails in the one or more images.

  In one aspect, the method also includes creating a reference visual profile from at least one of the one or more images that is compared to the reference visual profile to identify one or more differences.

  In one aspect, the method also includes one or more images of the section of the track and one or more additional images of the section of the track acquired by one or more other rail vehicles at one or more other times. And comparing to identify the degradation of the section of the line.

  In one aspect, one or more images of a section of the track are acquired while the rail vehicle is traveling at an upper speed limit (eg, track speed) of the section of the track.

  In another example of the inventive subject matter described herein, a system (eg, an optical path survey system) includes a camera and one or more computer processors. The camera is mounted on the rail car and is configured to capture one or more images of a section of the rail track while the rail car is moving along the rail track. One or more computer processors are configured to select a reference visual profile of a section of track that represents a specified layout of the track. The one or more computer processors also compare the one or more images of the section of the track to a reference visual profile of the track and determine one or more differences between the one or more images and the reference visual profile. It is configured to identify as a track misalignment segment.

  In one aspect, the one or more computer processors map one or more image pixels to corresponding locations in the reference visual profile, the one or more image pixels representing a line being a line in the reference visual profile. It is configured to compare the reference visual profile with one or more images of sections of the track by determining if they are in a common location.

  In one aspect, the one or more computer processors identify the portion of the one or more images representing the track by measuring the intensities of pixels in the one or more images to identify the portion of the one or more images representing the track. The portion is configured to distinguish it from other portions of the one or more images based on the intensity of the pixel.

  In one aspect, the reference visual profile visually represents where the railroad track was before the acquisition of the one or more images.

  In one aspect, the one or more computer processors are also configured to measure the distance between rails of the track by determining the number of pixels located between the rails in the one or more images.

  In one aspect, the one or more computer processors are configured to compare the distance to a specified distance and identify a changed standard gauge for that segment of the track.

  In one aspect, the one or more computer processors are configured to identify a switch in that section of the track by identifying a change in the number of pixels located between the rails in the one or more images. To be done.

  In one aspect, the one or more computer processors are configured to create a reference visual profile from at least one of the one or more images that is compared to the reference visual profile to identify the one or more differences.

  In one aspect, the one or more computer processors include one or more images of the section of track and one or more of the section of track acquired by one or more other rail vehicles at one or more other times. Is configured to identify deterioration of the section of the track.

  In one aspect, the camera is configured to acquire one or more images of a section of track and the one or more computer processors are configured to displace the track while the vehicle is traveling at an upper speed limit of the section of track. Configured to identify the partition.

  In another example of the inventive subject matter described herein, a method (e.g., an optical path-finding method) involves the use of one or more cameras on a vehicle moving along the path to determine the next segment of the path. Comparing the plurality of first images of the image of the first image with one or more computer processors to identify foreign objects on or near the next segment of the path. Identifying one or more differences between the first images using one or more processors and determining whether the foreign object is a temporary object or a permanent object based on the identified differences between the first images. The step of determining whether the foreign object is a temporary object, and the step of performing one or more mitigation operations according to the determination of whether the foreign object is a temporary object or a permanent object.

  In one aspect, the method also includes increasing the magnification level of the one or more cameras to zoom in on the foreign object and acquiring one or more second images of the foreign object. The foreign object can be determined to be a permanent object depending on the comparison between the first image and the one or more second images.

  In one aspect, the first image is acquired at different times and the step of performing the one or more mitigation operations is based on the difference in the first images acquired at the different time points. Priority is included.

  In one aspect, the method also includes calculating a depth of the foreign object and a distance from the vehicle to the foreign object based on the comparison of the first image and the second image.

  In one aspect, the step of performing one or more mitigation operations includes determining whether the foreign object is a permanent object or a temporary object and one or more differences due to the difference between the first image. Performed based on the depth of the foreign matter calculated by the computer processor and the distance from the vehicle to the foreign matter calculated by the one or more computer processors from the difference between the first images.

  In one aspect, the method also includes estimating the velocity of movement of the foreign object from the difference between the first images using one or more computer processors.

  In one aspect, the one or more cameras acquire a first image at a first frame rate and an additional second image at a different second frame rate. The method also includes the step of modifying at least one of the first frame rate and the second frame rate based on a change in the speed of travel of the vehicle.

  In one aspect, the method also includes comparing the first image with a plurality of additional images of the route acquired by a plurality of other vehicles at one or more other times to identify a deterioration of the route. Including.

  In another example of the inventive subject matter described herein, a system (e.g., an optical path inspection system) is mounted on a vehicle and comprises a plurality of next sections of the path while the vehicle is traveling along the path. Of one or more cameras configured to acquire a first image of the. The system also compares the first images to each other to identify differences between the first images and based on the identified differences between the first images on or near the next segment of the path. The foreign object is identified, and based on the difference between the identified first images, it is determined whether the foreign object is a temporary object or a permanent object, and whether the foreign object is a temporary object. , One or more computer processors configured to perform one or more mitigation operations in response to the determination of a permanent object.

  In one aspect, the one or more computer processors also directs the one or more cameras to increase the magnification level of the one or more cameras to zoom in on the foreign object and acquire one or more second images of the foreign object. Is configured to. A foreign object can be determined to be a permanent object by one or more computer processors in response to a comparison between the first image and the one or more second images.

  In one aspect, the one or more computer processors direct the one or more cameras to acquire the first image at different times and the one or more computer processors further acquire the first images at different times. The one or more mitigation operations are configured to be performed by prioritizing the one or more mitigation operations based on the differences in the images.

  In one aspect, the one or more computer processors are also configured to calculate the depth of the debris and the distance from the vehicle to the debris based on the comparison of the first images.

  In one aspect, the one or more computer processors are based on whether the foreign object is a permanent object or a temporary object and the difference between the first image. Perform one or more mitigation actions based on the depth of the foreign matter calculated by and the distance from the vehicle to the foreign matter calculated by the one or more computer processors based on the difference between the first images. Is configured as follows.

  In one aspect, the one or more computer processors are also configured to estimate the speed of movement of the foreign object from the difference between the first images.

  In one aspect, the one or more cameras acquire a first image at a first frame rate and an additional second image at a different second frame rate. The one or more computer processors can also be configured to modify at least one of the first frame rate and the second frame rate based on changes in vehicle speed.

  In one aspect, the one or more computer processors also compare the first image with a plurality of additional images of the route acquired by a plurality of other vehicles at one or more other points of time to determine degradation of the route. Configured to identify.

  In another embodiment, an optical path survey system surveys image data and uses a vehicle's on-board camera to detect markers alongside the route. Certain indicia (eg, distance markers) can be detected and stored in a memory structure such as a database or list. Image analysis (eg, optical character recognition) can be used to determine information about the sign (eg, letters, numbers, or symbols). The memory structure can be constructed or created to include an image of the sign, information about the sign, and/or the location of the sign. In that case, the memory structure may be used and/or updated for various purposes, such as for automatic control of a vehicle. For example, a positive train control (PTC) system or an onboard safety system, etc., uses information in a memory structure to determine when to slow down the movement of a vehicle within an area, when to allow the vehicle to accelerate, or the vehicle. Can automatically determine when to apply the brakes in the.

  FIG. 16 is a schematic diagram of an optical path survey system 1600 according to another embodiment. System 1600 is mounted on a vehicle 1602, such as a railroad vehicle. Vehicle 1602 may be the same as or different from vehicle 102 shown in FIG. For example, vehicle 1602 may represent vehicle 102. The vehicle 1602 may connect with one or more other vehicles, such as one or more locomotives or rail vehicles, to form a train that travels along a path 1620, such as a railroad track. Alternatively, vehicle 1602 may be another type of off-highway vehicle (eg, a vehicle that is not designed to travel on public roads, or is not allowed to travel on public roads), or another type of vehicle such as an automobile. May be In some configurations, the vehicle 1602 can pull and/or push passengers and/or cargo, such as in a train or other system of vehicles.

  System 1600 includes one or more cameras 1606, which may be one or more of cameras 106 shown in FIG. The camera 1606 can capture static (eg, still) images and/or video (eg, video). Optionally, the camera 1606 may be located inside the vehicle 1602. The camera 1606 may capture images of the route 1620 and/or video and/or signs located beside the route 1620 while the vehicle 1602 is moving at relatively high speeds. For example, the image shows that the vehicle 1602 is at or above the upper limit speed of the route 1620, such as the course speed of the route 1620 if maintenance is not being performed on the route 1620 or is not below the upper limit speed of the route 1620. It may be acquired while moving in the vicinity.

  The system 1600 includes a camera controller 1612, which may be the camera controller 112 shown in FIG. The camera controller 1612 can control the operation of the camera 1606, similar to that described above in connection with the camera controller 112. System 1600 may also include one or more image analysis processors 1616, which may represent one or more image analysis processors 116 shown in FIG. The image memory 1618 of the system 1600 can represent the image memory 118 shown in FIG. The vehicle controller 1614 may represent the vehicle controller 114 shown in FIG. As mentioned above, the vehicle controller 114 (and thus the vehicle controller 1614 in one embodiment) may include a positioning system that determines the position of the vehicle 102, 1602 along the route 120, 1620. Optionally, positioning system 1622 is separate from controller 1614 but may be operably connected to controller 1612 and/or 1614 (eg, by one or more wired and/or wireless connections), As a result, positioning system 1622 can communicate data indicating the position of vehicle 1602 to controllers 1612 and/or 1614. Examples of positioning system 1622 include global positioning systems, cellular triangulation systems, radio frequency identification (RFID) interrogators or readers (eg, reading a roadside transponder to determine position), or previous positions, vehicles. Such as a computer microprocessor that calculates position based on the speed of 1602 and/or the time elapsed from the layout of path 1620.

  System 1600 can include a communication device 1624 that represents transceiver circuitry and associated hardware (eg, antenna 1626) that can wirelessly communicate information to and/or from vehicle 1602. In one aspect, the communication device 1624 communicates information between the vehicle 1602 and another vehicle that is mechanically coupled to the vehicle 1602 (eg, directly or by one or more other vehicles). Connections are made using one or more wires, cables, buses, or the like (for example, a plurality of unit cables, electric wires, etc.).

  With continued reference to the system 1600 shown in FIG. 16, FIG. 17 illustrates image data 1700 acquired by the system 1600 according to one example. The camera 1606 can acquire or generate image data 1700 as the vehicle 1602 moves along the route 1620. Alternatively, the image data 1700 can be acquired or created while the vehicle 1602 is stationary. The target portion 1702 of the image data 1700 includes markers 1704 located beside or near the route 1620 (eg, within 10 feet of the route 1620 within the field of view of the camera 1606, ie, within 3 meters, or another distance). Can be represented. The image analysis processor 1616 may examine the image data 1700 to identify the portion of interest 1702 that includes the marking 1704 as the vehicle 1602 moves, and/or the portion of interest 1702 if the vehicle 1602 is stationary. Can be identified.

  In one aspect, the image analysis processor 1616 detects the indicia 1704, such as by based on the intensities of pixels in the image data 1700, or based on wireframe model data generated based on the image data 1700. You can For example, the pixels representing indicia 1704 may be more similar to each other than other pixels, such as in intensity or color. The image analysis processor 1616 may identify the indicia 1704 in the image data 1700 and store the image data 1700 and/or the portion of interest 1702 including the indicia 1704 in the image memory 1618. Image analysis processor 1616 can examine target portion 1702 of image data 1700 to determine what information is represented by indicia 1704. For example, image analysis processor 1616 may use optical character recognition to identify characters, numbers, symbols, or the like contained in indicia 1704. While indicia 1704 is shown as a printed indicia with a fixed numeric value, indicia 1704 may instead change the characters, numbers, or symbols that are displayed over time. For example, the indicator 1704 may be a display capable of changing the displayed information, and the indicator 1704 may have a placeholder capable of changing a character, a numerical value, a symbol, or the like. Alternatively, the image analysis processor 1616 can examine the target portion 1702 without first storing the image data 1700 and/or the target portion 1702 in the memory 1618.

  FIG. 18 is a schematic diagram of a memory structure 1800 created based on at least a portion of the inspection of the indicia 1704 and the inspecting of the indicia 1704 shown in the image data 1700 of FIG. 17, according to one embodiment. The image analysis processor 1616 shown in FIG. 16 can use optical character recognition or another technique to identify the information conveyed by the indicia 1704 (eg, as shown in indicia 1704). In the illustrated example, image analysis processor 1616 can examine target portion 1702 of image data 1700 and determine that indicia 1704 includes the number “225”. The image analysis processor 1616 can communicate with the positioning system 1622 shown in FIG. 16 to determine the position of the vehicle 1602 at the time the image data 1700 showing the sign 1704 was acquired.

  The image analysis processor 1616 may store the information shown on the sign 1704 and the location of the vehicle 1602, as determined by the positioning system 1622, in the memory structure 1800. Optionally, target portion 1702 and/or image data 1700 may be stored in memory structure 1800. Memory structure 1800 represents an organized list, table, database, or the like of different types of information associated with each other. For example, memory structure 1800 can store various different locations 1802 of different indicia 1704 and information 1804 shown in different indicia 1704.

  The memory structure 1800 may be stored locally in the memory 1618 and/or remotely in a memory device that is not mounted on the vehicle 1602. The information shown on sign 1704 and the location of sign 1704 can be updated by system 1600 of some vehicles 1602. For example, the communication device 1624 of the plurality of vehicles 1602 may store the information shown on the sign 1704 and the location of the sign 1704 at another vehicle or at a memory device at another location, such as a transmission facility or another location (eg, image memory). 1618). The information and location of sign 1704 can be updated and/or verified as multiple vehicles 1602 drive near sign 1704.

  The information and location of indicia 1704 can be used by system 1600 to determine if indicia 1704 has been damaged or obscured. The image analysis processor 1616 examines the image data 1700 and either does not identify the indicia 1704 in the image data 1700 where the indicia 1704 should be, or identifies the same information written on the indicia 1704 that should have been written on the indicia. If not, image analysis processor 1616 can determine that indicia 1704 is missing, damaged, or unreadable. For example, image analysis processor 1616 can examine memory structure 1800 and determine that indicia 1704 was previously identified at a particular location. The image analysis processor 1616 examines the image data 1700 acquired at the same location and determines if the indicia 1704 is shown in the image data 1700 and/or the information in the indicia 1704 is the same as the information stored in the memory structure 1800. Can be determined. If the indicia 1704 is not identified from the image data, the image analysis processor 1616 can determine that the indicia 1704 has been removed. If the image analysis processor 1616 cannot identify the information printed on the indicia 1704, the image analysis processor 1616 may determine that the indicia 1704 is damaged or at least partially out of sight (eg, due to condensation, ice, or vegetation). It can be judged that it was blocked. If the information shown on the indicia 1704 does not match the information stored in the memory structure 1800 associated with the location of the indicia 1704, the image analysis processor 1616 indicates that the indicia has been damaged and the indicia 1704 is at least partially out of view. The information obstructed and/or stored in the memory structure 1800 and/or shown in the indicia 1704 can be determined to be incorrect.

  In response to identifying one or more of these issues with indicator 1704 and/or memory structure 1800, image analysis processor 1616 can communicate one or more alert signals. These signals are communicated to another vehicle requesting that the system 1600 onboard the other vehicle confirm the image data of the sign 1704, or the non-installed equipment to the sign 1704 and/or the memory structure 1800. It is possible to request inspection, repair, or maintenance of the information recorded in.

  In one embodiment, the information stored in memory structure 1800 can be used by vehicle controller 1614 to control the operation of vehicle 1602. For example, some signs 1704 may indicate a speed limit for route 1620, and some signs 1704 indicate that a worker is working on or near route 1620. Or some signs 1704 can indicate that the vehicle 1602 should stop operating. The information read from indicia 1704 by system 1600 and stored in memory structure 1800 can be used to automatically control the operation of vehicle 1602. The vehicle controller 1614 can monitor the position of the vehicle 1602 based on the data communicated from the positioning system 1622. In response to the vehicle 1602 approaching or reaching the location associated with the sign 1704 in the memory structure 1800 (eg, being within the specified distance of the sign 1704), the vehicle controller 1614 causes the memory structure 1800 to move. Can be examined to determine what information is shown on sign 1704. If the information indicates a speed limit, a stop command, or the like, the vehicle controller 1614 may automatically change the speed, stop the vehicle 1602, and/or depending on the instructions displayed on the sign 1704. Instructions may be displayed to the driver to change speed or stop vehicle 1602. Optionally, memory structure 1800 can include information used as a positive train control system to automatically control the movement of vehicle 1602.

  FIG. 19 illustrates a flow chart for a method 1900 of identifying information indicated on a sign from image data, according to one embodiment. Method 1900 may be performed by one or more embodiments of the route survey system described herein. At 1902, image data of a route is acquired while a vehicle is traveling along the route. At 1904, it is determined whether the image data includes a sign. For example, pixel intensities in the image data can be examined to determine if a sign is present. If the indicia is shown in the image data, then the method 1900 flow may proceed to 1906. If no indicia are visible in the image data, the flow of method 1900 can return to 1902, where additional image data can be obtained.

  At 1906, the portion of the image data representing the sign is examined to determine what information is shown on the sign. For example, optical character recognition or another technique (eg, manual inspection) may be performed on the image data, or a portion of the image data representing the sign, to indicate what letter, number, or symbol, etc., on the sign. You may judge whether it is done.

  At 1908, the position of the sign is determined. The position of the sign can be determined by determining the position of the vehicle when the image data indicating the sign is obtained. Alternatively, the position of the sign may be manually input by the operator. At 1910, the location of the sign and the information shown on the sign are recorded, such as in a memory structure. In that case, as described above, this memory structure may be used to later check the status or condition of the sign, to automatically control the operation of the vehicle, or to instruct the driver how to control the operation of the vehicle. You can do things like The method 1900 flow may return to 1902, where additional image data is acquired.

  Returning to the description of the route survey system 1600 shown in FIG. 16, the system 1600 optionally surveys the image data and the safety equipment on the route 1620 is functioning as intended or as designed. Can be guaranteed. For example, the image analysis processor 1616 can analyze image data showing intersecting equipment. The image analysis processor 1616 examines this data to determine that the intersection facility has intersected another vehicle at the intersection (eg, the intersection between the route 1620 and another route such as a road for automobiles). It can be determined whether it is functioning to notify about the passage of a vehicle 1602 through the section.

  FIG. 20 shows image data 2000 representing an intersection 2002, according to an example. The image data 2000 can be acquired or generated by the camera 1606 (shown in FIG. 16) when the vehicle 1602 is moving toward the intersection 2002. Intersection 2002 represents the intersection between route 1620 and another route 2004, such as a road for an automobile. The image analysis processor 1616 (shown in FIG. 16) of the route survey system 1600 (shown in FIG. 16) mounted on the vehicle 1602 determines that the image data acquired or generated by the camera 1606 intersects based on the position of the vehicle 1602. A period can be determined that includes or indicates part 2002. For example, image analysis processor 1616 may be in communication with positioning system 1622 (shown in FIG. 16) and vehicle 1602 may be at or near intersection 2002 (eg, 1/4 mile, or 0.4 kilometers, or Within a specified distance from the intersection 2002, such as another distance). The location of the intersection 2002 may be programmed into the image analysis processor 1616 (eg, by hardwiring into the hardware circuitry of the processor 1616), stored in memory 1618 (shown in FIG. 16), or The processor 1616 may be accessible.

  In response to determining that the vehicle 1602 is at or approaching the intersection 2002, the image analysis processor 1616 determines the image data acquired or generated during the period when the vehicle 1602 is at or near the intersection 2002. Can be investigated. Processor 1616 examines the image data and determines whether safety equipment 2006 (eg, equipment 2006A-2006C) is present at or near intersection 2002 (eg, 50 feet, or 15 meters, or another). (E.g., within a specified distance of intersection 2002, such as distance), and/or whether safety equipment 2006 is operating.

  In the illustrated example, the safety facility 2006A indicates a level crossing sign. Similar to the sign 1704 shown in FIG. 17, the safety equipment 2006A can display characters, numerical values, symbols, or the like to warn the driver of the vehicle about the intersection 2002. The safety equipment 2006B shows an electric signal. The electrical signal may be, for example, that the vehicle is traveling on route 1620 toward intersection 2002 and/or within a specified distance of intersection 2002 (eg, 1/4 mile, or 0.4 kilometers, or other distance). Such as a light source that operates to generate light in response to coming to. These light sources may be constant light (for example, blinking, that is, a light source that does not repeat ON and OFF), blinking light (for example, a light source that alternately repeats ON and OFF), or a combination thereof. Safety facility 2006C represents a circuit breaker, such as a gate, that operates to move (eg, lower) a vehicle on a path 2004 that passes across a path 1620 through an intersection 2002. Safety facility 2006C may allow vehicles on route 1620 to travel on route 1620 toward intersection 2002 and/or a specified distance of intersection 2002 (eg, 1/4 mile, or 0.4 kilometers, or other. It can be actuated (e.g., lowered) in response to being within a distance.

  The image analysis processor 1616 may inspect the image data 2000 to ensure that the safety equipment 2006 is present, undamaged, and/or operating properly. The processor 1616 searches the entire image data 2000, and groups of pixels having the same or similar intensities (within a specified range of each other, such as 1%, 5%, or 10%) correspond. It is possible to determine whether or not the safety equipment 2006 is at or near the position in the image data 2000 where it is located. In one aspect, the processor 1616 compares the reference image data, such as an object template similar to that described above in connection with FIGS. 4, 5A, and 5B, with the image data 2000, and the safety facility 2006 determines the image data. It can be determined if it is in 2000. Alternatively, another technique may be used. With respect to safety facility 2006B, processor 1616 can examine image data acquired or generated at different times to determine whether the light source of safety facility 2006B is on and/or blinking.

  If the image analysis processor 1616 determines that one or more of the safety equipment 2006 is lost, damaged, or inoperable based on at least a portion of the inspection of the image data, then the image analysis processor 1616 may include one or more. Alarm signal can be generated. These signals may be provided to the driver of vehicle 1602 (eg, by displaying on a display screen or other output device of vehicle 1602 or controllers 114, 1614) for repair, inspection, and/or further inspection of safety equipment 2006. To warn other vehicles that unloaded equipment may be malfunctioning to request an investigation, or safety equipment 2006 may be malfunctioning or is not present (eg, route 1620 and/or It can communicate to other vehicles (traveling route 2004) and so on.

  Optionally, safety facility 2006 may be installed at a location other than intersection 2002. The image analysis processor 1616 can examine image data acquired or generated when the vehicle 1602 is positioned such that the view of the camera 1606 includes the safety equipment 2006. Image analysis processor 1616 can examine this image data in a manner similar to that described above to determine if safety equipment is present, damaged, or not functioning properly.

  Additionally or alternatively, equipment other than safety equipment 2006 may be investigated by image analysis processor 1616. The image analysis processor 1616 can inspect image data representative of equipment along the track, such as safety equipment or other equipment located along the path 1620. Facilities along the railroad track may include facilities that are within a specified distance of path 1620, such as 50 feet, or 15 meters, or another distance.

  FIG. 21 illustrates a flowchart of a method 2100 for surveying a facility along a track using image data, according to one embodiment. The method 2100 may be performed by one or more embodiments of the route probing system 100, 1600 described herein. At 2102, the position of the vehicle is determined while the vehicle is moving along the route. At 2104, it is determined whether the position of the vehicle is at the facility along the track or in the vicinity thereof. For example, the location of the vehicle is compared to a memory structure (eg, list, table, or database) that stores the locations of the various railroad equipment (eg, safety equipment, indicating equipment, or route switches). can do. If the vehicle's location is within a specified distance (eg, 50 feet, or 15 meters, or another distance) from the facility along the track, then the method 2100 flow may proceed to 2106. Alternatively, if the vehicle is not at or near equipment along the track, the method 2100 flow may return to 2102. For example, the further location of the vehicle can be identified and investigated to determine when the vehicle approaches equipment along the track.

  At 2106, the image data acquired or generated by a camera mounted on the vehicle is examined. For example, if the vehicle is at or near equipment along the track, the field of view of the onboard camera may include equipment along the track. Image data acquired or generated by the camera may be inspected during at least a portion of the time period during which visibility includes equipment along the track. At 2108, it is determined whether the image data indicates that the equipment along the track is damaged, lost, and/or is not operating properly. For example, if the equipment along the track is not displayed in the image data, the equipment along the track may have been lost. If the equipment along the railroad does not appear similar to the object template or previous images, etc., the equipment along the railroad may be damaged and/or malfunctioning. If the image data indicates that the equipment has been lost, damaged, and/or malfunctioned, then the method 2100 flow may proceed to 2110. Otherwise, the method 2100 flow may return to 2102, where additional image data may be explored elsewhere and other equipment along the track may be explored.

  At 2110, one or more alarm signals are generated. For example, a signal may be generated and/or communicated, such as to a display or monitor, to alert a driver in a vehicle that equipment along a railroad has been lost, damaged, and/or is malfunctioning. can do. As another example, the signal may be generated and/or communicated to unloaded equipment, requesting inspection, repair, and/or replacement of equipment along the track. Optionally, the signal may be communicated to one or more other vehicles to alert that equipment along the track has failed, lost, and/or is malfunctioning.

  In one or more embodiments described herein, the image data can be examined by an image analysis processor when the vehicle is moving and/or when the image data is output from a camera. For example, instead of taking image data and storing it for a long period of time (eg, until the vehicle moves so that the field of view of the camera is not included in any part of the image data), the image analysis processor may: The image may be inspected while the same object or path segment, etc., is within the field of view of the camera.

  In one embodiment, a method (eg, for surveying a route) includes image data of a field of view of a camera mounted on the first vehicle as the first vehicle travels along the first route. Optionally obtaining and automatically inspecting the image data on the first vehicle to identify one or more of the features of interest or the specified object.

  In one aspect, the feature of interest is a standard gauge distance between two or more portions of the first path, and the step of automatically examining the image data determines one or more changes in the standard gauge distance. Including the step of performing.

  In one aspect, the method is responsive to one or more changes in standard gauge distance that exhibit an increasing trend and a decreasing trend after the increasing trend and/or one or more of a decreasing trend and an increasing trend after the decreasing trend. The step of identifying the segment of the first path as damaged is included.

  In one aspect, the segment of the first route includes an increasing trend occurring over at least one or more of the first designated time or the first designated distance and at least one or more of the second designated time or the second designated distance. It is identified as damaged according to the decreasing trend over time.

  In one aspect, the first route segment is one or more of a first designated time or distance and one of a second designated time or distance that is within at least one of an outer designated time limit or an outer designated distance limit. According to the above, it is identified as damaged.

  In one aspect, the designated object is a sign and the method also includes the steps of determining the position of the sign and automatically examining the image data to determine the information displayed on the sign.

  In one aspect, the method uses the location of the sign and the information displayed on the sign by at least one of the first vehicle and the one or more second vehicles. Storing in a memory structure configured to automatically control the operation of at least one of the vehicles.

  In one aspect, the designated object is a facility along the railroad, and the step of automatically examining the image data is damaged or missing the facility along the railroad track based on at least a portion of the image data. , Or one or more of the malfunctioning states.

  In one aspect, the designated object is a safety facility located at an intersection between the first route and the second route on which the first vehicle travels, and the step of automatically examining the image data includes The facility's gate did not move to block the movement of one or more second vehicles through the intersection along the second path, and/or the safety facility's light signal did not work; and And/or determining that the safety equipment sign is one or more of being lost and/or damaged.

  In another embodiment, the system (eg, a route survey system) has one or more images configured to be mounted on the first vehicle when the first vehicle travels along the first path. It includes an analysis processor. One or more image analysis processors are also provided on the first vehicle to acquire image data of a field of view of a camera mounted on the first vehicle and identify one or more of the features or designated objects of interest. It is configured to automatically inspect image data.

  In one aspect, the characteristic of interest is a standard gauge distance between two or more portions of the first path, wherein the one or more image analysis processors automatically detect one or more changes in the standard gauge distance. Configured to judge.

  In one aspect, the one or more image analysis processors include one or more of the standard gauge distances that exhibit an increasing trend and/or a decreasing trend and/or a decreasing trend after the increasing trend and an increasing trend after the decreasing trend. In response to the change, the section of the first path is configured to identify as damaged.

  In one aspect, the one or more image analysis processors are configured to increase the segmentation of the first path over at least one or more of the first designated time or the first designated distance and the second designated time or the second designated time. Configured to identify as damaged in response to a decreasing trend occurring over at least one or more of the two specified distances.

  In one aspect, the one or more image analysis processors include one or more first designated times or distances and wherein the first path segment is within at least one of an outer designated time limit or an outer designated distance limit. Configured to identify as damaged according to one or more of the two specified times or distances.

  In one aspect, the designated object is a sign and the one or more image analysis processors determine the position of the sign and automatically inspect the image data to determine the information displayed on the sign. And the information displayed on the sign are used by at least one of the first vehicle and/or the one or more second vehicles to provide at least the first vehicle or the one or more second vehicles. It is configured to be stored in a memory structure that is configured to automatically control one operation.

  In one aspect, the designated object is a railroad facility, and the one or more image analysis processors are damaged, lost, or damaged along the railroad track based on at least a portion of the image data. It is configured to automatically determine one or more of the malfunctioning conditions.

  In one aspect, the designated object is a safety facility located at an intersection between a first route and a second route on which the first vehicle travels, and the one or more image analysis processors are The gate did not move to block the movement of one or more second vehicles through the intersection along the second path, or the safety equipment optical signal did not work, or the safety equipment sign Are automatically determined to be one or more of being lost and/or damaged.

  In another embodiment, another method (eg, for surveying a route) comprises surveying image data for a track having multiple rails. The image data can be acquired from a camera mounted on a vehicle moving along the railroad track. The method also includes determining a standard track distance for the track based on at least a portion of the image data, and identifying a section of the track having one or more damaged rails based on the trend in the standard track distance for the track. And steps.

  In one aspect, identifying the section of the track as one or more damaged rails identifies a first trend in standard gauge and a second opposite trend in standard gauge. Including steps.

  In one aspect, identifying the section of the track as one or more damaged rails comprises determining that each of the first trend and the second trend occurs over at least one or more of a specified time or specified distance. Occurs according to.

  The components of the systems described herein include hardware circuits or circuit components that include, or are associated with, one or more processors, such as one or more computer microprocessors. Or can be represented. The operations of the methods described herein and the system are sufficiently complex that the operations cannot be performed in the head by an average person or a person skilled in the art within a commercially reasonable time. It can be anything. For example, a survey of image data can take into account a large amount of information, and a human completes the survey of the image data within a commercially reasonable time period and controls the vehicle based on the survey of the image data. This can be done by a relatively complicated calculation that cannot be done. Hardware systems and/or processors of the system described herein may be used to significantly reduce the time required to acquire and examine image data, and may be safe and/or commercially available. Within a reasonable time, the image data can be examined and the damaged part of the path can be identified.

  As used herein, a structure, limitation, or element “configured” to perform a task or operation is specifically structured, configured, programmed, or adapted to correspond to the task or operation. It To be clear and avoid misunderstanding, an object that can be modified to simply perform a task or action is not “configured” to perform the task or action, as used herein. Instead, the use of "configured to" as used herein refers to performing the corresponding number or operation in a different manner than a "commercially available" structure or element that is not programmed to perform the task or operation. A structural application or characteristic of a structure or element, programming, and/or any structure, limitation, or structural requirement of an element described as "configured to" perform a task or operation.

  It will be appreciated that the above description is illustrative and not restrictive. For example, the above embodiments (and/or aspects thereof) may be used in combination with each other. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the subject matter of the invention without departing from the scope of the invention. The dimensions and types of materials described herein are intended to define the parameters of the present subject matter, but are not limiting and are exemplary embodiments. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. Accordingly, the scope of the present subject matter is determined with reference to the attached items, and the full scope of equivalents by such items. In the appended items, the terms "including" and "in while" are used as plain English equivalents of the terms "comprising" and "wherein", respectively. Furthermore, in the appended items, terms such as "first", "second" and "third" are used merely as signs and are not intended to impose numerical requirements on those objects. Furthermore, the limitations of the attached items are and unless such items explicitly use the phrase "means for" followed by a functional void statement of further structure. Until, it is not described in the means-plus-function format and is not intended to be interpreted under 35 USC 112(f).

  The present description discloses several embodiments of the present subject matter, and can be further read by those skilled in the art, including making and using any device or system, and performing any embedded method. Examples will be used to enable implementations of the subject matter of the invention. The patentable scope of the inventive subject matter may include other examples that occur to those skilled in the art. Such other embodiments are within the scope of the item if they have structural elements that do not differ from the wording of the item, or if they include equivalent structural elements that do not differ substantially from the wording of the item.

  The above description of certain embodiments of the present inventive subject matter will be better understood when read in conjunction with the accompanying drawings. As long as the drawings show functional block diagrams of various embodiments, functional blocks do not necessarily indicate divisions between hardware circuits. Thus, for example, one or more functional blocks (eg, processor or memory) may be implemented in one piece of hardware (eg, general purpose signal processor, microcontroller, random access memory, hard disk, etc.). Similarly, the program may be a stand-alone program, incorporated into the operating system as a subroutine, and may be a function within an installed software package. The various embodiments are not limited to the arrangements and means shown.

  As used herein, an element or step written in the singular and an element or step preceded by the word "a" or "an", unless the context clearly indicates otherwise, includes a plurality of such elements or steps. It should be understood that it does not exclude the possibility that Furthermore, references to "one embodiment" of the inventive subject matter should not be construed as excluding the existence of additional embodiments that include the recited features. Furthermore, embodiments that "comprise," "include," or "have" one or more elements that have a particular characteristic do not have that characteristic, unless such contrary is stated. May contain elements of.

  All of the subject matter of the invention in the above description and the accompanying drawings can be modified, as any changes may be made to the system and method described above without departing from the spirit and scope of the subject matter of the invention in this specification. The concept of the invention is to be construed as merely illustrative and should not be construed as limiting the invention.

100 optical path inspection system 102 vehicle 104 direction of travel or movement 106 camera 106a camera 106b camera 108 field of view 110 field of view 112 camera controller 114 vehicle controller 116 image analysis processor 118 image memory 120 path 200 camera acquired image 202 pixels, rails 204 rails 300 Reference visual profile 302 Designated area 304 Designated area 400 Visual mapping diagrams 402, 404, 406 Group 500 Standard gauge distance 500a Distance 500b Distance 600 Intersection (switching section)
602 route 604 route 606 first traveling direction 608 second traveling direction 610 third traveling direction 700 method 800 overlay display 802 foreign object 804 first position 806 second position 808 third position 900 method 1000 camera captured image 1002 Reference vision profile 1004 Reference vision profile 1100 Camera acquisition image 1102 Reference vision profile 1104 Reference vision profile 1106 Division 1200 Standard gauge distance 1202 Horizontal axis 1204 Vertical axes 1206, 1212 Distance or period 1208 Increasing tendency or pattern 1210 Decreasing tendency or pattern 1214 hours Or distance limit 1300 standard gauge distance 1408 increasing trend or pattern 1410 decreasing trend or pattern 1500 method 1600 optical path survey system 1602 vehicle 1606 camera 1612 camera controller 1614 vehicle controller 1616 image analysis processor 1618 image memory 1620 route 1622 positioning system 1624 communication Device 1626 Antenna 1700 Image data 1702 Target portion 1704 Sign 1800 Memory structure 1802 Position 1804 Information 2000 Image data 2002 Intersection 2004 Route 2006 Safety equipment 2006A Safety equipment 2006B Safety equipment 2006C Safety equipment

Claims (15)

The image data of the field of view (108, 110) of the camera (106, 1606) mounted on the first vehicle (102, 1602) is transferred to the first route (120, 602) by the first vehicle (102, 1602). , 604) as they move along.
Automatically examining the image data on board the first vehicle (102, 1602) to identify one or more of the features or designated objects of interest; and the first path (120, 602, 604). ),
Increasing trend (1208) and decreasing trend (1210) following the increasing trend (1208), or decreasing trend (1210) and increasing trend (1208) following the decreasing trend (1210)
A method of identifying damage in response to one or more changes in standard gauge distances (500, 1200, 1300) indicative of one or more of
The step of identifying the section of the track as having damaged rails comprises a first trend in the standard gauge distances (500, 1200, 1300) and the standard gauge distances (500, 1200, 1300) following the first trend. ), and the step of identifying the opposite second tendency in (1) .   The segment of the first path (120, 602, 604) is characterized by the increasing trend (1208) and the second designation occurring over at least one or more of a first designated time or a first designated distance (1206). The method of claim 1, wherein the method is identified as damaged in response to the decreasing trend (1210) occurring over at least one or more of a time or second designated distance (1212).   The section of the first route (120, 602, 604) is within the at least one of an outer designated time limit or an outer designated distance limit (1214), and the first of the first designated time or distance (1206). The method of claim 2, wherein one or more and said one or more of said second designated time or distance (1212) are identified as damaged. The designated object is a sign (1704),
Determining the position (1802) of the sign (1704),
Automatically examining the image data to determine the information (1804) displayed on the indicia (1704).   The position (1802) of the sign (1704) and the information (1804) displayed on the sign (1704) are used as the first vehicle (102, 1602) or one or more second vehicles (102, 1602) and is configured to automatically control the operation of the at least one of the first vehicle (102, 1602) or the one or more second vehicles (102, 1602). The method of claim 4, further comprising storing in a memory structure (1800).   The designated object is equipment along a railroad, and the step of automatically examining the image data is damaged or missing equipment along the railroad track based on at least a part of the image data. Or the method of claim 1, comprising determining that one or more of the malfunctioning conditions.   The designated object is an intersection (600) between the first route (120, 602, 604) and the second route (120, 602, 604) along which the first vehicle (102, 1602) travels. , 2002), wherein the step of automatically examining the image data is such that the gate of the safety equipment (2006) follows the second path (120, 602, 604). That one or more second vehicles (102, 1602) did not move to block the movement through the intersection (600, 2002), or that the safety equipment (2006) optical signal does not work; Or the method of claim 1, comprising determining that a sign (1704) of the safety equipment (2006) is one or more of being lost and/or damaged. One or more images configured to be disposed on a first vehicle (102, 1602) such that the first vehicle (102, 1602) moves along a first path (1620, 2004); An image of a field of view of a camera (106, 1606) mounted on the first vehicle (102, 1602), comprising an analysis processor (116, 1616). Configured to obtain data (1700) and automatically inspect the image data (1700) on the first vehicle (102, 1602) to identify one or more of the features or designated objects of interest. ,
The one or more image analysis processors (116, 1616) are
Increasing trend (1408) and decreasing trend (1410) after the increasing trend (1408), or the decreasing trend (1410) and the increasing trend (1408) after the decreasing trend (1410)
According to one or more changes in the standard gauge distance (500, 1200, 1300) indicating one or more of the above, the first tendency and the first tendency in the standard gauge distance (500, 1200, 1300) Configured to identify a segment of the first path (1620, 2004) as damaged by identifying a second opposite trend in the standard gauge distance (500, 1200, 1300) that follows. system. The one or more image analysis processors (116, 1616) determines that the section of the first path (1620, 2004) spans at least one or more of a first designated time or a first designated distance (1206). Configured to identify as damaged according to the increasing trend (1408) that has occurred and the decreasing trend (1410) that has occurred over at least one or more of a second specified time or a second specified distance (1212). 9. The system of claim 8, wherein the system is:   The one or more image analysis processors (116, 1616) are configured such that the segment of the first path (1620, 2004) is within at least one of an outside specified time limit or an outside specified distance limit (1214). The method of claim 9, configured to identify as damaged in response to one or more of one designated time or distance (1206) and one or more of the second designated time or distance (1212). System of.   The designated object is a sign (1704), and the one or more image analysis processors (116, 1616) determine the position (1802) of the sign (1704) and automatically generate the image data (1700). To determine the information (1804) displayed on the sign (1704), and the position (1802) of the sign (1704) and the information (1804) displayed on the sign (1704). Is used by at least one of the first vehicle (102, 1602) and/or one or more second vehicles (102, 1602) to provide the first vehicle (102, 1602) or the one or more The system of claim 8, wherein the system is configured to be stored in a memory structure (1800) configured to automatically control the operation of the at least one second vehicle (102, 1602).   The designated object is a facility along the railroad track, and the one or more image analysis processors (116, 1616) determine whether the facility along the railroad track is damaged based on at least a part of the image data (1700). 9. The system of claim 8, configured to automatically determine one or more of a present, missing, or malfunctioning condition.   The designated object is at an intersection (600, 2002) between the first route (1620, 2004) and the second route (1620, 2004) on which the first vehicle (102, 1602) travels. A safety facility (2006) located at the intersection of the one or more image analysis processors (116, 1616) where the gate of the safety facility (2006) is along the second path (1620, 2004). The one or more second vehicles (102, 1602) did not move to block the movement through (600, 2002), or the light signal of the safety equipment (2006) does not work, or the safety 9. The system of claim 8, configured to automatically determine that the indicia (1704) of the equipment (2006) is one or more of missing and/or damaged. A step of investigating image data (1700) of a track having a plurality of rails (204), wherein the image data (1700) is mounted on a vehicle (102, 1602) moving along the track (106, 106). , 1606), and
Determining a standard track distance (500, 1200, 1300) of the track based on at least a portion of the image data (1700);
Identifying a section of the track as having one or more damaged rails based on a trend in the standard track distance (500, 1200, 1300) of the track,
The step of identifying the section of the track as having one or more damaged rails comprises a first trend in the standard gauge distance (500, 1200, 1300) and the standard gauge distance (500 following the first trend. , 1200, 1300) and a second opposite trend.   Identifying the section of the track as having one or more damaged rails, wherein each of the first trend and the second trend spans at least one or more of a specified time or specified distance (1206, 1212). 15. The method of claim 14, occurring in response to the step of determining to occur.