JP2018508418A - Real-time machine vision and point cloud analysis for remote sensing and vehicle control - Google Patents

Abstract

Methods and apparatus for real-time machine vision and point cloud data analysis are provided for remote sensing and vehicle control. Point cloud data can be analyzed via a scalable centralized cloud computing system for asset information extraction and semantic map generation. The data storage / preprocessor (1220) subdivides the data set for streaming to the distributed processing unit (1240) and operation via the data analysis mechanism (1250). The output of the processing unit (1240) is aggregated by the map generator (1230). The machine learning component optimizes the data analysis mechanism to improve asset and feature extraction from sensor data. The optimized data analysis mechanism may be downloaded to the vehicle for use in an onboard system that analyzes vehicle sensor data. The semantic map data can be used locally in the vehicle, along with onboard sensors, leading to precise vehicle location and providing input to the vehicle to control the system.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is filed on Jan. 20, 2015 and is filed in US Provisional Patent Application No. 62 / 105,96, entitled A Scalable Approach To Point-Cloud Data Processing for Railroad Asset Location and Health Monitor. Claims benefit and priority to the application, which is incorporated by reference in its entirety.

  Automatic sensing of moving vehicles and remote sensing based on machines in the vehicle's local environment are becoming increasingly important in several different areas. One such area is automotive transportation. In recent years, many cars and trucks have implemented on-board global positioning system (GPS) receivers and navigation systems that utilize GPS data for driver guidance. However, as car manufacturers seek to implement more sophisticated autonomous driving, such as autonomous driving features, GPS based location systems may not be able to provide sufficiently accurate vehicle location They do not allow real-time sensing of the vehicle's local environment. Accordingly, highly sensitive infrastructure and landmark maps that potentially include ancillary sensing systems, as well as 3D semantic maps, would be desirable.

  Another application where vehicle location, local environment sensing, and 3D semantic maps may be desired is in train operation. In 2008, the US Congress S. Pass the Rail Safety Improvement Act to ensure that all trains are monitored in real time, enabling "Positive Train Control" (PTC). This method requires that all trains report their location information so that all train movements are tracked in real time. PTCs are required to function in both signaled and dark territories.

  To achieve this milestone, a large number of companies have tried to implement various PTC systems. The recurring problem is that the current PTC system can only track the train as it passes by the roadside repeater or signal station along the track, making the operator aware of the current state of the train between the roadside signals It is to make it not in a state. Thus, the distance between successive physical roadside signaling infrastructures determines the minimum safe distance required between trains (running intervals). Current signaling infrastructure also limits the extent to which roadside signaling equipment can be deployed due to the cost and complexity of building and maintaining a PTC infrastructure along the length of the line network. Current methodologies for detecting trains that have finally passed near roadside detectors suffer from a lack of location information in the middle of the repeater.

  Some companies are one step further and use radio towers along the length of the operator's tracking network to create virtual signals between trains, avoiding the need for roadside signaling equipment. . Radio towers still require signaling equipment that is deployed for wireless communication to take place. However, for reliable location information, additional repeaters need to be deployed along the train track to reliably determine the train location and the currently occupied track.

  One example of a PTC system that has been used is the European Train Control System (ETCS), which relies on trackside equipment and train-mounted controllers that react to information about signaling. The system relies heavily on infrastructure not deployed in the United States or in developing countries.

  A solution that requires minimal deployment of roadside signaling equipment would be beneficial to establish positive train control throughout the United States and in developing countries. It is not effective to deploy a million ground elements, repeaters used to detect and communicate the presence of trains and their locations every 1-15 km along the track. This is because the ground unit is adversely affected by environmental conditions and theft, requires regular maintenance, and the collected data may not be used in real time. Acquiring location data only through line-side equipment is not a scalable solution that takes into account the cost of using ground units throughout the line network PTC. Moreover, the train control and safety system cannot rely solely on the Global Positioning System (GPS). This is because it is not accurate enough to distinguish between trajectories, thereby requiring roadside signaling for position calibration.

  As autonomous operation, train control, and other vehicle operating systems evolve, these and other challenges can be addressed by the systems and methods described herein below.

  According to one aspect disclosed herein, a system and method for localization and / or control of a vehicle such as, for example, a train or automobile is described. For example, a local environment sensor that may include a machine vision system such as LiDAR may be mounted on the vehicle. A GPS receiver may also be included to provide the first geographical location of the vehicle. The remote database and processor store and process data collected from multiple sources, and the onboard vehicle processor downloads data related to the operation, safety, and / or control of a moving vehicle. The local environment sensor generates data describing the surrounding environment, for example, point cloud data generated by the LIDAR sensor. The collected data can be processed locally in the vehicle or uploaded to a remote data system for storage, processing and analysis. An analysis mechanism (onboard and / or implemented in a remote data system) manipulates the collected data to extract information from the sensor data, eg, object identification and location in the local environment Can do.

  Exemplary embodiments of the systems described herein are hardware components, remote databases, and analysis components mounted on a railroad or other vehicle that are moving and stationary Includes analysis components that process collected data regarding information about the transportation system, including vehicle, infrastructure, and roadway (eg, rail or road) conditions. The system can accurately estimate the precise position of the vehicle traveling along the path, for example by comparing the location of the object detected by the vehicle-mounted sensor against a known location of the object. Additional attributes for exemplary components are detailed herein and include the following.

  Hardware: Notifies the movement of the vehicle for safety, and for railway applications, among other features, the trajectory on which they travel, obstacles, the health of the track and rail system, some for automotive applications Among other features, it includes identifying the lane the vehicle is traveling on, the texture and soundness of the road, and the identification of assets in the vicinity.

  Remote database: Contains information about assets that can be remotely queried to obtain additional asset information.

  Asset information database entry: The method includes machine vision data collected by the traveling vehicle itself or by another vehicle (eg, a tracked vehicle, a track inspection vehicle, an aviation vehicle, a mobile mapping platform, etc.). This data is then processed to generate asset information (location, features, road / trajectory health, among other information).

  Data analysis mechanism: fuses several data and information streams (eg, from sensors, databases, roadside units, vehicle information buses, etc.) to provide an accurate estimate of lanes, track IDs, or other landmarks for location As a result.

  These and other aspects of the disclosure will be apparent in view of the text and drawings provided herein.

  Exemplary embodiments will now be further described with reference to the drawings, in which like designations represent like elements.

It is a typical flowchart of a train control system. It is a typical flow diagram of an on-board ecosystem. It is a typical flowchart for acquiring position information. 2 is an exemplary depiction of a train extrapolating signal conditions. FIG. 6 is an exemplary depiction of various interfaces available to the conductor as feedback. It is a typical flowchart for acquiring track ID occupied by a train. It is a typical flowchart explaining a track ID algorithm. FIG. 5 is a representative flow diagram illustrating a signal state algorithm. It is a typical flowchart explaining sensing and feedback. It is a typical flowchart of the image bonding technique for relative orbital positioning. It is a flowchart of a point cloud analysis process. It is a flowchart of a point cloud analysis process. It is a schematic block diagram of the apparatus for a point cloud analysis. FIG. 6 is a flow diagram of a process for analyzing point cloud data. FIG. 5 is a further flow diagram of a process for analyzing point cloud data. FIG. 4 illustrates point cloud tile size and density distribution in an exemplary point cloud survey. It is a schematic block diagram of a point cloud processing cluster. Figure 6 is a plot of characteristics for a compression mechanism that can be used with point cloud data. Figure 6 is a plot of characteristics for a compression mechanism that can be used with point cloud data. Figure 6 is a plot of characteristics for a compression mechanism that can be used with point cloud data. It is a flowchart of the process for orbit detection. It is visualization of the point cloud division with the extracted rail information. 6 is a histogram of point cloud intensity levels in an exemplary point cloud partition. This is a visualization of the output of the trajectory detection mechanism. 1 is a schematic block diagram of a map generation system that uses supervised machine learning. FIG. 1 is a schematic block diagram of a runtime system for vehicle location verification, vehicle control, and map auditing.

  According to one embodiment, the position of one or more moving vehicles, such as trains or autonomous vehicles, without relying on ground units / relays distributed throughout the operating environment for accurate position data. A method and apparatus for determining Some implementations based on trains of such embodiments may be referred to herein as BVRVB-PTCs, PTC vision systems, or machine vision systems.

  Solutions are also disclosed for using the position data to optimize vehicle control and operation, such as the operation of a train in a rail system. The track embodiment can use a series of sensor fusion and data fusion techniques to obtain the trajectory position with improved precision and reliability. Such an embodiment may also be used to synchronize train speeds and avoid red light to optimize fuel based on terrain for automatic braking of the train to handle red light violations on the track. It can be used for anti-collision systems and not only for trains but also for preventive maintenance of tracks, rails, and gravel substrates under the tracks. Some embodiments may use back-end processing and storage components to keep track of asset location and health information (accessible by moving vehicles or by railway operators through reports).

  In addition to location verification, it may be desirable for autonomous driving embodiments to make good use of highly detailed infrastructure and landmark maps. These maps can be used to direct the flow of traffic in the real world and to plan the route a vehicle travels from a source to a destination. The three-dimensional nature of the map, in addition to their accuracy in representing the material world, in predicting events beyond their sensing range, indenting those sensors into arbitrary assets, and in landmarks Assist the vehicle in estimating the location of the vehicle. By utilizing highly detailed three-dimensional (semantic) maps for pseudo-static assets, vehicle resources are freed up to observe dynamic objects around it.

  The PTC vision system may include modules that handle communications, image capture, image processing, computing devices, data aggregation platforms that interface with train signal buses and inertial sensors (including onboard and position sensors).

  FIG. 1 illustrates an exemplary flow operation of a train control system. In step S100, the train undergoes normal operation. In step S105, the train state is retrieved from the data aggregation platform (described below). In step S110, the train position is refined. In step S115, the arm signal state is identified from the local environment sensor information. In step S120, feedback is applied. Train speed can be adjusted (step S125) and alarms and / or notifications can be issued (step S130). Further details regarding each of these steps are described herein below.

  Referring to FIG. 2, the PTC vision system may include one or more of the following. Typically, data aggregation platform (DAP) 215, visual device (VA) 230, positive train control computer (PTCC) 210, human machine interface (HMI) 205, GPS reception, communicating via LAN or WAN communication network 240 And a vehicle communication device (VCD) 220.

  The components (eg, VCD, HMI, PTCC, VA, DAP, GPS) may be integrated into a single component or may be modular in nature, virtual software or physical hardware It may be a device. Each component in the PTC vision system may have its own power supply or may share one power supply with the PTC. The power supply used for the components in the PTC vision system may include components that cannot be disturbed due to a power outage.

  The PTCC module maintains the state of information passed between modules of the PTC vision system. PTCC communicates with HMI, VA, VCD, GPS, and DAP. The communication may include providing information (eg, data) and / or receiving information. The interface (eg, bus, connection) between any module in the ecosystem may include any conventional interface. The ecosystem modules may communicate with each other, human operators, and / or third parties (eg, another train, conductor, train operator) using any conventional communication protocol. Communication may be accomplished via wired and / or wireless communication links (eg, channels).

  The PTCC may be implemented using any conventional processing circuit including a microprocessor, computer, signal processor, memory, and / or bus. The PTCC may perform any computation suitable to perform the functions of the PTC vision system.

  The HMI module may receive information from the PTCC module. Information received by the HMI module includes geolocation (eg, GPS latitude and longitude coordinates), time, recommended speed, directional course (eg, azimuth), track ID, distance / driving between neighboring trains on the same track Distance, distance between neighboring trains on adjacent tracks / running interval, next station, previous station, or station of interest including station between origin and destination, for current track segment used by train The state of the virtual or physical arm signal, the state of the virtual or physical arm signal for the next and previous track segment in the route of the train, and the virtual or segment for the track segment sharing the track linkage with the current track It may include the state of the physical arm signal.

  The HMI module may provide information to the PTCC module. Information provided to the PTCC may include information and / or requests from the operator. The HMI may process (eg, format, reduce, adjust, correlate) the information before providing the information to the operator or PTCC module. Information provided to the PTCC module by the HMI includes a conductor command to decelerate the train, a conductor request to bypass certain parameters (eg, speed limits), and a conductor confirmation response of a message (eg, failure, status information) Any of the interests related to the conductor's train request for additional information (eg diagnostic procedures, events along the track, or other points of interest along the track), and the train movement of the conductor Other information may be included.

  The HMI provides a user interface (eg, GUI) to a human user (eg, conductor, operator). A human user may provide information to the HMI module by manipulating the controls (eg, buttons, levers, knobs, touch screen, keyboard) of the HMI module, or requesting information from the vision system. Also good. The operator may attach a user interface to the HMI module. The user interface may communicate with the HMI module by antennal operation, wired communication, and / or wireless communication. Information provided to the user by the HMI module includes recommended speed, current speed, efficiency score or indicator, driver profile, roadside signal status, station of interest, map view of inertial metrics, fault message, alarm, locomotive control A conductor interface for activation of the vehicle and a conductor interface for acknowledgment of messages or notifications.

  The VCD module performs communication (for example, wired or wireless). The VCD module allows the PTC vision system to communicate with other devices to turn the train on and off. The VCD module may provide wide area network (“WAN”) and / or local area network (“LAN”) communications. WAN communication may be performed using any conventional communication technology and / or protocol (eg, cellular, satellite, dedicated channel). LAN communication uses any conventional communication technology and / or protocol (eg, Ethernet, WiFi, Bluetooth®, WirelessHART, Low Power WiFi, Bluetooth Low Energy, Fiber Optics, IEEE 802.15.4e). You may go. Wireless communication may be performed using one or more antennas suitable for frequency and / or the protocol used.

  The VCD module may receive information from the PTCC module. The VCD may transmit information received from the PTCC module. Information may be transmitted to headquarters (eg, central location), roadside equipment, individual, and / or other trains. Information from the PTCC module includes packets addressed to other trains, packets addressed to the common back-end server, packets notifying operators of the location of the train, packets addressed to roadside equipment, addressing to roadside personnel And may include packets that communicate train locations, any payload from node to node, and packets addressed to third party listeners in the PTC vision system.

  The VCD module may also provide information to the PTCC module. A VCD may receive information from any source and the VCD may transmit information to that source. The information provided to the PTCC by the VCD includes packets addressed from other trains, packets addressed from the common back-end server, passing feedback to the conductor or train, packets addressed from roadside equipment, from roadside personnel It may include packets addressed and communicating personnel locations, any payload from node to node, and packets addressed by third party listeners of the PTC vision system.

  The GPS module may include a conventional global positioning system (“GPS”) receiver. The GPS module receives signals from GPS satellites and uses the information provided by the signals to determine the geographic location and time of the receiver (eg, UTC time). The GPS module may include one or more antennas for receiving signals from satellites. The antenna may be arranged to reduce and / or detect multipath signals and / or errors. The GPS module may maintain a historical record of geographic location and / or time. The GPS module may determine the traveling speed and direction of the train. The GPS module may receive correction information (eg, WAAS, difference) to improve the accuracy of geographic coordinates determined by the GPS receiver. The GPS module may provide information to the PTCC module. Information provided by the GPS module includes time (eg, UTC, local), geographic coordinates (eg, latitude and longitude, north and north east distance), correction information (eg, WAAS, difference), speed, and travel. Directions may be included.

  The DAP may receive (eg, determine, detect, request) information regarding trains, train systems (eg, hardware, software), and / or train operating conditions (eg, train conditions). For example, DAP is train speed, train acceleration, train deceleration, braking force (eg applied force), brake pressure, brake circuit status, train wheel traction, inertial metering, fluid (eg oil, fluid) Information regarding pressure and energy consumption may be received from the train system. Information from the train may be provided via a signal bus that is used by the train to transport information regarding the state and operation of the train system. The signal bus may be one or more conventional signal buses, such as a field bus (eg, IEC 61158), a multifunction vehicle bus (“MVB”), a wire train bus (“WTB”), a controller area network bus (“CanBus”). ), A train communication network (“TCN”) (eg, IEC 61375), a process field bus (“Profibus”), and the like. The signal bus may include devices that perform wired and / or wireless (eg, TTE Ethernet) communication using any conventional and / or proprietary protocol.

  The DAP may further include any conventional sensor for detecting information not provided by the train system. The sensor may be deployed anywhere on the train (eg, installed and attached). The sensor may provide information directly to the DAP and / or via another device or bus (eg, signal bus, vehicle control unit, regional train bus, multi-function vehicle bus). The sensor may detect any physical property (eg, density, elasticity, electrical properties, flow, magnetic properties, momentum, pressure, temperature, tension, speed, viscosity). The DAP may provide information about the train to other modules in the PTC ecosystem via the PTCC module.

  The DAP may receive information from any module in the PTC ecosystem via the PTCC module. The DAP may provide information received from any source to other modules in the PTC ecosystem via the PTCC module. Other modules may perform their respective functions using information provided by or through the DAP.

  The DAP may store the received data. The DAP may access stored data. The DAP may create a historical record of received data. A DAP may relate data from one source to another source. A DAP may relate one type of data to another type of data. The DAP may process (eg, format, manipulate, extrapolate) data. The DAP may store data that can be used at least in part to derive the signal status of the track the train is traveling on, the geographic location of the train, and other information used for positive train control. .

  The DAP may receive information from the PTCC module. Information received by the DAP from the PTCC module may include: Requests for train status data, requests for braking interface status, commands to drive train behavior (speed, braking, tractive force), requests for fault messages, acknowledgments for fault messages, requests for warning in trains Requests for notification of alarms issued in trains, and requests for roadside equipment conditions.

  The DAP may provide information to the PTCC module. Information provided by the DAP to the PTCC module may include data from the train signal bus regarding train status, request acknowledgments, fault messages on the train bus, and roadside equipment status.

  The VA module detects the environment around the train. The VA module detects the environment in which the train travels. The VA module is a trajectory on which the train travels, a trajectory adjacent to the trajectory traveled by the train, an aspect of the roadside signal (arm signal, mechanical, light, position) (eg, along the trajectory) (eg, appearance), Infrastructure (eg, bridges, elevated crossroads, tunnels) and / or objects (eg, humans, animals, vehicles) may be detected. Additional examples include PTC assets, ETCS assets, tracks, signals, signal lights, settling speed limits, catenary structures, catenary overhead lines, speed limit signs, roadside safety structures, intersections, pavements at intersections, main and side tracks Gap point location for rolling mills, gap / structural gauge / kinematic tolerance, start and end of trajectory detection circuit in no-signal territory, hut, station, tunnel, bridge, shelter, slope, curve, switch, sleeper , Ballasts, culverts, drainage structures, vegetation entrances, frogs (crossing of two rails), railroad crossings, integer miles, interchanges, interlock / control point locations, maintenance facilities, miles, and other signs and signals .

  The VA module may detect the environment using any type of conventional sensor that detects physical properties and / or physical characteristics. The sensors of the VA module may include cameras (eg, steel, video), remote sensors (eg, light detection and ranging), radar, infrared, motion, and distance sensors. The operation of the VA module may depend on the geographical location of the train, the track conditions, the environmental conditions (eg weather), the train speed. The operation of the VA may include the selection of sensors that collect sensor information and sampling rate.

  The VA module may receive information from the PTCC module. The information provided by the PTCC module may provide parameters and / or settings for controlling the operation of the VA module. For example, the PTCC may provide information for controlling the sampling frequency of one or more sensors of the VA. Information received by the VA from the PTCC module may include sampling frequency, thresholds for sensor data, and sensor configuration for timing and processing.

  The VA module may provide information to the PTCC module. Information provided to the PTCC module by the VA module includes current sensor configuration parameters, current status of sensor operation, sensor capabilities (eg, range, resolution, maximum operating parameters), raw or processed sensor data, processing capabilities, and Data format may be included.

  Raw or processed sensor data can be a point cloud (eg, 2D, 3D), an image (eg, jpg), a series of images, a video sequence (eg, live, recording / playback), a scanned map (eg, 2D, 3D), images detected by light detection and ranging (eg, LiDAR), infrared images, and / or low brightness images (eg, night vision). The VA module may perform some processing of sensor data. Processing may include data reduction, data expansion, data extrapolation, and object identification.

  The sensor data may be processed by the VA module and / or PTCC module to detect and / or identify: Tracks, tracks, distances to objects, and / or infrastructure used by trains, roadside signal indications (eg, meaning, messages, commands, status, current status), track status (eg, passable, substandard), tracks Curvature, direction of the next segment (eg, turn, straight), trajectory deviation from horizontal (eg, downhill, uphill), junction, intersection, interlocking device exchange, train position derived from environmental information, And a trajectory identifier (eg, trajectory ID).

  The VA module may be coupled (eg, attached) to the train. The VA modules may be connected at any position on the train (eg, upper, inner, lower). The connection may be fixed and / or adjustable. The adjustable connection allows the sensor viewpoint of the VA module to be moved relative to the train and / or the environment. The adjustment of the position of the VA may be performed manually or automatically. Adjustments may be made in response to the geographic location of the train, track conditions, environmental conditions around the train, and the current status of sensor operation.

  The PTCC uses its access to all subsystems (eg, modules) of the PTC system to derive trajectory ID and signal status from sensor data obtained from the VA module (eg, judgment, calculation, extrapolation). To do. In addition, the PTCC module may refine the geolocation data using the above-described train operation state information and data from the GPS receiver. The PTCC module may also use information from any module in the PTC environment, including the PTC vision system, to verify and / or interpret the sensor information provided by the VA module. For example, the PTCC may use geolocation information from the GPS module to determine whether the infrastructure or signaling data detected by the VA corresponds to a particular location. Velocity and course (eg, azimuth) information derived from video information provided by the VA module is compared with speed and course information provided by the GPS module to verify accuracy or allow correctness You may judge sex. The PTCC may use the image provided by the VA module along with the location information from the GPS module to prepare the map information provided to the operator via the HMI module user interface. The PTCC may use the current and historical data from the DAP to detect the position of the train using dead reckoning. The position determination may be correlated to location information provided by the VA module and / or the GPS module. The PTCC uses position information from the VA module and / or GPS module, or even dead reckoning position information from the DAP, for position determination that can be correlated and / or corrected (eg, refined). , Communication from other trains or roadside radio repeaters (eg, ground unit) may be received via the VCD module. In addition, track ID, signal status, or train position may be required to be entered by the operator via the HMI user interface for further correlation and / or verification.

  The PTCC module may also provide the conductor with information and calls for activities (eg, messages, warnings, suggested activities, commands) via the HMI user interface. Using the control algorithm, the PTCC ignores the conductor and uses the integration with the braking or traction interface to activate changes in train behavior (eg function, operation) to adjust the speed of the train. May be. The PTCC handles the routing of information by accounting for any receiver (s), data stream payload, frequency, route, and duration, and shares train status with third party listeners and devices.

  PTCC is also a roadside signaling system from a common backend server in the control room, from the track operator, from the control room terminal, or from the conductor, or in other third-party listeners who have contracted PTC vision systems or train data. Or, a packet of information may be dispatched / received from a module, either automatically or through a call to activity.

  The PTCC may also receive information regarding assets near the location of the moving vehicle. The PTCC may use VA to collect data regarding PTC and other assets. The PTCC may also process (or forward) newly collected data to audit and enhance information in the backend database.

  Algorithm: The track identification algorithm (TIA) depicted in FIGS. 6-7 determines which track is currently being used by all vehicles. TIA generates a superimposed feature data set by superimposing features from the 3D LIDAR scanner and FLIR camera on the on-board camera frame buffer. A superset of features (global feature vectors) allows three orthogonal measurements and predictions of the trajectory.

  Thermal features from the FLIR camera are used to identify (e.g., isolate, locate, detach) the track trajectory's thermal properties, and can use arbitrary regions (spatial and temporal filters) in the global feature vector. It may be generated.

  Range information from the 3D LIDAR scanner's 3D point cloud data set is used to identify the altitude of track trajectories and can also generate arbitrary regions (spatial and temporal filters) in the global feature vector. Good.

  Line detection algorithms may be utilized on 3D point cloud data sets for on-board cameras, FLIR cameras, and 3D LIDAR scanners to further increase confidence in trajectory identification.

  Color information from onboard cameras and FLIR cameras may be used to similarly generate arbitrary regions (spatial and temporal filters) in the global feature vector.

  The TIA may look for overlap in any region from multiple orthogonal measurements on the global feature vector to increase redundancy and confidence in the trajectory identification data.

  The TIA may use arbitrary region data to filter out false positives when any region does not overlap in the global feature vector.

  The TIA may process feature vectors in any region to identify trajectory width, distance, and curvature.

  The TIA may further verify the track identification process by examining the rate at which the track tracks concentrate towards a point. Moreover, the track trajectory slope may also be used to filter out noise in the global feature vector data set.

  The TIA may consider the spatial and temporal consistency of the feature vector before identifying the relative offset position of the train among the multiple track tracks.

  Directional paths may be obtained by sampling the GPS receiver multiple times to generate a temporal profile of movement in geographic coordinates.

  A list of potential absolute trajectory IDs may be obtained through a query to a locally cached GIS dataset or a remotely hosted backend server.

  In situations where the GPS receiver is out of sync with the GPS satellite, an odometer and directional track may be used to calculate the dead reckoning offset.

  The TIA compares the position of the relative offset of the train between multiple track tracks and the reference to a list of potential absolute track IDs to identify the absolute track ID that the train is using. .

  After the TIA has obtained an absolute trajectory ID, a global feature vector sample may be annotated with geolocation (eg, geographic coordinates) information and trajectory ID. This allows the TIA to directly determine the trajectory position in the future using the global feature vector data set. This machine learning approach reduces the computational cost of retrieving an absolute trajectory ID.

  The TIA may further match global feature vector samples from a local or backend database with a spatial transformation. Spatial transformation parameters may be used to calculate an offset position from a reference position generated from the matching.

  In addition, TIA utilizes global feature vectors from multiple points in space or using various image processing techniques (eg, image stitching, geometric registration, image calibration, image fusion). Features may be pasted from a single point in space. This results in a superset of feature data having collated global feature vectors from multiple points or a single point in space.

  Using the superset data, the TIA can normalize the offset position with respect to the relative trajectory ID before determining the absolute trajectory ID. This is useful when the trajectory is outside the range of the visual device (VA). This function is depicted in FIG.

  The TIA is a core component in a PTC vision system that eliminates the need for wireless repeaters, signal lights, or ground units to obtain location data. TIA also allows track operators to annotate newly constructed track trajectories for those network-wide GIS datasets that are authoritative for mapping roadside equipment and infrastructure assets.

  The signal state algorithm (SSA) described in FIG. 8 determines the signal state of the track currently used by the train. The purpose of this component is to ensure that the operation of the train follows the expected operating parameters of the track operator or mode control room or central control room. The compliance of the inertial metric of the train along the track can be audited in a distributed environment with multiple backend servers or in a centralized environment with a common backend server. The ability of the train to obtain an absolute track ID is important to correlate the brace signal state to the track ID utilized by the train. Audit signal compliance is enabled once the correlation between the arm signal status and the absolute trajectory ID is established. The placement of the sensor is important for efficiently determining the arm signal state. FIG. 4 depicts an example with a 3D LIDAR scanner facing forward and mounted on the top of the train roof.

  SSA considers the absolute track ID utilized by the train to audit train signal compliance. Once the trajectory correlation to the brace signal is complete, the signal state from that brace signal may trigger a call to activity as feedback to the train or conductor.

  Correlation of track trajectory to arm signal status is possible by analyzing regulatory specifications on roadside signal schemes from track operators. Using the regulatory document, the spatiotemporal consistency of the arm signal may be compared with the spatiotemporal consistency of the track. A scoring mechanism may be used to select the best candidate arm signal for the current rail track utilized by the train.

  A local or remote GIS dataset may be queried to confirm the geolocation of the brace signal.

  The local or remote signaling server may be queried to confirm that the signal state in the brace signal matches what the PTC vision system is extrapolating.

  Use areas where signal status is available to trains via wireless communication to verify the accuracy of the PTC vision system and further enhance the feedback provided to the machine learning device to help adjust the PTC vision system Good.

  The structure of the arm signal may be analyzed using a 3D point cloud data set acquired from the PTC vision system. If the structure of any object matches the expected specification as defined by the regulatory body for the arm signal in that rail passage, the object is annotated as a candidate scoring mechanism referred to above, May be added.

  Infrared images captured through a FLIR camera may be used to identify light emitted from roadside arm signals. In situations where a red signal is emitted from a candidate arm signal that is correlated to the track on which the train is currently on, a call to activity is dispatched to the HMI onboard the train for signal compliance. After a failure of the train that follows the arm signal that is correlated to the track on which the train is currently traveling, a call to activity is dispatched directly to the train with the braking interface for signal compliance.

  The spectrum of colors in an image captured through a PTC vision system may be segmented to calculate a centroid used to identify a blob resembling a green, red, yellow, or double yellow light signal . The spatio-temporal consistency of the arm signal may be verified using the specifications from the regulatory body using the spatial coordinates and size of the center of gravity of the small mass.

  A spatio-temporal consistency profile of the track may be generated by analyzing the curvature of the track, the spacing of the rails on the track, and the concentration of the track spacing toward a point on the horizon. The spatio-temporal consistency profile of the brace signal analyzes the following components: the altitude of the brace signal, the relative spatial distance between points in space, and the orientation and distance with respect to the track currently used by the train May be generated.

  The backend server may be queried to notify the train of expected arm signal conditions along the track trajectory segment that the train is currently using.

  The backend server may be queried to notify the train of expected arm signal conditions along the track track segment identified by absolute track ID and geolocation coordinates.

  As depicted in FIG. 3, the location refinement algorithm provides a highly reliable geolocation service onboard the train. The purpose of this algorithm is to ensure that no loss of geolocation service occurs when a single sensor fails. The PRA relies on redundant geolocation services to obtain trajectory positions.

  A fairly accurate geolocation coordinate may be obtained using GPS or differential GPS.

  The offset position can be calculated using tachometer data together with the directional course information.

  The WiFi antenna may scan the SSID along with the signal strength of each SSID while the GPS is moving, and later use the medium access control (MAC) address (or any unique identifier associated with the SSID). The geolocation coordinates may be quickly determined. The SSID signal strength during the scan with the WiFi antenna may be used to calculate the position relative to the original point of measurement. The PTC vision system may choose to insert the SSID profile (SSID name, MAC address, geolocation coordinates, signal strength) into the database as a reference point based on the trust in geolocation of the current train.

  The global feature vectors generated by the PTC vision system may be used to search for geolocation coordinates to further ensure the accuracy of the geolocation coordinates.

  A scoring mechanism that takes samples from all the components described above filters out inconsistent samples that may hinder the train's ability to obtain geolocation information. Moreover, the samples may have different weights based on the performance and accuracy of each sub-component in the PRA.

  High level process description of PTC vision system

  In this column, reference is made to the flowchart shown in FIG. The PTC vision system samples train conditions from the various subsystems described above. Train status is defined as a comprehensive overview of tracks, signals, and onboard information. In particular, the status includes the trajectory ID, the signal status of the associated signal, the associated onboard information, location information (pre- and subsequent refinements, reference PRA, TIA, and SSA algorithm described), and back-end server Consists of information obtained from. These backend servers maintain information about the railway infrastructure. The asset back-end database is accessed remotely by rail operators and personnel as well as moving vehicles. For example, a moving train and its conductor use this information to predict signals along the route. Operators and maintenance personnel can access track information, for example. These reports and notifications relate to signals and signs, structures, trajectory features and assets, safety information.

  After collecting this condition, the PTC vision system issues a notification (local or remote), optionally generates an alarm mounted on the train, and various onboard subsystems (eg, traction interface, braking interface, traction interface) By interfacing with the slip system), the inertial weighing of the train can be controlled automatically.

  Sensor stage

  Onboard data: The onboard data component represents the unit from which all of the data extracted from the various train systems is collected and made available. This data usually includes time information, diagnostic information from various onboard devices, energy monitoring information, braking interface information, location information, signal status obtained from the train interface to roadside equipment, onboard or on other trains Environmental conditions obtained through other VA devices, and any other data from components useful in positive train control.

  This data becomes available in the PTC vision system for other components and can be transmitted to remote servers, other trains, or roadside equipment.

  Location data is strategically important to ensure that trains operate within safety tolerances that meet Federal Railway Administration PTC standards. In this regard, roadside equipment is currently used by industry to accurately determine vehicle position. The output of the location service described above (eg, TIA & SSA) provides a relative trajectory position based on a computer vision algorithm.

  The relative position can be obtained by using a single sensor or multiple sensors. The position we obtain is usually returned as an offset position indicated as a relative trajectory number. Directional tracks can also be a factor in the construction of queries to obtain absolute position from feedback to the train.

  Absolute position is cached local database, or cached local data set, remote database, remote data set, relative offset position using onboard inertial metric data, GPS samples, Wi-Fi SSID, and Can be obtained from any of the respective signal strengths or through synchronization with existing roadside signaling equipment.

  The various types of data sets we use include, but are not limited to, 3D point cloud data sets from onboard cameras, FLIR imaging, and video buffer data.

  Once the location is known, this information can be used to correlate signal conditions from roadside signaling to the corresponding trajectory. Location services can also be made public to third party listeners. On-board components defined in the PTC vision system can function as listeners to location services. In addition, the train can scan the MAC IDs of networked devices in the surrounding area and utilize MAC ID filtering for any application that these networked devices are utilizing. This is useful for generating context-aware applications that follow the MAC ID pairing of third party devices (eg mobile phones, laptops, tablets, station servers, and other computing devices) with train geolocation information It is.

  The track signal condition is important to ensure that the train always follows the PTC safety tolerance. The functional scope of the PTC vision system includes extrapolating signal values from roadside signaling (arm signal conditions). In this regard, the communication module or visual device may identify the signal value of the roadside device. In areas where the signal cannot be recognized, the central back-end server can relay information to the train as feedback. This information can also enhance vision-based signal extrapolation algorithms (eg, TIA & SSA) when roadside devices are equipped with wireless communications. Data sets are used at the discretion of the PTC vision system.

  The data set collected by the PTC vision system can be used to identify trajectory features from the rest of the data in the device and to identify relative trajectory positions. A relative trajectory position can be transmitted to the back-end server together with the directional course information, and an absolute trajectory ID is obtained. The absolute trajectory ID indicates trajectory identification as listed by the operator. This payload is optional for the train and allows smooth operation between multiple operators without having an operator specific software stack on the train. Operator agnostic software allows the train to operate with excellent interoperability even when the train is traveling through infrastructure from different rail operators. Since the payload is arbitrary, the train is inherently interoperable even when switching between rails and operators. When all the vehicles travel along the track, the data necessary for updating the asset information is generated by the visual device. This data is then processed not only to verify the integrity of certain asset information, but also to update other asset information. Lost assets, damaged assets, or tampered ones can then be detected and reported. Each time a vehicle with a vision device goes on track, the current state of the infrastructure can also be verified and operational safety can be assessed. For example, gap measurements are taken to ensure that there are no obstructions blocking the train path. The amount of ballast that supports the track is estimated and monitored over time.

  Backend:

  The backend component has many purposes. For one, it receives, annotates, stores and forwards data from trains and algorithms to various local or remote subscribers. The back end also hosts many processes to analyze the data (in real time or offline) and then generates a corrected output. This output is then sent directly to the train as feedback or relayed to the command and dispatch center or train station.

  As part of the aforementioned process, an algorithm for reducing train operating intervals and optimizing the flow on a fixed path, an algorithm for optimizing the entire network flow by considering individual trains or paths, And a collision avoidance algorithm that constantly monitors the location and behavior of the train.

  The back end also hosts an asset database that is queried by moving trains to obtain asset and infrastructure information as required by all vehicle operating regulations. This database holds the following assets along with related information and features: PTC asset, ETCS asset, track, signal, signal light, settling speed limit, catenary structure, catenary overhead line, speed limit sign, roadside safety structure, intersection, pavement at intersection, gap point for main and side switch Location, gap / structure gauge / kinematic tolerance, start and end of trajectory detection circuit in no-signal territory, hut, station, tunnel, bridge, shelter, slope, curve, switch, sleeper, ballast, culvert, drainage Structures, vegetation entrances, frogs (intersection of two rails), railroad crossings, integer mileage marks, interchanges, interlock / control point locations, maintenance facilities, mileage signs, and other signs and signals.

  All vehicles use the information queried from the database to refine the trajectory identification algorithm, position refinement algorithm, and signal condition detection algorithm. A train moving along / in the vicinity of the track (or any other vehicle that utilizes a machine vision device) collects the data necessary to store, verify, and update information in the database . The backend infrastructure also generates alerts and reports on asset status for various railroad personnel.

  Feedback stage

  Automatic control:

  There are several ways in which a train can be controlled using a PTC vision system (eg, the application in FIG. 5). The output of the sensor stage can trigger certain activities independently of any other system. For example, after detecting a red light violation, the braking interface may be automatically triggered to attempt to stop the train.

  Certain control commands can also reach the train through its VCD. As such, the backend system can, for example, instruct the train to increase its speed, thereby reducing the operating interval between trains. Other train subsystems may also be operated through the PTC vision system as long as they have access to the locomotive itself.

  Built-in alarm:

  Feedback can also be delivered to locomotives and conductors through alarms. For example, in the case of a red light violation, an alarm can be displayed on the HMI. The alarm can be accompanied by any automatic control or can be present on its own. The alarm can be stopped by being acknowledged or stopped independently.

  Notification (local / remote):

  The feedback can be in the form of notification to the conductor via the user interface of the HMI module. These notifications may describe data detected and collected locally through the PTC vision system, or data obtained from the backend system through the VCD. These notifications may require a listener or may be enabled at all times. An example of a notification may relate to a speed recommendation for the conductor to follow.

  Backend architecture and data processing.

  The back end may have two modules: data aggregation and data processing. Data aggregation is a module that is responsible for aggregating information between the train and the central back end and defining the route. Make recommendations to trains using data processing components. Communication is bi-directional and this back-end server can supply all of the various possible applications from the PTC vision system.

  Possible applications for the PTC vision system include: signal detection, track detection, speed synchronization, extrapolation of track interlock and relaying it back to other trains in the network, fuel optimization, Collision prevention system, rail detection algorithm, track fault detection or preventive derailment detection, track performance metric, image stitching algorithm to generate comprehensive reference data set using samples from multiple runs, eg preventive maintenance Passing train imaging for fault detection and / or passing train vibration characteristics, geolocation or geofiltering services based on imaging, geolocation or geofiltering based on SSID, and GPS + inertial metric + computer vision Including sensor fusion based algorithm.

  According to other embodiments, remote sensing and location features can be utilized to implement a runtime system in an automated vehicle, such as an autonomous vehicle. FIG. 25 is a schematic block diagram of an exemplary in-vehicle system for vehicle location confirmation and / or control. The in-vehicle runtime engine (“IVRE”) 2500 and the vehicle decision engine 2510 are calculation and control modules, typically based on microprocessors that are mounted and implemented locally on the vehicle. The local 3D map cache 2530 stores map data associated with an area surrounding the rough location of the vehicle, as determined by the GPS and IMU sensor 2520, and a communication module (eg, may include a cellular data transceiver) It may be updated periodically or continuously from the remote map storage via 2540. The machine vision sensor 2550 may include one or more mechanisms for detecting the local environment closest to the vehicle, such as a LiDAR, a video camera, and / or a radar.

  During operation, the IVRE 2500 achieves vehicle position confirmation by obtaining a coarse vehicle position from the onboard GPS and IMU sensor 2520. Machine vision sensor 2550 generates an environmental characteristic indicative of the local environment around the vehicle, which is passed to IVRE 2500. The IVRE 2500 queries the local 3D map cache 2530 using the environmental characteristics received from the machine vision sensor 2550, and features or objects observed in the vehicle's local environment are stored in the 3D semantic map stored in the cache 2530. Match a known feature or object with a known location. By comparing the observed position of the vehicle relative to local features or objects to the location of those features and objects on the map, the vehicle position uses GPS where the margin of error is potentially measured in centimeters Thus, it can be refined with much greater accuracy than is typically possible.

  Detailed vehicle position and other observations or calculated information can be utilized to implement other functions, such as vehicle control and / or map auditing. For example, data from machine vision sensor 2550 may include graphs and other data, as described elsewhere in this specification, for IVRE 2500 to determine the centerline for the lane in which the vehicle is traveling. Analysis can be performed using an analysis mechanism. The IVRE 2500 can also operate to obtain meaning (e.g., events and triggers) along the route of the vehicle. Using available computing resources, asset information previously acquired from a centralized map and previously observed (and stored, for example, in the local 3D map cache 2530) is captured by the machine vision sensor 2550. A centralized map data source can be audited by comparing with asset information derived from temporal data. Thereby, IVRE 2500 may identify leakage errors (ie, observed assets that were leaked from centralized map data) as well as leakage errors (ie, assets in centralized map data that are not observed by machine vision sensor 2550). it can. Such errors may be stored in the cache 2530 and then communicated to the central map repository via the communication module 2540.

  In some embodiments, the map data audit by the local vehicle may be initiated by the centralized control server communicating with the vehicle via the communication module 2540. For example, if time passes, the last audit of the map segment exceeds the threshold, so the centralized control server can request an audit from a local vehicle traveling through the target area. In another example, if one vehicle reports a discrepancy between the centralized map data and the locally observed situation, the centralized control server may have one or more other vehicles moving in the discordant area. Confirmation audits may be required. The audit request may relate to various combinations of geographic areas and / or mapping layers.

  In some embodiments, it may be desirable to utilize information, such as precise vehicle location, assets and meaning, and navigation information, as input to the vehicle decision engine 2510. The vehicle decision engine 2510 can operate to control various other systems and functions of the vehicle. For example, in an autonomous driving implementation, the vehicle decision engine 2510 may use lane centerline information and precise vehicle position information to steer the vehicle and maintain the center lane position. These and other vehicle control operations can be beneficially realized using the systems and processes described herein.

  Semantic map generation using geospatial data

  A map is a collection of objects, their locations, and their properties. The map can be divided into multiple layers, each layer being a group of objects of the same type. The location of each object is defined with geometric attributes (eg, the location of a pole can be a point in 3D space, while the signal is drawn by drawing a polygon around it Can be specified). A map becomes “semantic” when semantic associations between different objects and layers are also recorded. For example, maps composed of the centerlines of various lanes on roads and signs located around the infrastructure are semantically labeled when the associations between the various signs and centerlines are recorded. . This can be achieved by generating a mapping between the unique identifier of the sign and the unique identifier of the lane with which the sign is associated. Map semantics generate more context for the vehicle or user consuming the map. Semantic maps can also be packaged with regulatory information from various transportation authorities.

  The physical geometry of any asset can be described in the map. The geometric features used to describe the shape include points, lines, polygons, and arcs. The features are typically three-dimensional, but they can be projected into a two-dimensional space with no depth / elevation. In general, a semantic map can be recorded and supplied with different coordinates and reference frames. There is also a transformation that allows the map to be projected from one coordinate reference frame to the next. These maps can be packaged and supplied in different formats. Common formats include GeoJSON, KML, shapefile, and the like.

  In some embodiments, the geospatial data used for semantic map generation comes from LiDAR, visible spectrum cameras, infrared cameras, and other optical equipment. The act of obtaining machine vision data for map generation is called surveying, and this data is georeferenced to a specific location on the earth. The output is a three-dimensional set of data points along with image and video supplies in the visible spectrum and other frequencies. There can be many different hardware platforms for data collection. The collection vehicle is also undefined (for aviation, for movement, for ground). Geospatial data is first collected using the collection vehicle that is the origin of the reference frame. By locating the vehicle throughout the survey (eg, using an inertial measurement unit (IMU) and a global positioning system (GPS)), the image, laser scan, and video supplies are then And it is georeferenced. Data generated in a survey can be streamed or stored locally for later consumption.

  Some embodiments of the vehicle localization and local environment sensing system described herein benefit from the use of point cloud survey data. A semantic map derived from point cloud survey data may provide the vehicle with a high level of detail and information regarding the current or anticipated local environment of the vehicle, which, for example, aids relative vehicle location. Or to serve as input data to an autonomous control decision making system (eg, automatic braking, steering, speed control, etc.). In addition or alternatively, the point cloud data measured by the vehicle is compared to the point cloud data measured previously, and the situation or change in the local environment, e.g. fallen trees, overgrown plants, changed signs, Lane closures, track or road obstacles, or the like may be detected. Detected changes in the environment can be used to further update the semantic map.

  However, an increase in the level of point cloud survey data detail can result in a fairly large data set, which can be expensive to service or process or store or process by the vehicle. Can be. For example, a 3D rail surveying system based on LiDAR that travels linearly along a rail track may generate more than 20 GB of geospatial data for a scan per kilometer. The raw point cloud data generated by the LiDAR scan typically then requires additional processing to extract useful asset information.

  A three-dimensional semantic map is traditionally generated from point cloud data and other geospatial data through the use of 3D visualization software. FIG. 11A illustrates a typical prior art process for extracting asset information from point cloud data. In step S1100, the surveying procedure generates a point cloud data set using, for example, a LiDAR surveying device. In step S1105, raw point cloud data is visualized. Typically, Geographic Information System (GIS) analysts use point and click methods to automatically identify, annotate, and classify important assets in the data. The first step in the GIS analyst process is to separate the terabyte point cloud data into smaller manageable segments. This is due to the fact that current personal computers (memory / computing capabilities) are limited and terabytes of LiDAR data cannot be managed immediately.

  The GIS analyst then traverses each of the smaller sections of the point cloud using 3D visualization software. As they progress through each of their divisions, GIS analysts delineate and annotate important assets. Finally, the annotated assets of each GIS analyst are combined into one map (step S1110). Various file formats and software systems can create additional difficulties in merging separate data sets.

  The extraction of values from point cloud data is limited by both prior art processes and infrastructure. Point-and-click annotation is manual, tends to be slow and error-prone. In addition, conventional file-based systems prevent GIS developers and administrators from effectively managing the growing point cloud data set.

  FIG. 11B illustrates an alternative approach for extracting asset information from raw point cloud data. In step S1150, surveying is performed to generate raw point cloud data. In step S1155, an asset map is generated directly from raw point cloud data without requiring visualization of a large complex data set or manual annotation of the data.

  FIG. 12 illustrates a computing device for quickly and effectively extracting asset information from a large number of point cloud data sets. FIG. 13 illustrates a process for using the apparatus of FIG. Preferably, the components in the apparatus of FIG. 12 are implemented using internet-connected cloud computing resources that may include one or more servers. The front end component 1200 includes a data upload tool 1205, a configuration tool 1210, and a map retrieval tool 1215. The front end component 1200 provides a mechanism for the end user to interact with and control the computing device.

  Using the data upload tool 1205, the user can upload LiDAR and other survey data from the local data storage device to the data storage component 1220 (step (S1300). The data storage component 1220 is a distributed file. A system (eg, a Hadoop distributed file system) or other mechanism for storing data may be implemented, and the configuration tool 1210 is accessed via a computing device (not shown) connected to the user's network. And allows the user to define other survey details as well as the format of the uploaded data, and to search and annotate by specifying assets (step S1305). After interacting with 1210 and selecting the desired asset, The user is provided with various options for configuring the output map format, preferably the configuration tool 1210 then requests the desired turnaround time from the configuring user and asks the user for analysis. (Step S1310) The cost estimate includes, for example, the size of the uploaded data set to be analyzed, the number of selected assets (and complexity), the output format, and the selected turnaround. Finally, when the configuration is complete, the user interacts with the configuration tool 1210 to start an analysis job (step S1315).

  Geospatial data uploaded through the front end 1200 is tracked in the database collection. This data is organized by category, geographic area, and other characteristics. As data evolves through the various stages of execution, the associated database entries are updated.

  Point cloud data uploaded through the front-end tool is stored in a secure and replicated manner. To simplify retrieval, data is tiled into different sized tiles in a Cartesian coordinate system. The tile itself is limited to two dimensions and is given a namespace accordingly. Preferably, the tiles are in the X and Y dimensions so that the tiles define a columnar area with a rectangular cross-section that is not highly limited (ie limited to only a range of available geospatial data). It is limited and not limited to the Z dimension that is perpendicular or parallel to the direction of Earth's gravity. In an exemplary implementation, tiles with sides of 1000 m (in the horizontal plane) can be utilized. The file representing the tile then keeps all of the points belonging to a particular geographic area delimited by the tiles and not others. In certain embodiments, tree structures (eg, quadtrees and octrees, etc.) are implemented according to a traversal style for the data.

  The processing of the data for automatically extracting the semantic map from the geospatial data takes place on the computational cluster and is implemented in the processing unit 1240 (embodiment described further below with reference to FIG. 16). . They can access point clouds and other data through a network accessible storage unit 1220. Not only the intermediate results but also the final results are stored in the same way.

  FIG. 14 illustrates a process that can be performed by the apparatus of FIG. 12 after the start of an analysis job. To simplify data processing and to allow implementation of the MapReduce data analysis framework, point cloud data is broken up into chunks by data storage / preprocessing component 1220 (step S1400). These chunks may be, for example, a subset of tiles or a combination of them that are potentially selected to optimize for the desired processing method, available memory, and other execution time considerations. Can do. Individual nodes in the computing cluster (ie, within processing unit 1240) may then process the given data chunk, ie, the geospatial and other data associated with the selected subset or tile combination. it can.

  Point cloud density can be an important factor in determining the number of tiles (or size of tile subsets) to process within the same compute node. In an exemplary embodiment, FIG. 15 illustrates exemplary data including tile size versus number of inner points (represented by diagonal lines), and LiDAR point cloud data measured along a 2 km section of the line. Illustrate the distribution of tile sizes for the set (each tile is represented by a hatch across the diagonal). Data storage and preprocessing component 1220 performs tile aggregation and / or fragmentation prior to supplying data to processing unit 1240 to optimize analysis performance.

  Considering the benefits of tile aggregation, as explained above, the reduced point cloud density can result in reduced processing time. However, low density generally makes the feature detection process more difficult and can result in a higher rate of false positives. The richer the point cloud data, the more accurate the detection process.

  Once the process is started, the job scheduler 1225 generates a queue that includes tasks related to the job as configured in steps S1305 and S1310. The job scheduler 1225 associates one or more of the analysis mechanisms 1250 (typically implementing a variety of different data analysis algorithms) with the task (step S1405) and creates a cluster of machines within the processing unit 1240. The data is processed (step S1410). Given the average time taken before a single node implements the data analysis mechanism (s) 1250 required for tile aggregation of a known average size (eg, 250 MB), the size of the cluster (ie, compute node May be determined so as to satisfy the turnaround time requested in step S1310. For example, considering a sample data set presented for processing, it is estimated that it takes approximately 240 hours of computation time on a desktop computer with 8 cores. Since the data analysis mechanism 1250 is preferably designed to run simultaneously, the job scheduler 1225 can start a cluster of 20 machines each having 4 cores, instead of about 24 hours, The same data set can be processed.

  The processing unit 1240 consists of a collection of computational clusters. The size of the cluster depends on the number of jobs. FIG. 16 illustrates an exemplary computational cluster. Each cluster includes a master instance 1605 responsible for managing the cluster, a fixed number of main compute nodes 1610 that also store data within the data storage system 1220, and a variable number of “spot” instances 1620. In some embodiments, it may be desirable to resize the primary instance 1610 so that it can process the entire data and meet turnaround time requirements, for example, Operates based on cost and / or job time constraints. In other embodiments, a computational cluster consisting entirely of spot instances or consisting entirely of primary nodes may be utilized.

  Once a properly configured compute cluster is generated, the data storage and preprocessor component 1220 may pass a stream of data chunks (eg, an aggregation of tiles that satisfy the desired data subset size) to the processing unit 1240. Guide (step S1415). The main node and spot instance in the processing unit 1240 execute an appropriate data analysis mechanism 1250 to extract asset or feature information from, for example, a 3D point cloud tile.

  Once the data set is processed by the processing unit 1240 and the desired information is extracted, the map generator 1230 is triggered. The map generator 1230 combines the outputs of the nodes in the processing unit 1240 into a semantic map (step S1420). Report analysis can be derived from a semantic map by performing a query to analyze specific assets and combinations thereof.

  Map generator 1230 may also include an annotation integrity verifier that operates to verify the integrity of the annotated data set over time. In some applications, the location may be surveyed repeatedly at different times. For example, in track applications, a train equipped with LiDAR or other track surveying vehicles may regularly survey the same length of track to monitor, for example, the health or current status of assets along the track. In some road applications, a survey vehicle equipped with LiDAR may travel along a given part of the road at different times. In other road applications, data captured by LiDAR equipped vehicles, such as autonomous vehicles, may be regularly analyzed, providing a potentially frequent analysis of the local environment at a given location. To do. Each time a new map is generated by the map generator 1230 for a given area, asset or local feature information may be compared with such information contained within the old map. An alert, notification, or event can be triggered when a discrepancy is detected.

  The output of the map generator 1230 is finally made available to the user via the front end 1200 and the map retrieval tool 1215 (step S1425). Once the job is complete and the map is generated, the scheduler 1225 (which monitors the current status of tasks and jobs) generates a notification for the end user.

  Feature maps (including only various asset locations, geometries, and features), as well as semantic maps, can also be stored in a remotely accessible geodatabase. Map data can be retrieved directly or through a server to facilitate querying and collection of results. Maps can be retrieved in their entirety or by selecting any particular area.

  Security, compression, and integrity

  Data and map security can be an important aspect of many embodiments. Preferably, the data upload step S1300 employs end-to-end encryption (eg, AES encryption, etc.) from the user data source to the cloud computing platform. Such encryption may also be utilized for communication between the user's system and the front end 1200.

  In some embodiments, it may be desirable to store raw point cloud data in a compressed format within the data storage component 1220. For example, an exemplary distributed computing cluster with 1 terabyte of storage for every 4 central processing unit (CPU) cores can store 3D point cloud data in its raw form, leading to slower processing times There is sex. This is because the storage infrastructure constrains I / O while the CPU core is often placed in an idle state. This means that the CPU substantially waits for the data to be read from the storage device before processing the data. Compressing raw point cloud data before storing allows the system to reduce the time spent reading data and writing to disk. Accordingly, the data storage component 1220 may include a compression mechanism for compressing point cloud data prior to storage.

  However, storing the compressed raw point cloud data increases processing time. This is because the data needs to be developed by the deployment mechanism before applying the data analysis mechanism 1250. There is typically a clear relationship between the compression ratio of the compressed data and the processing time required to compress and decompress the data. Thus, in some embodiments, the CPU time is continually measured to adjust the data compression ratio, the rate at which data can be read from the storage component 1220, and the rate at which the data can be expanded and processed by the processing unit 1240. It may be desirable to balance as close as possible.

  Many lossless data compression mechanisms may be utilized to handle large numbers of point cloud data sets, as described herein. Examples include LempelZivOberhumer (LZO), GZIP (also based on LempelZiv method), and LASzip (released by Rapidlasso GmbH, hereinafter referred to as LAZ). Figures 17, 18 and 19 show a comparative analysis of these three compression mechanisms. From a compression perspective, the LAZ method results in some CPU time across all compression levels (the higher the compression level, the smaller the output file that is compressed). This method is very attractive as it results in a smaller file size compared to LZO and GZIP. However, LZO and GZIP are optimized for deployment and thus provide an alternative to LAZ in terms of CPU time required for deployment. In some embodiments, a plurality of having different characteristics based on the nature of the data set and the characteristics of the available computing infrastructure (eg, cost and availability) while increasing the speed of data processing. It may be desirable to minimize storage requirements by selecting a compression mechanism from among these mechanisms.

  Machine vision analysis mechanism

  The data analysis mechanism 1250 is typically selected based on the nature of the information desired to be extracted from the point cloud data. It would be desirable to maintain an acceptable detection rate while designing the mechanism 1250 with a very low false positive rate. To add confidence to the generated map, in some applications, a subset of the results may be verified manually by inspecting the original point cloud and raw imaging data.

  Trajectory detection and traversal

  In embodiments that process track point cloud survey data, trajectory detection may be an important first step. Orbit detection can be important. This is because knowledge of trajectory position facilitates asset identification because rules often assign a specific location for each asset to the trajectory.

FIG. 20 illustrates a process for trajectory detection and traversal that may be implemented by the processing unit 1240, for example, in step S1415 of FIG. In step S2000, point cloud data of 100 m × 100 m section is identified for analysis. In step S2010, the 10,000 m ² point cloud segment geometry is analyzed to extract a subset of points associated with the trajectory. Many techniques can be employed to achieve the desired result. In some embodiments, previously classified trajectories can be studied from similar data sets to identify the characteristics of the data near those trajectories, and these characteristics can be derived from newly analyzed data. It serves as a mark of the orbital location in Other techniques include the projection of points in two-dimensional space (eg, based on altitude or pulse intensity) and the use of edge detection mechanisms and transformations to separate regions belonging to the trajectory. In an exemplary use case, the 10,000 m ² point cloud segment in step S2000 may consist of about 1 GB of data, while the trajectory subset output extracted in step S2010 is from about 1 MB of data. It may be made.

  FIG. 21 is a visualization of the point group segment input of 10,000 m 2 in step S2000 and the rail data output extracted in step S2010. Line 2100 represents a trajectory that can be recognized in the point cloud. Line 2110 represents a trajectory that has been obscured during the LiDAR data collection process and whose position has been estimated. This is the result of shadowing, a process that typically occurs when any object is hidden from the direct line of sight of the measuring instrument. Dot 2120 corresponds to the problematic positioning of the LiDAR tripod system, which results in some trajectory segments being disturbed. The location of the unrecognizable trajectory can be inferred by utilizing the known spatial continuity characteristics of the infrastructure (eg, spacing relative to other observed elements) (step S2020).

  Geospatial data presents many dimensions that can be exploited during asset extraction. Image, infrared, video feeds, and / or multispectral sensors can be combined to increase detection confidence and accuracy. Most LiDAR systems include intensity measurements for each point. Classification mechanisms and filters can be added to the system by analyzing the intensity of both in-orbit and off-orbit points for improved trajectory detection rates. FIG. 22 is a histogram of point cloud intensity levels in an exemplary trajectory detection implementation. FIG. 22a generally illustrates the amount of each measured intensity level in the analysis group of point cloud data. FIG. 22b illustrates the same histogram for points in the point cloud identified as corresponding to the trajectory. A simple bandpass filter may be effective in some cases to further narrow the search space for points belonging to the rail. Other classification methods can also be utilized.

  FIG. 23 is a visualization of a portion of an implementation output that includes a trajectory detection mechanism and other asset detection mechanisms. Through operation of the trajectory detection mechanism, trajectory segment 2300 is first identified and then a centerline marker 2310 is established for each trajectory. Once the trajectory and trajectory centerline are identified, subsequent analysis components can traverse the trajectory in the point cloud data, while the high-resolution point cloud data around each point in the centerline. Enjoy 360 degree view.

  Other analysis mechanisms identify and locate other assets or features for inclusion in the semantic map. For example, the aerial wire detection mechanism identifies and locates the aerial wires and delimits them with aerial wire centerline mark 2320. The pole detection mechanism identifies and locates the poles beside the line and positions them using landmarks 2330. These and other features may be included in the semantic map output generated through the systems and methods described herein.

  In some embodiments, the analysis mechanism may be applied continuously, and the output of one mechanism serves as an input to another mechanism. For example, in track applications, local environment assets and elements are regularly exchanged, added, removed, or moved. It may be desirable to regularly inspect gaps on and around the track to ensure safe operation and to ensure that the train does not come into contact with any obstacles. In such applications, trajectory detection mechanisms, such as those described above, may be implemented as part of a series of analysis mechanisms. The output of the trajectory detection mechanism including the trajectory centerline may then be used as an input to the trajectory gap inspection mechanism. A boundary frame is defined with respect to the trajectory centerline, and any object that enters within the boundary is reported. The dimensions of the border frame can be modified to meet various standards.

  The determination of the location of signs, signals, switches, roadside units, and the like is also possible using the detection framework. Once located, classification of these assets is possible given the geometric characteristics of each asset, according to the manufacturer's specifications or other object definitions.

  Another analysis mechanism that can be beneficially employed in track applications is overhead wire inspection. The fictitious wire can be identified in the point cloud data. The altitude of the wire is assessed relative to the track. Areas with curved lines are reported. By using the pole location information, the catenary shape of the wire can also be assessed.

  Although certain analysis components are described in the context of track trajectory detection, it is contemplated that similar analysis mechanisms and methods may be used to potentially identify other types of assets in other applications. And understood. For example, a mechanism similar to the trajectory detection mechanism described herein may be useful in road conditions to identify lane markings and / or curbs.

  Computing paradigm

  Automated extraction of maps can be accomplished by combining computational blocks (hereinafter referred to as “graphs”) into a directed acyclic graph. The blocks included in these graphs have variable complexity ranging from simple averaging and thresholding to transformations, filters, decompositions and the like. The output of one stage of the graph can be fed to any other subsequent stage. The stages do not need to move in sequence, but can be parallelized assuming sufficient information for each stage. When generating a feature map, the graph is generally used to classify or vectorize points within a point cloud that belong to the same category. Vectorization refers to the generation of a line or polygon (often virtual) that passes through a set of points that delimit their centers, boundaries, locations, etc. As such, computational graphs can be used to implement classifiers, clustering methods, routing fittings, neural networks, and the like. Rotation and projection are also often used in conjunction with machine vision processing techniques.

  In order to make the best use of distributed computing, semantic map generation from geospatial data may be parallelized. There are many levels of parallelism that can be implemented. At the highest level, survey data can be divided into arbitrary regions of regular shape, which are streamed to different machines and CPU processes. Similar to the process of FIG. 14, once all processes are complete, the results from each area must then be merged into a “reduction” step. Since boundary conditions arise, embedding an arbitrary region with extra data that is truncated at the end of the process typically removes malformations near the edges. The size of any region, as well as the embedding thickness, is determined by asset or feature graph extraction.

  At another level, parallel operations can be performed when processing is performed along pre-extracted vectors. For example, when searching for signs in the vicinity of a railway track, the data can be traversed by extracting regions around the waypoints along the previously extracted track centerline. Multiple processes can then be processed in parallel along different waypoints in the trajectory.

  Finally, each point can be considered individually when analyzing a particular area. In this traversal method, the voxels surrounding the point are usually extracted and analyzed. This process can also be paralleled if the result of the operation of one point does not affect the operation of any other point.

  There are several traversal methodologies that are employed in the map generation process, and several approaches in which data processing can be parallelized. In addition, using a GPU (graphics processing unit) with a conventional CPU can also lead to significant speed improvements and further assist in reducing turnaround time.

  Geospatial data is not limited to point clouds but extends to images, video supplies, multispectral data, radar, and the like. To improve the accuracy and correctness of the mapping, some embodiments may utilize any additional data source that is available. Several techniques can be utilized to use data from different sources. In some embodiments, the data set can be combined with a pre-processing stage (eg, step S1400) prior to feeding the calculation graph. This approach provides a computational graph with data from multiple sources for processing. In other embodiments, a set of data may be used to generate hypotheses about the asset and its characteristics. Data from other sources can then be used to verify and / or enhance the hypothesis by other analysis mechanisms.

  Machine learning:

  Many machine learning techniques can be implemented to assist the semantic map generation process. Existing annotated maps can be used to train graphs to optimize them and automatically generate accurate semantic maps from geospatial data. The input data to the machine learning system consists of survey data and a corresponding annotated output map. The output of the machine learning system is an elaborated graph that can then be applied to a wider range of survey data to extract the map on a large scale. In some cases, a classified point cloud (in this case, assigned to each point based on which asset the category belongs to) is used and fed into the training process. Otherwise, a vectorized map is used to learn the map generation process and adjust the processing graph. These methods fall into the supervised learning category and rely on enhancing evaluation performance (through error measurement) and desirable performance.

  FIG. 24 illustrates an embodiment of a system that implements supervised machine learning that includes a training component 2400 and a map generation component 2410. The training component 2400 receives raw point cloud data 2420 and sample output 2422 as inputs. In some situations, the sample output 2422 may be verified output data associated with about 1% of the total data set. Sample output 2422 can be classified point cloud data (points belonging to a particular asset category are grouped) and / or vectorization (points, lines, and polygons are drawn on any asset). A map may be included. Training component output 2424 defines an optimized categorization mechanism, such as algorithm coefficients for an analysis mechanism that can be compared to mechanism 1250 in the map generation system of FIG. The training component output 2424 may also define arbitrary regions for the algorithm that should be most effective, may define functional blocks within the computation graph to be utilized, and / or Any feature for a particular asset under consideration may be defined. Training component output 2424 is provided to map generation component 2410 along with the entire corpus of raw point cloud data 2420. Map generation component 2410 then operates to generate map output 2426.

  Unsupervised methods can also be implemented for map generation. Such a process may rely on scale-dependent features to describe context information for individual map points. They can also rely on deep learning to design feature transformations for use with map point features. A set of feature transformations generated by deep learning is used to encode map point context information. Asset members for points can then be transformed by a deep learning algorithm based on the features. Another method revolves around curriculum-based learning where assets are described in the curriculum and then learned in the computational graph. This method may be effective when the shape and characteristics of any asset are regular and do not exhibit a lot of spatial complexity.

  Using these learning schemes, neural networks are often trained in the first step and then applied to the rest of the geospatial data for map extraction.

  Thus, machine learning techniques can assist in the optimization and refinement of computational graphs. These graphs can be manipulated manually or can be learned using the methods described above. The parameter search component is useful for improving accuracy and reducing false positives and negatives. In this process, the various parameters of the computational graph (from any region to the parameters of each function, to the number and nature of the features used in the classifier) can all be adjusted and the output can be monitored. By using a search methodology, the best performance combination of parameters can be found and applied to the rest of the data. This process assumes the availability of a previously annotated semantic map.

  Once the computational graphs are refined to an acceptable performance level, they can be used directly in the vehicle. This corresponds to streaming intelligence from the cloud to the vehicle as opposed to streaming more traditional data from the local environment to the cloud system. With regard to geospatial data, large-scale sensor data may be prohibited. Thus, in some embodiments, locally acquired sensor data (eg, data acquired by sensors attached to a vehicle) is summarized via local computing resources, collected information and / or extracted. Only a subset of the content is sent back to the remote data system. For example, resources corresponding to data storage / preprocessor component 1220, processing unit 1240, and data analysis mechanism 1250 may be implemented in the vehicle to extract semantic map data from the on-board sensor system. Computational graphs similar to those described above for implementation in a cloud-based processing structure can be optimized and tested in a machine learning framework while providing local in-vehicle implementation opportunities. Such an embodiment can utilize the vehicle as a distributed computing platform, constantly updating the content of the centrally maintained map, while streaming all of it to a central cloud-based system Instead, it consumes most of the remotely sensed data in place.

  Although the machine learning implementation described herein can greatly accelerate the development of new graphs and map new features and assets, the learning task is related to lack of training data and accuracy. It can be troubled by problems. The consequences of these problems can include overfitting and performance limits. If the amount of training data is limited, the learning routine can distort the performance of the graph strongly towards the small amount of data available, and prone to failure when new, untrained cases are introduced. Let Regarding performance, the generation of maps for training data is typically a manual process, which is error prone. As such, when the training data itself is not completely accurate, the resulting graph is also not accurate. For example, if a GIS analyst achieves only 80% asset accuracy in their manually generated map, any trained graph on that data will exceed the 80% accuracy threshold. Will be very difficult.

  A simulation environment can be used to address these issues. In a simulation environment, a map is generated programmatically in a number of parameter substitutions to replicate terrain and landmark variations on the Earth's surface. A three-dimensional model is then generated from the map and ray-raced to generate point clouds in a manner as similar as possible to actual data collection. Since the location of every asset is known a priori, a complete map extracted from the point cloud is then available. The variability of the data and the fact that complete ground truth exists for each point cloud considerably increases the range of computational graphs and their accuracy. It also provides a mechanism for understanding the limitations of current computing paradigms.

  However, no matter how much the graph is trained and how many test cases it takes, automated map extraction cannot be perfect. For this reason, a manual quality control (QC) process can be introduced to help find any problem. In order to avoid having to perform QC throughout the map, a level of trust can be generated during the map creation process. This level represents how reliable the graph is in extracting the desired features from the map. QC can then be performed for the region at the lowest percentile of credit.

  Quality control can be performed in several ways. Similar to the generation of semantic maps, GIS analysts can use conventional visualization tools and can overlay raw survey data with automatically extracted maps. Any discrepancies can then be identified and corrected. Another way for QC would be to cloud source effort between multiple online agents. Since each of these agents may not be completely familiar with semantic map generation, the QC work needs to be replicated. The hypothesis can then be approved or rejected by each QC result, reaching the final conclusion with sufficient trials.

  It is important to augment the calculation graph by accumulating QC results. If a discrepancy is detected, a newly simulated area containing the problematic test case can be utilized. Further retraining of the graph may then be considered for use cases in future work.

  While certain embodiments have been described in detail herein for purposes of clarity and understanding, the foregoing description and drawings merely illustrate and illustrate the invention. Not limited to that. Those skilled in the art prior to this disclosure will recognize that these and other modifications and variations can be made to what is disclosed herein without departing from any claims. .

Claims (25)

A device for identifying assets in point cloud survey data,
A front-end component accessible via a digital communication network to receive the point cloud data set;
A data storage component for storing the point cloud data set and subdividing the point cloud data set into a plurality of data chunks;
A processing unit comprising a computing cluster, receiving a data chunk streamed from the data storage component, and applying one or more analysis mechanisms to each data chunk to extract asset information; and
A map generator that combines the asset information extracted from the data analysis mechanism into an output map.   The apparatus of claim 1, wherein each data chunk includes one or more tiles of point cloud data.   The apparatus of claim 2, wherein each tile includes a subset of point cloud data in a rectangular column extending longitudinally along the earth's gravity vector.   4. The apparatus of claim 3, wherein each data chunk contains a number of adjacent tiles that are optimized to achieve a target data chunk size.   Annotation complete where the map generator compares asset information in the output map with asset information in one or more previous output maps corresponding to a common local environment to generate a notification when a mismatch is detected The apparatus of claim 1, further comprising a verifier. A compression mechanism that operates to compress the point cloud data before storing in the data storage component;
A deployment mechanism that operates to deploy the point cloud data prior to application of the analysis mechanism by the processing unit;
2. The method of claim 1, wherein the compression mechanism adjusts its compression ratio to maintain a balance between a data processing rate achievable by the processing unit and a data retrieval rate from the data storage component. apparatus. A GPS receiver attached to the vehicle, the GPS receiver providing a first geographical location of the vehicle;
A local map cache in the vehicle that stores, for each asset, a local map of the asset, including location, characteristics associated with the asset, and one or more relationships to other assets. When,
One or more local environment sensors mounted on the vehicle to enable collection of data associated with the local environment in the vicinity of the vehicle;
One or more vehicle computers that receive the first geographic location from the GPS receiver and from the local map cache, previously mapped assets in the vicinity of the first geographic location; A vehicle computer that retrieves the associated record;
A feature extraction component implemented in the vehicle computer, the feature extraction component receiving the local environment sensor data and identifying and locating an observed asset currently in the vicinity of the vehicle Elements and
A location refinement component implemented in the vehicle computer, comparing the observed asset identification and location from the feature extraction component with asset information retrieved from the local map cache; A vehicle position confirmation device comprising: a position refinement component that determines a current state of the vehicle.   The vehicle position confirmation device according to claim 7, wherein the current state of the vehicle includes a location of the vehicle.   The vehicle position confirmation device according to claim 8, wherein the current state of the vehicle further includes a speed and a traveling direction of the vehicle.   8. The vehicle location verification of claim 7, further comprising a wireless vehicle communication device, wherein the local map cache is capable of downloading local map data from a remote database during vehicle operation by the wireless vehicle communication device. apparatus.   The vehicle location verification apparatus according to claim 7, wherein the one or more local environment sensors include one or more of a LIDAR sensor, a digital camera, and a radar sensor.   The vehicle position confirmation device according to claim 7, wherein the local environment attribute includes one or more of a sign, a roadside safety structure, and a brace signal.   The vehicle position confirmation device according to claim 7, wherein the local environment sensor data associated with a local environment in the vicinity of the vehicle includes three-dimensional point cloud data.   The vehicle position confirmation device according to claim 7, wherein the vehicle is a train.   The vehicle position confirmation device according to claim 14, wherein the current state of the vehicle includes a track identification.   The vehicle location device of claim 7, further comprising an interface component, through which the current state of the vehicle can be communicated to one or more vehicle control systems.   11. A map audit component that identifies a difference between the local map of the asset and the observed asset and outputs the difference to the vehicle communication device for transmission to the remote database. The vehicle position confirmation apparatus as described in.   The map audit component is an asset that exists in the observed asset and does not exist in the asset's local map, or an asset that does not exist in the observed asset and exists in the asset's local map. 18. A vehicle location verification device according to claim 17, comprising a lost asset detector for identification.   An asset alteration detector, wherein the map audit component identifies an asset in the observed asset that has a tampering characteristic that is different from a characteristic indicative of damage or a characteristic associated with the asset in the local map of the asset; The vehicle position confirmation device according to claim 17 provided. The vehicle is adapted for traveling on a track, and the device is
An orbital gap evaluation component that receives information from the feature extraction component that indicates a location of a first asset, identifying the first asset as an obstacle, and passing the obstacle through the vehicle communication device The vehicle location verification apparatus according to claim 10, further comprising a track clearance evaluation component that reports the location to a back-end server. A method for auditing map data by one or more networked servers that maintain maps in a database comprising:
Receiving a request for map data from a vehicle, wherein the vehicle has a local environment sensor and a local map cache; and
Transmitting map data to the vehicle in response to the request, wherein the map data includes asset information and the asset information includes identification, characteristics, and location of one or more assets. And a process of
Receiving from the vehicle a report indicating one or more differences between the map data and information detected by the vehicle local environment sensor;
Updating the database based on the information in the report.   The method of claim 21, wherein the report includes an identification and location of assets detected by the vehicle local environmental sensor that are not present in the database.   The method of claim 21, wherein the report includes an identification of assets present in the database, but not detected by the vehicle local environment sensor.   The method of claim 21, wherein the asset information includes one or more asset characteristics, and the report includes a difference between an asset characteristic using the database and an asset characteristic detected by the vehicle local environment sensor. . The step of receiving a report from the vehicle indicating one or more differences between the map data and information detected by the vehicle local environment sensor;
The method of claim 21, comprising receiving a report indicating an obstacle clearance for the train path.