KR102680231B1 - Real-time data acquisition and historical data sharing system - Google Patents

Abstract

Real-time data collection and archival data sharing systems provide real-time or near-real-time access to a wide range of data, such as event and operational data, video data, and audio data, to remote users such as asset owners, operators, and investigators. Works with data collection and recording systems and viewers. Data sharing systems allow users to share data obtained from data collection and recording systems with users in remote locations. Users can share a modern web browser in a secure, controlled, tracked, and audited manner with any remote recipient end-user with access to the Internet. Instead of sharing files, users share URLs to data. Users can control, track, and audit sensitive data through URL-based data sharing. By sharing their data, users can improve the safety of transportation systems around the world without fear of unauthorized data dissemination. Investigators using the web client can share data with remote users without having to find and download “black boxes.”

Description

Real-time data acquisition and historical data sharing system

This application claims priority from U.S. Provisional Application No. 62/680,907, filed June 5, 2018, which in turn claims priority from U.S. Provisional Application No. 62/825,943, filed March 29, 2019, and is a continuation-in-part thereof. , which claims priority and is in part a continuation of U.S. Provisional Application No. 62/337,227, filed May 16, 2016, and is a continuation-in-part of U.S. Non-Provisional Application No. 16/595,650, filed May 15, 2017 Application, now U.S. Application No. 9,934,623, filed April 3, 2018, claims priority and is a continuation-in-part of U.S. Non-Provisional Application No. 15/907,486, filed February 28, 2018, May 16, 2016 Claims priority and is a part-continuing application to U.S. Provisional Application No. 62/337,225, filed May 15, 2017, and is a continuation-in-part application and claims priority to U.S. Non-Provisional Application No. 15/595,689, filed May 15, 2016 Claims priority and is a continuation-in-part application of U.S. Provisional Application No. 62/337,228, filed May 16, 2017, and is a continuation-in-part application, claims priority of U.S. Non-Provisional Application No. 15/595,712, filed May 15, 2017, and is a continuation-in-part application The scope and content permitted by this are incorporated herein by reference in their entirety.

The present invention relates to systems and methods for viewing videos, images, and data from real-time data acquisition and recording systems and sharing videos, images, and/or data with other individuals.

Expensive mobile assets such as locomotives, aircraft, mass transit systems, mining equipment, transportable medical equipment, cargo, ships, and warships commonly use onboard data collection and recording “black box” systems. These data collection and recording systems, such as event data recorders or flight data recorders, record a variety of system parameters used in accident investigations, crew performance evaluations, fuel efficiency analysis, maintenance planning, and predictive diagnostics. A typical data acquisition and recording system consists of pressure switches and pressure transducers that record data from various onboard sensor devices as well as digital and analog inputs. Recorded data may include parameters such as speed, distance traveled, location, fuel level, revolutions per minute (RPM), fluid level, driver controls, pressure, current and forecast weather conditions, and ambient conditions. In addition to basic event and operational data, video and audio event/data recording capabilities are also used in many of these mobile assets. Typically, when an incident involving an asset occurs and an investigation is required, data is extracted from the data recorder once it has been recovered. Certain situations may arise where the data recorder cannot be recovered or the data becomes unusable. In these situations, data such as event and operational data, video data, and audio data collected by the data collection and recording system are immediately required and available to the user, regardless of whether physical access to the data collection and recording system or the data is unavailable. may share the data or portions thereof with other authorized individuals.

The present invention generally relates to real-time data acquisition and recording systems for use in high-value mobile assets. The disclosure herein can provide real-time or near-real-time access to data such as event and operational data, video data, and audio data recorded by real-time data acquisition and recording systems. One implementation of a method for processing, storing and transmitting data on one or more assets includes receiving a request from a first user using a web server, wherein the request is directed to specific data stored in a remote data store and an email address of the second user. configured-; determining a Uniform Resource Locator (URL) configured to provide access to the specified data; generating an email containing a URL; and sending an email to the email address.

Another implementation of a system for processing, storing and transmitting data from at least one asset includes receiving a request from a first user, determining a Uniform Resource Locator (URL) providing access to the specific data, and including the URL. A web server configured to generate an email, configured to send the email to an email address, wherein the request includes specific data stored within the remote data store and an email address of a second user, and at least one data recorder onboard the mobile asset. a local memory component, a data encoder, an onboard data manager, and a queue store, wherein the data recorder is configured to record data from at least one of at least one data source onboard the mobile asset and at least one data source remote from the mobile asset. It is configured to receive data based on one data signal, and the data encoder is configured to encode the data into encoded data.

These and other aspects of the invention are disclosed in the following detailed description of embodiments, appended claims, and accompanying drawings.

The various features, advantages, and other uses of the device will become more apparent by reference to the following detailed description and drawings, where like reference numerals refer to like parts in the various drawings. It is emphasized that, according to common practice, the various functions of the drawing are not expanded. Conversely, the sizes of various features are arbitrarily expanded or reduced for clarity.
1 depicts a field implementation of a first embodiment of an exemplary real-time data acquisition and recording system, in accordance with implementations of the present invention.
2 illustrates a field implementation of a second embodiment of an exemplary real-time data acquisition and recording system, in accordance with implementations of the present invention.
3 is a flow diagram of a process for recording data and/or information from a mobile asset, in accordance with an implementation of the present invention.
4 is a flow diagram of a process for adding data and/or information from a mobile asset after an outage, in accordance with an implementation of the present invention.
5 is a diagram illustrating exemplary intermediate record blocks and full record blocks stored in a conflict memory module, according to an implementation of the present invention.
6 is a diagram illustrating an exemplary intermediate record block of a crash memory module before a power outage and after power restoration, in accordance with an implementation of the present invention.
7 is a diagram illustrating an example record segment of a crash memory module after power has been restored, according to an implementation of the present invention.
8 shows a field implementation of a first embodiment of a real-time data acquisition and recording system viewer, in accordance with implementations of the present invention.
9 is a flow diagram of a process for recording video data, audio data, and/or information from a mobile asset, in accordance with an implementation of the present invention.
10 is a flow diagram of a process for recording video data, audio data, and/or information from a mobile asset, in accordance with an implementation of the present invention.
11 is a flow diagram illustrating an example fisheye view of a 360-degree camera of a real-time data acquisition and recording system viewer, in accordance with an implementation of the present invention.
12 is a diagram illustrating an exemplary panoramic view of a 360-degree camera of a real-time data acquisition and recording system viewer, according to an implementation of the present invention.
13 is a diagram illustrating an example quad view of a 360-degree camera of a real-time data acquisition and recording system viewer, according to an implementation of the present invention.
14 is a diagram illustrating an example distorted view of a 360-degree camera of a real-time data acquisition and recording system viewer, according to an implementation of the present invention.
15 shows a field implementation of a first embodiment of a data collection and recording system video content analysis system in accordance with implementations of the present invention.
16A is a diagram illustrating exemplary track sensing, in accordance with an implementation of the present invention.
FIG. 16B is a diagram illustrating exemplary track sensing and switch sensing, according to an implementation of the present invention.
FIG. 16C is a diagram illustrating exemplary track detection, track number count, and signal detection, in accordance with an implementation of the present invention.
16D is a diagram illustrating exemplary intersection and track detection, according to an implementation of the present invention.
FIG. 16E is a diagram illustrating exemplary dual overhead signal detection, in accordance with an implementation of the present invention.
16F is a diagram illustrating exemplary multi-track sensing, according to an implementation of the present invention.
Figure 16G is a diagram illustrating example switch and track sensing, in accordance with an implementation of the present invention.
Figure 16H is a diagram illustrating exemplary switch sensing, in accordance with an implementation of the present invention.
17 is a flow diagram of a process for determining the internal state of a mobile asset, in accordance with an implementation of the present invention.
18 is a flow diagram of a process for determining object detection and obstacle detection occurring external to a mobile asset, in accordance with an implementation of the present invention.
19 depicts a field implementation of a first embodiment of an exemplary real-time data acquisition and recording system, in accordance with implementations of the present invention.
20 illustrates a field implementation of a second embodiment of an exemplary real-time data acquisition and recording system, in accordance with implementations of the present invention.
21 is a flow diagram of a process for sharing data and/or information from assets, in accordance with implementations of the present invention.

A first embodiment of a real-time data acquisition and recording system described herein provides event and operational data, video data, and audio data associated with high-value assets to remote users, such as asset owners, operators, and investigators. The same provides real-time or near-real-time access to a wide range of data. Data collection and recording systems record data related to assets through data recorders and stream the data to remote data repositories and users at remote locations before, during, and after an incident. Data can be streamed to remote data repositories in real time or near real time so that the information is available at least until an incident or emergency occurs. Therefore, there is little need to locate and download “black boxes” to investigate incidents involving an asset, request downloads of specific data, and use custom applications to locate and transfer files and view data. There is no need to interact with The system of the present invention adds the ability to stream data to remote data storage and remote end users before, during, and after an incident while maintaining typical recording capabilities. In most situations, the information recorded in the data recorder is redundant and not necessary because the data has already been collected and stored in a remote data repository.

Prior to the system of the present invention, data was extracted from a "black box" or "event recorder" after an incident occurred and an investigation was required. Data files containing time segments recorded by the "black box" had to be downloaded from the "black box", retrieved, and then viewed by users using proprietary software. Users had to gain physical or remote access to the asset, select the desired data to download from a "black box," download a file containing the desired information to their computing device, and use a custom application running on the computing device to retrieve the desired data. I had to find the appropriate file with the data. The system of the present invention eliminates the need for the user to perform these steps; the user simply uses a common web browser to navigate to the desired data. Remotely located users can access a common web browser to explore desired data related to selected assets to view and analyze the asset's operational efficiency and safety in real-time or near real-time.

Remotely located users, such as asset owners, operators, and/or investigators, can access live and/or historical information related to selected assets by accessing a common web browser to view and analyze the operational efficiency and safety of assets in real-time or near real-time. You can explore data. The ability to view operations in real time or near real time allows actions to be quickly evaluated and adjusted. For example, during an incident, real-time information and/or data can facilitate triage of the situation and provide critical information to first responders. For example, during normal operations, near-real-time information and/or data can be used to audit crew performance and support network-wide situational awareness.

Data includes speed, pressure, temperature, current, voltage, and acceleration occurring on the asset and/or surrounding assets, Boolean data such as switch position, actuator position, warning light illumination, and actuator commands, position, speed, and altitude. Global Positioning System (GPS) data and/or Geographic Information System (GIS) data, such as internally generated information such as regulatory speed limits for the current location of the asset, video from cameras located at various locations within, on, or around the asset, and Image information, audio information from microphones located at various locations within, on, or around the asset, information about the asset's operational plan (e.g. route, schedule) transmitted from the data center to the asset, and manifest information, including information about the asset's current operation. Information about environmental conditions, including current and forecasted weather conditions in areas where operations are or are planned to be operated, asset control status and operational data generated by systems such as the locomotive's active train control (PTC), and additional data, video, and audio analytics. It may include, but is not limited to, analog and frequency parameters, such as data derived from a combination of the above, including but not limited to, analysis.

1 and 2 illustrate field implementations, respectively, of a first and second embodiment of an exemplary real-time data acquisition and recording system (DARS) 100, 200, in which aspects of the present invention may be implemented. DARS (100, 200) is a system that provides real-time information from a data recording device to a remote end user. DARS (100, 200) is installed in a vehicle or mobile asset (148, 248) and can be configured with any combination of onboard wired and/or wireless data links (170, 270), such as a wireless gateway/router or wireless data link (146). and data recorders 154, 254 that communicate with any number of different information sources, such as through data links such as off-board information sources through the data centers 150, 250 of DARS 100, 200. Data recorders (154, 254) include onboard data managers (120, 220), data encoders (122, 222), vehicle event detectors (156, 256), queue stores (158, 258), and wireless gateway/router (172, 272). Additionally, in such implementations, data recorders 154, 254 may include crash hardened memory modules 118, 218 and/or Ethernet switches 162, 262 with or without Power over Ethernet (PoE). . Exemplary enhanced memory modules 118, 218 may be, for example, crash capable event recorder memory modules that comply with federal regulations and Federal Railroad Administration regulations, and crash capable event recorder memory modules that comply with federal regulations and Federal Aviation Administration regulations. It may be a survivable memory unit, a crash hardened memory module that complies with applicable federal regulatory codes, or any other suitable hardened memory device known in the art. In a second embodiment, as shown in Figure 2, data recorder 254 may further include an optional non-collision enhanced removable storage device 219.

Wired and/or wireless data links 170, 270 may include any one or combination of discrete signal inputs, standard or proprietary Ethernet, serial connections, and wireless connections. Ethernet-connected devices can use the Ethernet switches 162 and 262 of the data recorders 154 and 254 and can use POE. Ethernet switches 162, 262 may be internal or external and may support POE. Additionally, in the implementation of Figures 1 and 2, data from remote data sources such as map components 164, 264, route/crew roster components 124, 224, and weather components 126, 226 may be wireless data. It is available from the data center 150, 250 via links 146, 246 and wireless gateways/routers 172, 272 to onboard data managers 120, 220 and vehicle event detectors 156, 256.

Data recorders 154, 254 collect data or information through onboard data links 170, 270 from a variety of sources, which can vary greatly depending on the configuration of the asset. Data encoders 122 and 222 encode minimal data sets typically defined by regulatory agencies. In this implementation, data encoders 122, 222 receive data from various asset 148, 248 sources and data centers 150, 250. Information sources include analog inputs (102, 202), digital inputs (104, 204), I/O modules (106, 206), vehicle controllers (108, 208), engine controllers (110, 210), and inertial sensors (112). , 212), global positioning system (GPS) (114, 214), cameras (116, 216), active train control (PTC)/signal data (166, 266), fuel data (168, 268), cellular transmission detector ( (not shown), internal drive data, and additional data signals, and any number of components of the data center 150, 250, such as any of the route/crew roster components 124, 224, weather components 126, 226), map components 164, 264, and any additional data signals. Data encoders 122 and 222 compress or encode data and time synchronize the data to facilitate efficient real-time transmission and replication to remote data storage 130 and 230. Data encoders (122, 222) transmit encoded data to the onboard data manager (120, 220), which transmits the encoded data to the data center (150) for replication to remote data storage (130, 230). The encrypted data is stored in the conflict memory modules 118, 218 and the queue storage 158, 258 via the remote data manager 132, 232 located at 250. Optionally, the onboard data manager 120, 220 may store a third copy of the encoded data in the non-conflict hardened removable storage device 219 of the second embodiment shown in FIG. 2. Onboard data managers 120, 220 and remote data managers 132, 232 work together to manage the data replication process. A single remote data manager 132, 232 within a data center 150, 250 may manage replication of data from multiple assets 148, 248.

Data from various input components and from the in-cab audio/graphical user interface (GUI) 160, 260 is transmitted to vehicle event detectors 156, 256. Vehicle event detectors 156 and 256 process data to determine whether an event, incident or other predefined situation related to an asset 148 or 248 has occurred. When the vehicle event detector 156, 256 detects a signal indicating that a predefined event has occurred, the vehicle event detector 156, 256 detects the processed data in which the predefined event occurred along with supporting data surrounding the predefined event. is transmitted to the onboard data manager (120, 220). Vehicle event detectors 156, 256 may include analog inputs 102, 202, digital inputs 104, 204, I/O modules 106, 206, and vehicle controllers 108, 208, which may vary depending on asset configuration. ), engine controller (110, 210), inertial sensors (112, 212), GPS (114, 214), camera (116, 216), route/crew roster component (124, 224), weather component (126, 226), map components 164, 264, PTC/signal data 166, 266, and fuel data 168, 268. When the vehicle event detector 156, 256 detects an event, the detected asset event information is stored in the queue repository 158, 258 and optionally sent to the asset via the in-cab audio/graphical user interface (GUI) 160, 260. Can be provided to crew members at (148, 248).

Onboard data managers 120, 220 also transmit data to queue storage 158. In a near real-time mode, the onboard data manager 120, 220 processes encoded data received from data encoders 122, 222 and random event information into collision hardened memory modules 118, 218 and queue stores 158, 258. Save it to In the second embodiment of FIG. 2 , onboard data manager 220 may selectively store encoded data in non-conflict hardened removable storage device 219 . After five minutes' worth of encoded data has been accumulated in the queue store (158, 258), the onboard data manager (120, 220) provides a wireless data link (146, 246) accessed via the wireless gateway/router (172, 272). ), 5 minutes of encoded data is stored in the remote data storage (130, 230) through the remote data manager (132, 232) in the data center (150, 250). In real-time mode, onboard data managers 120, 220 convert encoded data received from data encoders 122, 222 and any event information into a wireless data link accessed via wireless gateway/router 172, 272. 146, 246, through remote data managers 132, 232 in data centers 150, 250, to collision hardening memory modules 118, 218, and optionally, removing the non-collision hardening of FIG. 2. It is stored in a possible storage device 219 and remote data storage 130, 230. Onboard data managers 120, 220 and remote data managers 132, 232 can communicate via various wireless communication links such as Wi-Fi, cellular, satellite, and private wireless systems using wireless gateways/routers 172, 272. Can communicate. Wireless data links 146, 246 may be, for example, a wireless local area network (WLAN), a wireless metropolitan area network (WMAN), a wireless wide area network (WWAN), a private wireless system, a cellular network, or in this embodiment, DARS. There may be other means of transmitting data from the data recorders 154, 254 at 100, 200 to the remote data managers 130, 230 at DARS 100, 200. If a wireless data connection is not available, the data is stored in memory and waits in queue storage 158, 258 until the wireless connection is restored and the data replication process can resume.

In parallel with data recording, data recorders 154, 254 continuously and autonomously replicate data to remote data storage 130, 230. There are two modes for the replication process: real-time mode and near-real-time mode. In real-time mode, data is replicated to remote data storage 130, 230 every second. In near real-time mode, data is replicated to remote data storage 130, 230 every 5 minutes. The rate used for near real-time mode is configurable and the rate used for real-time mode can be adjusted to support high-resolution data by replicating data to remote data storage 130, 230 every 0.10 seconds. When DARS 100, 200 is in a near real-time mode, onboard data managers 120, 220 queue data in queue stores 158, 258 before replicating the data to remote data managers 132, 232. Onboard data managers 120, 220 also replicate vehicle event detector information waiting in queue storage 158, 258 to remote data managers 132, 232. Near real-time mode is used during normal operation under most conditions to increase the efficiency of the data replication process.

The real-time mode may be initiated based on events generated and detected by a vehicle event detector 156, 256 onboard the asset 148, 248 or may be initiated by a request originating from the data center 150, 250. When a remotely located user (152, 252) requests real-time information from a web client (142, 242), a typical data center (150, 250) startup request for real-time mode is initiated. Common reasons why real-time mode is initiated onboard an asset 148, 248 include: by a vehicle event detector 156, 256, an operator initiating an emergency stop request, emergency braking activity, rapid acceleration or deceleration in any axis, or a data recorder. 154, 254 is for detecting an event or accident, such as loss of input power. When switching from near-real-time mode to real-time mode, all data that has not yet been replicated to the remote data storage 130, 230 is replicated and stored in the remote data storage 130, 230 and then live replication begins. Switching between near-real-time mode and real-time mode typically occurs within 5 seconds. After a predetermined amount of time has passed following an incident or accident, a predetermined period of inactivity, or when the user 152, 252 no longer desires real-time information from the asset 148, 248, the data recorder 154, 254 can record near real-time information. Return to mode. The predetermined time required to initiate the conversion is configurable and is typically set to 10 minutes.

When the data recorders 154, 254 are in real-time mode, the onboard data managers 120, 220 store data in the crash hardened memory modules 118, 218 and the non-collision hardened removable storage devices of Figure 2. It continuously attempts to empty its queue to the remote data manager 132, 232 while selectively storing it at 219, while simultaneously transmitting data to the remote data manager 132, 232. Onboard data managers 120, 220 also transmit information about detected vehicles waiting in queue storage 158, 258 to remote data managers 132, 232.

Upon receiving data to be replicated from data recorders 154, 254, along with data from map components 164, 264, route/crew roster components 124, 224 and weather components 126, 226 ), the remote data managers 132 and 232 store the compressed data in the remote data storage 130 and 230 in the data center 150 and 250 of the DARS 100 and 200. Remote data store 130, 230 may be, for example, a cloud-based data store or any other suitable remote data store. When data is received, a process that causes a data decoder (136, 236) to decode recently replicated data to/from a remote data store (130, 230) and transmit the decoded data to a remote event detector (134, 234). begins. The remote data managers 132 and 232 store vehicle event information in the remote data storage 130 and 230. Once the remote event detector 134, 234 receives the decoded data, it processes the decoded data to determine whether an event of interest is found in the decoded data. The decoded information is used by the remote event detector 134, 234 to detect events, incidents or other predefined circumstances in data occurring with the asset 148, 248. Upon detecting an event of interest in the decoded data, the remote event detector 134, 234 stores the event information and supporting data in the remote data store 130, 230. When the remote data manager 132, 232 receives remote event detector 134, 234 information, the remote data manager 132, 232 stores the information in the remote data storage 130, 230.

A remotely located user 152, 252 may use a standard web client 142, 242, such as a web browser or a virtual reality device (not shown), which can display thumbnail images of selected cameras in implementations of the present invention. Thus, information including vehicle event detector information related to a specific asset 148, 248 or multiple assets may be accessed. Web clients 142, 242 forward user 152, 252 requests for information to web servers 140, 240 over networks 144, 244 using common web standards, protocols, and technologies. Networks 144, 244 may be the Internet, for example. Networks 144, 244 may be a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a virtual private network (VPN), a cellular network, or a web server 140, 240, of the present invention. In embodiments, there may be other means of transmitting data to web clients 142, 242. The web servers 140 and 240 request desired data from the data decoders 136 and 236. Data decoders 136, 236 obtain requested data associated with a particular asset 148, 248 or multiple assets from a remote data repository 130, 230 in response to a request from a web server 140, 240. Data decoders (136, 236) decode the requested data and transmit the decoded data to localizers (138, 238). Localization is the process of converting data into the format desired by the end user, including converting the data into the user's preferred language and unit of measurement. The localizer 138, 238 accesses the web client 142, 242 to identify the profile settings set by the user 152, 252, and ensures that the raw encoded data and sensed event information are in coordinated universal time. and information transmitted to a web client 142, 242 for presentation to the user 152, 252 using profile settings, as stored in a remote data repository 130, 230 using the International System of Units (SI units). prepare. Localizers 138 and 238 convert the decoded data into a format desired by the user 152 and 252, such as the user's 152 and 252 preferred language and unit of measurement. Localizers 138 and 238 transmit localized data in the user's preferred format to web servers 140 and 240 upon request. The web server 140, 240 then provides playback and real-time display of standard video and 360-degree video, and transmits localized data of the asset or multiple assets to the web client 142, 242 for viewing and analysis. do. Web clients 142, 242 may display and users 152, 252 may view data, video, and audio for a single asset or may view data, video, and audio for multiple assets simultaneously. Web clients 142, 242 may also store data, along with a plurality of video and audio data from both standard and 360-degree video sources, at locations located on, within, around the asset, and/or in remote locations. can provide synchronous playback and real-time display.

3 is a flow diagram illustrating a process 300 for recording data and/or information from assets 148, 248 in accordance with implementations of the present invention. Data recorders 154, 254 may record speed, latitude coordinates, longitude coordinates, horn detection, throttle position, weather data, map data, or route and/or crew data of assets 148, 248 and data centers 150, 250. Data signals are received from various input elements, including physical or computed data elements, such as data 302. Data encoders 122 and 222 generate records containing a structured series of bits that are used to construct and record data signal information 304. The encoded records are then sent to the onboard data manager 120, 220, which sequentially combines the series of records chronologically into record blocks containing up to 5 minutes of data 306. An intermediate record block contains less than 5 minutes of data, while a full record block contains a full 5 minutes of data. Each record block contains all the data needed to fully decode the signal it contains, including data integrity checks. At a minimum, a record block must begin with a start record and end with an end record.

Data recorder 154 to ensure that all encoded signal data is stored in the collision hardened memory module 118 and, optionally, in the non-collision hardened removable storage device 219 of FIG. , 254) loses power or is subjected to extreme temperature or mechanical stress due to a crash or other catastrophic event, the onboard data manager 120, 220 records intermediate records in the crash hardened memory module 118 at a predetermined rate 308. Store the blocks, optionally in the non-collision hardened removable storage device 219 of FIG. 2, where the predetermined rate is configurable and/or variable, as shown in exemplary representation in FIG. 5. Intermediate record blocks are stored at least once per second, but can be as frequent as once every tenth of a second. The rate at which intermediate record blocks are stored depends on the sampling rate of each signal. Every intermediate record block contains the entire record set since the last full record block. The data recorders 154, 254 may be configured to remove the non-collision hardening of the crash hardening memory modules 118, 218, or optionally the data recorder 254 of FIG. 2, to prevent damage or the data recorder 154, 254 When writing each intermediate record block, to prevent loss of more than a second of data when power is lost, while storing data in possible storage device 219, and optionally in crash hardening memory modules 118, 218. may alternately use two temporary storage locations in the non-collision hardened removable storage device 219 of FIG. 2 . Each time a new temporary record block is stored in a temporary conflict hardened memory location, any existing temporary record blocks previously stored in that location are overwritten.

In an implementation of the present invention, every five minutes, when the data recorders (154, 254) are in near-real-time mode, the onboard data managers (120, 220) crash-harden the entire record block containing the last five minutes of encoded signal data. A copy of an entire block of records to be stored in a record segment of a memory module 118, 218 and to be stored in a remote data store 130, 230 for a predetermined retention period, such as two years 310, as shown in Figure 7. is transmitted to the remote data manager (132, 232). The collision hardened memory modules 118, 218 of Figure 2, and/or the non-collision hardened removable storage device 219 of the data recorder 254 stores the record segments of the most recent record blocks for the required storage period, The storage period is the federally mandated period during which in implementations of the invention the data recorder 154, 254 must operate with an additional 24-hour buffer or the video data must be stored in the crash hardened memory module 118, 218 and overwritten.

4 is a flow diagram illustrating a process 400 for adding data and/or information from an asset 148, 248 after an outage, in accordance with an implementation of the present invention. When power is restored, data recorders 154, 254 identify the last intermediate record block stored in one of two temporary conflict hardened memory locations 402, and a 32-bit circular number contained in the final record of every record block 404. Redundancy checking is used to verify the last intermediate record block. The verified intermediate record blocks are then added to the crash hardened memory record segments, which may contain up to 5 minutes of data prior to the outage, to the remote data manager 132, 232 to be stored for a retention period 406. is transmitted. The encoded signal data is stored in the collision hardened memory modules 118, 218, and/or optionally in non-collision hardened removable storage of the data recorder 254 of FIG. 2, within a circular buffer of the required storage period. It is stored in device 219. Because a conflict hardened memory record segment is split into multiple record blocks, the data recorder 154, 254 removes old record blocks as needed to free memory space, each time an entire record block is transferred to the crash hardened memory module 118, 218. , and/or optionally stored in non-collision hardened removable storage device 219 of data recorder 254 of FIG. 2 .

FIG. 6 is a diagram illustrating an example intermediate write block prior to power loss and after power restoration to data recorders 154, 254. (2/1/2016 10:10:08 AM) If the intermediate record block stored in temporary location 2 at 602 is valid, then the temporary record block is stored in record segment 702 of conflict hardened memory module 118, 218. (FIG. 7), and/or optionally attached to the non-collision hardened removable storage device 219 of the data recorder 254 of FIG. 2, as shown in FIG. (2/1/2016 10:10:08 AM) If the intermediate record block stored in temporary location (2) is invalid, the temporary record block in temporary location (1) (2/1/2016 10:10:07 AM) is invalid. ) is validated and, if valid, added to the record segment of the crash hardened memory modules 118, 218, and/or optionally the non-collision hardened removable storage device 219 of the data recorder 254 of FIG. ) is added to

Whenever any block of records is to be stored in a crash hardened memory module 118, 218, and/or optionally in a non-collision hardened removable storage device 219 of data recorder 254 of FIG. Each time, the record segment is immediately flushed to disk. Because the data recorders 154, 254 alternate between two different temporary storage locations when storing intermediate record blocks, temporary storage locations that are not modified or flushed to conflict collision hardened memory or non-collision hardened removable storage devices. There is always one. Accordingly, at least one of the two intermediate record blocks stored in the temporary storage location is valid, and the data recorders 154, 254 do not lose more than one second of the majority of the data whenever the data recorders 154, 254 lose power. is guaranteed. Similarly, when data recorders 154, 254 write data to crash hardened memory modules 118, 218, and/or optionally to non-collision hardened removable storage devices of data recorder 254 of FIG. When recording data to 219, every tenth of a second, the data recorder 154, 254 will lose no more than a tenth of a second of data each time the data recorder 154, 254 loses power.

For simplicity of explanation, process 300 and process 400 are shown and described as a series of steps. However, the steps according to the invention may occur in various orders and/or simultaneously. Additionally, steps according to the invention may occur in other steps not presented and described herein. Additionally, not all steps shown are necessary to implement methods according to the disclosed subject matter.

Third embodiments of the real-time data collection and recording system and viewer described herein provide access to remote users, such as asset owners, operators, and investigators, to collect valuable assets, such as event and operational data, video data, and audio data. Provides real-time or near real-time access to a wide range of data. Data collection and recording systems record data related to assets through data recorders and stream the data to remote data repositories and users at remote locations before, during, and after an incident. Data can be streamed to remote data repositories in real time or near real time so that the information is available at least until an incident or emergency occurs. Therefore, there is little need to locate and download “black boxes” to investigate incidents involving an asset, request downloads of specific data, find and transfer files, and use custom applications to view data. There is no need to interact with the data recorder. The system of the present invention maintains general recording performance and adds the ability to stream data to remote data storage and remote end users before, during, and after an incident. In most situations, the information recorded in the data recorder is redundant and is not necessary because the data has already been acquired and stored in a remote data repository.

Prior to the system of the present invention, data was extracted from a "black box" or "event recorder" after an incident occurred and an investigation was required. Data files containing time segments recorded by the "black box" had to be downloaded from the "black box", retrieved, and then viewed by users using proprietary software. Users gain physical or remote access to an asset, select the data they want to download from a "black box," download a file containing the information they want to their computing device, and use a custom application that runs on the computing device to make the desired data available for download. I had to find the appropriate file with the data. The system of the present invention eliminates the need for the user to perform these steps, and the user only needs to use a common web browser to navigate to the desired data. Remotely located users can access a common web browser to explore desired data related to selected assets to view and analyze the asset's operational efficiency and safety in real-time or near real-time.

Remotely located users, such as asset owners, operators, and/or investigators, can access a common web browser to explore live and/or desired historical data related to selected assets to view and analyze the asset's operational efficiency and safety in real-time or near real-time. You can. The ability to view operations in real time or near real time allows you to quickly evaluate and adjust behavior. For example, during an incident, real-time information and/or data can facilitate triage of the situation and provide critical information to first responders. For example, during normal operations, near-real-time information and/or data can be used to audit crew performance and support network-wide situational awareness.

The real-time data collection and recording system of the third embodiment includes, as part of the data collection and recording system, at least one or more of an image measurement device, a video measurement device, and a distance measurement device within, on, or around a mobile asset. Use combinations. Image measurement devices and/or video measurement devices include, but are not limited to, 360-degree cameras, fixed cameras, narrow view cameras, wide view cameras, 360-degree fisheye view cameras, and/or other cameras. does not Ranging devices include, but are not limited to, radar and light detection and ranging (“LIDAR”). LIDAR is a surveying method that measures the distance to the target by illuminating the target with pulsed laser light and measuring the reflected pulse with a sensor. Prior to the system of the present invention, “black boxes” and/or “event recorders” did not include 360-degree cameras or other cameras within, on, or near mobile assets. Systems of the present invention include video capture using 360-degree cameras, fixed cameras, narrow-view cameras, wide-view cameras, 360-degree fisheye view cameras, radar, LIDAR, and/or other cameras that are part of a data collection and recording system. Adds the ability to use and record a 360-degree view, narrow view, wide view, fisheye view, and/or other views within, on, or near a mobile asset before, during, and after an incident involving the mobile asset occurs. Provides remote data storage and remote users and researchers. The ability to view operations, 360-degree video, and/or other video in real time or near real time allows crew behavior to be quickly assessed and adjusted. Owners, operators and investigators can view and analyze operational efficiency, safety of people, vehicles and infrastructure, and investigate or inspect accidents. The ability to view 360-degree video and/or other video from mobile assets allows crew behavior to be quickly assessed and adjusted. For example, 360-degree video and/or other video during an incident can facilitate triage of the situation and provide important information to first responders and investigators. For example, 360-degree video and/or other video could be used during normal operations to audit crew performance and support network-wide situational awareness. 360-degree cameras, fixed cameras, narrow-view cameras, wide-view cameras, 360-degree fisheye view cameras, radar, LIDAR and/or other cameras are used to provide surveillance video for law enforcement and/or rail police, inspect critical infrastructure, It provides a complete picture of the situation for monitoring railroad crossings, observing track work progress, auditing crews both inside the cab and in the yard, and for real-time remote monitoring.

Previous systems required users to download video files containing time segments to view them using proprietary software applications or other external video playback applications. The data acquisition and recording system of the present invention provides 360-degree video, other video, image information and audio information, and ranging information, which can be displayed to a remote user using a virtual reality device or through a standard web client. Therefore, there is no need to download and use external applications to watch videos. Additionally, remote users can view 360-degree video and/or other videos in a variety of modes using a virtual reality device or through a standard web client, such as a web browser. Therefore, there is no need to download and use external applications to watch videos. Previous video systems required users to download video files containing time segments of data that could only be viewed through proprietary application software or other external video playback applications that the user had to purchase separately.

Data may include, but is not limited to, video and image information from cameras located at various locations within, on, or around the asset and audio information from microphones located at various locations within, on, or around the asset. A 360-degree camera is a camera that provides a 360-degree spherical view, a 360-degree hemispherical view, and/or a 360-degree fisheye view. The use of 360-degree cameras, fixed cameras, narrow-view cameras, wide-view cameras, 360-degree fisheye view cameras, and/or other cameras on, on, or near a property are part of DARS and constitute 360-degree cameras, fixed cameras, narrow-view cameras. , provides the ability to use and record video using a wide view camera, a 360-degree fisheye view camera, and/or other cameras. Accordingly, a 360-degree view and/or other views within or around the asset are provided that can be used by remote data repositories, remotely located users, and investigators before, during, and after an incident.

8 shows a field implementation of a third embodiment of an exemplary real-time data acquisition and recording system (DARS) 800 in which aspects of the present invention may be implemented. DARS 800 is a system that delivers real-time information, video information, and audio information from a data recorder 808 on a mobile asset 830 through a data center 832 to remotely located end users. Data recorder 808 is installed in vehicle or mobile asset 830 and communicates with any number of different information sources through any combination of wired and/or wireless data links, such as a wireless gateway/router (not shown). . Data recorder 808 includes a crash hardening memory module 810, an onboard data manager 812, and a data encoder 814. In a fourth embodiment, data recorder 808 may also include a non-collision hardened removable storage device (not shown). An example hardened memory module 810 may be, for example, a crash event recorder memory module compliant with Federal and Federal Railroad Administration regulations, a crash survivable memory device compliant with Federal and Federal Aviation Administration regulations, It may be a crash hardened memory module that complies with applicable federal regulatory codes, or it may be any other suitable hardened memory device known in the art. Wired and/or wireless data links may include one or a combination of discrete signal inputs, standard or proprietary Ethernet, serial connections, and wireless connections.

Data recorder 808 collects video data, audio data, and other data and/or information from a variety of sources, which may vary depending on the configuration of the asset, via onboard data links. In this embodiment, data recorder 808 records video data from 360-degree cameras, fixed cameras, narrow-view cameras, wide-view cameras, 360-degree fisheye view cameras, radar, LIDAR, and/or other cameras 802. and a video management system 804 that continuously records audio data, wherein a fixed camera 806 is positioned within, on, or near the property 830, and the video management system 804 is configured to record video and audio data. Data may be stored in the crash hardened memory module 810 and video and audio data may also be stored in the non-collision hardened removable storage device of the second embodiment. Different versions of the video data are generated using different bit rates or spatial resolutions, and these versions are separated into variable length segments such as thumbnails, 5-minute low-resolution segments, and 5-minute high-resolution segments.

Data encoder 814 encodes a minimal data set, typically as defined by regulatory agencies. Data encoder 814 receives video and audio data from video management system 804, compresses or encodes the data, and time-synchronizes the data to facilitate efficient real-time transmission and replication to remote data storage 820. Data encoder 814 transmits the encoded data to onboard data manager 812, which then transmits the encoded data in response to an on-demand request from a remote user 834 or to a specific data observed on asset 830. In response to operating conditions, the encoded video and audio data is transmitted to a remote data store 820 via a remote data manager 818 located in the data center 830. The onboard data manager 812 and remote data manager 818 work together to manage the data replication process. A remote data manager 818 in data center 832 may manage replication of data from multiple assets. Video and audio data stored in remote data storage 820 is available to web server 822 for access by remote location users 834.

Onboard data manager 812 also transfers data to queue storage (not shown). The onboard data manager 812 monitors video and audio data stored in the collision hardened memory module 810 by the video management system 804 or, in a fourth embodiment, optional non-collision hardened removable storage. Monitors the device and determines whether it is in near real-time or real-time mode. In a near real-time mode, the onboard data manager 812 stores encoded data, including video data, audio data, and other data or information, received from the data encoder 814, and any event information, in a collision hardened memory module ( 810) and/or, in a fourth embodiment, optionally to a non-collision hardened removable storage device and/or to a queue storage. After five minutes' worth of encoded data has been accumulated in the queue store, the onboard data manager 812 sends the five minutes' worth of data via a wireless data link 816 to a remote data manager 818 within the data center 832. The encoded data is stored in the remote data storage 820. In real-time mode, onboard data manager 812 processes encoded data, including video data, audio data, and other data or information, received from data encoder 814 via wireless data link 816, and any events. The information is stored in remote data storage 820 via remote data manager 818 in data center 832, at configurable predetermined time intervals, such as every second or every 0.10 seconds. Onboard data manager 812 and remote data manager 818 may communicate via various wireless communication links. Wireless data link 816 may be, for example, from a wireless local area network (WLAN), a wireless metropolitan area network (WMAN), a wireless wide area network (WWAN), a private wireless system, a cellular telephone network, or a data recorder 808. , in this embodiment, may be another means of transmitting data to the remote data manager 818. The process of transmitting and retrieving video data and audio data remotely from asset 830 requires a wireless data connection between asset 830 and data center 832. If a wireless data connection is not available, the data is stored and stored in a crash hardened memory module 810 and/or, in a fourth embodiment, an optional non-collision hardened removable storage device until the wireless connection is restored. stored and waiting. As soon as the wireless connection is restored, the process of retrieving video, audio and other additional data resumes.

In parallel with data recording, data recorder 808 continuously and autonomously replicates data to remote data storage 820. The replication process has two modes: real-time mode and near-real-time mode. In real-time mode, data is replicated to remote data storage 820 every second. In near real-time mode, data is replicated to remote data storage 820 every five minutes. The rate used in near real-time mode is configurable and the rate used in real-time mode can be adjusted to support high-resolution data by replicating data to remote data storage 820 every 0.10 seconds. Near real-time mode is used during normal operation under most conditions to increase the efficiency of the data replication process.

Real-time mode can be initiated based on events occurring at asset 830 or by a request originating from data center 832. Once a remotely located user 834 has requested real-time information from web client 826, a typical data center 832 startup request for real-time mode is initiated. Common reasons why real-time mode is initiated on asset 830 are events, such as an operator initiating an emergency stop request, emergency braking activity, rapid acceleration or deceleration in any axis, or loss of input power to data recorder 808. Or it is the detection of an accident. When switching from near real-time mode to real-time mode, all data that has not yet been replicated to the remote data store 820 is replicated and stored in the remote data store 820 and then live replication begins. Switching between near-real-time mode and real-time mode typically occurs within 5 seconds. After a predetermined period of time following an incident or incident, a predetermined period of inactivity, or when the user 834 no longer desires real-time information from the asset 830, the data recorder 808 reverts to a near-real-time mode. The predetermined time required to initiate the conversion is configurable and is typically set to 10 minutes.

When data recorder 808 is in real-time mode, onboard data manager 812 stores data in crash hardened memory module 810, while optionally storing data in a non-collision hardened removable storage device in a second embodiment. While storing, it continuously attempts to empty its queue to the remote data manager 818, while simultaneously transmitting data to the remote data manager 818.

Upon receiving video data, audio data, and other data or information to be replicated from data recorder 808, remote data manager 818 stores the data in remote data storage 820 in data center 830. Remote data store 820 may be, for example, a cloud-based data store or any other suitable remote data store. Once data is received, a process begins that causes a data decoder (not shown) to decode recently replicated data from remote data storage 820 and transmit the decoded data to a remote event detector (not shown). Remote data manager 818 stores vehicle event information in remote data storage 820. Once the remote event detector receives the decoded data, it processes the decoded data to determine whether an event of interest is found in the decoded data. The decoded information is used by a remote event detector to detect events, incidents, or other predefined situations in data occurring on asset 830. Upon detecting an event of interest in decoded data previously stored in remote data store 820, the remote event detector stores event information and supporting data in remote data store 820.

Video data, audio data, and any other data or information may be made available to user 834 in response to on-demand requests by user 834 and/or onboard asset 830 to observe specific operating conditions. is transmitted to the remote data storage 820 by the onboard data manager 812 in response. Video data, audio data, and any other data stored in remote data storage 820 are available on web server 822 for access by user 834. A remotely located user 834 may use a standard web client 826, such as a web browser or virtual reality device 828, to retrieve video data, audio data, and certain assets 830 stored in a remote data store 820. Alternatively, web client 826 may display thumbnail images of selected cameras in this implementation. Web client 826 forwards the user's request 834 for video, audio and/or other information to web server 822 over network 824 using common web standard protocols and technologies. Network 824 may be, for example, the Internet. Network 824 may also be connected to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a virtual private network (VPN), a cellular network, or a web server 822, in this embodiment, a web server. There may be other means of transmitting data to the client 826. Web server 822 requests desired data from remote data storage 820. Web server 822 transmits the requested data to web client 826, which provides playback and real-time display of standard video, 360-degree video, and/or other video. Web client 826 provides video data, audio data, and any other data or information for user 834 with the ability to interact with 360-degree video data and/or other video data and/or still image data for viewing and analysis. Play . User 834 may also use web client 826 to download video data, audio data, and any other data or information, and then use a virtual reality device to interact with the 360-degree video data for viewing and analysis. (828) can be used.

Web client 826 may be enhanced with software applications that provide playback of 360-degree video and/or other videos in a variety of different modes. User 834 may select the mode in which the software application provides video playback (e.g., fisheye view shown in FIG. 11, panoramic view shown in FIG. 12, dual panoramic view (not shown), quad view shown in FIG. 13). , and the distorted view shown in Figure 14) can be selected.

9 is a flow diagram illustrating a process 840 for recording video data, audio data, and/or information from an asset 830 in accordance with an implementation of the present invention. Video management system 804 may include a 360-degree camera, fixed camera, narrow-view camera, wide-view camera, 360-degree fisheye view camera, radar, LIDAR, and/or other cameras on, inside, or near asset 830. Receives data signals from various input components 842, such as 802 and fixed cameras 806. Video management system 804 then crash-enhances the video data, audio data, and/or information, for example, using a combination of industry standard formats, such as still images, thumbnails, still image sequences, or compressed video formats. Stored in memory module 810 and/or, optionally, stored in non-collision hardened removable storage device 844 in a fourth embodiment. Data encoder 814 generates a record containing a structured series of bits that are used to construct and record data signal information 846. In a near real-time mode, video management system 804 stores video data in collision hardening memory module 810 while sending only limited video data, such as thumbnails or very short low-resolution video segments, to remote data stores 820, 848. Store and/or, optionally, in a fourth embodiment, store in a non-collision hardened removable storage device.

In another implementation, the encoded records are then sent to an onboard data manager 812, which sequentially combines the series of records chronologically into record blocks containing up to 5 minutes of data. A medium record block contains less than 5 minutes of data, and a full record block contains a full 5 minutes of data. Each record block contains all the data needed to fully decode the signal it contains, including data integrity checks. At a minimum, a record block must begin with a start record and end with an end record.

To ensure that all encoded signal data is stored in the collision hardened memory module 810 and/or optionally in a fourth embodiment to a non-collision hardened removable storage device, the data recorder 808 is powered. If lost, the onboard data manager 812 stores the intermediate record block in a crash hardened memory module 810 or, optionally, in a fourth embodiment at a predetermined rate to a non-collision hardened removable storage device. , where the predetermined speed is configurable and/or variable. Intermediate record blocks are stored at least once per second, but can be as frequent as once every tenth of a second. The rate at which intermediate record blocks are stored depends on the sampling rate of each signal. Every intermediate record block contains the entire record set since the last full record block. The data recorder 808 records each intermediate record block to prevent data corruption or loss of more than a second if the data recorder 808 loses power while storing data in the crash hardened memory module 810. , may alternate between two temporary storage locations in the collision hardened memory module 810. Each time a new temporary record block is stored in a temporary conflict hardened memory location, any existing temporary record blocks previously stored in that location are overwritten.

In this implementation, every five minutes, when the data recorder 808 is in near real-time mode, the onboard data manager 812 records an entire record block containing the last five minutes of encrypted signal data into the collision hardened memory module 810. and/or, optionally in a fourth embodiment, stored in a non-conflict hardened removable storage device for a period of time, such as 2 years, comprising 5 minutes of video data, audio data and/or information. A copy of the entire block of records to be stored in the remote data store 820 for a predetermined retention period is sent to the remote data manager 818. A collision hardened memory module 810, and an optional non-collision hardened removable storage device in a fourth embodiment, stores record segments of the most recent record block for a required storage period, wherein the required storage period is, in this implementation: This is the federally mandated period during which data recorder 808 must operate with an additional 24-hour buffer or store video data in crash hardened memory module 810 and then be overwritten.

10 is a flow diagram showing a process 850 for viewing data and/or information from an asset 830 via a web browser 826 or virtual reality device 828. When an event occurs or a remotely located authenticated user 834 requests a segment of video data stored in the crash hardened memory module 810 through the web client 826, depending on the event, the onboard data manager 812 Given the bandwidth of the wireless data link 816, it will begin transmitting video data off-board in real time at the highest resolution available. A remotely located user 834 initiates a request for specific video and/or audio data in a specific view mode 852 via a web client 826 which forwards the request to a web server 822 over the network 824. do. Web server 822 requests specific video and/or audio data from remote data storage 820 and transmits the requested video and/or audio data over network 824 to web clients 826, 854. Web client 826 displays video and/or audio data in a view mode specified by users 834 and 856. User 834 may download specific video and/or audio data for viewing on virtual reality device 828. In another implementation, in real-time mode, thumbnails are sent first at one-second intervals, followed by short segments of low-resolution video, followed by short segments of high-resolution video.

For simplicity of explanation, process 840 and process 850 are shown and described as a series of steps. However, the steps according to the invention may occur in various orders and/or simultaneously. Additionally, steps according to the invention may occur in other steps not presented and described herein. Additionally, not all steps shown are necessary to implement methods according to the disclosed subject matter.

Fifth embodiments of the real-time data collection and recording system and video analytics system described herein provide remote users with real-time or near-real-time access to a wide range of data, such as event and operational data, video data, and audio data from high-value assets. to provide. Data collection and recording systems record data related to assets and stream the data to remote data repositories and users at remote locations before, during, and after an incident. Data can be streamed to remote data repositories in real time or near real time so that the information is available at least until an incident or emergency occurs. Therefore, there is virtually no need to locate and download “black boxes” to investigate incidents involving assets by streaming information to remote data repositories in real time and ensuring that the information is available at least up to the time of the disaster. DARS performs video analysis of video data recorded for mobile assets to determine, for example, cab occupancy, track detection, and object detection near the track. Remotely located users can use a common web browser to browse and view desired data related to selected assets, request downloads of specific data, locate or transfer files, and use custom applications to view data. There is no need to interact with the data collection and recording system.

DARS provides remote location users access to video data and video analysis before, during, and after an incident, performed by a video analytics system by streaming data to remote data repositories and remote location users. Therefore, the user can review the video data to manually download the video to determine whether the cab is occupied, whether crew members or unauthorized individuals were present during the accident, track detection, object detection near the track, investigation, or other times of interest. , there is no need to extract, and reproduce. Additionally, the video analytics system processes image and video data in real time to provide cab occupancy determination, track detection, object detection near the track, and lead and trail unit determination. Therefore, it is guaranteed that the correct data is always available to users. For example, real-time video processing enhances railroad safety by ensuring that locomotives designated as trail locomotives do not receive lead service. Previous systems used train configuration functions in dispatch systems to provide the location of locomotives within a train. Because information is not updated in real time and crews can change locomotives if necessary, dispatch system information can sometimes become useless.

Prior to the system of the present invention, inspection personnel and/or property personnel had to manually inspect track conditions, manually determine if a vehicle was in a lead or trail position, manually survey the location of each individual object of interest, Manually create a database of the geographical location of all objects of interest, periodically conduct manual field surveys of each object of interest to confirm their location and identify changes in geographical location that differ from the original survey, and ensure that the original database is If the location of the object of interest changes after its creation due to repairs or additional infrastructure development, the database is manually updated, selecting and downloading the desired data from a digital video recorder and/or data recorder, and downloading the downloaded data and/or video offline. By inspecting and checking the track for obstacles, vehicle operators had to physically check for obstacles and/or change them. The system of the present invention eliminates the need for the user to perform these steps; the user only needs to use a common web browser to navigate to the desired data. Asset owners and operators can automate and improve the efficiency and safety of mobile assets in real time, actively monitor track conditions and obtain alert information in real time. The system of the present invention eliminates the need for asset owners and operators to download data from data loggers to monitor tracking status and investigate incidents. As an active safety system, DARS can help operators identify obstacles, send alerts in real time, and/or store information offline and transmit alert information for remote monitoring and storage. Current and historical track detection information and/or information related to detection of objects near the track may be stored in real time in a remote data storage to enable users to view the information when needed. Remotely located users can access a common web browser to explore desired data related to selected assets to view and analyze the asset's operational efficiency and safety in real-time or near real-time.

The real-time data collection and recording system of the fifth embodiment can be used to continuously monitor objects of interest and identify in real time when they are moved or damaged, obscured by foliage, and/or when they are broken and in need of maintenance. DARS utilizes video, image, and/or audio information to detect and identify a variety of infrastructure objects, such as railroad tracks in video, and can follow tracks as mobile assets move, identifying objects of interest along with their geographic location. Databases can be created, audited, and periodically updated. The real-time data collection and recording system of the fifth embodiment uses at least one or a combination of an image measurement device, a video measurement device, and a distance measurement device within, on, or near a mobile asset as part of the data collection and recording system. Image measurement devices and/or video measurement devices include, but are not limited to, 360-degree cameras, fixed cameras, narrow-view cameras, wide-view cameras, 360-degree fisheye view cameras, and/or other cameras. Ranging devices include, but are not limited to, radar and light detection and ranging (“LIDAR”). LIDAR is a surveying method that measures the distance to the target by illuminating the target with pulsed laser light and measuring the reflected pulse with a sensor.

DARS can automatically check track conditions, such as calculating the number of tracks present, identify the current track on which the mobile asset is traveling, detect any obstacles present, or loss of ballast, breakage of track, gauges, etc. Defects such as off-track, misaligned switches, switch run-overs, in-track flooding, snow accumulation, etc. can be detected and preventive maintenance plans can be made to avoid catastrophic events. DARS can also detect rail track switches and track track changes. DARS can additionally detect changes within the location of the data, including whether objects are missing, in the way, and/or not in expected locations. Tracking detection, infrastructure diagnostic information, and/or infrastructure monitoring information may be displayed to a user using a standard web client, such as a web browser. Therefore, there is no need to download files from the data recorder and use proprietary application software or other external applications to view information, as was required with previous systems. This process can be extended to automatically create, audit, and/or update a database with the geographic location of objects of interest and comply with federal regulations. With the system of the present invention, cameras previously installed to comply with federal regulations are utilized to perform a variety of tasks that previously required human interaction, special vehicles, and/or alternative equipment. With DARS, these tasks can be performed automatically as mobile assets move across a region as part of regular revenue services and daily operations. Using DARS, you can leverage the vehicle's normal operation and previously installed cameras to perform tasks that previously required manual labor, saving countless hours of manual labor and time. DARS can also perform tasks previously performed using specialized vehicles, such as when track segments are closed to inspect and locate tracks and objects of interest, which often result in loss of revenue service and require expensive equipment to purchase and maintain. can be prevented. DARS further reduces the amount of time people must be near railroad tracks, reducing overall accidents and potential loss of life.

Data includes measured analog and frequency parameters such as speed, pressure, temperature, current, voltage, and acceleration occurring on and/or near the mobile asset; Measured Boolean data such as switch positions, actuator positions, warning light illumination, and actuator commands; Additional data such as location, speed, and altitude information from the Global Positioning System (GPS) and the latitude and longitude of various objects of interest from a Geographic Information System (GIS); Internally generated information, such as regulatory speed limits for mobile assets based on your current location; Train control status and operational data generated by systems such as Active Train Control (PTC); Vehicle and inertial parameters such as speed, acceleration, and position as received from GPS; GIS data such as latitude and longitude of various objects of interest; video and image information from at least one camera located at various locations within, on, or around the mobile asset; audio information from at least one microphone located at various locations within, on, or around the mobile asset; Information about the mobile asset's operational plan transmitted from the data center to the mobile asset, such as route, schedule, and manifest information; Information about environmental conditions, such as current and expected weather, in the area where the mobile asset currently operates or plans to operate; and data derived from a combination of the above sources, including but not limited to additional data, video, audio analysis, and analytics.

“Track” may include, but is not limited to, the rails and sleepers of a railroad used for locomotive and/or train transportation. “Objects of Interest” may include, but are not limited to, various infrastructure objects installed and maintained near railroad tracks that can be identified using artificial intelligence, such as supervised learning or reinforcement learning on asset camera images and videos. . Supervised learning and/or reinforcement learning utilizes labeled datasets previously defined as “training” data to enable remote and autonomous identification of objects within camera views within, on, or near a mobile asset. . Supervised learning and/or reinforcement learning trains neural network models to identify patterns occurring within visual images obtained from a camera. These patterns - people, crossing gates, cars, trees, signals, switches, etc. - can only be found in a single image. Successive frames within a video can also be analyzed for patterns such as flashing lights, moving cars, sleeping people, etc. DARS may or may not require human interaction at all stages of implementation, including but not limited to labeling the training dataset required for supervised learning and/or reinforcement learning. Objects of interest include, but are not limited to, tracks, track centerline points, milestone signs, signals, intersection gates, switches, intersections, and text-based signs. “Video analytics” means at least one camera, such as a 360-degree camera inside, on, or near a mobile asset, a fixed camera, a narrow-field-of-view camera, a wide-view camera, a 360-degree fisheye view camera, radar, LIDAR, and/or Objects of interest, the geographic location of the objects, obstacles in their tracks, and the distance between the objects of interest and the mobile asset, collected by analyzing video and/or images recorded by imaging, video and/or ranging devices, such as cameras or other cameras; and track misalignment, etc., but is not limited to all understandable information. Video analytics systems can also be used in any mobile property, residential area, space or room with surveillance cameras to enhance video surveillance. In mobile assets, video analytics systems economically and efficiently provide autonomous cab occupancy event detection to remote users.

15 shows a field implementation of a fifth embodiment of a real-time data acquisition and recording system (DARS) 900, in which aspects of the present invention may be implemented. DARS 900 is a system that delivers real-time information, video information, and audio information from a data recorder 902 on a mobile asset 964 through a data center 966 to a remotely located end user 968. Data recorder 902 is installed in vehicle or mobile asset 964 and can record any number of different information via any combination of wired and/or wireless data links 942, such as a wireless gateway/router (not shown). Communicate with the source. Data recorder 902 collects video data, audio data, and other data or information via onboard data link 942 from a variety of sources, which may vary depending on the configuration of the asset. Data recorder 902 includes local memory components such as crash hardened memory module 904 of asset 964, onboard data manager 906, and data encoder 908. In a sixth embodiment, data recorder 902 may also include a non-collision hardened removable storage device (not shown). Exemplary enhanced memory modules 904 may include, for example, a crash event recorder memory module compliant with Federal Regulations and Federal Railroad Administration regulations, a crash survivable memory unit compliant with Federal Regulations and Federal Aviation Administration regulations, and applicable Federal Regulations codes. It may be a compliant crash hardening memory module, or other suitable hardening memory device known in the art. Wired and/or wireless data links may include one or a combination of discrete signal inputs, standard or proprietary Ethernet, serial connections, and wireless connections.

DARS 900 further includes a video analytics system 910 that includes track and/or object detection and infrastructure monitoring components 914. The track detection and infrastructure monitoring component 914 may include a supervised learning and/or reinforcement learning component 924, or other neural network or artificial intelligence component, an object detection and location component 926, and a supervised learning and/or reinforcement learning component 924, and an object detection and location component 926 on or near the track. and an obstacle detection component 928 that detects camera obstacles, such as obstacles present and/or people blocking the camera view. In this implementation, live video data is captured by at least one camera 940 mounted in the cab of asset 964, at or near asset 964. Camera 940 is positioned at an appropriate height and angle to capture video data within and around asset 964 and obtain a sufficient amount of view for further processing. Live video data and image data are captured in front of and/or around asset 964 by cameras 940 and fed to track and/or object detection and infrastructure monitoring component 914 for analysis. The track detection and infrastructure monitoring component 914 of the video analytics system 910 processes live video and image data frame by frame to detect the presence of rail tracks and any objects of interest. Camera position parameters such as height, angle, movement, focal length, and field of view may be provided for track and/or object detection, and the infrastructure monitoring component 914 or camera 940 may enable the video analytics system 910 to detect the camera. It can be configured to detect and determine positions and parameters.

To make status decisions, such as cab occupancy, video analytics system 910 may use, for example, video data from cameras 940, speed, GPS data, and inertial sensor data, weather component 936 data, and Uses supervised learning and/or reinforcement learning components 924 and/or other artificial intelligence and learning algorithms to evaluate asset data 934, such as route/crew manifests, and GIS component data 938. . Cab occupancy detection is inherently vulnerable to environmental noise sources, such as light reflecting from clouds and sunlight passing through buildings and trees while assets are in motion. To handle environmental noise, an asset component 934, which may include a supervised learning and/or reinforcement learning component 924, an object detection and location component 926, an obstacle detection component 928, and velocity. The data, GPS data, inertial sensor data, weather component 936 data, and other learning algorithms are assembled together to form a determination of the internal and/or external state comprising the mobile asset 964. Track and/or object detection and infrastructure monitoring component 914 may also be part of a locomotive security system, including a facial recognition system configured to authorize access to the locomotive, a fatigue detection component to monitor crew alerts, and permissions such as smoking. May include an activity detection component to detect unintended activity.

Additionally, video analytics system 910 may receive location information, including latitude and longitude coordinates of signals, such as stop signs, traffic signs, speed limit signs, and/or object signs near tracks, from the property owner. Video analysis system 910 determines whether location information received from the asset owner is accurate. If the location information is accurate, the video analytics system 910 will store the information and not check the location information again for a predetermined period of time, such as checking the location information monthly. If the location information is inaccurate, the video analysis system 910 determines the accurate location information, reports the accurate location information to the asset owner, stores the location information, and updates the location information for a predetermined period of time, such as checking the location information monthly. I won't check again. Storing location information makes it easier to detect signals such as stop signs, traffic signs, speed limit signs, and/or signs of objects near the track.

Using the supervised learning and/or reinforcement learning component 924, supervised learning and/or reinforcement learning of the tracks is performed using various information obtained from successive frames of video and/or images, also in the data center 966. and additional information received from vehicle data component 934, including inertial sensor data and GPS data, to determine learned data. Object detection and location component 926 may utilize training data received from supervised learning and/or utilize reinforcement learning component 924 and other objects to determine track width and curvature, sleeper location, and object detection data. Utilizes specific information about the mobile asset 964 and the railway, such as vehicle speed, signs, signals, etc., to distinguish between railway tracks. Obstacle detection component 928 provides information about obstacles on or near the track and/or camera obstacles, such as people blocking the camera view, and additional information from weather component 936, route/crew information. Object detection and location components 926, such as manifest data and GIS data components 938, and vehicle data components 934 that include inertial sensor data and GPS data to increase accuracy and determine obstacle detection data. Uses object detection data received from. Mobile asset data from vehicle data component 934 includes, but is not limited to, speed, position, acceleration, yaw/pitch rate, and rail crossing. Any additional information received and utilized from data center 966 includes, but is not limited to, daytime and nighttime details and geographic location of mobile asset 964.

Infrastructure objects of interest, tracks and/or information, diagnostic, and monitoring information processed by the object detection and infrastructure monitoring component 914 can be sent to a data recorder 902 via an onboard data link 942 to encode the data. ) is transmitted to the data encoder 908. Data recorder 902 stores the encrypted data in a collision hardened memory module 904, optionally within an optional non-collision hardened removable storage device of the sixth embodiment, and stores the encoded information over a wireless data link ( It is transmitted to the remote data manager 946 of the data center 966 through 944). Remote data manager 946 stores the encoded data in remote data storage 948 in data center 966.

Obstacle detection 928 or object detection 926, such as the presence of a track in front of an asset, an object on and/or near the track, an obstacle on or near the track, and/or an obstacle blocking the camera view 964 To determine, vehicle analysis system 910 may use supervised and/or reinforcement learning components 924 to process and evaluate camera images and video data from cameras 940 in real time, or other artificial Intelligence, object detection and location components 926, and obstacle detection components 928, and other image processing algorithms are used. The track and/or object detection and infrastructure monitoring component 914 may include speed, GPS data and inertial sensor data, weather component 936 data, route/crew, manifest and GIS component 938 data. Using the processed video data along with the asset component 934 data, external status determinations, such as lead and trail mobile assets, are made in real time. For example, when processing image and video data for track and/or object detection, the video analysis system 910 automatically configures the camera 940 parameters necessary for track detection, detects execution via switches, and , calculate the number of tracks, detect additional tracks along the sides of the asset 964, determine the track on which the asset 964 is currently traveling, detect defects in the track geometry, and detect nearby tracks within the defined limits of the track. Detect track loss scenarios, such as detecting water on the ground, and detect missing slope or track scenarios. Object detection accuracy depends on existing lighting conditions within and around the asset 964. DARS 900 handles various lighting conditions with the help of data onboard assets 964 and additional data collected from data center 966. DARS (900) operates in a variety of lighting conditions, operates in a variety of weather conditions, detects more objects of interest, integrates with existing database systems to automatically generate, audit, and update data, detects multiple tracks, and , works well with curved tracks, detects obstacles, detects track defects that can cause safety issues, and has been enhanced to work in low-cost embedded systems.

Obstacle detection, such as obstacles on or near the track and obstacles blocking cameras, such as cab occupancy, object detection and location, object detection and detection from the video analytics system 910, interior and /or a determination of the external condition is provided to the data recorder 902 along with any data from a vehicle management system (VMS) or digital video recorder component 932 via an onboard data link 942. Data recorder 902 stores internal and/or external state determination, object detection and location component 926 data, and obstacle detection component 928 data in collision hardening memory module 904, optionally In six embodiments the storage is to a non-conflict hardened removable storage device and to a remote data store 948 via a remote data manager 946 located in a data center 966. Web server 958 provides internal and/or external state determination, object detection and location component 926 information and obstacle detection component 928 to a remotely located user 968 via web client 962 upon request. Provides information.

Data encoder 908 encodes a minimal data set, typically as defined by regulatory agencies. Data encoder 908 may output video, It receives image and audio data, compresses or encodes the data, and synchronizes time. The data encoder 908 transmits the encoded data to the onboard data manager 906, which then converts the encoded video, image, and audio data to the on-board data manager 906 in response to on-demand requests from the user 968 or to the In response to certain operating conditions observed on-board at 964, it transmits to remote data storage 948 via remote data manager 946 located in data center 966. The onboard data manager 906 and remote data manager 946 work together to manage the data replication process. A remote data manager 946 in data center 966 may manage replication of data from multiple assets 964.

The onboard data manager 908 determines whether an event has been detected, determines internal and/or external status, detects and locates objects, and/or detects obstacles, and prioritizes detected events. Depending on this, you decide whether to wait or send it immediately. For example, under normal operating conditions, detecting obstacles on the track is much more urgent than detecting whether someone is in the cab of asset 964. Onboard data manager 908 also transfers data to queue storage (not shown). In a near real-time mode, the onboard data manager stores encoded data received from data encoder 908 and any event information in collision hardened memory module 904 and queue storage. After 5 minutes of encoded data have been accumulated in the queue store, the onboard data manager 906 transmits the 5 minutes of encoded data via a wireless data link 944 to a remote data manager 946 in the data center 966. The encoded data is stored in the remote data storage 948. In real-time mode, the onboard data manager 908 converts encoded data received from the data encoder 908 and any event information to the wireless data link 944 at configurable predetermined time intervals, such as every second or every 0.10 seconds. ), through the remote data manager 946 of the data center 966, and stored in the collision hardened memory module 904 and the remote data storage 948.

In this implementation, the onboard data manager 906, via a wireless data link 944, via a remote data manager 946 within the data center 966, receives video data, audio data, determines internal and/or external state, and objects. Sensing and location information, obstacle detection information, and any other data or event information are transmitted to remote data storage 948. Wireless data link 944 may be, for example, a wireless local area network (WLAN), a wireless metropolitan area network (WMAN), a wireless wide area network (WWAN), a wireless virtual private network (WVPN), a cellular telephone network, or a data recorder. From 902, in this embodiment, there may be another means of transmitting data to the remote data manager 946. The process of remotely retrieving data from asset 964 requires a wireless connection between asset 964 and data center 966. If a wireless data connection is unavailable, data is stored and queued until the wireless connection is restored.

In parallel with data recording, data recorder 902 continuously and autonomously replicates data to remote data storage 948. There are two modes for the replication process: real-time mode and near-real-time mode. In real-time mode, data is replicated to the remote data storage 10 every second. In near real-time mode, data is replicated to remote data storage 15 every 5 minutes. The rate used in near real-time mode is configurable, and the rate used in real-time mode can be adjusted to support high-resolution data by replicating data to remote data storage 15 every 0.10 seconds. Near real-time mode is used during normal operation under most conditions to increase the efficiency of the data replication process.

Real-time mode may be initiated based on events occurring at asset 964 or by a request originating from data center 966. A typical data center 966 startup request for real-time mode is initiated when a remotely located user 968 requests real-time information from a web client 962. Common reasons why real-time mode is initiated onboard an asset 964 include: an emergency stop request, an operator initiating an emergency braking activity, rapid acceleration or deceleration on any axis, or loss of input power to the data recorder 902. , This is because it detects events or accidents related to the asset 964. When switching from near real-time mode to real-time mode, all data that has not yet been replicated to the remote data store 948 is replicated and stored in the remote data store 948 and then live replication begins. Switching between near-real-time mode and real-time mode typically occurs within 5 seconds. After a predetermined time has elapsed following an event or incident, after a predetermined period of inactivity has elapsed, or when the user 968 no longer desires real-time information from the asset 964, the data recorder 902 switches to a near real-time mode. returns to The predetermined time required to initiate the conversion is configurable and is typically set to 10 minutes.

When the data recorder 902 is in real-time mode, the onboard data manager 906 continuously attempts to empty its queue to the remote data manager 946 while storing data in the collision hardened memory module 940. , and optionally, in a sixth embodiment, attempts to flush the data to an optional non-conflict hardened removable storage device, while simultaneously transmitting the data to the remote data manager 946.

Upon receiving video data, audio data, internal and/or external state determinations, object detection and location information, obstacle detection information, and other data or information to be replicated from data recorder 902, remote data manager 946 may encode Data received from the onboard data manager 906, such as received data and sensed event data, is stored in a remote data storage 948 within the data center 966. Remote data store 948 may be, for example, a cloud-based data store or any other suitable remote data store. Once the data is received, the data decoder 954 decodes the recently replicated data from the remote data store 948 and combines the decoded data into visible track/object detection/location data stored for further 'post-processed' events. The process for transmitting to information component 950 begins. In this embodiment, tracking/object detection/location information component 950 includes an object/obstacle detection component for determining internal and/or external state, object detection and location information, and obstacle detection information. Upon detecting internal and/or external information, object detection and location information, and/or obstacle detection information, track/object detection/location information component 950 stores the information in remote data storage 948.

A remotely located user 968 may be configured to: determine video data, audio data, internal and/or external conditions, including track information, asset information, and cab occupancy information, with respect to a particular asset 964 or multiple assets; Object detection and location information, obstacle detection information, and other information stored in remote data storage 948 may be accessed by a web browser or virtual reality device (not shown), such as virtual reality device 828 of FIG. 8 (in this implementation). , which can display thumbnail images of selected cameras). Web client 962 forwards the user's 968 request for information over network 960 to web server 958 using common web standards, protocols, and technologies. For example, network 960 may be the Internet. Network 960 may also be from a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a virtual private network (VPN), a cellular telephone network, or a web server 958, in this embodiment: There may be other means of transmitting data to web client 962. The web server 958 requests the desired data from the remote data store 948, and the data decoder 954, in response to the request from the web server 958, retrieves a request related to a particular asset 964 from the remote data store 948. Obtain the data. Data decoder 954 decodes the requested data and transmits the decoded data to localizer 956. Raw encoded data and detected track/object detection/position information are stored in remote data storage 948 using Coordinated Universal Time (UTC) and International System of Units (SI units), according to localizer 956. identifies profile settings set by user 968 by accessing web client 962 and uses the profile settings to prepare information transmitted to web client 962 for presentation to user 968. Localizer 956 converts the decoded data into a format desired by the user 968, such as the user's 968 preferred unit of measurement and language. Localizer 956 transmits localized data in the user's 968 preferred format to web server 958 upon request. Web server 958 then performs track and/or object detection (FIG. 16A), track and switch detection (FIG. 16B), track and/or object detection, track counting and signal detection (FIG. 16C), intersection and track detection (FIG. 16C), and and/or object detection (FIG. 16D), dual overhead signal detection (FIG. 16E), multi-track and/or multi-object detection (FIG. 16F), switch and tracking and/or object detection (FIG. 16G), and switch detection (FIG. 16G). 16h), providing playback and real-time display of standard video and 360-degree video, along with internal and/or external state determination, object detection and location information, and obstacle detection information, and a web client for observation and analysis (FIG. 16H). 962) and transmit localized data.

Web client 962 is enhanced with a software application that provides playback of 360-degree video and/or other videos in a variety of different modes. User 968 can select the mode in which the software application provides video playback, such as fisheye view, distorted view, panoramic view, dual panoramic view, and quad view.

FIG. 17 is a flow diagram illustrating a process 970 for determining the internal state of an asset 964 in accordance with an implementation of the present invention. Video analytics system 910 may include a 360-degree camera, fixed camera, narrow-view camera, wide-view camera, 360-degree fisheye view camera, radar, LIDAR, and/or on, inside, or around asset 964. Various input components, such as cameras 940, vehicle data components 934, weather components 936, and route/manifest and GIS components 938, including but not limited to other cameras ( 972). Video analytics system 910 uses supervised learning and/or reinforcement learning components 974 to process data signals and determine internal states 976, such as cab occupancy.

FIG. 18 is a flow diagram illustrating a process 980 for determining object detection/location and obstacle detection occurring externally and internally to an asset 964, in accordance with this implementation of the invention. Video analytics system 910 may include a 360-degree camera, fixed camera, narrow-view camera, wide-view camera, 360-degree fisheye view camera, radar, LIDAR, and/or on, inside, or around asset 964. Various input components, such as cameras 940, vehicle data components 934, weather components 936, and route/manifest and GIS components 938, including but not limited to other cameras ( 982). Video analytics system 910 processes data signals using supervised learning and/or reinforcement learning components 924, object detection/location components 926, and obstacle detection components 928, 984, and detects obstacles. Detect 986, and determine location 988, such as object detection and track presence.

For simplicity of explanation, process 970 and process 980 are shown and described as a series of steps. However, the steps according to the invention may occur in various orders and/or simultaneously. Additionally, steps according to the invention may occur in other steps not presented and described herein. Additionally, not all steps shown are necessary to implement methods according to the disclosed subject matter.

Real-time data collection and historical data sharing systems provide real-time or near-real-time access to a wide range of data, for example, event and operational data, video data, and audio data of expensive assets such as locomotives, with asset owners, operators, and investigators. It works in conjunction with a real-time data collection and recording system and viewer, providing users located in remote locations as well. Data collection and recording systems record data related to assets and stream data to remote data repositories and remote location users before, during, and after an incident. Data can be streamed to remote data repositories in real time or near real time so that the information is available at least until an incident or emergency occurs. Therefore, there is virtually no need to locate and download a “black box” to investigate an incident involving an asset, request the download of specific data, find and transfer files, and use custom applications to view the data. There is no need to interact with the collection and recording system. Real-time data collection and recording systems maintain typical logging capabilities and add the ability to stream data to remote data storage and remote end users before, during, and after an incident.

Remotely located users, such as asset owners, operators, and/or investigators, can access live and/or historical data related to selected assets by accessing a common web browser to observe and analyze the operational efficiency and safety of assets in real time or near real time. You can explore data. The ability to observe operations in real time or near real time allows you to quickly evaluate and adjust behavior. For example, during an incident, real-time information and/or data can facilitate triage of the situation and provide critical information to first responders. For example, during normal operation, near real-time information and/or data can be used to audit crew performance and support network-wide situational awareness.

Remotely located users can access a common web browser to use the viewer and explore the desired data related to the selected asset to observe and analyze the operational efficiency and safety of the asset in real-time or near real-time. Viewers provide the ability to view operations and/or 360-degree video in real time or near real time, allowing crew behavior to be quickly assessed and adjusted. Owners, operators and investigators can observe and analyze operational efficiency, safety of people, vehicles and infrastructure, and investigate or inspect accidents. For example, during an incident, 360-degree video can help triage the situation and provide critical information to first responders and investigators. For example, during normal operation, 360-degree video can be used to audit crew performance and support network-wide situational awareness. Additionally, a remote user may view the 360-degree video with a viewer in various modes using a virtual reality device, such as virtual reality device 828 in Figure 8, or through a standard web client, such as a web browser. Therefore, there is no need to download and use external applications to watch videos.

A data sharing system allows users to share data obtained from a data acquisition and recording system with remotely located users. Users can share modern web browsers with end users who are remote recipients with Internet access in a secure, controlled, tracked and audited manner. Instead of sharing files, users share URLs to data. URL-based data sharing allows users to control, track, and audit sensitive data. By sharing their data, users can improve the safety of transportation systems around the world without fear of unauthorized data dissemination. Investigators using the web client can share data with remote users without having to find and download “black boxes.”

Data includes: speed, pressure, temperature, current, voltage, and acceleration occurring on and/or near the asset; Boolean data such as switch positions; actuator position; warning light illumination; and actuator commands; location, speed, and altitude; and current location of the asset. Global Positioning System (GPS) data and/or Geographic Information System (GIS) data, such as internally generated information such as speed limits prescribed for the property, video and images from cameras located at various locations on, on, or near the property. information, audio information from microphones located at various locations on, on, or near the asset, route, schedule, cargo manifest information, and environmental conditions, including current and expected weather conditions in the areas where the asset is currently operating or scheduled to operate. Information about the asset's operational plan transferred from the data center to the asset, including asset management status and operational data generated by the system, active train control (PTC) of the locomotive, and additional data, video, and audio analysis and analytics. However, it may include, but is not limited to, analog and frequency parameters, such as, but not limited to, data derived from combinations of the above.

19 and 20 illustrate field implementations, respectively, of a first and second embodiment of an exemplary real-time data acquisition and recording system (DARS) 100, 200, in which aspects of the present invention may be implemented. DARS 100, 200 may be installed on a vehicle or mobile asset 148, 248 and/or via a data link, such as a wireless data link 146, and/or be connected to the data center 150 of DARS 100, 200. , data recorders 154, 254, which communicate with various information sources via any combination of wireless data links 170, 270, such as a wireless gateway/router or off-board information sources via 250). Data recorders (154, 254) include onboard data managers (120, 220), data encoders (122, 222), vehicle event detectors (156, 256), queue stores (158, 258), and wireless gateway/router (172, 272). Additionally, in such implementations, data recorders 154, 254 may include crash hardened memory modules 118, 218 and/or Ethernet switches 162, 262, with or without Power over Ethernet (PoE). You can. Exemplary hardened memory modules 118, 218 may be, for example, crash event recorder memory modules compliant with Code of Federal Regulations and Federal Railroad Administration regulations, and crash survivable memory units compliant with Federal Regulations and Federal Aviation Administration regulations. It may be a crash hardened memory module that complies with applicable federal regulatory codes, or it may be any other suitable hardened memory device known in the art. In a second embodiment, as shown in FIGS. 2 and 20, data recorder 254 may further include an optional non-collision hardened removable storage device 219.

Wired and/or wireless data links 170, 270 may include any one or combination of discrete signal inputs, standard or proprietary Ethernet, serial connections, and wireless connections. Ethernet-connected devices can use the Ethernet switches 162 and 262 of the data recorders 154 and 254, and can use POE. Ethernet switches 162, 262 may be internal or external and may support POE. Additionally, remote data such as map components 164, 264, route/crew roster components 124, 224, and weather components 126, 226 in the implementations of FIGS. 1, 2, 19, and 20. Data from sources is transmitted from the data center 150, 250 via wireless data links 146, 246 and wireless gateways/routers 172, 272, onboard data managers 120, 220, and vehicle event detectors 156. , 256).

Data recorders 154, 254, via onboard data links 170, 270, collect data or information from a variety of sources, which may vary greatly depending on asset configuration. Data encoders 122 and 222 encode minimal data sets typically defined by regulatory agencies. In this embodiment, data encoders 122, 222 receive data from various asset 148, 248 sources and data centers 150, 250. Information sources include analog inputs (102, 202), digital inputs (104, 204), I/O modules (106, 206), vehicle controllers (108, 208), engine controllers (110, 210), and inertial sensors (112). , 212), global positioning system (GPS) (114, 214), cameras (116, 216), active train control (PTC)/signal data (166, 266), fuel data (168, 268), cellular transmission detector ( (not shown), internal driving data and additional data signals, and all components within the data center 150, 250, such as one of the route/crew roster components 124, 224, weather components 126, 226, maps. It may include any components of assets 148, 248, such as any of components 164, 264, and any additional data signals. Cameras 116, 216, or image measurement devices and/or video measurement devices, may include, inside and outside asset 148, 360-degree cameras, fixed cameras, narrow-view cameras, wide-view cameras, 360-degree fisheye view cameras, and /or other cameras, including but not limited to. Data encoders 122 and 222 compress or encode data and time synchronize the data to facilitate efficient real-time transmission and replication to remote data storage 130 and 230. Data encoders 122, 222 send encoded data to an onboard data manager for replicating the data to a remote data repository 130, 230 via a remote data manager 132, 232 located in the data center 150, 250. (120, 220), and the on-board data manager (120, 220) then stores the encrypted data in the crash-hardened memory modules (118, 218) and queue storage (158, 258). Optionally, the onboard data manager 120, 220 may store a third copy of the encoded data in the non-conflict hardened removable storage device 219 of the second embodiment shown in FIGS. 2 and 20. Onboard data managers 120, 220 and remote data managers 132, 232 work together to manage the data replication process. A single remote data manager 132, 232 within a data center 150, 250 may manage replication of data from multiple assets 148, 248.

Data from various input components and from the in-cab audio/graphical user interface (GUI) 160, 260 is transmitted to vehicle event detectors 156, 256. Vehicle event detectors 156 and 256 process data to determine whether an event, incident or other predefined situation related to an asset 148 or 248 has occurred. When the vehicle event detector 156, 256 detects a signal indicating that a predefined event has occurred, the vehicle event detector 156, 256 detects the processed data in which the predefined event occurred along with supporting data surrounding the predefined event. is transmitted to the onboard data manager (120, 220). Vehicle event detectors 156, 256 may include analog inputs 102, 202, digital inputs 104, 204, I/O modules 106, 206, and vehicle controllers 108, 208, which may vary depending on asset configuration. ), engine controller (110, 210), inertial sensors (112, 212), GPS (114, 214), camera (116, 216), route/crew roster component (124, 224), weather component (126, 226), map components 164, 264, PTC/signal data 166, 266, and fuel data 168, 268. When the vehicle event detector 156, 256 detects an event, the detected asset event information is stored in the queue repository 158, 258 and optionally sent to the asset via the in-cab audio/graphical user interface (GUI) 160, 260. Can be provided to crew members at (148, 248).

Onboard data managers 120, 220 also transmit data to queue storage 158. In a near real-time mode, the onboard data manager 120, 220 processes encoded data received from data encoders 122, 222 and random event information into collision hardened memory modules 118, 218 and queue stores 158, 258. Save it to In the second embodiment of FIGS. 2 and 20 , onboard data manager 220 may selectively store encoded data in non-conflict hardened removable storage device 219 . After 5 minutes of encoded data has been accumulated in the queue store 158, 258, the onboard data manager 120, 220 provides a wireless data link 146, accessed via the wireless gateway/router 172, 272. 256), 5 minutes of encoded data is stored in the remote data storage (130, 230) through the remote data manager (132, 232) in the data center (150, 250). In real-time mode, onboard data managers 120, 220 store encoded data and arbitrary event information received from data encoders 122, 222 in collision hardened memory modules 118, 218, optionally as shown in Figures 2 and 218. 20, and accessed via wireless gateways/routers 172, 272, via wireless data links 146, 246, in data centers 150, 250. It is stored in the remote data storage (130, 230) through the remote data manager (132, 232). Onboard data managers (120, 220) and remote data managers (132, 232) support various wireless communication links such as Wi-Fi, cellular, satellite, and private wireless systems using wireless gateways/routers (172, 272). You can communicate through The wireless data link 146, 246 may be, for example, a wireless local area network (WLAN), a wireless metropolitan area network (WMAN), a wireless wide area network (WWAN), a private wireless system, a cellular telephone network, or DARS (100, 200). )'s data recorders 154, 254, in this example, to the remote data managers 130, 230 of DARS 100, 200. If a wireless data connection is not available, the data is stored in memory and waits in queue storage 158, 258 until the wireless connection is restored and the data replication process can resume.

In parallel with data recording, data recorders 154, 254 continuously and autonomously replicate data to remote data storage 130, 230. There are two modes for the replication process: real-time mode and near-real-time mode. In real-time mode, data is replicated to remote data storage 130, 230 every second. In near real-time mode, data is replicated to remote data storage 130, 230 every 5 minutes. The rate used for near real-time mode is configurable, and the rate used for real-time mode can be adjusted to support high-resolution data by replicating data to remote data storage 130, 230 every 0.10 seconds. When DARS 100, 200 is in a near real-time mode, onboard data managers 120, 220 queue data in queue stores 158, 258 before replicating the data to remote data managers 132, 232. Onboard data managers 120, 220 also replicate vehicle event detector information waiting in queue storage 158, 258 to remote data managers 132, 232. Near real-time mode is used during normal operation under most conditions to increase the efficiency of the data replication process.

The real-time mode can be initiated based on events raised and detected by a vehicle event detector 156, 256 onboard the asset 148, 248, or by a request initiated from the data center 150, 250. It can be started based on a given event. When a remotely located user (152, 252) requests real-time information from a web client (142, 242), a typical data center (150, 250) startup request for real-time mode is initiated. Common reasons why real-time mode occurs on an asset (148, 248) are: an operator initiates an emergency stop request, emergency braking activity, rapid acceleration or deceleration on any axis, or loss of input power to the data recorder (154, 254). Such as, event or accident detection by vehicle event detectors 156 and 256. When switching from near-real-time mode to real-time mode, all data that has not yet been replicated to the remote data storage 130, 230 is copied and stored in the remote data storage 130, 230, and then live replication begins. Switching between near-real-time mode and real-time mode typically occurs within 5 seconds. After a predetermined amount of time has passed following an incident or accident, a predetermined period of inactivity, or when the user 152, 252 no longer desires real-time information from the asset 148, 248, the data recorder 154, 254 can record near real-time information. Return to mode. The predetermined time required to initiate the conversion is configurable and is typically set to 10 minutes.

When the data recorder (154, 254) is in real-time mode, the onboard data manager (120, 220) stores data in the collision hardened memory module (118, 218) while also directing its queue to the remote data manager (132, 232). ), and optionally to the non-conflict hardened removable storage device 219 of FIGS. 2 and 20, while simultaneously transmitting data to the remote data manager 132, 232. Onboard data managers 120, 220 also transmit information about detected vehicles waiting in queue storage 158, 258 to remote data managers 132, 232.

Upon receiving the data to be replicated from the data recorders 154, 254 along with the data in the map component 164, 264, the route/crew roster component 124, 224 and weather component 1d26, 226, the remote data manager. (132, 232) stores the compressed data in a remote data storage (130, 230) within the data center (150, 250) of DARS (100, 200). Remote data store 130, 230 may be, for example, a cloud-based data store or any other suitable remote data store. When data is received, a process that causes a data decoder (136, 236) to decode recently replicated data to/from a remote data store (130, 230) and transmit the decoded data to a remote event detector (134, 234). begins. The remote data managers 132 and 232 store vehicle event information in the remote data storage 130 and 230. When the remote event detector 134, 234 receives the decoded data, it processes the decoded data to determine if an event of interest is found in the decoded data. The decoded information is then used by the remote event detector 134, 234 to detect events, incidents, or other predefined circumstances within the data occurring with the asset 148, 248. Upon detecting an event of interest in decoded data previously stored within the remote data store 130, 230, the remote event detector 134, 234 stores event information and supporting data in the remote data store 130, 230. When the remote data manager 132, 232 receives remote event detector 134, 234 information, the remote data manager 132, 232 stores the information in the remote data store 130, 230.

A remotely located user 152, 252 may use a standard browser, such as a web browser or a virtual reality device (not shown), such as virtual reality device 828 of FIG. 8, which in this implementation can display thumbnail images of the selected camera. Web clients 142 and 242 may be used to access information, including vehicle event detector information, associated with a specific asset 148 or multiple assets. Web clients 142, 242 forward user's 152, 252 requests for information to web servers 140, 240 over networks 144, 244, using common web standards, protocols, and techniques. do. Networks 144, 244 may be, for example, the Internet. Networks 144, 244 may also host a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a virtual private network (VPN), a cellular telephone network, or a web client on a web server 140, 240. (142, 242) may be another means of transmitting data. The web servers 140 and 240 request desired data from the data decoders 136 and 236. Data decoders 136, 236 obtain requested data associated with a particular asset 148, 248 or multiple assets from a remote data repository 130, 230 in response to a request from a web server 140, 240. Data decoders 136 and 236 decode the requested data and transmit the decoded data to localizers 138 and 238. Localization is the process of converting data into the format desired by the end user, including converting the data into the user's preferred language and unit of measurement. As raw encoded data and sensed event information are stored in remote data repositories 130, 230 using Coordinated Universal Time (UTC) and International System of Units (SI units), localizers 138, 238 can access web clients ( accesses 142, 242 to identify profile settings established by user 152, 252, and uses the profile settings to prepare information to be transmitted to web client 142, 242 for presentation to user 152, 252; do. Localizers 138, 238 convert the decoded data into a format desired by the user 152, 252, such as the user's 152, 252 preferred language and unit of measurement. Localizers 138 and 238 transmit localized data in the user's preferred format to web servers 140 and 240 upon request. The web server 140, 240 then provides playback and real-time display of standard video and 360-degree video through a viewer, while providing localized data of the asset or a plurality of assets for review and analysis to the web client 142, 242). Web clients 142, 242 may display and users 152, 252 may view data, video, and audio for a single asset or may view data, video, and audio for multiple assets simultaneously. Web clients 142, 242 may also provide synchronous playback and real-time display of data along with multiple video and audio data from both standard and 360-degree video sources, on, within or near the asset and/or remotely. You can. Web clients 142, 242 may play video data in a viewer for users 152, 252, who may interact with the video for viewing and analysis. Users 152, 252 may also use web clients 142, 242 to download video data, which can then be viewed and analyzed using a virtual reality device, such as virtual reality device 828 of FIG. 8. You can interact with the video data on the viewer.

Web clients 142, 242 are enhanced with software applications that provide playback of video data and/or 360-degree video in a variety of different modes. Users 152, 252 can select a mode in which the software application provides video playback, such as, for example, fisheye view, distorted panoramic view, distorted double panoramic view, and distorted quad view.

Using the data sharing system of the present invention, users 152, 252 may further share data in a secure, controlled, trackable, and auditable manner with recipient end-users in remote locations who have Internet access and a modern web browser. Instead of sharing files, users 152 and 252 share URLs to data. Through URL-based data sharing, users can control, track, and audit sensitive data. By sharing their data, users can improve the safety of transportation systems around the world without fear of unauthorized data dissemination. The administrator has the authority to increase and/or decrease the default privileges of users 152, 252 and each remotely located recipient end user. The inherent permissions of users 152, 252 and each remote location recipient end user determine whether a particular remote location recipient end user has permission to view data on web client 142, 242. The data sharing system is used by owners, operators, and investigators of assets 148, 248 to share real-time data on the operational efficiency and safety of the vehicles. Sharing data allows behavior to be quickly assessed and adjusted.

21 illustrates a process 1000 for sharing data and/or information from an asset 148, 248 via a virtual reality device, such as the web browser 142, 242 described above, or virtual reality device 828 of FIG. 8. ) is a flow chart showing. Typically, a user 152, 252 will request that a data center 150, 250 share asset 148, 248 data using a web client 142, 242 1002 (FIG. 3). Common reasons for data sharing are incident detection, such as an operator initiating an emergency stop request, emergency braking activity, rapid acceleration or deceleration on any axis, or loss of input power to DARS (100, 200). No files are downloaded or transmitted to remote recipient end users. Users 152, 252 may not share more than their inherent permissions with the web client 142, 242 allow. Receiving end users located remotely may view the data based on their own privileges on the web client 142, 242. This sharing activity is recorded by the web client 142, 242 of the data center 150, 250. The administrator may share data with a plurality of users 152, 252 who do not have access to the data through the web client 142, 242 using escalation of authority. This privilege escalation activity will also be recorded by the web client 142, 242 of the data center 150, 250.

As previously discussed, users 152, 252 use web clients 142, 242 to access information, including vehicle event detector 156, 256 information. Using common web standards, protocols, and technologies, such as the Internet or private networks 144, 244, web clients 142, 242 communicate information desired by users 152, 252 with web servers 140, 240. do. The web servers 140 and 240 request desired data from the data decoders 136 and 236. Data is extracted by data decoders 136, 236 and then localized by localizers 138, 238, which then convert the data to users 152, 252 as described above. Convert to desired format. Web servers 140, 240 then transmit localized data to web clients 142, 242 for observation and analysis 1004 (FIG. 21).

The sharing end-user 152, 252 uses the web client 142, 242 to view the vehicle event detector 156, regardless of whether the recipient end-user has a pre-registered account with the web client 142, 242. 256) This information, including information and video data, can be shared with multiple recipient end-users located in remote locations. Sharing end-users 152, 252 may share information and data with multiple remotely located recipient end-users, regardless of whether the recipient end-users have pre-registered accounts with web clients 142, 242. In this process, the web client 142, 242 will generate an email with a URL pointing to data in data center 150, 250 1006 (FIG. 21). The recipient end-user, located in a remote location, receives an email containing a URL address to access the data. A URL address is not a link to a file. Files are not shared with recipient end users. The data is not a discrete file, but a range of data extracted from a remote data repository (15) based on a shared web-based viewer link. The URL address sent via email is a link to a web-based viewer that allows the recipient end-user to view specific data segments synchronized with still images and video through the web-based viewer. When a remote recipient end-user clicks on the URL, the end-user can access the shared information using his or her web client 142, 242, and the sharing activity is transmitted to the web client 142 in the data center 150, 250. , 242).

For simplicity of explanation, process 1000 is shown and described as a series of steps. However, the steps according to the invention may occur in various orders and/or simultaneously. Additionally, steps according to the invention may occur in other steps not presented and described herein. Additionally, not all steps shown are necessary to implement methods according to the disclosed subject matter.

As used herein, the term “or” is intended to mean an inclusive “or” and not an exclusive “or.” That is, unless otherwise specified or clear from context, "X contains either A or B" means the natural inclusive permutation. That is, X includes A; X includes B; Or, if Also, “X contains at least one of A and B” means any natural inclusive permutation. That is, X includes A; X includes B; Or, if X includes both A and B, then “X includes at least one of A and B” is satisfied in any of the preceding cases. As used in this application and the appended claims, the articles “a” and “an” should generally be construed to mean “one or more,” and the context shall take the singular form. Additionally, use of the terms “one implementation” or “an implementation” are not intended to refer to the same embodiment, aspect, or implementation, unless so described.

Although the present invention has been described in connection with specific embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, includes various modifications and corresponding arrangements included within the scope of the appended claims, the scope of which It is to be construed most broadly to include all modifications and corresponding structures permitted by law.

Claims (21)

1. A method for providing access to view data from at least one mobile asset, said method comprising:
Using a data recorder onboard the mobile asset, record data based on at least one data signal from at least one of at least one data source onboard the mobile asset and at least one data source remote from the mobile asset. receiving;
Using a data encoder of the data recorder, encoding the predetermined amount of data into encoded data;
appending the encoded data to a data segment using an onboard data manager of the data recorder;
Using the onboard data manager, store at least one of the encoded data and the data segment, at a first configurable predetermined rate, in at least one local memory component of the data recorder and within a queue store of the data recorder. steps;
transmitting, using the onboard data manager, at least one of the encoded data and the data segment to a remote data manager via a wireless data link at a second, configurable, predetermined rate;
storing, using the remote data manager, at least one of the encoded data and the data segment to a remote data storage;
Using a web server, receive a request from a first user, wherein the request includes at least one email address of at least one second user and specific data selected by the first user stored in the remote data storage. Contains-;
identifying the specific data within the remote data store;
A uniform resource locator comprising a shared web-based viewer link configured to provide access for the first user to view the specific data under a first condition that includes a first default right to view the specific data. (URL));
generating an email including the URL;
Using the web server, sending the email to the at least one email address; and
in the web browser of the at least one second user if the second user selects the URL provided in the email in a second condition that includes a second default permission allowing access to view the specific data. A method comprising displaying the specific data. delete According to paragraph 1,
requesting the specific data from a data decoder using the web server; and
decoding the specific data using the data decoder;
A method further comprising: According to paragraph 3,
The method further comprising processing, using a localizer, the specific data into processed specific data comprising a predetermined language and at least one predetermined unit of measurement. According to paragraph 1,
further comprising, using a web client, storing a record within the remote data store, wherein the record includes at least one of the email, the URL, the specific data, the first user, and the second user. A method characterized by: According to paragraph 1,
The at least one data source onboard the mobile asset may include analog inputs, digital inputs, input and output modules, vehicle controllers, engine controllers, inertial sensors, global positioning systems (GPS), at least one camera, fuel data, and active trains. Control (PTC) signal data, comprising at least one of a 360-degree camera, a fixed camera, a narrow view camera, a wide view camera, and a 360-degree fisheye view camera,
The at least one data source remote from the mobile asset may include a map component, a route and crew roster component, a weather component, a 360-degree camera, a fixed camera, a narrow-view camera, a wide-view camera, and a 360-degree camera. Also includes at least one of a fisheye view camera,
The data may include speed, pressure, temperature, current, voltage, acceleration into the mobile asset, acceleration from a remote mobile asset, switch position, actuator position, warning light illumination, actuator commands, position, altitude, internally generated information, video information, A method comprising at least one of audio information, route, schedule, manifest information, environmental conditions, current weather conditions, and forecasted weather conditions. A system for providing access to view data from at least one mobile asset, the system comprising:
A data recorder onboard the mobile asset including at least one local memory component, a data encoder, an onboard data manager, and a queue store.
- wherein the data recorder is configured to include at least one of at least one data source onboard the mobile asset and at least one data source remote from the mobile asset,
the data encoder is configured to encode a predetermined amount of data into encoded data,
The onboard data manager adds the encoded data to a data segment, and adds at least one of the encoded data and the data segment at a first predetermined rate configurable to at least one of the at least one local memory component and the queue store. store and transmit a predetermined amount of at least one of the encoded data and the data segment to a remote data manager via a wireless data link at a second configurable predetermined rate;
It is composed,
the second predetermined rate is in the range of 0 seconds to 5 minutes, and the remote data manager is configured to store a predetermined amount of at least one of the encoded data and the data segment in a remote data storage;
Receiving a request from a first user, and providing access to view the specific data selected by the first user under first conditions including a first basic right allowing the first user access to view the specific data Determine a uniform resource locator (URL) containing a shared web-based viewer link configured to generate an email containing the URL, and transmit the email to at least one email address of at least one second user. server
Including,
The request includes the at least one email address of the at least one second user and a URL that allows the at least one second user to view the specific data stored in the remote data storage.
A system characterized by: In clause 7,
The system further comprising a remote data decoder configured to decode the specific data into decoded data. In clause 7,
The system further comprising a localizer configured to process the specific data into processed specific data comprising a predetermined language and at least one predetermined unit of measurement. In clause 7,
further comprising a web client configured to store a record within the remote data store, wherein the record includes the URL, the specific data, the first user, the first user's email address, the at least one second user, and the A system comprising at least one of said at least one email address of at least one second user. According to paragraph 1,
The method of claim 1 , wherein an administrator can increase or decrease the first basic authority of the first user and the second basic authority of the at least one second user. According to clause 11,
the first user may request the specific data on the condition that the administrator expands the first basic authority of the first user;
The at least one second user can view the specific data on the condition that the administrator extends the second basic permission of the at least one second user.
A method characterized by at least one of the following. According to paragraph 1,
transmitting a request for the specific data from a first web client of the first user to the web server;
requesting the specific data from a data decoder using the web server;
Using the data decoder, decoding the specific data into decoded specific data; and
Using a localizer, processing the decoded specific data into processed specific data comprising a predetermined language and at least one predetermined unit of measurement.
A method further comprising: 2. The method of claim 1, wherein the first user can share the URL with a plurality of remotely located recipient end users. 2. The method of claim 1, wherein the at least one second user does not need to have a pre-registered account. In clause 7,
A first device configured to transmit a request for specific data to the web server, wherein the web server is configured to request the specific data from a data decoder, and the data decoder is configured to decode the specific data into decoded specific data. web client; and
a localizer configured to process the decoded specific data into processed specific data comprising a predetermined language and at least one predetermined unit of measure;
A system further comprising: 8. The system of claim 7, wherein the first user can share the URL with a plurality of remotely located recipient end users. 8. The system of claim 7, wherein the at least one second user does not need to have a pre-registered account. In clause 7,
When the at least one second user selects the URL provided in the email under the second condition that the URL includes a second default permission allowing access to view the specific data, the at least one second user's web The system further comprising at least one second web client configured to display the specific data on a browser. According to clause 19,
A system wherein an administrator can at least one of increase or decrease the first basic authority of the first user and the second basic authority of the at least one second user. According to clause 19,
the first user may request the specific data on the condition that the administrator expands the first basic authority of the first user;
The at least one second user can view the specific data on the condition that the administrator extends the second basic permission of the at least one second user.
A system characterized by at least one of the following.

Images