WO2004036926A2 - Video and telemetry apparatus and methods - Google Patents

Video and telemetry apparatus and methods Download PDF

Info

Publication number
WO2004036926A2
WO2004036926A2 PCT/US2003/033119 US0333119W WO2004036926A2 WO 2004036926 A2 WO2004036926 A2 WO 2004036926A2 US 0333119 W US0333119 W US 0333119W WO 2004036926 A2 WO2004036926 A2 WO 2004036926A2
Authority
WO
WIPO (PCT)
Prior art keywords
video
data
user
meta
file
Prior art date
Application number
PCT/US2003/033119
Other languages
French (fr)
Other versions
WO2004036926A3 (en
Inventor
Daniel Pearlson
Original Assignee
Global Epoint, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Global Epoint, Inc. filed Critical Global Epoint, Inc.
Priority to AU2003286491A priority Critical patent/AU2003286491A1/en
Publication of WO2004036926A2 publication Critical patent/WO2004036926A2/en
Publication of WO2004036926A3 publication Critical patent/WO2004036926A3/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23116—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving data replication, e.g. over plural servers
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19639—Details of the system layout
    • G08B13/19647—Systems specially adapted for intrusion detection in or around a vehicle
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665—Details related to the storage of video surveillance data
    • G08B13/19669—Event triggers storage or change of storage policy
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665—Details related to the storage of video surveillance data
    • G08B13/19671—Addition of non-video data, i.e. metadata, to video stream
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665—Details related to the storage of video surveillance data
    • G08B13/19676—Temporary storage, e.g. cyclic memory, buffer storage on pre-alarm
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/27—Server based end-user applications
    • H04N21/278—Content descriptor database or directory service for end-user access
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6377—Control signals issued by the client directed to the server or network components directed to server
    • H04N21/6379—Control signals issued by the client directed to the server or network components directed to server directed to encoder, e.g. for requesting a lower encoding rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H04N21/6582—Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H04N21/8405—Generation or processing of descriptive data, e.g. content descriptors represented by keywords
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention is related to video and telemetry, and in particular to methods and systems for acquiring, storing, and distributing video and telemetry information. Description of the Related Art
  • the present invention is related to methods and systems for acquiring, storing, and distributing video and telemetry.
  • Example embodiments further provide methods and systems for the comprehensive management of audio, video and data recorded by one or more sensors.
  • embodiments of the present invention enables users to select and play back video records based on meta-data embedded within the video record.
  • One embodiment is a recording and transmission system comprising: a recorder mountable in a police vehicle configured to record video and meta-data synchronized with the video, wherein the recorded video includes a plurality of segments having fixed lengths, the recorder including a wireless transmitter configured to wirelessly transmit the recorded video segments and meta-data, wherein the meta-data includes: data related to vehicle status; subjective time information; objective time information; serial data.
  • An assembler is configured to wirelessly receive the transmitted video segments and assemble the video segments into a file including the meta-data, the assembler further including a mass storage device used to store the assembled video and transmit the assembled video.
  • An information management system is coupled to the assembler to receive the assembled video and to store the meta-data in a relational database, wherein the information management system is configured to receive a user search query from a first terminal having a first resolution, conduct a search of the meta-data based at least in part on the search query, transmit the results of the search to the first terminal, receive a user selection of at least a first of the results, configure the video corresponding to the selected result for the first terminal based at least in part on the first resolution, and transmit the configured video to the first terminal.
  • Another embodiment is a method of storing and providing video data, the method comprising: receiving a first video file including video and embedded meta-data, the metadata including at least timing information and status information; storing copies of at least a portion of the first video file in a second video file on a first data storage system and in a third file on a second storage system; generating at least a first string searchable file based at least in part on the meta-data; receiving a user search query; conducting a search of information in the first string searchable file based at least in part on the user search query; presenting results of the search to the user via a user terminal; receiving a user search results selection corresponding to the second video file; retrieving the second video file and the third video file based at least in part on the user selection; comparing at least a first portion of the meta-data in the second video file with a corresponding portion in the third video file in order to determine if file tampering has occurred; and transmitting at least a portion of the second video file to the user terminal.
  • One embodiment is a method of storing and providing video data, the method comprising: receiving and storing data including a video sequence and synchronized metadata; generating at least a first string searchable data structure based at least in part on the meta-data; receiving a user search query; and conducting a search of information in the first string searchable data structure based at least in part on the user search query.
  • One embodiment is an apparatus for storing and providing video data, the apparatus comprising: a storage device configured to store data including a video sequence and synchronized meta-data; a first instruction configured to generate at least a first string searchable data structure based at least in part on the meta-data; a second instruction receive a user search query; and a third instruction configured to conduct a search of information in the first string searchable data structure based at least in part on the user search query.
  • Figure 1 illustrates an example video acquisition, storage and distribution system.
  • Figure 2 illustrates an example authentication process and system.
  • Figure 3 illustrates an example Web page used to display a retrieved video.
  • Figures 4A, 4B and 4C illustrate example video files with embedded timing information.
  • Figure 5 illustrates example video segments.
  • Figure 6 illustrates an example assembler file conversion process.
  • Figure 7 illustrates an example of merging data from multiple sources for rendering on a user terminal.
  • Figure 8 illustrates an example search page.
  • Figure 9 illustrates an example file access, download and rendering process.
  • Figure 10 illustrates an example encoding process.
  • Example embodiments further provide video archive management that enables users to select and play back video records based on meta-data embedded within the video record.
  • the meta-data when used in a law enforcement patrol vehicle scenario, may include, by way of example, date, time, vehicle identification, personnel names, global positioning coordinates, and a comprehensive variety of additional data, including mobile data terminal traffic, event logs or intrusion alarms.
  • automated upload of video files onto archive storage media and automatic entry into a corresponding database is also provided, optionally with little to no user intervention.
  • files including video, audio, and meta-data, are temporarily stored onboard vehicle-based recording units which are transmitted to an archive database when the vehicle returns to its home station.
  • the meta-data is optionally stored both in the home station archive database, as well as on a remote centralized database that stores meta-data from a plurality of station archive databases.
  • a variety of electronic sensors and associated electronics and software, that encode, transmit, and record video, audio, and many channels of streaming real-time meta-data, such as telemetry and event information, can be included in an embodiment of a surveillance system to detect a variety of events or occurrences.
  • the meta-data is embedded and synchronized with the recorded video.
  • the sensors may be positioned on mobile platforms, such as cars, trains, or planes, or on stationary platforms, such as buildings, posts, fences, and the like.
  • the sensors can include, by way of example, one or more of pan/tilt/zoom, passive and active infrared and visible light cameras, covert wireless cameras and microphones, glass breakage detectors, door open/close detectors, gun discharge detectors, crash detectors, light bar on/off inputs, siren on/off inputs, gun rack detectors that sense when a gun is removed from and/or placed in the rack, vehicle speed sensors, tachometer, radar gun, fuel level, heartbeat monitors, pulse monitors, chemical weapons detectors, GPS systems, radio receive and transmit switches, public address ports, and the like.
  • the sensors and supporting circuitry are optionally capable of simultaneously transmitting live (low latency) video, audio, and data telemetry wirelessly over wireless networks while recording, and further provide data terminal functionality.
  • a variety of server components are used to cost effectively and securely deliver video, audio, and data telemetry, such as that recorded via the sensors and supporting circuitry, to long-term storage archives, optionally in accordance with Federal Evidentiary Rules.
  • an example apparatus Upon receiving the recorded files that include video, audio, and data telemetry, an example apparatus automatically extracts and parses pre-determined data-points from the recorded telemetry to construct a scalable video meta-data "comparator" file that is used by a central database to enter the data to fields for sorting; seek extract, and include selected data and information from other databases related to the recording for central access; and provide for string-searches of meta-data embedded in videos. For example, in response to user string searches and using the meta-data entered into the meta-data database, specific videos and even specific segments or frames within a video sequence are located and retrieved from the video archives.
  • the located videos can be substantially instantly delivered to the requesting user starting from the moment of a selected meta-data "incident" or data-points, or beginning at a selected time period relative to the incident or data point.
  • use of an embodiment of the present invention can result in the reduction or elimination in the need to spend time manually locating and retrieving video records, or manually viewing such records in order to locate specific events.
  • the system advantageously optionally provides secure user access locally and/or remotely, optionally without needing customized client display software on the user computer or terminal.
  • access to and display of videos and meta-data can be provided using a standard browser, such as Microsoft Internet Explorer or the like, and a standard media player, such as Windows Media Player or Apple QuickTime, executing on a personal or server computer, personal digital assistant, cell phone, and the like, thereby lowering the client cost and easing maintenance.
  • custom media players and browsers can be used as well.
  • video teleconferencing interfaces such as the standard H.323 teleconferencing, is provided for remote observers. By optionally using an existing teleconferencing standard the system is compatible with legacy video teleconferencing systems.
  • the system and applicable sensors can utilize one or more of ISO MPEG-4, MPEG-1, MPEG-2, and H.323 encoding/decoding.
  • the system employs extremely compressed video to reduce bandwidth and storage requirements.
  • an I (intracoded) frame is only transmitted when at least a certain number or percentage, such as 25% of pixels change luminance and/or chrominance relative to the previous intracoded frame, otherwise P (predicted) frames or B (bi-directionally interpolated) frames are used.
  • P (predicted) frames or B (bi-directionally interpolated) frames are used.
  • P frames predicted frames
  • B bi-directionally interpolated
  • I frames generally utilize many more bits than P or B frames.
  • this process allows six frames/second to be transmitted using as little as about 5.9 kbs, and at a nominal transmission rate of 7.2kbps, which is well within the capacity of existing cellular data transmission services.
  • One embodiment advantageously utilizes asymmetrical encoding which provides a multi-session digital video compression process that enables multiple simultaneous transmissions of various video resolutions and frames rates while a high quality recording is created or played. This process permits various users utilizing dissimilar devices to view live and/or prerecorded video transmissions appropriate for the type of viewing device in use.
  • Web site is used to refer to a user-accessible server site that implements the basic World Wide Web standards for the coding and transmission of hypertextual documents. These standards currently include HTML (the Hypertext Markup Language) and HTTP (the Hypertext Transfer Protocol).
  • Java script also referred to as JavaScript
  • JavaScript JavaScript
  • VBScript Active X objects
  • site is not intended to imply a single geographic location, as a Web or other network site can, for example, include multiple geographically distributed computer systems that are appropriately linked together.
  • other networks such as networked interactive televisions, and other protocols may be used as well.
  • the functions described herein are preferably performed by executable code and instructions running on one or more general-purpose computers, terminals, personal digital assistants, cellular phones, or the like.
  • the present invention can also be implemented using special purpose computers, state machines, and/or hardwired electronic circuits.
  • the example processes described herein do not necessarily have to be performed in the described sequence, and not all states have to be reached or performed.
  • the commands or selections can also be made using other input techniques, such as using voice input, pen input, mousing or hovering over an input area, and/or the like.
  • the overall system is shown in Figure 1 to which reference is hereby made.
  • the system includes a mobile video/audio data recorder 100 which records high quality video/audio and serial data telemetry while simultaneously transmitting specially formatted live video/audio and information to personnel using a variety of reception devices or terminals.
  • the mobile video/audio data recorder 100 optionally records at 30 frames per second, though other frame rates can be used as well.
  • the mobile video/audio data recorder 100 utilizes RTSP (real time streaming protocol), a client-server multimedia presentation control protocol, which provides efficient delivery of streamed multimedia over IP networks.
  • the RTSP may utilize RTP (Realtime Transport Protocol), which is a packet format for multimedia data streams.
  • RTP Realtime Transport Protocol
  • the mobile video/audio data recorder 100 optionally saves video files in multiple file segments, the length and/or duration of which is selectable by a user or system administrator. For example, if segment sizes of 300 seconds are selected with a recording bit-rate of 800kbits per second, mobile video/audio data recorder 100 will break long duration recordings into files of approximately 30Mbytes each.
  • the foregoing process advantageously provides enhanced file resiliency, thereby reducing the incidence of file corruption, and facilitating the recovery of damaged or corrupted files by limiting the size of each binary recording.
  • the recorder 100 can transmit files, including video records, audio records, and/or meta-data, using one or more of CDPD, GPRS, GSM, satellite, dial- up, or broadband Internet networks.
  • the frame rate, bit rate and resolution for transmission can be selected based on the wired or wireless network and the receiving terminal.
  • some devices such as palm-held personal digital assistants (PDAs) and hybrid cell phones may have limited display resolution and processor power
  • the recorder 100 maintains compatibility with the target device by optionally formatting video specifically for the target device.
  • personnel equipped with a PDA that may only be capable of displaying 15 video frames (pictures) per second at a resolution of 320x240 pixels connect to a special URL (Universal Resource Location) associated with a Web site, and are provided video specifically formatted for the receiving device.
  • URL Universal Resource Location
  • live video can selectively be transmitted at one or more of the following video frame/bit rates, while at the same time optionally recording at a minimum frame rate and resolution, such as 30 FPS at a minimum resolution of 352 x 240 with 48KHz audio:
  • FPS frames per second
  • a resolution of 160 x 120 utilizing less than 8 kilobits per second via CDPD, GSM, GPRS, 802.1 lg, 802.11b, 802.11a, or Ethernet 10/100.
  • the mobile video/audio data recorder 100 can be located in and transmit from a variety of stationary and mobile locations, such as aircraft, ships, patrol vehicles, buildings, lots, fields and the like. i one example embodiment, a ring buffer in a recorder, such as recorder 100, captures video, audio and data until an instruction or trigger to start the recording process occurs. When the instruction or trigger is received, the content of the ring buffer is assigned a new serial record number.
  • a file is created in the system's local temporary archive with a file name including the serial record number, system identifier, and the user assigned file naming convention is appended.
  • the start instruction can be the result of any particular event, such as a patrol vehicle light bar being turned on, or as a result of a user providing a manual signal start instruction.
  • Files, including video, audio, and meta-data, are then recorded and closed in optionally fixed duration intervals. The file intervals can be specified by the user or administrator during system set-up.
  • the files can include embedded video, audio, and meta-data. Meta-data is recorded as virtual channels utilizing a packet-based addressing scheme, enabling a large number of virtual channels, such as up to 256 or more virtual channels.
  • the meta-data can optionally correspond to inputs from one or more of the sensors and detectors discussed above.
  • a buffer such as the ring buffer, captures the data for the next file segment so that data is not lost.
  • the data, video, and audio sources and encoders can remain active during file close intervals.
  • the files are stored in an archive clearing directory, and are serialized and identified by a record event string in preparation for automated file upload to an assembler server system 120.
  • the video/audio data signals recorded by the mobile video/audio data recorder 100 are uploaded either wirelessly or via a wired connection to the assembler server system 120.
  • the recorder 100 can automatically upload the recorded video/audio and data using 802.11(b, g, or a) protocol upon reaching the police station at which the vehicle is based.
  • the data transmitted with the video/audio can, for example, include the date and time of recorded segments, together with organization and sequencing information, as well as digital serial data and event data. This data may also include police vehicle number, shift, time, personnel designations, and other fields, as desired.
  • the date and time information can be embedded in the video file with high accuracy, such as, in one embodiment, to within 1/30 of a second accuracy.
  • the date and time are recorded as meta-data, the date and time information does not have to be recorded in the actual visible video frame.
  • the date and time do not obstruct information in the video frame, and instead are optionally displayed in a separate window.
  • the serial data can include records of data communications of an external remote data terminal, alarm data from an external security sensor system, or other data from other serial interfaces, such as an RS-232 or IP interface.
  • the serial data can be embedded in the video stream and recorded in the video file to within 1/30 of a second accuracy, though greater or less accuracy can be used in other embodiments, hi one embodiment, up to 256 external data sources can be simultaneously supported, though in other embodiments more than 256 data sources can be recorded, h addition to serial data, parallel data can be recorded as well.
  • parallel data is first serialized and then recorded as serial data.
  • the data from any virtual channel are recorded as meta-data, the data and/or information derived therefrom does not have to be recorded in the actual visible video frame. Thus, upon viewing or playback, the data and/or information do not obstruct information in the video frame, and instead are optionally displayed in a separate window.
  • One embodiment of the recorder 100 provides event-based recording that enables users to activate a recording session by either pressing a button or activating another device used as an event trigger such as a vehicle light bar, or an alarm contact or other sensor.
  • an event trigger such as a vehicle light bar, or an alarm contact or other sensor.
  • a digital "flag" is created and embedded that enables a viewer to find the event during playback. Recording may optionally be started remotely by network connection, either manually, or automatically using a plurality of predetermined external events, parameters, or triggers.
  • a record of the location of a vehicle, such as a patrol vehicle or aircraft can be embedded in the video file.
  • the GPS data can be provided by a GPS system within the vehicle and embedded in the same fashion as serial data.
  • the assembler 120 recognizes each recorder 100, and interrogates the recorder 100 for new records or files. If the recorder 100 contains new recordings, the assembler 120 automatically commences the file upload process. Once the files have been transferred and verified, the assembler 120 notifies the recorder 100 that the file upload is complete, and instructs the recorder 100 to clear its record volume for new recordings. The assembler 120 optionally returns the recorder 100 to normal operations mode, or shuts the recorder 100 down.
  • the assembler 120 can, for example, be located in a facility with which the recorder 100 is likely to come within close proximity with on a regular basis, such as a station house. This allows a relatively short-range wireless network, such as 802.1 lg, to be used to communicate information and video between the recorder and the assembler 120.
  • the assembler 120 is coupled to a media server 123, such as Microsoft Media
  • VAC video access controller
  • archive controller 126 is coupled to mass storage units 160, 162.
  • the assembler server 120 optionally "assembles" associated files into a one cohesive and user-playable video recording.
  • the assembler extracts and parses meta-data from the meta-data channels of the newly archived video file.
  • a string searchable format file such as an XML document, is then created by an XML generator 128 and associated with the video recording, parsed by information type, to enable "string" searches of the data embedded with the video and audio.
  • the searchable document is stored in an information management system (IMS) 132.
  • IMS information management system
  • the assembler 120 combines related files into a single Internet streamable file containing the original data while preserving encoded RTP/RTSP streaming protocols embedded in the files by the recorder 100.
  • the video/audio stream is then transmitted, either immediately, or by predetermined schedule, by the assembler 120 to an appropriate video-on-demand database or storage media, such as the storage units 160 and 162 controlled by the archive controller 126.
  • the storage unit 160 can be implemented as a RAID (redundant array of inexpensive disks) system providing high speed storage and retrieval, while the storage unit 162 can be implemented as a plurality of high capacity optical digital data storage disks providing relatively more capacity at lower overall long-term storage cost than the storage unit 160.
  • the storage unit 162 provides a more permanent record of video stored in the RAID system 160, however, the write/read time may be slower than that provided by RAID system 160.
  • the storage unit 162 may be in the form of an optical WORM (write once, read many) drive or carrousel, including DVD disk or disks, the use of which prevents the editing of recorded files.
  • the system may include one or more digital recorder transmitter units 169, which can include or be connected to a plurality of mobile video/audio data recorders.
  • the digital recorder transmitter unit 169 can optionally record on internal magnetic or optical media 165 large amounts of video, such as one year or more of GIF video from, each of several cameras at relatively high frame rates. For example, one embodiment records at 30 frames per second (fps) for 160 video cameras at GIF simultaneously, with multiple audio channels, though other frame rates, resolutions, and quantity of video cameras can be used as well.
  • the digital recorder transmitter unit 169 can further record one or more audio sources and multiple streaming data sources, such as data from alarm panels and machine controls.
  • the system can also include one or more encoder systems 172, which is an MPEG multichannel encoder coupled to a plurality cameras transmitting live video, coupled to assembler keeper 139 via (Video Access System) VAS 170.
  • the digital recorder transmitter unit 169 also includes an omniport which includes or acts as a television tuner card.
  • the omniport acts as a capture device so that the digital recorder transmitter unit 169 can be tuned to receive CCTV cable channels. This provides the capability for frequency division multiplexing to view desired images. This capability enables the user of the digital recorder transmitter unit 169 to capture the video/audio images that may be generated in such a closed circuit television system of the type often utilized for security purposes in various facilities.
  • the omniport interfaces into existing frequency modulation CCTV signals. This may include a number of different cameras, for example 160 or more cameras, the images of which are substantially simultaneously transmitted within a particular facility.
  • the user of the digital recorder transmitter unit 169 may then select the desired channel in addition to being able to record images from other cameras.
  • One such specific application where security is involved is on trains.
  • the digital recorder transmitter unit 169 also includes an assembler server system
  • the output from the digital recorder transmitter units 169 may then be transmitted to the assembler keeper 139 as was the XML signal from the assembler 120.
  • the video/audio signal is then transmitted directly to a storage medium, such as the RAID system 165.
  • the meta-data and other identifying information regarding the video/audio is transmitted to the IMS 132.
  • the archive controllers 126 and 168 respectively control the recording of the video in the RAID systems 160 and 165.
  • the storage unit 162 stores an exact copy of at least a portion of the meta-data and/or video on more permanent storage, such as on a optical media, of selected files recorded in the RAID system 160 for authentication purposes, as discussed below, and for backup purposes. Further, having the recorded copy can be kept at a site geographically remote, such as more than 150 meters distant, from the RAID system 160 provides for increased security.
  • the IMS 132 includes one or more relational databases, such as Oracle 9i or lOg, an ASP (Active Server Page)/Fusion module 134, a Java generator 138, an assembler keeper 139, and a computer aided dispatch (CAD) keeper 140.
  • the CAD keeper 140 is further coupled to a CAD system 175.
  • the ASP/Fusion module 134 is used to generate pages, such as HTML pages, that include one or more scripts, such as JavaScript or VBScript.
  • the module 134 processes the scripts on a Web server associated with the module 134 to customize the pages for the user and to provide transaction security. The customization can be based on information received from the user or the user terminal.
  • the information can be provided by way of a URL or address for an appropriate Web site page.
  • the URL can be provided by the user browser when requesting a page to access data from one or more of the system databases.
  • the ASP/Fusion module 134 can then build the requested page in substantially real time and send it, via a Web server, to the requesting user terminal for display in a browser or other application.
  • the ASP/Fusion module 134 can be implemented using Macromedia's ColdFusion product, including the ColdFusion Studio and the ColdFusion Server, though other applications and Web servers can be used as well.
  • the CAD system 175 can, for example, be used to process emergency or help calls from the public and handle police dispatches to police officers or police vehicles. Of course the CAD system 175 can also be used to handle dispatches from other entitites, such as fire department dispatches, hospital dispatches, and dispatches of other governmental or non-governmental entities. By way of further example, a CAD system 175 can be used to guide a call-taker or dispatcher into collecting relevant information, such as the address or location, related to an incident or compliant being reported, which the call-taker then enters into a form presented by the CAD system 175 on the call-taker's computer terminal. The information is optionally then stored in a CAD database.
  • the CAD system 175 may check to make sure certain information, such as the address, is valid.
  • the CAD system 175 or other entity may assign an incident or case number or code to the incident.
  • Appropriate resources such as one or more police cars, foot patrols, motorcycle officers, and/or ambulance units are then selected and dispatched to handle the incident.
  • the CAD database also stores information on which resources were selected and dispatched to handle the incident.
  • One or more status events for each resource may also be recorded in the database. For example, the status can include an acknowledgement from the resource that the dispatch was received. Another status can include a report from the resource that the resource has arrived at the scene of the incident. Still another status might relate to the resource leaving the scene.
  • the IMS 132 provides users with the ability to view a particular video record which has been made of a particular event in the past.
  • the IMS 132 further enables the user to search for meta-data associated with archived videos, permits users to play back recordings at subjective times based on the meta-data and data field searches, and enables playback of specific records when selected by a user.
  • Authorized users also have the ability to navigate and play back requested files on the storage system, provided that they have clearance to do so. There can be multiple levels of clearance, such as up to 30 or more levels of authorization that are used to determine user access to file types and specific data.
  • the authorization can be centrally specified by the system administrator.
  • the IMS 132 tracks user access to files and user activity on the IMS 132.
  • a user chooses a file by going directly to the storage system to commence playback directly from archive, the user's activities are automatically logged. User's are optionally prevented from erasing archived files. Access to "housekeeping" functions are optionally reserved to the system administrator.
  • the IMS 132 extracts files created by the assembler 120 and automatically enters files to its database that are initially designated by the system administrator or other user during system set-up. Video files are stored in a separated storage volume, such as storage unit 160, accessible to users provided that they receive access authorization from the IMS 132.
  • a user may access the IMS 132 over a network 204 through the utilization of a terminal 212, which can be a personal computer, personal digital assistant, cell phone, interactive television, or the like.
  • the network 204 can be a local or wide area network, the Internet, or an intranet.
  • the user logs on by providing a user identifier and pass code.
  • the active server page/fusion module 134 After the log in has been completed and appropriately acknowledged by the IMS 132, the active server page/fusion module 134 generates a web page which will appear in a browser on the user terminal's screen 302, as illustrated in Figure 3.
  • the web page is configured specifically for this particular user and will identify for the user, depending upon his position, the various files to which the user is entitled to have access.
  • This web page is built in conjunction with the JAVA. generator 138 which generates appropriate JavaScript.
  • a web page provides the ability for the user to enter a specific string of information into a database search field or value field for the purpose of generating a search for a specific event or for a specific period of time. The user can initiate a search by clicking on the "find” button, cancel a search in progress by clicking on the "cancel” button, initiate a new search by clicking on the "new search” button, add criteria by clicking on the "add criteria” button. The user can also specify as to whether subfolders are to be searched by clicking on the "recurse" field.
  • the query can include a Boolean combination of one or more of an incident or case identification code, the identity (for example, name or employee identification code) of a person or officer responding to a dispatch, the name of a person reporting an incident, the name of a person arrested, cited, or questioned during an incident, the police vehicle number, the video camera identification number, the date and/or time of an incident, the date and/or time of a video recording, an event, such as a weapon removal from a rack, a light bar turning on/off, a siren turning on/off, a vehicle speed, a vehicle door opening/closing, a gun discharge, an alarm signal, a crash signal, geographical location, GPS geographical position coordinates, and the like.
  • an incident or case identification code the identity (for example, name or employee identification code) of a person or officer responding to a dispatch, the name of a person reporting an incident, the name of a person arrested, cited, or questioned during an incident, the police vehicle number, the video camera identification number, the
  • the user can request or search for files associated with a particular vehicle that were recorded in response to the vehicle light bar being turned on in order to locate the beginning of all situations in which the officer issued a traffic ticket, hi another example, in order to determine if an office was driving unnecessarily fast, the user can request files wherein a police vehicle driven by a named officer exceeded 65 miles/hour when the vehicle light bar was not turned on.
  • the user can request files associated with a specific incident number to locate videos and meta-data associated with the incident, even when the video and meta-data was recorded by different recorders, such as recorders in all police vehicles assigned to respond to the incident.
  • the user can request files associated a specific incident number that were originally recorded within 1 hour after an officer responded to a dispatch.
  • the user can request files associated with the swiping or entering a driver's license number taken during a traffic stop in order to locate video for a particular citation incident.
  • the user will be provided a list of files which matches the search string and which the user is authorized to access.
  • the list can be sorted by time of date, time, size, name, incident code, officer name, name of person reporting an incident, event, and/or on other fields, including customizable fields.
  • the user can specify one or more search properties, one or more operators, including Boolean and other operators (such as AND, OR, XOR, NOT, exact match, NEAR), values, and the like.
  • the particular file of interest may not be stored in the IMS 132, and is accessed from the storage device 160 coupled to the assembler 120 or alternatively the RAID storage system 165 associated with a corresponding digital recorder transmitter unit 169.
  • Access to video files and data can be restricted and controlled.
  • the video access controller (VAC) 124 when the user clicks on a link corresponding to a particular video file of interest, the video access controller (VAC) 124 generates a certificate in the form of an ultra long alphanumeric string, such as a 64, 128, or 256 bit string, which can be arbitrary in nature. That certificate is transmitted as shown at 208 to the user terminal 212 along with a command to redirect the user's browser to the particular file which resides on the storage system 160, for example, as is shown at 210. Substantially simultaneously, the video access controller (VAC) 124 will transmit the identical certificate directly to the archive manager or RAID system 160 as shown at 206.
  • the certificates, transmitted by the video access controller 124 as shown at 206 and redirected by the computer 212 as shown at 210, are then matched to ascertain identical confirmation thereof, hi the event that a match is obtained within a predetermined of time such as a very short period of time (seconds), access to the selected file is then granted and the file is forwarded to the computer 212 and appears on the screen 302 as illustrated graphically at 304.
  • a predetermined of time such as a very short period of time (seconds)
  • access to the selected file is then granted and the file is forwarded to the computer 212 and appears on the screen 302 as illustrated graphically at 304.
  • the video associated with the selected file is displayed, as well as the synchronized meta-data that was recorded and embedded in the video file.
  • a recorded event occurs, such as the light bar being turned on
  • the user will see that discrete recorded event happening at a certain time during the video.
  • the user desires to search for the light bar "on" event, the video will begin playing at the moment when the light bar is activated.
  • the user can search on other data or events, such as a drivers license number, which was either embedded in the video file at the time of the recording, or associated with the file during the data mining procedure after the comparator file was delivered to the IMS.
  • a user can search for the removal of a weapon from a weapon rack, activation of a siren, the vehicle reaching a specified speed, specified data communication transmitted to police car terminal, that were either recorded at the time of the creation of the record, or associated with the video during the data mining procedure.
  • the user may control what appears on the screen. For example, the user may play the whole video in full screen.
  • multiple frames or windows can be provided.
  • a GPS map may be generated and displayed, as well as terminal traffic (including data entered by the officer or other user), speedometer readings, date and time, and like in corresponding windows or frames.
  • Other displayed information can show the status of various other events, such as the status of firearms, seat belts, and the like. The foregoing information optionally will appear scrolling up synchronized to the video/audio, map location, speedometer reading, etc. If the recorded video did not have a data terminal, then that information would not be displayed.
  • the page gets reconfigured automatically and so what is displayed is something that is size appropriate reconstructed in substantially real time because it is generated for the specific user. So now the video is larger and the telemetry data is repositioned to whatever is most convenient.
  • the ASP/fusion generator module 134 automatically determines the best or preferred way to display the desired file.
  • the following process is optionally performed.
  • the connection is immediately terminated.
  • the period of time within which this match of the ultra long alphanumeric string needs to occur can be any time desired for the particular application, such as for example, a few seconds or a few minutes. If the match does not occur within the specific period of time, then the user is optionally required to again log on to the IMS 132, establish the secure connection, enter the pass code and again request the file as described above.
  • the user may control viewing of the selected video by going forward, backward, stop framing or the like as long as the socket remains open.
  • the user will have no overwrite capabilities of any type and may only view the video.
  • the URL for the media server for that particular storage media is provided by the video access control (VAC) and is contained in the redirect 210 generated by the video access controller 124 and transmitted to the user's computer 212.
  • the video is streamed to the user's terminal or computer 212, and so television-quality video playback can start very quickly, such as within seconds or faster, without having to wait for files to transfer to the local user's machine.
  • the entire file can be transferred to the local user terminal 212 before playback begins, though this may be a more time consuming process.
  • the media storage may also include information which is obtained from alien databases obtained and associated with the specific file during a data mining procedure as described below.
  • Alien databases are databases controlled or existing outside of the information management system 132.
  • a search is made of the alien or external databases to obtain information which is relevant to the information being generated from the video stored in the overall system. That information is automatically entered into the IMS relational database stored on the RAID 142 that is connected to the IMS 132, and optionally associated with corresponding video files. This information is then also available and will appear for utilization by the user on the web page generated and displayed on the terminal 212 when log in occurs and is authenticated.
  • the database searches can be performed in substantially real time by generating quires, or in batch mode.
  • the alien databases can include one or more of a DMV (department of motor vehicle records) database, a CAD database included in the CAD system 175, a database of police reports filed by police officers, a database of 911 emergency phone calls, a database of incident reports, and so on.
  • the CAD database can include information related to an incident or compliant being reported, such as the incident or case number, the date and time the incident was reported, the date, time and location of the incident, the type of incident, which resources were assigned to handle the incident, the status of those resources before during and after handling the incident, and related incident reports.
  • the IMS 132 when a record is being uploaded for storage in a database, the IMS 132 creates a second duplicate meta-data record and stores it in a file, and optionally on a different storage device at a removed location, separate from the original uploaded record.
  • the IMS 132 automatically compares the two stored meta-data records, including for example the objective time ticks, subjective time ticks, event data, serial data, and so on, to authenticate the record.
  • the file is generated by the assembler 120 and is used to transmit extracted data embedded by a recorder 100 into a video data chamiel of an archive file for automated data entry by the assembler keeper to the IMS relational database.
  • the file is used to authenticate an archived record by comparing subjective time tick proximity to data for file validation.
  • the file is further used to format the ASP user interface for concurrent data display of video and meta-data from multiple sources.
  • Lines 1-14 comprise the header information used to define constants, such as recorder identifier, date, time increments, and the like.
  • the other lines are used to specify information specific to a segment, such as the value or change in status of events, time tick values, and the like.
  • Line 1 of the example file format provides the version number and the encoding type.
  • UTF-8 is used, though other encoding types can be used as well.
  • UCS characters U+0000 to U+007F (ASCII) are encoded as bytes 0x00 to 0x7F (ASCII compatibility).
  • files and strings which contain 7-bit ASCII characters have the same encoding under both ASCII and UTF-8.
  • Line 2 is used to declare the namespace.
  • Line 3 is used to define the mobile video/audio data recorder file identifier.
  • Line 4 is used to define the mobile video/audio data recorder identifier.
  • Line 5 is used to specify the event date using an objective date, such as the Greenwich Mean Time (GMT) date.
  • GTT Greenwich Mean Time
  • Line 6 is used to specify an event start time using an objective time, such as GMT.
  • Line 7 is used to specify the time zone being used in lines 5 and 6.
  • the objective time zone can be GMT or Pacific standard time (PST).
  • Line 8 is used to specify the user label, which can be a easily remembered label, such a "Bank” recorder if the recorder is located in a bank, or "Intersection of Westwood and Wilshire” if the recorder is located on a post located at the intersection of Westwood and Wilshire.
  • Line 9 is used to specify the subjective duration
  • Line 10 is used to specify the file size.
  • Line 11 is used to specify the time increment used for the subjective time ticks.
  • Line 12 is used to specify the system label, such as the System ID which can indicate that the system is associated with a certain entity, such as the Los Angeles Police Department, California Highway Patrol, FBI, and so on.
  • Line 13 is used to specify the element list that assigns channels to data sources. For example, GPS coordinates can be assigned to channel 1, and terminal traffic can be assigned to terminal 2. This information can also be stored in a unit configuration file in the IMS database.
  • Line 14 is used to specify the video format, such as MPEG 1, MPEG 2, MPEG 4, H.323 encoding.
  • Line 15 is used to as compositor to define the virtual data channel sequence.
  • the sequence provides an efficient way of delineating what the ASM keeper is about to parse.
  • Line 16 is used to specify the current subjective time tick for the corresponding segment.
  • Line 17 is used to specify state change values for labels assigned to the mobile video/audio data recorder digital IO ports corresponding to a given application.
  • Line 18 is used to specify the serial data values for labels assigned to the mobile video/audio data recorder serial data or network IO ports corresponding to a given application.
  • Line 19 is used to specify the objective data and time for the corresponding segment.
  • Lines 20-31 are used to specify the subjective time ticks, state change values, serial data values and objective dates and times for additional segments.
  • each of the video streams A, B and C have an objective time recorded thereon as shown at 404, 412 and 416.
  • the time indicators 404, 412 and 416 are each objective time.
  • the objective time can be Greenwich Mean Time (GMT), sometimes referred to as ZULU time, though other time zones can be used as well.
  • GMT Greenwich Mean Time
  • the video stream A and the video stream B may be synchronized so that the objective times 404 and 412 are identical or substantially identical, as well as having the video stream C also synchronized so that the events occurring on each of the video streams A, B and C are all occurring based upon commencement at the objective times 404, 412 and 416.
  • each of the video streams will have a subjective time marking as shown at 406, 410 and 414, respectively, on the video streams A, B and C. Each of these subjective time markings are relative to the duration of the specified video file.
  • the ticks of the clock are recorded for each video frame or picture within the file.
  • the user may then be able to correspond the events being depicted on each of the video steams A, B and C with the local time at the particular venue for purposes of analysis, comparison to other events, or the like.
  • This ability to synchronize is provided using a UTBC (Universal Time Base Controller) included in the assembler 120.
  • Figure 5 illustrates an example video sequence 500 which includes video 502, a first audio channel 504, a second audio channel 506, and meta-data 508.
  • the variable vA is the bit rate variable, which is related to the number of frames/second and the image quality.
  • the variable vB is the audio bit rate, which can relate to bit depth, how many audio channels are being recorded, the dynamic range.
  • the variable vC is the meta-data bandwidth which is related to maximum bit-per second of aggregate channels.
  • a recorder such as recorder 100, can be constantly capturing data 510 in a buffer, such as a ring buffer or FIFO, until an instruction or start event starts the recording. The duration of the buffer is vl. The recorder then records multiple segments, such as multiple segments 512, 514, 516, 518, 520 of duration v2.
  • the time v2 can be specified by the user and/or the system administrator.
  • Figure 6 illustrates the conversion of the segments 512-520 illustrated in Figure 5 into an assembled file 602 as assembled or merged by the assembler.
  • Figure 7 illustrates the process of converting assembled video/audio and meta-data files into a Web page for display on a user terminal.
  • the meta-data in file 602 is packetized for transmission over the meta-channel 710 using an address header 704 and a barer channel variable length packet 706, wherein the meta-data can include channels corresponding to multiple datasources.
  • Figure 9 illustrates an example process of accessing and rendering video streams.
  • a user accesses the IMS. As similarly described above, the user may need to log on and provide a user identifier and password.
  • a determination is made as to what files the user is authorized to view, and the list of authorized files are sorted and transmitted to the user terminal. The list can be sorted by time of day, date, size, event, or on other fields, including customizable fields.
  • the ASP downloads the selected file or files.
  • the frame rate, bit rate and resolution for transmission can be selected based on the wired or wireless network being used and/or on the receiving terminal resolution and frame rate rending capability.
  • data including video, audio, and meta-data may be transmitted from more than one source, such as the IMS and/or ASM.
  • the IMS may transit a GPS map and the ASM can transmit GPS coordinates that was recorded by a vehicle in synchronization with video taken by a video camera mounted in the vehicle.
  • the data in rendered on the user terminal screen, such as in the user's browser.
  • Figure 10 illustrates an example encoding process for live source transmission and recording from up to four different simultaneous streams generated from a single camera source, though fewer or more cameras can be used and fewer or more streams can be transmitted and recorded.
  • the example encoding process optionally uses a dedicated processor for MPEG 1, MPEG 2, MPEG 4, CIF, D-l, and High Definition, as well as using real-time software-based encoders that substantially simultaneously encode at various bitrates/frame rates/frame sizes for playback or viewing on a plurality of different user devices such as PDAs, notebooks, and desktop computers.
  • the "Stream controller” is a infinite pin tee that replicates video/audio data from a single sample source device to enable multiple encoders to simultaneously access different memory addresses for the same data stream.
  • this is also a software driver "wrapper" used to "spoof the functionality of a device driver for each of the replicated signal sources.
  • Each of the encoded video streams may be recorded, formatted for RTP/RTSP, while any or all of them are simultaneously available for live transmission . It should be understood that certain variations and modifications of this invention would suggest themselves to one of ordinary skill in the art. The scope of the present invention is not to be limited by the illustrations or the foregoing descriptions thereof.

Abstract

The present invention is related to methods and systems for acquiring, storing, and distributing video and telemetry. Methods and systems are provided for the comprehensive management of audio, video and data recorded by one or more sensors (100), such as surveillance sensors. Users can select and play back video records based on meta-data embedded within the video record. The frame rate and resolution of the video can be customized to be appropriate for the user terminal.

Description

VIDEOANDTELEMETRYAPPARATUS AND METHODS
Background of the Invention Field of the Invention The present invention is related to video and telemetry, and in particular to methods and systems for acquiring, storing, and distributing video and telemetry information. Description of the Related Art
Conventional video surveillance systems often suffer from many deficiencies. For example, many conventional video surveillance systems fail to deliver television-quality streaming video with synchronized audio. Instead, many systems use a technique called motion JPEG or Wavelet "push" technology. Disadvantageous!/, motion JPEG and Wavelet systems are primarily designed to provide a series of individual "photographs" from a video camera source. This process of transporting individual pictures is sometimes called "Serial Still Frame" transport, or "SSF." SSF does not adequately synchronize audio and other data. Further, the bandwidth needed for SSF transmission can be relatively high, as with conventional systems each individual picture is separately encoded, recorded and transmitted.
Further, once the image data is captured, it can be very difficult to find a particular frame or sequence of frames without having to manually search by viewing large number of the frames to locate the desired frame or sequence of frames.
In addition, many conventional systems do not provide tamper-proof archiving technologies to ensure that archived data is not later improperly modified or edited. For example, some conventional video archiving systems rely on watermarks embedded within the video frames during the recording process to detect tampering. This method however, can often be circumvented, and so these systems often do not have adequate evidentiary credibility.
Summary of the Invention
The present invention is related to methods and systems for acquiring, storing, and distributing video and telemetry. Example embodiments further provide methods and systems for the comprehensive management of audio, video and data recorded by one or more sensors. Thus, embodiments of the present invention enables users to select and play back video records based on meta-data embedded within the video record. One embodiment is a recording and transmission system comprising: a recorder mountable in a police vehicle configured to record video and meta-data synchronized with the video, wherein the recorded video includes a plurality of segments having fixed lengths, the recorder including a wireless transmitter configured to wirelessly transmit the recorded video segments and meta-data, wherein the meta-data includes: data related to vehicle status; subjective time information; objective time information; serial data. An assembler is configured to wirelessly receive the transmitted video segments and assemble the video segments into a file including the meta-data, the assembler further including a mass storage device used to store the assembled video and transmit the assembled video. An information management system is coupled to the assembler to receive the assembled video and to store the meta-data in a relational database, wherein the information management system is configured to receive a user search query from a first terminal having a first resolution, conduct a search of the meta-data based at least in part on the search query, transmit the results of the search to the first terminal, receive a user selection of at least a first of the results, configure the video corresponding to the selected result for the first terminal based at least in part on the first resolution, and transmit the configured video to the first terminal. Another embodiment is a method of storing and providing video data, the method comprising: receiving a first video file including video and embedded meta-data, the metadata including at least timing information and status information; storing copies of at least a portion of the first video file in a second video file on a first data storage system and in a third file on a second storage system; generating at least a first string searchable file based at least in part on the meta-data; receiving a user search query; conducting a search of information in the first string searchable file based at least in part on the user search query; presenting results of the search to the user via a user terminal; receiving a user search results selection corresponding to the second video file; retrieving the second video file and the third video file based at least in part on the user selection; comparing at least a first portion of the meta-data in the second video file with a corresponding portion in the third video file in order to determine if file tampering has occurred; and transmitting at least a portion of the second video file to the user terminal. One embodiment is a method of storing and providing video data, the method comprising: receiving and storing data including a video sequence and synchronized metadata; generating at least a first string searchable data structure based at least in part on the meta-data; receiving a user search query; and conducting a search of information in the first string searchable data structure based at least in part on the user search query.
One embodiment is an apparatus for storing and providing video data, the apparatus comprising: a storage device configured to store data including a video sequence and synchronized meta-data; a first instruction configured to generate at least a first string searchable data structure based at least in part on the meta-data; a second instruction receive a user search query; and a third instruction configured to conduct a search of information in the first string searchable data structure based at least in part on the user search query.
Brief Description of the Drawings Figure 1 illustrates an example video acquisition, storage and distribution system. Figure 2 illustrates an example authentication process and system. Figure 3 illustrates an example Web page used to display a retrieved video. Figures 4A, 4B and 4C illustrate example video files with embedded timing information.
Figure 5 illustrates example video segments. Figure 6 illustrates an example assembler file conversion process. Figure 7 illustrates an example of merging data from multiple sources for rendering on a user terminal.
Figure 8 illustrates an example search page.
Figure 9 illustrates an example file access, download and rendering process.
Figure 10 illustrates an example encoding process.
Detailed Description of the Preferred Embodiment The present invention is related to methods and systems for acquiring, storing, and distributing video and telemetry. Example embodiments further provide video archive management that enables users to select and play back video records based on meta-data embedded within the video record.
For example, when used in a law enforcement patrol vehicle scenario, the meta-data may include, by way of example, date, time, vehicle identification, personnel names, global positioning coordinates, and a comprehensive variety of additional data, including mobile data terminal traffic, event logs or intrusion alarms. As discussed in greater detail below, automated upload of video files onto archive storage media and automatic entry into a corresponding database is also provided, optionally with little to no user intervention.
For example, in the law enforcement patrol vehicle scenario, in one embodiment, files, including video, audio, and meta-data, are temporarily stored onboard vehicle-based recording units which are transmitted to an archive database when the vehicle returns to its home station. The meta-data is optionally stored both in the home station archive database, as well as on a remote centralized database that stores meta-data from a plurality of station archive databases.
A variety of electronic sensors and associated electronics and software, that encode, transmit, and record video, audio, and many channels of streaming real-time meta-data, such as telemetry and event information, can be included in an embodiment of a surveillance system to detect a variety of events or occurrences. The meta-data is embedded and synchronized with the recorded video. The sensors may be positioned on mobile platforms, such as cars, trains, or planes, or on stationary platforms, such as buildings, posts, fences, and the like. The sensors can include, by way of example, one or more of pan/tilt/zoom, passive and active infrared and visible light cameras, covert wireless cameras and microphones, glass breakage detectors, door open/close detectors, gun discharge detectors, crash detectors, light bar on/off inputs, siren on/off inputs, gun rack detectors that sense when a gun is removed from and/or placed in the rack, vehicle speed sensors, tachometer, radar gun, fuel level, heartbeat monitors, pulse monitors, chemical weapons detectors, GPS systems, radio receive and transmit switches, public address ports, and the like.
Advantageously, the sensors and supporting circuitry are optionally capable of simultaneously transmitting live (low latency) video, audio, and data telemetry wirelessly over wireless networks while recording, and further provide data terminal functionality. In one embodiment, a variety of server components are used to cost effectively and securely deliver video, audio, and data telemetry, such as that recorded via the sensors and supporting circuitry, to long-term storage archives, optionally in accordance with Federal Evidentiary Rules. Upon receiving the recorded files that include video, audio, and data telemetry, an example apparatus automatically extracts and parses pre-determined data-points from the recorded telemetry to construct a scalable video meta-data "comparator" file that is used by a central database to enter the data to fields for sorting; seek extract, and include selected data and information from other databases related to the recording for central access; and provide for string-searches of meta-data embedded in videos. For example, in response to user string searches and using the meta-data entered into the meta-data database, specific videos and even specific segments or frames within a video sequence are located and retrieved from the video archives. The located videos can be substantially instantly delivered to the requesting user starting from the moment of a selected meta-data "incident" or data-points, or beginning at a selected time period relative to the incident or data point. Thus, use of an embodiment of the present invention can result in the reduction or elimination in the need to spend time manually locating and retrieving video records, or manually viewing such records in order to locate specific events.
Further, the system advantageously optionally provides secure user access locally and/or remotely, optionally without needing customized client display software on the user computer or terminal. For example, access to and display of videos and meta-data can be provided using a standard browser, such as Microsoft Internet Explorer or the like, and a standard media player, such as Windows Media Player or Apple QuickTime, executing on a personal or server computer, personal digital assistant, cell phone, and the like, thereby lowering the client cost and easing maintenance. However, custom media players and browsers can be used as well. In addition, video teleconferencing interfaces, such as the standard H.323 teleconferencing, is provided for remote observers. By optionally using an existing teleconferencing standard the system is compatible with legacy video teleconferencing systems. Optionally, the system and applicable sensors can utilize one or more of ISO MPEG-4, MPEG-1, MPEG-2, and H.323 encoding/decoding. Optionally, for certain application, the system employs extremely compressed video to reduce bandwidth and storage requirements. For example, in one embodiment, an I (intracoded) frame is only transmitted when at least a certain number or percentage, such as 25% of pixels change luminance and/or chrominance relative to the previous intracoded frame, otherwise P (predicted) frames or B (bi-directionally interpolated) frames are used. As is well known in the field, I frames generally utilize many more bits than P or B frames. Optionally, this process allows six frames/second to be transmitted using as little as about 5.9 kbs, and at a nominal transmission rate of 7.2kbps, which is well within the capacity of existing cellular data transmission services.
One embodiment advantageously utilizes asymmetrical encoding which provides a multi-session digital video compression process that enables multiple simultaneous transmissions of various video resolutions and frames rates while a high quality recording is created or played. This process permits various users utilizing dissimilar devices to view live and/or prerecorded video transmissions appropriate for the type of viewing device in use.
The foregoing methods and systems will be described in greater detail below with reference to the figures. Throughout the following description, the term "Web site" is used to refer to a user-accessible server site that implements the basic World Wide Web standards for the coding and transmission of hypertextual documents. These standards currently include HTML (the Hypertext Markup Language) and HTTP (the Hypertext Transfer Protocol). In addition, reference is made to Java script (also referred to as JavaScript), though other types of script, programming languages, and code can be used as well. For example, VBScript or Active X objects can be used with appropriate platforms. It should be understood that the term "site" is not intended to imply a single geographic location, as a Web or other network site can, for example, include multiple geographically distributed computer systems that are appropriately linked together. Furthermore, while the following description relates to an embodiment utilizing the Internet and related protocols, other networks, such as networked interactive televisions, and other protocols may be used as well.
In addition, unless otherwise indicated, the functions described herein are preferably performed by executable code and instructions running on one or more general-purpose computers, terminals, personal digital assistants, cellular phones, or the like. However, the present invention can also be implemented using special purpose computers, state machines, and/or hardwired electronic circuits. The example processes described herein do not necessarily have to be performed in the described sequence, and not all states have to be reached or performed. Further, while the following description may refer to "clicking on" a link or button, or pressing a key in order to provide a command or make a selection, the commands or selections can also be made using other input techniques, such as using voice input, pen input, mousing or hovering over an input area, and/or the like.
The overall system is shown in Figure 1 to which reference is hereby made. The system includes a mobile video/audio data recorder 100 which records high quality video/audio and serial data telemetry while simultaneously transmitting specially formatted live video/audio and information to personnel using a variety of reception devices or terminals. For example, the mobile video/audio data recorder 100 optionally records at 30 frames per second, though other frame rates can be used as well. Optionally, the mobile video/audio data recorder 100 utilizes RTSP (real time streaming protocol), a client-server multimedia presentation control protocol, which provides efficient delivery of streamed multimedia over IP networks. The RTSP may utilize RTP (Realtime Transport Protocol), which is a packet format for multimedia data streams.
The mobile video/audio data recorder 100 optionally saves video files in multiple file segments, the length and/or duration of which is selectable by a user or system administrator. For example, if segment sizes of 300 seconds are selected with a recording bit-rate of 800kbits per second, mobile video/audio data recorder 100 will break long duration recordings into files of approximately 30Mbytes each. The foregoing process advantageously provides enhanced file resiliency, thereby reducing the incidence of file corruption, and facilitating the recovery of damaged or corrupted files by limiting the size of each binary recording. The recorder 100 can transmit files, including video records, audio records, and/or meta-data, using one or more of CDPD, GPRS, GSM, satellite, dial- up, or broadband Internet networks.
Optionally the frame rate, bit rate and resolution for transmission can be selected based on the wired or wireless network and the receiving terminal. For example, some devices, such as palm-held personal digital assistants (PDAs) and hybrid cell phones may have limited display resolution and processor power, hi applications where such devices are used to view the video, such as by remote field persomiel performing surveillance, the recorder 100 maintains compatibility with the target device by optionally formatting video specifically for the target device. By way of example, personnel equipped with a PDA that may only be capable of displaying 15 video frames (pictures) per second at a resolution of 320x240 pixels connect to a special URL (Universal Resource Location) associated with a Web site, and are provided video specifically formatted for the receiving device. At the same time, when video and data streams to devices that can display full resolution and frame rates, such as notebooks and PCs, the devices receive full resolution frames at the recorded frame rate.
Thus, for example, live video can selectively be transmitted at one or more of the following video frame/bit rates, while at the same time optionally recording at a minimum frame rate and resolution, such as 30 FPS at a minimum resolution of 352 x 240 with 48KHz audio:
6 FPS (frames per second) with a resolution of 160 x 120 utilizing less than 8 kilobits per second via CDPD, GSM, GPRS, 802.1 lg, 802.11b, 802.11a, or Ethernet 10/100.
15 FPS with a resolution of 320 x 240 at 47 kilobits per second via GPRS, GSM, 802.1 lg, 802.11b, 802.11a, or Ethernet 10/100.
30 FPS with a resolution of 352 x 240 at 192 to 600 kilobits per second via 802.1 lg, 802.11b, 802.11a, or Ethernet 10/100. The mobile video/audio data recorder 100 can be located in and transmit from a variety of stationary and mobile locations, such as aircraft, ships, patrol vehicles, buildings, lots, fields and the like. i one example embodiment, a ring buffer in a recorder, such as recorder 100, captures video, audio and data until an instruction or trigger to start the recording process occurs. When the instruction or trigger is received, the content of the ring buffer is assigned a new serial record number. A file is created in the system's local temporary archive with a file name including the serial record number, system identifier, and the user assigned file naming convention is appended. The start instruction can be the result of any particular event, such as a patrol vehicle light bar being turned on, or as a result of a user providing a manual signal start instruction. Files, including video, audio, and meta-data, are then recorded and closed in optionally fixed duration intervals. The file intervals can be specified by the user or administrator during system set-up. As discussed above, the files can include embedded video, audio, and meta-data. Meta-data is recorded as virtual channels utilizing a packet-based addressing scheme, enabling a large number of virtual channels, such as up to 256 or more virtual channels. The meta-data can optionally correspond to inputs from one or more of the sensors and detectors discussed above. As each file is closed, a buffer, such as the ring buffer, captures the data for the next file segment so that data is not lost. Advantageously, the data, video, and audio sources and encoders can remain active during file close intervals.
The files are stored in an archive clearing directory, and are serialized and identified by a record event string in preparation for automated file upload to an assembler server system 120.
The video/audio data signals recorded by the mobile video/audio data recorder 100 are uploaded either wirelessly or via a wired connection to the assembler server system 120. By way of further example, if the mobile video/audio data recorder 100 is located in a police or other vehicle, the recorder 100 can automatically upload the recorded video/audio and data using 802.11(b, g, or a) protocol upon reaching the police station at which the vehicle is based. The data transmitted with the video/audio can, for example, include the date and time of recorded segments, together with organization and sequencing information, as well as digital serial data and event data. This data may also include police vehicle number, shift, time, personnel designations, and other fields, as desired. The date and time information can be embedded in the video file with high accuracy, such as, in one embodiment, to within 1/30 of a second accuracy.
Advantageously, because the date and time are recorded as meta-data, the date and time information does not have to be recorded in the actual visible video frame. Thus, upon viewing or playback, the date and time do not obstruct information in the video frame, and instead are optionally displayed in a separate window.
The serial data, for example, can include records of data communications of an external remote data terminal, alarm data from an external security sensor system, or other data from other serial interfaces, such as an RS-232 or IP interface. The serial data can be embedded in the video stream and recorded in the video file to within 1/30 of a second accuracy, though greater or less accuracy can be used in other embodiments, hi one embodiment, up to 256 external data sources can be simultaneously supported, though in other embodiments more than 256 data sources can be recorded, h addition to serial data, parallel data can be recorded as well. In one embodiment, parallel data is first serialized and then recorded as serial data. Advantageously, because the data from any virtual channel are recorded as meta-data, the data and/or information derived therefrom does not have to be recorded in the actual visible video frame. Thus, upon viewing or playback, the data and/or information do not obstruct information in the video frame, and instead are optionally displayed in a separate window.
One embodiment of the recorder 100 provides event-based recording that enables users to activate a recording session by either pressing a button or activating another device used as an event trigger such as a vehicle light bar, or an alarm contact or other sensor. In cases where a recording is already in progress, a digital "flag" is created and embedded that enables a viewer to find the event during playback. Recording may optionally be started remotely by network connection, either manually, or automatically using a plurality of predetermined external events, parameters, or triggers. Optionally, a record of the location of a vehicle, such as a patrol vehicle or aircraft, can be embedded in the video file. For example, the GPS data can be provided by a GPS system within the vehicle and embedded in the same fashion as serial data. hi particular, when a recorder 100 comes into proximity with its designated home network, using, by way of example, an 802.11 wireless network, the assembler 120 recognizes each recorder 100, and interrogates the recorder 100 for new records or files. If the recorder 100 contains new recordings, the assembler 120 automatically commences the file upload process. Once the files have been transferred and verified, the assembler 120 notifies the recorder 100 that the file upload is complete, and instructs the recorder 100 to clear its record volume for new recordings. The assembler 120 optionally returns the recorder 100 to normal operations mode, or shuts the recorder 100 down.
The assembler 120 can, for example, be located in a facility with which the recorder 100 is likely to come within close proximity with on a regular basis, such as a station house. This allows a relatively short-range wireless network, such as 802.1 lg, to be used to communicate information and video between the recorder and the assembler 120. The assembler 120 is coupled to a media server 123, such as Microsoft Media
Server, which is coupled to a video access controller (VAC) 124, which is coupled to an archive controller 126. The archive controller 126 is coupled to mass storage units 160, 162.
The assembler server 120 optionally "assembles" associated files into a one cohesive and user-playable video recording. The assembler extracts and parses meta-data from the meta-data channels of the newly archived video file. A string searchable format file, such as an XML document, is then created by an XML generator 128 and associated with the video recording, parsed by information type, to enable "string" searches of the data embedded with the video and audio. The searchable document is stored in an information management system (IMS) 132. In one example embodiment, the assembler 120 combines related files into a single Internet streamable file containing the original data while preserving encoded RTP/RTSP streaming protocols embedded in the files by the recorder 100.
The video/audio stream is then transmitted, either immediately, or by predetermined schedule, by the assembler 120 to an appropriate video-on-demand database or storage media, such as the storage units 160 and 162 controlled by the archive controller 126. The storage unit 160 can be implemented as a RAID (redundant array of inexpensive disks) system providing high speed storage and retrieval, while the storage unit 162 can be implemented as a plurality of high capacity optical digital data storage disks providing relatively more capacity at lower overall long-term storage cost than the storage unit 160. The storage unit 162 provides a more permanent record of video stored in the RAID system 160, however, the write/read time may be slower than that provided by RAID system 160. Optionally, the storage unit 162 may be in the form of an optical WORM (write once, read many) drive or carrousel, including DVD disk or disks, the use of which prevents the editing of recorded files.
The system may include one or more digital recorder transmitter units 169, which can include or be connected to a plurality of mobile video/audio data recorders. For example, the digital recorder transmitter unit 169 can optionally record on internal magnetic or optical media 165 large amounts of video, such as one year or more of GIF video from, each of several cameras at relatively high frame rates. For example, one embodiment records at 30 frames per second (fps) for 160 video cameras at GIF simultaneously, with multiple audio channels, though other frame rates, resolutions, and quantity of video cameras can be used as well. The digital recorder transmitter unit 169 can further record one or more audio sources and multiple streaming data sources, such as data from alarm panels and machine controls. The system can also include one or more encoder systems 172, which is an MPEG multichannel encoder coupled to a plurality cameras transmitting live video, coupled to assembler keeper 139 via (Video Access System) VAS 170.
The digital recorder transmitter unit 169 also includes an omniport which includes or acts as a television tuner card. The omniport acts as a capture device so that the digital recorder transmitter unit 169 can be tuned to receive CCTV cable channels. This provides the capability for frequency division multiplexing to view desired images. This capability enables the user of the digital recorder transmitter unit 169 to capture the video/audio images that may be generated in such a closed circuit television system of the type often utilized for security purposes in various facilities.
The omniport interfaces into existing frequency modulation CCTV signals. This may include a number of different cameras, for example 160 or more cameras, the images of which are substantially simultaneously transmitted within a particular facility. The user of the digital recorder transmitter unit 169 may then select the desired channel in addition to being able to record images from other cameras. One such specific application where security is involved is on trains.
Trains or other vehicles traveling through various geographic regions will encounter communication and other device frequencies that could interfere with- the wireless transmission of video/audio data signals. Under such circumstances, the video/audio data signals in analog format are frequency modulated and then are applied to the AC or DC electrical network utilized within the trains or other like vehicles. This frequency modulated signal may then be acquired through the omniport resident on the digital recorder transmitter unit structure in much the same manner as acquisition of a frequency modulated CCTV signal as described above. The digital recorder transmitter unit 169 also includes an assembler server system
163, a VAC 164, a media server 166, and an archive controller 168, which functions in the same manner as described above with respect to the assembler server system 120. The output from the digital recorder transmitter units 169 may then be transmitted to the assembler keeper 139 as was the XML signal from the assembler 120. The video/audio signal is then transmitted directly to a storage medium, such as the RAID system 165. As previously discussed with regard to the mobile video/audio data recorder 100, the meta-data and other identifying information regarding the video/audio is transmitted to the IMS 132.
With regard to both the mobile video/audio data recorder 100 and the digital recorder transmitter units 169, the archive controllers 126 and 168 respectively control the recording of the video in the RAID systems 160 and 165. Optionally, the storage unit 162 stores an exact copy of at least a portion of the meta-data and/or video on more permanent storage, such as on a optical media, of selected files recorded in the RAID system 160 for authentication purposes, as discussed below, and for backup purposes. Further, having the recorded copy can be kept at a site geographically remote, such as more than 150 meters distant, from the RAID system 160 provides for increased security.
The IMS 132 will now be discussed. The IMS 132 includes one or more relational databases, such as Oracle 9i or lOg, an ASP (Active Server Page)/Fusion module 134, a Java generator 138, an assembler keeper 139, and a computer aided dispatch (CAD) keeper 140. The CAD keeper 140 is further coupled to a CAD system 175.
The ASP/Fusion module 134 is used to generate pages, such as HTML pages, that include one or more scripts, such as JavaScript or VBScript. The module 134 processes the scripts on a Web server associated with the module 134 to customize the pages for the user and to provide transaction security. The customization can be based on information received from the user or the user terminal. By way of example, the information can be provided by way of a URL or address for an appropriate Web site page. The URL can be provided by the user browser when requesting a page to access data from one or more of the system databases. The ASP/Fusion module 134 can then build the requested page in substantially real time and send it, via a Web server, to the requesting user terminal for display in a browser or other application. By way of example, the ASP/Fusion module 134 can be implemented using Macromedia's ColdFusion product, including the ColdFusion Studio and the ColdFusion Server, though other applications and Web servers can be used as well.
The CAD system 175 can, for example, be used to process emergency or help calls from the public and handle police dispatches to police officers or police vehicles. Of course the CAD system 175 can also be used to handle dispatches from other entitites, such as fire department dispatches, hospital dispatches, and dispatches of other governmental or non-governmental entities. By way of further example, a CAD system 175 can be used to guide a call-taker or dispatcher into collecting relevant information, such as the address or location, related to an incident or compliant being reported, which the call-taker then enters into a form presented by the CAD system 175 on the call-taker's computer terminal. The information is optionally then stored in a CAD database. The CAD system 175 may check to make sure certain information, such as the address, is valid. The CAD system 175 or other entity may assign an incident or case number or code to the incident. Appropriate resources, such as one or more police cars, foot patrols, motorcycle officers, and/or ambulance units are then selected and dispatched to handle the incident. The CAD database also stores information on which resources were selected and dispatched to handle the incident. One or more status events for each resource may also be recorded in the database. For example, the status can include an acknowledgement from the resource that the dispatch was received. Another status can include a report from the resource that the resource has arrived at the scene of the incident. Still another status might relate to the resource leaving the scene.
In particular, the IMS 132 provides users with the ability to view a particular video record which has been made of a particular event in the past. The IMS 132 further enables the user to search for meta-data associated with archived videos, permits users to play back recordings at subjective times based on the meta-data and data field searches, and enables playback of specific records when selected by a user. Authorized users also have the ability to navigate and play back requested files on the storage system, provided that they have clearance to do so. There can be multiple levels of clearance, such as up to 30 or more levels of authorization that are used to determine user access to file types and specific data. The authorization can be centrally specified by the system administrator. The IMS 132 tracks user access to files and user activity on the IMS 132. Thus, for example, if a user chooses a file by going directly to the storage system to commence playback directly from archive, the user's activities are automatically logged. User's are optionally prevented from erasing archived files. Access to "housekeeping" functions are optionally reserved to the system administrator.
The IMS 132 extracts files created by the assembler 120 and automatically enters files to its database that are initially designated by the system administrator or other user during system set-up. Video files are stored in a separated storage volume, such as storage unit 160, accessible to users provided that they receive access authorization from the IMS 132.
For example, upon occasion an individual may desire to view a particular video record which has been made of a particular event in the past. With reference to Figure 2, when such is desirable, a user may access the IMS 132 over a network 204 through the utilization of a terminal 212, which can be a personal computer, personal digital assistant, cell phone, interactive television, or the like. The network 204 can be a local or wide area network, the Internet, or an intranet. When contact has been made between the user's terminal 212 and the IMS 132, the user logs on by providing a user identifier and pass code. After the log in has been completed and appropriately acknowledged by the IMS 132, the active server page/fusion module 134 generates a web page which will appear in a browser on the user terminal's screen 302, as illustrated in Figure 3.
Optionally, the web page is configured specifically for this particular user and will identify for the user, depending upon his position, the various files to which the user is entitled to have access. This web page is built in conjunction with the JAVA. generator 138 which generates appropriate JavaScript. A web page, an example of which is illustrated in Figure 8, provides the ability for the user to enter a specific string of information into a database search field or value field for the purpose of generating a search for a specific event or for a specific period of time. The user can initiate a search by clicking on the "find" button, cancel a search in progress by clicking on the "cancel" button, initiate a new search by clicking on the "new search" button, add criteria by clicking on the "add criteria" button. The user can also specify as to whether subfolders are to be searched by clicking on the "recurse" field.
The query can include a Boolean combination of one or more of an incident or case identification code, the identity (for example, name or employee identification code) of a person or officer responding to a dispatch, the name of a person reporting an incident, the name of a person arrested, cited, or questioned during an incident, the police vehicle number, the video camera identification number, the date and/or time of an incident, the date and/or time of a video recording, an event, such as a weapon removal from a rack, a light bar turning on/off, a siren turning on/off, a vehicle speed, a vehicle door opening/closing, a gun discharge, an alarm signal, a crash signal, geographical location, GPS geographical position coordinates, and the like.
For example, the user can request or search for files associated with a particular vehicle that were recorded in response to the vehicle light bar being turned on in order to locate the beginning of all situations in which the officer issued a traffic ticket, hi another example, in order to determine if an office was driving unnecessarily fast, the user can request files wherein a police vehicle driven by a named officer exceeded 65 miles/hour when the vehicle light bar was not turned on. Similarly, the user can request files associated with a specific incident number to locate videos and meta-data associated with the incident, even when the video and meta-data was recorded by different recorders, such as recorders in all police vehicles assigned to respond to the incident. By way of further example, the user can request files associated a specific incident number that were originally recorded within 1 hour after an officer responded to a dispatch. By way of yet another example, the user can request files associated with the swiping or entering a driver's license number taken during a traffic stop in order to locate video for a particular citation incident. hi response to the query, the user will be provided a list of files which matches the search string and which the user is authorized to access. The list can be sorted by time of date, time, size, name, incident code, officer name, name of person reporting an incident, event, and/or on other fields, including customizable fields. Optionally, the user can specify one or more search properties, one or more operators, including Boolean and other operators (such as AND, OR, XOR, NOT, exact match, NEAR), values, and the like.
As similarly discussed above, in one optional embodiment, the particular file of interest may not be stored in the IMS 132, and is accessed from the storage device 160 coupled to the assembler 120 or alternatively the RAID storage system 165 associated with a corresponding digital recorder transmitter unit 169.
Access to video files and data can be restricted and controlled. For example, when the user clicks on a link corresponding to a particular video file of interest, the video access controller (VAC) 124 generates a certificate in the form of an ultra long alphanumeric string, such as a 64, 128, or 256 bit string, which can be arbitrary in nature. That certificate is transmitted as shown at 208 to the user terminal 212 along with a command to redirect the user's browser to the particular file which resides on the storage system 160, for example, as is shown at 210. Substantially simultaneously, the video access controller (VAC) 124 will transmit the identical certificate directly to the archive manager or RAID system 160 as shown at 206. The certificates, transmitted by the video access controller 124 as shown at 206 and redirected by the computer 212 as shown at 210, are then matched to ascertain identical confirmation thereof, hi the event that a match is obtained within a predetermined of time such as a very short period of time (seconds), access to the selected file is then granted and the file is forwarded to the computer 212 and appears on the screen 302 as illustrated graphically at 304. Thus, the video associated with the selected file is displayed, as well as the synchronized meta-data that was recorded and embedded in the video file.
For example, if a recorded event occurs, such as the light bar being turned on, the user will see that discrete recorded event happening at a certain time during the video. If the user desires to search for the light bar "on" event, the video will begin playing at the moment when the light bar is activated. Similarly, the user can search on other data or events, such as a drivers license number, which was either embedded in the video file at the time of the recording, or associated with the file during the data mining procedure after the comparator file was delivered to the IMS. Likewise, a user can search for the removal of a weapon from a weapon rack, activation of a siren, the vehicle reaching a specified speed, specified data communication transmitted to police car terminal, that were either recorded at the time of the creation of the record, or associated with the video during the data mining procedure.
As the video plays, the user may control what appears on the screen. For example, the user may play the whole video in full screen. Alternatively, as illustrated by display 718 in Figure 7, multiple frames or windows can be provided. For example, a GPS map may be generated and displayed, as well as terminal traffic (including data entered by the officer or other user), speedometer readings, date and time, and like in corresponding windows or frames. Other displayed information can show the status of various other events, such as the status of firearms, seat belts, and the like. The foregoing information optionally will appear scrolling up synchronized to the video/audio, map location, speedometer reading, etc. If the recorded video did not have a data terminal, then that information would not be displayed. Under those circumstances, the page gets reconfigured automatically and so what is displayed is something that is size appropriate reconstructed in substantially real time because it is generated for the specific user. So now the video is larger and the telemetry data is repositioned to whatever is most convenient. The ASP/fusion generator module 134 automatically determines the best or preferred way to display the desired file.
If there is a failure to obtain the desired match up between the certificate directed by the VAC to the storage device 160 with the redirect from the computer 32 to the storage device 160, then the following process is optionally performed. The connection is immediately terminated. The period of time within which this match of the ultra long alphanumeric string needs to occur can be any time desired for the particular application, such as for example, a few seconds or a few minutes. If the match does not occur within the specific period of time, then the user is optionally required to again log on to the IMS 132, establish the secure connection, enter the pass code and again request the file as described above. If the system fails to obtain a match up of the alphanumeric string for a predetermined number of times, for example, three, then the user attempting to obtain access is optionally locked out of the system and will not be able to obtain access to the files desired without obtaining clearance through the system administrator. This process for video access control is referred to as a temporally extinguished certificate. It will be recognized by those skilled in the art that what has occurred is that when the user has established initial user authentication and when the temporally extinguished certificate matches within the specified period of time, then the particular file which the user has selected and particularly for the specific moment from which playback is to start appears on the screen as shown in 304 in Figure 3. Once access has been granted, then the user may control viewing of the selected video by going forward, backward, stop framing or the like as long as the socket remains open. Optionally, unless appropriately authorized by the appropriate supervisor or administrator, the user will have no overwrite capabilities of any type and may only view the video.
Because the user is requesting a specific video file identified on the web page which has been generated specifically for that user by the ASP/fusion module 134 and- JAVA generator 138, the URL for the media server for that particular storage media is provided by the video access control (VAC) and is contained in the redirect 210 generated by the video access controller 124 and transmitted to the user's computer 212. The video is streamed to the user's terminal or computer 212, and so television-quality video playback can start very quickly, such as within seconds or faster, without having to wait for files to transfer to the local user's machine. Optionally, however, the entire file can be transferred to the local user terminal 212 before playback begins, though this may be a more time consuming process.
The media storage may also include information which is obtained from alien databases obtained and associated with the specific file during a data mining procedure as described below. Alien databases are databases controlled or existing outside of the information management system 132. In one embodiment, through the utilization of the CAD keeper 140 a search is made of the alien or external databases to obtain information which is relevant to the information being generated from the video stored in the overall system. That information is automatically entered into the IMS relational database stored on the RAID 142 that is connected to the IMS 132, and optionally associated with corresponding video files. This information is then also available and will appear for utilization by the user on the web page generated and displayed on the terminal 212 when log in occurs and is authenticated.
The database searches can be performed in substantially real time by generating quires, or in batch mode. By way of example, the alien databases can include one or more of a DMV (department of motor vehicle records) database, a CAD database included in the CAD system 175, a database of police reports filed by police officers, a database of 911 emergency phone calls, a database of incident reports, and so on. By way of further example, as similarly discussed above with respect to CAD system 175, the CAD database can include information related to an incident or compliant being reported, such as the incident or case number, the date and time the incident was reported, the date, time and location of the incident, the type of incident, which resources were assigned to handle the incident, the status of those resources before during and after handling the incident, and related incident reports.
In some applications, such as for evidentiary purposes law enforcement, it can be important to ensure that archived video and meta-data has not been tampered with. Therefore, in one embodiment when a record is being uploaded for storage in a database, the IMS 132 creates a second duplicate meta-data record and stores it in a file, and optionally on a different storage device at a removed location, separate from the original uploaded record. When a user requests a record the IMS 132 automatically compares the two stored meta-data records, including for example the objective time ticks, subjective time ticks, event data, serial data, and so on, to authenticate the record. If the original digital record has been modified, such as by deleting or replacing one or more video frames, the duplicate meta-data record will not match, and the IMS will alert the user and/or system administrator of the match failure. Further, because most of the encoded, stored video frames are predicted or interpolated from other frames, the deletion of one or more frames can cause the other frames to visibly render properly, further indicating tampering. An XML compare file is provided below. The file is generated by the assembler 120 and is used to transmit extracted data embedded by a recorder 100 into a video data chamiel of an archive file for automated data entry by the assembler keeper to the IMS relational database. In addition, the file is used to authenticate an archived record by comparing subjective time tick proximity to data for file validation. The file is further used to format the ASP user interface for concurrent data display of video and meta-data from multiple sources.
1. <?xml version="1.0" encoding="UTF-8"?>
2. <xs:schema xmlns:xs= (ASM Comparator File) 3. <xs:element name- 'Mobile video/audio data recorder File ID" type- ' xs:string "/>
4. <xs:element name="Mobile video/audio data recorder ID" type- ' xs:string "/>
5. <xs:element name- 'Event Date Objective Value" type- ' xs:string "/>
6. <xs:element name- 'Event Start Objective Time" type="xs:string"/>
7. <xs:element name- 'Event Start Objective TimeZone" type="xs:string"/> 8. <xs:element name- 'User Label" type- ' xs:string"/>
9. <xs:element name- ' Subjective Duration" type="xs:duration"/>
10. <xs : element name- 'EndingValue" type="xs:fioat"/>
11. <xs:element name- 'Increment" type="xs:int"/>
12. <xs:element name- 'System Label" type- 'xs:string"/> 13. <xs:element name- 'Element List" type- ' xs:string .
14. <xs:element name- 'Video Format" type- ' xs:string "/>
15. </xs:sequence>
16. <xs:element ref="subjective time tick"/>
17. <xs:element name- 'State Change Name" type="xs:float"/> 18. <xs:element name="Serial Data Channel Name" type="String"/>
19. <xs:element name- ' Objective DateTime" type="String"/>
20. <xs:element ref=" subjective time tick"/>
21. <xs:element name- 'State Change Name" type="xs: float "/> (state change labels assigned to Mobile video/audio data recorder digital IO ports corresponding to application — any number of different elements to maximum available IOs)
22. <xs:element name- 'Serial Data Channel Name" type="String"/> (serial data labels assigned to Mobile video/audio data recorder serial data or network IO ports corresponding to application - any number of different elements to maximum available IOs) 23. <xs:element name- ' Objective DateTime" type="String"/>
24. <xs:element ref- ' subjective time tick"/>
25. <xs:element name- 'State Change Name" type="xs:float"/> (state change labels assigned to Mobile video/audio data recorder digital IO ports corresponding to application - any number of different elements to maximum available IOs) 26. <xs:element name- 'Serial Data Channel Name" type="String"/> (serial data labels assigned to Mobile video/audio data recorder serial data or network IO ports corresponding to application - any number of different elements to maximum available IOs) 27. <xs:element name- Objective DateTime" type="String"/> 28. <xs:element ref="subjective time tick"/> 29. <xs:element name- 'State Change Name" tyρe="xs:float"/> (state change labels assigned to Mobile video/audio data recorder digital IO ports corresponding to application - any number of different elements to maximum available IOs)
30. <xs:element name- 'Serial Data Channel Name" type- 'String'7> (serial data labels assigned to Mobile video/audio data recorder serial data or network IO ports corresponding to application - any number of different elements to maximum available IOs)
31. <xs:element name- 'Objective DateTime" type="String"/>
Lines 1-14 comprise the header information used to define constants, such as recorder identifier, date, time increments, and the like. The other lines are used to specify information specific to a segment, such as the value or change in status of events, time tick values, and the like.
Line 1 of the example file format provides the version number and the encoding type. In this example, UTF-8 is used, though other encoding types can be used as well. UCS characters U+0000 to U+007F (ASCII) are encoded as bytes 0x00 to 0x7F (ASCII compatibility). Advantageously, files and strings which contain 7-bit ASCII characters have the same encoding under both ASCII and UTF-8. Line 2 is used to declare the namespace. Line 3 is used to define the mobile video/audio data recorder file identifier.
Line 4 is used to define the mobile video/audio data recorder identifier. Line 5 is used to specify the event date using an objective date, such as the Greenwich Mean Time (GMT) date.
Line 6 is used to specify an event start time using an objective time, such as GMT. Line 7 is used to specify the time zone being used in lines 5 and 6. For example, the objective time zone can be GMT or Pacific standard time (PST).
Line 8 is used to specify the user label, which can be a easily remembered label, such a "Bank" recorder if the recorder is located in a bank, or "Intersection of Westwood and Wilshire" if the recorder is located on a post located at the intersection of Westwood and Wilshire.
Line 9 is used to specify the subjective duration
Line 10 is used to specify the file size.
Line 11 is used to specify the time increment used for the subjective time ticks. Line 12 is used to specify the system label, such as the System ID which can indicate that the system is associated with a certain entity, such as the Los Angeles Police Department, California Highway Patrol, FBI, and so on.
Line 13 is used to specify the element list that assigns channels to data sources. For example, GPS coordinates can be assigned to channel 1, and terminal traffic can be assigned to terminal 2. This information can also be stored in a unit configuration file in the IMS database.
Line 14 is used to specify the video format, such as MPEG 1, MPEG 2, MPEG 4, H.323 encoding. Line 15 is used to as compositor to define the virtual data channel sequence.
Because, in one embodiment, channels that contain no data are not transmitted, the sequence provides an efficient way of delineating what the ASM keeper is about to parse.
Line 16 is used to specify the current subjective time tick for the corresponding segment. Line 17 is used to specify state change values for labels assigned to the mobile video/audio data recorder digital IO ports corresponding to a given application.
Line 18 is used to specify the serial data values for labels assigned to the mobile video/audio data recorder serial data or network IO ports corresponding to a given application. Line 19 is used to specify the objective data and time for the corresponding segment.
Lines 20-31 are used to specify the subjective time ticks, state change values, serial data values and objective dates and times for additional segments.
As will be clearly understood from the foregoing description, the user may wish to simultaneously access different video files from the storage media. For example, as illustrated in Figure 4, three different video files labeled A, B, and C may be retrieved for simultaneous display on the user's screen. It will often be desired to synchronize the video streams A, B and C in order to assure that the events being viewed arc being viewed with relation to the exact time which they occurred even though the video streams A, B and C may have been generated by totally different cameras and, even in some instances, totally different parts of the world. To facilitate such synchronization, each of the video streams A, B and C have an objective time recorded thereon as shown at 404, 412 and 416. The time indicators 404, 412 and 416 are each objective time. The objective time can be Greenwich Mean Time (GMT), sometimes referred to as ZULU time, though other time zones can be used as well. As a result, the video stream A and the video stream B may be synchronized so that the objective times 404 and 412 are identical or substantially identical, as well as having the video stream C also synchronized so that the events occurring on each of the video streams A, B and C are all occurring based upon commencement at the objective times 404, 412 and 416. hi addition, each of the video streams will have a subjective time marking as shown at 406, 410 and 414, respectively, on the video streams A, B and C. Each of these subjective time markings are relative to the duration of the specified video file. As an example, the ticks of the clock are recorded for each video frame or picture within the file. Once the video streams A, B and C are properly synchronized according to the objective time designations times 404, 412 and 416, the user may then be able to correspond the events being depicted on each of the video steams A, B and C with the local time at the particular venue for purposes of analysis, comparison to other events, or the like. This ability to synchronize is provided using a UTBC (Universal Time Base Controller) included in the assembler 120. Figure 5 illustrates an example video sequence 500 which includes video 502, a first audio channel 504, a second audio channel 506, and meta-data 508. The variable vA is the bit rate variable, which is related to the number of frames/second and the image quality. The variable vB is the audio bit rate, which can relate to bit depth, how many audio channels are being recorded, the dynamic range. The variable vC is the meta-data bandwidth which is related to maximum bit-per second of aggregate channels. A recorder, such as recorder 100, can be constantly capturing data 510 in a buffer, such as a ring buffer or FIFO, until an instruction or start event starts the recording. The duration of the buffer is vl. The recorder then records multiple segments, such as multiple segments 512, 514, 516, 518, 520 of duration v2. The time v2 can be specified by the user and/or the system administrator. Figure 6 illustrates the conversion of the segments 512-520 illustrated in Figure 5 into an assembled file 602 as assembled or merged by the assembler. Figure 7 illustrates the process of converting assembled video/audio and meta-data files into a Web page for display on a user terminal. The meta-data in file 602 is packetized for transmission over the meta-channel 710 using an address header 704 and a barer channel variable length packet 706, wherein the meta-data can include channels corresponding to multiple datasources.
Figure 9 illustrates an example process of accessing and rendering video streams. At state 902 a user accesses the IMS. As similarly described above, the user may need to log on and provide a user identifier and password. At state 904 a determination is made as to what files the user is authorized to view, and the list of authorized files are sorted and transmitted to the user terminal. The list can be sorted by time of day, date, size, event, or on other fields, including customizable fields. In response to the user selecting one or more of the files, at state 906 the ASP downloads the selected file or files. The frame rate, bit rate and resolution for transmission can be selected based on the wired or wireless network being used and/or on the receiving terminal resolution and frame rate rending capability. At state 908 data, including video, audio, and meta-data may be transmitted from more than one source, such as the IMS and/or ASM. For example, the IMS may transit a GPS map and the ASM can transmit GPS coordinates that was recorded by a vehicle in synchronization with video taken by a video camera mounted in the vehicle. At state 910 the data in rendered on the user terminal screen, such as in the user's browser. Figure 10 illustrates an example encoding process for live source transmission and recording from up to four different simultaneous streams generated from a single camera source, though fewer or more cameras can be used and fewer or more streams can be transmitted and recorded. The example encoding process optionally uses a dedicated processor for MPEG 1, MPEG 2, MPEG 4, CIF, D-l, and High Definition, as well as using real-time software-based encoders that substantially simultaneously encode at various bitrates/frame rates/frame sizes for playback or viewing on a plurality of different user devices such as PDAs, notebooks, and desktop computers. In this example, the "Stream controller" is a infinite pin tee that replicates video/audio data from a single sample source device to enable multiple encoders to simultaneously access different memory addresses for the same data stream. Those familiar with the art will recognize that this is also a software driver "wrapper" used to "spoof the functionality of a device driver for each of the replicated signal sources. Each of the encoded video streams may be recorded, formatted for RTP/RTSP, while any or all of them are simultaneously available for live transmission . It should be understood that certain variations and modifications of this invention would suggest themselves to one of ordinary skill in the art. The scope of the present invention is not to be limited by the illustrations or the foregoing descriptions thereof.

Claims

WHAT IS CLAIMED IS:
1. A recording and transmission system comprising: a recorder mountable in a police vehicle configured to record video and meta-data synchronized with the video, wherein the recorded video includes a plurality of segments having fixed lengths, the recorder including a wireless transmitter configured to wirelessly transmit the recorded video segments and metadata, wherein the meta-data includes: data related to vehicle status; subjective time information; obj ective time information; serial data; an assembler configured to wirelessly receive the transmitted video segments and assemble the video segments into a file including the meta-data, the assembler further including a mass storage device used to store the assembled video and transmit the assembled video; and an information management system coupled to the assembler to receive the assembled video and to store the meta-data in a relational database, wherein the information management system is configured to receive a user search query from a first terminal having a first resolution, conduct a search of the meta-data based at least in part on the search query, transmit the results of the search to the first terminal, receive a user selection of at least a first of the results, configure the video corresponding to the selected result for the first terminal based at least in part on the first resolution, and transmit the configured video to the first terminal.
2. The system as defined in Claim 1, wherein the assembler is. configured to provide the information management system with the meta-data as a string searchable file.
3. The system as defined in Claim 1, wherein the assembler is configured to provide the information management system with the meta-data as an XML file parsed by information type to enable string searches of the meta-data.
4. The system as defined in Claim 1, further comprising a certificate configured to be passed to the user and at least one other destination.
5. The system as defined in Claim 1, wherein the mass storage device includes both a magnetic and optical memory, and the assembled video is stored on both the magnetic and optical memory.
6. The system as defined in Claim 1, wherein the recorder records in an MPEG compatible format.
7. The system as defined in Claim 1, wherein the wireless transmitter is compatible with at least one of CDPD, GSM, GPRS, and 802.11.
8. The system as defined in Claim 1, wherein the video is configured using an Active Server Page generator.
9. The system as defined in Claim 1, wherein the vehicle status is related to the status of a light bar.
10. The system as defined in Claim 1, wherein the vehicle status is related to the status of vehicle speed.
11. The system as defined in Claim 1, wherein the vehicle status is related to the status of a weapon rack.
12. A method of storing and providing video data, the method comprising: receiving a first video file including video and embedded meta-data, the meta-data including at least: timing information; status information; storing copies of at least a portion of the first video file in a second video file on a first data storage system and in a third file on a second storage system; generating at least a first string searchable file based at least in part on the meta-data; receiving a user search query; conducting a search of information in the first string searchable file based at least in part on the user search query; presenting results of the search to the user via a user terminal; receiving a user search results, selection corresponding to the second video file; retrieving the second video file and the third video file based at least in part on the user selection; comparing at least a first portion of the meta-data in the second video file with a corresponding portion in the third video file in order to determine if file tampering has occurred; and transmitting at least a portion of the second video file to the user terminal.
13. The method as defined in Claim 12, further comprising: determining the display capabilities of the user terminal based at least in part on a URL received from the user terminal; and tailoring the portion of the second video file based at least in part on the user terminal display capabilities.
14. The method as defined in Claim 12, wherein the timing information includes subjective timing information, and comparing at least a first portion of the meta-data in the second video file with a corresponding portion in the third video file further comprises comparing subjective timing information.
15. The method as defined in Claim 12, wherein the meta-data includes serial digital data, and comparing at least a first portion of the meta-data in the second video file with a corresponding portion in the third video file further comprises comparing at least a portion of digital serial data.
16. The method as defined in Claim 12, wherein the first video file is received from a storage device located in a vehicle.
17. The method as defined in Claim 12, wherein tailoring the portion of the second video file is performed in substantially real time.
18. The method as defined in Claim 12, wherein tailoring the portion of the second video file is based at least in part of a URL provided by the user terminal.
19. The method as defined in Claim 12, wherein the status includes at least a vehicle status.
20. The method as defined in Claim 12, wherein the status includes at least a weapon status.
21. The method as defined in Claim 12, wherein the status includes at least an alarm status.
22. The method as defined in Claim 12, further comprising: generating a certificate; transmitting a first instance of the certificate to the user terminal and a second instance of the certificate to the first data storage system; and comparing the first instance of the certificate with the second instance of the certificate to determine whether to transmit the portion of the second video file to the user terminal.
23. The method as defined in Claim 12, further comprising: transmitting GPS coordinate information from a first data storage device to the user terminal; and transmitting from a second data storage device a map corresponding to the GPS coordinate information to the user terminal.
24. A method of storing and providing video data, the method comprising: receiving and storing data including a video sequence and synchronized meta-data; generating at least a first string searchable data structure based at least in part on the meta-data; receiving a user search query; and conducting a search of information in the first string searchable data structure based at least in part on the user search query.
25. The method as defined in Claim 24, wherein the search query includes an incident code.
26. The method as defined in Claim 24, wherein the search query includes a police officer identifier.
27. The method as defined in Claim 24, wherein the search query relates to a vehicle light bar status.
28. The method as defined in Claim 24, wherein the search query relates to a vehicle speed.
29. The method as defined in Claim 24, wherein the search query relates to a vehicle identifier.
30. The method as defined in Claim 24, wherein the search query relates to a driver license identifier.
31. The method as defined in Claim 24, wherein the search query relates to a time.
32. The method as defined in Claim 24, wherein the search query relates to a weapon status.
33. The method as defined in Claim 24, wherein the search query relates to a sensor status.
34. The method as defined in Claim 24, wherein the search query relates to a geographic location.
35. The method as defined in Claim 24, further comprising: transmitting at least a portion of the results of the search to the user via a user terminal; receiving a user search results selection corresponding to the video sequence; retrieving the video sequence based at least in part on the user selection; and transmitting at least a portion of the video sequence to the user terminal.
36. The method as defined in Claim 24, further comprising: transmitting at least a portion of the results of the search to the user via a user terminal. receiving a user search results selection corresponding to the video sequence; retrieving at least a first copy and a second copy of the video sequence based at least in part on the user selection; comparing at least a first portion of the meta-data in the first copy with a corresponding portion in the second copy in order to determine if file tampering has occurred; and transmitting at least a portion of the first copy to the user terminal.
37. The method as defined in Claim 24, further comprising displaying at least a portion of the video sequence on a first area of a user terminal display at the same as displaying meta-data in a second area of the user terminal display, wherein the first area and the second area do not overlap.
38. The method as defined in Claim 24, further comprising associating data from a plurality of databases with the video sequence.
39. The method as defined in Claim 24, further comprising: transmitting at least a portion of the results of the search to the user via a user terminal; receiving a user search results selection corresponding to the video sequence; retrieving the video sequence based at least in part on the user selection; retrieving related data from a plurality of alien databases; and transmitting at least a portion of the second video file and related data to the user terminal.
40. The method as defined in Claim 24, further comprising: transmitting at least a portion of the results of the search to the user via a user terminal; receiving a user search results selection corresponding to the video sequence; retrieving the video sequence based at least in part on the user selection; retrieving related data from a computer aided dispatch system database; and transmitting at least a portion of the second video file and related data to the user terminal.
41. The method as defined in Claim 24, further comprising: transmitting at least a portion of the results of the search to the user via a user terminal; receiving a user search results selection corresponding to the video sequence; retrieving the video sequence based at least in part on the user selection, wherein the video sequence is related to an incident reported to a government entity; retrieving data related to a resource dispatched to respond to the incident, the data including one or more of an acknowledgement from the resource that the dispatch was received, and data indicating that the resource has arrived at the scene of the incident; and transmitting at least a portion of the second video file and related data to the user terminal.
42. The method as defined in Claim 24, wherein the video is recorded using intracoded frames, predicted frames, and bi-directional frames, wherein at least a first quantity of intracoded frames are generated when at least 25%, of pixels change in at least one of luminance and chrominance relative to a previous intracoded frames.
43. The method as defined in Claim 24, wherein the video sequence is received at 6 FPS (frames per second) with a resolution of 160 x 120 pixels.
44. The method as defined in Claim 24, wherein the video sequence is received
15 FPS with a resolution of 320 x 240 pixels.
45. The method as defined in Claim 24, wherein the video sequence is received at 30 FPS with a resolution of 352 x 240 pixels.
46. The method as defined in Claim 24, wherein the video sequence is received as streaming video .
47. An apparatus for storing and providing video data, the apparatus comprising: a storage device configured to store data including a video sequence and synchronized meta-data; a first instruction configured to generate at least a first string searchable data structure based at least in part on the meta-data; a second instruction receive a user search query; and a third instruction configured to conduct a search of information in the first string searchable data structure based at least in part on the user search query.
48. The apparatus as defined in Claim 47, further comprising a recorder system configured to be mounted in a vehicle, wherein the recorder is configured to wirelessly transmit the video sequence for storage on the storage device.
49. The apparatus as defined in Claim 47, further comprising a recorder system configured to record data in non- volatile memory in response to an event trigger.
50. The apparatus as defined in Claim 47, further comprising a recorder system, wherein when the recorder system comes into proximity with a designated network, the recorder system is interrogated for new records, and if the recorder contains new records, uploads the new records to a recipient.
51. The apparatus as defined in Claim 47, further comprising a recorder system configured to receive and record a plurality of closed circuit television channels.
52. The apparatus as defined in Claim 47, further comprising a video camera.
53. The apparatus as defined in Claim 47, further comprising a light bar on/off detector input.
54. The apparatus as defined in Claim 47, further comprising a weapon rack sensor input.
55. The apparatus as defined in Claim 47, further comprising a weapon discharge sensor input.
56. The apparatus as defined in Claim 47, further comprising a vehicle speed sensor input.
57. The apparatus as defined in Claim 47, further comprising a tachometer sensor input.
58. The apparatus as defined in Claim 47, further comprising a radar sensor input.
59. The apparatus as defined in Claim 47, further comprising a vehicle door status input.
60. The apparatus as defined in Claim 47, further comprising a fuel level detector input.
61. The apparatus as defined in Claim 47, further comprising at least one of a heartbeat sensor input and a pulse rate sensor input.
62. The apparatus as defined in Claim 47, further comprising a chemical weapons sensor input.
63. The apparatus as defined in Claim 47, further comprising a GPS signal input.
PCT/US2003/033119 2002-10-18 2003-10-17 Video and telemetry apparatus and methods WO2004036926A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2003286491A AU2003286491A1 (en) 2002-10-18 2003-10-17 Video and telemetry apparatus and methods

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US41951502P 2002-10-18 2002-10-18
US60/419,515 2002-10-18
US47870703P 2003-06-13 2003-06-13
US60/478,707 2003-06-13

Publications (2)

Publication Number Publication Date
WO2004036926A2 true WO2004036926A2 (en) 2004-04-29
WO2004036926A3 WO2004036926A3 (en) 2005-03-03

Family

ID=32110252

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2003/033119 WO2004036926A2 (en) 2002-10-18 2003-10-17 Video and telemetry apparatus and methods

Country Status (2)

Country Link
AU (1) AU2003286491A1 (en)
WO (1) WO2004036926A2 (en)

Cited By (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2152004A2 (en) 2008-08-08 2010-02-10 Samsung Electronics Co., Ltd. Image providing apparatus, image displaying apparatus, image transmitting system and image transmitting method using the same
EP1936577A3 (en) * 2006-12-20 2010-05-19 Omvox Telecom Corp. Enhanced multimedia intrusion notification system and method
WO2010139306A1 (en) * 2009-06-05 2010-12-09 Mobotix Ag Method for transmitting video data
EP2958321A4 (en) * 2013-04-17 2016-03-30 Huawei Tech Co Ltd Method, device, and system for playing surveillance video
CN105518697A (en) * 2013-07-31 2016-04-20 萨卢德·马丁内斯蒙雷亚尔 Method implemented by computer for capturing evidentiary audiovisual and/or multimedia information and computer program
US9560309B2 (en) 2004-10-12 2017-01-31 Enforcement Video, Llc Method of and system for mobile surveillance and event recording
US9602761B1 (en) 2015-01-22 2017-03-21 Enforcement Video, Llc Systems and methods for intelligently recording a live media stream
US9660744B1 (en) 2015-01-13 2017-05-23 Enforcement Video, Llc Systems and methods for adaptive frequency synchronization
US9712730B2 (en) 2012-09-28 2017-07-18 Digital Ally, Inc. Portable video and imaging system
US9841259B2 (en) 2015-05-26 2017-12-12 Digital Ally, Inc. Wirelessly conducted electronic weapon
US9860536B2 (en) 2008-02-15 2018-01-02 Enforcement Video, Llc System and method for high-resolution storage of images
US10013883B2 (en) 2015-06-22 2018-07-03 Digital Ally, Inc. Tracking and analysis of drivers within a fleet of vehicles
US10074394B2 (en) 2013-08-14 2018-09-11 Digital Ally, Inc. Computer program, method, and system for managing multiple data recording devices
US10075681B2 (en) 2013-08-14 2018-09-11 Digital Ally, Inc. Dual lens camera unit
CN109154860A (en) * 2016-05-18 2019-01-04 微软技术许可有限责任公司 Emotion/cognitive state trigger recording
US10172436B2 (en) 2014-10-23 2019-01-08 WatchGuard, Inc. Method and system of securing wearable equipment
US10192277B2 (en) 2015-07-14 2019-01-29 Axon Enterprise, Inc. Systems and methods for generating an audit trail for auditable devices
US10250433B1 (en) 2016-03-25 2019-04-02 WatchGuard, Inc. Method and system for peer-to-peer operation of multiple recording devices
US10271015B2 (en) 2008-10-30 2019-04-23 Digital Ally, Inc. Multi-functional remote monitoring system
US10272848B2 (en) 2012-09-28 2019-04-30 Digital Ally, Inc. Mobile video and imaging system
WO2019094569A1 (en) * 2017-11-12 2019-05-16 Bendix Commercial Vehicle Systems Llc Driving oriented digital video recorder system
US10341605B1 (en) 2016-04-07 2019-07-02 WatchGuard, Inc. Systems and methods for multiple-resolution storage of media streams
US10390732B2 (en) 2013-08-14 2019-08-27 Digital Ally, Inc. Breath analyzer, system, and computer program for authenticating, preserving, and presenting breath analysis data
US10409621B2 (en) 2014-10-20 2019-09-10 Taser International, Inc. Systems and methods for distributed control
US10521675B2 (en) 2016-09-19 2019-12-31 Digital Ally, Inc. Systems and methods of legibly capturing vehicle markings
US10730439B2 (en) 2005-09-16 2020-08-04 Digital Ally, Inc. Vehicle-mounted video system with distributed processing
US10853435B2 (en) 2016-06-17 2020-12-01 Axon Enterprise, Inc. Systems and methods for aligning event data
US10904474B2 (en) 2016-02-05 2021-01-26 Digital Ally, Inc. Comprehensive video collection and storage
US10911725B2 (en) 2017-03-09 2021-02-02 Digital Ally, Inc. System for automatically triggering a recording
US11024137B2 (en) 2018-08-08 2021-06-01 Digital Ally, Inc. Remote video triggering and tagging
CN114466238A (en) * 2020-11-09 2022-05-10 华为技术有限公司 Frame demultiplexing method, electronic device and storage medium
US11950017B2 (en) 2022-05-17 2024-04-02 Digital Ally, Inc. Redundant mobile video recording

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6167253A (en) * 1995-01-12 2000-12-26 Bell Atlantic Network Services, Inc. Mobile data/message/electronic mail download system utilizing network-centric protocol such as Java
US6185409B1 (en) * 1995-11-30 2001-02-06 Amsc Subsidiary Corporation Network engineering/systems engineering system for mobile satellite communication system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6167253A (en) * 1995-01-12 2000-12-26 Bell Atlantic Network Services, Inc. Mobile data/message/electronic mail download system utilizing network-centric protocol such as Java
US6185409B1 (en) * 1995-11-30 2001-02-06 Amsc Subsidiary Corporation Network engineering/systems engineering system for mobile satellite communication system

Cited By (62)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9560309B2 (en) 2004-10-12 2017-01-31 Enforcement Video, Llc Method of and system for mobile surveillance and event recording
US10075669B2 (en) 2004-10-12 2018-09-11 WatchGuard, Inc. Method of and system for mobile surveillance and event recording
US9871993B2 (en) 2004-10-12 2018-01-16 WatchGuard, Inc. Method of and system for mobile surveillance and event recording
US10063805B2 (en) 2004-10-12 2018-08-28 WatchGuard, Inc. Method of and system for mobile surveillance and event recording
US9756279B2 (en) 2004-10-12 2017-09-05 Enforcement Video, Llc Method of and system for mobile surveillance and event recording
US10730439B2 (en) 2005-09-16 2020-08-04 Digital Ally, Inc. Vehicle-mounted video system with distributed processing
EP1936577A3 (en) * 2006-12-20 2010-05-19 Omvox Telecom Corp. Enhanced multimedia intrusion notification system and method
US9860536B2 (en) 2008-02-15 2018-01-02 Enforcement Video, Llc System and method for high-resolution storage of images
US10334249B2 (en) 2008-02-15 2019-06-25 WatchGuard, Inc. System and method for high-resolution storage of images
US8854460B2 (en) 2008-08-08 2014-10-07 Samsung Techwin Co., Ltd. Image providing apparatus, image displaying apparatus, image transmitting system and image transmitting method using the same
EP2152004A2 (en) 2008-08-08 2010-02-10 Samsung Electronics Co., Ltd. Image providing apparatus, image displaying apparatus, image transmitting system and image transmitting method using the same
KR101315040B1 (en) 2008-08-08 2013-10-14 삼성테크윈 주식회사 Image providing apparatus, image displaying apparatus, image transmitting system and image transmitting method using the same
EP2152004A3 (en) * 2008-08-08 2011-09-28 Samsung Techwin Co., Ltd Image providing apparatus, image displaying apparatus, image transmitting system and image transmitting method
US10917614B2 (en) 2008-10-30 2021-02-09 Digital Ally, Inc. Multi-functional remote monitoring system
US10271015B2 (en) 2008-10-30 2019-04-23 Digital Ally, Inc. Multi-functional remote monitoring system
US9117320B2 (en) 2009-06-05 2015-08-25 Mobotix Ag Method for transmitting video data
WO2010139306A1 (en) * 2009-06-05 2010-12-09 Mobotix Ag Method for transmitting video data
US11310399B2 (en) 2012-09-28 2022-04-19 Digital Ally, Inc. Portable video and imaging system
US11667251B2 (en) 2012-09-28 2023-06-06 Digital Ally, Inc. Portable video and imaging system
US9712730B2 (en) 2012-09-28 2017-07-18 Digital Ally, Inc. Portable video and imaging system
US10272848B2 (en) 2012-09-28 2019-04-30 Digital Ally, Inc. Mobile video and imaging system
US10257396B2 (en) 2012-09-28 2019-04-09 Digital Ally, Inc. Portable video and imaging system
US9843775B2 (en) 2013-04-17 2017-12-12 Huawei Technologies Co., Ltd. Surveillance video playback method, device, and system
EP2958321A4 (en) * 2013-04-17 2016-03-30 Huawei Tech Co Ltd Method, device, and system for playing surveillance video
CN105518697A (en) * 2013-07-31 2016-04-20 萨卢德·马丁内斯蒙雷亚尔 Method implemented by computer for capturing evidentiary audiovisual and/or multimedia information and computer program
US20160188593A1 (en) * 2013-07-31 2016-06-30 Salud MARTÍNEZ MONREAL Method implemented by computer for capturing evidentiary audiovisual and/or multimedia information and computer program
US10075681B2 (en) 2013-08-14 2018-09-11 Digital Ally, Inc. Dual lens camera unit
US10757378B2 (en) 2013-08-14 2020-08-25 Digital Ally, Inc. Dual lens camera unit
US10390732B2 (en) 2013-08-14 2019-08-27 Digital Ally, Inc. Breath analyzer, system, and computer program for authenticating, preserving, and presenting breath analysis data
US10074394B2 (en) 2013-08-14 2018-09-11 Digital Ally, Inc. Computer program, method, and system for managing multiple data recording devices
US10885937B2 (en) 2013-08-14 2021-01-05 Digital Ally, Inc. Computer program, method, and system for managing multiple data recording devices
US10901754B2 (en) 2014-10-20 2021-01-26 Axon Enterprise, Inc. Systems and methods for distributed control
US11900130B2 (en) 2014-10-20 2024-02-13 Axon Enterprise, Inc. Systems and methods for distributed control
US11544078B2 (en) 2014-10-20 2023-01-03 Axon Enterprise, Inc. Systems and methods for distributed control
US10409621B2 (en) 2014-10-20 2019-09-10 Taser International, Inc. Systems and methods for distributed control
US10172436B2 (en) 2014-10-23 2019-01-08 WatchGuard, Inc. Method and system of securing wearable equipment
US9660744B1 (en) 2015-01-13 2017-05-23 Enforcement Video, Llc Systems and methods for adaptive frequency synchronization
US9923651B2 (en) 2015-01-13 2018-03-20 WatchGuard, Inc. Systems and methods for adaptive frequency synchronization
US9888205B2 (en) 2015-01-22 2018-02-06 WatchGuard, Inc. Systems and methods for intelligently recording a live media stream
US9602761B1 (en) 2015-01-22 2017-03-21 Enforcement Video, Llc Systems and methods for intelligently recording a live media stream
US9841259B2 (en) 2015-05-26 2017-12-12 Digital Ally, Inc. Wirelessly conducted electronic weapon
US10337840B2 (en) 2015-05-26 2019-07-02 Digital Ally, Inc. Wirelessly conducted electronic weapon
US11244570B2 (en) 2015-06-22 2022-02-08 Digital Ally, Inc. Tracking and analysis of drivers within a fleet of vehicles
US10013883B2 (en) 2015-06-22 2018-07-03 Digital Ally, Inc. Tracking and analysis of drivers within a fleet of vehicles
US10848717B2 (en) 2015-07-14 2020-11-24 Axon Enterprise, Inc. Systems and methods for generating an audit trail for auditable devices
US10192277B2 (en) 2015-07-14 2019-01-29 Axon Enterprise, Inc. Systems and methods for generating an audit trail for auditable devices
US10904474B2 (en) 2016-02-05 2021-01-26 Digital Ally, Inc. Comprehensive video collection and storage
US10848368B1 (en) 2016-03-25 2020-11-24 Watchguard Video, Inc. Method and system for peer-to-peer operation of multiple recording devices
US10250433B1 (en) 2016-03-25 2019-04-02 WatchGuard, Inc. Method and system for peer-to-peer operation of multiple recording devices
US10341605B1 (en) 2016-04-07 2019-07-02 WatchGuard, Inc. Systems and methods for multiple-resolution storage of media streams
CN109154860A (en) * 2016-05-18 2019-01-04 微软技术许可有限责任公司 Emotion/cognitive state trigger recording
US10853435B2 (en) 2016-06-17 2020-12-01 Axon Enterprise, Inc. Systems and methods for aligning event data
US10521675B2 (en) 2016-09-19 2019-12-31 Digital Ally, Inc. Systems and methods of legibly capturing vehicle markings
US10911725B2 (en) 2017-03-09 2021-02-02 Digital Ally, Inc. System for automatically triggering a recording
WO2019094569A1 (en) * 2017-11-12 2019-05-16 Bendix Commercial Vehicle Systems Llc Driving oriented digital video recorder system
US10719722B2 (en) 2017-11-12 2020-07-21 Bendix Commercial Vehicle Systems Llc Driving oriented digital video recorder system
US10755121B2 (en) 2017-11-12 2020-08-25 Bendix Commercial Vehicle Systems Llc Driving oriented digital video recorder system
EP4203482A1 (en) * 2017-11-12 2023-06-28 Bendix Commercial Vehicle Systems, LLC Driving oriented digital video recorder system
US11024137B2 (en) 2018-08-08 2021-06-01 Digital Ally, Inc. Remote video triggering and tagging
CN114466238A (en) * 2020-11-09 2022-05-10 华为技术有限公司 Frame demultiplexing method, electronic device and storage medium
CN114466238B (en) * 2020-11-09 2023-09-29 华为技术有限公司 Frame demultiplexing method, electronic device and storage medium
US11950017B2 (en) 2022-05-17 2024-04-02 Digital Ally, Inc. Redundant mobile video recording

Also Published As

Publication number Publication date
AU2003286491A1 (en) 2004-05-04
AU2003286491A8 (en) 2004-05-04
WO2004036926A3 (en) 2005-03-03

Similar Documents

Publication Publication Date Title
WO2004036926A2 (en) Video and telemetry apparatus and methods
US8041829B2 (en) System and method for remote data acquisition and distribution
US10419722B2 (en) Correlated media source management and response control
CN110519477B (en) Embedded device for multimedia capture
US20080212685A1 (en) System for the Capture of Evidentiary Multimedia Data, Live/Delayed Off-Load to Secure Archival Storage and Managed Streaming Distribution
US6360234B2 (en) Video cataloger system with synchronized encoders
US8311983B2 (en) Correlated media for distributed sources
US6567980B1 (en) Video cataloger system with hyperlinked output
US20060274828A1 (en) High capacity surveillance system with fast search capability
US7295752B1 (en) Video cataloger system with audio track extraction
US6463444B1 (en) Video cataloger system with extensibility
US20060274829A1 (en) Mobile surveillance system with redundant media
US20070217763A1 (en) Robust surveillance system with partitioned media
US20030086000A1 (en) Remote surveillance system
US20130084053A1 (en) System to merge multiple recorded video timelines
US20070217501A1 (en) Surveillance system with digital tape cassette
US20050066371A1 (en) Mobile digital security system and method
WO2001028251A1 (en) System and method for controlling the storage and remote retrieval of surveillance video images
CN112969049A (en) Intelligent detection system for ship violation behaviors
Doermann et al. Issues in the transmission, analysis, storage and retrieval of surveillance video
US20240037135A1 (en) System and method for multi-media content bookmarking with provenance
US20080320043A1 (en) OfficerAssist
CA2914803C (en) Embedded appliance for multimedia capture
Annesley et al. An extension of the AVC file format for video surveillance

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP