WO2016123251A1

WO2016123251A1 - System and method for providing and responding to missing person alerts

Info

Publication number: WO2016123251A1
Application number: PCT/US2016/015191
Authority: WO
Inventors: Marios Savvides; George CONTIS; Nathan KHOSLA; Jordan Brown; Ryan P. FLOOD
Original assignee: Marios Savvides; Contis George; Khosla Nathan; Jordan Brown; Flood Ryan P
Priority date: 2015-01-27
Filing date: 2016-01-27
Publication date: 2016-08-04

Abstract

The system and method described herein are directed to a real-time and crowd-source based information exchange system for missing person cases. The system and method include user interfaces, such as a web browser or a mobile application, servers, and network communication platform giving parents and communities the ability to report missing children as soon as possible, and allowing people to anonymously give tips while tying in international law enforcement agencies. The system provides open API access for recognized law enforcement agencies, standardization and translation of report languages, and localization of public-facing information. The system includes servers which establish an international backend, allowing it to be tied into any law enforcement internal software and can be used as a collaboration backend where any developments in the case can be instantly seen by all relevant parties and evaluated.

Description

SYSTEM AND METHOD FOR PROVIDING AND RESPONDING TO MISSING PERSON

ALERTS

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No. 62/108,375, filed January 27, 2015, the contents of which are incorporated herein in their entirety.

BACKGROUND

The abduction of people, and in particular children, poses a major problem in societies around the world. However, such incidents have often been dealt with locally, with regional searches and involvement of only nearby law enforcement agencies (LEAs). For example, according to the US State Department, as many as 27 million people each year are a victim of human trafficking. Other estimates by the International Labor Organization (focusing only on children) estimate the number at 1 .2 million. Importantly, family members or others reporting a missing person are typically faced with a 24hr waiting period before law enforcement is able to take any action. However, by that time an abducted child could have been taken far away from the place of abduction, significantly lessening the chances that they can be recovered. With little international collaboration, many children (and adults) can be transferred forcibly across borders, leaving the LEA local to the place of abduction running a futile search and the LEA where the abducted person has been moved completely unaware of the situation.

Current systems for missing children in place and accepted by the law enforcement community have significant flaws that limit their efficacy, particularly in cross-border situations. These systems lack real time information sharing, limit how information on a missing person can be distributed, limit who may contribute and review information, and limit how quickly the information is disseminated to the general public. In the case of cross- border situations, language barriers may also prevent collaboration from occurring. Further, current systems lack automated messaging between platforms, and thus information is not quickly and automatically shared between different LEAs, or between LEAs and the public.

What is needed is a real time international information exchange, providing both LEAs and civilians with access to real-time data on missing persons in their area and the ability to add to the information on a missing person in real time.

SUMMARY OF THE INVENTION

The system and method described herein are directed to a real-time and crowd- source based information exchange system for missing person cases. The system and method include user interfaces, such as a web browser or a mobile application, servers, and network communication platform giving parents and communities the ability to report missing children as soon as possible, and allowing people to anonymously give tips while tying in international law enforcement agencies. The system provides open API access for recognized law enforcement agencies, standardization and translation of report languages, and localization of public-facing information. The system includes servers which establish an international backend, allowing it to be tied into any law enforcement internal software and can be used as a collaboration backend where any developments in the case can be instantly seen by all relevant parties and evaluated.

Further, the user interface, such as the mobile application, allows the general public to view reported missing persons that may be near them, and provides an interface for them to send tips or other information they may have. For example, if a user sees a reported missing person, she can use the mobile application to indicate a potential sighting, which will be tied to the missing person profile in the system. In order to verify the accuracy of reported sightings or updated information on a missing person, the system may include authentication procedures. These procedures may include a registration and video authentication process for a user to access the system, access missing person information, and add sightings or other information to a missing person report in the system. When new information and sightings on a given missing person are added to the system, backend servers may run sophisticated pattern recognition analysis on the sightings, user reported information, any attached media such as images, and metadata about the report such as report location, reporter identity (if available) and any other relevant data. Through this pattern recognition analysis, and other processing of verified information and tips from users, the system provides helpful information for LEAs and others to use in locating missing persons.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is an illustration of adding missing person reports and generating notifications using the system.

Figure 2 is an illustration of uploading and distributing tips relevant to reported missing persons using the system.

Figure 3 is an illustration of the mobile application presented to a user through a user interface.

Figure 4 is an illustration of the processes involved in generating reports, viewing missing person profiles and information, and adding tips to missing person profiles using the mobile application. DETAILED DESCRIPTION OF THE INVENTION

The system and method are directed to a real-time, crowd sourced information exchange for missing person cases. The system includes a user interface, such a website or mobile application, which communicate with one or more servers having access to information databases. The at least one server may be a cloud based server, or may be one or more computer servers accessible over wired or wireless communication interfaces. The system may include any collection of computers used for the organization, interpretation, analysis, or distribution of information, metadata, or media related to missing persons. The databases may be any digital technology used for the structural organization of information. Law enforcement agencies can be any federal, regional, local, or international agency or group involved in the recovery of missing persons.

The servers and the databases can be synchronized with law enforcement's similar functioned system if necessary. The system further includes user interfaces displayed on user devices. For example, the user interface may be a mobile application running on a smartphone or tablet, or may be a website accessible to a user with a computing device connected to the internet or a local network. The user device may include a camera or imaging component, allowing the device to acquire the video data necessary to complete a video verification process. The mobile application, or website, is a user platform program that includes instructions executable by a computer processor in the user device. The mobile application includes an authentication feature for verifying a user's identity, an authorization feature for authorizing a user to communicate with the one or more servers and access information in the one or more databases, and an accounting feature that tracks a user's access of missing report reports and other data within the system.

The mobile application allows a user to report a missing child, send a tip about the location of an already reported missing child, and view all nearby reports or missing persons. To view nearby missing persons, the mobile application may access a local location device within the user device, such as a GPS receiver, or ask the user to input location information. The mobile application may also run in the background on the user device, alerting users to missing children reports nearby using notifications. These notifications may be real-time notifications such as push, email, ftp, http, TCP/IP, udp, internet client/server or SMS, or any other method of communicating relevant information from the system to an end user. For example, messages may be delivered as phone calls, messages over third-party social platforms, or third party notification services. The mobile application may provide the user with a selectable option as to whether they want to automatically receive alerts, or whether they only want alerts when they actively log in to the application. The mobile application may include an interface with a messaging and report component that users can regularly check for updates. The mobile application may also offer the user an option to sign up for notifications to be delivered in a particular format with a selected frequency. For example, a user may select notifications to be delivered once daily in a text message, or once a week in an e-mail.

As soon as a person is reported missing in an area and the information on this missing person is input into the system, all nearby users will receive a notification including a picture of the missing person, full name, a description including identifying features, and other relevant information.

The missing person reporting system allows users to input missing person reports, update information on an already reported missing person, and receive notifications regarding missing persons reported within the system. As seen in Figure 1 , a user may initiate a missing persons report within the system in two ways. As shown at 101 A, a user having a user device running the mobile application, or accessing a system web page, may directly submit a missing report to the system servers and databases 102. Alternatively, as shown at 101 B, a person may initiate a missing person report with an LEA or other agency. This agency may file the report in their database, and also send the missing report to the system servers and databases 102. Upon receiving a missing person report, the server and databases 102 first checks previously filed missing person reports stored in the databases. The server and databases 102 may additionally send the missing person report to LEAs and other agencies to be added to the internal databases at those agencies. The servers and databases 102 then notifications to system users 103 of the newly added missing person report. These notifications to users may be geographically based, such that only users within a certain geographical range receive an alert about a missing person. The geographical range may be established based on, for example, where the person is believed to have gone missing or where the person is believed to have been transported to. The users receive the alerts on user interfaces implemented on a user device. The user devices may be a smartphone or tablet running a mobile application, or a computer accessing a web page interface.

The system further allows user to submit a tip on a reported missing person within the system. As shown in Figure 2, if a system user sees a person they believe to be reported as missing within the system, they may send a tip to the system servers using the mobile application or web page interface as shown at 201 . A tip may also be generated if a user notices a person identified in a missing person report as being a potential suspect, or being the last known person the missing person was seen with. Tips may be any information that could help with locating the reported missing person. When the user is accessing the system through a mobile application on a smartphone, tablet, or other device that includes a camera, the tip may include photographs or images of any potentially relevant individuals taken by the user. The tip may be delivered to a communication server 202, which sends the tip to the system servers 203 and databases 204, where the tip is stored in an information record associated with the missing person. The system servers 203 check the incoming tip to ensure that the tip is for an open and existing case, and that the tip appears credible. When photographs or images are sent with the tip, the servers may analyze the photographs and perform a validity scoring by comparing the photographs to images of individuals associated with the missing person report already stored in the databases. This validity scoring may include facial recognition analysis if faces are visible in the photographs or images sent with the tip. After the servers determine that the tip is credible, a new alert that includes the information from the tip is generated and

communicated to system users as shown at 205. This process may be performed in real time, which allows system users to receive updated information on a missing person as quickly as possible.

Figure 3 provides an illustration of the features of the user interface of the mobile application program. The mobile application user interface may include a home screen 300 showing a listing of reported missing persons. The user interface may allow a user to select from the reported missing persons to view missing person profiles. As shown at 301 , a missing person profile in the system may include names, aliases, photographs of the reported missing person, a detailed description of physical characteristics of the person, such as height, weight, eye color, hair color, age, gender, and/or distinguishing features, and any other information that may help to identify the missing person. This other information could include personality descriptions, listings of other individuals who may have information on the missing person, or information about schools, careers, or hobbies. Within a missing person profile, the mobile application may provide a selectable option for a user to submit a tip regarding that missing person. When a user chooses to submit a tip, she may be directed to a report generation feature 302. The user may be asked to first provide personal information identifying herself. This personal identifying information may include name, phone number, address, and e-e-mail. The user may then provide a tip report on the missing person. This report can include locations, photographs, images, or any other information that the user believes may be relevant to locating the reporting missing person. Once the personal information and report have been entered, the user may submit the report. As shown at 303, a user may also generate a new missing person report within the system. Once information on a potentially missing person is input, the user may be directed to the report submission feature 302, which will first ask for the personal identifying information before generating the report. The newly generated missing person report will be sent to the system, and a missing person profile will be created and added to the databases. Users may also choose to view missing person information on a map 304, with reports, tips, and locations relevant to the missing persons identified within a certain geographic area.

Figure 4 illustrates the process for using the mobile application on a user device.

When a user accesses the mobile application 401 , they are presented with a variety of options regarding missing persons, including but not limited to the ability to report a new missing person, the ability to view a list of missing persons sorted by some combination of at least one of user input/search, geodistance from the user or regional groupings, and in addition the ability to view various reported missing people on a map. Upon choosing to report a new missing person 402 (Path R), the user may be presented with the ability to provide a picture(s) (step R1 ) that will be used as the basis for identifying the missing person. This identification may be done by facial recognition or other recognition software implemented by the mobile application or servers, or through recognition when presented to another human user. The user may then proceed to Step R2 where they are prompted to provide more detailed information about the missing person such as height, weight, eye color, hair color, age, gender, distinguishing features, or any other information that may help to identify the missing person. The user may then add additional information that will aid in the recovery of the reported missing person (Step R3). Such information may include, in addition to other relevant information, the location at which the person was last seen, possible destination locations, place of employment or school, and any persons the missing person was last seen with. Finally, the user may provide verifiable, identifiable information (Step R4). Such information may be provided directly to the system through the mobile application or through a third party with whom the user has previously identified themselves. This information may be used to track reports of missing persons, perform fraud analytics to prevent the propagation of fake reports, and to give proper law enforcement agencies the identity of the reporter in special cases or cases of fraud. While the user is providing the above information (but prior to submission of the report) the mobile application implements a process (Process C1 ) to collect pertinent data to provide additional data with the report and allow for the possibility of identifying the user in the case that the police wish to pursue more information. This information can include, but is not limited to, the location of the phone, the type of phone (identified through many factors including but not limited to screen resolution, operative system version, hardware components such as GPS, camera, or front-facing camera), the phone number, any name given to the phone, or any other identifying aspect or attribute of the device. Once the report is completed, and Process C1 adds any additional information to the report, the report with additional information is communicated to the system servers for processing and storage in the databases, as shown at 402.

A user also has the option to view the detailed profile of a currently missing person from within the mobile application (through Path T). Having viewed the profile, the user may then choose to go back to the home screen or take further action on the current profile, including sending a tip (Step T1 ). When a user chooses to submit a tip, the user is prompted for as much relevant information as they can provide including a description of the person, a description of anyone the person is with, the location (through one or more of geo- coordinates, addresses, and more descriptive text) of any potential sighting of the missing person or other person relevant to a missing person, possible media they may have collected such as audio/visual recordings or pictures, descriptions of vehicles the person was in, and any other information that may be helpful in locating the missing person (Step T2). While the user is providing the above tip information (but prior to submission of the tip) the mobile application implements a process (Process C1 ) to collect pertinent data to provide additional data with the tip and allow for the possibility of identifying the tip-giver in the case that the police wish to pursue more information. This information can include, but is not limited to, the location of the phone, the type of phone (identified through many factors including but not limited to screen resolution, operative system version, hardware components such as GPS, camera, or front-facing camera), the phone number, any name given to the phone, or any other identifying aspect or attribute of the device. Once the tip is completed, and Process C1 adds any additional information to the tip, the tip with additional information is communicated to the system servers for processing and storage in the databases, as shown at 403.

The mobile application may also allow a user to view a map of all missing people reported near them displayed on a map (Path M). The map may include icons of missing persons or tips. The user may select any of the icons to receive more information about the person or tip, and may further be transferred to the beginning of Path T if they wish to submit a tip.

Before a user submits any report or tip regarding a missing person, he or she may be asked to provide further identification than that automatically gathered. This identification may be used in an authentication process. First, the mobile application may request the user to present an identification card, such as government issued ID or official school ID. An image of the identification card may be captured with a camera on the user device running the mobile application. The user device may be, for example, a smartphone or tablet. The user device may also be a personal computer with a camera input attached. Second, the mobile application may capture the text and the images of label or human face on the identification card through consecutive frames comparison. Third, the mobile application may instruct the user to make certain movement such as smiling, blinking, or moving the ID in a certain direction or pattern in order to verify a real person with a real ID, not a photocopy of the ID was presented in front of a camera of the user device. Liveliness tests can also include time domain scanning for color differentials that indicate the presence of a pulse or asking the user to strike a specific pose that can then be detected. Facial recognition procedures may be implemented to prove that the person in the video is the same person shown on the picture in the identification card. Fourth, the mobile application may recognize and verify the identification card by recognizing a holograph embedded in a valid government ID or school ID. If the identification card does not include a hologram, other identifying images, logos, or words on the ID may be used for verification purposes. Fifth, the mobile application checks for matches to known holograph patterns for the type of identification card presented. Sixth, the mobile application gives permission when the identification card and user movements are verified, or denies registration to the system if the user movements or identification card are not verified. The mobile application implementing the authentication process may be programed to perform optical character recognition, pattern recognition and/or refection and brightness recognition between video frames captured by the device camera.

To recognize the holograph pattern on an identification card such as a state driver license, the camera on the user device captures consecutive frames of the images of the user and the identification card first through an instructed video pasture. The mobile application then processes the non-changing part of the frames and recognizes the text and images for the primary identification card information. The mobile application then identifies the pixels having high brightness value, identifies background pixels, and identifies the value change with movement of the identification card. The mobile application further filters the images by comparing the brightness values to thresholds, and generates an image of the holograph. The image of the holograph is then checked against a database of known holographs for the particular identification card presented. When a match is found, the identification card may be verified.

In addition to the mobile application, the system may include a website interface.

The website interface may include a section for the general public. This section may allow the generation public to view open missing persons with geographic information and to submit tips if and only if valid identifying information is given or otherwise available for verification purposes. The mobile application may also include a section for allowing individuals to report a missing person and edit existing missing person profiles under their authority. This section may allow users to access, edit, and delete missing person reports and profiles that are under their authority within the system. It also provides them the ability to review, confirm, and edit associated attachments such as pictures, sound files, or image files related to missing persons under their authority. Finally, it provides the ability to review, edit, and create alerts.

The website interface may further have sections specifically for LEAs. Special log in information or other identification may be required for access to this section. The LEA section may provide LEAs with:

• ability to edit, delete missing persons under their authority

• ability to review, confirm, edit associated attachments such as pictures, sound files or image files related to missing persons under their authority.

• ability to review, edit, create alerts.

• ability to review sources of information and flag as abuse

• access to API to automate previous abilities

• API ability to update police system with new information as it arises

Claims

1 . A system for reporting information regarding missing persons, the system comprising:

a system server in communication with at least one information database;

a user device in communication with the system server, the user device including an imaging element and a processor, wherein the processor implements computer-readable instructions to run a mobile application program, and wherein the mobile application program is configured to:

generate missing reports and communicate the missing person reports to the system server;

receive missing person report notifications from the system server;

generate tips related to an existing missing person report and communicate the tips to the system server;

receive missing person tip notifications from the system server; and

authenticate users, tips, and reports before communication to the system server.

2. The system of claim 1 , wherein, as part of authenticating users, tips, and reports, the mobile application is configured to request a user to present an identification card having identifiable features, capture text and images from the identification card, instruct the user to perform certain motions, capture the motions using the camera on the user device, and compare the identifiable features on the card to a database of identifiable features expected to be on the identification card.

3. The system of claim 2, wherein the identifiable features include a hologram.