US20220036073A1 - Method, system and computer program product that provides virtual assistance in facilitating visual comparison - Google Patents
Method, system and computer program product that provides virtual assistance in facilitating visual comparison Download PDFInfo
- Publication number
- US20220036073A1 US20220036073A1 US16/945,969 US202016945969A US2022036073A1 US 20220036073 A1 US20220036073 A1 US 20220036073A1 US 202016945969 A US202016945969 A US 202016945969A US 2022036073 A1 US2022036073 A1 US 2022036073A1
- Authority
- US
- United States
- Prior art keywords
- metadata
- information
- information component
- computer
- processor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G06K9/00671—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G06K9/00281—
-
- G06K9/00295—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
- G06V40/173—Classification, e.g. identification face re-identification, e.g. recognising unknown faces across different face tracks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
- H04N7/185—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
-
- G06K2209/15—
-
- G06K2209/27—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/10—Recognition assisted with metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/179—Human faces, e.g. facial parts, sketches or expressions metadata assisted face recognition
Definitions
- a command center of a public safety or other security entity can often quickly identify a suspect's appearance (for example, identifying the suspect using appearance search and/or other video analytics).
- BLO Be On the Look Out
- Real-time action may be called for, but the patrolling officer may be unable to immediately carry out a visual assessment of reference image(s) because, for example, he is driving a car or Sports Utility Vehicle (SUV), operating a motorbike, riding a horse, etc.
- SUV Sports Utility Vehicle
- Patrolling officers normally carry a variety of different types of equipment to do their jobs.
- a piece of equipment that may be at the patrolling officer's disposal: a sophisticated two-way radio (for example, APX NEXTTM two-way radios, sold by Motorola Solutions, and integrated with ViQi Virtual PartnerTM, also a Motorola Solutions product).
- ViQi Virtual PartnerTM the patrolling officer presses and holds a button on his radio (or uses a remote speaker microphone), and makes a request using natural language.
- Frequently-asked types of queries include drivers' licenses, license plates and vehicle identification numbers.
- a body camera is another example of equipment that may be available, which is a wearable video or photographic recording system, typically worn in one of three locations: on the torso, on or built into a helmet, and on or built into glasses.
- FIG. 1 is a block diagram of a system in accordance with example embodiments
- FIG. 2 is a schematic diagram of a practical implementation, in accordance with example embodiments, of the system of FIG. 1 ;
- FIG. 3 is an example two-way radio that may be used in example embodiments
- FIG. 4 is a flow chart illustrating a method for automated object comparison assistance in accordance with an example embodiment
- FIG. 5 is another schematic diagram of another practical implementation, in accordance with another example embodiment consistent with the method of FIG. 4 ;
- FIG. 6 is a flow chart illustrating another method for automated object comparison assistance in accordance with another example embodiment.
- a computer-implemented method for providing live virtual assistance to a user of the live virtual assistance includes receiving, at an at least one processor, first metadata generated from at least one user-provided image that depicts an unknown object.
- the method also includes providing, to the at least one processor, second metadata generated from at least one reference image of a known object.
- the method also includes comparing, using the at least one processor, the first metadata to the second metadata to determine difference information as between the unknown and known objects.
- the difference information includes at least: a first information component; a second information component; or a third information component.
- the first information component relates to appearance, the second information component relates to size, and the third information component is different from the first and second information components.
- a first audio message is caused to be generated for delivery to the user.
- the first audio message includes at least some of the difference information.
- a system that includes a non-stationary camera that is configured to capture at least one first image that depicts an unknown object, and also configured to output the at least one first image for generating first metadata elsewhere within the system.
- the system also includes storage that is configured to store second metadata generated from at least one second image of a known object.
- the system also includes at least one processor that is in communication with the storage and the non-stationary camera. The at least one processor is configured to compare the first metadata to the second metadata to determine difference information as between the unknown and known objects.
- the difference information includes at least: a first information component; a second information component; or a third information component.
- the first information component relates to appearance
- the second information component relates to size
- the third information component is different from the first and second information components.
- the system also includes a mobile communications device that includes a speaker.
- the mobile communications device is in communication with the at least one processor and is configured to cause the speaker to emit, when the difference information indicates that the unknown object is different than the known object, a first audio message that includes at least some of the difference information.
- a computer-implemented method for providing live virtual assistance to a user of the live virtual assistance includes receiving, at an at least one processor: first metadata generated from a first region of at least one user-provided image that depicts a first unknown object; and second metadata generated from a second region of the at least one user-provided image that depicts a second unknown object.
- the method also includes providing, to the at least one processor, third metadata generated from at least one reference image of a known object.
- the method also includes processing the first metadata, the second metadata and the third metadata to determine, using the at least one processor, that the first unknown object is more similar than the second unknown object to the known object.
- the method also includes comparing, using the at least one processor, the first metadata to the third metadata to determine difference information as between the unknown and known objects.
- the method also includes generating a customized be on the lookout message that includes at least some of the difference information.
- Example embodiments are herein described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to example embodiments. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions.
- These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- the methods and processes set forth herein need not, in some embodiments, be performed in the exact sequence as shown and likewise various blocks may be performed in parallel rather than in sequence. Accordingly, the elements of methods and processes are referred to herein as “blocks” rather than “steps.”
- These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
- the computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational blocks to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide blocks for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. It is contemplated that any part of any aspect or embodiment discussed in this specification can be implemented or combined with any part of any other aspect or embodiment discussed in this specification.
- object as used herein is understood to have the same meaning as would normally be given by one skilled in the art of video analytics, and examples of objects may include humans, vehicles, animals, other entities, etc.
- objects include both moving and stationary objects.
- objects may include only objects that are not always stationary within the video within which they appear.
- FIG. 1 is a block diagram of a system 10 .
- the illustrated system 10 includes a plurality of moving or non-stationary cameras 20 1 - 20 n (for example, body-worn cameras, police vehicle cameras, dash cameras, drone-mounted cameras, etc.) which are coupled to a network 30 (which may comprise a plurality of networks, even though shown as a single network in FIG. 1 for convenience of illustration).
- the network 30 can include the Internet, or one or more other public/private networks coupled together by communication elements: for example, one or more network switches 32 , one or more routers 34 , and/or one or more gateways 36 .
- the network 30 could be of the form of, for example, client-server networks, peer-to-peer networks, etc.
- Data connections between any of the moving cameras 20 1 - 20 n and other network devices can be any number of known arrangements for accessing a data communications network, such as, for example, dial-up Serial Line Interface Protocol/Point-to-Point Protocol (SLIP/PPP), Integrated Services Digital Network (ISDN), dedicated lease line service, broadband (e.g. cable) access, Digital Subscriber Line (DSL), Asynchronous Transfer Mode (ATM), Frame Relay, or other known access techniques (for example, radio frequency (RF) links).
- the moving cameras 20 1 - 20 n and one or more of the other illustrated network devices are within the same Local Area Network (LAN).
- LAN Local Area Network
- the network 30 may include a Radio Access Network (RAN).
- RANs may operate according to an industry standard Land Mobile Radio (LMR) or cellular protocol such as, for example, the Project 25 (P25) standard defined by the Association of Public Safety Communications Officials International (APCO), the TErrestrial Trunked RAdio (TETRA) standard defined by the European Telecommunication Standards Institute (ETSI), the Digital Private Mobile Radio (dPMR) standard also defined by the ETSI, the Digital Mobile Radio (DMR) standard also defined by the ETSI, the Long Term Evolution (LTE) including LTE-Advanced or LTE-Advanced Pro (compliant with, for example, the 3GPP TS 36 specification series), or the 5G (including a network architecture compliant with, for example, the 3GPP TS 23 specification series and a New Radio (NR) air interface compliant with the 3GPP TS 38 specification series) standard, among other possibilities, and over which Multimedia Broadcast Multicast Services (MBMS), Single-Cell Point-To
- the moving cameras 20 1 - 20 n communicate data and information to and from other network devices via the network 30 .
- the moving cameras 20 1 - 20 n transmit video data to one or more other network devices via the network 30 .
- the moving cameras 20 1 - 20 n receive control data from other network devices via the network 30 .
- the moving cameras 20 1 - 20 n need not all be of homogeneous type, and any suitable combination of cameras of different types (i.e. a heterogeneous combination of cameras) is also contemplated.
- a plurality of mobile communication devices 35 1 - 35 m are also included within the system 10 which, in some examples are two-way radios (each formally or informally assigned to a respective individual working within an identifiable area).
- the mobile communication devices 35 1 - 35 m communicate data and information to and from other network devices via the network 30 .
- each of the mobile communication devices 35 1 - 35 m includes, amongst other components, a microphone and a speaker.
- any suitable number of the moving cameras 20 1 - 20 n may optionally be configured to directly transmit wireless communications to any suitable number of the mobile communication devices 35 1 - 35 m that may be configured to receive such wireless communications.
- any suitable number of the mobile communication devices 35 1 - 35 m may optionally be configured to directly transmit wireless communications to any suitable number of the moving cameras 20 1 - 20 n that may be configured to receive such wireless communications.
- a server 40 which is coupled to the network 30 to receive data and information from other devices on the network 30 such as, for example, any of the moving cameras 20 1 - 20 n .
- the server 40 is also coupled to any suitable number of client devices 70 via a network 60 in order that the server 40 may, for example, send and receive data and information as between the client devices 70 and the server 40 .
- the network 60 may comprise a plurality of networks even though shown as a single network in FIG. 1 for convenience of illustration.
- the network 60 can include the Internet, or one or more other public/private networks coupled together by communication elements: for example, one or more network switches 62 , one or more routers 64 , and/or one or more gateways 66 .
- the network 60 could be of the form of, for example, client-server networks, peer-to-peer networks, etc.
- Data connections between any of the client devices 70 and other network devices can be any number of known arrangements for accessing a data communications network, such as, for example, dial-up SLIP/PPP, ISDN, dedicated lease line service, broadband (e.g.
- the network 30 and the network 60 are shown as separate, in some examples there may be some overlap and commonality between the network 30 and the network 60 . In at least one example, the network 60 and the network 30 may be the same network.
- the illustrated server 40 includes a database 81 maintained within storage 83 .
- the database 81 is organized storage for images and/or video footage, video and/or other metadata, etc.
- the server 40 also includes a query manager module 85 (which, for example, provides any of the client devices 70 an interface for retrieving information from the database 81 ), a neural network module 87 (which, for example, enables artificial intelligence functions as herein explicitly and implicitly described), a media server module 89 (which, for example, manages the transmission of stored media to the client devices 70 , the moving cameras 20 1 - 20 n , the mobile communication devices 35 1 - 35 m , etc.), an appearance search module 90 (providing server-side appearance search functionality more details of which are described in, for example, US Pat. Publ. No.
- the server 40 also includes other software components 93 . These other software components will vary depending on the requirements of the server 40 within the overall system. As just one example, the other software components 93 might include special test and debugging software, or software to facilitate version updating of modules within the server 40 .
- FIG. 2 is a schematic diagram of a practical implementation, in accordance with example embodiments, of the system of FIG. 1 .
- a police officer 201 is on patrol within a geographic region 203 .
- the police officer 201 is carrying a body-worn camera 206 , which may correspond to one of moving cameras 20 1 - 20 n , and a two-way radio 208 (shown in more detail in FIG. 3 ) which may correspond to one of the mobile communication devices 35 1 - 35 m .
- the two-way radio 208 includes a display screen 209 , a speaker 210 and a microphone 211 .
- the police officer 201 receives a BOLO on the two-way radio 208 .
- this BOLO includes an image 212 of a person of interest that the police officer 201 is tasked to try and find.
- the police officer 201 may, for instance, look at the image 212 and commit the shown physical characteristics of the person of interest to memory.
- the image 212 is a mug shot of a suspect.
- the police officer 201 will observe each of the three people 220 , 230 and 240 .
- the police officer 201 may immediately determine (i.e. without automated assistance) that the person 220 does not match the subject of the BOLO.
- the police officer 201 may immediately notice that the gender of the person 220 does not match the subject of the BOLO (i.e. the person 220 is not a male individual).
- the police officer 201 may observe the person 230 who, for the purposes of this illustrated example, has more similar physical characteristics to the subject of the BOLO as compared to the previously observed person 220 .
- the police officer 201 may seek automated assistance and, in this regard, reference is now made to FIG. 4 .
- FIG. 4 is a flow chart illustrating a method 400 for automated object comparison assistance in accordance with an example embodiment.
- First, at least one image of an unknown object is captured ( 410 ) using a camera.
- the police officer 201 may point his body-worn camera 206 towards the person 230 and capture video footage (a plurality of images) of the person 230 .
- the at least one image of the unknown object is compared ( 420 ) against at least one reference image of a known object to determine difference information.
- the video footage of the person 230 captured by the body-worn camera 206 may be transmitted through the network 30 to the server 40 , and then subsequently the appearance search module 90 operates cooperatively with the neural network module 87 and the video analytics module 91 to compare one or more images of the person 230 against one or more reference images of the subject of the BOLO. Resultant from this comparison is difference information.
- an audio message that provides the difference information may be transmitted to the police officer 201 .
- a speaker of the two-way mobile radio 208 may emit the audio message.
- Other forms of audio delivery to the police officer 201 are contemplated.
- the police officer 201 may have a BluetoothTM earpiece that emits the audio message into an ear of the police officer 201 .
- any suitable audio message that delivers the difference information in a manner that helps police officer 201 understand why the person 230 is not a potential match may be: “The person is too tall. The person of interest is about a head shorter than this person.”
- the police officer 201 moves on to capture video footage of the next person, namely the person 240 .
- the person 240 is sufficiently similar to the subject of the BOLO.
- an audio message may delivered (in any of the manners previously herein described) to the police officer 201 indicating that the person 240 may be a match to the subject of the BOLO. Accordingly an appropriate action may be taken by the police officer such as, for example, approaching the person 240 if it is permissible for him to do so by himself.
- the security cameras 250 and 252 are communicatively connected to a network 260 (which may be similar to the network 30 shown in FIG. 1 ).
- the network 260 is communicatively connected to a server 270 (which may be similar to the server 40 shown in FIG. 1 ).
- LPR License Plate Recognition
- Example embodiments are not limited to merely safety or security-related applications.
- FIG. 5 shown is a schematic diagram of a store 460 within which the method 400 may be carried out in accordance with an example embodiment.
- Store employee 464 is equipped with a BluetoothTM earpiece 466 and a body-worn camera 468 .
- the employee 464 can operate the earpiece 466 and a body-worn camera 468 in a manner similar to as was described in relation to similar devices which may be worn by the police officer 201 ( FIG. 2 ); however instead of searching for the subject of a BOLO, instead the employee 464 of this illustrated example embodiment is searching for a specific shoe amongst a wall of shoes 470 .
- Difference information may be provided in relation to any one or more of the shoes on the wall as compared to a particular specific shoe of interest.
- appearance search technology as it relates to moving objects is presently well commercialized. Extending appearance search technology to stationary, inanimate objects like shoes, paintings etc. can be achieved through suitable conventional modifications and instructive guidance from teachings accessible to those skilled in the art such as, for example, teachings found the 2015 non-patent literature entitled “Stationary Object Detection in Video” by Filip Leiding.
- FIG. 6 is a flow chart illustrating a method 500 for automated object comparison assistance in accordance with another example embodiment.
- a camera For instance, the police officer 201 shown in FIG. 2 may point his body-worn camera 206 towards both the person 220 and the person 230 (i.e. making the assumption that these two people may come to be sufficiently close to each other in order for both to be within a common field of view of the body-worn camera 206 ) and the body-worn camera 206 thus captures video footage (a plurality of images) of the two people.
- n equals two; however it will be understood that n may be any suitable integer greater than zero.
- a first of the n unknown objects is set ( 520 ) as an initial most similar object.
- the person 220 may, by arbitrary assignment, be a “first” unknown object, and thus set to the initial object that is most similar to the known object (the known object may be the person shown in the image 212 of FIG. 3 , for example).
- decision action 530 namely whether or not there are further unknown objects to assess. If “YES”, action 540 follows, where metadata of the next unknown object is compared against metadata of the known object to assess similarity. So with continuing reference to the aforementioned two people example, the person 230 is the next unknown object. Thus, metadata for the person 230 is compared against metadata for the known person corresponding to the image 212 .
- the person assessed in the previous action 540 is a new most similar object. For example, is the person 230 more similar to the known person than the person 220 . If the object assessed in the previous action 540 is not a new most similar object, then the decision action 530 follows. If however the object assessed in the previous action 540 is a new most similar object, then this object is set ( 560 ) as the new most similar object, and then the decision action 530 follows the action 560 .
- action 570 follows “NO” (no more unknown object to assess).
- this is generation of a customized BOLO based on difference information as between the most similar object of the n unknown objects and the known object. For example, a message along the lines of the following might be delivered to the two-way radio 208 of the police officer 201 : “Officer John, please be on the lookout for a person of interest that looks similar to that man in front of you in the red t-shirt. The person of interest is about a head shorter than that man.”
- Electronic computing devices such as set forth herein are understood as requiring and providing speed and accuracy and complexity management that are not obtainable by human mental steps, in addition to the inherently digital nature of such operations (e.g., a human mind cannot interface directly with RAM or other digital storage, cannot transmit or receive electronic messages, electronically encoded video, electronically encoded audio, etc., and also, among other features and functions set forth herein, it will be understood that well designed computer vision may provide more reliable comparison of images as compared to achievable limits of normal human vision).
- modules herein described as forming part of the server 40 may alternatively be found, in whole or in part, in one or more of the edge devices of the system 10 such as, for example, one or more of the moving cameras 20 1 - 20 n or one or more of the mobile communication devices 35 1 - 35 m .
- a includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element.
- the terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein.
- the terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%.
- a device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
- Coupled can have several different meanings depending in the context in which these terms are used.
- the terms coupled, coupling, or connected can have a mechanical or electrical connotation.
- the terms coupled, coupling, or connected can indicate that two elements or devices are directly connected to one another or connected to one another through an intermediate elements or devices via an electrical element, electrical signal or a mechanical element depending on the particular context.
- processors such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein.
- processors or “processing devices” such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein.
- FPGAs field programmable gate arrays
- unique stored program instructions including both software and firmware
- an embodiment can be implemented as a computer-readable storage medium having computer readable code stored thereon for programming a computer (e.g., comprising a processor) to perform a method as described and claimed herein.
- a computer e.g., comprising a processor
- Any suitable computer-usable or computer readable medium may be utilized. Examples of such computer-readable storage mediums include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory.
- a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- computer program code for carrying out operations of various example embodiments may be written in an object oriented programming language such as Java, Smalltalk, C++, Python, or the like.
- object oriented programming language such as Java, Smalltalk, C++, Python, or the like.
- computer program code for carrying out operations of various example embodiments may also be written in conventional procedural programming languages, such as the “C” programming language or similar programming languages.
- the program code may execute entirely on a computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or server or entirely on the remote computer or server.
- the remote computer or server may be connected to the computer through a LAN or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
- WAN Wide Area Network
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
- Alarm Systems (AREA)
Abstract
Description
- To the extent that there exists availability of video security footage, a command center of a public safety or other security entity can often quickly identify a suspect's appearance (for example, identifying the suspect using appearance search and/or other video analytics). By contrast, it can be challenging for the command center to effectively convey descriptive and identifying information about the subject of a Be On the Look Out (BOLO) to a patrolling officer searching for that subject around a city or other patrolled area. Real-time action may be called for, but the patrolling officer may be unable to immediately carry out a visual assessment of reference image(s) because, for example, he is driving a car or Sports Utility Vehicle (SUV), operating a motorbike, riding a horse, etc. Even in other cases (such as, for example, when the patrolling officer is on foot) it may still be difficult for the patrolling officer to unrestrictedly look at one or more reference images (sent to him on his two-way radio or smart phone) because, for example, the patrolling officer may feel the need to keep uninterrupted eye attention on the surroundings in support of his effort to look and search for the subject of the BOLO (this is especially true when such searching is taking place in a crowded area).
- Patrolling officers normally carry a variety of different types of equipment to do their jobs. One example of a piece of equipment that may be at the patrolling officer's disposal: a sophisticated two-way radio (for example, APX NEXT™ two-way radios, sold by Motorola Solutions, and integrated with ViQi Virtual Partner™, also a Motorola Solutions product). Using the ViQi Virtual Partner™, the patrolling officer presses and holds a button on his radio (or uses a remote speaker microphone), and makes a request using natural language. Frequently-asked types of queries include drivers' licenses, license plates and vehicle identification numbers. A body camera is another example of equipment that may be available, which is a wearable video or photographic recording system, typically worn in one of three locations: on the torso, on or built into a helmet, and on or built into glasses.
- In the accompanying figures similar or the same reference numerals may be repeated to indicate corresponding or analogous elements. These figures, together with the detailed description, below are incorporated in and form part of the specification and serve to further illustrate various embodiments of concepts that include the claimed invention, and to explain various principles and advantages of those embodiments.
-
FIG. 1 is a block diagram of a system in accordance with example embodiments; -
FIG. 2 is a schematic diagram of a practical implementation, in accordance with example embodiments, of the system ofFIG. 1 ; -
FIG. 3 is an example two-way radio that may be used in example embodiments; -
FIG. 4 is a flow chart illustrating a method for automated object comparison assistance in accordance with an example embodiment; -
FIG. 5 is another schematic diagram of another practical implementation, in accordance with another example embodiment consistent with the method ofFIG. 4 ; and -
FIG. 6 is a flow chart illustrating another method for automated object comparison assistance in accordance with another example embodiment. - Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of embodiments of the present disclosure.
- The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
- According to one example embodiment, there is provided a computer-implemented method for providing live virtual assistance to a user of the live virtual assistance. The method includes receiving, at an at least one processor, first metadata generated from at least one user-provided image that depicts an unknown object. The method also includes providing, to the at least one processor, second metadata generated from at least one reference image of a known object. The method also includes comparing, using the at least one processor, the first metadata to the second metadata to determine difference information as between the unknown and known objects. The difference information includes at least: a first information component; a second information component; or a third information component. The first information component relates to appearance, the second information component relates to size, and the third information component is different from the first and second information components. When the difference information indicates that the unknown object is different than the known object, a first audio message is caused to be generated for delivery to the user. The first audio message includes at least some of the difference information.
- According to another example embodiment, there is provided a system that includes a non-stationary camera that is configured to capture at least one first image that depicts an unknown object, and also configured to output the at least one first image for generating first metadata elsewhere within the system. The system also includes storage that is configured to store second metadata generated from at least one second image of a known object. The system also includes at least one processor that is in communication with the storage and the non-stationary camera. The at least one processor is configured to compare the first metadata to the second metadata to determine difference information as between the unknown and known objects. The difference information includes at least: a first information component; a second information component; or a third information component. The first information component relates to appearance, the second information component relates to size, and the third information component is different from the first and second information components. The system also includes a mobile communications device that includes a speaker. The mobile communications device is in communication with the at least one processor and is configured to cause the speaker to emit, when the difference information indicates that the unknown object is different than the known object, a first audio message that includes at least some of the difference information.
- According to yet another example embodiment, there is provided a computer-implemented method for providing live virtual assistance to a user of the live virtual assistance. The method includes receiving, at an at least one processor: first metadata generated from a first region of at least one user-provided image that depicts a first unknown object; and second metadata generated from a second region of the at least one user-provided image that depicts a second unknown object. The method also includes providing, to the at least one processor, third metadata generated from at least one reference image of a known object. The method also includes processing the first metadata, the second metadata and the third metadata to determine, using the at least one processor, that the first unknown object is more similar than the second unknown object to the known object. The method also includes comparing, using the at least one processor, the first metadata to the third metadata to determine difference information as between the unknown and known objects. The method also includes generating a customized be on the lookout message that includes at least some of the difference information.
- Each of the above-mentioned embodiments will be discussed in more detail below, starting with example system and device architectures of the system in which the embodiments may be practiced, followed by an illustration of processing blocks for achieving an improved technical method, device, and system for providing virtual assistance in facilitating visual comparison. Example embodiments are herein described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to example embodiments. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. The methods and processes set forth herein need not, in some embodiments, be performed in the exact sequence as shown and likewise various blocks may be performed in parallel rather than in sequence. Accordingly, the elements of methods and processes are referred to herein as “blocks” rather than “steps.”
- These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
- The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational blocks to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide blocks for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. It is contemplated that any part of any aspect or embodiment discussed in this specification can be implemented or combined with any part of any other aspect or embodiment discussed in this specification.
- The term “object” as used herein is understood to have the same meaning as would normally be given by one skilled in the art of video analytics, and examples of objects may include humans, vehicles, animals, other entities, etc. In some example embodiments objects include both moving and stationary objects. In other example embodiments objects may include only objects that are not always stationary within the video within which they appear.
- Further advantages and features consistent with this disclosure will be set forth in the following detailed description, with reference to the figures.
- Referring now to the drawings, and in particular
FIG. 1 , which is a block diagram of asystem 10. The illustratedsystem 10 includes a plurality of moving or non-stationary cameras 20 1-20 n (for example, body-worn cameras, police vehicle cameras, dash cameras, drone-mounted cameras, etc.) which are coupled to a network 30 (which may comprise a plurality of networks, even though shown as a single network inFIG. 1 for convenience of illustration). Thenetwork 30 can include the Internet, or one or more other public/private networks coupled together by communication elements: for example, one ormore network switches 32, one ormore routers 34, and/or one ormore gateways 36. Thenetwork 30 could be of the form of, for example, client-server networks, peer-to-peer networks, etc. Data connections between any of the moving cameras 20 1-20 n and other network devices can be any number of known arrangements for accessing a data communications network, such as, for example, dial-up Serial Line Interface Protocol/Point-to-Point Protocol (SLIP/PPP), Integrated Services Digital Network (ISDN), dedicated lease line service, broadband (e.g. cable) access, Digital Subscriber Line (DSL), Asynchronous Transfer Mode (ATM), Frame Relay, or other known access techniques (for example, radio frequency (RF) links). In at least one example embodiment, the moving cameras 20 1-20 n and one or more of the other illustrated network devices are within the same Local Area Network (LAN). - It is contemplated that the
network 30 may include a Radio Access Network (RAN). RANs may operate according to an industry standard Land Mobile Radio (LMR) or cellular protocol such as, for example, the Project 25 (P25) standard defined by the Association of Public Safety Communications Officials International (APCO), the TErrestrial Trunked RAdio (TETRA) standard defined by the European Telecommunication Standards Institute (ETSI), the Digital Private Mobile Radio (dPMR) standard also defined by the ETSI, the Digital Mobile Radio (DMR) standard also defined by the ETSI, the Long Term Evolution (LTE) including LTE-Advanced or LTE-Advanced Pro (compliant with, for example, the3GPP TS 36 specification series), or the 5G (including a network architecture compliant with, for example, the 3GPP TS 23 specification series and a New Radio (NR) air interface compliant with the 3GPP TS 38 specification series) standard, among other possibilities, and over which Multimedia Broadcast Multicast Services (MBMS), Single-Cell Point-To-Multipoint (SC-PTM) services, or Mission Critical Push-to-talk (MCPTT) services may be provided, or over which an Open Mobile Alliance (OMA) Push To Talk (PTT) over cellular, a Voice over IP (VoIP), or a PTT over IP (PoW) application may be implemented. - Still with reference to
FIG. 1 , the moving cameras 20 1-20 n communicate data and information to and from other network devices via thenetwork 30. For instance, the moving cameras 20 1-20 n transmit video data to one or more other network devices via thenetwork 30. As another example, the moving cameras 20 1-20 n receive control data from other network devices via thenetwork 30. In at least some example embodiments, the moving cameras 20 1-20 n need not all be of homogeneous type, and any suitable combination of cameras of different types (i.e. a heterogeneous combination of cameras) is also contemplated. - A plurality of mobile communication devices 35 1-35 m are also included within the
system 10 which, in some examples are two-way radios (each formally or informally assigned to a respective individual working within an identifiable area). The mobile communication devices 35 1-35 m communicate data and information to and from other network devices via thenetwork 30. Also, each of the mobile communication devices 35 1-35 m includes, amongst other components, a microphone and a speaker. - Still with reference to the moving cameras 20 1-20 n and the mobile communication devices 35 1-35 m, those skilled in the art will appreciate that any suitable number of the moving cameras 20 1-20 n may optionally be configured to directly transmit wireless communications to any suitable number of the mobile communication devices 35 1-35 m that may be configured to receive such wireless communications. Similarly any suitable number of the mobile communication devices 35 1-35 m may optionally be configured to directly transmit wireless communications to any suitable number of the moving cameras 20 1-20 n that may be configured to receive such wireless communications.
- Also shown in
FIG. 1 is aserver 40 which is coupled to thenetwork 30 to receive data and information from other devices on thenetwork 30 such as, for example, any of the moving cameras 20 1-20 n. Theserver 40 is also coupled to any suitable number ofclient devices 70 via anetwork 60 in order that theserver 40 may, for example, send and receive data and information as between theclient devices 70 and theserver 40. - Regarding the
network 60, this may comprise a plurality of networks even though shown as a single network inFIG. 1 for convenience of illustration. Thenetwork 60 can include the Internet, or one or more other public/private networks coupled together by communication elements: for example, one or more network switches 62, one ormore routers 64, and/or one ormore gateways 66. Thenetwork 60 could be of the form of, for example, client-server networks, peer-to-peer networks, etc. Data connections between any of theclient devices 70 and other network devices can be any number of known arrangements for accessing a data communications network, such as, for example, dial-up SLIP/PPP, ISDN, dedicated lease line service, broadband (e.g. cable) access, DSL, ATM, Frame Relay, or other known access techniques (for example, RF links). Although in the illustrated example embodiment thenetwork 30 and thenetwork 60 are shown as separate, in some examples there may be some overlap and commonality between thenetwork 30 and thenetwork 60. In at least one example, thenetwork 60 and thenetwork 30 may be the same network. - Still with reference to
FIG. 1 , the illustratedserver 40 includes adatabase 81 maintained withinstorage 83. Amongst other things, thedatabase 81 is organized storage for images and/or video footage, video and/or other metadata, etc. - The
server 40 also includes a query manager module 85 (which, for example, provides any of theclient devices 70 an interface for retrieving information from the database 81), a neural network module 87 (which, for example, enables artificial intelligence functions as herein explicitly and implicitly described), a media server module 89 (which, for example, manages the transmission of stored media to theclient devices 70, the moving cameras 20 1-20 n, the mobile communication devices 35 1-35 m, etc.), an appearance search module 90 (providing server-side appearance search functionality more details of which are described in, for example, US Pat. Publ. No. 2018/0157939 entitled “System and Method for Appearance Search”), and a video analytics module 91 (providing server-side video analytics functionality, the details of which will be understood by those skilled in the art and which may be expected to vary depending upon specifics of different suitable alternative implementations). Theserver 40 also includesother software components 93. These other software components will vary depending on the requirements of theserver 40 within the overall system. As just one example, theother software components 93 might include special test and debugging software, or software to facilitate version updating of modules within theserver 40. - Reference is now made to
FIG. 2 .FIG. 2 is a schematic diagram of a practical implementation, in accordance with example embodiments, of the system ofFIG. 1 . In the illustrated example implementation, apolice officer 201 is on patrol within ageographic region 203. Thepolice officer 201 is carrying a body-worncamera 206, which may correspond to one of moving cameras 20 1-20 n, and a two-way radio 208 (shown in more detail inFIG. 3 ) which may correspond to one of the mobile communication devices 35 1-35 m. Amongst other things, the two-way radio 208 includes adisplay screen 209, aspeaker 210 and amicrophone 211. - In accordance with at least one example, the
police officer 201 receives a BOLO on the two-way radio 208. As shown inFIG. 3 , this BOLO includes animage 212 of a person of interest that thepolice officer 201 is tasked to try and find. Thepolice officer 201 may, for instance, look at theimage 212 and commit the shown physical characteristics of the person of interest to memory. In at least one example, theimage 212 is a mug shot of a suspect. - Referring once again to
FIG. 2 , shown are threepeople police officer 201. In accordance with the illustrated example, thepolice officer 201 will observe each of the threepeople person 220, thepolice officer 201 may immediately determine (i.e. without automated assistance) that theperson 220 does not match the subject of the BOLO. For example, thepolice officer 201 may immediately notice that the gender of theperson 220 does not match the subject of the BOLO (i.e. theperson 220 is not a male individual). - Next, the
police officer 201 may observe theperson 230 who, for the purposes of this illustrated example, has more similar physical characteristics to the subject of the BOLO as compared to the previously observedperson 220. Here thepolice officer 201 may seek automated assistance and, in this regard, reference is now made toFIG. 4 . -
FIG. 4 is a flow chart illustrating amethod 400 for automated object comparison assistance in accordance with an example embodiment. First, at least one image of an unknown object is captured (410) using a camera. For instance, thepolice officer 201 may point his body-worncamera 206 towards theperson 230 and capture video footage (a plurality of images) of theperson 230. - Next, the at least one image of the unknown object is compared (420) against at least one reference image of a known object to determine difference information. For example, the video footage of the
person 230 captured by the body-worncamera 206 may be transmitted through thenetwork 30 to theserver 40, and then subsequently theappearance search module 90 operates cooperatively with theneural network module 87 and thevideo analytics module 91 to compare one or more images of theperson 230 against one or more reference images of the subject of the BOLO. Resultant from this comparison is difference information. - Next, it is determined (430) whether or not (based on the images comparison described above) there is sufficient likelihood that the unknown object is the known object. If “NO”,
action 440 follows. If “YES”,action 450 follows. - With continuing reference to the practical implementation of
FIG. 2 , following comparison of the video footage of theperson 230 captured by the body-worncamera 206 to reference image(s), it may be determined that theperson 230 is not sufficiently similar to the subject of the BOLO. Accordingly and consistent with theaction 440, an audio message that provides the difference information may be transmitted to thepolice officer 201. For instance, a speaker of the two-waymobile radio 208 may emit the audio message. Other forms of audio delivery to thepolice officer 201 are contemplated. For instance, thepolice officer 201 may have a Bluetooth™ earpiece that emits the audio message into an ear of thepolice officer 201. Also, it will be understood that any suitable audio message that delivers the difference information in a manner that helpspolice officer 201 understand why theperson 230 is not a potential match. As just one example amongst many, the audio message in relation to theperson 230 may be: “The person is too tall. The person of interest is about a head shorter than this person.” - Still with reference to the practical implementation of
FIG. 2 , say that thepolice officer 201 moves on to capture video footage of the next person, namely theperson 240. Following comparison of the video footage of theperson 240 captured by the body-worncamera 206 to reference image(s), it may be determined that theperson 240 is sufficiently similar to the subject of the BOLO. In accordance with theaction 450, an audio message may delivered (in any of the manners previously herein described) to thepolice officer 201 indicating that theperson 240 may be a match to the subject of the BOLO. Accordingly an appropriate action may be taken by the police officer such as, for example, approaching theperson 240 if it is permissible for him to do so by himself. - Also within the
geographic region 203 are twostationary security cameras security cameras camera 206 to capture additional video images of any one or more of thepeople security cameras network 30 shown inFIG. 1 ). Thenetwork 260 is communicatively connected to a server 270 (which may be similar to theserver 40 shown inFIG. 1 ). - Also within the
geographic region 203 are twovehicles - Example embodiments are not limited to merely safety or security-related applications. Referring to
FIG. 5 , shown is a schematic diagram of astore 460 within which themethod 400 may be carried out in accordance with an example embodiment.Store employee 464 is equipped with aBluetooth™ earpiece 466 and a body-worncamera 468. Theemployee 464 can operate theearpiece 466 and a body-worncamera 468 in a manner similar to as was described in relation to similar devices which may be worn by the police officer 201 (FIG. 2 ); however instead of searching for the subject of a BOLO, instead theemployee 464 of this illustrated example embodiment is searching for a specific shoe amongst a wall ofshoes 470. Difference information may be provided in relation to any one or more of the shoes on the wall as compared to a particular specific shoe of interest. Also, those skilled in the art will appreciate that appearance search technology as it relates to moving objects is presently well commercialized. Extending appearance search technology to stationary, inanimate objects like shoes, paintings etc. can be achieved through suitable conventional modifications and instructive guidance from teachings accessible to those skilled in the art such as, for example, teachings found the 2015 non-patent literature entitled “Stationary Object Detection in Video” by Filip Leiding. - Continuing on, reference is now made to
FIG. 6 .FIG. 6 is a flow chart illustrating amethod 500 for automated object comparison assistance in accordance with another example embodiment. First, at least one image, within which are n unknown objects, is captured (510) using a camera. For instance, thepolice officer 201 shown inFIG. 2 may point his body-worncamera 206 towards both theperson 220 and the person 230 (i.e. making the assumption that these two people may come to be sufficiently close to each other in order for both to be within a common field of view of the body-worn camera 206) and the body-worncamera 206 thus captures video footage (a plurality of images) of the two people. (In this example n equals two; however it will be understood that n may be any suitable integer greater than zero.) - Next in the
method 500, a first of the n unknown objects is set (520) as an initial most similar object. For example, theperson 220 may, by arbitrary assignment, be a “first” unknown object, and thus set to the initial object that is most similar to the known object (the known object may be the person shown in theimage 212 ofFIG. 3 , for example). - Next is
decision action 530, namely whether or not there are further unknown objects to assess. If “YES”,action 540 follows, where metadata of the next unknown object is compared against metadata of the known object to assess similarity. So with continuing reference to the aforementioned two people example, theperson 230 is the next unknown object. Thus, metadata for theperson 230 is compared against metadata for the known person corresponding to theimage 212. - Next, it is determined (550) whether or not the person assessed in the
previous action 540 is a new most similar object. For example, is theperson 230 more similar to the known person than theperson 220. If the object assessed in theprevious action 540 is not a new most similar object, then thedecision action 530 follows. If however the object assessed in theprevious action 540 is a new most similar object, then this object is set (560) as the new most similar object, and then thedecision action 530 follows theaction 560. - With reference once again to the
decision action 530,action 570 follows “NO” (no more unknown object to assess). In regards to theaction 570, this is generation of a customized BOLO based on difference information as between the most similar object of the n unknown objects and the known object. For example, a message along the lines of the following might be delivered to the two-way radio 208 of the police officer 201: “Officer John, please be on the lookout for a person of interest that looks similar to that man in front of you in the red t-shirt. The person of interest is about a head shorter than that man.” - It will be appreciated that variations on the
method 500 are contemplated. For example, instead of the BOLO message being tailored based on a single object-to-object comparison, a BOLO message tailored based on a plurality of object-to-object comparisons is also contemplated. - As should be apparent from this detailed description, the operations and functions of the electronic computing device are sufficiently complex as to require their implementation on a computer system, and cannot be performed, as a practical matter, in the human mind. Electronic computing devices such as set forth herein are understood as requiring and providing speed and accuracy and complexity management that are not obtainable by human mental steps, in addition to the inherently digital nature of such operations (e.g., a human mind cannot interface directly with RAM or other digital storage, cannot transmit or receive electronic messages, electronically encoded video, electronically encoded audio, etc., and also, among other features and functions set forth herein, it will be understood that well designed computer vision may provide more reliable comparison of images as compared to achievable limits of normal human vision).
- In the foregoing specification, specific embodiments have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. For example, at least some of the modules herein described as forming part of the
server 40 may alternatively be found, in whole or in part, in one or more of the edge devices of thesystem 10 such as, for example, one or more of the moving cameras 20 1-20 n or one or more of the mobile communication devices 35 1-35 m. - Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
- Moreover in this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art, and in one non-limiting embodiment the term is defined to be within 10%, in another embodiment within 5%, in another embodiment within 1% and in another embodiment within 0.5%. The term “one of”, without a more limiting modifier such as “only one of”, and when applied herein to two or more subsequently defined options such as “one of A and B” should be construed to mean an existence of any one of the options in the list alone (e.g., A alone or B alone) or any combination of two or more of the options in the list (e.g., A and B together).
- A device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
- The terms “coupled”, “coupling” or “connected” as used herein can have several different meanings depending in the context in which these terms are used. For example, the terms coupled, coupling, or connected can have a mechanical or electrical connotation. For example, as used herein, the terms coupled, coupling, or connected can indicate that two elements or devices are directly connected to one another or connected to one another through an intermediate elements or devices via an electrical element, electrical signal or a mechanical element depending on the particular context.
- It will be appreciated that some embodiments may be comprised of one or more generic or specialized processors (or “processing devices”) such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and/or apparatus described herein. Alternatively, some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more Application Specific Integrated Circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic. Of course, a combination of the two approaches could be used.
- Moreover, an embodiment can be implemented as a computer-readable storage medium having computer readable code stored thereon for programming a computer (e.g., comprising a processor) to perform a method as described and claimed herein. Any suitable computer-usable or computer readable medium may be utilized. Examples of such computer-readable storage mediums include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory. In the context of this document, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation. For example, computer program code for carrying out operations of various example embodiments may be written in an object oriented programming language such as Java, Smalltalk, C++, Python, or the like. However, the computer program code for carrying out operations of various example embodiments may also be written in conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on a computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer or server or entirely on the remote computer or server. In the latter scenario, the remote computer or server may be connected to the computer through a LAN or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
- The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/945,969 US11443510B2 (en) | 2020-08-03 | 2020-08-03 | Method, system and computer program product that provides virtual assistance in facilitating visual comparison |
PCT/US2021/042501 WO2022031437A1 (en) | 2020-08-03 | 2021-07-21 | Method, system and computer program product that provides virtual assistance in facilitating visual comparison |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/945,969 US11443510B2 (en) | 2020-08-03 | 2020-08-03 | Method, system and computer program product that provides virtual assistance in facilitating visual comparison |
Publications (2)
Publication Number | Publication Date |
---|---|
US20220036073A1 true US20220036073A1 (en) | 2022-02-03 |
US11443510B2 US11443510B2 (en) | 2022-09-13 |
Family
ID=77265347
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/945,969 Active 2040-10-07 US11443510B2 (en) | 2020-08-03 | 2020-08-03 | Method, system and computer program product that provides virtual assistance in facilitating visual comparison |
Country Status (2)
Country | Link |
---|---|
US (1) | US11443510B2 (en) |
WO (1) | WO2022031437A1 (en) |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5091780A (en) * | 1990-05-09 | 1992-02-25 | Carnegie-Mellon University | A trainable security system emthod for the same |
US20020071033A1 (en) * | 2000-12-12 | 2002-06-13 | Philips Electronics North America Corporation | Apparatus and methods for resolution of entry/exit conflicts for security monitoring systems |
US20020135483A1 (en) * | 1999-12-23 | 2002-09-26 | Christian Merheim | Monitoring system |
US20170076140A1 (en) * | 2015-09-14 | 2017-03-16 | Panasonic Intellectual Property Management Co., Ltd. | Wearable camera system and method of notifying person |
US20180146169A1 (en) * | 2016-02-26 | 2018-05-24 | Ring Inc. | Locating a Person of Interest Using Shared Video Footage from Audio/Video Recording and Communication Devices |
US20180268818A1 (en) * | 2017-03-20 | 2018-09-20 | Ebay Inc. | Detection of mission change in conversation |
US20180357870A1 (en) * | 2017-06-07 | 2018-12-13 | Amazon Technologies, Inc. | Behavior-aware security systems and associated methods |
US20190122516A1 (en) * | 2017-10-19 | 2019-04-25 | L3 Technologies, Inc. | Threat detection and warning system |
US20190156640A1 (en) * | 2017-11-17 | 2019-05-23 | Motorola Solutions, Inc. | Systems and methods for surveillance-assisted patrol |
US20190278976A1 (en) * | 2018-03-11 | 2019-09-12 | Krishna Khadloya | Security system with face recognition |
US20200238952A1 (en) * | 2019-01-28 | 2020-07-30 | Jeffrey Dean Lindsay | Facial recognition systems for enhanced security in vehicles and other devices |
US20200279381A1 (en) * | 2017-11-15 | 2020-09-03 | Panasonic Corporation | Communication device, communication system, and mobile body tracking method |
US20200327347A1 (en) * | 2019-04-15 | 2020-10-15 | Panasonic I-Pro Sensing Solutions Co., Ltd. | Person monitoring system and person monitoring method |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060282771A1 (en) | 2005-06-10 | 2006-12-14 | Tad Vinci | Verifying document compliance to a subsidiary standard |
US20120288140A1 (en) | 2011-05-13 | 2012-11-15 | Carnegie Mellon University | Method and system for selecting a video analysis method based on available video representation features |
US8873813B2 (en) | 2012-09-17 | 2014-10-28 | Z Advanced Computing, Inc. | Application of Z-webs and Z-factors to analytics, search engine, learning, recognition, natural language, and other utilities |
US8892419B2 (en) | 2012-04-10 | 2014-11-18 | Artificial Solutions Iberia SL | System and methods for semiautomatic generation and tuning of natural language interaction applications |
US8892132B2 (en) | 2012-09-05 | 2014-11-18 | Motorola Solutions, Inc. | Analytic and tracking systems and methods using over-the-air identifiers of mobile devices |
US8837906B2 (en) | 2012-12-14 | 2014-09-16 | Motorola Solutions, Inc. | Computer assisted dispatch incident report video search and tagging systems and methods |
US10061835B2 (en) | 2013-10-28 | 2018-08-28 | Motorola Solutions, Inc. | Establishing user-confidence levels of data inputs |
US10714079B2 (en) | 2015-12-23 | 2020-07-14 | Motorola Solutions, Inc. | Methods and system for analyzing conversational statements and providing feedback in real-time |
CN109416692B (en) | 2016-06-02 | 2023-08-11 | 柯达阿拉里斯股份有限公司 | Method for making and distributing one or more customized media-centric products |
US10204274B2 (en) | 2016-06-29 | 2019-02-12 | Cellular South, Inc. | Video to data |
WO2018076201A1 (en) | 2016-10-26 | 2018-05-03 | Motorola Solutions, Inc. | Systems and methods for prioritizing wireless communication |
CN110235138B (en) | 2016-12-05 | 2023-09-05 | 摩托罗拉解决方案公司 | System and method for appearance search |
PL424877A1 (en) | 2018-03-14 | 2019-09-23 | Waldemar Kowalski | Method and the system for feeding combustion engines |
US11036979B2 (en) | 2018-03-15 | 2021-06-15 | Motorola Solutions, Inc. | Device, system and method for generating an alert and an automatic search for a candidate subject |
US10867219B2 (en) | 2018-08-30 | 2020-12-15 | Motorola Solutions, Inc. | System and method for intelligent traffic stop classifier loading |
US20220012469A1 (en) | 2019-03-29 | 2022-01-13 | Wojciech Kucharski | Method for sharing information for identifying a person or object |
WO2020263110A1 (en) | 2019-06-25 | 2020-12-30 | Motorola Solutions, Inc | System and method for saving bandwidth in performing facial recognition |
US11062162B2 (en) | 2019-08-27 | 2021-07-13 | Motorola Solutions, Inc. | Breaking pursuit and strategy change |
-
2020
- 2020-08-03 US US16/945,969 patent/US11443510B2/en active Active
-
2021
- 2021-07-21 WO PCT/US2021/042501 patent/WO2022031437A1/en active Application Filing
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5091780A (en) * | 1990-05-09 | 1992-02-25 | Carnegie-Mellon University | A trainable security system emthod for the same |
US20020135483A1 (en) * | 1999-12-23 | 2002-09-26 | Christian Merheim | Monitoring system |
US20020071033A1 (en) * | 2000-12-12 | 2002-06-13 | Philips Electronics North America Corporation | Apparatus and methods for resolution of entry/exit conflicts for security monitoring systems |
US20170076140A1 (en) * | 2015-09-14 | 2017-03-16 | Panasonic Intellectual Property Management Co., Ltd. | Wearable camera system and method of notifying person |
US20180146169A1 (en) * | 2016-02-26 | 2018-05-24 | Ring Inc. | Locating a Person of Interest Using Shared Video Footage from Audio/Video Recording and Communication Devices |
US20180268818A1 (en) * | 2017-03-20 | 2018-09-20 | Ebay Inc. | Detection of mission change in conversation |
US20180357870A1 (en) * | 2017-06-07 | 2018-12-13 | Amazon Technologies, Inc. | Behavior-aware security systems and associated methods |
US20190122516A1 (en) * | 2017-10-19 | 2019-04-25 | L3 Technologies, Inc. | Threat detection and warning system |
US20200279381A1 (en) * | 2017-11-15 | 2020-09-03 | Panasonic Corporation | Communication device, communication system, and mobile body tracking method |
US20190156640A1 (en) * | 2017-11-17 | 2019-05-23 | Motorola Solutions, Inc. | Systems and methods for surveillance-assisted patrol |
US20190278976A1 (en) * | 2018-03-11 | 2019-09-12 | Krishna Khadloya | Security system with face recognition |
US20200238952A1 (en) * | 2019-01-28 | 2020-07-30 | Jeffrey Dean Lindsay | Facial recognition systems for enhanced security in vehicles and other devices |
US20200327347A1 (en) * | 2019-04-15 | 2020-10-15 | Panasonic I-Pro Sensing Solutions Co., Ltd. | Person monitoring system and person monitoring method |
Also Published As
Publication number | Publication date |
---|---|
WO2022031437A1 (en) | 2022-02-10 |
US11443510B2 (en) | 2022-09-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2882693C (en) | Method, apparatus and system for performing facial recognition | |
US11620470B2 (en) | Device, system and method for generating an alert and an automatic search for a candidate subject | |
CN111937050B (en) | Passenger related item loss reduction | |
US10489982B2 (en) | Device, system and method for controlling a display screen using a knowledge graph | |
AU2018281924B2 (en) | System and method for tailoring an electronic digital assistant inquiry response as a function of previously detected user ingestion of related video information | |
CN109636368B (en) | Internet taxi taking method and device, taxi taking equipment and computer readable storage medium | |
US9854015B2 (en) | Incident data collection for public protection agencies | |
US20200204848A1 (en) | Device and method for providing relevant video content to members of a communication group | |
US20190005310A1 (en) | Public service system and method using autonomous smart car | |
US20160063335A1 (en) | A method and technical equipment for people identification | |
US20170064529A1 (en) | System for automatically transmitting emergency signal using smart phone, and method therefor | |
CN110909203A (en) | Video analysis method and device, electronic equipment and storage medium | |
JP2023539250A (en) | Lost item notification methods, devices, electronic devices, storage media and computer programs | |
CN109214326A (en) | A kind of information processing method, device and system | |
US10891753B2 (en) | Device, system and method for notifying a person-of-interest of their location within an estimated field-of-view of a camera | |
JP7428682B2 (en) | Investigation support system and investigation support method | |
CN113486957A (en) | Neural network training and image processing method and device | |
US11443510B2 (en) | Method, system and computer program product that provides virtual assistance in facilitating visual comparison | |
US11096017B2 (en) | Broadcasting received identifying characteristic | |
US20220121700A1 (en) | Picture-based message generation and retrieval | |
US20210027605A1 (en) | Method and apparatus for warning a user about a suspicious vehicle | |
US20220414141A1 (en) | Image-assisted field verification of query response |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MOTOROLA SOLUTIONS INC., ILLINOIS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SOO, SWEE YEE;WONG, CHOON KANG;LOH, WAI SEONG;AND OTHERS;SIGNING DATES FROM 20200802 TO 20200803;REEL/FRAME:053379/0069 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |