US20190156495A1 - Systems and Methods of Tracking of the Moving Objects on the Video Image - Google Patents
Systems and Methods of Tracking of the Moving Objects on the Video Image Download PDFInfo
- Publication number
- US20190156495A1 US20190156495A1 US16/044,332 US201816044332A US2019156495A1 US 20190156495 A1 US20190156495 A1 US 20190156495A1 US 201816044332 A US201816044332 A US 201816044332A US 2019156495 A1 US2019156495 A1 US 2019156495A1
- Authority
- US
- United States
- Prior art keywords
- image
- image capture
- moving object
- capture device
- trajectory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 16
- 238000013500 data storage Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 241001422033 Thestylus Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/7867—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G06K9/00771—
-
- G06K9/3241—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
- G08B13/19608—Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/029—Location-based management or tracking services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/33—Services specially adapted for particular environments, situations or purposes for indoor environments, e.g. buildings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Definitions
- the present invention belongs to area of video surveillance and, more specifically, to systems and methods of processing of the video information received from surveillance cameras for tracking of the moving objects in real time or when viewing archival video.
- Systems of video surveillance are understood as the hardware-software providing or technical means using including methods of computer vision for the automated data collection, on the basis of the analysis of stream video (video analysis).
- Systems of video surveillance can lean on the algorithms of processing of the image and recognition of images allows analyzing video without direct participation of the person.
- Systems of video surveillance are used as a part of intellectual systems of video surveillance and video search.
- Systems of video surveillance can realize a set of functions, for example such as: detection of objects, tracking the movement of objects, classification of objects, identification of objects, detection or recognition of various situations, including disturbing events and etc.
- the system of information processing is configured for identification of the direction of movement of the object which appeared in the video image of the first video camera from a set of video cameras, definition of set of probabilities and a prediction on the basis of the received set of probabilities of one or several following video cameras for which the probability of emergence of the tracked object is higher, than for other video cameras.
- a lack of this solution is the impossibility of the select of a concrete object and recognition of its movement, and also lack of highlight of a moving object as areas of interest at recognition of the movement. Besides, the solution is not including a stage of a linking a set of video cameras to location map of an object of video surveillance.
- the specified shortcomings do not allow reaching the necessary level of accuracy of tracking of a concrete moving object in preset territory.
- the device is configured to display in real time on the device of display of a set of types of display of the taken images made the corresponding cameras, and a set of types of display are located on the image of the map representing a controlled zone, and the device contains the block of installation of the purpose, for the choice and installation of a moving object as the purpose which has to be tracked, the block of the indication of the purpose which on the basis of information of tracking obtained as a result of processing of images specifies a moving object on each of types on which there is a moving object, the prediction block, for a prediction of the following type of display on which a moving object will appear next time, and the block of the indication of a type of display which specifies the following type of display.
- This solution is characterized the insufficient accuracy of tracking of the moving objects due to lack of an appropriate linking a set of video cameras to location map of an object of video surveillance. Besides, this solution is not applicable for tracking of the moving objects when viewing archival video. Lack of the fact of recognition of the movement and assessment of the direction of the movement of the selected moving object also belongs to shortcomings.
- the system for tracking of the moving objects on the video image in real time contains: set of image capture devices; storage device; image-display device; data input device; a set of data processing device configured for performance of the stages including: a linking a set of image capture devices to location map, to create links between objects on location map and the same objects on the video image; display the video image from the first image capture device; select a moving object which is tracked by operator as an object of interest; recognition of the movement of the selected moving object and considering it on the video image as area of interest; assessment of the direction of the movement of the selected moving object; prediction of the second image capture device in the field of which view there can be a selected moving object after leaves area of view of the first image capture device if it continues to move in the same direction, as at the time of an exit from area of view of the first image capture device; display on the image-display device the video image from the second image capture device, right after that as the selected moving object will leave area of view of the first image capture device.
- the system for tracking of the moving objects on the video image when viewing archival video contains: set of image capture devices; storage device; image-display device; data input device; a set of data processing device configured for performance of the stages including: a linking a set of image capture devices to location map by creating links between objects on location map and the same objects on the video image; display of the archival video image from the first image capture device; select a moving object which is tracked by operator as an object of interest; recognition of the movement of the selected moving object and considering it on the video image as area of interest; assessment of the direction of the movement of the selected moving object; prediction of the second image capture device in the field of which view there can be a selected moving object after leaves area of view of the first image capture device; display on the image-display device of the archival video image from the first image capture device, right after that as a moving object was selected by the operator as an object of interest, and display of the archival video image from the second image capture device begins with that time point in which the selected
- the yielded technical result is also achieved due to performance of a method of tracking of the moving objects on the video image, and the method is carried out in real time or when viewing archival video and contains stages at which: linking a set of image capture devices to location map by creating links between objects on location map and the same objects on the video image, and links are set by virtual line segments which one end corresponds to an object arrangement on the video image and other end corresponds to an object arrangement on location map; display the video image from the first image capture device; select a moving object which is tracked by operator as an object of interest; identify the movement of the selected moving object and estimate its direction of the movement; predict the second image capture device in the field of which view there can be a selected moving object after leaves area of view of the first image capture device; displaying on the image-display device of the video image from the second image capture device.
- Also technical result is achieved also due to the performance of a method of tracking of the moving objects on the video image which is carried out in real time or when viewing archival video and containing stages at which: linking a set of image capture devices to location map by creating links between objects on location map and the same objects on the video image; display the video image from the image capture device; select a moving object which is tracked by operator as an object of interest; identify the movement of the selected moving object by definition positions of a moving object on location map from coordinates of the positions received from the image capture device, and determination of coordinates of positions of a moving object is carried out by receiving metadata from the image capture device; after recognition of the movement highlights a moving object on the video image as the area of interest, and area of interest is displayed on the image-display device as the translucent rectangular area imposed on the video image; estimate the direction of the movement of the selected moving object on history of positions of the selected moving object;
- each of a set of data processing device is in addition configured for display on the image-display device of the highlighted trajectory of the movement of the selected moving object.
- only one object can be selected when selecting a moving object.
- recognition of the movement of a moving object includes definition of positions of a moving object on location map from coordinates of the positions received from the image capture device.
- determination of coordinates of positions of a moving object is carried out by receiving metadata from the image capture device.
- location map is the open map of streets (OpenStreetMap) or the building map or the map of the protected object.
- the area of interest is translucent rectangular area.
- the moving object is a person or group of close going people or the vehicle.
- FIG. 1 flowchart of system for tracking of the moving objects
- FIG. 2 flowchart of one of options of realization of a method of tracking of the moving objects
- FIG. 3 an example of a linking a set of image capture devices to location map by creating links
- FIG. 4 an example of work of system of tracking of the moving objects
- FIG. 5 flowchart of other option of realization of a method for tracking of the moving objects
- FIG. 1 shows a flowchart of system for tracking of the moving objects.
- the system includes: set of data processing device ( 10 , . . . , 1 n ), set of image capture devices ( 20 , . . . , 2 m ), data storage device ( 30 ), data input device ( 40 ) and image-display devices ( 50 ).
- the image capture device usually is the video camera.
- the processor the microprocessor, the computer (electronic computer), PLC (programmable logical controller) or integrated circuit configured for execution of certain commands (instructions, programs) for data processing (images) can act as the data processing device.
- the computer electronic computer
- PLC programmable logical controller
- integrated circuit configured for execution of certain commands (instructions, programs) for data processing (images) can act as the data processing device.
- Hard drives HDD
- flash memory ROM (read-only memory), solid-state stores (SSD) and etc. can act as the data storage device.
- ROM read-only memory
- SSD solid-state stores
- the display, the monitor, the video terminal and others can act as the image-display devices.
- the data input device can be represented by the mouse manipulator, the keyboard, the touchpad, the stylus, the joystick, the trackpad and etc.
- FIG. 2 shows the flowchart of one of options of performance of method of tracking of the moving objects.
- the specified method can be realized by computer system in two modes: in real time and in browse mode of archival video.
- Location map is a kind of the topographic map or the drawing of the small site of the area in the set scale.
- the building map or the protected object, and also the open map of streets (OpenStreetMap) can be location map.
- Location map is the image in a public format (.jpeg or .png) and can be two-dimensional or three-dimensional. The map can be loaded into system of video surveillance. The possibility of automatic and/or manual scaling and shift is characteristic of location map.
- the linking a set of cameras to the area is carried out by creating links between motionless objects on location map and the same objects on the video image.
- Various pieces of furniture, office and/or household appliances, doors, window sills and other objects having volume and spatial structure can act as motionless objects.
- the specified links are set by virtual line segments, one end (point) of which corresponds to an arrangement of a motionless object on the video image, and other end (point) corresponds to an arrangement of a motionless object on location map.
- the operator sets final points then the system sets virtual line segments. The operator can correct and/or delete the set links.
- FIG. 3 shows an example of a linking a set of image capture devices to location map by creating four links.
- the realization of correlation of the video image with objects on location map can be enabled in the immersion mode.
- the operator can “move” between cameras, choosing any camera from the current area of visibility.
- the video image from the selected video camera is displayed over the image of location map.
- the video image from the selected video camera is displayed over the image of location map.
- Video cameras can be installed in more than one room. At the same time to each camera from a set of cameras there can correspond the location map (that is in any area only one video camera can be installed), or several video cameras are installed in one area and the map of this area with an opportunity to switch between video cameras is used. Video images can be displayed imposed over location map on the display screen.
- the operator selects the moving object which is interesting for him. It is only possible to select one interesting moving object, and the person, an animal, the vehicle or group of close going people can be a moving object.
- recognition of the movement of a moving object includes definition of positions of a moving object on location map from coordinates of the positions received from the image capture device, and determination of coordinates of positions of a moving object is carried out by receiving metadata from the image capture device.
- the system receives metadata if the tracker (for example, AxxonSoft) or an external tracker works at the camera in the field of which view a moving object is located.
- Tracking of objects identifies existence of a moving object in the field of view of the camera and dynamically highlights it on the video image with translucent rectangular area as area of interest.
- Location of moving objects in view of the video camera can be defined by video of the analytics which is built in each video camera.
- the specified location of objects is determined by the sensors fixing various parts of a range (visible, thermal) or sensors, difference by the principle of action from the video camera, such as radars.
- History of positions is characteristics of several previous received positions of movements of objects, the speed of the movement, an orientation of the movement on the basis of which one or more data processing device can predict the further direction of the movement of an object of interest.
- the specified prediction of the camera is carried out in the referral of the movement got at the previous stage.
- the system counts a further trajectory of the movement of the selected object and defines the camera to which area of view it most likely will get.
- This stage has essential differences when using various browse modes of the video image.
- Such approach reduces time of viewing of archival video and increases monitoring speed with the purpose to monitor movement of the selected object of interest as allows the operator not to watch in the specified browse mode of archival video all archive from the first camera (not to wait when a moving object leaves area of view of the first camera), and at once to pass to viewing of archive from the second camera (the third, etc.), and from that time point when an object of interest has to appear in the field of its view.
- FIG. 4 shows a concrete example of work of the system of tracking of the moving objects.
- FIG. 5 shows the flowchart of other option of realization of a method of tracking of the moving objects.
- the specified method is carried out by computer means as in real time, and when viewing archival video. Initial stages of this method are similar to the stages which are in detail described above.
- Main distinctive and essential features are that after a stage of recognition of the movement automatic highlighting of a moving object on the video image as area of interest is carried out and the area of interest is displayed on the image-display device as the translucent rectangular area imposed on the video image. Further the direction of the movement of the selected moving object is estimated on stories of positions of the selected moving object and displaying on the image-display device of the video image with the highlighted trajectory of the movement of the selected moving object.
- Detecting of a start of motion is made on a gradient of an interpersonal difference of the video image in time.
- the trajectory of the movement for different objects of interest can be highlighted in different colors.
- the trajectory of the movement is imposed on the video image also, as well as the translucent rectangular area characterizing area of interest.
- Highlight system of the movement trajectory provides an opportunity to the operator visually to track the direction of the movement of the selected object and if desired by the data input device, to pass independently to viewing of that camera to which area of view the highlighted object of interest goes. It is easily realized solution on condition of correlation of a set of cameras with location map.
Abstract
Description
- This application claims priority to Russian Patent Application No. RU 2017141015, filed Nov. 24, 2017, which is incorporated herein by reference in its entirety.
- The present invention belongs to area of video surveillance and, more specifically, to systems and methods of processing of the video information received from surveillance cameras for tracking of the moving objects in real time or when viewing archival video.
- Systems of video surveillance are understood as the hardware-software providing or technical means using including methods of computer vision for the automated data collection, on the basis of the analysis of stream video (video analysis). Systems of video surveillance can lean on the algorithms of processing of the image and recognition of images allows analyzing video without direct participation of the person.
- Systems of video surveillance are used as a part of intellectual systems of video surveillance and video search.
- Systems of video surveillance, depending on specific goals, can realize a set of functions, for example such as: detection of objects, tracking the movement of objects, classification of objects, identification of objects, detection or recognition of various situations, including disturbing events and etc.
- Tracking the movements of objects by using a set of cameras which are located in a control zone and also a prediction of the camera in the field of which view there can be a selected moving object after leaves area of view of the current camera with which video image is at the moment checked by the operator belongs to tasks which are possible for solving by systems of video surveillance.
- The solution known from the patent U.S. Pat. No. 9,390,332 B2, published in Jul. 12, 2016, in which systems and methods of information (images) processing directed to tracking of the person on a set of cameras are opened and capable to foretell in the field of what view of the following camera there will be a tracked person. The system of information processing is configured for identification of the direction of movement of the object which appeared in the video image of the first video camera from a set of video cameras, definition of set of probabilities and a prediction on the basis of the received set of probabilities of one or several following video cameras for which the probability of emergence of the tracked object is higher, than for other video cameras. A lack of this solution is the impossibility of the select of a concrete object and recognition of its movement, and also lack of highlight of a moving object as areas of interest at recognition of the movement. Besides, the solution is not including a stage of a linking a set of video cameras to location map of an object of video surveillance. The specified shortcomings do not allow reaching the necessary level of accuracy of tracking of a concrete moving object in preset territory.
- The closest solution at the technical level known from the RU 2551128 C1, published on May 20, 2015, which is characterizes device, the system and a method directed to assistance to the controlling person in performance of a problem of tracking of a moving object. The device is configured to display in real time on the device of display of a set of types of display of the taken images made the corresponding cameras, and a set of types of display are located on the image of the map representing a controlled zone, and the device contains the block of installation of the purpose, for the choice and installation of a moving object as the purpose which has to be tracked, the block of the indication of the purpose which on the basis of information of tracking obtained as a result of processing of images specifies a moving object on each of types on which there is a moving object, the prediction block, for a prediction of the following type of display on which a moving object will appear next time, and the block of the indication of a type of display which specifies the following type of display. This solution is characterized the insufficient accuracy of tracking of the moving objects due to lack of an appropriate linking a set of video cameras to location map of an object of video surveillance. Besides, this solution is not applicable for tracking of the moving objects when viewing archival video. Lack of the fact of recognition of the movement and assessment of the direction of the movement of the selected moving object also belongs to shortcomings.
- This technical solution is directed to elimination of the shortcomings inherent in the previous equipment level.
- Technical result of the declared group of inventions is increase in accuracy of tracking of the moving objects.
- The yielded technical result is achieved by what the system for tracking of the moving objects on the video image in real time contains: set of image capture devices; storage device; image-display device; data input device; a set of data processing device configured for performance of the stages including: a linking a set of image capture devices to location map, to create links between objects on location map and the same objects on the video image; display the video image from the first image capture device; select a moving object which is tracked by operator as an object of interest; recognition of the movement of the selected moving object and considering it on the video image as area of interest; assessment of the direction of the movement of the selected moving object; prediction of the second image capture device in the field of which view there can be a selected moving object after leaves area of view of the first image capture device if it continues to move in the same direction, as at the time of an exit from area of view of the first image capture device; display on the image-display device the video image from the second image capture device, right after that as the selected moving object will leave area of view of the first image capture device.
- The specified technical result is also achieved by the system for tracking of the moving objects on the video image when viewing archival video contains: set of image capture devices; storage device; image-display device; data input device; a set of data processing device configured for performance of the stages including: a linking a set of image capture devices to location map by creating links between objects on location map and the same objects on the video image; display of the archival video image from the first image capture device; select a moving object which is tracked by operator as an object of interest; recognition of the movement of the selected moving object and considering it on the video image as area of interest; assessment of the direction of the movement of the selected moving object; prediction of the second image capture device in the field of which view there can be a selected moving object after leaves area of view of the first image capture device; display on the image-display device of the archival video image from the first image capture device, right after that as a moving object was selected by the operator as an object of interest, and display of the archival video image from the second image capture device begins with that time point in which the selected moving object left area of view of the first image capture device.
- The yielded technical result is also achieved due to performance of a method of tracking of the moving objects on the video image, and the method is carried out in real time or when viewing archival video and contains stages at which: linking a set of image capture devices to location map by creating links between objects on location map and the same objects on the video image, and links are set by virtual line segments which one end corresponds to an object arrangement on the video image and other end corresponds to an object arrangement on location map; display the video image from the first image capture device; select a moving object which is tracked by operator as an object of interest; identify the movement of the selected moving object and estimate its direction of the movement; predict the second image capture device in the field of which view there can be a selected moving object after leaves area of view of the first image capture device; displaying on the image-display device of the video image from the second image capture device.
- Also technical result is achieved also due to the performance of a method of tracking of the moving objects on the video image which is carried out in real time or when viewing archival video and containing stages at which: linking a set of image capture devices to location map by creating links between objects on location map and the same objects on the video image; display the video image from the image capture device; select a moving object which is tracked by operator as an object of interest; identify the movement of the selected moving object by definition positions of a moving object on location map from coordinates of the positions received from the image capture device, and determination of coordinates of positions of a moving object is carried out by receiving metadata from the image capture device; after recognition of the movement highlights a moving object on the video image as the area of interest, and area of interest is displayed on the image-display device as the translucent rectangular area imposed on the video image; estimate the direction of the movement of the selected moving object on history of positions of the selected moving object;
- displaying on the image-display device of the video image with the highlighted trajectory of the movement of the selected moving object.
- In one particular case of the declared solution each of a set of data processing device is in addition configured for display on the image-display device of the highlighted trajectory of the movement of the selected moving object.
- In another particular case of the declared solution of link between objects on location map and the same objects on the video image are set by virtual line segments which one end corresponds to an object arrangement on the video image, and other end corresponds to an object arrangement on location map.
- In one particular option only one object can be selected when selecting a moving object.
- In other particular option recognition of the movement of a moving object includes definition of positions of a moving object on location map from coordinates of the positions received from the image capture device.
- In one particular option determination of coordinates of positions of a moving object is carried out by receiving metadata from the image capture device.
- In other particular option assessment of the direction of the movement is carried out on the basis of the history of positions of a moving object kept on the storage device.
- In one particular option location map is the open map of streets (OpenStreetMap) or the building map or the map of the protected object.
- In other particular option the area of interest is translucent rectangular area.
- In one particular option the moving object is a person or group of close going people or the vehicle.
-
FIG. 1 —flowchart of system for tracking of the moving objects; -
FIG. 2 —flowchart of one of options of realization of a method of tracking of the moving objects; -
FIG. 3 —an example of a linking a set of image capture devices to location map by creating links; -
FIG. 4 —an example of work of system of tracking of the moving objects; -
FIG. 5 —flowchart of other option of realization of a method for tracking of the moving objects - The declared technical solution in various options of implementation can be executed in the form of the systems or methods realized by various computer means.
-
FIG. 1 shows a flowchart of system for tracking of the moving objects. The system includes: set of data processing device (10, . . . , 1 n), set of image capture devices (20, . . . , 2 m), data storage device (30), data input device (40) and image-display devices (50). - The image capture device usually is the video camera.
- The processor, the microprocessor, the computer (electronic computer), PLC (programmable logical controller) or integrated circuit configured for execution of certain commands (instructions, programs) for data processing (images) can act as the data processing device.
- Hard drives (HDD), flash memory, ROM (read-only memory), solid-state stores (SSD) and etc. can act as the data storage device.
- The display, the monitor, the video terminal and others can act as the image-display devices.
- The data input device can be represented by the mouse manipulator, the keyboard, the touchpad, the stylus, the joystick, the trackpad and etc.
-
FIG. 2 shows the flowchart of one of options of performance of method of tracking of the moving objects. The specified method can be realized by computer system in two modes: in real time and in browse mode of archival video. - Further stages of performance of a method of tracking of the moving objects, and also basic distinctions of work of systems for tracking the objects in the different modes will be briefly considered.
- 1. Linking a set of image capture devices to location map.
- Location map is a kind of the topographic map or the drawing of the small site of the area in the set scale. In the context of this invention the building map or the protected object, and also the open map of streets (OpenStreetMap) can be location map. Location map is the image in a public format (.jpeg or .png) and can be two-dimensional or three-dimensional. The map can be loaded into system of video surveillance. The possibility of automatic and/or manual scaling and shift is characteristic of location map.
- The linking a set of cameras to the area is carried out by creating links between motionless objects on location map and the same objects on the video image. Various pieces of furniture, office and/or household appliances, doors, window sills and other objects having volume and spatial structure can act as motionless objects. The specified links are set by virtual line segments, one end (point) of which corresponds to an arrangement of a motionless object on the video image, and other end (point) corresponds to an arrangement of a motionless object on location map. The operator sets final points then the system sets virtual line segments. The operator can correct and/or delete the set links.
-
FIG. 3 shows an example of a linking a set of image capture devices to location map by creating four links. - The realization of correlation of the video image with objects on location map can be enabled in the immersion mode. In this mode the operator can “move” between cameras, choosing any camera from the current area of visibility.
- In the immersion mode the video image from the selected video camera is displayed over the image of location map. At the same time on the basis of in advance set linking such foreshortening at which subjects to observation on the video image and on location map will coincide is selected. The operator sees location map and the video image imposed on it on the image-display device.
- 2. Displaying of the video image from the first image capture device.
- Video cameras can be installed in more than one room. At the same time to each camera from a set of cameras there can correspond the location map (that is in any area only one video camera can be installed), or several video cameras are installed in one area and the map of this area with an opportunity to switch between video cameras is used. Video images can be displayed imposed over location map on the display screen.
- 3. Selecting a moving object which is tracked by operator as an object of interest.
- At this stage by using of the data input device the operator selects the moving object which is interesting for him. It is only possible to select one interesting moving object, and the person, an animal, the vehicle or group of close going people can be a moving object.
- 4. Recognition of the movement of the selected moving object and assessment of its direction of the movement.
- Proceeding from placement of video cameras on location map calculation of positions of moving objects from coordinates on the current camera in coordinates on location map is made. In one of particular cases recognition of the movement of a moving object includes definition of positions of a moving object on location map from coordinates of the positions received from the image capture device, and determination of coordinates of positions of a moving object is carried out by receiving metadata from the image capture device. The system receives metadata if the tracker (for example, AxxonSoft) or an external tracker works at the camera in the field of which view a moving object is located. It is necessary for the operator for activation of the specified function by the data input device in a video surveillance window to click on a track of the object of interest, at the same time in a window of video surveillance tracking of objects has to be activated (location definition of the moving object in time by camera where the algorithm analyzes video images and issues the position of moving target objects concerning images).
- Tracking of objects identifies existence of a moving object in the field of view of the camera and dynamically highlights it on the video image with translucent rectangular area as area of interest.
- Difference of the proposed technical solution from the solutions, known from area of the equipment, is that highlighting of a moving object is carried out not at a stage of the choice of an object of interest, but only after the system defines that an object of interest moves. Such solution allows the operator to see more precisely a controlled object; at the same time system is more effective to turn the operator's attention to the movements of objects.
- Location of moving objects in view of the video camera can be defined by video of the analytics which is built in each video camera. The specified location of objects is determined by the sensors fixing various parts of a range (visible, thermal) or sensors, difference by the principle of action from the video camera, such as radars.
- Further, on the basis of the history of positions of a moving object kept on the storage device assessment of the direction of the movement of an object is carried out. History of positions is characteristics of several previous received positions of movements of objects, the speed of the movement, an orientation of the movement on the basis of which one or more data processing device can predict the further direction of the movement of an object of interest.
- 5. A prediction of the second (following) image capture device in the field of which view there can be a selected moving object after leaves area of view of the first (current) image capture device.
- The specified prediction of the camera is carried out in the referral of the movement got at the previous stage. The system counts a further trajectory of the movement of the selected object and defines the camera to which area of view it most likely will get.
- 6. Displaying on the image-display device of the video image from the second image capture device.
- This stage has essential differences when using various browse modes of the video image.
- When viewing the video image in real time display on the image-display device of the video image from the second image capture device is carried out right after the selected moving object leaves area of view of the first image capture device.
- When viewing the archival video image display on the image-display device of the archival video image from the second image capture device is carried out right after a moving object was selected by the operator as an object of interest. And display of the archival video image from the second image capture device begins with that time point in which the selected moving object left area of view of the first image capture device. Such approach reduces time of viewing of archival video and increases monitoring speed with the purpose to monitor movement of the selected object of interest as allows the operator not to watch in the specified browse mode of archival video all archive from the first camera (not to wait when a moving object leaves area of view of the first camera), and at once to pass to viewing of archive from the second camera (the third, etc.), and from that time point when an object of interest has to appear in the field of its view.
-
FIG. 4 shows a concrete example of work of the system of tracking of the moving objects. - Let's consider the room consisting of 5 rooms (zones). In each zone video cameras which survey only the zone, depending on area of view of the video camera are installed. Subject to observation is in a
zone 1 in area of view of theCamera 1. If an object moves to the left and will be gone from a camera view 1 (will enter a door), then on the monitor of the operator theCamera 2 will be displayed. If an object moves to the right and will be gone from a camera view 1 (will enter a door), then on the monitor of the operator theCamera 5 will be displayed. If an object moves up and will be gone from a camera view 1 (will enter a door), then on the monitor of the operator theCamera 3 will be displayed. If an object moves down and will be gone from a camera view 1 (will enter a door), then on the monitor of the operator theCamera 4 will be displayed. -
FIG. 5 shows the flowchart of other option of realization of a method of tracking of the moving objects. - The specified method is carried out by computer means as in real time, and when viewing archival video. Initial stages of this method are similar to the stages which are in detail described above.
- Main distinctive and essential features are that after a stage of recognition of the movement automatic highlighting of a moving object on the video image as area of interest is carried out and the area of interest is displayed on the image-display device as the translucent rectangular area imposed on the video image. Further the direction of the movement of the selected moving object is estimated on stories of positions of the selected moving object and displaying on the image-display device of the video image with the highlighted trajectory of the movement of the selected moving object.
- Detecting of a start of motion is made on a gradient of an interpersonal difference of the video image in time. At the same time the trajectory of the movement for different objects of interest can be highlighted in different colors. The trajectory of the movement is imposed on the video image also, as well as the translucent rectangular area characterizing area of interest. Highlight system of the movement trajectory provides an opportunity to the operator visually to track the direction of the movement of the selected object and if desired by the data input device, to pass independently to viewing of that camera to which area of view the highlighted object of interest goes. It is easily realized solution on condition of correlation of a set of cameras with location map.
- If necessary, at least, a part of various operations considered in the description of this invention can be executed in different ways from presented an order and/or at the same time with each other.
- Though this invention was described in detail for an illustration of the most practical way and preferable options of implementation, it is necessary to understand that this invention is not limited to the opened options of implementation and moreover, is intended for modification and various other combinations of characteristics from the described implementation options. For example, it is necessary to understand that the present invention assumes that in possible degree, one or more characteristics of any option of implementation can be combined with other one or more characteristics of any other option of implementation.
Claims (13)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
RU2017141015A RU2670429C1 (en) | 2017-11-24 | 2017-11-24 | Systems and methods of tracking moving objects on video image |
RU2017141015 | 2017-11-24 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190156495A1 true US20190156495A1 (en) | 2019-05-23 |
US10839533B2 US10839533B2 (en) | 2020-11-17 |
Family
ID=63923475
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/044,332 Active US10839533B2 (en) | 2017-11-24 | 2018-07-24 | Systems and methods of tracking of the moving objects on the video image |
Country Status (3)
Country | Link |
---|---|
US (1) | US10839533B2 (en) |
DE (1) | DE102018118423A1 (en) |
RU (1) | RU2670429C1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180182168A1 (en) * | 2015-09-02 | 2018-06-28 | Thomson Licensing | Method, apparatus and system for facilitating navigation in an extended scene |
US10635925B2 (en) * | 2018-04-10 | 2020-04-28 | Ooo Itv Group | Method and system for display the data from the video camera |
CN112001941A (en) * | 2020-06-05 | 2020-11-27 | 成都睿畜电子科技有限公司 | Piglet supervision method and system based on computer vision |
CN113115216A (en) * | 2021-02-22 | 2021-07-13 | 浙江大华技术股份有限公司 | Indoor positioning method, service management server and computer storage medium |
US20220366575A1 (en) * | 2021-05-14 | 2022-11-17 | Fyma OÜ | Method and system for gathering information of an object moving in an area of interest |
EP4030751A4 (en) * | 2020-07-27 | 2022-11-23 | Huawei Technologies Co., Ltd. | Method, device, and system for video stitching |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2701985C1 (en) * | 2018-12-06 | 2019-10-02 | Общество с ограниченной ответственностью "Ай Ти Ви групп" | System and method of searching objects on trajectories of motion on plan of area |
RU2742582C1 (en) * | 2020-06-25 | 2021-02-08 | Общество с ограниченной ответственностью "Ай Ти Ви групп" | System and method for displaying moving objects on local map |
JP7402784B2 (en) * | 2020-11-13 | 2023-12-21 | 本田技研工業株式会社 | Remote display systems, robots, and display terminals |
RU2755775C1 (en) * | 2021-01-14 | 2021-09-21 | Виктор Михайлович Крылов | Method for automatically generating digital model of protected object, analyzing the security systems of protected object and assessing vulnerability of a protected object, taking into account aforementioned analysis |
CN113377977A (en) * | 2021-06-17 | 2021-09-10 | 深圳市睿联技术股份有限公司 | Video information generation method, device, system and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150015718A1 (en) * | 2013-07-11 | 2015-01-15 | Panasonic Corporation | Tracking assistance device, tracking assistance system and tracking assistance method |
US20160275358A1 (en) * | 2011-06-13 | 2016-09-22 | Tyco Integrated Security, LLC | System to provide a security technology and management portal |
US20160335505A1 (en) * | 2014-04-10 | 2016-11-17 | Smartvue Corporation | Systems and Methods for Automated 3-Dimensional (3D) Cloud-Based Analytics for Security Surveillance in Operation Areas |
US20170094165A1 (en) * | 2004-08-31 | 2017-03-30 | Visual Real Estate, Inc. | Method and apparatus for providing image data of property |
US20170372575A1 (en) * | 2007-05-15 | 2017-12-28 | Ipsotek Ltd | Data processing apparatus to generate an alarm |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2452033C2 (en) * | 2005-01-03 | 2012-05-27 | Опсигал Контрол Системз Лтд. | Systems and methods for night surveillance |
TWI405457B (en) * | 2008-12-18 | 2013-08-11 | Ind Tech Res Inst | Multi-target tracking system, method and smart node using active camera handoff |
US8781293B2 (en) * | 2012-08-20 | 2014-07-15 | Gorilla Technology Inc. | Correction method for object linking across video sequences in a multiple camera video surveillance system |
BR112015008383A2 (en) | 2012-10-18 | 2017-07-04 | Nec Corp | information processing system, method and program for information processing |
JP5506990B1 (en) * | 2013-07-11 | 2014-05-28 | パナソニック株式会社 | Tracking support device, tracking support system, and tracking support method |
JP5999394B2 (en) * | 2015-02-20 | 2016-09-28 | パナソニックIpマネジメント株式会社 | Tracking support device, tracking support system, and tracking support method |
JP6730079B2 (en) * | 2016-04-28 | 2020-07-29 | 東芝テック株式会社 | Monitoring device and program |
-
2017
- 2017-11-24 RU RU2017141015A patent/RU2670429C1/en active
-
2018
- 2018-07-24 US US16/044,332 patent/US10839533B2/en active Active
- 2018-07-30 DE DE102018118423.9A patent/DE102018118423A1/en not_active Withdrawn
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170094165A1 (en) * | 2004-08-31 | 2017-03-30 | Visual Real Estate, Inc. | Method and apparatus for providing image data of property |
US20170372575A1 (en) * | 2007-05-15 | 2017-12-28 | Ipsotek Ltd | Data processing apparatus to generate an alarm |
US20160275358A1 (en) * | 2011-06-13 | 2016-09-22 | Tyco Integrated Security, LLC | System to provide a security technology and management portal |
US20150015718A1 (en) * | 2013-07-11 | 2015-01-15 | Panasonic Corporation | Tracking assistance device, tracking assistance system and tracking assistance method |
US20160335505A1 (en) * | 2014-04-10 | 2016-11-17 | Smartvue Corporation | Systems and Methods for Automated 3-Dimensional (3D) Cloud-Based Analytics for Security Surveillance in Operation Areas |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180182168A1 (en) * | 2015-09-02 | 2018-06-28 | Thomson Licensing | Method, apparatus and system for facilitating navigation in an extended scene |
US11699266B2 (en) * | 2015-09-02 | 2023-07-11 | Interdigital Ce Patent Holdings, Sas | Method, apparatus and system for facilitating navigation in an extended scene |
US10635925B2 (en) * | 2018-04-10 | 2020-04-28 | Ooo Itv Group | Method and system for display the data from the video camera |
CN112001941A (en) * | 2020-06-05 | 2020-11-27 | 成都睿畜电子科技有限公司 | Piglet supervision method and system based on computer vision |
EP4030751A4 (en) * | 2020-07-27 | 2022-11-23 | Huawei Technologies Co., Ltd. | Method, device, and system for video stitching |
CN113115216A (en) * | 2021-02-22 | 2021-07-13 | 浙江大华技术股份有限公司 | Indoor positioning method, service management server and computer storage medium |
US20220366575A1 (en) * | 2021-05-14 | 2022-11-17 | Fyma OÜ | Method and system for gathering information of an object moving in an area of interest |
US11727580B2 (en) * | 2021-05-14 | 2023-08-15 | Fyma OÜ | Method and system for gathering information of an object moving in an area of interest |
Also Published As
Publication number | Publication date |
---|---|
RU2670429C1 (en) | 2018-10-23 |
US10839533B2 (en) | 2020-11-17 |
DE102018118423A1 (en) | 2019-05-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10839533B2 (en) | Systems and methods of tracking of the moving objects on the video image | |
US11343575B2 (en) | Image processing system, image processing method, and program | |
US10148914B2 (en) | Intrusion detection with directional sensing | |
KR100885465B1 (en) | Video tracking system, and method and apparatus for selecting a target in an automated video tracking system | |
JP4852765B2 (en) | Estimating connection relationship between distributed cameras and connection relationship estimation program | |
US10445887B2 (en) | Tracking processing device and tracking processing system provided with same, and tracking processing method | |
JP6806188B2 (en) | Information processing system, information processing method and program | |
US9984300B2 (en) | Image processing system, image processing method, and program | |
JP2008172765A (en) | System and computer implemented method for tracking object using surveillance database | |
US9671493B1 (en) | Automated scheduling of radar-cued camera system for optimizing visual inspection (detection) of radar targets | |
JP7085812B2 (en) | Image processing device and its control method | |
EP1927947A1 (en) | Computer implemented method and system for tracking objects using surveillance database | |
EP2618288A1 (en) | Monitoring system and method for video episode viewing and mining | |
JPWO2014050432A1 (en) | Information processing system, information processing method, and program | |
JP6575845B2 (en) | Image processing system, image processing method, and program | |
KR20170100204A (en) | Apparatus and method for target tracking using 3d bim | |
RU2703152C1 (en) | System and method of displaying objects movement scheme | |
JP2009182624A (en) | Target tracking device | |
WO2020026325A1 (en) | Evaluation device, derivation device, monitoring method, monitoring device, evaluation method, computer program, and derivation method | |
US11070770B2 (en) | Method and system for auto-calibration of multiple sensors for event tracking | |
RU2701985C1 (en) | System and method of searching objects on trajectories of motion on plan of area | |
CN113916233A (en) | Navigation route determining method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
AS | Assignment |
Owner name: OOO ITV GROUP, RUSSIAN FEDERATION Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SUCHKOV, EGOR P.;REEL/FRAME:046714/0562 Effective date: 20180514 Owner name: OOO ITV GROUP, RUSSIAN FEDERATION Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ALTUEV, MURAT K.;REEL/FRAME:046714/0445 Effective date: 20180514 Owner name: OOO ITV GROUP, RUSSIAN FEDERATION Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KALININ, ROMAN V.;REEL/FRAME:046714/0669 Effective date: 20180514 Owner name: OOO ITV GROUP, RUSSIAN FEDERATION Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FALOMKIN, IGOR I.;REEL/FRAME:046714/0616 Effective date: 20180514 Owner name: OOO ITV GROUP, RUSSIAN FEDERATION Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHMELYOV, VADIM V.;REEL/FRAME:046714/0719 Effective date: 20180514 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |