WO2016149938A1 - 视频监控方法、视频监控系统以及计算机程序产品 - Google Patents

视频监控方法、视频监控系统以及计算机程序产品 Download PDF

Info

Publication number
WO2016149938A1
WO2016149938A1 PCT/CN2015/075126 CN2015075126W WO2016149938A1 WO 2016149938 A1 WO2016149938 A1 WO 2016149938A1 CN 2015075126 W CN2015075126 W CN 2015075126W WO 2016149938 A1 WO2016149938 A1 WO 2016149938A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
video
determining
coordinate system
video data
Prior art date
Application number
PCT/CN2015/075126
Other languages
English (en)
French (fr)
Inventor
俞刚
李超
尚泽远
Original Assignee
北京旷视科技有限公司
北京小孔科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京旷视科技有限公司, 北京小孔科技有限公司 filed Critical 北京旷视科技有限公司
Priority to US14/888,357 priority Critical patent/US9792505B2/en
Priority to CN201580000334.1A priority patent/CN105519102B/zh
Priority to PCT/CN2015/075126 priority patent/WO2016149938A1/zh
Publication of WO2016149938A1 publication Critical patent/WO2016149938A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection

Definitions

  • the present disclosure relates to the field of video surveillance, and more particularly, to a depth video based video surveillance method, a video surveillance system, and a computer program product.
  • the purpose of intelligent monitoring is to automatically track pedestrians in the video scene based on image data, and analyze and analyze the characteristics and behavior of each pedestrian.
  • smart monitoring often relies solely on traditional non-depth cameras (RGB cameras). Due to the limitation of the camera itself, the accuracy of pedestrian tracking is not accurate, and it is limited by the pedestrian movement posture in the scene, so the pedestrian-based feature analysis effect is not expected.
  • Depth cameras (depth cameras) have been widely used in application scenarios such as human-computer interaction, but there are no mature systems and methods that can be extended to the field of intelligent monitoring. In particular, the existing various monitoring systems are unable to achieve accurate analysis of pedestrian characteristics (such as height, weight, speed of movement) and effective detection of pedestrian abnormal behavior.
  • the present disclosure has been made in view of the above problems.
  • the present disclosure provides a video monitoring method based on depth video, a video monitoring system, and a computer program product, which can track pedestrians in a scene at high speed and effectively analyze feature information of pedestrians in real time, thereby realizing statistical analysis of the scene. And abnormal situation monitoring.
  • a video monitoring method comprising: acquiring video data acquired via a video capture device; determining an object as a monitoring target based on the video data; and extracting feature information of the object;
  • the video data is video data including depth information.
  • the video monitoring method further includes: configuring the view Frequency acquisition device and determining coordinate parameters of the video capture device.
  • determining a coordinate parameter of the video capture device includes: selecting a plurality of reference points on a predetermined reference plane; determining, based on coordinate information of the plurality of reference points Determining a transformation relationship between a camera coordinate system of the video capture device and a world coordinate system; and determining coordinate parameters of the video capture device based on the transformation relationship.
  • a video monitoring method includes: determining background information in the video data; determining, based on the background information, The foreground information in each frame of the video data; the edge contour information corresponding to the foreground region of the foreground information; and the object is determined based on the edge contour information.
  • determining the object based on the edge contour information includes: acquiring a candidate block based on the edge contour information; and determining the candidate larger than a first predetermined threshold The block is a candidate object; and the evaluation value of the candidate object is acquired based on a predetermined algorithm, and the candidate object whose evaluation value is greater than a second predetermined threshold is determined to be the object.
  • determining the object based on the edge contour information further comprises: matching each of the objects determined in a previous frame and a current frame to determine to leave The object of the previous frame.
  • a video monitoring method wherein the feature information of the object includes: body type information of the object and a moving speed.
  • a video monitoring method wherein extracting the body type information of the object includes: selecting a closest point of the object from the video capture device as a head point of the object; based on the video Determining a coordinate relationship between a camera coordinate system of the acquisition device and the world coordinate system, determining a coordinate parameter of the head point in the world coordinate system; and determining the object based on the coordinate parameter of the head point in the world coordinate system The distance of the head from the ground is used as the height information of the object.
  • extracting the body type information of the object includes: selecting a closest point of the object from the video collection device as a head point of the object; selecting the object a point of a maximum value on a longitudinal axis of the image coordinate system as a sole point of the object; determining a head of the object in the world coordinate system based on a transformation relationship between a camera coordinate system of the video capture device and a world coordinate system The distance between the point and the sole point of the object The height information as the object.
  • the video monitoring method further includes:
  • extracting the body type information of the object further comprises: extracting height information and contour information of the object; and determining height information, contour information, and weight based on different objects collected in advance Correspondence between the information, the weight information of the object is determined according to the height information and the outline information of the object.
  • the video monitoring method further includes: analyzing the feature information to determine an abnormal event of the object, wherein analyzing the feature information, determining that the abnormal event of the object includes: at a predetermined time An abnormal event of the object is determined when the change in the body shape information of the object in the segment is greater than a predetermined third threshold and/or the moving speed of the object is greater than a fourth threshold.
  • a video surveillance system comprising: a processor; a memory; and computer program instructions stored in the memory, executed when the computer program instructions are executed by the processor The following steps: acquiring video data collected via a video capture device; determining, based on the video data, an object that is a monitoring target; and extracting feature information of the object; wherein the video data is video data including depth information.
  • a video surveillance system further includes the video capture device for acquiring the video data.
  • a video surveillance system wherein when the computer program instructions are executed by the processor, the step of: selecting a plurality of reference points on a predetermined reference plane; The coordinate information of the reference point determines a transformation relationship between the camera coordinate system of the video capture device and the world coordinate system; and determines a coordinate parameter of the video capture device based on the transformation relationship.
  • a video monitoring system wherein the step of determining an object as a monitoring target based on the video data performed when the computer program instruction is executed by the processor comprises: determining Determining background information in the video data; determining foreground information in each frame of the video data based on the background information; acquiring a foreground area corresponding to the foreground information Edge contour information of the domain; and determining the object based on the edge contour information.
  • a video surveillance system wherein the determining the object based on the edge contour information performed when the computer program instructions are executed by the processor comprises: Edge candidate information, acquiring a candidate block; determining the candidate block that is greater than a first predetermined threshold as a candidate object; and acquiring an evaluation value of the candidate object based on a predetermined algorithm, determining the candidate whose evaluation value is greater than a second predetermined threshold The object is the object.
  • a video surveillance system according to another embodiment of the present disclosure, wherein the step of determining the object based on the edge contour information performed when the computer program instruction is executed by the processor further comprises: matching The previous frame and each of the objects determined in the current frame are determined to determine the object leaving the previous frame.
  • a video monitoring system wherein the feature information of the object includes: body type information of the object and a moving speed.
  • a video surveillance system wherein the extracting the body type information of the object performed when the computer program instruction is executed by the processor comprises: selecting the object distance to the a closest point of the video capture device as a head point of the object; determining a coordinate parameter of the head point in the world coordinate system based on a transformation relationship between a camera coordinate system of the video capture device and a world coordinate system; The coordinate parameter of the head point in the world coordinate system determines the distance of the head of the object from the ground as the height information of the object.
  • a video surveillance system wherein the extracting the body type information of the object performed when the computer program instruction is executed by the processor comprises: selecting the object distance to the a closest point of the video capture device as a head point of the object; a point at which a maximum value of the object on a vertical axis of the image coordinate system is selected as a sole point of the object; a camera coordinate system based on the video capture device The transformation relationship with the world coordinate system determines the distance between the head point of the object and the sole point of the object in the world coordinate system as the height information of the object.
  • a video monitoring system wherein the extracting the feature information of the object performed when the computer program instruction is executed by the processor comprises: calculating a first of the object a moving distance of the fixed point between the first selected frame and the second selected frame in the world coordinate system; based on a time interval between the first selected frame and the second selected frame and the motion The distance determines the moving speed of the object.
  • a video monitoring method wherein the extracting the body type information of the object performed when the computer program instruction is executed by the processor further comprises: extracting a height of the object Information and contour information; determining weight information of the object based on the height information and the contour information of the object based on the height information of the different objects collected in advance, the correspondence between the contour information and the weight information.
  • a video surveillance system wherein when the computer program instructions are executed by the processor, performing: analyzing the feature information, determining an abnormal event of the object, and wherein The step of determining an abnormal event of the object performed by the computer program instruction being executed by the processor comprises: when a change in the body shape information of the object is greater than a predetermined third threshold and/or of the object within a predetermined time period When the moving speed is greater than the fourth threshold, an abnormal event of the object is determined.
  • a computer program product comprising a computer readable storage medium on which computer program instructions are stored, the computer program instructions being executed while being executed by a computer The following steps: acquiring video data collected via a video capture device; determining, based on the video data, an object that is a monitoring target; and extracting feature information of the object; wherein the video data is video data including depth information.
  • FIG. 1 is a flowchart illustrating a video monitoring method according to an embodiment of the present invention
  • FIG. 2 is a functional block diagram illustrating a video surveillance system in accordance with an embodiment of the present invention
  • FIG. 3 is a flow chart further illustrating configuring and determining parameters of a video capture device in a video surveillance method in accordance with an embodiment of the present invention
  • FIG. 5 is a flowchart further illustrating determining an object as a monitoring target in a video monitoring method according to an embodiment of the present invention
  • FIG. 6 is a schematic diagram illustrating determining foreground information in a video monitoring method according to an embodiment of the present invention
  • FIG. 7 is a flowchart further illustrating a first example of determining height information of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 8 is a flowchart further illustrating a second example of determining height information of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram illustrating determination of height information of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 10 is a flowchart further illustrating determining a moving speed of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 11 is a diagram illustrating determining a moving speed of an object in a video monitoring method according to an embodiment of the present invention
  • FIG. 12 is a flowchart further illustrating determining weight information of an object in a video monitoring method according to an embodiment of the present invention
  • FIG. 13 is a flowchart further illustrating determining an abnormal event of an object in a video monitoring method according to an embodiment of the present invention
  • FIG. 14 is a schematic block diagram illustrating a video monitoring system according to an embodiment of the present invention.
  • FIG. 1 is a flow chart illustrating a video monitoring method according to an embodiment of the present invention. As shown in Figure 1, A video monitoring method according to an embodiment of the present invention includes the following steps.
  • step S101 video data acquired via the video capture device is acquired.
  • the video capture device is a depth camera (depth camera) capable of acquiring depth information of a subject.
  • Acquiring video data collected via the video capture device includes, but is not limited to, receiving video data transmitted from the video capture device via a wired or wireless method after the video capture device configured by the physical location is separately configured to acquire video data.
  • the video capture device may be physically co-located with other modules or components in the video surveillance system or even within the same chassis, and other modules or components in the video surveillance system are received from the video capture device via the internal bus. Video data. Thereafter, the processing proceeds to step S102.
  • an object that is a monitoring target is determined based on the video data.
  • the object to be monitored is a pedestrian and/or other object recorded in the video data, such as a vehicle, an animal, or the like.
  • Determining the object as the monitoring target generally includes performing background modeling on the depth video data, then determining foreground information in each frame according to the background data, and obtaining edge contour information of the foreground region based on a predetermined algorithm, thereby based on the contour of each foreground region To determine if it is the object to be monitored.
  • the processing proceeds to step S103.
  • step S103 feature information of the object is extracted.
  • the feature information of the object includes, but is not limited to, at least one of a height, a weight, a moving speed, and the like of the object.
  • the video monitoring method uses a depth camera/camera as a video capture device, thereby realizing object detection and feature information extraction based on depth video information, and providing accurate and efficient for further statistical analysis and abnormal situation monitoring. Data guarantee.
  • the video monitoring system 20 includes a video data acquiring module 21, a monitoring object determining module 22, a feature information extracting module 23, and a feature information analyzing module 24.
  • the video data acquisition module 21, the monitoring object determination module 22, the feature information extraction module 23, and the feature information analysis module 24 may be configured, for example, by hardware (server, dedicated computer, etc.), software, firmware, and any feasible combination thereof.
  • the video data acquiring module 21 is configured to acquire video data.
  • the video data acquisition module 21 may include a video capture device of a depth camera (depth camera) capable of acquiring depth information of a subject.
  • the video capture device may be physically separated from the subsequent monitoring object determining module 22, the feature information extracting module 23, and the feature information analyzing module 24, or physically located at the same location or even inside the same chassis.
  • the video data acquisition module 21 further transmits the video via a wired or wireless manner.
  • the depth video data acquired by the acquisition device is sent to the subsequent module.
  • the video data acquiring module 21 is internally The bus transmits the depth video data acquired by the video capture device to a subsequent module.
  • the video data may include depth video data as well as color video data. More specifically, the three-dimensional positional parameter of each pixel in the video data can be determined according to the positional parameters of the video capture device and the depth information value of each pixel in the video data.
  • its predetermined format can be encoded and compressed into video data packets to reduce the amount of traffic and bandwidth required for transmission.
  • the monitoring object determining module 22 is configured to determine an object that is a monitoring target based on the video data acquired from the video data acquiring module 21.
  • the monitoring object determining module 22 performs background modeling on the depth video data, for example, and then determines foreground information in each frame according to the background data, and obtains edge contour information of the foreground region based on a predetermined algorithm, thereby based on the contour of each foreground region. Determine if it is the object to be monitored. More specifically, the monitored object determination module 22 can detect whether each of the frames of the video is stored in a pedestrian and determine the specific location of each pedestrian. Matching associations are performed for pedestrians in different frames to achieve preparation tracking for pedestrians in different frames.
  • the feature information extraction module 23 is configured to extract feature information of each object to be monitored based on the video data acquired from the video data acquiring module 21 for the object to be monitored determined by the monitoring object determining module 22 Such as the height, weight, speed of movement, etc. of the subject.
  • the feature information analysis module 24 is configured to perform storage, statistical analysis, and monitoring of abnormal conditions on the feature information extracted by the feature information extraction module 23. More specifically, the stored data includes, but is not limited to, the position of the pedestrian at each frame, the feature information of each pedestrian, the specific time of each frame of the image, and the like.
  • the feature information analysis module 24 analyzes feature information in real time to monitor possible occurrences. Abnormal conditions, and the long-term storage of feature information is organized and statistically extracted. For example, at which time a pedestrian appears more frequently, the pedestrian's average height, weight and other characteristics.
  • FIGS. 3 and 4 are flow charts further illustrating configuring and determining parameters of a video capture device in a video surveillance method in accordance with an embodiment of the present invention.
  • 4 is a schematic diagram illustrating a camera coordinate system and a world coordinate system for determining parameters of a video capture device.
  • the flow of configuring and determining parameters of the video collection device in the video monitoring method according to the embodiment of the present invention includes the following steps.
  • the video capture device is configured.
  • a depth camera (depth camera) as the video capture device is installed in a scene to be monitored.
  • the depth camera (depth camera) is mounted at a height of 2-3.5 meters with a viewing angle looking down on the ground (shown schematically in Figure 4).
  • the video capture device may be a single depth camera (ie, only a depth camera lens) or a deep color dual lens camera. In the case of a deep color dual lens camera, the camera needs to be calibrated so that the images obtained by the two lenses are corresponding and synchronized.
  • the process proceeds to step S302 to determine coordinate parameters such as the actual height and angle from the reference plane for the video capture device after installation.
  • step S302 a plurality of reference points on a predetermined reference plane are selected.
  • the predetermined reference plane may be a ground plane, and the greater the number of selected reference points (for example, 5 or more), the higher the accuracy. Thereafter, the processing proceeds to step S303.
  • step S303 based on the coordinate information of the selected plurality of reference points, the conversion relationship between the camera coordinate system of the video capture device and the world coordinate system is determined.
  • the Cartesian coordinate system composed of the points Oc and Xc, Yc, and Zc axes is a camera coordinate system.
  • a world coordinate system is introduced, and a Cartesian coordinate system composed of points Ow and Xw, Yw, and Zw axes is a world coordinate system.
  • the transformation matrix of the camera coordinate system to the world coordinate system that is, the transformation relationship between the camera coordinate system and the world coordinate system, can be estimated based on the least square method by selecting a plurality of reference points. Thereafter, the processing proceeds to step S304.
  • a coordinate parameter of the video capture device is determined based on the transformation relationship.
  • coordinate parameters such as the actual height and angle of the video capture device can be determined.
  • the complete ground plane position in the video scene can be determined.
  • FIG. 5 is a flowchart further illustrating determining an object as a monitoring target in a video monitoring method according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram illustrating determining foreground information in a video monitoring method according to an embodiment of the present invention.
  • the flow of determining an object as a monitoring target in the video monitoring method includes the following steps.
  • step S501 background information in the video data is determined.
  • a background frame without any monitoring object may be captured as background information, as shown in FIG. 6(A) as a background frame 601 as background information.
  • the process proceeds to step S502.
  • step S502 foreground information in each frame of the video data is determined based on the background information.
  • the current frame 602 of the video data is determined as foreground information by comparing the background frame 601 with the current frame 602 by depth difference.
  • the foreground area 603 is as shown in Fig. 6(C). Thereafter, the processing proceeds to step S503.
  • step S503 edge contour information corresponding to the foreground region of the foreground information is acquired.
  • the edge contour information of the foreground region is obtained using an edge detection algorithm based on the foreground region determined in step S502. Thereafter, the processing proceeds to step S504.
  • step S504 a candidate block is acquired based on the edge contour information.
  • a two-layer contour is used to obtain a plurality of candidate blocks of the foreground. Thereafter, the processing proceeds to step S505.
  • step S505 the candidate block that is greater than the first predetermined threshold is determined to be a candidate object.
  • the first predetermined threshold is a minimum acceptable area of the candidate block, and the candidate block whose area is not greater than the first predetermined threshold may be generated due to noise, thereby eliminating the area Too small candidate blocks, each of the remaining candidate blocks represents a candidate object. Thereafter, the processing proceeds to step S506.
  • an evaluation value of the candidate object is acquired based on a predetermined algorithm, and the candidate object whose evaluation value is greater than a second predetermined threshold is determined to be the object.
  • each candidate object may be evaluated based on a pedestrian detection algorithm and/or a human head detection algorithm of a color image, and a candidate whose evaluation value is greater than a second predetermined threshold is determined as the object (eg, determined as a pedestrian).
  • the pedestrian detection algorithm is applied, all possible rectangular frames in the depth video frame are traversed, each rectangular frame representing a candidate region of a pedestrian.
  • the image features of the gradient histogram are extracted, and then classified according to the use of the support vector machine to determine whether there is a pedestrian in the rectangular region.
  • the human head detection algorithm all possible rectangular frames in the depth video frame are traversed, and for each rectangular frame, color and texture features are extracted, and then the trained support vector machine is used to determine whether there is a human head in the region. Thereafter, the processing proceeds to step S507.
  • each of the objects determined in the previous frame and the current frame is matched to determine an object that leaves the previous frame.
  • the matching criterion may be the travel distance of the pedestrian and the color texture information extracted on the candidate object.
  • the tracking result of the same pedestrian on different frames can be obtained. For example, suppose there are M detected pedestrians in the t-1th frame, and there are N detected pedestrians in the tth frame, then the pedestrian detected in the t-1th frame and the detected in the tth frame are required.
  • the distance between each pedestrian which can be defined as the spatial position distance between the head points of the pedestrian in two frames.
  • the pedestrian detected by the t-1th frame is considered to match the pedestrian with the smallest distance in the t-th frame. If the minimum distance is greater than the predetermined threshold, then the pedestrian detected by the t-1th frame is considered to have left the tth frame, so that the pedestrian is no longer tracked.
  • FIG. 7 is a flowchart further illustrating a first example of determining height information of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 8 is a flowchart further illustrating a second example of determining the height information of an object in the video monitoring method according to an embodiment of the present invention.
  • FIG. 9 is a diagram illustrating determining height information of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 10 is a flowchart further illustrating determining a moving speed of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 11 is a diagram illustrating determining a moving speed of an object in a video monitoring method according to an embodiment of the present invention.
  • FIG. 12 is a flowchart further illustrating determining weight information of an object in a video monitoring method according to an embodiment of the present invention.
  • the flow of the first example of determining the height information of an object in the video monitoring method according to an embodiment of the present invention includes the following steps.
  • step S701 the closest point of the object to the video capture device is selected as the pair.
  • the head of the elephant In one embodiment of the present invention, as shown in FIG. 9, the closest point of the object to the video capture device is selected. Since the pedestrian is upright, the closest point can be assumed to be the head point 901. Thereafter, the processing proceeds to step S702.
  • a coordinate parameter of the head point in the world coordinate system is determined based on a transformation relationship between the camera coordinate system of the video capture device and the world coordinate system. As already described above with reference to Figures 3 and 4, the coordinate parameters of the head point in the camera coordinate system of the video capture device can be converted to coordinate parameters of the world coordinate system. Thereafter, the processing proceeds to step S703.
  • step S703 based on the coordinate parameter of the head point in the world coordinate system, the distance of the head of the object from the ground is determined as the height information of the object.
  • the distance of the head point 901 from the ground is determined to be the height 902 of the object.
  • the method of determining the height information of the object in the video monitoring method according to the embodiment of the present invention is not limited to the first example as shown in FIG. As shown in FIG. 8, the flow of the second example of determining the height information of the object in the video monitoring method according to the embodiment of the present invention includes the following steps.
  • step S801 similar to step S701, the closest point of the object from the video capture device is selected as the head point of the object. Thereafter, the processing proceeds to step S802.
  • step S802 a point of the maximum value of the object on the vertical axis of the image coordinate system is selected as the sole point of the object.
  • the sole point may be defined as the point at which the foreground region of the pedestrian is at a maximum on the longitudinal axis of the image coordinate system.
  • step S803 based on a transformation relationship between the camera coordinate system of the video capture device and the world coordinate system, determining a distance between a head point of the object and a sole point of the object in the world coordinate system as the The height information of the object.
  • the feature information of the object further includes the moving speed of the object, the weight information, and the like.
  • the flow of determining the moving speed of an object in the video monitoring method according to an embodiment of the present invention includes the following steps.
  • step S1001 a moving distance of the first fixed point of the object between the first selected frame and the second selected frame in the world coordinate system is calculated.
  • the first fixed point may be a head point of the object.
  • the invention is not limited thereto, and the first fixed point may also be a sole point of the object or any fixed point on the object.
  • the first selected frame is a Tth frame and the second selected frame is a T+t frame.
  • the object is in the first In a selected frame is an object 1101, and in a second selected frame is an object 1102.
  • the moving distance of the first fixed point between the first selected frame T frame and the second selected frame T+t frame in the world coordinate system is a moving distance 1103. Thereafter, the process proceeds to step S1002.
  • the flow of determining the weight information of an object in the video monitoring method according to an embodiment of the present invention includes the following steps.
  • step S1201 the height information and the outline information of the object are extracted. Thereafter, the process proceeds to step S1202.
  • the weight information of the object is determined based on the height information and the contour information of the object based on the correspondence relationship between the height information, the contour information, and the weight information of the different objects collected in advance.
  • a data set containing pedestrians of different heights and weights is first collected. Thereafter, corresponding contour information for each pedestrian at different depths under the depth camera is acquired. For each contour area, normalize according to the depth distance at which the contour is located. Then, the heights of all pedestrians are quantified (eg, quantized into multiple height intervals), and all data falling within the same interval are fitted to establish a normalized contour area to body weight relationship.
  • the model according to the corresponding relationship is obtained according to the linear model of the region where the pedestrian is located and the contour area of the normalized pedestrian. Weight value.
  • FIG. 13 is a flowchart further illustrating determining an abnormal event of an object in a video monitoring method according to an embodiment of the present invention.
  • the flow of determining an abnormal event of an object in the video monitoring method includes the following steps. Among them, steps S1301 through S1303 are the same as steps S101 to S103 described with reference to FIG. 1, and a repetitive description thereof will be omitted.
  • step S1304 the feature information is analyzed.
  • the analysis of the feature information can be controlled and performed by the feature information analysis module 24 described above.
  • analyzing the feature information comprises analyzing the acquired object body shape information and the moving speed of the object. Thereafter, the process proceeds to step S1305.
  • step S1305 it is determined whether the change in the body shape information of the object is greater than a predetermined threshold. In one embodiment of the present invention, if the change in the height information of the pedestrian compared to the previous frame is greater than a predetermined threshold (for example, 1 meter), it may be determined that an abnormal event such as a pedestrian falls occurs. If an affirmative result is obtained in step S1305, the processing proceeds to step S1307.
  • a predetermined threshold for example, 1 meter
  • step S1307 it is judged that an abnormal event has occurred.
  • an alert to the monitoring personnel of the video surveillance system that an abnormal event has occurred may be issued in an appropriate manner.
  • step S1305 If a negative result is obtained in step S1305, the processing proceeds to step S1306.
  • step S1306 it is continued to determine whether the moving speed of the object is greater than a predetermined threshold. In one embodiment of the present invention, if the moving speed of the pedestrian suddenly exceeds a predetermined threshold (for example, 4 meters/second), it can be judged that an abnormal event has occurred. If an affirmative result is obtained in step S1306, the processing proceeds to step S1307. Conversely, if an affirmative result is obtained in step S1306, it is determined that based on the analysis of the feature information of step S1304, no abnormal event occurs, and the process returns to step S1301. Thereby, the video data collected via the video capture device is continuously acquired in real time and the monitoring is continued.
  • a predetermined threshold for example, 4 meters/second
  • steps S1301 to S1307 are only one exemplary flow of determining an abnormal event of an object.
  • only the moving speed of the object may be monitored, or only the change of the body shape information of the object may be monitored, and the change of the body shape information of the object and the moving speed of the object may be monitored at the same time, and the monitoring may be performed first.
  • the movement speed of the object re-monitoring the change of the body shape information of the object is not limited herein.
  • FIG. 14 is a schematic block diagram illustrating a video monitoring system according to an embodiment of the present invention.
  • a video surveillance system in accordance with an embodiment of the present invention includes a processor 141, a memory 142, and computer program instructions 143 stored in the memory 142.
  • the computer program instructions 143 may implement the functions of the various functional modules of the video surveillance system in accordance with embodiments of the present invention while the processor 141 is running, and/or may perform various steps of the video surveillance method in accordance with an embodiment of the present invention.
  • the following steps are performed: acquiring video data collected via a video capture device; determining an object as a monitoring target based on the video data; and extracting the object Characteristic information; wherein the video data is video data including depth information.
  • the following steps are further performed: selecting a plurality of reference points on a predetermined reference plane; determining the video capture device based on coordinate information of the plurality of reference points Transformation relationship between the camera coordinate system and the world coordinate system; Deriving a transformation relationship to determine coordinate parameters of the video capture device.
  • the step of determining, as the monitoring target, based on the video data, when the computer program instructions 143 are executed by the processor 141, includes: determining background information in the video data; based on the background Information, determining foreground information in each frame of the video data; acquiring edge contour information corresponding to the foreground region of the foreground information; and determining the object based on the edge contour information.
  • the step of determining the object based on the edge contour information when the computer program instructions 143 are executed by the processor 141, includes: acquiring a candidate block based on the edge contour information; determining that is greater than the first The candidate block of the predetermined threshold is a candidate object; and the evaluation value of the candidate object is acquired based on a predetermined algorithm, and the candidate object whose evaluation value is greater than a second predetermined threshold is determined to be the object.
  • the step of determining the object based on the edge contour information performed when the computer program instructions 143 are executed by the processor 141 further comprises: matching the previous frame with the object determined in the current frame Each one to determine the object that left the previous frame.
  • the feature information of the object includes: body shape information of the object and a moving speed.
  • the step of extracting the body shape information of the object performed when the computer program instructions 143 are executed by the processor 141 includes: selecting a closest point of the object from the video capture device as a header of the object a coordinate point based on a transformation relationship between a camera coordinate system of the video capture device and a world coordinate system, determining a coordinate parameter of the head point in a world coordinate system; and a coordinate based on the head point in a world coordinate system A parameter determining a distance of a head of the object from the ground as height information of the object.
  • the step of extracting the body shape information of the object performed when the computer program instructions 143 are executed by the processor 141 includes: selecting a closest point of the object from the video capture device as a header of the object a point of selecting a maximum value of the object on a vertical axis of the image coordinate system as a sole point of the object; determining a world coordinate based on a transformation relationship between a camera coordinate system of the video capture device and a world coordinate system The distance between the head point of the object and the sole point of the object in the system is used as the height information of the object.
  • the step of extracting feature information of the object performed when the computer program instructions 143 are executed by the processor 141 comprises calculating a first fixed point of the object in a first selection in a world coordinate system a moving distance between the frame and the second selected frame; determining a moving speed of the object based on a time interval between the first selected frame and the second selected frame and the moving distance degree.
  • the step of extracting the body shape information of the object performed when the computer program instructions 143 are executed by the processor 141 further comprises: extracting height information and contour information of the object; based on different objects collected in advance The correspondence between the height information, the outline information, and the weight information determines the weight information of the object based on the height information and the outline information of the object.
  • the method further comprises: analyzing the feature information, determining an abnormal event of the object, and wherein the computer program instructions are executed by the processor Determining the abnormal event of the object includes determining the object when a change in the body shape information of the object is greater than a predetermined third threshold within a predetermined time period and/or a moving speed of the object is greater than a fourth threshold Anomalous event.
  • Modules in a video surveillance system in accordance with embodiments of the present invention may be implemented by computer programs stored in a memory in a video surveillance system in accordance with an embodiment of the present invention, or may be in a computer in accordance with an embodiment of the present invention
  • the computer instructions stored in the computer readable storage medium of the program product are implemented by the computer when executed.
  • the computer readable storage medium can be any combination of one or more computer readable storage media, for example, a computer readable storage medium includes computer readable program code for randomly generating a sequence of action instructions, and another computer can The read storage medium contains computer readable program code for performing face activity recognition.
  • the computer readable storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet, a hard disk of a personal computer, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory. (EPROM), Portable Compact Disk Read Only Memory (CD-ROM), USB memory, or any combination of the above storage media.
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read only memory
  • CD-ROM Portable Compact Disk Read Only Memory
  • USB memory or any combination of the above storage media.

Abstract

本公开涉及基于深度视频的视频监控方法、视频监控系统以及计算机程序产品。一种视频监控方法，包括：获取经由视频采集装置采集的视频数据；基于所述视频数据，确定作为监控目标的对象；以及提取所述对象的特征信息；其中，所述视频数据为包含深度信息的视频数据。

Description

视频监控方法、视频监控系统以及计算机程序产品 技术领域
本公开涉及视频监控领域，更具体地，本公开涉及基于深度视频的视频监控方法、视频监控系统以及计算机程序产品。
背景技术
当前的图像或者视频监控往往需要依赖于工作人员人工的检测和处理。虽然越多来多的场景(例如机场、车站、商场、街道等)有摄像头覆盖，但是因为监控系统自身无法实现对行人特征的分析和跟踪，所以需要很多人力来进行处理和监视。这样需要配备大量的人力去进行监控和管理，并且随着摄像头规模进一步扩大后，难以高效的处理和反应一些突发事件。
智能监控的目的是根据图像数据，自动跟踪视频场景中的行人，并且对每个行人的特点和行为做分析处理。目前，智能监控往往仅仅依赖于传统非深度相机(RGB摄像机)。由于相机本身的限制，导致行人跟踪的精度不准，并且受限于场景中的行人动作姿态，所以基于行人的特征分析效果达不到预期。深度相机(深度摄像机)目前已被广泛用于人机交互等应用场景，但目前还不存在成熟的系统和方法能将其推广到智能监控领域。特别地，现有的各种监控系统，均无法实现对行人的特征(比如身高、体重、运动速度)的准确分析和对行人异常行为的有效检测。
发明内容
鉴于上述问题而提出了本公开。本公开提供了一种基于深度视频的视频监控方法、视频监控系统以及计算机程序产品，其能够高速有效地跟踪场景中的行人，并且准确实时地分析行人的特征信息，从而实现对于场景的统计分析以及异常情况监控。
根据本公开的一个实施例，提供了一种视频监控方法，包括：获取经由视频采集装置采集的视频数据；基于所述视频数据，确定作为监控目标的对象；以及提取所述对象的特征信息；其中，所述视频数据为包含深度信息的视频数据。
此外，根据本公开的一个实施例的视频监控方法，还包括：配置所述视 频采集装置，并且确定所述视频采集装置的坐标参数。
此外，根据本公开的一个实施例的视频监控方法，其中确定所述视频采集装置的坐标参数包括：选择预定基准面上的多个基准点；基于所述多个基准点的坐标信息，确定所述视频采集装置的相机坐标系统与世界坐标系统的变换关系；基于所述变换关系，确定所述视频采集装置的坐标参数。
此外，根据本公开的一个实施例的视频监控方法，其中，所述基于所述视频数据，确定作为监控目标的对象包括：确定所述视频数据中的背景信息；基于所述背景信息，确定所述视频数据的每帧中的前景信息；获取对应于所述前景信息的前景区域的边缘轮廓信息；以及基于所述边缘轮廓信息，确定所述对象。
此外，根据本公开的一个实施例的视频监控方法，其中，基于所述边缘轮廓信息，确定所述对象包括：基于所述边缘轮廓信息，获取候选块；确定大于第一预定阈值的所述候选块为候选对象；以及基于预定算法获取所述候选对象的评估值，确定所述评估值大于第二预定阈值的所述候选对象为所述对象。
此外，根据本公开的一个实施例的视频监控方法，其中，基于所述边缘轮廓信息，确定所述对象还包括：匹配前一帧与当前帧中确定的所述对象的每一个，以确定离开前一帧的对象。
此外，根据本公开的一个实施例的视频监控方法，其中所述对象的特征信息包括：对象的体型信息以及移动速度。
此外，根据本公开的一个实施例的视频监控方法，其中提取所述对象的体型信息包括：选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定所述头部点在世界坐标系统中的坐标参数；以及基于所述头部点在世界坐标系统中的坐标参数，确定所述对象的头部距离地面的距离作为所述对象的身高信息。
此外，根据本公开的一个实施例的视频监控方法，其中提取所述对象的体型信息包括：选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；选取所述对象在图像坐标系的纵轴上的最大值的点作为所述对象的脚底点；基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定在世界坐标系统中所述对象的头部点和所述对象的脚底点之间的距 离作为所述对象的身高信息。
此外，根据本公开的一个实施例的视频监控方法，还包括：
计算所述对象的第一固定点在世界坐标体系中在第一选定帧与第二选定帧之间的运动距离；
基于所述第一选定帧与所述第二选定帧之间的时间间隔以及所述运动距离，确定所述对象的移动速度。
此外，根据本公开的一个实施例的视频监控方法，其中提取所述对象的体型信息还包括：提取所述对象的身高信息和轮廓信息；基于预先采集的不同对象的身高信息、轮廓信息与体重信息之间的对应关系，根据所述对象的身高信息和轮廓信息确定所述对象的体重信息。
此外，根据本公开的一个实施例的视频监控方法，还包括：分析所述特征信息，确定所述对象的异常事件，其中分析所述特征信息，确定所述对象的异常事件包括：在预定时间段内所述对象的体型信息的变化大于预定第三阈值时和/或所述对象的移动速度大于第四阈值时，确定所述对象的异常事件。
根据本公开的另一个实施例，提供了一种视频监控系统，包括：处理器；存储器；和存储在所述存储器中的计算机程序指令，在所述计算机程序指令被所述处理器运行时执行以下步骤：获取经由视频采集装置采集的视频数据；基于所述视频数据，确定作为监控目标的对象；以及提取所述对象的特征信息；其中，所述视频数据为包含深度信息的视频数据。
此外，根据本公开的另一个实施例的视频监控系统，还包括用于采集所述视频数据的所述视频采集装置。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时还执行以下步骤：选择预定基准面上的多个基准点；基于所述多个基准点的坐标信息，确定所述视频采集装置的相机坐标系统与世界坐标系统的变换关系；基于所述变换关系，确定所述视频采集装置的坐标参数。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的基于所述视频数据，确定作为监控目标的对象的步骤包括：确定所述视频数据中的背景信息；基于所述背景信息，确定所述视频数据的每帧中的前景信息；获取对应于所述前景信息的前景区 域的边缘轮廓信息；以及基于所述边缘轮廓信息，确定所述对象。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的基于所述边缘轮廓信息，确定所述对象的步骤包括：基于所述边缘轮廓信息，获取候选块；确定大于第一预定阈值的所述候选块为候选对象；以及基于预定算法获取所述候选对象的评估值，确定所述评估值大于第二预定阈值的所述候选对象为所述对象。
此外，根据本公开的另一个实施例述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的基于所述边缘轮廓信息，确定所述对象的步骤还包括：匹配前一帧与当前帧中确定的所述对象的每一个，以确定离开前一帧的对象。
此外，根据本公开的另一个实施例的视频监控系统，其中所述对象的特征信息包括：对象的体型信息以及移动速度。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的体型信息的步骤包括：选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定所述头部点在世界坐标系统中的坐标参数；以及基于所述头部点在世界坐标系统中的坐标参数，确定所述对象的头部距离地面的距离作为所述对象的身高信息。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的体型信息的步骤包括：选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；选取所述对象在图像坐标系的纵轴上的最大值的点作为所述对象的脚底点；基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定在世界坐标系统中所述对象的头部点和所述对象的脚底点之间的距离作为所述对象的身高信息。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的特征信息的步骤包括：计算所述对象的第一固定点在世界坐标体系中在第一选定帧与第二选定帧之间的运动距离；基于所述第一选定帧与所述第二选定帧之间的时间间隔以及所述运动距离，确定所述对象的移动速度。
此外，根据本公开的另一个实施例的视频监控方法，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的体型信息的步骤还包括：提取所述对象的身高信息和轮廓信息；基于预先采集的不同对象的身高信息、轮廓信息与体重信息之间的对应关系，根据所述对象的身高信息和轮廓信息确定所述对象的体重信息。
此外，根据本公开的另一个实施例的视频监控系统，其中在所述计算机程序指令被所述处理器运行时还执行：分析所述特征信息，确定所述对象的异常事件，并且其中在所述计算机程序指令被所述处理器运行所执行的确定所述对象的异常事件的步骤包括：在预定时间段内所述对象的体型信息的变化大于预定第三阈值时和/或所述对象的移动速度大于第四阈值时，确定所述对象的异常事件。
根据本公开的又一个实施例，提供了一种计算机程序产品，包括计算机可读存储介质，在所述计算机可读存储介质上存储了计算机程序指令，所述计算机程序指令在被计算机运行时执行以下步骤：获取经由视频采集装置采集的视频数据；基于所述视频数据，确定作为监控目标的对象；以及提取所述对象的特征信息；其中，所述视频数据为包含深度信息的视频数据。
要理解的是，前面的一般描述和下面的详细描述两者都是示例性的，并且意图在于提供要求保护的技术的进一步说明。
附图说明
通过结合附图对本发明实施例进行更详细的描述，本发明的上述以及其它目的、特征和优势将变得更加明显。附图用来提供对本发明实施例的进一步理解，并且构成说明书的一部分，与本发明实施例一起用于解释本发明，并不构成对本发明的限制。在附图中，相同的参考标号通常代表相同部件或步骤。
图1是图示根据本发明实施例的视频监控方法的流程图；
图2是图示根据本发明实施例的视频监控系统的功能性框图；
图3是进一步图示根据本发明实施例的视频监控方法中配置和确定视频采集装置的参数的流程图；
图4是图示用于确定视频采集装置的参数的相机坐标系统和世界坐标系 统的示意图；
图5是进一步图示根据本发明实施例的视频监控方法中确定作为监控目标的对象的流程图；
图6是图示根据本发明实施例的视频监控方法中确定前景信息的示意图；
图7是进一步图示根据本发明实施例的视频监控方法中确定对象的身高信息的第一示例的流程图；
图8是进一步图示根据本发明实施例的视频监控方法中确定对象的身高信息的第二示例的流程图；
图9是图示根据本发明实施例的视频监控方法中确定对象的身高信息的示意图；
图10是进一步图示根据本发明实施例的视频监控方法中确定对象的移动速度的流程图；
图11是图示根据本发明实施例的视频监控方法中确定对象的移动速度的示意图；
图12是进一步图示根据本发明实施例的视频监控方法中确定对象的体重信息的流程图；
图13是进一步图示根据本发明实施例的视频监控方法中确定对象的异常事件的流程图；
图14是图示根据本发明实施例的视频监控系统的示意性框图。
具体实施方式
为了使得本发明的目的、技术方案和优点更为明显，下面将参照附图详细描述根据本发明的示例实施例。显然，所描述的实施例仅仅是本发明的一部分实施例，而不是本发明的全部实施例，应理解，本发明不受这里描述的示例实施例的限制。基于本公开中描述的本发明实施例，本领域技术人员在没有付出创造性劳动的情况下所得到的所有其它实施例都应落入本发明的保护范围之内。
以下，将参考附图详细描述本发明的优选实施例。
图1是图示根据本发明实施例的视频监控方法的流程图。如图1所示， 根据本发明实施例的视频监控方法包括以下步骤。
在步骤S101中，获取经由视频采集装置采集的视频数据。在本发明的一个实施例中，所述视频采集装置为能够获取被摄体的深度信息的深度相机(深度摄像机)。获取经由视频采集装置采集的视频数据包括但不限于，在由物理位置上分离配置的视频采集装置采集视频数据之后，经由有线或者无线方式，接收从所述视频采集装置发送的视频数据。可替代地，视频采集装置可以与视频监控系统中的其他模块或组件物理上位于同一位置甚至位于同一机壳内部，视频监控系统中的其他模块或组件经由内部总线接收从所述视频采集装置发送的视频数据。此后，处理进到步骤S102。
在步骤S102中，基于所述视频数据，确定作为监控目标的对象。在本发明的一个实施例中，所述作为监控目标的对象为所述视频数据中记录的行人和/或其他物体，诸如交通工具、动物等。确定作为监控目标的对象一般地包括对于深度视频数据进行背景建模，然后根据背景数据来判断各个帧中的前景信息，并且基于预定算法获得前景区域的边缘轮廓信息，从而基于各个前景区域的轮廓来判断是否为待监控的对象。以下，将参照附图进一步详细描述如何确定作为监控目标的对象的流程。此后，处理进到步骤S103。
在步骤S103中，提取所述对象的特征信息。在本发明的一个实施例中，所述对象的特征信息包括但不限于所述对象的身高、体重、移动速度等中的至少一种。以下，将参照附图进一步详细描述如何提取所述对象的特征信息的流程。
上述根据本发明实施例的视频监控方法，采用深度相机/摄像机作为视频采集装置，从而实现基于深度视频信息的对象检测和特征信息提取，为进一步的统计分析和异常情况监控提供了准确和高效的数据保证。
以下，将参照图2进一步描述执行上述视频监控方法的视频监控系统。
图2是图示根据本发明实施例的视频监控系统的功能性框图。如图2所示，根据本发明实施例的视频监控系统20包括视频数据获取模块21、监控对象确定模块22、特征信息提取模块23以及特征信息分析模块24。所述视频数据获取模块21、监控对象确定模块22、特征信息提取模块23以及特征信息分析模块24例如可以由诸如硬件(服务器、专用计算机等)、软件、固件以及它们的任意可行的组合配置。
具体地，所述视频数据获取模块21用于获取视频数据。在本发明的一 个实施例中，所述视频数据获取模块21可以包括能够获取被摄体的深度信息的深度相机(深度摄像机)的视频采集装置。所述视频采集装置可以与其后的监控对象确定模块22、特征信息提取模块23以及特征信息分析模块24物理上分离，或者物理上位于同一位置甚至位于同一机壳内部。在所述视频采集装置与其后的监控对象确定模块22、特征信息提取模块23以及特征信息分析模块24物理上分离的情况下，所述视频数据获取模块21进一步经由有线或者无线方式将所述视频采集装置获取的深度视频数据发送给其后的模块。在所述视频采集装置与其后的监控对象确定模块22、特征信息提取模块23以及特征信息分析模块24物理上位于同一位置甚至位于同一机壳内部的情况下，所述视频数据获取模块21经由内部总线将所述视频采集装置获取的深度视频数据发送给其后的模块。所述视频数据可以包括深度视频数据以及彩色视频数据。更具体地，可以根据视频采集装置的位置参数以及视频数据中每个像素点的深度信息值，确定视频数据中每个像素的三维位置参数。在经由有线或者无线方式或者经由内部总线发送所述视频数据之前，可以将其预定格式进行编码和压缩为视频数据包，以减少发送需要占用的通信量和带宽。
所述监控对象确定模块22用于基于从所述视频数据获取模块21获取的所述视频数据，确定作为监控目标的对象。所述监控对象确定模块22例如对于深度视频数据进行背景建模，然后根据背景数据来判断各个帧中的前景信息，并且基于预定算法获得前景区域的边缘轮廓信息，从而基于各个前景区域的轮廓来判断是否为待监控的对象。更具体地，所述监控对象确定模块22可以检测视频的每一帧中是否存储在行人，并且确定每个行人的具体位置。对于不同帧中的行人进行匹配关联，实现对于行人在不同帧中的准备追踪。
所述特征信息提取模块23用于对于由所述监控对象确定模块22确定的待监控的对象，基于从所述视频数据获取模块21获取的所述视频数据，提取各个待监控的对象的特征信息，诸如所述对象的身高、体重、移动速度等。
所述特征信息分析模块24用于对于由所述特征信息提取模块23提取的特征信息进行存储、统计分析以及异常情况的监控。更具体地，存储的数据包括但不限于行人在每一帧的位置、每个行人的特征信息、每一帧图像的具体时刻等。所述特征信息分析模块24实时分析特征信息以监控可能出现的 异常情况，并且对于长期存储的特征信息进行整理和统计规律提取。例如，行人在哪个时刻比较多地出现，行人的平均身高、体重等特征。
以下，将进一步参照附图详细描述由根据本发明实施例的视频监控系统的各个模块执行的根据本发明实施例的视频监控方法的各个具体步骤流程。
首先，参照图3和图4描述视频采集装置的配置以及视频采集装置的坐标参数的确定。可以由上述视频数据获取模块21控制和执行视频采集装置的配置以及视频采集装置的坐标参数的确定。图3是进一步图示根据本发明实施例的视频监控方法中配置和确定视频采集装置的参数的流程图。图4是图示用于确定视频采集装置的参数的相机坐标系统和世界坐标系统的示意图。
如图3所示，根据本发明实施例的视频监控方法中配置和确定视频采集装置的参数的流程包括以下步骤。
在步骤S301中，配置视频采集装置。将作为所述视频采集装置的深度相机(深度摄像机)安装在需要监控的场景中。通常，深度相机(深度摄像机)的安装高度为2-3.5米，其视角为俯视地面(如图4示意性所示)。在此，所述视频采集装置可以是单一深度相机(即，只有深度相机镜头)或者深度彩色双镜头相机。在深度彩色双镜头相机的情况下，需要对相机进行校准，使得两个镜头得到的图像相对应和同步。此后，处理进到步骤S302，以便对于安装后的所述视频采集装置，确定其离基准面的实际高度和角度等坐标参数。
在步骤S302中，选择预定基准面上的多个基准点。如图4所示，所述预定基准面可以是地平面，选择的基准点的数目越大(例如，大于等于5个)，精度越高。此后，处理进到步骤S303。
在步骤S303中，基于选择的多个基准点的坐标信息，确定视频采集装置的相机坐标系统与世界坐标系统的变换关系。如图4所示，由点Oc与Xc、Yc和Zc轴构成的直角坐标系为相机坐标系统。为了描述相机的位置，引入世界坐标系统，由点Ow与Xw、Yw和Zw轴构成的直角坐标系为世界坐标系统。可以通过选择多个基准点，基于最小二乘法来估计相机坐标系统到世界坐标系统的转换矩阵，即相机坐标系统与世界坐标系统的变换关系。此后，处理进到步骤S304。
在步骤S304中，基于所述变换关系，确定视频采集装置的坐标参数。 通过利用所述变换关系，将相机坐标系统转换到世界坐标系统，可以确定视频采集装置的实际高度和角度等坐标参数。同样地，将视频采集装置采集的视频中的像素点转换到世界坐标系统，可以确定视频场景中的完整地平面位置。
以下，将参照图5和图6描述作为监控目标的对象的确定。可以由上述监控对象确定模块22控制和执行监控目标的对象的确定。图5是进一步图示根据本发明实施例的视频监控方法中确定作为监控目标的对象的流程图。图6是图示根据本发明实施例的视频监控方法中确定前景信息的示意图。
如图5所示，根据本发明实施例的视频监控方法中确定作为监控目标的对象的流程包括以下步骤。
在步骤S501中，确定视频数据中的背景信息。在本发明的一个实施例中，在根据深度视频进行背景建模时，例如可以拍摄没有任何监控对象存在的背景帧作为背景信息，如图6(A)示出了作为背景信息的背景帧601。此后，处理进到步骤S502。
在步骤S502中，基于所述背景信息，确定所述视频数据的每帧中的前景信息。在本发明的一个实施例中，例如如图6(B)示出了所述视频数据的当前帧602，通过将所述背景帧601与所述当前帧602进行深度差比较，确定作为前景信息的前景区域603，如图6(C)所示。此后，处理进到步骤S503。
在步骤S503中，获取对应于所述前景信息的前景区域的边缘轮廓信息。在本发明的一个实施例中，根据在步骤S502中确定的前景区域，使用边缘检测算法来得到所述前景区域的边缘轮廓信息。此后，处理进到步骤S504。
在步骤S504中，基于所述边缘轮廓信息，获取候选块。在本发明的一个实施例中，基于在步骤S503中获取的边缘轮廓，使用双层轮廓来得到前景的多个候选块。此后，处理进到步骤S505。
在步骤S505中，确定大于第一预定阈值的所述候选块为候选对象。具在本发明的一个实施例中，所述第一预定阈值为候选块的最小可接受面积，其面积不大于所述第一预定阈值的候选块可能是由于噪声而产生的，从而剔除该面积过小的候选块，剩余的每一个候选块代表一个候选对象。此后，处理进到步骤S506。
在步骤S506中，基于预定算法获取所述候选对象的评估值，确定所述评估值大于第二预定阈值的所述候选对象为所述对象。在本发明的一个实施 例中，可以基于彩色图像的行人检测算法和/或人头检测算法给每个候选对象进行评估，将其评估值大于第二预定阈值的候选对象确定为所述对象(例如，确定为行人)。在应用所述行人检测算法的情况下，遍历深度视频帧中所有可能的矩形框，每一个矩形框代表一个行人的候选区域。对于该区域，提取梯度直方图的图像特征，然后根据使用支持向量机来做分类，判断该矩形区域是否有行人存在。在应用人头检测算法的情况下，遍历深度视频帧中所有可能的矩形框，对于每个矩形框，提取颜色和纹理特征，此后采用训练好的支持向量机来判断该区域是否有人头存在。此后，处理进到步骤S507。
在步骤S507中，匹配前一帧与当前帧中确定的所述对象的每一个，以确定离开前一帧的对象。在本发明的一个实施例中，对于当前帧中确定的行人和前一帧中确定的行人进行匹配，匹配的标准可以是行人的行进距离以及候选对象上提取的颜色纹理信息。根据匹配的结果，可以得到同一个行人在不同帧上面的跟踪结果。例如，假设在第t-1帧中存在M个检测到的行人，在第t帧存在N个检测到的行人，那么需要求得第t-1帧检测到的行人与第t帧检测到的每个行人之间的距离，该距离可以定义为行人在两帧中头部点之间的空间位置距离。如果第t-1帧检测到的行人到第t帧的所有行人的最小距离小于预定阈值，则认为该第t-1帧检测到的行人匹配到第t帧中具有最小距离的行人。如果该最小距离大于预定阈值，则认为该第t-1帧检测到的行人已经离开第t帧，从而不再继续跟踪该行人。
以下，将参照图7到图12描述提取所述对象的特征信息。可以由上述特征信息提取模块23控制和执行所述对象的特征信息的提取。图7是进一步图示根据本发明实施例的视频监控方法中确定对象的身高信息的第一示例的流程图。图8是进一步图示根据本发明实施例的视频监控方法中确定对象的身高信息的第二示例的流程图。图9是图示根据本发明实施例的视频监控方法中确定对象的身高信息的示意图。图10是进一步图示根据本发明实施例的视频监控方法中确定对象的移动速度的流程图。图11是图示根据本发明实施例的视频监控方法中确定对象的移动速度的示意图。图12是进一步图示根据本发明实施例的视频监控方法中确定对象的体重信息的流程图。
如图7所示，根据本发明实施例的视频监控方法中确定对象的身高信息的第一示例的流程包括以下步骤。
在步骤S701中，选取对象距离所述视频采集装置的最近点作为所述对 象的头部点。在本发明的一个实施例中，如图9所示，选取对象距离所述视频采集装置的最近点，因为行人为直立，所以该最近点可以假设为头部点901。此后，处理进到步骤S702。
在步骤S702中，基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定所述头部点在世界坐标系统中的坐标参数。如上参照图3和图4已经描述的，可以将所述头部点在所述视频采集装置的相机坐标系统中的坐标参数转换为世界坐标系统的坐标参数。此后，处理进到步骤S703。
在步骤S703中，基于所述头部点在世界坐标系统中的坐标参数，确定所述对象的头部距离地面的距离作为所述对象的身高信息。在本发明的一个实施例中，如图9所示，确定头部点901距离地面的距离为所述对象的身高902。
根据本发明实施例的视频监控方法中确定对象的身高信息的方法不限于如图7所示的第一示例。如图8所示，根据本发明实施例的视频监控方法中确定对象的身高信息的第二示例的流程包括以下步骤。
在步骤S801中，类似于步骤S701，选取对象距离所述视频采集装置的最近点作为所述对象的头部点。此后，处理进到步骤S802。
在步骤S802中，选取所述对象在图像坐标系的纵轴上的最大值的点作为所述对象的脚底点。在本发明的一个实施例中，脚底点可以定义为行人的前景区域在图像坐标系的纵轴上的最大值的点。此后，处理进到步骤S803。
在步骤S803中，基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定在世界坐标体系中所述对象的头部点和所述对象的脚底点之间的距离作为所述对象的身高信息。
除了参照图7到图9描述的身高信息，所述对象的特征信息还包括所述对象的移动速度以及体重信息等。
如图10所示，根据本发明实施例的视频监控方法中确定对象的移动速度的流程包括以下步骤。
在步骤S1001中，计算所述对象的第一固定点在世界坐标体系中在第一选定帧与第二选定帧之间的运动距离。在本发明的一个实施例中，所述第一固定点可以是所述对象的头部点。本发明不限于此，所述第一固定点还可以是所述对象的脚底点或对象上的任一固定点。在本发明的一个实施例中，第一选定帧是第T帧，第二选定帧是第T+t帧。如图11所示，所述对象在第 一选定帧中是对象1101，在第二选定帧中是对象1102。所述第一固定点在世界坐标体系中在第一选定帧第T帧与第二选定帧第T+t帧之间的运动距离为运动距离1103。此后，处理进到步骤S1002。
在步骤S1002中，基于所述第一选定帧与所述第二选定帧之间的时间间隔以及所述运动距离，确定所述对象的移动速度。也就是说，所述对象的移动速度可以确定为V＝运动距离1103/t。
如图12所示，根据本发明实施例的视频监控方法中确定对象的体重信息的流程包括以下步骤。
在步骤S1201中，提取所述对象的身高信息和轮廓信息。此后，处理进到步骤S1202。
在步骤S1202中，基于预先采集的不同对象的身高信息、轮廓信息与体重信息之间的对应关系，根据所述对象的身高信息和轮廓信息确定所述对象的体重信息。在本发明的一个实施例中，首先采集包含不同身高体重的行人的数据集。其后，采集每个行人在深度相机下处于不同深度时的相应轮廓信息。对于每个轮廓的面积，根据该轮廓所在的深度距离进行正规化。然后，将所有行人的身高进行量化(例如，量化为多个身高区间)，对于落在同一区间内的所有数据拟合，从而建立正规化的轮廓面积与体重之间的关系。当已经建立和训练好对应关系的模型之后，当需要测量行人的体重时，根据该行人的身高所在区域的线性模型，以及正规化后的行人的轮廓面积，得到根据所述对应关系的模型确定的体重值。
以上参照图7到图12描述的所述对象的特征信息可以进一步用于对于对象的异常事件的监控。图13是进一步图示根据本发明实施例的视频监控方法中确定对象的异常事件的一种流程图。
如图13所示，根据本发明实施例的视频监控方法中确定对象的异常事件的流程包括以下步骤。其中，步骤S1301到步骤S1303与参照图1描述的步骤S101到S103相同，将省略其重复描述。
在步骤S1304中，分析所述特征信息。可以由上述特征信息分析模块24控制和执行所述特征信息的分析。在本发明的一个实施例中，分析所述特征信息包括分析获取的对象体型信息以及对象的移动速度。此后，处理进到步骤S1305。
在步骤S1305中，判断所述对象的体型信息的变化是否大于预定阈值。 在本发明的一个实施例中，如果行人的身高信息与之前帧相比出现的变化大于预定阈值(例如，1米)，则可以判断出现诸如行人摔倒的异常事件。如果在步骤S1305获得肯定结果，则处理进到步骤S1307。
在步骤S1307中，判断出现异常事件。在本发明的一个实施例中，响应于出现异常事件，可以以适当的方式向视频监控系统的监控人员发出出现异常事件的警报。
如果在步骤S1305获得否定结果，则处理进到步骤S1306。
在步骤S1306中，继续判断所述对象的移动速度是否大于预定阈值。在本发明的一个实施例中，如果行人的移动速度突然大于预定阈值(例如，4米/秒)，则可以判断有异常事件出现。如果在步骤S1306获得肯定结果，则处理进到步骤S1307。相反地，如果在步骤S1306获得肯定结果，则确定基于步骤S1304的特征信息的分析，没有异常事件出现，处理返回步骤S1301。从而继续实时获取经由视频采集装置采集的视频数据并且继续执行监控。
以上步骤S1301至步骤S1307仅为确定对象的异常事件的一种示例性流程。在确定对象的异常事件的其他实现方式中，可以仅监控对象的移动速度，也可以仅监控对象的体型信息的变化，可以同时监控对象的体型信息的变化和对象的移动速度，还可以先监控对象的移动速度再监控对象的体型信息的变化，在此并不进行限定。
图14是图示根据本发明实施例的视频监控系统的示意性框图。如图14所示，根据本发明实施例的视频监控系包括：处理器141、存储器142、以及在所述存储器142的中存储的计算机程序指令143。
所述计算机程序指令143在所述处理器141运行时可以实现根据本发明实施例的视频监控系统的各个功能模块的功能，并且/或者可以执行根据本发明实施例的视频监控方法的各个步骤。
具体地，在所述计算机程序指令143被所述处理器141运行时执行以下步骤：获取经由视频采集装置采集的视频数据；基于所述视频数据，确定作为监控目标的对象；以及提取所述对象的特征信息；其中，所述视频数据为包含深度信息的视频数据。
例如，在所述计算机程序指令143被所述处理器141运行时还执行以下步骤：选择预定基准面上的多个基准点；基于所述多个基准点的坐标信息，确定所述视频采集装置的相机坐标系统与世界坐标系统的变换关系；基于所 述变换关系，确定所述视频采集装置的坐标参数。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的基于所述视频数据，确定作为监控目标的对象的步骤包括：确定所述视频数据中的背景信息；基于所述背景信息，确定所述视频数据的每帧中的前景信息；获取对应于所述前景信息的前景区域的边缘轮廓信息；以及基于所述边缘轮廓信息，确定所述对象。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的基于所述边缘轮廓信息，确定所述对象的步骤包括：基于所述边缘轮廓信息，获取候选块；确定大于第一预定阈值的所述候选块为候选对象；以及基于预定算法获取所述候选对象的评估值，确定所述评估值大于第二预定阈值的所述候选对象为所述对象。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的基于所述边缘轮廓信息，确定所述对象的步骤还包括：匹配前一帧与当前帧中确定的所述对象的每一个，以确定离开前一帧的对象。
此外，所述对象的特征信息包括：对象的体型信息以及移动速度。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的提取所述对象的体型信息的步骤包括：选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定所述头部点在世界坐标系统中的坐标参数；以及基于所述头部点在世界坐标系统中的坐标参数，确定所述对象的头部距离地面的距离作为所述对象的身高信息。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的提取所述对象的体型信息的步骤包括：选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；选取所述对象在图像坐标系的纵轴上的最大值的点作为所述对象的脚底点；基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定在世界坐标系统中所述对象的头部点和所述对象的脚底点之间的距离作为所述对象的身高信息。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的提取所述对象的特征信息的步骤包括：计算所述对象的第一固定点在世界坐标体系中在第一选定帧与第二选定帧之间的运动距离；基于所述第一选定帧与所述第二选定帧之间的时间间隔以及所述运动距离，确定所述对象的移动速 度。
此外，在所述计算机程序指令143被所述处理器141运行时所执行的提取所述对象的体型信息的步骤还包括：提取所述对象的身高信息和轮廓信息；基于预先采集的不同对象的身高信息、轮廓信息与体重信息之间的对应关系，根据所述对象的身高信息和轮廓信息确定所述对象的体重信息。
此外，在所述计算机程序指令143被所述处理器141运行时还执行：分析所述特征信息，确定所述对象的异常事件，并且其中在所述计算机程序指令被所述处理器运行所执行的确定所述对象的异常事件的步骤包括：在预定时间段内所述对象的体型信息的变化大于预定第三阈值时和/或所述对象的移动速度大于第四阈值时，确定所述对象的异常事件。
根据本发明实施例的视频监控系统中的各模块可以通过根据本发明实施例的视频监控系统中的处理器运行在存储器中存储的计算机程序指令来实现，或者可以在根据本发明实施例的计算机程序产品的计算机可读存储介质中存储的计算机指令被计算机运行时实现。
所述计算机可读存储介质可以是一个或多个计算机可读存储介质的任意组合，例如一个计算机可读存储介质包含用于随机地生成动作指令序列的计算机可读的程序代码，另一个计算机可读存储介质包含用于进行人脸活动识别的计算机可读的程序代码。
所述计算机可读存储介质例如可以包括智能电话的存储卡、平板电脑的存储部件、个人计算机的硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器、或者上述存储介质的任意组合。
在上面详细描述的本发明的示例实施例仅仅是说明性的，而不是限制性的。本领域技术人员应该理解，在不脱离本发明的原理和精神的情况下，可对这些实施例进行各种修改，组合或子组合，并且这样的修改应落入本发明的范围内。

Claims (25)

  1. 一种视频监控方法，包括：
    获取经由视频采集装置采集的视频数据；
    基于所述视频数据，确定作为监控目标的对象；以及
    提取所述对象的特征信息；
    其中，所述视频数据为包含深度信息的视频数据。
  2. 如权利要求1所述的视频监控方法，还包括：
    配置所述视频采集装置，并且确定所述视频采集装置的坐标参数。
  3. 如权利要求2所述的视频监控方法，其中确定所述视频采集装置的坐标参数包括：
    选择预定基准面上的多个基准点；
    基于所述多个基准点的坐标信息，确定所述视频采集装置的相机坐标系统与世界坐标系统的变换关系；
    基于所述变换关系，确定所述视频采集装置的坐标参数。
  4. 如权利要求1所述的视频监控方法，其中，所述基于所述视频数据，确定作为监控目标的对象包括：
    确定所述视频数据中的背景信息；
    基于所述背景信息，确定所述视频数据的每帧中的前景信息；
    获取对应于所述前景信息的前景区域的边缘轮廓信息；以及
    基于所述边缘轮廓信息，确定所述对象。
  5. 如权利要求4所述的视频监控方法，其中，基于所述边缘轮廓信息，确定所述对象包括：
    基于所述边缘轮廓信息，获取候选块；
    确定大于第一预定阈值的所述候选块为候选对象；以及
    基于预定算法获取所述候选对象的评估值，确定所述评估值大于第二预定阈值的所述候选对象为所述对象。
  6. 如权利要求5所述的视频监控方法，其中，基于所述边缘轮廓信息，确定所述对象还包括：
    匹配前一帧与当前帧中确定的所述对象的每一个，以确定离开前一帧的对象。
  7. 如权利要求1所述的视频监控方法，其中所述对象的特征信息包括：对象的体型信息以及移动速度。
  8. 如权利要求7所述的视频监控方法，其中提取所述对象的体型信息包括：
    选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；
    基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定所述头部点在世界坐标系统中的坐标参数；以及
    基于所述头部点在世界坐标系统中的坐标参数，确定所述对象的头部距离地面的距离作为所述对象的身高信息。
  9. 如权利要求7所述的视频监控方法，其中提取所述对象的体型信息包括：
    选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；
    选取所述对象在图像坐标系的纵轴上的最大值的点作为所述对象的脚底点；
    基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定在世界坐标系统中所述对象的头部点和所述对象的脚底点之间的距离作为所述对象的身高信息。
  10. 如权利要求7所述的视频监控方法，还包括：
    计算所述对象的第一固定点在世界坐标体系中在第一选定帧与第二选定帧之间的运动距离；
    基于所述第一选定帧与所述第二选定帧之间的时间间隔以及所述运动距离，确定所述对象的移动速度。
  11. 如权利要求7所述的视频监控方法，其中提取所述对象的体型信息还包括：
    提取所述对象的身高信息和轮廓信息；
    基于预先采集的不同对象的身高信息、轮廓信息与体重信息之间的对应关系，根据所述对象的身高信息和轮廓信息确定所述对象的体重信息。
  12. 如权利要求7所述的视频监控方法，还包括：
    分析所述特征信息，确定所述对象的异常事件，
    其中分析所述特征信息，确定所述对象的异常事件包括：
    在预定时间段内所述对象的体型信息的变化大于预定第三阈值时和/或 所述对象的移动速度大于第四阈值时，确定所述对象的异常事件。
  13. 一种视频监控系统，包括：
    处理器；
    存储器；和
    存储在所述存储器中的计算机程序指令，在所述计算机程序指令被所述处理器运行时执行以下步骤：
    获取经由视频采集装置采集的视频数据；
    基于所述视频数据，确定作为监控目标的对象；以及
    提取所述对象的特征信息；
    其中，所述视频数据为包含深度信息的视频数据。
  14. 如权利要求13所述的视频监控系统，还包括用于采集所述视频数据的所述视频采集装置。
  15. 如权利要求14所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时还执行以下步骤：
    选择预定基准面上的多个基准点；
    基于所述多个基准点的坐标信息，确定所述视频采集装置的相机坐标系统与世界坐标系统的变换关系；
    基于所述变换关系，确定所述视频采集装置的坐标参数。
  16. 如权利要求13所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的基于所述视频数据，确定作为监控目标的对象的步骤包括：
    确定所述视频数据中的背景信息；
    基于所述背景信息，确定所述视频数据的每帧中的前景信息；
    获取对应于所述前景信息的前景区域的边缘轮廓信息；以及
    基于所述边缘轮廓信息，确定所述对象。
  17. 如权利要求16所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的基于所述边缘轮廓信息，确定所述对象的步骤包括：
    基于所述边缘轮廓信息，获取候选块；
    确定大于第一预定阈值的所述候选块为候选对象；以及
    基于预定算法获取所述候选对象的评估值，确定所述评估值大于第二预 定阈值的所述候选对象为所述对象。
  18. 如权利要求17所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的基于所述边缘轮廓信息，确定所述对象的步骤还包括：
    匹配前一帧与当前帧中确定的所述对象的每一个，以确定离开前一帧的对象。
  19. 如权利要求13所述的视频监控系统，其中所述对象的特征信息包括：对象的体型信息以及移动速度。
  20. 如权利要求19所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的体型信息的步骤包括：
    选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；
    基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定所述头部点在世界坐标系统中的坐标参数；以及
    基于所述头部点在世界坐标系统中的坐标参数，确定所述对象的头部距离地面的距离作为所述对象的身高信息。
  21. 如权利要求19所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的体型信息的步骤包括：
    选取所述对象距离所述视频采集装置的最近点作为所述对象的头部点；
    选取所述对象在图像坐标系的纵轴上的最大值的点作为所述对象的脚底点；
    基于所述视频采集装置的相机坐标系统与世界坐标系统的变换关系，确定在世界坐标系统中所述对象的头部点和所述对象的脚底点之间的距离作为所述对象的身高信息。
  22. 如权利要求19所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的特征信息的步骤包括：
    计算所述对象的第一固定点在世界坐标体系中在第一选定帧与第二选定帧之间的运动距离；
    基于所述第一选定帧与所述第二选定帧之间的时间间隔以及所述运动距离，确定所述对象的移动速度。
  23. 如权利要求19所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时所执行的提取所述对象的体型信息的步骤还包括：
    提取所述对象的身高信息和轮廓信息；
    基于预先采集的不同对象的身高信息、轮廓信息与体重信息之间的对应关系，根据所述对象的身高信息和轮廓信息确定所述对象的体重信息。
  24. 如权利要求19所述的视频监控系统，其中在所述计算机程序指令被所述处理器运行时还执行：分析所述特征信息，确定所述对象的异常事件，并且
    其中在所述计算机程序指令被所述处理器运行所执行的确定所述对象的异常事件的步骤包括：
    在预定时间段内所述对象的体型信息的变化大于预定第三阈值时和/或所述对象的移动速度大于第四阈值时，确定所述对象的异常事件。
  25. 一种计算机程序产品，包括计算机可读存储介质，在所述计算机可读存储介质上存储了计算机程序指令，所述计算机程序指令在被计算机运行时执行以下步骤：
    获取经由视频采集装置采集的视频数据；
    基于所述视频数据，确定作为监控目标的对象；以及
    提取所述对象的特征信息；
    其中，所述视频数据为包含深度信息的视频数据。
PCT/CN2015/075126 2015-03-26 2015-03-26 视频监控方法、视频监控系统以及计算机程序产品 WO2016149938A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US14/888,357 US9792505B2 (en) 2015-03-26 2015-03-26 Video monitoring method, video monitoring system and computer program product
CN201580000334.1A CN105519102B (zh) 2015-03-26 2015-03-26 视频监控方法和视频监控系统
PCT/CN2015/075126 WO2016149938A1 (zh) 2015-03-26 2015-03-26 视频监控方法、视频监控系统以及计算机程序产品

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/075126 WO2016149938A1 (zh) 2015-03-26 2015-03-26 视频监控方法、视频监控系统以及计算机程序产品

Publications (1)

Publication Number Publication Date
WO2016149938A1 true WO2016149938A1 (zh) 2016-09-29

Family

ID=55725079

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/075126 WO2016149938A1 (zh) 2015-03-26 2015-03-26 视频监控方法、视频监控系统以及计算机程序产品

Country Status (3)

Country Link
US (1) US9792505B2 (zh)
CN (1) CN105519102B (zh)
WO (1) WO2016149938A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111582242A (zh) * 2020-06-05 2020-08-25 上海商汤智能科技有限公司 滞留检测方法、装置、电子设备和存储介质

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104994360B (zh) * 2015-08-03 2018-10-26 北京旷视科技有限公司 视频监控方法和视频监控系统
US10157452B1 (en) * 2015-09-28 2018-12-18 Amazon Technologies, Inc. Image processing system for image rectification
CN106204549B (zh) * 2016-06-30 2018-09-11 北京文安智能技术股份有限公司 一种基于视频分析的广告牌监测方法、装置及电子设备
CN106529399A (zh) * 2016-09-26 2017-03-22 深圳奥比中光科技有限公司 人体信息采集方法、装置及系统
CN106529400A (zh) * 2016-09-26 2017-03-22 深圳奥比中光科技有限公司 移动终端及其人体监测方法、装置
US10535142B2 (en) * 2017-01-10 2020-01-14 Electronics And Telecommunication Research Institute Method and apparatus for accelerating foreground and background separation in object detection using stereo camera
TW201904265A (zh) * 2017-03-31 2019-01-16 加拿大商艾維吉隆股份有限公司 異常運動偵測方法及系統
CN108364316A (zh) * 2018-01-26 2018-08-03 阿里巴巴集团控股有限公司 交互行为检测方法、装置、系统及设备
CN109087275A (zh) * 2018-03-21 2018-12-25 杨娟 计算机评估方法
CN111354034A (zh) * 2019-04-02 2020-06-30 北京航空航天大学 面向轨道交通乘客表面积估算的图像识别系统及方法
CN110338800A (zh) * 2019-07-17 2019-10-18 成都泰盟软件有限公司 自动测量身高体重的躺椅
CN110418112A (zh) * 2019-08-09 2019-11-05 上海商汤智能科技有限公司 一种视频处理方法及装置、电子设备和存储介质
CN111767850A (zh) * 2020-06-29 2020-10-13 北京百度网讯科技有限公司 突发事件的监控方法、装置、电子设备和介质
CN112560610B (zh) * 2020-12-03 2021-09-28 西南交通大学 一种视频监控对象分析方法、装置、设备及可读存储介质
CN112528854B (zh) * 2020-12-12 2023-07-25 南方电网调峰调频发电有限公司 振动对象监测方法、装置、计算机设备和存储介质
CN112700657B (zh) * 2020-12-21 2023-04-28 阿波罗智联(北京)科技有限公司 用于生成检测信息的方法、装置、路侧设备和云控平台
CN113099173A (zh) * 2021-03-25 2021-07-09 湖南桥康智能科技有限公司 一种具有物体位移测量计算功能的智能网络摄像机

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040061781A1 (en) * 2002-09-17 2004-04-01 Eastman Kodak Company Method of digital video surveillance utilizing threshold detection and coordinate tracking
CN101068342A (zh) * 2007-06-05 2007-11-07 西安理工大学 基于双摄像头联动结构的视频运动目标特写跟踪监视方法
CN101727671A (zh) * 2009-12-01 2010-06-09 湖南大学 基于路面共线三点及其平行线的单摄像机标定方法
CN102722721A (zh) * 2012-05-25 2012-10-10 山东大学 一种基于机器视觉的人体跌倒检测方法
US20130182114A1 (en) * 2012-01-17 2013-07-18 Objectvideo, Inc. System and method for monitoring a retail environment using video content analysis with depth sensing
CN103716579A (zh) * 2012-09-28 2014-04-09 中国科学院深圳先进技术研究院 视频监控方法及系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7263209B2 (en) * 2003-06-13 2007-08-28 Sarnoff Corporation Vehicular vision system
US8500284B2 (en) * 2008-07-10 2013-08-06 Real View Imaging Ltd. Broad viewing angle displays and user interfaces
EP2154650A1 (en) * 2008-08-12 2010-02-17 IEE INTERNATIONAL ELECTRONICS & ENGINEERING S.A. 3D time-of-flight camera system and position/orientation calibration method therefor
US8417062B2 (en) * 2009-10-19 2013-04-09 Ut-Battelle, Llc System and method for stabilization of fisheye video imagery
EP2615580B1 (en) * 2012-01-13 2016-08-17 Softkinetic Software Automatic scene calibration
US9165190B2 (en) * 2012-09-12 2015-10-20 Avigilon Fortress Corporation 3D human pose and shape modeling

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040061781A1 (en) * 2002-09-17 2004-04-01 Eastman Kodak Company Method of digital video surveillance utilizing threshold detection and coordinate tracking
CN101068342A (zh) * 2007-06-05 2007-11-07 西安理工大学 基于双摄像头联动结构的视频运动目标特写跟踪监视方法
CN101727671A (zh) * 2009-12-01 2010-06-09 湖南大学 基于路面共线三点及其平行线的单摄像机标定方法
US20130182114A1 (en) * 2012-01-17 2013-07-18 Objectvideo, Inc. System and method for monitoring a retail environment using video content analysis with depth sensing
CN102722721A (zh) * 2012-05-25 2012-10-10 山东大学 一种基于机器视觉的人体跌倒检测方法
CN103716579A (zh) * 2012-09-28 2014-04-09 中国科学院深圳先进技术研究院 视频监控方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111582242A (zh) * 2020-06-05 2020-08-25 上海商汤智能科技有限公司 滞留检测方法、装置、电子设备和存储介质
CN111582242B (zh) * 2020-06-05 2024-03-26 上海商汤智能科技有限公司 滞留检测方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
US9792505B2 (en) 2017-10-17
CN105519102A (zh) 2016-04-20
US20170032194A1 (en) 2017-02-02
CN105519102B (zh) 2018-12-28

Similar Documents

Publication Publication Date Title
WO2016149938A1 (zh) 视频监控方法、视频监控系统以及计算机程序产品
US10672140B2 (en) Video monitoring method and video monitoring system
US9646212B2 (en) Methods, devices and systems for detecting objects in a video
CN106650620B (zh) 一种应用无人机监测的目标人员识别追踪方法
Charfi et al. Definition and performance evaluation of a robust SVM based fall detection solution
CN105493502B (zh) 视频监控方法、视频监控系统以及计算机可读存储介质
JP6488083B2 (ja) 駐車区画占有率判定のための映像および視覚ベースのアクセス制御のハイブリッド方法およびシステム
CN104933414B (zh) 一种基于wld-top的活体人脸检测方法
JP6237804B2 (ja) 異常状況の検出方法及び装置
Avgerinakis et al. Recognition of activities of daily living for smart home environments
WO2015131734A1 (zh) 一种前视监视场景下的行人计数方法、装置和存储介质
US20160140399A1 (en) Object detection apparatus and method therefor, and image recognition apparatus and method therefor
JP2015207280A (ja) 目標識別方法及び目標識別装置
US10402633B2 (en) Human detection in high density crowds
US20170039431A1 (en) Video monitoring method, video monitoring apparatus and video monitoring system
CN103824070A (zh) 一种基于计算机视觉的快速行人检测方法
WO2018228174A1 (zh) 获取对象密度的方法、装置、设备及存储介质
CN115620212B (zh) 一种基于监控视频的行为识别方法及系统
CN110633648B (zh) 一种自然行走状态下的人脸识别方法和系统
WO2023000253A1 (zh) 攀爬行为预警方法和装置、电子设备、存储介质
US20220366570A1 (en) Object tracking device and object tracking method
WO2022127181A1 (zh) 客流的监测方法、装置、电子设备及存储介质
CN103903020A (zh) 一种基于CodeBook的火灾图像识别方法及装置
US20220076005A1 (en) Person authentication apparatus, control method, and non-transitory storage medium
CN104123569A (zh) 一种基于有监督学习的视频人数信息统计方法

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 14888357

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15885877

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15885877

Country of ref document: EP

Kind code of ref document: A1