WO2020111932A1

WO2020111932A1 - A system and method for detecting aggressive behaviour

Info

Publication number: WO2020111932A1
Application number: PCT/MY2019/050099
Authority: WO
Inventors: Nik Ahmad Akram BIN NIK ZULKEPELI; Zulaikha BINTI KADIM; Hock Woon Hon; Khairunnisa BINTI MOHAMED JOHARI
Original assignee: Mimos Bhd
Current assignee: Mimos Bhd
Priority date: 2018-11-29
Filing date: 2019-11-26
Publication date: 2020-06-04
Anticipated expiration: 2021-05-29
Also published as: MY200192A

Abstract

The present invention relates to a system (200) and method for detecting aggressive behaviour. The system (200) comprising an optical flow module (105), a statistical computing module (106), and a classifier module (107). The optical flow module (105) is configured for subtracting background of an image frame and compute optical flow each image frame. The statistical computing module (106) is configured for computing statistical value of the optical flow based on a dimension of the enclosed space and location of a camera (101). The classifier module (107) is configured for classifying the statistical value to determine if an aggressive behaviour occurs. The system (200) further comprising an image calibration module (104) configured for calibrating the image frame according to a dimension of the enclosed space and the location of the camera (101) by correcting perspective errors in the image frame.

Description

A SYSTEM AND METHOD FOR DETECTING AGGRESSIVE BEHAVIOUR

FIELD OF INVENTION

The present invention relates to a system and method for detecting aggressive behaviour. More particularly, the present invention relates to a system and method for detecting aggressive behaviour based on video surveillance.

BACKGROUND OF THE INVENTION

Nowadays, many enclosed spaces are used to quarantine people such as prisons, police lock-up cells, psychiatric wards and elderly centres. Sometimes, certain people are quarantined in an enclosed space against their will and cause them to act aggressively and therefore they have to be monitored consistently. Conventionally, a number of surveillance cameras are installed in these enclosed spaces to monitor the occupant. In addition to the surveillance cameras, a system to detect aggressive behaviour is also installed to detect aggressive behaviour by the occupant.

An example of a system and method for detecting aggressive behaviour is disclosed in a United States Patent Application No. 2012/0321 138 A1 which relates to a system for detecting a suspicious behaviour. The system detects suspicious behaviour of a monitored subject by using images captured by a stereo camera. The system then acquires ambulatory path information of the monitored subject from the captured images and identifies the behaviour of the monitored based on the ambulatory path information. Finally, the system automatically determines the suspicious behaviour of the monitored subject.

Another example of a system and method for detecting aggressive behaviour is disclosed in a China Patent No. 101557506 A. The system comprises a video sensor arranged at the top part of an elevator car, an embedded system used for transmitting video data, and a monitoring centre computer used for monitoring the interior of the elevator car. The monitoring centre computer comprises a video image reading module used for displaying the video data in the elevator car in real time and a microprocessor used for the detecting aggressive behaviour in the elevator car. The microprocessor distinguishes between normal behaviour and abnormal behaviour by converting all codebook sets to feature vectors observation, obtaining optimal hidden Markov model parameters and calculating forward-backward algorithm observation sequence.

Although there are many systems and methods for detecting aggressive behaviour, most of the systems and methods analyse perspective view of the images of the enclosed space which may result in a false detection. This is because an object appears bigger when the object is situated nearer to the camera but the same object appears smaller when the object is situated further away from the camera. Consequently, the optical values for the same object in two different places give different optical values which cause false detection. Therefore, there is a need for a system and method for detecting aggressive behaviour which addresses the mentioned problems.

SUMMARY OF INVENTION

The present invention relates to a system (200) and method for detecting aggressive behaviour comprising. The system (200) comprising an optical flow module

(105), a statistical computing module (106), and a classifier module (107). The optical flow module (105) is configured for subtracting background of an image frame and compute optical flow each image frame, wherein the optical flow is a pattern of apparent motion of objects, surfaces, and edges from the image frame due to relative motion between an observer and the image frame. The statistical computing module

(106) is configured for computing statistical value of the optical flow based on a dimension of the enclosed space and location of a camera (101). The classifier module

(107) is configured for classifying the statistical value to determine if an aggressive behaviour occurs. The system (200) further comprising an image calibration module (104) configured for calibrating the image frame according to a dimension of an enclosed space and location of the camera (101) by correcting perspective errors in the image frame.

A method for detecting aggressive behaviour includes the steps of calibrating a plurality of image frames based on a dimension of the enclosed space and location of a camera (101) to correct perspective errors in the image frame by an image calibration module (104), subtracting background of the calibrated image frames to obtain foreground of the calibrated image frame by an optical flow module (105) and computing optical flow based on the foreground of the calibrated image frames by the optical flow module (105), wherein the optical flow is the pattern of apparent motion of objects, surfaces, and edges from the image frame due to relative motion between an observer and the image frame. Thereon, a statistical computing module (106) computes statistical value of the optical flow to identify the pattern in the optical flow distribution. Finally, a classifier module (107) determines if an aggressive behaviour occurs in the enclosed space.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

FIG. 1 illustrates a block diagram of a system (200) for detecting aggressive behaviour according to an embodiment of the present invention.

FIG. 2 illustrates a flowchart of a method for detecting aggressive behaviour according to an embodiment of the present invention.

FIG. 3 illustrates an example of a camera (101) installed in an enclosed space.

FIG. 4 (a) illustrates an example of an image frame with a perspective view captured by the camera (101) of FIG. 3.

FIG. 4 (b) illustrates an example of an image frame captured by the camera (101) of FIG. 3 with a calibrated view.

DESCRIPTION OF THE PREFERRED EMBODIMENT

A preferred embodiment of the present invention will be described herein below with reference to the accompanying drawings. In the following description, well known functions or constructions are not described in detail since they would obscure the description with unnecessary detail.

Initial reference is made to FIG. 1 which illustrates a block diagram of a system (200) for detecting aggressive behaviour according to an embodiment of the present invention. The system (200) comprises a video acquisition module (102), a video streaming module (103), an image calibration module (104), an optical flow module (105), a statistical computing module (106), a classifier module (107), a monitoring module (108), and a repository (109). The system (200) is connected to at least one camera (101) to acquire a stream of video from the camera (101), wherein the camera

(101) is installed in an enclosed space. The system (200) analyses the sequence of images to detect any aggressive behaviour in the enclosed space such as prison, psychiatric ward or elderly centre.

Although it is described herein that one camera (101) is installed in the enclosed space, it may be understood that there may be a plurality of cameras (101) installed in the enclosed space. Additionally, although it is described herein that the system (200) detects aggressive behaviour in one enclosed space, it may be understood that the system (200) may detect aggressive behaviour in a plurality of enclosed spaces simultaneously.

The video acquisition module (102) is connected to the camera (101) to acquire video from the camera (101). The video acquisition module (102) is configured to convert the video into a plurality of image frames. The video acquisition module (102) is further connected to the video streaming module (103) to send the image frames for streaming process.

The video streaming module (103) is connected to the video acquisition module

(102) and image calibration module (104). The video streaming module (103) is configured to stream the image frames from the video acquisition module (102) to the image calibration module (104) for a video calibration.

The image calibration module (104), which is connected to the repository (109), is configured to acquire the streamed image frames and calibrate the image frames according to a dimension of the enclosed space and the location of the camera (101) from the repository (109). The image calibration module (104) corrects perspective errors in the image frame by converting the image frame from perspective view to a linear view. The image calibration module (104) is further connected to the optical flow module (105) and is further configured to send the calibrated image frames to the optical flow module (105) for video analytic processing. The optical flow module (105) is configured to acquire the calibrated image frames from the image calibration module (104) and subtract background of the image frames. The optical flow module (105) is also configured to compute optical flow of each image frame, wherein the optical flow is a pattern of apparent motion of objects, surfaces, and edges from the image frame due to relative motion between an observer and the image frame. The optical flow contains movement flow information of an object from a previous image frame to a current image frame in terms of magnitude and direction of the object. The magnitude of the optical flow represents the spread of the optical values in the scene.

The statistical computing module (106) is connected to the optical flow module (105), the classifier module (107), and repository (109). The statistical computing module (106) obtains the dimension of the enclosed space and the location of the camera (101) from the repository (109) and obtains the optical flow from the optical flow module (105). The statistical computing module (106) is configured to compute the statistical value of the optical flow based on the dimension of the enclosed space and the location of the camera (101) and send the statistical value to the classifier module (107) for classification. The statistical value includes but not limited to an average magnitude of the optical flow, variance and standard deviation of the magnitude of the optical flow, as well as variance and standard deviation of the direction of the optical flow.

The classifier module (107), which is connected to the repository (109) and statistical computing module (106), is configured to classify the statistical value based on the input from the repository (109) and statistical computing module (106) to determine if an aggressive behaviour occurs. The classifier module (107) acquires the dimension of the enclosed space and the location of the camera (101) from the repository (109) and selects a classifier model stored in the repository (109) to be used to classify the statistical value based on the dimension of the enclosed space and the location of the camera (101). The classifier model includes but not limited to a decision tree, Naive Bayes, Artificial Neural Networks, and k-Nearest Neighbour.

The repository (109), which is connected to the image calibration module (104), the statistical computing module (106), and classifier module (107), stores the dimension of each enclosed space and the location of each camera (101). The repository (109) also stores the classifier model to be used to classify the statistical value.

The monitoring module (108) is connected to the classifier module (107). The monitoring module (108) is configured to display an area where the aggressive behaviour occurs. Additionally, the monitoring module (108) is configured to alert a person in charge if any aggressive behaviour is detected.

Reference is now made to FIG. 2 which illustrates a flowchart of a method for detecting aggressive behaviour according to an embodiment of the present invention. Initially, the video acquisition module (102) acquires the video from the camera (101) and converts the video into the sequence of image frames as in step 2100. The image frames are then streamed by the video streaming module (103) for video analytic process as in step 2200.

Thereon, the image calibration module (104) calibrates each image frame based on the dimension of the enclosed space and the location of the camera (101) obtained from the repository (109) to correct perspective errors in the image frame as in step 2300. The image calibration module (104) applies an image calibration algorithm to the image frame to convert the image frame in the perspective view to a linear view, wherein the image calibration algorithm includes but not limited to a transformation matrix and a projection matrix. Consequently, an object appears relatively same size with respect to any position in the image frame. For example, the camera (101) is installed in an enclosed space as shown FIG.

3, wherein the enclosed space has a space width and a space height which are stored in the repository (109) as the dimension of the enclosed space. As shown in FIG. 3, the camera (101) is installed at one wall ofthe enclosed space at a device height which is stored in the repository (109) as the location of the camera (101).

As the camera (101) is installed at one end of the enclosed space, the camera (101) captures the video of the enclosed space in a perspective view as shown in FIG. 4 (a). As seen in FIG. 4 (a) which illustrates the perspective view of the floor of the enclosed space, the side of the floorwhich is closerto the camera (101) appears longer than the side of the floor which is further away from the camera (101). Therefore, the floor of the enclosed space appears to be in a parallelogram shape.

The image calibration module (104) calibrates each image frame based on the dimension of the enclosed space and the location of the camera (101) obtained from the repository (109) by applying an image calibration algorithm to the image frame. The image calibration module (104) converts the image frame which is in the perspective view to the linear view. As a result of the calibration, the floor of the enclosed space which initially appears to be in a parallelogram shape in the perspective view is now appeared in a rectangle shape in the linear view as shown in FIG. 4 (b). Consequently, an object in the enclosed room also appears relatively same size regardless whether the object is situated far away from the camera (101) or close to the camera (101).

Thereafter, the optical flow module (105) acquires the calibrated image frames from the image calibration module (104) and subtracts background of the calibrated image frames as in step 2400. Subsequently, the optical flow module (105) obtains foreground of the calibrated image frames and computes optical flow based on the foreground of the calibrated image frames as in step 2500. The optical flow is the pattern of apparent motion of objects, surfaces, and edges from the image frame due to relative motion between an observer and the image frame. The optical flow contains movement flow information of an object from the previous image frame to the current image frame in terms of magnitude and direction of the object. The magnitude of the optical flow represents the spread of the optical values in the scene.

The optical flow is sent to the statistical computing module (106) and the statistical computing module (106) acquires the dimension of the enclosed space and the location of the camera (101) from the repository (109). The statistical computing module (106) then filters any outliers in the data and computes the statistical value of the optical flow to identify the pattern in the optical flow distribution as in step 2600. The outliers which are also known as noises refer to data points that are distant from the rest of the data points. The outliers are filtered out because the outliers affect the statistical analysis and result in an inaccurate statistical value. The statistical value includes but not limited to an average magnitude of the optical flow, variance and standard deviation of the magnitude of the optical flow, and variance and standard deviation of the direction of the optical flow. The statistical value of the optical flow is then normalised by using the dimension of the enclosed space and location of the camera (101). As a result of normalisation, the optical flow of an object movement shows relatively same optical flow range of values with respect to the different distance between the camera (101) and the object.

The classifier module (107) acquires the statistical value from the statistical computing module (106) and the dimension of the enclosed space and the location of the camera (101) from the repository (109). The classifier module (107) selects a classifier model to classify the statistical value based on the dimension of the enclosed space and location of the camera (101). The classifier module (107) selects the classifier model only once during an initialisation process. The classifier model includes but not limited to a decision tree, Naive Bayes, Artificial Neural Networks, and k- Nearest Neighbour. The classifier module (107) employs the selected classifier model on the statistical value to determine if an aggressive behaviour occurs in the enclosed space as in step 2700.

Finally, the monitoring module (108) displays the area where the aggressive behaviour is detected and alerts the person in charge if any aggressive behaviour is detected as in step 2800.

While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrate and describe all possible forms of the invention. Rather, the words used in the specifications are words of description rather than limitation and various changes may be made without departing from the scope of the invention.

Claims

1 . A system (200) for detecting aggressive behaviour comprising:

a) an optical flow module (105) configured for subtracting background of an image frame and compute optical flow for each image frame, wherein the optical flow is a pattern of apparent motion of objects, surfaces, and edges from the image frame due to relative motion between an observer and the image frame;

b) a statistical computing module (106) configured for computing statistical value of the optical flow based on a dimension of an enclosed space and location of a camera (101); and

c) a classifier module (107) configured for classifying the statistical value to determine if the aggressive behaviour occurs,

characterised in that the system (200) further comprising:

d) an image calibration module (104) configured for calibrating the image frame according to the dimension of the enclosed space and location of the camera (101) by correcting perspective errors in the image frame.

2. The system (200) as claimed in claim 1 , wherein the system (200) further comprising:

a) a video acquisition module (102) configured for acquiring video from the camera (101) and converting the video into a plurality of image frames; and

b) a video streaming module (103) configured for streaming the image frames from the video acquisition module (102) to the image calibration module (104).

3. The system (200) as claimed in claim 1 , wherein the system (200) further comprising a repository (109) configured for storing the dimension of each enclosed space, the location of the camera (101) in the enclosed space and classifier models to be used by the classifier module (107) to classify the statistical value.

4. The system (200) as claimed in claim 1 , wherein the system (200) further comprising a monitoring module (108) configured for displaying an area where the aggressive behaviour occurs and alerting a person in charge if any aggressive behaviour is detected.

5. A method for detecting aggressive behaviour is characterised by the steps of:

a) calibrating a plurality of image frames based on a dimension of an enclosed space and location of a camera (101) to correct perspective errors in the image frame by an image calibration module (104);

b) subtracting background of the calibrated image frames to obtain foreground of the calibrated image frame by an optical flow module (105);

c) computing optical flow based on the foreground of the calibrated image frames by the optical flow module (105), wherein the optical flow is the pattern of apparent motion of objects, surfaces, and edges from the image frame due to relative motion between an observer and the image frame;

d) computing statistical value of the optical flow to identify the pattern in the optical flow distribution by a statistical computing module (106); and e) determining if an aggressive behaviour occurs in the enclosed space by a classifier module (107).

6. The method as claimed in claim 5, wherein prior to calibrating a plurality of image frames based on the dimension of the enclosed space and location of a camera (101) to correct perspective errors in the image frame by an image calibration module (104) the method includes the steps of:

a) converting a video acquired from the camera (101) into a sequence of image frames by a video acquisition module (102); and b) streaming the sequence of image frames by a video streaming module (103).

7. The method as claimed in claim 5, wherein the method further includes the steps of:

a) displaying the enclosed area where the aggressive behaviour is detected by a monitoring module (108); and

b) alerting a person in charge if any aggressive behaviour is detected by the monitoring module (108).