WO2012173465A1 - System and method of validation of object counting - Google Patents

System and method of validation of object counting Download PDF

Info

Publication number
WO2012173465A1
WO2012173465A1 PCT/MY2012/000120 MY2012000120W WO2012173465A1 WO 2012173465 A1 WO2012173465 A1 WO 2012173465A1 MY 2012000120 W MY2012000120 W MY 2012000120W WO 2012173465 A1 WO2012173465 A1 WO 2012173465A1
Authority
WO
WIPO (PCT)
Prior art keywords
count
blob
current
motion
blobs
Prior art date
Application number
PCT/MY2012/000120
Other languages
French (fr)
Inventor
Kim Meng Liang
Sze Ling Tang
Kadim Zulaikha
Samudin Norshuhada
Original Assignee
Mimos Berhad
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mimos Berhad filed Critical Mimos Berhad
Publication of WO2012173465A1 publication Critical patent/WO2012173465A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Definitions

  • the present invention relates generally to surveillance technologies, and more particularly to a method and system for object counting applicable in surveillance technologies.
  • Object counting is an important part of video surveillance and video analysis systems.
  • the object counting is an automatic process that detects and counts the objects present in a scene or passing through a monitored area or line.
  • the accuracy of the object counting in the prior arts is still unsatisfactory.
  • the issue of accuracy is heighted.
  • the count of the number of objects can vary from one frame to another due to different movement patterns of objects at each frame.
  • one objective of the present invention is to provide a system and method for accurate object counting with validating the count of objects in each motion blob for each frame and normalizing the inconsistent count of objects from each frame in order to derive the final count of objects that pass a monitored area or line.
  • One aspect of the present invention provides a method of object counting with validation of the number of count in motion blobs, wherein the method is executed in a computer-executable electronic medium.
  • the method comprises obtaining a temporal sequence of image frames for an interest area from a capturing device; detecting and tracking motion blobs in each of the frames; calculating the number of count of each of the motion blobs for each of the frames; validating the number of count of each of motion blobs as valid or invalid by analysis of the relationships of motion blobs in a previous frame and a current frame; wherein the current frame is one of the frames where the motion blobs are to be validated, and the detected motion blobs in the current frame are designated as the current generation; and wherein the previous frame is the frame prior to the current frame, and the detected motion blobs in the previous frame are designated as the previous generation (source); and determining the final number of count by validating the tolerance range of the number of counts; thereby the final number
  • the step of validating the number of count of each of motion blobs comprises identifying corresponding motion blobs in the current and previous frames, wherein the corresponding motion blobs refer to the motion blobs in current and previous frames that are overlapped with each other; determining the blob relationship between the current and previous generations of corresponding motion blobs; if the blob relationship is blob merging, the total number of count from merged motion blobs at the current frame is assigned as current total count, and the total number of count from the motion blobs that participated in merging in the previous frame is determined as source total count; and if the blob relationship is not blob merging, further determining whether the blob relationship is a splitting one; if it is not blob splitting, the status of the number of count for the motion blob at current frame is assigned as valid; and if it is blob splitting, the number of count from the motion blobs that are the resultant splitted
  • the computing tolerance range is based on an offset value, wherein the offset value is approximated based on the calculation error of the estimated number of count for any motion blob; wherein the tolerance range for a generation count is defined as the range between its minimum and maximum counts; and wherein the minimum count for a generation count is computed by subtracting the generation count by the offset value, while maximum count is computed by adding the offset value to the generation count.
  • the step of calculating overlapping ratio comprises dividing the overlapping area between the tolerance ranges of previous generation count and current generation count to the area in the tolerance range of previous generation count or area in the tolerance range of current generation count.
  • the step of determining the final number of count comprises deciding whether the number of count of each of the motion blobs in the current frame after validation is valid; if it is valid, extracting the number of entries in the buffer, wherein the number of entries represents the actual number of validated counts stored in the buffer, in terms of number of frames; and deciding whether the extracted numbers of count are more than a predefined threshold; if the extracted numbers of count is more than the predefined threshold, calculating the majority value of the extracted numbers of count; determining the tolerance range values of the majority values; deciding whether the number of count at the current frame is within the tolerance range of the majority value; if yes, storing the number of count at the current frame into the buffer, and then determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count; if no, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count; and if it is not valid, determining the average value
  • the system comprises an image capturing device for capturing the images of objects in/out a monitored area; a microprocessor electronically coupled with the capturing device for importing the captured images; and a displaying device electronically coupled with the microprocessor for displaying the number of count output from the microprocessor; wherein the microprocessor comprises a computer- executable electronic medium embedded therein a software program for performing a method of object counting with validation of the number of count in motion blobs.
  • FIG 1 is a flowchart illustrating the method of accurate object counting by validating the number of count in each frame in accordance with one embodiment of the present invention.
  • FIG 2 is a flowchart of validating the number of count of each motion blob at each individual frame in accordance with one embodiment of the present invention.
  • FIG 3 illustrates blob merging
  • FIG 4 illustrates blob splitting
  • FIG 5 is a flowchart of determining the final number of count in accordance with one embodiment of the present invention.
  • known automatic counting systems are able to track and detect the objects in the form of motion blobs in a monitored area, and the number of count in each frame is estimated from the detected blobs.
  • the number of counts means the number of detected objects.
  • the captured images of the monitored area are converted into a temporal sequence of image frames.
  • a frame to be validated is designated as the current frame, and the frame prior to the current frame is designated as the previous frame; correspondingly, the detected blobs in the current frame are designated as the current generation, and the detected blobs in the previous frame as the previous generation (source).
  • the relationship of the detected blobs between the current generation and previous generation is measured by their status changes, where the relationship ranges from blob splitting to blob merging.
  • the blob splitting refers to a blob from the source that is split into two or more blobs in the current generation
  • the blob merging means that two or more blobs from the source are merged into one blob in the current generation.
  • One aspect of the present invention provides a method for accurate object counting by validating the number of count for motion blobs in each frame, wherein the method is implemented in a computer-executable electronic medium.
  • the validation is based on the blob relationship between previous and current generations.
  • FIG I there is provided a flowchart illustrating the method of accurate object counting by validating the number of count in each frame in accordance with one embodiment of the present invention.
  • the method commences to detect and track objects in each frame in the form of blobs 100, then calculates the number of count in each frame 200, then validates the number of count of each frame based on the relationships of detected blobs 300, and determines the final number of count at multiple frames 400.
  • the number of count at multiple frames means that the number of objects is determined throughout the sequential frames and the value represents the final number of count.
  • the validated number of count is output to a displaying device.
  • the capturing and displaying devices can be any suitable ones known in the art. In order not to unnecessarily obscure the present invention, the operation of the steps 100 and 200 will not be described in details herein because they are well known to one skilled in the art.
  • FIG 2 there is provided a flowchart of validating the number of count of each motion blob at each individual frame in accordance with one embodiment of the present invention.
  • the validation starts by first identifying the corresponding motion blobs in current and previous frames and determining the blob relationship between the current and previous generations of corresponding motion blobs 301, and then to determine whether the blob relationship of corresponding motion blobs is a merging one 302.
  • the corresponding motion blobs refer to the motion blobs in current and previous frame that are overlapped with each other.
  • the blob relationship is blob merging
  • the total number of count from merged motion blobs at current frame is assigned as current total count 303, and the total number of count from the motion blobs that participated in merging in the previous frame is determined as source total count 304; then the tolerance range for source total count and current total count is computed 309, and the overlapping ratio between the computed tolerance ranges is calculated 310; then a decision is made to whether the overlapping ratio is larger than a predefined threshold 311: if yes, the status of the number of count at current frame from step 200 is assigned as valid 306; if no, the status of the number of count at current frame from step 200 is assigned as invalid 312.
  • the blob relationship is a splitting one 305; if it is not blob splitting, the status of the number of count for that motion blob at current frame from step 200 is assigned as valid and the number of count for that motion blob at the current frame is stored in a buffer for further processing 306; if it is blob splitting, the number of count from the motion blobs that are the resultant splitted blobs in the current frame is accumulated as current total count 307, and the number of count from the motion blobs that participate in splitting in the previous frame is assigned as source total count 308; then the tolerance range for source total count and current total count is computed 309, and the overlapping ratio between the computed tolerance ranges is calculated 310; then a decision is made to whether the overlapping ratio is larger than a predefined threshold 311 : if yes, the status of the number of count for that motion blob at current frame from step 200 is assigned as
  • the buffer is a repository to store the valid number of count of each motion blob throughout the frames where the motion blob enters into the region of interest and out from the region of interest. It is to be noted that each motion blob in the current frame is validated individually based on its corresponding blobs in the previous frame. For example, if there are 2 motion blobs entering the region of interest at one time, then both blobs need to be validated individually. In each frame (while the blobs still within the ROI), the number of count for each blob will be estimated and then validated based on their corresponding blobs in previous frame. If current count for each blob is valid, then both counts will be stored in buffers corresponding to each of the two blobs.
  • the tolerance range for previous (source) generation count and current generation count is computed based on offset value.
  • the offset value is approximated based on the calculation error of the estimated number of count for any motion blob. For example, calculation error of 2 means that the estimated number of count for any blob is either less than or more than 2 from the actual number of count for the blob. Thus in this case, the offset value will be set at 2.
  • the tolerance range for a generation count is defined as the range between its minimum and maximum counts. Minimum count for a generation count is computed by subtracting the generation count by an offset value, while maximum count is computed by adding the offset value to the generation count. Thus, for example, the tolerance range of a generation count of 3 with offset value of 2 is 1 to 5.
  • step 310 it involves calculation of overlapping ratio between the computed tolerance range for previous generation count and the computed tolerance range for current generation count.
  • the overlapping ratio is computed by dividing the overlapping area between the tolerance ranges of previous generation count and current generation count to the area in the tolerance range of previous generation count or area in the tolerance range of current generation count. If the overlapping ratio is larger than a predefined threshold, the status of number of count at individual frame is assigned as valid. Otherwise, the status of number of count per frame basis is assigned as invalid.
  • the computation of the overlapping ratio is exemplarily illustrated herein. Assuming previous generation count is 3, current generation count is 5 and offset is 2, the tolerance range for previous generation count is 1 to 5, while the tolerance range for current generation count is 3 to 7.
  • the overlapping area between previous and current ranges is 3 (from 3 to 5), and the area in the tolerance range of previous or current generation counts is 5 (from 1-5 or from 3-7).
  • the overlapping ratio can be computed by dividing the overlapping area to the area in the tolerance range of previous or current generation count (3/5); this will result in an overlapping ratio of 0.6.
  • FIG 5 there is provided a flowchart of determining the final number of count in accordance with one embodiment of the present invention.
  • a decision is made based on the validation 401. If the number of count of the blob in current frame is valid, extract the number of entries in the buffer 402. The number of entries represents the actual number of validated counts stored in the buffer, in terms of number of frames. For example, if a motion blob appears within the ROI for 3 frames, and in each frame, the number of count for the blob is valid then the count in each frame will be stored in the buffer. Thus the number of entries in the buffer, in this case is 3.
  • the predefined threshold of entries refers to the desired numbers of frames; for example, the threshold can be predefined as 5 to 25 frames. If the extracted numbers of count is more than the predefined threshold, the majority value of the extracted numbers of count is calculated 404.
  • the 'majority value' here means the value of the majority counts stored in the buffer. For example, assuming that the extracted number of count is 5, thus there are 5 count values in the buffer. Further assuming that the count values are ⁇ 3,3,2,1 ,4 ⁇ , then the majority count value is 3. And then the tolerance range values of the majority values are determined 405.
  • the average value of the entries in the buffer is determined 408, where the average value is assigned 409 as the final number of count that is output to the displaying device.
  • the average value In determining the average value, all entries in the buffer are included. It is to be noted that the average values are computed no matter of whether the number of count at the current frame is valid or invalid, but the computation is different. If the current count at the current frame is valid, then the average value is computed based on current content of the buffer including current valid count; but, if the current count at the current frame is invalid, then the average value is computed only based on current content of the buffer not including current count.
  • Buffer is a repository to store the valid number of count at individual frame throughout the frames where the motion blob enter the region of interest and out from the region of interest. If the number of entry is lesser than a predefined threshold, the number of count at individual frame is updated into the buffer. Otherwise, the majority values of the entries in the buffer are calculated.
  • One of the proposed methods to calculate the majority value is by calculating the mode value of the entries in the buffer. In this majority value calculation, it can be more than one majority values. For each majority value, the tolerance range is calculated based on a predefined offset value. The offset value will determine the minimum and maximum values of the tolerance range. If the number of count at individual frame is within the tolerance ranges, the number of count at the individual frame is stored into the buffer.
  • the average value of the entries in the buffer is determined. In the final step, the average value is assigned as the final number of count. If the status of the number of count of the motion blob is invalid, the average value is computed from the entries in the buffer and the average value is assigned as the final number of count.
  • the system comprises an image capturing device for capturing the images of objects in out a monitored area, a microprocessor electronically coupled with the capturing device for importing the captured images, and a displaying device electronically coupled with the microprocessor for displaying the number of count output from the microprocessor.
  • the system can be a surveillance system.
  • the image capturing device can be any suitable one including camera.
  • the displaying device can be any suitable one including a flat monitor.
  • the microprocessor comprises a computer-executable electronic medium embedded therein a software program for performing the method as described above, where the microprocessor can be a computer, PDA, a notebook or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

The present invention provides a system and method for accurate object counting with validating the count of objects in each motion blob for each frame and normalizing the inconsistent count of objects from each frame in order to derive the final count of objects that pass a monitored area or line.

Description

SYSTEM AND METHOD OF VALIDATION OF OBJECT COUNTING
Field of the Invention [0001] The present invention relates generally to surveillance technologies, and more particularly to a method and system for object counting applicable in surveillance technologies.
Background of the Invention
[0002] Object counting is an important part of video surveillance and video analysis systems. In the video surveillance and video analysis systems, the object counting is an automatic process that detects and counts the objects present in a scene or passing through a monitored area or line. The accuracy of the object counting in the prior arts is still unsatisfactory. When two or more objects in a group are under monitoring, the issue of accuracy is heighted. As the group passing through the monitored area or line, the count of the number of objects can vary from one frame to another due to different movement patterns of objects at each frame.
[0003] It is known to be an added advantage if the temporal information of the counts of objects for all the related frames is analyzed together to derive a final number of objects in the particular group. However, if the count of objects in any of the frames is not valid due to the error of counting, the final count of objects will be inaccurate. Furthermore, each count of objects for the same group in each frame is not consistent due to the various movement patterns of the objects in the group. The various movement patterns of the objects causes that similar objects appear in different motion blobs, where the motion blobs are merged and split throughout the frames.
Summary of the Invention [0004] Therefore, one objective of the present invention is to provide a system and method for accurate object counting with validating the count of objects in each motion blob for each frame and normalizing the inconsistent count of objects from each frame in order to derive the final count of objects that pass a monitored area or line.
[0005] One aspect of the present invention provides a method of object counting with validation of the number of count in motion blobs, wherein the method is executed in a computer-executable electronic medium. In one embodiment, the method comprises obtaining a temporal sequence of image frames for an interest area from a capturing device; detecting and tracking motion blobs in each of the frames; calculating the number of count of each of the motion blobs for each of the frames; validating the number of count of each of motion blobs as valid or invalid by analysis of the relationships of motion blobs in a previous frame and a current frame; wherein the current frame is one of the frames where the motion blobs are to be validated, and the detected motion blobs in the current frame are designated as the current generation; and wherein the previous frame is the frame prior to the current frame, and the detected motion blobs in the previous frame are designated as the previous generation (source); and determining the final number of count by validating the tolerance range of the number of counts; thereby the final number of count is displayed in a displaying device.
[0006] In another embodiment of the method, the step of validating the number of count of each of motion blobs comprises identifying corresponding motion blobs in the current and previous frames, wherein the corresponding motion blobs refer to the motion blobs in current and previous frames that are overlapped with each other; determining the blob relationship between the current and previous generations of corresponding motion blobs; if the blob relationship is blob merging, the total number of count from merged motion blobs at the current frame is assigned as current total count, and the total number of count from the motion blobs that participated in merging in the previous frame is determined as source total count; and if the blob relationship is not blob merging, further determining whether the blob relationship is a splitting one; if it is not blob splitting, the status of the number of count for the motion blob at current frame is assigned as valid; and if it is blob splitting, the number of count from the motion blobs that are the resultant splitted blobs in the current frame is accumulated as current total count, and the number of count from the motion blobs that participate in splitting in the previous frame is assigned as source total count; computing tolerance range for the source and current total counts; calculating overlapping ratio between the computed tolerance ranges; determining whether the overlapping ratio is larger than a predefined threshold: if yes, the status of the number of count of the motion blob at the current frame is assigned as valid; if no, the status of the number of count of the motion blob at current frame is assigned as invalid, whereby all valid number of counts are stored in a buffer of the electronic medium for further processing.
[0007] In another embodiment of the method, the computing tolerance range is based on an offset value, wherein the offset value is approximated based on the calculation error of the estimated number of count for any motion blob; wherein the tolerance range for a generation count is defined as the range between its minimum and maximum counts; and wherein the minimum count for a generation count is computed by subtracting the generation count by the offset value, while maximum count is computed by adding the offset value to the generation count.
[0008] In another embodiment of the method, the step of calculating overlapping ratio comprises dividing the overlapping area between the tolerance ranges of previous generation count and current generation count to the area in the tolerance range of previous generation count or area in the tolerance range of current generation count.
[0009] In another embodiment of the method, the step of determining the final number of count comprises deciding whether the number of count of each of the motion blobs in the current frame after validation is valid; if it is valid, extracting the number of entries in the buffer, wherein the number of entries represents the actual number of validated counts stored in the buffer, in terms of number of frames; and deciding whether the extracted numbers of count are more than a predefined threshold; if the extracted numbers of count is more than the predefined threshold, calculating the majority value of the extracted numbers of count; determining the tolerance range values of the majority values; deciding whether the number of count at the current frame is within the tolerance range of the majority value; if yes, storing the number of count at the current frame into the buffer, and then determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count; if no, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count; and if it is not valid, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count. [0010] Another aspect of the present invention provides a system for object counting. In one embodiment, the system comprises an image capturing device for capturing the images of objects in/out a monitored area; a microprocessor electronically coupled with the capturing device for importing the captured images; and a displaying device electronically coupled with the microprocessor for displaying the number of count output from the microprocessor; wherein the microprocessor comprises a computer- executable electronic medium embedded therein a software program for performing a method of object counting with validation of the number of count in motion blobs.
[0011] The objectives and advantages of the invention will become apparent from the following detailed description of preferred embodiments thereof in connection with the accompanying drawings.
Brief Description of the Drawings [0012] Preferred embodiments according to the present invention will now be described with reference to the Figures, in which like reference numerals denote like elements.
[0013] FIG 1 is a flowchart illustrating the method of accurate object counting by validating the number of count in each frame in accordance with one embodiment of the present invention.
[0014] FIG 2 is a flowchart of validating the number of count of each motion blob at each individual frame in accordance with one embodiment of the present invention.
[0015] FIG 3 illustrates blob merging.
[0016] FIG 4 illustrates blob splitting.
[0017] FIG 5 is a flowchart of determining the final number of count in accordance with one embodiment of the present invention.
Detailed Description of the Invention [0018] The present invention may be understood more readily by reference to the following detailed description of certain embodiments of the invention. [0019] Throughout this application, where publications are referenced, the disclosures of these publications are hereby incorporated by reference, in their entireties, into this application in order to more fully describe the state of art to which this invention pertains.
[0020] It is to be noted that known automatic counting systems are able to track and detect the objects in the form of motion blobs in a monitored area, and the number of count in each frame is estimated from the detected blobs. The number of counts means the number of detected objects. In practice, the captured images of the monitored area are converted into a temporal sequence of image frames. For the convenience of description, a frame to be validated is designated as the current frame, and the frame prior to the current frame is designated as the previous frame; correspondingly, the detected blobs in the current frame are designated as the current generation, and the detected blobs in the previous frame as the previous generation (source). The relationship of the detected blobs between the current generation and previous generation is measured by their status changes, where the relationship ranges from blob splitting to blob merging. Throughout this application, the blob splitting refers to a blob from the source that is split into two or more blobs in the current generation, and the blob merging means that two or more blobs from the source are merged into one blob in the current generation.
[0021] One aspect of the present invention provides a method for accurate object counting by validating the number of count for motion blobs in each frame, wherein the method is implemented in a computer-executable electronic medium. In principle, the validation is based on the blob relationship between previous and current generations.
[0022] Now referring to FIG I, there is provided a flowchart illustrating the method of accurate object counting by validating the number of count in each frame in accordance with one embodiment of the present invention. Briefly, after the images captured by the capturing device for a monitored area are processed by the pre-registration into a temporal sequence of frames, the method commences to detect and track objects in each frame in the form of blobs 100, then calculates the number of count in each frame 200, then validates the number of count of each frame based on the relationships of detected blobs 300, and determines the final number of count at multiple frames 400. The number of count at multiple frames means that the number of objects is determined throughout the sequential frames and the value represents the final number of count. The validated number of count is output to a displaying device. The capturing and displaying devices can be any suitable ones known in the art. In order not to unnecessarily obscure the present invention, the operation of the steps 100 and 200 will not be described in details herein because they are well known to one skilled in the art.
[0023] Now referring to FIG 2, there is provided a flowchart of validating the number of count of each motion blob at each individual frame in accordance with one embodiment of the present invention. The validation starts by first identifying the corresponding motion blobs in current and previous frames and determining the blob relationship between the current and previous generations of corresponding motion blobs 301, and then to determine whether the blob relationship of corresponding motion blobs is a merging one 302. The corresponding motion blobs refer to the motion blobs in current and previous frame that are overlapped with each other. If the blob relationship is blob merging, the total number of count from merged motion blobs at current frame is assigned as current total count 303, and the total number of count from the motion blobs that participated in merging in the previous frame is determined as source total count 304; then the tolerance range for source total count and current total count is computed 309, and the overlapping ratio between the computed tolerance ranges is calculated 310; then a decision is made to whether the overlapping ratio is larger than a predefined threshold 311: if yes, the status of the number of count at current frame from step 200 is assigned as valid 306; if no, the status of the number of count at current frame from step 200 is assigned as invalid 312.
[0024] If the blob relationship is not blob merging, it is further determined whether the blob relationship is a splitting one 305; if it is not blob splitting, the status of the number of count for that motion blob at current frame from step 200 is assigned as valid and the number of count for that motion blob at the current frame is stored in a buffer for further processing 306; if it is blob splitting, the number of count from the motion blobs that are the resultant splitted blobs in the current frame is accumulated as current total count 307, and the number of count from the motion blobs that participate in splitting in the previous frame is assigned as source total count 308; then the tolerance range for source total count and current total count is computed 309, and the overlapping ratio between the computed tolerance ranges is calculated 310; then a decision is made to whether the overlapping ratio is larger than a predefined threshold 311 : if yes, the status of the number of count for that motion blob at current frame from step 200 is assigned as valid and the number of count for that motion blob at the current frame is stored in a buffer for further processing 306; if no, the status of the number of count for that motion blob at current frame from step 200 is assigned as invalid 312. The buffer is a repository to store the valid number of count of each motion blob throughout the frames where the motion blob enters into the region of interest and out from the region of interest. It is to be noted that each motion blob in the current frame is validated individually based on its corresponding blobs in the previous frame. For example, if there are 2 motion blobs entering the region of interest at one time, then both blobs need to be validated individually. In each frame (while the blobs still within the ROI), the number of count for each blob will be estimated and then validated based on their corresponding blobs in previous frame. If current count for each blob is valid, then both counts will be stored in buffers corresponding to each of the two blobs.
[0025] The calculation of the current total count and source total count for blob merging condition is depicted in FIG 3; and the calculation of the current total count and source total count for blob splitting condition is depicted in FIG 4.
[0026] For step 309, the tolerance range for previous (source) generation count and current generation count is computed based on offset value. The offset value is approximated based on the calculation error of the estimated number of count for any motion blob. For example, calculation error of 2 means that the estimated number of count for any blob is either less than or more than 2 from the actual number of count for the blob. Thus in this case, the offset value will be set at 2. The tolerance range for a generation count is defined as the range between its minimum and maximum counts. Minimum count for a generation count is computed by subtracting the generation count by an offset value, while maximum count is computed by adding the offset value to the generation count. Thus, for example, the tolerance range of a generation count of 3 with offset value of 2 is 1 to 5.
[0027] For step 310, it involves calculation of overlapping ratio between the computed tolerance range for previous generation count and the computed tolerance range for current generation count. The overlapping ratio is computed by dividing the overlapping area between the tolerance ranges of previous generation count and current generation count to the area in the tolerance range of previous generation count or area in the tolerance range of current generation count. If the overlapping ratio is larger than a predefined threshold, the status of number of count at individual frame is assigned as valid. Otherwise, the status of number of count per frame basis is assigned as invalid. The computation of the overlapping ratio is exemplarily illustrated herein. Assuming previous generation count is 3, current generation count is 5 and offset is 2, the tolerance range for previous generation count is 1 to 5, while the tolerance range for current generation count is 3 to 7. Then the overlapping area between previous and current ranges is 3 (from 3 to 5), and the area in the tolerance range of previous or current generation counts is 5 (from 1-5 or from 3-7). Then the overlapping ratio can be computed by dividing the overlapping area to the area in the tolerance range of previous or current generation count (3/5); this will result in an overlapping ratio of 0.6.
[0028] Now referring to FIG 5, there is provided a flowchart of determining the final number of count in accordance with one embodiment of the present invention. After validating the number of count of each individual motion blobs in the current frame in step 300, a decision is made based on the validation 401. If the number of count of the blob in current frame is valid, extract the number of entries in the buffer 402. The number of entries represents the actual number of validated counts stored in the buffer, in terms of number of frames. For example, if a motion blob appears within the ROI for 3 frames, and in each frame, the number of count for the blob is valid then the count in each frame will be stored in the buffer. Thus the number of entries in the buffer, in this case is 3. And then decide whether the extracted numbers of count are more than a predefined threshold 403. The predefined threshold of entries refers to the desired numbers of frames; for example, the threshold can be predefined as 5 to 25 frames. If the extracted numbers of count is more than the predefined threshold, the majority value of the extracted numbers of count is calculated 404. The 'majority value' here means the value of the majority counts stored in the buffer. For example, assuming that the extracted number of count is 5, thus there are 5 count values in the buffer. Further assuming that the count values are {3,3,2,1 ,4} , then the majority count value is 3. And then the tolerance range values of the majority values are determined 405. Then a decision is made as to whether the number of count at the current frame is within the tolerance range of the majority value 406; if yes, the number of count at the current frame is stored into the buffer 407, and then the average value of the entries in the buffer is determined 408, where the average value is assigned 409 as the final number of count that is output to the displaying device; if no, the average value of the entries in the buffer is determined 408, where the average value is assigned 409 as the final number of count that is output to the displaying device. Back to step 401, if the number of count at the current frame is invalid, the average value of the entries in the buffer is determined 408, where the average value is assigned 409 as the final number of count that is output to the displaying device. In determining the average value, all entries in the buffer are included. It is to be noted that the average values are computed no matter of whether the number of count at the current frame is valid or invalid, but the computation is different. If the current count at the current frame is valid, then the average value is computed based on current content of the buffer including current valid count; but, if the current count at the current frame is invalid, then the average value is computed only based on current content of the buffer not including current count.
[0029] Buffer is a repository to store the valid number of count at individual frame throughout the frames where the motion blob enter the region of interest and out from the region of interest. If the number of entry is lesser than a predefined threshold, the number of count at individual frame is updated into the buffer. Otherwise, the majority values of the entries in the buffer are calculated. One of the proposed methods to calculate the majority value is by calculating the mode value of the entries in the buffer. In this majority value calculation, it can be more than one majority values. For each majority value, the tolerance range is calculated based on a predefined offset value. The offset value will determine the minimum and maximum values of the tolerance range. If the number of count at individual frame is within the tolerance ranges, the number of count at the individual frame is stored into the buffer. Otherwise, the average value of the entries in the buffer is determined. In the final step, the average value is assigned as the final number of count. If the status of the number of count of the motion blob is invalid, the average value is computed from the entries in the buffer and the average value is assigned as the final number of count.
[0030] Another aspect of the present invention provides a system for object counting. The system comprises an image capturing device for capturing the images of objects in out a monitored area, a microprocessor electronically coupled with the capturing device for importing the captured images, and a displaying device electronically coupled with the microprocessor for displaying the number of count output from the microprocessor. The system can be a surveillance system. The image capturing device can be any suitable one including camera. The displaying device can be any suitable one including a flat monitor. The microprocessor comprises a computer-executable electronic medium embedded therein a software program for performing the method as described above, where the microprocessor can be a computer, PDA, a notebook or the like.
[0031] While the present invention has been described with reference to particular embodiments, it will be understood that the embodiments are illustrative and that the invention scope is not so limited. Alternative embodiments of the present invention will become apparent to those having ordinary skill in the art to which the present invention pertains. Such alternate embodiments are considered to be encompassed within the scope of the present invention. Accordingly, the scope of the present invention is defined by the appended claims and is supported by the foregoing description.

Claims

What is claimed is: 1. A method of object counting with validation of the number of count in motion blobs, wherein the method is executed in a computer-executable electronic medium, said method comprising:
obtaining a temporal sequence of image frames for an interest area from a capturing device;
detecting and tracking motion blobs in each of the frames;
calculating the number of count of each of the motion blobs for each of the frames; validating the number of count of each of motion blobs as valid or invalid by analysis of the relationships of motion blobs in a previous frame and a current frame; wherein the current frame is one of the frames where the motion blobs are to be validated, and the detected motion blobs in the current frame are designated as the current generation; and wherein the previous frame is the frame prior to the current frame, and the detected motion blobs in the previous frame are designated as the previous generation (source); and determining the final number of count by validating the tolerance range of the number of counts;
thereby the final number of count is displayed in a displaying device.
2. The method of claim 1 , wherein the step of validating the number of count of each of motion blobs comprises:
identifying corresponding motion blobs in the current and previous frames, wherein the corresponding motion blobs refer to the motion blobs in current and previous frames that are overlapped with each other;
determining the blob relationship between the current and previous generations of corresponding motion blobs;
if the blob relationship is blob merging, the total number of count from merged motion blobs at the current frame is assigned as current total count, and the total number of count from the motion blobs that participated in merging in the previous frame is determined as source total count; and if the blob relationship is not blob merging, further determining whether the blob relationship is a splitting one;
if it is not blob splitting, the status of the number of count for the motion blob at current frame is assigned as valid; and
if it is blob splitting, the number of count from the motion blobs that are the resultant splitted blobs in the current frame is accumulated as current total count, and the number of count from the motion blobs that participate in splitting in the previous frame is assigned as source total count;
computing tolerance range for the source and current total counts;
calculating overlapping ratio between the computed tolerance ranges;
determining whether the overlapping ratio is larger than a predefined threshold: if yes, the status of the number of count of the motion blob at the current frame is assigned as valid; if no, the status of the number of count of the motion blob at current frame is assigned as invalid,
whereby all valid number of counts are stored in a buffer of the electronic medium for further processing.
3. The method of claim 2, wherein the computing tolerance range is based on an offset value, wherein the offset value is approximated based on the calculation error of the estimated number of count for any motion blob; wherein the tolerance range for a generation count is defined as the range between its minimum and maximum counts; and wherein the minimum count for a generation count is computed by subtracting the generation count by the offset value, while maximum count is computed by adding the offset value to the generation count.
4. The method of claim 3, wherein the step of calculating overlapping ratio comprises dividing the overlapping area between the tolerance ranges of previous generation count and current generation count to the area in the tolerance range of previous generation count or area in the tolerance range of current generation count.
5. The method of claim 4, wherein the step of determining the final number of count comprises: deciding whether the number of count of each of the motion blobs in the current frame after validation is valid;
if it is valid, extracting the number of entries in the buffer, wherein the number of entries represents the actual number of validated counts stored in the buffer, in terms of number of frames; and deciding whether the extracted numbers of count are more than a predefined threshold;
if the extracted numbers of count is more than the predefined threshold, calculating the majority value of the extracted numbers of count; determining the tolerance range values of the majority values;
deciding whether the number of count at the current frame is within the tolerance range of the majority value;
if yes, storing the number of count at the current frame into the buffer, and then determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count;
if no, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count; and
if it is not valid, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count.
6. A system for object counting, comprising:
an image capturing device for capturing the images of objects in/out a monitored area;
a microprocessor electronically coupled with the capturing device for importing the captured images; and
a displaying device electronically coupled with the microprocessor for displaying the number of count output from the microprocessor;
wherein the microprocessor comprises a computer-executable electronic medium embedded therein a software program for performing a method of object counting with validation of the number of count in motion blobs, said method comprising: obtaining a temporal sequence of image frames for an interest area from a capturing device;
detecting and tracking motion blobs in each of the frames;
calculating the number of count of each of the motion blobs for each of the frames; validating the number of count of each of motion blobs as valid or invalid by analysis of the relationships of motion blobs in a previous frame and a current frame; wherein the current frame is one of the frames where the motion blobs are to be validated, and the detected motion blobs in the current frame are designated as the current generation; and wherein the previous frame is the frame prior to the current frame, and the detected motion blobs in the previous frame are designated as the previous generation (source); and determining the final number of count by validating the tolerance range of the number of counts;
thereby the final number of count is displayed in a displaying device.
7. The system of claim 6, wherein the step of validating the number of count of each of motion blobs comprises:
identifying corresponding motion blobs in the current and previous frames, wherein the corresponding motion blobs refer to the motion blobs in current and previous frames that are overlapped with each other;
determining the blob relationship between the current and previous generations of corresponding motion blobs;
if the blob relationship is blob merging, the total number of count from merged motion blobs at the current frame is assigned as current total count, and the total number of count from the motion blobs that participated in merging in the previous frame is determined as source total count; and
if the blob relationship is not blob merging, further determining whether the blob relationship is a splitting one;
if it is not blob splitting, the status of the number of count for the motion blob at current frame is assigned as valid; and
if it is blob splitting, the number of count from the motion blobs that are the resultant splitted blobs in the current frame is accumulated as current total count, and the number of count from the motion blobs that participate in splitting in the previous frame is assigned as source total count;
computing tolerance range for the source and current total counts;
calculating overlapping ratio between the computed tolerance ranges;
determining whether the overlapping ratio is larger than a predefined threshold: if yes, the status of the number of count of the motion blob at the current frame is assigned as valid; if no, the status of the number of count of the motion blob at current frame is assigned as invalid,
whereby all valid number of counts are stored in a buffer of the electronic medium for further processing.
8. The system of claim 7, wherein the computing tolerance range is based on an offset value, wherein the offset value is approximated based on the calculation error of the estimated number of count for any motion blob; wherein the tolerance range for a generation count is defined as the range between its minimum and maximum counts; and wherein the minimum count for a generation count is computed by subtracting the generation count by the offset value, while maximum count is computed by adding the offset value to the generation count.
9. The system of claim 8, wherein the step of calculating overlapping ratio comprises dividing the overlapping area between the tolerance ranges of previous generation count and current generation count to the area in the tolerance range of previous generation count or area in the tolerance range of current generation count.
10. The system of claim 9, wherein the step of determining the final number of count comprises:
deciding whether the number of count of each of the motion blobs in the current frame after validation is valid;
if it is valid, extracting the number of entries in the buffer, wherein the number of entries represents the actual number of validated counts stored in the buffer, in terms of number of frames; and deciding whether the extracted numbers of count are more than a predefined threshold; if the extracted numbers of count is more than the predefined threshold, calculating the majority value of the extracted numbers of count; determining the tolerance range values of the majority values;
deciding whether the number of count at the current frame is within the tolerance range of the majority value;
if yes, storing the number of count at the current frame into the buffer, and then determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count;
if no, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count; and
if it is not valid, determining the average value of the entries in the buffer, so that the average value is assigned as the final number of count.
11 , The system of claim 6, wherein the image capturing device is a camera.
PCT/MY2012/000120 2011-06-17 2012-06-14 System and method of validation of object counting WO2012173465A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2011002836 2011-06-17
MYPI2011002836A MY167117A (en) 2011-06-17 2011-06-17 System and method of validation of object counting

Publications (1)

Publication Number Publication Date
WO2012173465A1 true WO2012173465A1 (en) 2012-12-20

Family

ID=46704987

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2012/000120 WO2012173465A1 (en) 2011-06-17 2012-06-14 System and method of validation of object counting

Country Status (2)

Country Link
MY (1) MY167117A (en)
WO (1) WO2012173465A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3156972A1 (en) * 2015-10-13 2017-04-19 Fujitsu Limited Counting apparatus and method for moving objects
US10448006B2 (en) 2016-02-11 2019-10-15 Signify Holding B.V. People sensing system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090296989A1 (en) * 2008-06-03 2009-12-03 Siemens Corporate Research, Inc. Method for Automatic Detection and Tracking of Multiple Objects
WO2011019266A1 (en) * 2009-08-10 2011-02-17 Mimos Berhad Object tracking system and method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090296989A1 (en) * 2008-06-03 2009-12-03 Siemens Corporate Research, Inc. Method for Automatic Detection and Tracking of Multiple Objects
WO2011019266A1 (en) * 2009-08-10 2011-02-17 Mimos Berhad Object tracking system and method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BEIBEI ZHAN ET AL: "Crowd analysis: a survey", MACHINE VISION AND APPLICATIONS, SPRINGER, BERLIN, DE, vol. 19, no. 5-6, 10 April 2008 (2008-04-10), pages 345 - 357, XP019651735, ISSN: 1432-1769, DOI: 10.1007/S00138-008-0132-4 *
TANG S L ET AL: "Hybrid blob and particle filter tracking approach for robust object tracking", PROCEDIA COMPUTER SCIENCE, ELSEVIER, AMSTERDAM, NL, vol. 1, no. 1, 1 May 2010 (2010-05-01), pages 2549 - 2557, XP027492187, ISSN: 1877-0509, [retrieved on 20100501], DOI: 10.1016/J.PROCS.2010.04.288 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3156972A1 (en) * 2015-10-13 2017-04-19 Fujitsu Limited Counting apparatus and method for moving objects
US10448006B2 (en) 2016-02-11 2019-10-15 Signify Holding B.V. People sensing system

Also Published As

Publication number Publication date
MY167117A (en) 2018-08-10

Similar Documents

Publication Publication Date Title
EP2835763B1 (en) A hybrid method and system of video and vision based access control for parking stall occupancy determination
CN108875465B (en) Multi-target tracking method, multi-target tracking device and non-volatile storage medium
CN103975343B (en) For enhancing the system and method that the mankind count by merging the result of mankind's sensed-mode
JP6270433B2 (en) Information processing apparatus, information processing method, and information processing system
JP6508041B2 (en) Object monitoring system, object monitoring method and monitoring target extraction program
US10791252B2 (en) Image monitoring device, image monitoring method, and recording medium
US10701352B2 (en) Image monitoring device, image monitoring method, and recording medium
JP2010056692A5 (en)
JP4600929B2 (en) Stop low-speed vehicle detection device
US20110135158A1 (en) Image processing device, image processing method and program
JP2017534858A5 (en)
US9842452B2 (en) Banknote jam determination system and method
CN111583118B (en) Image stitching method and device, storage medium and electronic equipment
EP2107524A3 (en) Image signal processing apparatus, image signal processing method, and program
WO2014175356A1 (en) Information processing system, information processing method, and program
JP2011258180A5 (en)
JP6792722B2 (en) Vehicle number measurement system
CN113869137A (en) Event detection method and device, terminal equipment and storage medium
US20220366691A1 (en) Object detection
WO2012173465A1 (en) System and method of validation of object counting
CN110889347B (en) Density traffic flow counting method and system based on space-time counting characteristics
WO2014112407A1 (en) Information processing system, information processing method, and program
US9589191B2 (en) Method for evaluating a plurality of time-offset pictures, device for evaluating pictures, and monitoring system
CN104455121A (en) Method and system for detecting residual thickness of brake shoe
CN111784750A (en) Method, device and equipment for tracking moving object in video image and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12748560

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12748560

Country of ref document: EP

Kind code of ref document: A1