WO2022045877A1

WO2022045877A1 - A system and method for identifying occupancy of parking lots

Info

Publication number: WO2022045877A1
Application number: PCT/MY2020/050191
Authority: WO
Inventors: Siti Sofiah BINTI MOHD RADZI; Hock Woon Hon
Original assignee: Mimos Berhad
Priority date: 2020-08-28
Filing date: 2020-12-04
Publication date: 2022-03-03

Abstract

The present invention discloses a system for identifying occupancy of parking lots from a sequence of captured images, the system is characterized by having an instance segmentation module (3), a vehicle orientation module (4), a parking lot occupancy module (5), and a parking lot assignment module (6). The system executes the following steps: performing instance segmentation (205) to segment a vehicle of interest from other detected vehicles in the captured images, determining a center point (404) and a bounding box (306) of the vehicle of interest in the captured images, computing a center point (505) of a parking lot, and matching the vehicle of interest to a designated parking lot by pairing the respective vehicle's center point (404) to the closest parking lot's center point (505) which falls within the bounding box (306) of the vehicle of interest.

Description

A SYSTEM AND METHOD FOR IDENTIFYING OCCUPANCY OF PARKING LOTS

FIELD OF INVENTION

The present invention relates to a system and method for identifying occupancy of parking lots, more particularly, identifying occupancy of parking lots from a perspective view.

BACKGROUND OF THE INVENTION

Conventional approaches for monitoring areas of the environment, such as parking structures, employs cameras installed at multiple locations. For example, outdoor parking lots may be monitored by closed-circuit television (CCTV) installed at a high elevation which allows larger fields of view to be monitored, longer viewing distances, and greater per-camera coverage of people and vehicles on the ground. These CCTV cameras give rise to limitation in terms of evaluating the parking lot availability. When parking lots are not accompanied by vehicles, the parking lots can be seen clearly as individual lots. However, when vehicles are parked in the parking lots, the parking lot areas are often covered or occluded by the vehicle adjacent to it.

Many technologies related to identifying parking lot occupancies have been proposed to further improve the system. For example, a United States patent with publication no. US9477892B2 discloses a method for training a vehicle detection system used in a street occupancy estimation of stationary vehicles. The method includes defining first and second areas on an image plane of an image capture device associated with monitoring the detection of vehicles. The method further includes receiving video-data from a sequence of frames captured from the image capture device and subsequently determining candidate frames which include objects relevant to a classification task in the second area. In addition, the method includes extracting the objects from the candidate frames, extracting features of each extracted object, assigning labels to each extracted object and finally training at least one classifier using the labels and extracted features of the objects.

Another United States patent with publication no. US20180301031A1 discloses a system and method for automatically detecting and mapping points-of-interest (POI) such as parking spaces, and accordingly locating and directing drivers to available parking spaces as close as possible to desired POI and locations. According to the document, the system is completely autonomous and independent and uses a Parking Space Detection module which employs machine learning and computer vision techniques for learning the surface of the parking area, the unoccupied life span of a parking space, the occupancy life span of the parking space, detection of suspicious vehicles in terms of parking searcher to independently predict in which available parking space they may have parked, and navigate in real-time a user to a parking space that has the highest probability to remain free on arrival of said user.

A Chinese patent with publication no. CN108766022B discloses a method and system for parking lot state identification based on machine learning, wherein the method comprises the steps of determining specific installation positions of a camera according to the size information of a specific parking space of the parking lot, in which the camera will acquire images of the parking lots in real-time, mark parking lots and stores the marked image data. All vehicles in the marked parking lots are labelled and the labelled image data are learned in which the machine will map out a parking space model, establish a vehicle identification model and non-vehicle object model.

Another technology as disclosed in United States patent with publication no. US9672434B2 recites a spatiotemporal system and method for parking occupancy detection. According to the document, the system can generally include suitable image acquisition, processing, transmission and data storage devices configured to carry out the method which includes generating and processing spatiotemporal images to detect the presence of an object in a region of interest, such as a vehicle in a parking stall.

The aforementioned patent documents describe the many systems for identifying or detecting the occupancy of parking lots. A major drawback arises therefrom as the systems generally function in general detection of an empty parking lot and assigning labels to the detected vehicles and parking lots. No system have a method for differentiating an empty parking lot with a form of occlusion with respect to the adjacent detected vehicles.

There exists a need to provide a system of such configuration, particularly being able to differentiate the parking lots with respect to the vehicle of interest occupying said parking lots by pairing and computing the shortest distance between their respective center points coordinates.

SUMMARY OF INVENTION

The present invention discloses a method for identifying occupancy of parking lots from a sequence of captured images, the method is characterized by the steps of performing instance segmentation to segment a vehicle of interest from other detected vehicles in the captured images, by an instance segmentation module; determining a center point and a bounding box of the vehicle of interest in the captured images by a vehicle orientation module; computing a center point of a parking lot, by a parking lot occupancy module; and matching the vehicle of interest to a designated parking lot by pairing the respective vehicle's center point to the closest parking lot's center point which falls within the bounding box of the vehicle of interest, by a parking lot assignment module. Preferably, the instance segmentation module is updated by the steps of obtaining a first output from combining a trained semantic segmentation model and a trained object detection model, obtaining a second output from combining the trained object detection model with a traditional segmentation model, and applying the first and second output to a trained instance segmentation model to produce the instance segmentation of the vehicle of interest in the captured images.

Preferably, the first output is created by the steps of training the semantic segmentation model using sample images with a pixel-wise labelled vehicle dataset, applying the training semantic segmentation model to an inferencing engine to produce a semantic segmentation of the vehicles only, and combining the output semantic segmentation with an output instance bounding box from the trained object detection model to be input to the trained instance segmentation model for producing instance segmentation of each detected vehicle.

Preferably, the second output is created by the steps of training the object detection model using sample images with a bounding-box labelled vehicle dataset, applying the trained object detection model to the inferencing engine, producing the instance detection bounding box of the vehicles without instance segmentation, and combining the output instance bounding box of the vehicles with traditional segmentation models to be input to the trained instance segmentation model for producing instance segmentation of each detected vehicle.

Preferably, the trained object detection model is created by the steps of training a model using sample images with a vehicle dataset, applying the trained model to the inferencing engine, and inserting a raw captured image through the trained model to obtain the trained object detection model.

Preferably, the trained model module further comprising the steps of creating an instance segmentation model for vehicles at different perspectives training the instance segmentation model using sample images with the pixel-wise labelled vehicle dataset, and applying the trained instance segmentation model to the inferencing engine to produce the instant segmentation of each detected vehicle with the inputs from the combined first and second outputs from the trained semantic segmentation model and the trained object detection model respectively.

Preferably, the method further comprises the steps of computing an orientation of the vehicle of interest and its bounding box respectively from the captured images, by the vehicle orientation module.

Preferably, the orientation of the bounding box is determined by the steps of computing, a contour from a segmentation of the vehicle of interest to obtain a list of points of the contour, converting the contour to a convex hull to obtain a list of points of the convex hull, calculating a minimum area from the lists of points of the convex hull and the contour, and determining the bounding box from the calculated minimum area, by the vehicle orientation module.

Preferably, the vehicle center point is determined by the steps of converting the instance segmentation of each detected vehicle to a binary image based on a thresholding technique, calculating a moment from the binary image of each detected vehicle, and computing the vehicle’s center point from the moment of each detected vehicle, by the vehicle orientation module.

Preferably, the vehicle of interest is matched to the designated parking lot by the steps of computing parallel heights and widths of parking lot center points to the center point of the bounding box for the vehicle of interest, normalizing the parallel heights and widths of the parking lot center points to the height and widths of the bounding box, computing distances from the parking lot inputs to the center point of the bounding box using the normalized heights and widths of said parking lot center points, and comparing the distances between the parking lots center points to the center point of the bounding box, wherein the parking lots center points with a shortest distance to the center point of the bounding box is the correct designated parking lot for the vehicle of interest.

In another aspect of the present invention, there is provided a system for identifying occupancy of parking lots from a sequence of captured images, the system comprising of an instance segmentation module, configured to perform instance segmentation on to segment a vehicle of interest from other detected vehicles in the captured images; a vehicle orientation module, configured to determine a center point and a bounding box of the vehicle of interest in the captured images; a parking lot occupancy module, configured to define a center point of a parking lot; and a parking lot assignment module, configured to match the vehicle of interest to a designated parking lot by pairing the respective vehicle's center point to the closest parking lot's center point which falls within the bounding box of the vehicle of interest.

One skilled in the art will readily appreciate that the present invention is well adapted to carry out the objects and obtain the ends and advantages mentioned, as well as those inherent therein. The embodiment described herein is not intended as limitations on the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

For the purpose of facilitating an understanding of the invention, there is illustrated in the accompanying drawing the preferred embodiments from an inspection of which when considered in connection with the following description, the invention, its construction and operation and many of its advantages would be readily understood and appreciated. FIG. 1 is a block diagram of a system for identifying occupancy of parking lots from a sequence of captured images, according to the present invention.

FIG. 2 is a flow chart illustrating a preferred embodiment for a method for identifying occupancy of parking lots from a sequence of captured images based on the above- mentioned system.

FIG. 3A illustrates a flowchart for determining an orientation of a vehicle of interest.

FIG. 3B illustrates another exemplary embodiment for determining the orientation of the vehicle of interest.

FIG. 4A illustrates a flowchart for determining a center point of the vehicle of interest.

FIG. 4B illustrates another exemplary embodiment for determining the center point of the vehicle of interest.

FIG. 5A illustrates a flowchart for determining the occupancy of the parking lot.

FIG. 5B illustrates another exemplary embodiment for determining occupancy of the parking lot.

FIG. 6A illustrates a flowchart for differentiating an occupied parking lot with respect to the vehicle of interest.

FIG. 6B illustrates an exemplary embodiment of the center points of adjacent parking lots with respect to the center point of the oriented bounding box. FIG. 6C illustrates an exemplary embodiment of normalizing parallel heights of the center points of adjacent parking lots with respect to the center point of the bounding box of the vehicle of interest.

FIG. 7 illustrates a flowchart for creating a trained model for performing instance segmentation on detected vehicles in the captured images.

FIG. 8 illustrates a flowchart for creating a trained model for detecting vehicles in the captured images.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, the invention shall be described according to the preferred embodiments of the present invention and by referring to the accompanying description and drawings. However, it is to be understood that limiting the description to the preferred embodiments of the invention is merely to facilitate discussion of the present invention and it is envisioned that those skilled in the art may devise various modifications without departing from the scope of the appended claim.

It will be understood that each block of the flowchart illustrations and combinations of blocks in the flowchart illustrations can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, or a special purpose computer and the likes, such that the instructions that execute via the processor of the computer, create means for implementing the functions specified in the flowchart or block diagrams.

These computer program instructions may be stored in a computer-readable memory that can direct a computer or a programmable data processing apparatus to function in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in the flowchart or block diagrams.

The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or block diagrams.

The invention will now be described in greater detail, by way of example, with reference to the drawings.

Referring to FIG. 1, there is provided a system for identifying occupancy of parking lots from a sequence of captured images, the system comprising a video acquisition module 1, wherein the video acquisition module 1 may be a digital camera, digital video recorder, a network video camera or any other mediums suitable for capturing images in a sequence. Preferably, the video acquisition module 1 may include a closed-circuit television (CCTV) camera, which is used for monitoring specific areas of interest. The captured images may then be transferred wirelessly over a communication network to an image processing module 2, wherein the communication network may be a wireless network connection established via a wireless protocol cloud such as a Long-Term Evolution (LTE) cloud, Code Division Multiple Access (CDMA) and its derivatives, Enhanced Data Rates for GSM Evolution (EDGE), 3G protocol, High Speed Packet Access (HSPA), 4G protocol, 5G protocol and the likes, in accordance to the advancement of wireless technology with time. Alternatively, the captured images may be transferred wiredly via a local network port connected from a computer to the video acquisition module 1. In a preferred embodiment, the captured images from the video acquisition module 1 may be stored in an internal memory storage device which can be installed within the video acquisition module or externally linked therewith, wherein the internal or external memory storage device may include primary storage devices such as Raw Access Memory (RAM), which is used by computer systems to temporarily store and retrieve data, or secondary storage devices such as Hard Disk Drives (HDD) or Solid State Drives (SSD) to store data permanently. In a preferred embodiment, the output from the image processing module 2 will be transmitted to a data visualization module 7 which is configured to visualize an occupancy status of each parking lot visible in the captured images. Preferably, the plurality of modules in the image processing module 2 may communicate with one another to form an interconnected communication network.

In one particular embodiment, the image processing module 2 comprises an instance segmentation module 3, configured to perform instance segmentation 205 on each vehicle detected in the captured images. In a preferred embodiment, instance segmentation 205 is a subtype of image segmentation which identifies each instance of each object within an image at a pixel level. Instance segmentation 205 is a computer vision process designed to simplify image analysis by splitting the visual input into segments that represent objects or parts of the objects and form a collection of pixels or “super-pixels”, wherein it sorts pixels into larger components and eliminates the need to consider each pixel as a unit of observation. Preferably, instance segmentation 205 identifies each instance of each object featured in the image instead of categorizing each pixel such as in semantic segmentation. In a preferred embodiment, the instance segmentation module 3 is updated with a trained model module 8 which provides trained models to update the instance segmentation module 3 in order to perform the instance segmentation 205 on detected vehicles in the captured images. In a preferred embodiment, the online trained model module 8 includes trained instance segmentation models 701 such as Mask-RCNN (Regions with Convolutional Neural Networks), DeepMask, and Yolact. Besides that, the online trained model module 8 further includes trained semantic segmentation models 702 such as DeepLab, SegNet, Fully Convolutional Network (FCN), UNET, ENet with trained You Only Look Once (YOLO) object detection models, wherein the trained object detection models 703 include Faster RCNN, Single Shot Detector (SSD), and YOLO with trained semantic segmentation. Alternatively, object detection of the vehicles in the captured images may be conducted with traditional vision segmentation methods such as thresholding, edgebased, region-based, watershed, grab-cut and the likes.

In another particular embodiment, the image processing module 2 comprises a vehicle orientation determination module 4. The vehicle orientation module 4 is configured to provide inputs from the captured images in order to compute an appropriate orientation for a bounding box 306 surrounding a vehicle of interest. Such inputs include the instance segmentation 205 of the detected vehicle from the instance segmentation module 3, which is then computed to obtain various characteristics of the segmented vehicle, such as the contour 307 and convex hull 308 in order to obtain the bounding box 306 corresponding to the orientation of the vehicle of interest in the captured images.

In another particular embodiment, the image processing module 2 comprises a parking lot occupancy module 5, configured to compute a center point 505 of the parking lot. Preferably, the parking lots in the captured images should have one center point 505 for each parking lot. However, due to the perspective view of the video acquisition module 1, the captured images may show parking lots which are occluded from view due to vehicles blocking a section of adjacent parking lots. As a result, the parking lot center points 505 may fall within an area of the bounding box 306 and may cause inaccuracies in determining the occupancy of said parking lot. Therefore, the parking lot occupancy module 5 will further examine whether said center points 505 fall within the oriented bounding box 306 of the occupying detected vehicles to transmit this information to the subsequent module for further analysis which will be discussed further herein. In another particular embodiment, the image processing module 2 comprises a parking lot assignment module 6, configured to compute an actual parking lot occupied by the vehicle of interest by first examining whether the center points 505 of adjacent parking lots fall within the bounding box 306 of the vehicle of interest occupying the parking lot, in which an algorithm may then be applied to determine the detected vehicle is corresponded to which parking lot. The algorithm, in use, computes the distances of the parking lots center points 505 with respect to the center point 404 of the bounding box 306 in order to match the vehicle of interest to its designated parking lot. The steps will be further discussed herein.

FIG. 2 illustrates an exemplary embodiment for a method for identifying occupancy of parking lots from a sequence of captured images based on the above-mentioned system. At Step 201, the video acquisition module 1 first captures a sequence of images of the parking lot configuration in the area of interest, in which the trained model module 8 will input the trained model for detecting vehicles in the captured images at Step 202. At Step 203, the acquired sequence of captured images will be input into the system whereby each captured image will be extracted at Step 204 to be analysed further in the image processing module 2. At Step 205, the instance segmentation module 3 will use the trained model from the trained model 8 module to perform instance segmentation 205 for all detected vehicles in the captured images. At Step 206, the vehicle orientation module 4 will then compute the center point 404 and the orientation of the vehicle of interest and its bounding box 306 respectively based on the instance segmentation 205 information obtained from the instance segmentation module 3. At Step 207, the parking lot occupancy module 5 will determine the occupancy of each parking lot by computing the center points 505 of each parking lot. Further, at Step 208, the parking lot assignment module 6 will differentiate the occupied parking lot with respect to the detected vehicle by applying an algorithm to match the vehicle of interest to its designated parking lot by pairing the vehicle’s center points 404 and the parking lots center points 505. The outcome of the parking lot assignment module 6 is the occupancy status of each individual parking lots which will be visualised by the data visualisation module 7 at Step 209. The steps shown in FIG. 2 are further elaborated in FIG. 3 to FIG. 8

FIG. 3A illustrate a flowchart for determining orientation of the vehicle of interest by the vehicle orientation module 4. At Step 301, the contour 307 of the vehicle of interest is computed from the instance segmentation 205 of the detected vehicles in the captured images obtained by the instance segmentation module 2 in which a list of points of contour 307 may be obtained therefrom, as illustrated in FIG. 3B. The contour 307 is an outline representing or bounding the shape of the detected vehicle. At Step 302, a convex hull 308 is then computed from the contour 307 of the vehicle of interest, in which a list of points of convex hull 308 of the vehicle of interest is subsequently obtained. The convex hull 308 represents a set of points defined as the smallest convex polygons which encloses all the points in the set. At Step 303 and Step 304, a minimum area is calculated from the list of points of both the convex hull 308 and the contour 308 respectively, in which the bounding box 306 of the vehicle of interest is subsequently computed at Step 305 having a corresponding oriented angle 309 with respect to a right angle bounding box 310, which is visualized in a 2D plane of the captured images, as shown in FIG. 3B.

FIG. 4A and FIG. 4B illustrate a flowchart and an exemplary embodiment for determining the center point 404 of the vehicle of interest by the vehicle orientation module 4. At Step 401, the vehicle orientation module 4 will convert the instance segmentation 205 of the detected vehicle to a binary image 405 using a thresholding method, such as illustrated in FIG. 4B. By way of example, the thresholding method involves selecting a threshold value for a colour of a pixel in the captured image, wherein any colour value which is below the selected threshold value is classified as 0, which is black, and all the colour value which is equal or greater than the threshold value are classified as 1, which is white. At Step 402, a moment from the binary image 405 of the vehicle of interest is then calculated, wherein the moment is a weighted average of pixel density for each segmented vehicle and is represented by an equation as follows:

Mij= _x,y(array (x, y) . xA y^f) wherein, M is the center point 404 moment of the vehicle of interest, and x and y represent the 2D coordinates in the captured images for each detected vehicle.

At Step 403, the center point 404 of the vehicle of interest may then be calculated and obtained by employing the following equation:

Wherein, M is the center point 404 moment of the detected vehicle, and x and y represent the 2D coordinates in the captured image for each detected vehicle.

FIG. 5A illustrates a flowchart for determining the parking lot occupancy by the parking lot occupancy module 5. At Step 501, the parking lot occupancy module 5 will determine whether each parking lot center point 505 is inside the oriented bounding box 306 of each detected vehicle. With reference to FIG. 5B, Step 501a illustrates each parking lot in the captured images being defined with respective center points 505. Step 501b and Step 501c illustrate the center point 505 of the parking lot being represented by P, whereby the parking lot center point 505 may be determined by intersecting diagonal lines from one corner of the bounding box 306 to an opposing corner. By doing so, several triangles such as the triangle APD shown in FIG. 5B may be formed in which the area of the bounding box 306 may also be calculated using the areas of said triangles. The area of the bounding box is calculated using the equation as follows:

Area of rectangle ABCD = Area of Triangle APD +Area of Triangle ABP + Area of Triangle BPC + Area of Triangle DPC

If the parking lot center point 505 is not within the bounding box 306 of the vehicle of interest, this means that the parking lot is unoccupied and this information will be transmitted to the data visualization module 7 at Step 503. However, if the parking lot center point 505 is found to be within the bounding box 306 of the vehicle of interest, the parking lot occupancy module 5 will then determine the number of parking lot center points 505 which are within the bounding box 306 of the vehicle of interest at Step 502. If there are more than 1 parking lot center points 505 within the bounding box 306, the parking lot occupancy module 5 will transmit this information to the next module at Step 504. If there is only 1 parking lot center point 505 within the bounding box 306 of the vehicle of interest, this shows that the parking lot is occupied by the vehicle of interest and this information will be subsequently transmitted to the data visualization module 7 at Step 503.

FIG. 6A illustrates an exemplary embodiment of differentiating the occupied parking lot with respect to the detected vehicle by the parking lot assignment module 6. At Step 601, the parking lot assignment module 6 will firstly obtain coordinates of the parking lot center points 505 within the bounding box 306 of the vehicle of interest. With reference to FIG. 6B, the center point 404 of the bounding box 306 is represented by Q, and the parking lot center points 505a, 505b, 505c which fall within the bounding box 306 are represented by Pl, P2 and Pn respectively. Referring to FIG. 6B and FIG. 6C, the coordinates for the parking lot center points 505 may be obtained using a straight line equation just as follows: y = mx + c Referring to FIG. 6C, the coordinates for the center points 505 of the parking lot as denoted as m, n, and o, whereby according to the straight line equation as shown above, are the gradients for the straight line. As illustrated in FIG. 6C, the gradient of the parallel heights 608 with respect to the center point 404 of the bounding box 306 are represented by Q_m, Q_n, and Q_o and the widths 609 are represented by Pim, P2it, and P_no, whereby the gradient of said parallel heights 608 and widths 609 are equal to the gradient of the heights 610, BC and AD, and widths 611, AB and DC, of the bounding box 306.

At Step 602, the parallel heights 608, Q_m, Q„ and Q_o and the widths 609, Pim, P2tt, and P_no of each parking lot center points 505, Pi, P2 and P_n are computed with reference to the center point 404, height 610 and width 611 of the oriented bounding box 306. The computed parallel heights 608 and widths 609 are then normalized at Step 603 with reference to the center point 404, height 610 and width 611 of the oriented bounding box 306 as well, as illustrated in FIG. 6C. At Step 604, the parking lot assignment module 6 will compute the shortest distances from the parking lot center points 505, Pi, P2 and P_n, to the center point 404 of the oriented bounding box 306, Q, using the normalized heights, Q_m, Q_n and Q_o, and normalized widths, Pim, P21 and P_no, in which the minimum shortest distances, PiQ, P2Q, and P_nQ, subsequently determines the occupancy of a corresponded parking lot at Step 605. At Step 606, the parking lot center point 505 with the minimum shortest distance is determined to be the correct occupied parking lot for the corresponding vehicle of interest, wherein the occupancy status of the correct parking lot corresponding with the vehicle of interest will be visualized at Step 607 in the data visualization module 7. In a preferred embodiment, the algorithm applied by the parking lot assignment module 6 to determine which parking lot corresponds to the vehicle of interest is as follows:

If PiQ = mm{PiQ, P2Q,. . P_nQ}, Pi occupied If P2Q = min{PiQ,P2Q,. . PnQ}, P2 occupied If P3Q = mm{PiQ,P2Q,. . PnQ}, P3 occupied

FIG. 7 illustrates an exemplary embodiment for creating the instance segmentation trained model by the trained model module 8. At Step 701, 702 and 703, the online trained model module 8 will create a trained instance segmentation model 701, a trained semantic segmentation model 702 and a trained object detection model 703 respectively for vehicles at different perspectives. At Step 704, the trained instance segmentation model 701 and the trained semantic segmentation model 702 are trained using sample images with a pixel-wise labelled vehicle dataset 704 which may be obtained from online databases. At Step 705, the object detection model 703 is trained using sample images with a bounding box labelled vehicle dataset 705 instead. Subsequently, at Steps 706, 707 and 708 respectively, the trained instance segmentation model 701, the trained semantic segmentation model 702 and the trained object detection model 703 are applied to an inferencing engine. The inferencing engine will then produce the instance segmentation 205 of each detected vehicle in the captured images for the trained instance segmentation model 701 at Step 709. At Step 710, the semantic segmentation 702 of the detected vehicle only is produced from the inferencing engine. At Step 711, an instance detection bounding box without instance segmentation 205 for the trained object detection model 703 is also produced by the inferencing engine. At Step 713, the produced semantic segmentation of the detected vehicles from Step 710 and the instance detection bounding box without instance segmentation from Step 711 are combined to be used as inputs for performing the instance segmentation on the detected vehicle at Step 709. At Step 714, the produced instance detection bounding box is combined with traditional segmentation methods and are then combined to also be used as inputs for producing the instance segmentation for each detected vehicle at Step 709. At Step 712, raw captured images are passed through the trained model from the sample images to obtain a trained model to detect and perform instance segmentation 205 on the detected vehicles in the captured images. FIG. 8 illustrates an exemplary embodiment of creating a trained model for detecting vehicles in the captured images. At Step 801, the online trained model module 8 will create a trained model to detect the vehicles in the parking lot at different perspectives. At Step 802, the model is trained with sample images with a vehicle dataset which may be obtained from online databases. The sample images may include by way of example but not limited to, vehicles of various sizes, shapes, orientation and the likes. At Step 803, the trained model is then applied to the inferencing engine in the online trained model module 8 to be further analysed and computed, in which the raw captured images are passed through the trained model to be further computed at Step 804. Upon completion of the computational step at Step 804, the trained model for detecting vehicles in the captured images is obtained and subsequently input into the image processing module 2.

The present disclosure includes as contained in the appended claims, as well as that of the foregoing description. Although this invention has been described in its preferred form with a degree of particularly, it is understood that the present disclosure of the preferred form has been made only by way of example and that numerous changes in the details of construction and the combination and arrangements of parts may be resorted to without departing from the scope of the invention.

Claims

1. A method for identifying occupancy of parking lots from a sequence of captured images, the method is characterized by the steps of: performing, by an instance segmentation module (3), instance segmentation (205) to segment a vehicle of interest from other detected vehicles in the captured images; determining, by a vehicle orientation module (4), a center point (404) and a bounding box (306) of the vehicle of interest in the captured images; computing, by a parking lot occupancy module (5), a center point (505) of a parking lot; and matching, by a parking lot assignment module (6), the vehicle of interest to a designated parking lot by pairing the respective vehicle's center point (404) to the closest parking lot's center point (505) which falls within the bounding box (306) of the vehicle of interest.

2. The method according to Claim 1, wherein instance segmentation (205) to segment a vehicle of interest is produced by the steps of: obtaining a first output from combining a trained semantic segmentation model

(702) with a trained object detection model (703); obtaining a second output from combining the trained object detection model

(703) with a traditional segmentation model; and applying the first and second output to a trained instance segmentation model (701) to produce an instance segmentation (205) of the vehicle of interest in the captured images.

3. The method according to Claim 2, wherein the first output is created by the steps of: training the semantic segmentation model (702) using sample images with a pixel-wise labelled vehicle dataset (704); applying the training semantic segmentation model (702) to an inferencing engine to produce a semantic segmentation of the vehicles only; and combining the output semantic segmentation with an output instance bounding box from the trained object detection model (703) which will be input to the trained instance segmentation model (701) for producing instance segmentation (205) of each detected vehicle. The method according to Claim 2, wherein the second output is created by the steps of: training the object detection model (703) using sample images with a bounding- box labelled vehicle dataset (705); applying the trained object detection model (703) to the inferencing engine; producing the instance detection bounding box of the vehicles without instance segmentation (205); and combining the output instance bounding box of the vehicles with traditional segmentation models which will be input to the trained instance segmentation model (701) for producing instance segmentation (205) of each detected vehicle. The method according to Claim 4, wherein the trained obj ect detection model (703) is created by the steps of: training a model using sample images with a vehicle dataset; applying the trained model to the inferencing engine; and inserting a raw captured image through the trained model to obtain the trained object detection model (703). The method according to Claim 2 further comprises the steps of: creating an instance segmentation model for vehicles at different perspectives; training the instance segmentation model using sample images with the pixel- wise labelled vehicle dataset; and applying the trained instance segmentation model to the inferencing engine to produce the instant segmentation (205) of each detected vehicle with the inputs from the combined first and second outputs from the trained semantic segmentation model (702) and the trained object detection model (703) respectively. The method according to Claim 1 further comprises the step of computing, by the vehicle orientation module (4), an orientation of the vehicle of interest and its bounding box (306) respectively from the captured images. The method according to Claim 7, wherein the orientation of the bounding box

(306) is determined by the steps of: computing, by the vehicle orientation module (4), a contour (307) from a segmentation (205) of the vehicle of interest to obtain a list of points of the contour

(307); converting, by the vehicle orientation module (4), the contour (307) to a convex hull (308) to obtain a list of points of the convex hull (308); calculating, by the vehicle orientation module (4), a minimum area from the lists of points of the convex hull (308) and the contour (307); and determining, by the vehicle orientation module (4), the bounding box (306) from the calculated minimum area. The method according to Claim 1, wherein the vehicle center point (404) is determined by the steps of: converting, by the vehicle orientation module (4), the instance segmentation (205) of each detected vehicle to a binary image (405) based on a thresholding technique; calculating, by the vehicle orientation module (4), a moment from the binary image (405) of each detected vehicle; and 22 computing, by the vehicle orientation module (4), the vehicle’s center point (404) from the moment of each detected vehicle. The method according to Claim 1, wherein the vehicle of interest is matched to the designated parking lot by the steps of: computing parallel heights (608) and widths (609) of the parking lot center points (505a, 505b, and 505c) to the center point (404) of the bounding box (306) for the vehicle of interest; normalizing the parallel heights (608) and widths (609) of the parking lot center points (505a, 505b and 505c) to the height (610) and widths (611) of the bounding box (306); computing distances from the parking lot center points (505a, 505b, and 505c) to the center point (404) of the bounding box (306) using the normalized heights (608) and widths (609) of said parking lot center points (505a, 505b, and 505c); and comparing the distances between the parking lots center points (505a, 505b, and 505c) to the center point (404) of the bounding box (306), wherein the parking lots center points (505a, 505b, 505c) with a shortest distance to the center point (404) of the bounding box (306) is the correct designated parking lot for the vehicle of interest. A system for identifying occupancy of parking lots from a sequence of captured images, the system is characterized by having: an instance segmentation module (3), configured to perform instance segmentation (205) to segment a vehicle of interest from other detected vehicles in the captured images; a vehicle orientation module (4), configured to determine a center point (404) and a bounding box (306) of the vehicle of interest in the captured images; a parking lot occupancy module (5), configured to define a center point (505) 23 of a parking lot; and a parking lot assignment module (6), configured to match the vehicle of interest to a designated parking lot by pairing the respective vehicle's center point to the closest parking lot's center point which falls within the bounding box of the vehicle of interest.