WO2020107251A1

WO2020107251A1 - System and method for detecting in-vehicle headcount

Info

Publication number: WO2020107251A1
Application number: PCT/CN2018/117856
Authority: WO
Inventors: Haifeng Shen; Yuan Zhao
Original assignee: Beijing Didi Infinity Technology and Development Co Ltd
Current assignee: Beijing Didi Infinity Technology and Development Co Ltd
Priority date: 2018-11-28
Filing date: 2018-11-28
Publication date: 2020-06-04
Anticipated expiration: 2021-05-28
Also published as: CN111566660A

Abstract

A system for determining a headcount of occupants in a vehicle (100), the system includes at least one camera (110), which is configured to capture at least one image in the vehicle. The system further includes a controller (120) in communication with the camera (110). The controller (120) is configured to detect a plurality of human objects (410) from the image, detect one or more vehicle occupants in each human object (410), and determine the headcount based on the detected vehicle occupants.

Description

SYSTEM AND METHOD FOR DETECTING IN-VEHICLE HEADCOUNT

TECHNICAL FIELD

The present disclosure relates to a system and method for detecting a headcount in a vehicle, and more particularly to, a system and method for automatically detecting a headcount in the vehicle based on images taken inside the vehicle.

BACKGROUND

An online hailing platform (e.g., DiDi ^TM online) provides rideshare services to passengers by dispatching transportation service vehicles (e.g., a taxi, a private car, or the like) . Certain circumstances may cause a high demand of rideshare service, e.g., during rush hours, under inclement weather conditions, or before/after large social gatherings. Accordingly, it becomes difficult to find a transportation service vehicle. Rideshare drivers and passengers may be motivated to overload a service vehicle, in order to reduce wait time to find a ride.

However, vehicle overload causes accidents and poses safety concerns. For example, when a vehicle carries more passengers than its capacity, the weight may affect vehicle steering. In addition, the overload passenger does not have his own seat belt and may sustain more severe injuries during an accident. Therefore, it is important to detect overloaded vehicles and stop them before any accident happens.

Existing vehicle overload detection methods include setting up check points to manually screen vehicles and count the people inside the vehicle. However, manual screening can only spot check vehicles (only those passing the check points) , but cannot effectively check all the vehicles on the road. In addition, the cost of setting up check points and hiring screening officers is inevitably high. Screen may also slow down traffic and cause traffic jams.

Embodiments of the disclosure address the above problems by automatically detect a headcount in the vehicle using images captured by at least one camera inside the vehicle.

SUMMARY

Embodiments of the disclosure provide a system for determining a headcount of occupants in a vehicle. The system includes at least one camera, which is configured to capture at least one image in the vehicle. The system further includes a controller in communication with the at least one camera. The controller is configured to detect a plurality of human objects from the image, detect one or more vehicle occupants in each human object, and determine the headcount based on the detected vehicle occupants.

Embodiments of the disclosure also provide a method for determining a headcount of occupants in a vehicle. The method includes capturing, by at least one camera, at least one image in the vehicle. The method further includes detecting, by a processor, a plurality of human objects from the image, and detecting, by the processor, one or more vehicle occupants in each human object. The method also includes determining, by the processor, the headcount based on the detected vehicle occupants.

Embodiments of the disclosure further provide a non-transitory computer-readable medium that stores a set of instructions. When executed by at least one processor of an electronic device, the set of instructions cause the electronic device to perform a method for determining a headcount of occupants in a vehicle. The method includes capturing at least one image in the vehicle. The method further includes detecting a plurality of human objects from the image, and detecting one or more vehicle occupants in each human object. The method also includes determining the headcount based on the detected vehicle occupants.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a schematic diagram of an exemplary interior of a vehicle equipped with a headcount detection system, according to embodiments of the disclosure.

FIG. 2 illustrates a block diagram of an exemplary controller, according to embodiments of the disclosure.

FIG. 3 illustrates a data flow diagram of an exemplary processor in the controller illustrated in FIG. 2, according to embodiments of the disclosure.

FIG. 4 illustrates a data flow diagram of an exemplary rough headcount estimation unit of FIG. 3, according to embodiments of the disclosure.

FIG. 5 illustrates a data flow diagram of an exemplary fine headcount estimation unit of FIG. 3, according to embodiments of the disclosure.

FIG. 6 illustrates a flowchart of an exemplary method for determining a headcount in a vehicle, according to embodiments of the disclosure.

DETAILED DESCRIPTION

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

FIG. 1 illustrates a schematic diagram of an exemplary vehicle 100 equipped with a conflict detection system, according to embodiments of the disclosure. Consistent with some embodiments, vehicle 100 may be configured to be operated by an operator occupying the vehicle, remotely controlled, and/or autonomous. It is contemplated that vehicle 100 may be an electric vehicle, a fuel cell vehicle, a hybrid vehicle, or a conventional internal combustion engine vehicle. Vehicle 100 may have a body that may be any body style, such as a sports vehicle, a coupe, a sedan, a pick-up truck, a station wagon, a sports utility vehicle (SUV) , a minivan, or a conversion van.

As shown in FIG. 1, the interior of vehicle 100 surrounded by the body may include one or more rows of seats to accommodate people inside the vehicle. For example, the front-row seats may accommodate a driver 102, and a passenger (not show) . The back-row seats 106 may accommodate one or more passengers, such as a passenger 104. Vehicle 100 may include more than two rows of seats to accommodate more passengers. In some embodiments, an arm rest or a cup holder may be installed between the seats. For example, a cup holder may accommodate a water bottle 108.

Vehicle 100 may be designed to accommodate a limited number of occupants, which is known as a vehicle capacity. For example, a coupe may have a capacity of 2-4, a compact vehicle or a sedan may have a capacity of 4-5, a SUV may have a capacity of 5-7, and a minivan may have a capacity of 7-8. If more occupants than its designed capacity is loaded in vehicle 100, vehicle 100 is overloaded. In some embodiments, vehicle 100 may be equipped with a headcount detection system to automatically determine a headcount in the vehicle in order to detect an overload condition.

As illustrated in FIG. 1, the headcount detection system includes, among other things, at least one camera 110 and a controller 120. Camera 110 may be mounted or otherwise installed inside vehicle 100. In some embodiments, camera 110 may be installed on the dashboard, above the windshield, on the ceiling, in the corner, etc. In some embodiments, camera 110 may be integrated in a mobile device, such as a mobile phone, a tablet, or a global positioning system (GPS) navigation device mounted on the dashboard of vehicle 100. In some embodiments, camera 110 may be configured to capture images inside vehicle 100 when vehicle 100 is fulfilling a service trip. Consistent with the present disclosure, cameras 110 may be a digital camera or a digital video camera configured to take pictures or videos of the interior of vehicle 100. The images may capture various objects inside vehicle 100, such as driver 102, passenger 104, empty seat 106, and water bottle 108.

In some embodiments, multiple cameras 110 may be installed at different locations inside vehicle 100 and take pictures of the interior from different view angles. As vehicle 100 travels towards the destination, camera 110 may continuously capture images. Each image captured at a certain time point is known as an image frame. For example, camera 110 may record a video consisting of multiple image frames captured at multiple time points.

Returning to FIG. 1, in some embodiments, camera 110 may communicate with controller 120. In some embodiments, controller 120 may be a controller onboard of vehicle 100, e.g., the electronic control unit, or a vehicle infortainment controller. In some embodiments, controller 120 may be part of a local physical server, a cloud server (as illustrated in FIG. 1) , a virtual server, a distributed server, or any other suitable computing device. Controller 120 may communicate with camera 110, and/or other components of vehicle 100 via a network, such as a Wireless Local Area Network (WLAN) , a Wide Area Network (WAN) , wireless networks such as radio waves, a cellular network, a satellite communication network, and/or a local or short-range wireless network (e.g., Bluetooth ^TM) .

Consistent with the present disclosure, controller 120 may be responsible for processing images captured by cameras 110 and detect an in-vehicle conflict based on the images. In some embodiments, controller 120 may identify human objects, such as driver 102 and one or more passengers 104, using various image processing methods. For example, controller 120 may perform image segmentation and object classification methods to identify the human objects and determine a rough headcount based thereon. Depending on the view angle of camera 110 and the seating positions of the occupants, one occupant may be entirely or partially hidden in the images because of the occupants in front of him. Accordingly, the detected human object may sometimes contain more than one occupant. In some embodiments, controller 120 may further detect one or more vehicle occupants in each human object and determine a fine headcount based on the total vehicle occupants detected in vehicle 100. For example, if two human objects are detected, one including one occupant and the other including two, the fine headcount is three. In some embodiments, controller 120 may compare the determined headcount with the capacity of vehicle 100 to detect an overload condition.

For example, FIG. 2 illustrates a block diagram of an exemplary controller 120, according to embodiments of the disclosure. Consistent with the present disclosure, controller 120 may receive image data 203 from one or more camera 110. In some embodiments, image data 203 may contain two-dimensional (2D) images or three-dimensional (3D) images. In some embodiments, when multiple cameras 110 are installed at different locations inside vehicle 100, image data 203 may contain image data captured from different view angles.

Controller 120 may determine a rough headcount based on human objects detected from image data 203 and determine a fine headcount based on vehicle occupants detected from the human objects. The headcount may be then used to detect an overload condition in vehicle 100. In some embodiments, as shown in FIG. 2, controller 120 includes a communication interface 202, a processor 204, a memory 206, and a storage 208. In some embodiments, controller 120 includes different modules in a single device, such as an integrated circuit (IC) chip (implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA) ) , or separate devices with dedicated functions. In some embodiments, one or more components of controller 120 may be located in a cloud, or may be alternatively in a single location (such as inside vehicle 100 or a mobile device) or distributed locations. Components of controller 120 may be in an integrated device, or distributed at different locations but communicate with each other through a network (not shown) .

Communication interface 202 may send data to and receive data from components such as camera 110 via communication cables, a Wireless Local Area Network (WLAN) , a Wide Area Network (WAN) , wireless networks such as radio waves, a cellular network, and/or a local or short-range wireless network (e.g., Bluetooth ^TM) , or other communication methods. In some embodiments, communication interface 202 can be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection. As another example, communication interface 202 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links can also be implemented by communication interface 202. In such an implementation, communication interface 202 can send and receive electrical, electromagnetic or optical signals that carry digital data streams representing various types of information via a network.

Consistent with some embodiments, communication interface 202 may receive image data 203 captured by cameras 110. Communication interface 202 may further provide the received data to storage 208 for storage or to processor 204 for processing.

Processor 204 may include any appropriate type of general-purpose or special-purpose microprocessor, digital signal processor, or microcontroller. Processor 204 may be configured as a separate processor module dedicated to performing in-vehicle conflict detection based on image data captured by cameras 110. Alternatively, processor 204 may be configured as a shared processor module for performing other functions.

As shown in FIG. 2, processor 204 includes multiple modules, such as a rough headcount estimation unit 210, a fine headcount estimation unit 212, headcount determination unit 214, and the like. In some embodiments, processor 204 may additionally include an overload detection unit 216. These modules (and any corresponding sub-modules or sub-units) can be hardware units (e.g., portions of an integrated circuit) of processor 204 designed for use with other components or software units implemented by processor 204 through executing at least part of a program. The program may be stored on a computer-readable medium, and when executed by processor 204, it may perform one or more functions. Although FIG. 2 shows units 210-216 all within one processor 204, it is contemplated that these units may be distributed among multiple processors located near or remotely with each other.

FIG. 3 illustrates a data flow diagram 300 of processor 204 in controller 120 illustrated in FIG. 2, according to embodiments of the disclosure. As shown in FIG. 3, rough headcount estimation unit 210 may receive image data 203 from communication interface 202 and be configured to determine a rough headcount based on human objects detected from image data 203. Fine headcount estimation unit 212 may further detect one or more vehicle occupants in each human object detected in rough headcount estimation unit 210, and determine a fine headcount based on the total detected vehicle occupants. The rough headcount and the fine headcount may be provided to headcount determination unit 214, which determines a final headcount 302 based thereon.

In some embodiments, image segmentation and object detection methods may be applied by rough headcount estimation unit 210 to identify the human objects. For example, FIG. 4 illustrates a data flow diagram 400 of an exemplary rough headcount estimation unit 210 of FIG. 3, according to embodiments of the disclosure. As shown in FIG. 4, rough headcount estimation unit 210 may further include an object segmentation unit 402 and a human object detection unit 404. Object segmentation unit 402 may receive image data 203 from communication interface 202 and apply segmentation on image data 203 to identify objects from the images. The objects identified through image segmentation may include various objects inside vehicle 100, e.g., human objects, empty seats, bags, seat belts, bottles or cups placed in the cup holders, as well as other objects that may be installed or brought into vehicle 100. In some embodiments, object segmentation unit 402 may apply an object segmentation model 406 to perform the image segmentation. Object detection model 408 may be a machine learning model, such as a CNN model, trained using training images and corresponding objects in those images.

Human object detection unit 404 may then use object detection model 408 to detect human objects among the identified objects. In some embodiments, object detection model 408 may be a machine learning model, such as a CNN model, trained using training object images and corresponding types of objects in those images. For example, the training object images may be labeled with the known objects (e.g., human object, seats, water bottles, etc. ) depicted therein. In some embodiments, the human objects may be identified by determining their contour information.

In some alternative embodiments, object segmentation unit 402 and human object detection unit 404 may be switched in order such that object detection is performed before human object segmentation. For example, human object detection unit 404 may determine bounding areas containing human objects from image data 203, e.g., by applying object detection model 408. The bounding areas may be in any suitable shape, such as rectangular, square, circular, oval, diamond, etc. Object segmentation unit 402 may then apply object segmentation model 406 to segment each bounding area to identify the human objects therein.

Rough headcount estimation unit 210 may provide two outputs: the detected human objects 410 and a rough headcount 412. In some embodiments, rough headcount 412 is the number of human objects 410 detected. Human objects 410 may be received and used by fine headcount estimation unit 212 to further determine a fine headcount. Rough headcount 412 may be received by headcount determination unit 214 to determine a final headcount.

In some embodiments, fine headcount estimation unit 212 may apply head detection and/or skeleton key points detection to detect one or more vehicle occupants in each human object. For example, FIG. 5 illustrates a data flow diagram 500 of an exemplary fine headcount estimation unit 212 of FIG. 3, according to embodiments of the disclosure. As shown in FIG. 5, fine headcount estimation unit 212 may further include a head detection unit 502, a skeleton detection unit 504, and a fusion unit 510. In some embodiments, fine headcount estimation unit 212 may include only one of head detection unit 502 and skeleton detection unit 504, and fusion unit 510 may be omitted.

Head detection unit 502 and skeleton detection unit 504 may separately receive human objects 410 from rough headcount estimation unit 210, and further detect one or more vehicle occupants in each human object. In some embodiments, the processing of head detection unit 502 and skeleton detection unit 504 may be performed in parallel. Head detection unit 502 may apply a head detection model 506 to detect human heads. Head detection model 506 may be a machine learning model, such as a CNN model, trained using training images and human heads labeled in the training images. In some embodiments, fine headcount estimation unit 212 may use the total number of human heads detected across all the human objects as the fine headcount. For example, if two heads are detected in human object I, and another two heads are detected in human object II, the fine headcount is determined as four.

Skeleton detection unit 504 may apply a skeleton detection model 508 to detect human skeletons in each human object. Unlike head detection model 506 that focuses on features of human heads, skeleton detection unit 504 focuses on key point of human skeletons to detect distinct skeletons. Skeleton detection model 508 may be a machine learning model, such as a CNN model, trained using training images and human skeletons labeled in the training images. A human skeleton structure can be defined by a number of key points, such as head, neck, shoulder, wrist, legs, feet, arms, hands, etc. Such key points may be labeled in the training images. In some cases, skeleton detection may be more accurate than head detection for the purpose of detecting distinct occupants in a human object. For example, if a passenger behind a driver has his head entirely invisible in the image, head detection methods may not be able to tell there is another occupant behind the driver. However, as long as some key skeleton points of the passenger is visible in the image, skeleton detection methods may be able to identify the passenger as a distinct occupant.

In some embodiments, both head detection and skeleton detection may be performed, as shown in FIG. 5, to further improve detection accuracy. The detection results from head detection unit 502 and skeleton detection unit 504 may be provided to fusion unit 510, which fuse the detection results to provide the final occupant detection. In some embodiments, fusion unit 510 may perform an OR operation on the two detection results. That is, if one detection method returns two occupants in a human object, and the other detection method returns one occupant in that same human object, fusion unit 510 will adopt the result of two. In some other embodiments, head detection model 506 and skeleton detection model 508 may be jointly trained and applied by fusion unit 510 to detect the occupants. Fusion unit 510 outputs a fine headcount 512 to headcount determination unit 214.

Referring back to FIG. 3, headcount determination unit 214 determines final headcount 302 based on rough headcount 412 and fine headcount 512. In some embodiments, headcount determination unit 214 may perform a max operation. For example, if rough headcount 412 is a, and fine headcount 512 is b, final headcount 302 may be determined as c= max (a, b) .

In some embodiments, processor 204 may execute data flow diagram 300 repeatedly to confirm the headcount or detect any change in headcount in the vehicle. If a headcount is detected based on image data acquired at a particular time point or over a short time period, the detection result may not be reliable. For example, passenger 104 may occasionally bend to pick up an item from the floor, and thus be entirely missing from image data 203. Therefore, processor 204 may periodically repeat the headcount detection to confirm the final headcount and reduce the likelihood of under counting. In some embodiments, processor 204 may generate control signals to cause camera 110 to acquire more images over a relatively long time period, e.g., 10, 20 or 30 seconds. Alternatively, if camera 110 captures a video containing multiple image frames, processor 204 may sample image frames in a span of time, e.g., 10, 20 or 30 seconds. Processor 204 may repeat the detection process performed by units 210-214 for each image frame. If the same headcount is detected persistently across the sampled image frames, headcount determination unit 214 may confirm the final headcount. If the headcount changes over time, headcount determination unit 214 may inquire vehicle operation information, such as vehicle stops, door opening, weight change, etc. to determine if the headcount change is caused by passenger loading or unloading.

Referring back to FIG. 2, overload detection unit 216 may detect an overload condition by comparing the final headcount with a threshold. In some embodiments, the threshold may be pre-determined as the vehicle capacity. For example, if five occupants are detected in a 4-passenger compact vehicle, an overload condition is detected. Upon the detection, processor 204 may generate a control signal to trigger an alarm and send the control signal to a terminal 230 via communication interface 202. In some embodiments, terminal 230 may be a driver terminal or a passenger terminal, such as a smart phone, a PDA, a wearable device, etc. For example, the driver/passenger terminal may have a rideshare application installed that the driver/passenger uses for the transportation service. The overload condition may be notified to driver/passenger through terminal 230 to urge the driver/passenger to end the overload condition. In some embodiments, the control signal may cause a warning notice to be generated by terminal 230, such as a pop-out window on a display screen of terminal 230, a beeping sound, vibrating, or an audio alarm, etc.

In some embodiments, terminal 230 may be a regulation module of the service platform, or a server/controller of a police department. In some embodiments, the control signal may trigger a phone call to terminal 230 to report the overload condition. In some other embodiments, the control signal may trigger a data transmission, including, e.g., vehicle registration information, driver information, passenger information, vehicle location, and the final headcount, to terminal 230. Terminal 230 may intervene to ask the driver/passenger to stop the overload condition immediately. For example, the police department may dispatch an officer near the vehicle location to chase and stop vehicle 100.

Memory 206 and storage 208 may include any appropriate type of mass storage provided to store any type of information that processor 204 may need to operate. Memory 206 and storage 208 may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or tangible (i.e., non-transitory) computer-readable medium including, but not limited to, a ROM, a flash memory, a dynamic RAM, and a static RAM. Memory 206 and/or storage 208 may be configured to store one or more computer programs that may be executed by processor 204 to perform image data processing and conflict detection disclosed herein. For example, memory 206 and/or storage 208 may be configured to store program (s) that may be executed by processor 204 to determine a headcount in vehicle 100, and detect an overload condition based on the headcount.

Memory 206 and/or storage 208 may be further configured to store information and data used by processor 204. For instance, memory 206 and/or storage 208 may be configured to store the various types of data (e.g., image data 203) captured by camera 110 and data related to camera setting. Memory 206 and/or storage 208 may also store intermediate data such as the human objects, head and skeleton features, etc. Memory 206 and/or storage 208 may further store the various learning models used by processor 204, such as object segmentation model 406, object detection model 408, head detection model 506, and skeleton detection model 508. The various types of data may be stored permanently, removed periodically, or disregarded immediately after each frame of data is processed.

FIG. 6 illustrates a flowchart of an exemplary method 600 for determining a headcount in a vehicle, according to embodiments of the disclosure. In some embodiments, method 600 may be implemented by controller 120 that includes, among other things, processor 204. However, method 600 is not limited to that exemplary embodiment. Method 600 may include steps S602-S618 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 6.

In step S602, camera 110 captures image data 203 of at least one object within vehicle 100 when vehicle 100 is fulfilling a service trip. In some embodiments, multiple cameras 110 may be installed at various places inside vehicle 100 and capture image data simultaneously from different angles. For example, camera 110 may be a backward-facing camera installed at the dashboard of vehicle 100 or embedded in a GPS navigation device or cell phone mounted on the dashboard of vehicle 100. In some embodiments, the objects may include a driver (e.g., driver 102) , one or more passengers (e.g., passenger 104) , empty seats (e.g., empty seat 106) , seat belts, and any other items installed inside vehicle 100 or brought into vehicle 100 (e.g., water bottle 108) .

Camera 110 may be configured to capture image data 203 continuously or at certain time points. For example, camera 110 may be a video camera configured to capture a video containing multiple image frames. In some embodiments, image data 203 may contain 2D images and/or 3D images. Image data 203 captured by camera 110 may be transmitted to controller 120, e.g., via a network.

In step S604, controller 120 identifies objects from the images within image data 203 using object segmentation model 406. The objects identified through image segmentation may include various objects inside vehicle 100, e.g., human objects, empty seats, bags, seat belts, bottles or cups placed in the cup holders, as well as other objects that may be installed or brought into vehicle 100. Object detection model 408 may be trained using training images and corresponding objects in those images.

In step S606, controller 120 may identify human objects among the objects detected in step S604, using object detection model 408. Object detection model 408 may be trained using training object images and labeled objects in those images. In some embodiments, the human objects may be identified by determining their contour information.

In some embodiments, step S604 and step S606 may be switched in order. That is, controller 120 may perform object detection first using object detection model 408, to determine bounding areas containing human objects, and then segment each bounding area to identify the human objects using object segmentation model 406. In step S608, controller 120 determines a rough headcount based on the human objects detected in step S606.

In step S610, controller 120 detects heads in each human object using a head detection model 506. Head detection model 506 may be trained using training images and human heads labeled in the training images. In step S612, controller 120 detects skeleton key points in each human object using a skeleton detection model 508. Skeleton detection model 508 may be trained using training images and human skeleton key points labeled in the training images. In some embodiments, controller 120 may perform steps S610 and S612 in parallel to obtain head detection and skeleton detection results. In some embodiments, one of step S610 and S612 may be optional and omitted from method 600.

In step S614, controller 120 determines a fine headcount in the vehicle. In some embodiments, controller 120 may use the total number of human heads detected across all the human objects as the fine headcount. In some other embodiments, controller 120 may use the total number of distinct human skeleton structures detected across all the human objects as the fine headcount. In yet some other embodiments, the detection results from steps S610 and S612 may be fused to determine the final occupant detection. For example, controller 120 may perform an OR operation on the two detection results.

In step S616, controller 120 may compare the final headcount with a preset threshold. For example, the threshold may be set as the vehicle capacity. If the headcount exceeds the threshold (S616: yes) , a vehicle overload condition is detected and method 600 proceeds to step S618 to generate an alarm. Otherwise (S616: no) , method 600 returns to step S602 to continue capturing images inside vehicle 100 and then repeats steps S604-S616 to determine whether vehicle 100 is overloaded. In some embodiments, if the overload condition detected in step S616 is detected persistently across multiple image frames captured by camera 110, the overload may be confirmed.

In step S618, controller 120 generates a control signal to trigger an alarm and sends the control signal to terminal 230. In some embodiments, terminal 230 may be a driver terminal or a passenger terminal used for the rideshare service. Through terminal 230, the driver or the passenger inside vehicle 100 may be notified the overload condition and urged to stop the condition. For example, the control signal may cause a warning notice to be generated by terminal 230, such as a pop-out window on a display screen of terminal 230, a beeping sound, vibrating, or an audio alarm, etc. In some embodiments, if the condition persists after warning, controller 120 may further generate control signal to trigger an alarm to other terminals 230 such as the service platform or a police department. In some embodiments, the control signal may trigger a phone call or a data transmission to alarm receiver 230. For example, the data transmission may include, e.g., vehicle registration information, driver information, passenger information, vehicle location, and the final headcount in the vehicle.

Another aspect of the disclosure is directed to a non-transitory computer-readable medium storing instructions which, when executed, cause one or more processors to perform the methods, as discussed above. The computer-readable medium may include volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of computer-readable medium or computer-readable storage devices. For example, the computer-readable medium may be the storage device or the memory module having the computer instructions stored thereon, as disclosed. In some embodiments, the computer-readable medium may be a disc or a flash drive having the computer instructions stored thereon.

It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed system and related methods. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed system and related methods.

It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the following claims and their equivalents.

Claims

A system for automatically determining a headcount of occupants in a vehicle, comprising:

at least one camera, the camera being configured to capture at least one image in the vehicle; and

a controller in communication with the at least one camera and configured to:

detect a plurality of human objects from the image;

detect one or more vehicle occupants in each human object; and

determine the headcount based on the detected vehicle occupants.
The system of claim 1, wherein to detect the human objects, the controller is configured to:

determine bounding areas containing the human objects from the image; and

segment the bounding areas to detect the human objects.
The system of claim 1, wherein to detect the human objects, the controller is configured to:

segment the at least one image to identify objects; and

detect human objects among the objects based on an object detection model.
The system of claim 1, wherein to detect one or more vehicle occupants in each human object, the controller is configured to detect at least one head in the human object based on a head detection model.
The system of claim 4, wherein the headcount is a total number of heads detected in the human objects.
The system of claim 1, wherein to detect one or more vehicle occupants in each human object, the controller is configured to:

detect key skeleton points in the human object based on a skeleton detection model; and

map the key skeleton points to the one or more vehicle occupants.
The system of claim 1, wherein the controller is further configured to determine a rough headcount in the vehicle based on the identified human objects.
The system of claim 1, wherein the controller is further configured to:

detect an overload condition by comparing the headcount with a vehicle capacity; and

generate an alarm upon detection of the overload condition.
The system of claim 1, wherein the controller is further configured to detect a loading or a unloading event of passengers based on a change of the headcount over time.
A method for automatically determining a headcount of occupants in a vehicle, comprising:

capturing, by at least one camera, at least one image in the vehicle;

detecting, by a processor, a plurality of human objects from the image;

detecting, by the processor, one or more vehicle occupants in each human object; and

determining, by the processor, the headcount based on the detected vehicle occupants.
The method of claim 10, wherein detecting the human objects comprises:

determining bounding areas containing the human objects from the image; and

segmenting the bounding areas to detect the human objects.
The method of claim 10, wherein detecting the human objects comprises:

segmenting the at least one imags to identify objects; and

detecting human objects among the objects based on an object detection model.
The method of claim 10, wherein detecting the one or more vehicle occupants in each human object comprises detecting at least one head in the human object based on a head detection model.
The method of claim 13, wherein the headcount is a total number of heads detected in the human objects.
The method of claim 10, wherein detecting the one or more vehicle occupants in each human object comprises:

detecting key skeleton points in the human object based on a skeleton detection model; and

mapping the key skeleton points to the one or more vehicle occupant.
The method of claim 10, further comprising determining a rough headcount in the vehicle based on the identified human objects.
The method of claim 10, further comprising:

detecting an overload condition by comparing the headcount with a vehicle capacity; and

generating an alarm upon detection of the overload condition.
The method of claim 10, further comprising detecting a loading or a unloading event of passengers based on a change of the headcount over time.
A non-transitory computer-readable medium that stores a set of instructions, when executed by at least one processor of an electronic device, cause the electronic device to perform a method for automatically determining a headcount of occupants in a vehicle, comprising:

capturing at least one image in the vehicle;

detecting a plurality of human objects from the image;

detecting one or more vehicle occupants in each human object; and

determining the headcount based on the detected vehicle occupants.
The non-transitory computer-readable medium of claim 19, wherein the method further comprises:

detecting an overload condition by comparing the headcount with a vehicle capacity; and

generating an alarm upon detection of the overload condition.