WO2020107251A1 - Système et procédé de détection d'un effectif de passagers dans un véhicule - Google Patents

Système et procédé de détection d'un effectif de passagers dans un véhicule Download PDF

Info

Publication number
WO2020107251A1
WO2020107251A1 PCT/CN2018/117856 CN2018117856W WO2020107251A1 WO 2020107251 A1 WO2020107251 A1 WO 2020107251A1 CN 2018117856 W CN2018117856 W CN 2018117856W WO 2020107251 A1 WO2020107251 A1 WO 2020107251A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
headcount
human
objects
detect
Prior art date
Application number
PCT/CN2018/117856
Other languages
English (en)
Inventor
Haifeng Shen
Yuan Zhao
Original Assignee
Beijing Didi Infinity Technology And Development Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Didi Infinity Technology And Development Co., Ltd. filed Critical Beijing Didi Infinity Technology And Development Co., Ltd.
Priority to PCT/CN2018/117856 priority Critical patent/WO2020107251A1/fr
Priority to CN201880081102.7A priority patent/CN111566660A/zh
Publication of WO2020107251A1 publication Critical patent/WO2020107251A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K28/00—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
    • B60K28/08—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the cargo, e.g. overload
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/593—Recognising seat occupancy
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/24—Character recognition characterised by the processing or recognition method
    • G06V30/248—Character recognition characterised by the processing or recognition method involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q"
    • G06V30/2504—Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/049—Number of occupants
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07B—TICKET-ISSUING APPARATUS; FARE-REGISTERING APPARATUS; FRANKING APPARATUS
    • G07B15/00—Arrangements or apparatus for collecting fares, tolls or entrance fees at one or more control points
    • G07B15/06—Arrangements for road pricing or congestion charging of vehicles or vehicle users, e.g. automatic toll systems

Definitions

  • the present disclosure relates to a system and method for detecting a headcount in a vehicle, and more particularly to, a system and method for automatically detecting a headcount in the vehicle based on images taken inside the vehicle.
  • An online hailing platform (e.g., DiDi TM online) provides rideshare services to passengers by dispatching transportation service vehicles (e.g., a taxi, a private car, or the like) .
  • transportation service vehicles e.g., a taxi, a private car, or the like
  • rideshare services e.g., a taxi, a private car, or the like
  • Certain circumstances may cause a high demand of rideshare service, e.g., during rush hours, under inclement weather conditions, or before/after large social gatherings. Accordingly, it becomes difficult to find a transportation service vehicle.
  • Rideshare drivers and passengers may be motivated to overload a service vehicle, in order to reduce wait time to find a ride.
  • Embodiments of the disclosure address the above problems by automatically detect a headcount in the vehicle using images captured by at least one camera inside the vehicle.
  • Embodiments of the disclosure provide a system for determining a headcount of occupants in a vehicle.
  • the system includes at least one camera, which is configured to capture at least one image in the vehicle.
  • the system further includes a controller in communication with the at least one camera.
  • the controller is configured to detect a plurality of human objects from the image, detect one or more vehicle occupants in each human object, and determine the headcount based on the detected vehicle occupants.
  • Embodiments of the disclosure also provide a method for determining a headcount of occupants in a vehicle.
  • the method includes capturing, by at least one camera, at least one image in the vehicle.
  • the method further includes detecting, by a processor, a plurality of human objects from the image, and detecting, by the processor, one or more vehicle occupants in each human object.
  • the method also includes determining, by the processor, the headcount based on the detected vehicle occupants.
  • Embodiments of the disclosure further provide a non-transitory computer-readable medium that stores a set of instructions.
  • the set of instructions When executed by at least one processor of an electronic device, the set of instructions cause the electronic device to perform a method for determining a headcount of occupants in a vehicle.
  • the method includes capturing at least one image in the vehicle.
  • the method further includes detecting a plurality of human objects from the image, and detecting one or more vehicle occupants in each human object.
  • the method also includes determining the headcount based on the detected vehicle occupants.
  • FIG. 1 illustrates a schematic diagram of an exemplary interior of a vehicle equipped with a headcount detection system, according to embodiments of the disclosure.
  • FIG. 2 illustrates a block diagram of an exemplary controller, according to embodiments of the disclosure.
  • FIG. 3 illustrates a data flow diagram of an exemplary processor in the controller illustrated in FIG. 2, according to embodiments of the disclosure.
  • FIG. 4 illustrates a data flow diagram of an exemplary rough headcount estimation unit of FIG. 3, according to embodiments of the disclosure.
  • FIG. 5 illustrates a data flow diagram of an exemplary fine headcount estimation unit of FIG. 3, according to embodiments of the disclosure.
  • FIG. 6 illustrates a flowchart of an exemplary method for determining a headcount in a vehicle, according to embodiments of the disclosure.
  • FIG. 1 illustrates a schematic diagram of an exemplary vehicle 100 equipped with a conflict detection system, according to embodiments of the disclosure.
  • vehicle 100 may be configured to be operated by an operator occupying the vehicle, remotely controlled, and/or autonomous. It is contemplated that vehicle 100 may be an electric vehicle, a fuel cell vehicle, a hybrid vehicle, or a conventional internal combustion engine vehicle. Vehicle 100 may have a body that may be any body style, such as a sports vehicle, a coupe, a sedan, a pick-up truck, a station wagon, a sports utility vehicle (SUV) , a minivan, or a conversion van.
  • SUV sports utility vehicle
  • the interior of vehicle 100 surrounded by the body may include one or more rows of seats to accommodate people inside the vehicle.
  • the front-row seats may accommodate a driver 102, and a passenger (not show) .
  • the back-row seats 106 may accommodate one or more passengers, such as a passenger 104.
  • Vehicle 100 may include more than two rows of seats to accommodate more passengers.
  • an arm rest or a cup holder may be installed between the seats.
  • a cup holder may accommodate a water bottle 108.
  • Vehicle 100 may be designed to accommodate a limited number of occupants, which is known as a vehicle capacity.
  • a coupe may have a capacity of 2-4
  • a compact vehicle or a sedan may have a capacity of 4-5
  • a SUV may have a capacity of 5-7
  • a minivan may have a capacity of 7-8. If more occupants than its designed capacity is loaded in vehicle 100, vehicle 100 is overloaded.
  • vehicle 100 may be equipped with a headcount detection system to automatically determine a headcount in the vehicle in order to detect an overload condition.
  • the headcount detection system includes, among other things, at least one camera 110 and a controller 120.
  • Camera 110 may be mounted or otherwise installed inside vehicle 100.
  • camera 110 may be installed on the dashboard, above the windshield, on the ceiling, in the corner, etc.
  • camera 110 may be integrated in a mobile device, such as a mobile phone, a tablet, or a global positioning system (GPS) navigation device mounted on the dashboard of vehicle 100.
  • GPS global positioning system
  • camera 110 may be configured to capture images inside vehicle 100 when vehicle 100 is fulfilling a service trip.
  • cameras 110 may be a digital camera or a digital video camera configured to take pictures or videos of the interior of vehicle 100. The images may capture various objects inside vehicle 100, such as driver 102, passenger 104, empty seat 106, and water bottle 108.
  • multiple cameras 110 may be installed at different locations inside vehicle 100 and take pictures of the interior from different view angles. As vehicle 100 travels towards the destination, camera 110 may continuously capture images. Each image captured at a certain time point is known as an image frame. For example, camera 110 may record a video consisting of multiple image frames captured at multiple time points.
  • controller 120 may be a controller onboard of vehicle 100, e.g., the electronic control unit, or a vehicle infortainment controller.
  • controller 120 may be part of a local physical server, a cloud server (as illustrated in FIG. 1) , a virtual server, a distributed server, or any other suitable computing device.
  • Controller 120 may communicate with camera 110, and/or other components of vehicle 100 via a network, such as a Wireless Local Area Network (WLAN) , a Wide Area Network (WAN) , wireless networks such as radio waves, a cellular network, a satellite communication network, and/or a local or short-range wireless network (e.g., Bluetooth TM ) .
  • WLAN Wireless Local Area Network
  • WAN Wide Area Network
  • wireless networks such as radio waves, a cellular network, a satellite communication network, and/or a local or short-range wireless network (e.g., Bluetooth TM ) .
  • controller 120 may be responsible for processing images captured by cameras 110 and detect an in-vehicle conflict based on the images.
  • controller 120 may identify human objects, such as driver 102 and one or more passengers 104, using various image processing methods. For example, controller 120 may perform image segmentation and object classification methods to identify the human objects and determine a rough headcount based thereon.
  • image segmentation and object classification methods may be performed by controller 120 and perform image segmentation and object classification methods to identify the human objects and determine a rough headcount based thereon.
  • one occupant may be entirely or partially hidden in the images because of the occupants in front of him. Accordingly, the detected human object may sometimes contain more than one occupant.
  • controller 120 may further detect one or more vehicle occupants in each human object and determine a fine headcount based on the total vehicle occupants detected in vehicle 100. For example, if two human objects are detected, one including one occupant and the other including two, the fine headcount is three. In some embodiments, controller 120 may compare the determined headcount with the capacity of vehicle 100 to detect an overload condition.
  • FIG. 2 illustrates a block diagram of an exemplary controller 120, according to embodiments of the disclosure.
  • controller 120 may receive image data 203 from one or more camera 110.
  • image data 203 may contain two-dimensional (2D) images or three-dimensional (3D) images.
  • image data 203 may contain image data captured from different view angles.
  • Controller 120 may determine a rough headcount based on human objects detected from image data 203 and determine a fine headcount based on vehicle occupants detected from the human objects. The headcount may be then used to detect an overload condition in vehicle 100.
  • controller 120 includes a communication interface 202, a processor 204, a memory 206, and a storage 208.
  • controller 120 includes different modules in a single device, such as an integrated circuit (IC) chip (implemented as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA) ) , or separate devices with dedicated functions.
  • IC integrated circuit
  • ASIC application-specific integrated circuit
  • FPGA field-programmable gate array
  • controller 120 may be located in a cloud, or may be alternatively in a single location (such as inside vehicle 100 or a mobile device) or distributed locations. Components of controller 120 may be in an integrated device, or distributed at different locations but communicate with each other through a network (not shown) .
  • Communication interface 202 may send data to and receive data from components such as camera 110 via communication cables, a Wireless Local Area Network (WLAN) , a Wide Area Network (WAN) , wireless networks such as radio waves, a cellular network, and/or a local or short-range wireless network (e.g., Bluetooth TM ) , or other communication methods.
  • communication interface 202 can be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection.
  • ISDN integrated services digital network
  • communication interface 202 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
  • Wireless links can also be implemented by communication interface 202.
  • communication interface 202 can send and receive electrical, electromagnetic or optical signals that carry digital data streams representing various types of information via a network.
  • communication interface 202 may receive image data 203 captured by cameras 110. Communication interface 202 may further provide the received data to storage 208 for storage or to processor 204 for processing.
  • Processor 204 may include any appropriate type of general-purpose or special-purpose microprocessor, digital signal processor, or microcontroller. Processor 204 may be configured as a separate processor module dedicated to performing in-vehicle conflict detection based on image data captured by cameras 110. Alternatively, processor 204 may be configured as a shared processor module for performing other functions.
  • processor 204 includes multiple modules, such as a rough headcount estimation unit 210, a fine headcount estimation unit 212, headcount determination unit 214, and the like.
  • processor 204 may additionally include an overload detection unit 216.
  • These modules can be hardware units (e.g., portions of an integrated circuit) of processor 204 designed for use with other components or software units implemented by processor 204 through executing at least part of a program.
  • the program may be stored on a computer-readable medium, and when executed by processor 204, it may perform one or more functions.
  • FIG. 2 shows units 210-216 all within one processor 204, it is contemplated that these units may be distributed among multiple processors located near or remotely with each other.
  • FIG. 3 illustrates a data flow diagram 300 of processor 204 in controller 120 illustrated in FIG. 2, according to embodiments of the disclosure.
  • rough headcount estimation unit 210 may receive image data 203 from communication interface 202 and be configured to determine a rough headcount based on human objects detected from image data 203.
  • Fine headcount estimation unit 212 may further detect one or more vehicle occupants in each human object detected in rough headcount estimation unit 210, and determine a fine headcount based on the total detected vehicle occupants.
  • the rough headcount and the fine headcount may be provided to headcount determination unit 214, which determines a final headcount 302 based thereon.
  • image segmentation and object detection methods may be applied by rough headcount estimation unit 210 to identify the human objects.
  • FIG. 4 illustrates a data flow diagram 400 of an exemplary rough headcount estimation unit 210 of FIG. 3, according to embodiments of the disclosure.
  • rough headcount estimation unit 210 may further include an object segmentation unit 402 and a human object detection unit 404.
  • Object segmentation unit 402 may receive image data 203 from communication interface 202 and apply segmentation on image data 203 to identify objects from the images.
  • the objects identified through image segmentation may include various objects inside vehicle 100, e.g., human objects, empty seats, bags, seat belts, bottles or cups placed in the cup holders, as well as other objects that may be installed or brought into vehicle 100.
  • object segmentation unit 402 may apply an object segmentation model 406 to perform the image segmentation.
  • Object detection model 408 may be a machine learning model, such as a CNN model, trained using training images and corresponding objects in those images.
  • Human object detection unit 404 may then use object detection model 408 to detect human objects among the identified objects.
  • object detection model 408 may be a machine learning model, such as a CNN model, trained using training object images and corresponding types of objects in those images.
  • the training object images may be labeled with the known objects (e.g., human object, seats, water bottles, etc. ) depicted therein.
  • the human objects may be identified by determining their contour information.
  • object segmentation unit 402 and human object detection unit 404 may be switched in order such that object detection is performed before human object segmentation.
  • human object detection unit 404 may determine bounding areas containing human objects from image data 203, e.g., by applying object detection model 408.
  • the bounding areas may be in any suitable shape, such as rectangular, square, circular, oval, diamond, etc.
  • Object segmentation unit 402 may then apply object segmentation model 406 to segment each bounding area to identify the human objects therein.
  • Rough headcount estimation unit 210 may provide two outputs: the detected human objects 410 and a rough headcount 412.
  • rough headcount 412 is the number of human objects 410 detected.
  • Human objects 410 may be received and used by fine headcount estimation unit 212 to further determine a fine headcount.
  • Rough headcount 412 may be received by headcount determination unit 214 to determine a final headcount.
  • fine headcount estimation unit 212 may apply head detection and/or skeleton key points detection to detect one or more vehicle occupants in each human object.
  • FIG. 5 illustrates a data flow diagram 500 of an exemplary fine headcount estimation unit 212 of FIG. 3, according to embodiments of the disclosure.
  • fine headcount estimation unit 212 may further include a head detection unit 502, a skeleton detection unit 504, and a fusion unit 510.
  • fine headcount estimation unit 212 may include only one of head detection unit 502 and skeleton detection unit 504, and fusion unit 510 may be omitted.
  • Head detection unit 502 and skeleton detection unit 504 may separately receive human objects 410 from rough headcount estimation unit 210, and further detect one or more vehicle occupants in each human object. In some embodiments, the processing of head detection unit 502 and skeleton detection unit 504 may be performed in parallel. Head detection unit 502 may apply a head detection model 506 to detect human heads. Head detection model 506 may be a machine learning model, such as a CNN model, trained using training images and human heads labeled in the training images. In some embodiments, fine headcount estimation unit 212 may use the total number of human heads detected across all the human objects as the fine headcount. For example, if two heads are detected in human object I, and another two heads are detected in human object II, the fine headcount is determined as four.
  • Skeleton detection unit 504 may apply a skeleton detection model 508 to detect human skeletons in each human object. Unlike head detection model 506 that focuses on features of human heads, skeleton detection unit 504 focuses on key point of human skeletons to detect distinct skeletons. Skeleton detection model 508 may be a machine learning model, such as a CNN model, trained using training images and human skeletons labeled in the training images.
  • a human skeleton structure can be defined by a number of key points, such as head, neck, shoulder, wrist, legs, feet, arms, hands, etc. Such key points may be labeled in the training images.
  • skeleton detection may be more accurate than head detection for the purpose of detecting distinct occupants in a human object. For example, if a passenger behind a driver has his head entirely invisible in the image, head detection methods may not be able to tell there is another occupant behind the driver. However, as long as some key skeleton points of the passenger is visible in the image, skeleton detection methods may be able to identify the passenger as a distinct occupant.
  • both head detection and skeleton detection may be performed, as shown in FIG. 5, to further improve detection accuracy.
  • the detection results from head detection unit 502 and skeleton detection unit 504 may be provided to fusion unit 510, which fuse the detection results to provide the final occupant detection.
  • fusion unit 510 may perform an OR operation on the two detection results. That is, if one detection method returns two occupants in a human object, and the other detection method returns one occupant in that same human object, fusion unit 510 will adopt the result of two.
  • head detection model 506 and skeleton detection model 508 may be jointly trained and applied by fusion unit 510 to detect the occupants. Fusion unit 510 outputs a fine headcount 512 to headcount determination unit 214.
  • headcount determination unit 214 determines final headcount 302 based on rough headcount 412 and fine headcount 512.
  • processor 204 may execute data flow diagram 300 repeatedly to confirm the headcount or detect any change in headcount in the vehicle. If a headcount is detected based on image data acquired at a particular time point or over a short time period, the detection result may not be reliable. For example, passenger 104 may occasionally bend to pick up an item from the floor, and thus be entirely missing from image data 203. Therefore, processor 204 may periodically repeat the headcount detection to confirm the final headcount and reduce the likelihood of under counting. In some embodiments, processor 204 may generate control signals to cause camera 110 to acquire more images over a relatively long time period, e.g., 10, 20 or 30 seconds.
  • processor 204 may sample image frames in a span of time, e.g., 10, 20 or 30 seconds. Processor 204 may repeat the detection process performed by units 210-214 for each image frame. If the same headcount is detected persistently across the sampled image frames, headcount determination unit 214 may confirm the final headcount. If the headcount changes over time, headcount determination unit 214 may inquire vehicle operation information, such as vehicle stops, door opening, weight change, etc. to determine if the headcount change is caused by passenger loading or unloading.
  • vehicle operation information such as vehicle stops, door opening, weight change, etc.
  • overload detection unit 216 may detect an overload condition by comparing the final headcount with a threshold.
  • the threshold may be pre-determined as the vehicle capacity. For example, if five occupants are detected in a 4-passenger compact vehicle, an overload condition is detected.
  • processor 204 may generate a control signal to trigger an alarm and send the control signal to a terminal 230 via communication interface 202.
  • terminal 230 may be a driver terminal or a passenger terminal, such as a smart phone, a PDA, a wearable device, etc.
  • the driver/passenger terminal may have a rideshare application installed that the driver/passenger uses for the transportation service.
  • the overload condition may be notified to driver/passenger through terminal 230 to urge the driver/passenger to end the overload condition.
  • the control signal may cause a warning notice to be generated by terminal 230, such as a pop-out window on a display screen of terminal 230, a beeping sound, vibrating, or an audio alarm, etc.
  • terminal 230 may be a regulation module of the service platform, or a server/controller of a police department.
  • the control signal may trigger a phone call to terminal 230 to report the overload condition.
  • the control signal may trigger a data transmission, including, e.g., vehicle registration information, driver information, passenger information, vehicle location, and the final headcount, to terminal 230.
  • Terminal 230 may intervene to ask the driver/passenger to stop the overload condition immediately.
  • the police department may dispatch an officer near the vehicle location to chase and stop vehicle 100.
  • Memory 206 and storage 208 may include any appropriate type of mass storage provided to store any type of information that processor 204 may need to operate.
  • Memory 206 and storage 208 may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or tangible (i.e., non-transitory) computer-readable medium including, but not limited to, a ROM, a flash memory, a dynamic RAM, and a static RAM.
  • Memory 206 and/or storage 208 may be configured to store one or more computer programs that may be executed by processor 204 to perform image data processing and conflict detection disclosed herein.
  • memory 206 and/or storage 208 may be configured to store program (s) that may be executed by processor 204 to determine a headcount in vehicle 100, and detect an overload condition based on the headcount.
  • Memory 206 and/or storage 208 may be further configured to store information and data used by processor 204.
  • memory 206 and/or storage 208 may be configured to store the various types of data (e.g., image data 203) captured by camera 110 and data related to camera setting.
  • Memory 206 and/or storage 208 may also store intermediate data such as the human objects, head and skeleton features, etc.
  • Memory 206 and/or storage 208 may further store the various learning models used by processor 204, such as object segmentation model 406, object detection model 408, head detection model 506, and skeleton detection model 508.
  • the various types of data may be stored permanently, removed periodically, or disregarded immediately after each frame of data is processed.
  • FIG. 6 illustrates a flowchart of an exemplary method 600 for determining a headcount in a vehicle, according to embodiments of the disclosure.
  • method 600 may be implemented by controller 120 that includes, among other things, processor 204.
  • controller 120 includes, among other things, processor 204.
  • method 600 is not limited to that exemplary embodiment.
  • Method 600 may include steps S602-S618 as described below. It is to be appreciated that some of the steps may be optional to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 6.
  • camera 110 captures image data 203 of at least one object within vehicle 100 when vehicle 100 is fulfilling a service trip.
  • multiple cameras 110 may be installed at various places inside vehicle 100 and capture image data simultaneously from different angles.
  • camera 110 may be a backward-facing camera installed at the dashboard of vehicle 100 or embedded in a GPS navigation device or cell phone mounted on the dashboard of vehicle 100.
  • the objects may include a driver (e.g., driver 102) , one or more passengers (e.g., passenger 104) , empty seats (e.g., empty seat 106) , seat belts, and any other items installed inside vehicle 100 or brought into vehicle 100 (e.g., water bottle 108) .
  • Camera 110 may be configured to capture image data 203 continuously or at certain time points.
  • camera 110 may be a video camera configured to capture a video containing multiple image frames.
  • image data 203 may contain 2D images and/or 3D images.
  • Image data 203 captured by camera 110 may be transmitted to controller 120, e.g., via a network.
  • controller 120 identifies objects from the images within image data 203 using object segmentation model 406.
  • the objects identified through image segmentation may include various objects inside vehicle 100, e.g., human objects, empty seats, bags, seat belts, bottles or cups placed in the cup holders, as well as other objects that may be installed or brought into vehicle 100.
  • Object detection model 408 may be trained using training images and corresponding objects in those images.
  • controller 120 may identify human objects among the objects detected in step S604, using object detection model 408.
  • Object detection model 408 may be trained using training object images and labeled objects in those images.
  • the human objects may be identified by determining their contour information.
  • step S604 and step S606 may be switched in order. That is, controller 120 may perform object detection first using object detection model 408, to determine bounding areas containing human objects, and then segment each bounding area to identify the human objects using object segmentation model 406. In step S608, controller 120 determines a rough headcount based on the human objects detected in step S606.
  • controller 120 detects heads in each human object using a head detection model 506.
  • Head detection model 506 may be trained using training images and human heads labeled in the training images.
  • controller 120 detects skeleton key points in each human object using a skeleton detection model 508.
  • Skeleton detection model 508 may be trained using training images and human skeleton key points labeled in the training images.
  • controller 120 may perform steps S610 and S612 in parallel to obtain head detection and skeleton detection results.
  • one of step S610 and S612 may be optional and omitted from method 600.
  • controller 120 determines a fine headcount in the vehicle.
  • controller 120 may use the total number of human heads detected across all the human objects as the fine headcount.
  • controller 120 may use the total number of distinct human skeleton structures detected across all the human objects as the fine headcount.
  • the detection results from steps S610 and S612 may be fused to determine the final occupant detection. For example, controller 120 may perform an OR operation on the two detection results.
  • controller 120 may compare the final headcount with a preset threshold. For example, the threshold may be set as the vehicle capacity. If the headcount exceeds the threshold (S616: yes) , a vehicle overload condition is detected and method 600 proceeds to step S618 to generate an alarm. Otherwise (S616: no) , method 600 returns to step S602 to continue capturing images inside vehicle 100 and then repeats steps S604-S616 to determine whether vehicle 100 is overloaded. In some embodiments, if the overload condition detected in step S616 is detected persistently across multiple image frames captured by camera 110, the overload may be confirmed.
  • a preset threshold For example, the threshold may be set as the vehicle capacity. If the headcount exceeds the threshold (S616: yes) , a vehicle overload condition is detected and method 600 proceeds to step S618 to generate an alarm. Otherwise (S616: no) , method 600 returns to step S602 to continue capturing images inside vehicle 100 and then repeats steps S604-S616 to determine whether vehicle 100 is overloaded. In
  • controller 120 In step S618, controller 120 generates a control signal to trigger an alarm and sends the control signal to terminal 230.
  • terminal 230 may be a driver terminal or a passenger terminal used for the rideshare service. Through terminal 230, the driver or the passenger inside vehicle 100 may be notified the overload condition and urged to stop the condition.
  • the control signal may cause a warning notice to be generated by terminal 230, such as a pop-out window on a display screen of terminal 230, a beeping sound, vibrating, or an audio alarm, etc.
  • controller 120 may further generate control signal to trigger an alarm to other terminals 230 such as the service platform or a police department.
  • the control signal may trigger a phone call or a data transmission to alarm receiver 230.
  • the data transmission may include, e.g., vehicle registration information, driver information, passenger information, vehicle location, and the final headcount in the vehicle.
  • the computer-readable medium may include volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other types of computer-readable medium or computer-readable storage devices.
  • the computer-readable medium may be the storage device or the memory module having the computer instructions stored thereon, as disclosed.
  • the computer-readable medium may be a disc or a flash drive having the computer instructions stored thereon.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Transportation (AREA)
  • Mathematical Physics (AREA)
  • Automation & Control Theory (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un système de détermination d'un effectif de passagers dans un véhicule (100), le système comprenant au moins une caméra (110), qui est configurée pour capturer au moins une image dans le véhicule. Le système comprend en outre un dispositif de commande (120) en communication avec la caméra (110). Le dispositif de commande (120) est configuré pour détecter une pluralité d'objets humains (410) à partir de l'image, détecter qu'un ou plusieurs des objets humains (410) sont des passagers du véhicule, et déterminer l'effectif sur la base des passagers du véhicule détectés.
PCT/CN2018/117856 2018-11-28 2018-11-28 Système et procédé de détection d'un effectif de passagers dans un véhicule WO2020107251A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/117856 WO2020107251A1 (fr) 2018-11-28 2018-11-28 Système et procédé de détection d'un effectif de passagers dans un véhicule
CN201880081102.7A CN111566660A (zh) 2018-11-28 2018-11-28 用于检测车辆中的人数的系统和方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/117856 WO2020107251A1 (fr) 2018-11-28 2018-11-28 Système et procédé de détection d'un effectif de passagers dans un véhicule

Publications (1)

Publication Number Publication Date
WO2020107251A1 true WO2020107251A1 (fr) 2020-06-04

Family

ID=70854258

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/117856 WO2020107251A1 (fr) 2018-11-28 2018-11-28 Système et procédé de détection d'un effectif de passagers dans un véhicule

Country Status (2)

Country Link
CN (1) CN111566660A (fr)
WO (1) WO2020107251A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931644A (zh) * 2020-08-10 2020-11-13 济南博观智能科技有限公司 一种车载人数检测的方法、系统、设备及可读存储介质
CN112188702A (zh) * 2020-09-30 2021-01-05 中车青岛四方机车车辆股份有限公司 轨道车辆的照明设备的控制方法、控制装置和控制系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114407735A (zh) * 2022-02-17 2022-04-29 芜湖雄狮汽车科技有限公司 汽车座舱的控制方法、装置、车辆及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101770642A (zh) * 2008-12-26 2010-07-07 深圳先进技术研究院 一种车内人数计数方法和系统
US20140327752A1 (en) * 2013-05-01 2014-11-06 Nissan North America, Inc. Vehicle occupancy detection system
WO2015052896A1 (fr) * 2013-10-09 2015-04-16 日本電気株式会社 Dispositif et procédé de comptage de passagers et support d'enregistrement de programme
CN204547917U (zh) * 2015-04-15 2015-08-12 郭帅 一种轿车超员检测报警系统
US20170150252A1 (en) * 2014-12-08 2017-05-25 Harman International Industries, Inc. Adjusting speakers using facial recognition

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003046843A (ja) * 2001-08-02 2003-02-14 Omron Corp 撮像装置及び車両における人体検知システム
CN103280108B (zh) * 2013-05-20 2015-04-22 中国人民解放军国防科学技术大学 基于视觉感知和车联网的客车安全预警系统
CN108416254A (zh) * 2018-01-17 2018-08-17 上海鹰觉科技有限公司 一种用于人流行为识别和人数统计的统计系统以及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101770642A (zh) * 2008-12-26 2010-07-07 深圳先进技术研究院 一种车内人数计数方法和系统
US20140327752A1 (en) * 2013-05-01 2014-11-06 Nissan North America, Inc. Vehicle occupancy detection system
WO2015052896A1 (fr) * 2013-10-09 2015-04-16 日本電気株式会社 Dispositif et procédé de comptage de passagers et support d'enregistrement de programme
US20170150252A1 (en) * 2014-12-08 2017-05-25 Harman International Industries, Inc. Adjusting speakers using facial recognition
CN204547917U (zh) * 2015-04-15 2015-08-12 郭帅 一种轿车超员检测报警系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931644A (zh) * 2020-08-10 2020-11-13 济南博观智能科技有限公司 一种车载人数检测的方法、系统、设备及可读存储介质
CN112188702A (zh) * 2020-09-30 2021-01-05 中车青岛四方机车车辆股份有限公司 轨道车辆的照明设备的控制方法、控制装置和控制系统

Also Published As

Publication number Publication date
CN111566660A (zh) 2020-08-21

Similar Documents

Publication Publication Date Title
US10304165B2 (en) Vehicle stain and trash detection systems and methods
US10112585B1 (en) Vehicle cleanliness detection systems and methods
US10509974B2 (en) Stain and trash detection systems and methods
US20240166144A1 (en) Interior Camera System for a Self Driving Car
US11615545B2 (en) System and method for detecting in-vehicle conflicts
WO2020107251A1 (fr) Système et procédé de détection d'un effectif de passagers dans un véhicule
JP7032295B2 (ja) 車両制御システム、車両制御方法、およびプログラム
CN104776849A (zh) 车辆定位装置与方法
CN106164931B (zh) 用于在车辆显示装置上显示目标的方法和装置
CN111615721A (zh) 基于车辆与乘客之间进行识别的接送服务
US20190384277A1 (en) Remote driving managing apparatus and computer-readable storage medium
US11270525B2 (en) Automated vehicle occupancy detection
AU2018102204A4 (en) System and method for detecting in-vehicle conflicts
CN114475511B (zh) 基于视觉的气囊启动
US20230126957A1 (en) Systems and methods for determining fault for a vehicle accident
JPWO2019181096A1 (ja) 画像処理装置、画像処理方法及びプログラム
CN113537117B (zh) 车载遗留物监控告警方法及其5g系统
CN114898544A (zh) 一种基于网络的交通接驳方法及系统
CN110475694A (zh) 车辆控制装置、车辆控制方法及程序
CN113408320A (zh) 用于车辆避撞的方法、电子设备和计算机存储介质
WO2023097125A1 (fr) Systèmes et procédés d'étalonnage automatique de caméra
JP2023143183A (ja) 駐車位置調整装置および駐車位置調整方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18941798

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18941798

Country of ref document: EP

Kind code of ref document: A1