WO2023124384A1 - 确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品 - Google Patents

确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品 Download PDF

Info

Publication number
WO2023124384A1
WO2023124384A1 PCT/CN2022/124919 CN2022124919W WO2023124384A1 WO 2023124384 A1 WO2023124384 A1 WO 2023124384A1 CN 2022124919 W CN2022124919 W CN 2022124919W WO 2023124384 A1 WO2023124384 A1 WO 2023124384A1
Authority
WO
WIPO (PCT)
Prior art keywords
seat
detection frame
target
occupant
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/124919
Other languages
English (en)
French (fr)
Inventor
李阳阳
许亮
毛宁元
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Sensetime Intelligent Technology Co Ltd
Original Assignee
Shanghai Sensetime Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Sensetime Intelligent Technology Co Ltd filed Critical Shanghai Sensetime Intelligent Technology Co Ltd
Priority to JP2024539629A priority Critical patent/JP7792005B2/ja
Priority to KR1020247024146A priority patent/KR20240124371A/ko
Publication of WO2023124384A1 publication Critical patent/WO2023124384A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60N—SEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2/00—Seats specially adapted for vehicles; Arrangement or mounting of seats in vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/29—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/593—Recognising seat occupancy
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0881—Seat occupation; Driver or passenger presence
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera

Definitions

  • the embodiment of the present disclosure is based on the Chinese patent application with the application number 202111682157.3, the application date is December 31, 2021, and the application name is "Determining the seat of the occupants in the vehicle and the vehicle control method and device", and requires the Chinese patent application Priority, the entire content of the Chinese patent application is hereby incorporated by reference into this disclosure.
  • the present disclosure relates to the technical field of intelligent driving, and relates to a method, device, electronic device, medium and program product for determining the seat of an occupant in a vehicle and controlling the vehicle.
  • Embodiments of the present disclosure at least provide a method, device, electronic device, medium, and program product for determining the seat of an occupant in a vehicle and controlling the vehicle.
  • an embodiment of the present disclosure provides a method for determining a seat of an occupant in a vehicle, including:
  • each of the first seat areas corresponds to a seat
  • each of the second seating areas corresponds to a row of seats or a row of seats
  • the seat corresponding to the detection frame of the target occupant is determined according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant.
  • the area range of the first seat area is expanded, and based on the expanded area range of the second seat area, Then determine the seat corresponding to the detection frame of the target occupant, thereby reducing the probability of occupant and seat matching or binding failure, and improving the success rate of occupant seat matching.
  • the determining a plurality of first seat areas in the vehicle image includes:
  • a first seat area corresponding to each seat in the vehicle image is determined according to the seat distribution configuration information of the vehicle.
  • the seat distribution configuration information of different models may be different, and the first seat area can be determined more accurately through the seat distribution configuration information of the vehicle.
  • the detection frame for each occupant determines the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas, include:
  • the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to other detection frames, bind the seat corresponding to the first seat area to the detection frame, And determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area.
  • detecting whether other detection frames are bound to the first seat area can avoid binding multiple detection frames to one seat area, thereby avoiding When the vehicle is controlled, it affects the driving of the vehicle.
  • the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.
  • the target second seat area corresponding to the detection frame of the target occupant is determined based on the second matching degree between the detection frame of the target occupant and each of the second seat areas, include:
  • the second seating area is determined to be a target second seating area corresponding to the detection frame of the target occupant.
  • the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant are determined according to the location of the target occupant.
  • Seats corresponding to the detection frame including:
  • the target second seat area corresponding to the detection frame of the target occupant is determined based on the second matching degree between the detection frame of the target occupant and each of the second seat areas, Also includes:
  • the method also includes:
  • the target second seat area corresponding to the detection frame of the target occupant is not determined, determine the seat corresponding to the first seat area closest to the detection frame of the target occupant and not bound to other detection frames is the seat corresponding to the detection frame of the target occupant.
  • an embodiment of the present disclosure also provides a vehicle control method, including:
  • a target action made by the target user within the target detection frame is detected, and vehicle control is performed based on the target action and the seat area corresponding to the target detection frame.
  • an embodiment of the present disclosure also provides a device for determining the seat of an occupant in a vehicle, including:
  • an acquisition part configured to acquire a vehicle image within the vehicle
  • a first determining part configured to determine a plurality of first seat areas in the vehicle image, each first seat area corresponding to a seat;
  • a second determining section configured to detect occupants in the vehicle based on the vehicle image, and determine a detection frame for each of the occupants in the vehicle image
  • the third determining part is configured to, for each occupant's detection frame, determine the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas;
  • a fourth determining part configured to determine a plurality of second seating areas in the vehicle image, each second seating area corresponding to a row of seats or a row of seats;
  • the fifth determining part is configured to, for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the detection frame of the target occupant and each of the second seating areas The second matching degree, determining the target second seat area corresponding to the detection frame of the target occupant;
  • the sixth determining part is configured to determine the detection frame of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area, and the detection frame of the target occupant corresponding seat.
  • the first determination part is further configured to: determine the first seat area corresponding to each seat in the vehicle image according to the seat distribution configuration information of the vehicle.
  • the third determining part is further configured to: traverse each of the first seating areas in a preset order, and for each first seating area traversed to: calculate the detection The degree of overlap between the frame and the first seating area is used as the first matching degree between the detection frame and the first seating area; when the first matching degree reaches a first threshold and the first seating area If the corresponding seat is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame, and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area .
  • the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.
  • the fifth determining part is further configured to: for each detection frame of the target occupant, traverse the second seating area in a preset order, and for each second Seat area: calculating the degree of overlap between the frame of the target occupant and the second seat area as the second matching degree; when the second matching degree reaches a second threshold, determine the second matching degree
  • the second seat area is a target second seat area corresponding to the detection frame of the target occupant.
  • the sixth determining part is further configured to: determine the target occupant's location based on the detection frame of the target occupant and the position range of each seat in the target second seating area The seat corresponding to the position range where the center point of the detection frame falls is the candidate seat corresponding to the detection frame of the target occupant; if the candidate seat is not bound to other detection frames, determine that the candidate seat is A seat corresponding to the detection frame of the target occupant, and binding the seat corresponding to the candidate seat to the detection frame of the target occupant.
  • the fifth determining part is further configured to: for each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine the The second seat area with the largest overlapping area of the detection frames of the target occupant is the target second seat area.
  • the seventh determining part is further configured to: if the target second seat area corresponding to the detection frame of the target occupant is not determined, set the corresponding first seat area The seat closest to the detection frame of the target occupant and not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.
  • an embodiment of the present disclosure also provides a vehicle control device, including:
  • the eighth determination part is configured to determine the user detection frame corresponding to each seat area based on the method for determining the seat of the occupant in the vehicle described in the first aspect or any possible implementation manner of the first aspect;
  • the detection part is configured to detect the target action made by the target user within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.
  • an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the steps in any one of the above possible implementation manners are performed.
  • the embodiments of the present disclosure also provide a computer-readable storage medium, where any computer-readable storage medium stores a computer program, and any computer program executes any of the above-mentioned possibilities when run by a processor. steps in the implementation of .
  • an embodiment of the present disclosure further provides a computer program product, including computer readable code, when the computer readable code is run in an electronic device, the processor in the electronic device executes any one of the above-mentioned steps in a possible implementation.
  • Fig. 1 shows a flow chart of a method for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure
  • Fig. 2 shows a schematic diagram of the division method of the first seat area of a 4-seater vehicle provided by an embodiment of the present disclosure
  • Fig. 3 shows a flow chart of determining a seat corresponding to the detection frame provided by an embodiment of the present disclosure
  • Fig. 4 shows a schematic diagram of a method for calculating the degree of overlap between the detection frame and the first seat area provided by an embodiment of the present disclosure
  • Figure 5(a) shows a schematic diagram of the second seating area of a 4-seater vehicle divided by row provided by an embodiment of the present disclosure
  • Fig. 5(b) shows a schematic diagram of the second seat area of a 4-seater vehicle divided by column provided by an embodiment of the present disclosure
  • Fig. 6 shows a flow chart of determining a target second seat area corresponding to the detection frame of the target occupant provided by an embodiment of the present disclosure
  • Fig. 7(a) shows a schematic diagram of determining a target second seat area corresponding to the detection frame of the target occupant provided by an embodiment of the present disclosure
  • Fig. 7(b) shows another schematic diagram of determining the target second seat area corresponding to the detection frame of the target occupant provided by the embodiment of the present disclosure
  • Fig. 8 shows a flow chart of determining the seat corresponding to the detection frame of the target occupant based on the target second seat area provided by an embodiment of the present disclosure
  • Fig. 9 shows a schematic diagram of determining the seat corresponding to the detection frame of the target occupant based on the second target seat area provided by an embodiment of the present disclosure
  • Fig. 10 shows a detection method that distances the corresponding first seat area from the target occupant when the target second seat area corresponding to the detection frame of the target occupant is not determined according to an embodiment of the present disclosure.
  • Fig. 11 shows a flowchart of a vehicle control method provided by an embodiment of the present disclosure
  • Fig. 12 shows a schematic structural diagram of a device for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure
  • Fig. 13 shows a schematic structural diagram of a vehicle control device provided by an embodiment of the present disclosure
  • FIG. 14 shows a schematic structural diagram of a computer device 1400 provided by an embodiment of the present disclosure.
  • At least one (item) means one or more
  • “multiple” means two or more
  • at least two (items) means two or three And three or more
  • "and/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, "A and/or B” can mean: only A exists, only B exists, and A exists at the same time and B, where A and B can be singular or plural.
  • the character "/" can indicate that the contextual objects are an "or” relationship, which refers to any combination of these items, including any combination of single items (items) or plural items (items).
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.
  • an embodiment of the present disclosure provides a method for determining the seat of an occupant in a vehicle.
  • the area range of the first seat area is expanded, Based on the expanded second seat area, the seat corresponding to the detection frame of the target occupant is determined, thereby reducing the probability of occupant and seat matching or binding failure, and improving the success rate of occupant seat matching.
  • the method for determining the seat of an occupant in a vehicle provided by the embodiment of the present disclosure,
  • the execution subject of the vehicle control method is a vehicle control device (such as a rail transit cockpit intelligent analysis box, a bus cockpit intelligent analysis box), or a server.
  • FIG. 1 is a flow chart of a method for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure, the method includes steps S101 to S107, wherein:
  • Step S101 acquiring a vehicle image in the vehicle
  • Step S102 determining a plurality of first seat areas in the vehicle image, each of the first seat areas corresponds to a seat;
  • Step S103 Detect occupants in the vehicle based on the vehicle image, and determine a detection frame for each occupant in the vehicle image;
  • Step S104 for each occupant's detection frame, based on the first matching degree between the detection frame and each of the first seat areas, determine the seat corresponding to the detection frame;
  • Step S105 determining a plurality of second seat areas in the vehicle image, each of the second seat areas corresponds to a row of seats or a row of seats;
  • Step S106 for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching degree between the detection frame of the target occupant and each of the second seating areas , determining a target second seat area corresponding to the detection frame of the target occupant;
  • Step S107 Determine the seat corresponding to the detection frame of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant.
  • Step S101 to step S107 will be described in detail below.
  • step S101 may include: acquiring a vehicle image in the vehicle in response to satisfying an image acquisition trigger condition.
  • the satisfying the image acquisition trigger condition and acquiring the vehicle image of the vehicle can be implemented as any one of the following scenarios:
  • Scenario A The driver turns on the vehicle to start the vehicle.
  • the vehicle can be started by igniting or electrifying. After the vehicle is started, the vehicle circuit system is connected, the vehicle is connected to the network, and the vehicle image in the vehicle is started to be acquired.
  • Scenario B The driver opens the car door or triggers the car's start switch, wherein triggering the car's start switch can be done in at least one of the following ways, such as turning on the vehicle's remote control switch, inserting the vehicle's start key, or using a wearable device to touch the car body. Among them, after detecting that the driver opens the door or triggers the car start switch, the image of the vehicle inside the vehicle is acquired.
  • scene C when the vehicle is running, images in the vehicle are collected regularly according to a preset cycle, that is, images of the vehicle in the vehicle can be acquired at intervals.
  • the vehicle image is an image including the environment inside the vehicle.
  • the content of the vehicle image may include all seats in the vehicle cabin, people in the vehicle, and other pets.
  • the vehicle image may be acquired by using a driver monitoring system (driver monitoring system, DMS), and may be photographed by a DMS camera during the acquisition process.
  • DMS driver monitoring system
  • the detection frame and seat area can be determined based on steps S102 to S107.
  • the method of collecting and determining the occupant's seat at time intervals may be used. After one collection, after a preset time interval, the vehicle image is collected again according to the current vehicle condition. Adopting this method can reduce the collection pressure of the DMS system and reduce resource occupation. In actual use, the preset time period can be set as small as possible, so that the DMS system can update the vehicle image in a more timely manner according to the vehicle situation in the vehicle, so that while reducing resource occupation, it can also minimize collection The error between the vehicle image and the real-time vehicle situation in the vehicle.
  • the server in response to satisfying an image acquisition trigger condition, performs an operation of acquiring vehicle images in the vehicle.
  • the server after it is detected that the vehicle is connected to a preset system, the server corresponding to the preset system can detect whether the vehicle satisfies the image acquisition trigger condition.
  • the server may send a vehicle image acquisition request to the vehicle camera or the vehicle host connected to the vehicle camera, so as to acquire the vehicle image in the vehicle.
  • the vehicle accessing a preset system may refer to accessing a preset system through a vehicle-mounted system (such as a navigation system built into a vehicle); or it may be through a wearable device (such as smart watch, etc.), connected to the preset system.
  • a vehicle-mounted system such as a navigation system built into a vehicle
  • a wearable device such as smart watch, etc.
  • the server may not acquire the vehicle image in the vehicle.
  • the vehicle does not need to be connected to a preset system, and after the vehicle satisfies the image acquisition trigger condition, the vehicle host terminal directly acquires the vehicle image in the vehicle.
  • the DMS system on the vehicle side can directly monitor the vehicle in real time, collect vehicle images in the vehicle, and improve the safety of the vehicle.
  • the built-in DMS of the vehicle can The system detects the vehicle condition in the vehicle and updates the current vehicle image; after the vehicle returns to normal condition, the server can continue to obtain the vehicle image in the vehicle.
  • a plurality of first seat areas in the vehicle image may be determined.
  • each said first seat area is an independently seatable seat area inside the vehicle, when dividing the first seat area, each independently seatable seat inside the vehicle can be divided into a first seat area, a There are multiple first seating areas within the vehicle. Exemplarily, as shown in FIG. 2 , a way of dividing the first seating area of a 4-seater vehicle is shown in FIG. 2 .
  • this 4-seater vehicle comprises 4 seats respectively, and the upper left is the driver's seat 201, the upper right is the co-driver's seat 202, the lower left and the lower right correspond to two passenger seats (including 203 and 204) respectively, and each seat can be Travel alone with at least one passenger.
  • each seat can be divided into a corresponding first seating area, therefore, this 4-seater vehicle includes 4 first seating areas, namely the driver's seat area 201 and the passenger's seat area 202 , the passenger A seating area 203 , and the passenger B seating area 204 .
  • the first seating area corresponding to each seat in the vehicle image may be determined according to the seat distribution configuration information of the vehicle. seating area.
  • the seat distribution configuration information of the vehicle may exemplarily include the first seat range information of each seat inside the vehicle, the seat position arrangement and the number of seats of the vehicle, and the like.
  • the manner of determining the seat distribution configuration information of the vehicle may include: determining the seat distribution configuration information corresponding to the vehicle type information according to the vehicle type information. Wherein, the corresponding relationship between the vehicle type information and the seat distribution configuration information may be stored in advance, and the seat distribution configuration information corresponding to the vehicle type information may be searched based on the corresponding relationship after the seat distribution configuration information of the vehicle is determined.
  • the vehicle in the process of determining the vehicle type information of the vehicle, if the vehicle can implement a preset policy of automatically reporting the vehicle type information, the vehicle can automatically report the vehicle type information when the vehicle is connected to the preset system; If the vehicle model used by the occupant is too old to implement the preset automatic reporting strategy, the occupant can report the relevant vehicle type information by himself.
  • the first seat range information may include the corresponding image coordinate range information of the first seat area in the vehicle cabin of the vehicle type, and the image coordinate range information may be fixed installed in the vehicle for collecting the above-mentioned vehicle image. After that, it is obtained by dividing the seats of the collected in-vehicle images and calculating the boundary coordinates corresponding to each of the seat areas. Based on the corresponding coordinate range information of the first seating area, each of the first seating areas may be divided from the vehicle image.
  • the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.
  • the vehicle image may be input into a pre-trained detection model to determine a detection frame corresponding to each occupant in the vehicle image.
  • the detection model used to detect the face detection frame may be a first model
  • the detection model used to detect the human body detection frame may be a second model
  • the supervised data of the models may be different
  • the supervised data of the first model may be pre-marked human face frames
  • the supervised data of the second model may be pre-marked human body frames.
  • the detection model used to detect the face detection frame and the detection model used to detect the human body detection frame may be the same detection model, and the supervision data of the detection model includes two types, namely, pre- Annotated face frames and pre-annotated body frames.
  • step S102 and step S103 can be executed synchronously based on the order shown in FIG. 1 ; or step S102 and step S103 can be executed sequentially. This solution does not impose too many restrictions on the distinction between the above two steps.
  • the first matching degree between the above-mentioned detection frame and each of the first seating areas indicates the correlation of spatial positions, and may be based on the overlapping area, degree of overlap, etc. between the detection frame and each of the first seating areas inferred.
  • the detection frame of each occupant can be matched with each of the first seating areas.
  • the overlapping area between the detection frame and the first seating area can be calculated to determine the Among the plurality of seat areas within the detection frame, the seat area with the largest overlapping area with the detection frame is the seat corresponding to the detection frame, so as to determine the seat occupied by the occupant in the detection frame.
  • the above-mentioned step of determining the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas can be realized through steps S301 to S303 :
  • Step S301 traversing each of the first seating areas according to a preset order
  • the traversal may be performed from the driver's seat first from left to right, and then from top to bottom.
  • FIG. 2 there are 4 first seating areas in FIG.
  • the order of seat A 203 and passenger B seat 204 is traversed in sequence.
  • the traversal may be performed from top to bottom, and then from left to right.
  • the driver's seat 201, passenger A seat 203, co-pilot The order of the passenger seat 202 and the passenger B seat 204 is traversed in sequence.
  • step S302 For each first seating area traversed to, perform step S302 and step S303:
  • Step S302 taking the degree of overlap between the detection frame and the first seat area as the first matching degree between the detection frame and the first seat area;
  • the degree of overlap may be, for example, an intersection-over-union (IoU), that is, a ratio of an intersection and a union between the detection frame and the first seating area.
  • IOU intersection-over-union
  • the first seat area in FIG. 4 is the driver's seat area 401, wherein the middle gray highlighted part is the area 403 jointly contained by the driver's seat area 401 and the detection frame 402, that is, and set area; the synthesis of the driver's seat area and all areas of the detection frame is the intersection area, and the ratio of the intersection area and the union area is obtained to obtain the overlap between the driver's seat area and the detection frame, and this overlap is used as the The first matching degree between the detection frame and the driver's seat area.
  • IoU intersection-over-union
  • Step S303 when the first matching degree reaches the first threshold and the seat corresponding to the first seating area is not bound to other detection frames, bind the seat corresponding to the first seating area to the A frame is detected, and the seat corresponding to the detection frame is determined as the seat corresponding to the first seat area.
  • a first seat area is bound to a certain detection frame, indicating that the first seat area is the seating position of the occupant corresponding to the detection frame.
  • the first seat area cannot accommodate multiple people, so in the first matching
  • the degree reaches the first threshold, it is detected whether the first seat area is bound to other detection frames, which can avoid the unreasonable situation where multiple detection frames are bound to one seat area and multiple people sit on the same seat. Test results.
  • the first threshold of the face detection frame and the first threshold corresponding to the human body detection frame may be different. Since there may be differences in distances and relative angles between multiple different seating areas and the cameras that collect vehicle images, the face detection frame of each seating area can correspond to a first threshold, and the human body detection frame of each seating area can be respectively corresponds to a first threshold.
  • the first matching degree between a detection frame and the first seat area reaches the corresponding first threshold, and the seat corresponding to the first seat area If no other detection frame is bound, the seat corresponding to the first seat area is used as the seat corresponding to the detection frame, and the first seat area is bound to the first seat corresponding to its first matching degree. Threshold detection box.
  • the first matching degree calculated between the face detection frame of a certain occupant and the first seat area reaches the corresponding first threshold, but the human body detection frame of the occupant and the first seat area If the first matching degree of the area does not reach the corresponding first threshold, bind the first seat area with the face detection frame of the occupant, and determine that the seat corresponding to the face detection frame is the first seat The seat corresponding to the area, and then the face of the occupant is bound to the seat corresponding to the first seating area.
  • the human body detection frame of a certain occupant and the first matching degree calculated in the first seat area reach the corresponding first threshold, but the occupant's face detection frame and the first matching degree calculated in the first seating area If the corresponding first threshold is not reached, bind the first seat area to the human body detection frame of the occupant, determine that the seat corresponding to the human body detection frame is the seat corresponding to the first seat area, and then The occupant's human body is bound to the seat corresponding to the first seat area.
  • a double matching method can be adopted, and the seat area matching is performed on the human body detection frame and the face detection frame respectively.
  • the first matching degree of the area reaches the first threshold, bind the occupant's face detection frame and human body detection frame with the first seat area, and then bind the occupant with the first seat area , so as to improve the binding efficiency.
  • the seat area matching can be performed by only using the human body detection frame or the face detection frame, wherein, in the scheme of only using the face detection frame for detection, the face detection frame When the first matching degree with the first seat area reaches the corresponding first threshold, and the seat corresponding to the first seat area is not bound to other detection frames, bind the face detection frame to the first seat area Determine, determine that the seat corresponding to the face detection frame is the seat corresponding to the first seat area, and then bind the occupant's face with the first seat area; when the first matching degree does not exceed the corresponding matching threshold In the case of , the occupant's face is not bound to the first seat area.
  • the method of detection using the human body detection frame is similar to the method of detection using the face detection frame.
  • the area range of the second seat area is larger than that of the first seat area.
  • the second seat area is a seat area including a part of the vehicle gap, and each of the second seat areas Corresponds to a row of seats or a row of seats.
  • the second seating area may be determined based on the determined first seating area. Wherein, based on the determined first seating area and the vehicle image, some gaps in the first seating area can be supplemented, and each row or column can be determined as a second seating area; and then the second seating area can be divided according to the vehicle model information
  • a plurality of areas within, where each second seating area corresponds to a row of seats, each second seating area can be further divided exemplarily as follows: In the case of two seats in each row of the vehicle , can be divided at 1/2, divided into the left column area and the right column area; in the case of three seats in each row of the vehicle, it can be divided at 1/3, and so on.
  • the second seating area can be determined directly based on the vehicle image.
  • the gap interval of each row in the vehicle image can be directly removed, and each row in the vehicle is determined as a second seat area, and then the left column area and the second seat area in the second seat area can be divided.
  • the right column area similarly, in the scheme of dividing by column, the gap interval of each column in the vehicle image can be directly removed, and each column in the vehicle is determined as a second seat area, and then divided into the second seat area Areas listed above and areas below
  • Figure 5(a) what Figure 5(a) shows is a schematic diagram of the second seat area of a 4-seater vehicle divided by row, wherein, Figure 5(a) contains two The second seating area is respectively the second seating area A501 and the second seating area B502; each row includes a left column area and a right column area, and each left column area and each right column area correspond to a seat respectively.
  • Figure 5(b) what Figure 5(b) shows is a schematic diagram of the second seat area of a 4-seater vehicle divided by columns, wherein, in Figure 5(b) Contains two second seating areas, respectively the second seating area C503 and the second seating area D504; each row includes a front row area and a rear row area, and each front row area and each rear row area correspond to a seat .
  • the step of determining the target second seat area corresponding to the detection frame of the target occupant can be realized through steps S601 to S603, wherein:
  • Step S601 traversing the second seating areas in a preset order, for each of the second seating areas traversed;
  • the second seat area where the driver's seat is located may be selected according to the actual situation. Region starts and traverses row by row or column by column.
  • Step S602 taking the degree of overlap between the detection frame of the target occupant and the second seat area as the second matching degree between the detection frame of the target occupant and the second seat area;
  • Step S603 when the second matching degree reaches a second threshold, determine that the second seating area is a target second seating area corresponding to the detection frame of the target occupant.
  • the degree of overlap between the detection frame 701 of the target occupant and the second seat area E702, and the detection frame 701 of the target occupant and the second seat area F703 is calculated as the corresponding second matching degree, If the second matching degree corresponding to the detection frame of the target occupant and the second seat area E reaches a second threshold, the second seat area E is determined to be the target second seat area corresponding to the detection frame of the target occupant.
  • the overlapping area with the detection frame of the target occupant is the largest
  • the second seating area of is the target second seating area.
  • the overlapping area of the detection frame 701 of the target occupant and the second seat area G704 is larger than the overlapping area of the detection frame 701 of the target occupant and the second seat area H705, and the The second seat area G704 is determined as the target second seat area corresponding to the detection frame of the target occupant.
  • the target occupant is determined according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area, and the detection frame of the target occupant.
  • the steps of the seat corresponding to the detection frame can be as shown in Figure 8, including step S801 to step S802, wherein:
  • Step S801 based on the detection frame of the target occupant and the position range of each seat in the target second seat area, determine that the seat corresponding to the position range where the center point of the detection frame of the target occupant falls is the The candidate seat corresponding to the detection frame of the target occupant;
  • the center point between the detection frame of the target occupant and the position range of each seat may be calculated For example, as shown in Figure 9, the Euclidean distance between the detection frame 901 of the target occupant and the center point 902 between the left column area and the center point 903 between the detection frame 901 of the target occupant and the right column area are calculated. Euclidean distance, that is, the straight-line distance between the detection frame of the target occupant and the two center points.
  • the seat corresponding to the position range with the shortest straight line distance is determined as the candidate seat corresponding to the detection frame of the target occupant.
  • the center point of the detection frame 901 of the target occupant to be bound is shorter than the distance from the left column area to the right column area, and the seat corresponding to the left column area is determined as the target occupant.
  • the candidate seat corresponding to the detection box is determined as the candidate seat corresponding to the detection box.
  • intersection between the detection frame of the target occupant and each seat in the target second seating area may be calculated respectively, and the larger intersection is taken as the detection frame of the target occupant corresponding candidate seats.
  • Step S802 if the candidate seat is not bound to other detection frames, determine that the candidate seat is the seat corresponding to the detection frame of the target occupant, and bind the seat corresponding to the candidate seat to the The detection box of the target occupant.
  • the corresponding first seat area is closest to the detection frame of the target occupant and The seat not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.
  • the detection frame of the target occupant and the center point of each of the first seating areas may be respectively determined first;
  • the seats of other detection frames are determined as the seats corresponding to the detection frame of the target occupant.
  • Fig. 10 includes a plurality of second seating areas, namely the first seating area A1001, the first seating area B1002, the first seating area C1003 and the first seating area D1004, the target occupant's
  • the detection frame 1005 does not intersect with the above-mentioned four first seating areas. Since the center point of the detection frame of the target occupant has the shortest straight-line distance from the center point of the first seat area A, and no other detection frames are bound, the first seat area A1001 is determined as the detection frame of the target occupant The seat corresponding to 1005.
  • an embodiment of the disclosure also provides a vehicle control method, which is executed by an electronic device, as shown in FIG. 11 , which is a method provided by an embodiment of the disclosure.
  • Schematic diagram of the vehicle control method including the following steps:
  • Step S1101 based on the method for determining the seat of the occupant in the vehicle described in the above embodiment, determine the detection frame corresponding to each of the seat areas;
  • Step S1102 Detect the target action made by the target occupant within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.
  • the control of the vehicle can be realized, wherein, in response to the target instruction corresponding to the occupant's target action, the vehicle executes a corresponding target operation.
  • the vehicle control method can be applied in the field of intelligent driving, when it is detected that the target detection frame corresponding to the target occupant is bound to the target seat area.
  • the vehicle opens the window above the corresponding door according to the gesture instruction of the passenger and the position where the passenger is sitting. For example, if the passenger who issued the gesture instruction is a passenger in the passenger seat, the window above the door on the passenger side can be opened, thereby realizing targeted control based on passenger seat detection.
  • the embodiment of the present disclosure also provides a device for determining the seat of the occupant in the vehicle corresponding to the method for determining the seat of the occupant in the vehicle.
  • the above method for determining the seat of the occupant in the vehicle is similar, so the implementation of the device can refer to the implementation of the method.
  • FIG. 12 it is a schematic structural diagram of a device for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure.
  • the device includes: an acquisition part 1201, a first determination part 1202, a second determination part 1203, a third Determination part 1204, fourth determination part 1205, fifth determination part 1206, sixth determination part 1207; wherein,
  • An acquisition part 1201 configured to acquire a vehicle image in the vehicle
  • the first determining part 1202 is configured to determine a plurality of first seat areas in the vehicle image, and each first seat area corresponds to a seat;
  • the second determining part 1203 is configured to detect occupants in the vehicle based on the vehicle image, and determine a detection frame of each occupant in the vehicle image;
  • the third determining part 1204 is configured to, for each occupant's detection frame, determine the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas;
  • the fourth determination part 1205 is configured to determine a plurality of second seating areas in the vehicle image, each second seating area corresponds to a row of seats or a row of seats;
  • the fifth determining part 1206 is configured to, for the detection frame of the target occupant whose corresponding target first seat area is not determined based on the first matching degree, based on the detection frame of the target occupant and each of the second seat The second matching degree of the area, determining the target second seat area corresponding to the detection frame of the target occupant;
  • the sixth determining part 1207 is configured to determine the detection of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant box corresponding to the seat.
  • the first determining part 1202 is further configured to: determine the first seat area corresponding to each seat in the vehicle image according to the seat distribution configuration information of the vehicle.
  • the third determining part 1204 is further configured to: traverse each of the first seating areas in a preset order, and for each first seating area traversed to: calculate the The degree of overlap between the detection frame and the first seat area is used as a first matching degree between the detection frame and the first seat area; when the first matching degree reaches a first threshold and the first seat If the seat corresponding to the area is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame, and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area seat.
  • the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.
  • the fifth determining part 1206 is further configured to: for each target occupant's detection frame, traverse the second seat area in a preset order, and for each Second seat area: calculate the degree of overlap between the frame of the target occupant and the second seat area as the second matching degree; when the second matching degree reaches a second threshold, determine the The second seat area is a target second seat area corresponding to the detection frame of the target occupant.
  • the sixth determining part 1207 is further configured to: determine the target occupant based on the detection frame of the target occupant and the position range of each seat in the target second seating area.
  • the seat corresponding to the position range where the center point of the detection frame falls into is the candidate seat corresponding to the detection frame of the target occupant; if the candidate seat is not bound to other detection frames, determine the candidate seat is the seat corresponding to the detection frame of the target occupant, and the seat corresponding to the candidate seat is bound to the detection frame of the target occupant.
  • the fifth determining part 1206 is further configured to: for each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine The second seat area with the largest overlapping area of the detection frames of the target occupant is the target second seat area.
  • the seventh determination part 1208 is further configured to: if the target second seat area corresponding to the detection frame of the target occupant is not determined, set the corresponding first seat area The seat whose area is closest to the detection frame of the target occupant and which is not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.
  • the embodiment of the present disclosure also provides a vehicle control device corresponding to the vehicle control method, because the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned method of determining the seat of the occupant in the vehicle in the embodiment of the present disclosure , so the implementation of the device can refer to the implementation of the method.
  • FIG. 13 is a schematic structural diagram of a vehicle control device provided by an embodiment of the present disclosure
  • the device includes: an eighth determination part 1301 and a detection part 1302; wherein,
  • the eighth determining part 1301 is configured to determine the detection frame corresponding to each seat area based on the method for determining the seat of the occupant in the vehicle described in the above embodiment;
  • the detection part 1302 is configured to detect a target action made by a target occupant within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.
  • a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course it may also be a unit, a module or a non-modular one.
  • FIG. 14 it is a schematic structural diagram of a computer device 1400 provided by an embodiment of the present disclosure, including a processor 1401 , a memory 1402 , and a bus 1403 .
  • the memory 1402 is used to store execution instructions, including the memory 14021 and the external memory 14022; the memory 14021 here is also called the internal memory, and is used to temporarily store the operation data in the processor 1401 and the data exchanged with the external memory 14022 such as the hard disk,
  • the processor 1401 exchanges data with the external memory 14022 through the memory 14021.
  • the processor 1401 communicates with the memory 1402 through the bus 1403, so that the processor 1401 executes the following instructions:
  • each first seat area corresponding to a seat
  • each second seating area corresponding to a row of seats or a row of seats
  • the seat corresponding to the detection frame of the target occupant is determined according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant.
  • processor 1401 may also execute the following instructions:
  • a target action made by a target occupant within the target detection frame is detected, and vehicle control is performed based on the target action and the seat area corresponding to the target detection frame.
  • An embodiment of the present disclosure also provides a computer-readable storage medium, any computer-readable storage medium stores a computer program, and any computer program executes the determination described in the above-mentioned method embodiments when run by a processor A method of seating an occupant in a vehicle, steps of a vehicle control method.
  • any one of the storage media may be a volatile or non-volatile computer-readable storage medium.
  • the above-mentioned processor can be an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic At least one of Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor. Understandably, the electronic device that implements the above processor function may also be other, which is not specifically limited in this embodiment of the present disclosure.
  • the above-mentioned computer storage medium/memory can be read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Magnetic Random Access Memory (Ferromagnetic Random Access Memory, FRAM), Flash Memory (Flash Memory), Magnetic Surface Memory, CD-ROM, or CD-ROM (Compact Disc Read-Only Memory, CD-ROM) and other memories; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. .
  • An embodiment of the present disclosure also provides a computer program product, any of the computer products carries a program code, and the instructions included in the program code can be used to execute the method for determining the seat of an occupant in a vehicle described in the method embodiment above, For the steps of the vehicle control method, reference may be made to the foregoing method embodiments.
  • the above-mentioned computer program product may be implemented by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • a storage medium includes several instructions for enabling a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .
  • An embodiment of the present disclosure provides a method and device for determining the seat of an occupant in a vehicle and controlling the vehicle, including: acquiring a vehicle image in the vehicle; determining a plurality of first seat areas in the vehicle image; detecting an occupant in the vehicle based on the vehicle image, Determining the detection frame of each occupant in the vehicle image; for the detection frame of each occupant, based on the first matching degree between the detection frame and each of the first seat areas, determining the seat corresponding to the detection frame; determining the vehicle image a plurality of second seating areas; for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching between the detection frame of the target occupant and each of the second seating areas Determine the target second seat area corresponding to the detection frame of the target occupant; determine the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant
  • the area range of the first seat area is expanded, and based on the expanded area range of the second seat area, the seat corresponding to the detection frame of the target occupant is determined , thereby reducing the probability of occupant and seat matching or binding failure, and improving the success rate of occupant seat matching.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Transportation (AREA)
  • Human Computer Interaction (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Seats For Vehicles (AREA)
  • Air Bags (AREA)
  • Image Analysis (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)

Abstract

一种确定车辆内乘员的座位及车辆控制方法及装置，方法包括：获取车辆内的车辆图像；确定车辆图像中多个第一座位区域；基于车辆图像检测车辆内的乘员，确定车辆图像中每一乘员的检测框；针对每一乘员的检测框，基于检测框与每一第一座位区域的第一匹配度，确定与检测框对应的座位；确定车辆图像中多个第二座位区域；针对基于第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于目标乘员的检测框与每一第二座位区域的第二匹配度，确定与目标乘员的检测框对应的目标第二座位区域；根据与目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及目标乘员的检测框确定目标乘员的检测框对应的座位。

Description

确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品
相关申请的交叉引用
本公开实施例基于申请号为202111682157.3、申请日为2021年12月31日、申请名称为“确定车辆内乘员的座位及车辆控制方法及装置”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开涉及智能驾驶技术领域，涉及一种确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品。
背景技术
随着汽车产业的高速发展，除汽车必备的发动机等重要零部件的迭代更新外，许多新兴的与汽车产业相关的技术也在随着汽车产业的高速发展而诞生，其中就包括车辆智能驾驶技术。在车辆智能驾驶技术中，为了能够实现让车辆根据人的动作做出与之相应的反馈的效果，就需要预先将人与车辆座位进行绑定，进而再根据人的不同的动作由车辆内的智能系统做出相应的判断。
相关技术中，当人与车辆座位进行绑定时，经常会出现因为坐姿或身高等客观因素导致人与车辆座位出现绑定错误的情况，因此，对于如何让人与车辆座位正确的进行绑定成为亟待解决的问题。
发明内容
本公开实施例至少提供一种确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品。
一方面，本公开实施例提供了一种确定车辆内乘员的座位的方法，包括：
获取车辆内的车辆图像；
确定所述车辆图像中多个第一座位区域，每一所述第一座位区域对应一个座位；
基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；
确定所述车辆图像中多个第二座位区域，每一所述第二座位区域对应一排座位或一列座位；
针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位。
本公开实施例所提供的方案中，在通过第一座位区域无法确定座位的目标乘员的检测框的情况下，扩大了第一座位区域的区域范围，并基于扩大区域范围的第二座位区域，再确定目标乘员的检测框对应的座位，由此可以降低乘员与座位匹配或绑定失败的概率，提升乘员座位匹配的成功率。
一种可能的实施方式中，所述确定所述车辆图像中多个第一座位区域，包括：
根据所述车辆的座位分布配置信息，确定所述车辆图像中每一座位对应的第一座位区域。
不同车型的座位分布配置信息可能不同，通过车辆的座位分布配置信息，可以更加精 准的确定出第一座位区域。
一种可能的实施方式中，所述针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位，包括：
按照预设顺序遍历每一所述第一座位区域，针对遍历至的每一所述第一座位区域：
计算所述检测框与所述第一座位区域的交叠度，作为所述检测框与所述第一座位区域的第一匹配度；
在所述第一匹配度达到第一阈值、且所述第一座位区域对应的座位未绑定其他检测框的情况下，将所述第一座位区域对应的座位绑定至所述检测框，并确定所述检测框对应的座位为所述第一座位区域对应的座位。
在所述第一匹配度达到第一阈值时，去检测所述第一座椅区域是否绑定有其他检测框，可以避免一个座椅区域绑定多个检测框，进而避免在基于绑定结果进行车辆控制时，影响车辆的行驶。
一种可能的实施方式中，所述车辆图像中每一所述乘员的检测框为以下至少之一：人脸检测框、人体检测框。
一种可能的实施方式中，所述基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域，包括：
针对每一目标乘员的检测框，按照预设顺序遍历所述第二座位区域，针对遍历至的每一第二座位区域：
计算所述目标乘员的检测框与所述第二座位区域的交叠度，作为所述第二匹配度；
在所述第二匹配度达到第二阈值的情况下，确定所述第二座位区域为与目标乘员的检测框对应的目标第二座位区域。
通过这种方法，可以先检测出目标成员是哪一排成员，或哪一列成员，然后再去进行进一步的座位绑定，由此可以提高绑定结果的精度。
一种可能的实施方式中，所述根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位，包括：
基于所述目标乘员的检测框与所述目标第二座位区域每一座位的位置范围，确定所述目标乘员的检测框的中心点落入的所述位置范围对应的座位为所述目标乘员的检测框对应的候选座位；
在所述候选座位未绑定至其他检测框的情况下，确定所述候选座位为与所述目标乘员的检测框对应的座位，并将所述候选座位对应的座位绑定至所述目标乘员的检测框。
一种可能的实施方式中，所述基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域，还包括：
针对遍历至的每一第二座位区域，在所述交叠度未达到第二阈值的情况下，确定与所述目标乘员的检测框的交叠面积最大的第二座位区域为目标第二座位区域。
一种可能的实施方式中，所述方法还包括：
在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
另一方面，本公开实施例还提供一种车辆控制方法，包括：
基于第一方面或第一方面任一种可能的实施方式所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的用户检测框；
检测目标检测框内的目标用户做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
基于上述实施例中所述的确定车辆内乘员的座位的方法，当用于车辆控制时，可以响 应于用户发出的动作指令，根据用户的动作和其座位进行针对性的反馈，提升了用户在使用车辆时的交互感受。
再一方面，本公开实施例还提供一种确定车辆内乘员的座位的装置，包括：
获取部分，被配置为获取车辆内的车辆图像；
第一确定部分，被配置为确定所述车辆图像中多个第一座位区域，每一第一座位区域对应一个座位；
第二确定部分，被配置为基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
第三确定部分，被配置为针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；
第四确定部分，被配置为确定所述车辆图像中多个第二座位区域，每一第二座位区域对应一排座位或一列座位；
第五确定部分，被配置为针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
第六确定部分，被配置为根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位。
在一种可能的实施方式中，所述第一确定部分还被配置为：根据所述车辆的座位分布配置信息，确定所述车辆图像中每一座位对应的第一座位区域。
在一种可能的实施方式中，所述第三确定部分，还被配置为：按照预设顺序遍历每一所述第一座位区域，针对遍历至的每一第一座位区域：计算所述检测框与所述第一座位区域的交叠度，作为所述检测框与所述第一座位区域的第一匹配度；在所述第一匹配度达到第一阈值、且所述第一座位区域对应的座位未绑定其他检测框的情况下，将所述第一座位区域对应的座位绑定至所述检测框，并确定所述检测框对应的座位为所述第一座位区域对应的座位。
在一种可能的实施方式中，所述车辆图像中每一所述乘员的检测框为以下至少之一：人脸检测框、人体检测框。
在一种可能的实施方式中，所述第五确定部分，还被配置为：针对每一目标乘员的检测框，按照预设顺序遍历所述第二座位区域，针对遍历至的每一第二座位区域：计算所述目标乘员的测框与所述第二座位区域的交叠度，作为所述第二匹配度；在所述第二匹配度达到第二阈值的情况下，确定所述第二座位区域为与目标乘员的检测框对应的目标第二座位区域。
在一种可能的实施方式中，所述第六确定部分，还被配置为：基于所述目标乘员的检测框与所述目标第二座位区域每一座位的位置范围，确定所述目标乘员的检测框的中心点落入的所述位置范围对应的座位为所述目标乘员的检测框对应的候选座位；在所述候选座位未绑定至其他检测框的情况下，确定所述候选座位为与所述目标乘员的检测框对应的座位，并将所述候选座位对应的座位绑定至所述目标乘员的检测框。
在一种可能的实施方式中，所述第五确定部分，还被配置为：针对遍历至的每一第二座位区域，在所述交叠度未达到第二阈值的情况下，确定与所述目标乘员的检测框的交叠面积最大的第二座位区域为目标第二座位区域。
在一种可能的实施方式中，所述第七确定部分，还被配置为：在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
又一方面，本公开实施例还提供一种车辆控制装置，包括：
第八确定部分，被配置为基于第一方面或第一方面任一种可能的实施方式所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的用户检测框；
检测部分，被配置为检测目标检测框内的目标用户做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
又一方面，本公开实施例还提供一种计算机设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当计算机设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行上述任一种可能的实施方式中的步骤。
又一方面，本公开实施例还提供一种计算机可读存储介质，所述任一计算机可读存储介质上存储有计算机程序，所述任一计算机程序被处理器运行时执行上述任一种可能的实施方式中的步骤。
又一方面，本公开实施例还提供一种计算机程序产品，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行时执行上述任一种可能的实施方式中的步骤。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本公开实施例所提供的一种确定车辆内乘员的座位的方法的流程图；
图2示出了本公开实施例所提供的一种4人座车辆的第一座位区域的划分方式示意图；
图3示出了本公开实施例所提供的一种确定与所述检测框对应的座位的流程图；
图4示出了本公开实施例所提供的一种检测框与第一座位区域之间交叠度计算方式的示意图；
图5(a)示出了本公开实施例所提供的一种按排划分的4人座车辆的第二座位区域的示意图；
图5(b)示出了本公开实施例所提供的一种按列划分的4人座车辆的第二座位区域的示意图；
图6示出了本公开实施例所提供的一种确定与所述目标乘员的检测框对应的目标第二座位区域的流程图；
图7(a)示出了本公开实施例所提供的一种确定与所述目标乘员的检测框对应的目标第二座位区域的示意图；
图7(b)示出了本公开实施例所提供的另一种确定与所述目标乘员的检测框对应的目标第二座位区域的示意图；
图8示出了本公开实施例所提供的一种基于目标第二座位区域确定所述目标乘员的检测框对应的座位的流程图；
图9示出了本公开实施例所提供的一种基于目标第二座位区域确定所述目标乘员的检测框对应的座位的示意图；
图10示出了本公开实施例所提供的一种在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位的示意图；
图11示出了本公开实施例所提供的一种车辆控制方法的流程图；
图12示出了本公开实施例所提供的一种确定车辆内乘员的座位的装置的架构示意图；
图13示出了本公开实施例所提供的一种车辆控制装置的架构示意图；
图14示出了本公开实施例所提供的一种计算机设备1400的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本公开方案，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可能还包括没有列出的步骤或单元，或可能还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上，“至少两个(项)”是指两个或三个及三个以上，“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”可表示前后关联对象是一种“或”的关系，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。字符“/”还可表示数学运算中的除号，例如，a/b＝a除以b；6/3＝2。“以下至少一项(个)”或其类似表达。
经研究发现，在现有的已公开的关于人与车辆座位绑定的技术中，在人与车辆座位进行绑定的过程中，经常会出现因为人的坐姿或身高等客观因素造成机器无法识别的情况，例如儿童身形较小、身高不够且易动，导致机器监测不到。
并且，在现有的已公开的关于人与车辆座位绑定的技术中，人与车辆座位进行绑定的过程中一般仅采取一种绑定方式，并且只进行一次绑定，无相关兜底策略，容易造成执行设备误判的情况，从而导致人与车辆座位出现绑定错误的情况。
基于上述研究，本公开实施例提供了一种确定车辆内乘员的座位的方法，在通过第一座位区域无法确定座位的目标乘员的检测框的情况下，扩大了第一座位区域的区域范围，并基于扩大区域范围的第二座位区域，再确定目标乘员的检测框对应的座位，由此可以降低乘员与座位匹配或绑定失败的概率，提升乘员座位匹配的成功率。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种确定车辆内乘员的座位的方法、车辆控制方法进行详细介绍，本公开实施例所提供的确定车辆内乘员的座位的方法、车辆控制方法的执行主体车辆控制设备(如轨交驾驶舱智能分析盒、公交驾驶舱智能分析盒)，或服务器。
参见图1所示，为本公开实施例提供的确定车辆内乘员的座位的方法的流程图，所述方法包括步骤S101至步骤S107，其中：
步骤S101、获取车辆内的车辆图像；
步骤S102、确定所述车辆图像中多个第一座位区域，每一所述第一座位区域对应一个座位；
步骤S103、基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
步骤S104、针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；
步骤S105、确定所述车辆图像中多个第二座位区域，每一所述第二座位区域对应一排座位或一列座位；
步骤S106、针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
步骤S107、根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位。
下面将对步骤S101至步骤S107进行详细介绍。
针对步骤S101、
在一种可能的实施方式中，步骤S101可以包括：响应于满足图像采集触发条件，获取车辆内的车辆图像。
其中，所述满足图像采集触发条件，获取车辆的车辆图像，可以实现为以下所描述的多种场景中任意一种：
场景A、驾驶员打开车辆使车辆启动。其中，车辆可以通过打火或者通电的方式进行启动，在车辆启动后，车辆电路系统接通，车辆接入网络，开始获取车辆内的车辆图像。
场景B、驾驶员打开车门或者触发汽车启动开关，其中，触发汽车启动开关可以通过下列方式的至少之一进行，打开车辆远程遥控开关、插入车辆启动钥匙或者使用可穿戴设备接触车体等。其中，在检测到驾驶员打开车门或者触发汽车启动开关后，获取车辆内的车辆图像。
场景C、车辆行驶中，按照预设的周期定期采集车辆内的图像，即可以每隔一段时间获取一次车辆内的车辆图像。
在一种可能的实施方式中，所述车辆图像为包含车内环境的图像，示例性的，车辆图像的内容可以包括车舱内所有的座位、车辆内人以及其他的宠物等。
在一种可能的实施方式中，所述车辆图像可以使用驾驶员监控系统(driver monitoring system，DMS)进行采集获取，在采集过程中可以使用DMS摄像头进行拍摄。在使用DMS采集车辆图像的过程中，因为车辆内的情况实时都在动态的变化，所以在车辆行驶过程中需要实时采集车辆图像。
实际应用中，由于在车辆驾驶过程中，车辆内的乘员所坐的位置可能会发生变化，对于采集的每一帧车辆图像，都可以基于步骤S102至步骤S107来确定检测框与座位区域。
在一种可能的实施方式中，可以使用时间段间隔采集和确定乘员座位的方式，在进行一次采集后，经过预设时间间隔，针对当前的车辆情况再次进行车辆图像的采集。采用此方式可以降低DMS系统的采集压力，减少资源占用。在实际使用中，可以将预设的时间段设置的尽可能小，使DMS系统更加及时的针对于车辆内的车辆情况进行车辆图像更新，做到在减少资源占用的同时，也可以尽量降低采集的车辆图像与实时车辆内车辆情况的误差。
在一种可能的实施方式中，响应于满足图像采集触发条件，服务器执行获取车辆内的车辆图像的操作。其中，可以在检测到车辆接入预设的系统后，所述预设的系统对应的服务器可以检测所述车辆是否满足图像采集触发条件。在检测到车辆满足图像采集触发条件的情况下，可以由服务器向车载摄像头或与车载摄像头连接的车载主机发出车辆图像获取请求，从而获取车辆内的车辆图像。
其中，所述车辆接入预设的系统，示例性的，可以是指通过车载系统(例如车辆自带导航系统)接入预设的系统；或者可以通过驾驶人自带的可穿戴设备(例如智能手表等)，接入到预设的系统中。
在车辆只是接入预设的系统但是并未满足图像采集触发条件的情况下，为了减少资源占用，服务器可以不获取车辆内的车辆图像。
在另一种可能的实施方式中，车辆无需接入至预设的系统，在车辆满足图像采集触发条件后，由车载主机端直接获取车辆内的车辆图像。采用此种方式，可以由车辆端的DMS系统直接对于车辆进行实时监控，采集车辆内的车辆图像，提升车辆使用的安全性。
以上两种实施方式可以在特定的场景下结合使用，其中，在车辆因为极端情况无法接入服务器的情况下，例如，汽车泡水熄火、车辆行驶至无信号路段等，可以由车辆内置的DMS系统对于车辆的车辆内的车辆情况进行检测，更新当前的车辆图像；在车辆恢复至正常情况至后，可以继续由服务器获取车辆内的车辆图像。
针对步骤S102、
在获取到车辆内的车辆图像后，可以确定所述车辆图像中多个第一座位区域。
其中，每一所述第一座位区域为车辆内部的一个独立可乘坐的座位区域，在划分第一座位区域时，可以将车辆内部每一独立可乘坐的座位划分为一个第一座位区域，一个车辆内有多个第一座位区域。示例性的，如图2，在图2中展示了一种4人座车辆的第一座位区域的划分方式。
其中，此4人座车辆分别包含4个座位，左上为驾驶人座位201、右上为副驾驶人座位202、左下与右下分别对应两个乘员座位(包括203和204)，每一座位均可单独乘坐至少一名乘坐人。在划分第一座位区域时，将每一座位划可以对应一个第一座位区域，因此，此4人座车辆包含4个第一座位区域，分别是驾驶人座位区域201、副驾驶人座位区域202、乘员A座位区域203、乘员B座位区域204。
在一种可能的实施方式中，在确定所述车辆图像中多个第一座位区域的过程中，可以根据所述车辆的座位分布配置信息，确定所述车辆图像中每一座位对应的第一座位区域。
所述车辆的座位分布配置信息示例性的可以包括车辆内部的各个座位的第一座位范围信息、车辆的座位位置排布以及座位数量等。
在一种可能的实施方式中，确定所述车辆的座位分布配置信息的方式，可以包括：根据所述车辆的车型信息，确定与所述车型信息对应的座位分布配置信息。其中，可以预先存储车型信息与座位分布配置信息的对应关系，在确定所述车辆的座位分布配置信息，可以基于所述对应关系，查找与所述车型信息对应的座位分布配置信息。
其中，在确定所述车辆的车型信息的过程中，在车辆可执行预设的自动上报车型信息的策略的情况下，可以在车辆接入预设的系统时，由车辆自动上报车型信息；在乘员使用的车型过老无法执行预设的自动上报的策略的情况下，可以由乘员自行上报相关的车型信息。
其中，所述第一座位范围信息可以包括该车型的车舱内的第一座位区域的对应的图像坐标范围信息，该图像坐标范围信息可以是在用于采集上述车辆图像的摄像头固定安装在车内之后，通过对采集的车内图像的座位进行划分并计算相应每一所述座位区域的边界坐标获得的。基于所述第一座位区域的对应的坐标范围信息，可以从所述车辆图像中划分出每一所述第一座位区域。
针对步骤S103、
所述车辆图像中每一所述乘员的检测框为以下至少之一：人脸检测框、人体检测框，在一种可能的实施方式中，在确定所述车辆图像中每一所述乘员的检测框的过程中，可以将所述车辆图像输入至预先训练的检测模型中，确定所述车辆图像中每一所述乘员分别对应的检测框。
在一种可能的实施方式中，用于检测人脸检测框的检测模型可以是第一模型，用于检测人体检测框的检测模型可以是第二模型，所述第一模型和所述第二模型的监督数据可以不同，所述第一模型的监督数据可以是预先标注的人脸框，所述第二模型的监督数据可以 是预先标注的人体框。
或者，在另外一种可能的实施方式中，用于检测人脸检测框的检测模型和用于检测人体检测框的检测模型可以是同一检测模型，该检测模型的监督数据包括两种，即预先标注的人脸框和预先标注的人体框。
在实际应用中，步骤S102、与步骤S103、可以基于图1顺序同步执行；或者先后执行步骤S102、与步骤S103，关于以上两个步骤的先后区分，本方案不作过多的限制。
针对步骤S104、
上述检测框与每一所述第一座位区域之间的第一匹配度表示空间位置的相关性，可以基于检测框与每一所述第一座位区域之间的交叠面积、交叠度等得出。
在一种可能的实现方式中，可以针对每一乘员的检测框与每一所述第一座位区域进行匹配，示例性的，可以计算检测框与第一座位区域的交叠面积，确定车舱内的多个座位区域中与检测框交叠面积最大的座位区域为该检测框对应的座位，从而确定出该检测框内的乘员所乘坐的座位。
在一种可能的实施方式中，上述基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位的步骤，可通过步骤S301至步骤S303实现：
步骤S301、按照预设顺序遍历每一所述第一座位区域；
在一种可能的实施方式中，在按照预设顺序遍历每一所述第一座位区域的过程中，可以从驾驶人座位开始进行先从左至右，再从上至下的方式进行遍历。示例性的，如图2所示，图2中含有4个第一座位区域，在遍历每一所述第一座位区域的过程中，可以依次按照驾驶人座位201、副驾驶人座位202、乘员A座位203、乘员B座位204的顺序依次进行遍历。
在另一种可能的实施方式中，可以从驾驶人座位开始进行先从上至下，再从左至右的方式进行遍历。示例性的，如图2所示，图2中含有4个第一座位区域，在遍历每一所述第一座位区域的过程中，可以依次按照驾驶人座位201、乘员A座位203、副驾驶人座位202、乘员B座位204的顺序依次进行遍历。
针对遍历至的每一第一座位区域，执行步骤S302和步骤S303：
步骤S302、将所述检测框与所述第一座位区域的交叠度，作为所述检测框与所述第一座位区域的第一匹配度；
所述交叠度例如可以是交并比(Intersection-over-Union，IoU)，即，检测框与第一座位区域之间交集与并集的比值。示例性的，如图4所示，图4中所述第一座位区域为驾驶人座位区域401，其中，中间灰色突出部分为驾驶人座位区域401与检测框402共同包含的区域403，即并集区域；驾驶人座位区域与检测框所有区域的综合为交集区域，将交集区域与并集区域做出比值，即得到驾驶人座位区域与检测框的交叠度，将此交叠度作为所述检测框与所述驾驶人座位区域的第一匹配度。
步骤S303、在所述第一匹配度达到第一阈值、且所述第一座位区域对应的座位未绑定其他检测框的情况下，将所述第一座位区域对应的座位绑定至所述检测框，并确定所述检测框对应的座位为所述第一座位区域对应的座位。
在按照预设顺序遍历第一座位区域的过程中，可以检测第一座位区域是否绑定其他检测框。一个第一座位区域绑定至某一个检测框，表明该第一座位区域为该检测框对应的乘员的乘坐位置，通常情况下该第一座位区域不能乘坐多人，故而在所述第一匹配度达到第一阈值的情况下，去检测所述第一座椅区域是否与其他检测框绑定，可以避免一个座椅区域绑定多个检测框，出现多个人乘坐于同一座位的不合理的检测结果。
在一种可能的实施方式中，所述第一座位区域中，人脸检测框的第一阈值与人体检测框各自对应的第一阈值可以不同。由于多个不同座位区域与采集车辆图像的摄像头之间的距离和相对角度可能存在差异，每一座位区域的人脸检测框可以分别对应一个第一阈值， 每一座位区域的人体检测框可以分别对应一个第一阈值。
在一种可能的实施方式中，针对当前遍历至的第一座位区域，在一个检测框与该第一座位区域的第一匹配度达到对应的第一阈值，且该第一座位区域对应的座位未绑定其他检测框的情况下，将该所述第一座位区域对应的座位作为所述检测框对应的座位，并将该第一座位区域绑定至与其第一匹配度达到对应的第一阈值的检测框。
在一种可能的实施方式中，在某乘员的人脸检测框与所述第一座位区域计算的第一匹配度达到对应的第一阈值，但是该乘员的人体检测框与所述第一座位区域的第一匹配度未达到对应的第一阈值的情况下，将第一座位区域与该乘员的人脸检测框进行绑定，确定所述人脸检测框对应的座位为所述第一座位区域对应的座位，进而将该乘员的人脸与第一座位区域对应的座位进行绑定。
反之，在某乘员的人体检测框与所述第一座位区域计算的第一匹配度达到对应的第一阈值，但是该乘员的人脸检测框与所述第一座位区域计算的第一匹配度未达到对应的第一阈值的情况下，将第一座位区域与该乘员的人体检测框进行绑定，确定所述人体检测框对应的座位为所述第一座位区域对应的座位，进而将该乘员的人体与第一座位区域对应的座位进行绑定。
在实际应用中，可以采取双重匹配的方式，先后就人体检测框与人脸检测框分别进行座位区域匹配，在一个乘员的人体检测框和人脸检测框中的任一检测框与第一座位区域的第一匹配度达到第一阈值的情况下，将该乘员的人脸检测框和人体检测框都与所述第一座位区域进行绑定，进而将该乘员与第一座位区域进行绑定，以此来提升绑定的效率。
在另一种可能的实施方式中，可以采取单独只使用人体检测框或者人脸检测框的方式进行座位区域匹配，其中，在仅使用人脸检测框进行检测的方案中，在人脸检测框与第一座位区域的第一匹配度达到对应的第一阈值，且所述第一座位区域对应的座位未绑定其他检测框的情况下，将该人脸检测框与第一座位区域进行绑定，确定所述人脸检测框对应的座位为所述第一座位区域对应的座位，进而将该乘员的人脸与第一座位区域进行绑定；在第一匹配度不超过对应的匹配阈值的情况下，不将该乘员的人脸与第一座位区域进行绑定。使用人体检测框进行检测的方法与使用人脸检测框进行检测的方法类似。
针对步骤S105、
所述第二座位区域的区域范围大于所述第一座位区域，在一种可能的实施方式中，所述第二座位区域为包含部分车辆空隙间隔的座位区域，每一所述第二座位区域对应一排座位或一列座位。
在一种可能的实施方式中，可以基于已经确定的第一座位区域，确定第二座位区域。其中，可以基于已经确定的第一座位区域与车辆图像，补充第一座位区域的部分间隔空隙，将每一排或一列确定为一个第二座位区域；再针对车辆的车型信息划分第二座位区域内的多个区域，其中，在每一第二座位区域对应一排座位，示例性地可以按照如下方式对每一第二座位区域进行进一步划分：在车辆每一排有两个座位的情况下，可以在1/2处进行划分，划分为左列区域和右列区域；在车辆每一排有三个座位的情况下，可以在1/3处进行划分，以此类推。
在另一种可能的实施方式中，可以基于车辆图像直接确定第二座位区域。其中，按照排进行划分的方案中，可以直接去掉车辆图像内的每排的空隙间隔，将车辆内的每一排确定为一个第二座位区域，再划分第二座位区域内的左列区域和右列区域；相似的，按照列进行划分的方案中，可以直接去掉车辆图像内的每列的空隙间隔，将车辆内的每一列确定为一个第二座位区域，再划分第二座位区域内的上列区域和下列区域
示例性的，如图5(a)所示，图5(a)示出的是一种按排划分的4人座车辆的第二座位区域的示意图，其中，图5(a)中包含两个第二座位区域，分别为第二座位区域A501与第二座位区域B502；每一排包括左列区域和右列区域，每一左列区域和每一右列区域都 分别对应一个座位。
或者，示例性的，如图5(b)所示，图5(b)示出的是一种按列划分的4人座车辆的第二座位区域的示意图，其中，图5(b)中包含两个第二座位区域，分别为第二座位区域C503与第二座位区域D504；每一排包括前排区域和后排区域，每一前排区域和每一后排区域都分别对应一个座位。
针对步骤S106、
在一种可能的实施方式中，针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，上述基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域的步骤，如图6，可以通过步骤S601至步骤S603实现，其中：
步骤S601、按照预设顺序遍历所述第二座位区域，针对遍历至的每一所述第二座位区域；
在一种可能的实施方式中，在按照预设顺序遍历所述第二座位区域过程中，在出现多排或者多列座位的情况下，可以根据实际情况按照从驾驶人座位所在的第二座位区域开始，逐排或者逐列的进行遍历。
针对遍历至的每一第二座位区域，执行以下步骤S602和步骤S603：
步骤S602、将所述目标乘员的检测框与所述第二座位区域的交叠度，作为所述目标乘员的检测框与所述第二座位区域的第二匹配度；
步骤S603、在所述第二匹配度达到第二阈值的情况下，确定所述第二座位区域为与所述目标乘员的检测框对应的目标第二座位区域。
示例性的，如图7(a)，分别计算目标乘员的检测框701与第二座位区域E702、目标乘员的检测框701与第二座位区域F703的交叠度作为对应的第二匹配度，若目标乘员的检测框与第二座位区域E对应的第二匹配度达到第二阈值，则确定所述第二座位区域E为与目标乘员的检测框对应的目标第二座位区域。
在一种可能的实施方式中，针对遍历至的每一第二座位区域，在所述交叠度未达到第二阈值的情况下，可以确定与所述目标乘员的检测框的交叠面积最大的第二座位区域为目标第二座位区域。
示例性的，如图7(b)，所述目标乘员的检测框701与第二座位区域G704交叠面积大于所述目标乘员的检测框701与第二座位区域H705的交叠面积，将所述第二座位区域G704确定为所述目标乘员的检测框对应的目标第二座位区域。
针对步骤S107、
在一种的可能的实施方式中，上述根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位的步骤，可以如图8，包含步骤S801至步骤S802，其中：
步骤S801、基于所述目标乘员的检测框与所述目标第二座位区域中每一座位的位置范围，确定所述目标乘员的检测框的中心点落入的所述位置范围对应的座位为所述目标乘员的检测框对应的候选座位；
在确定所述目标乘员的检测框的中心点落入的所述位置范围时，在一种可能的实施方式中，可以通过计算目标乘员的检测框与所述每一座位的位置范围的中心点的欧式距离进行判断，示例性的，如图9，计算目标乘员的检测框901与左列区域之间中心点902的欧式距离、目标乘员的检测框901与右列区域之间中心点903的欧式距离，即目标乘员的检测框与两中心点间的直线距离。
然后，将直线距离最短的所述位置范围对应的座位，确定为所述目标乘员的检测框对应的候选座位。在如图9中，所述待绑定的目标乘员的检测框901的中心点距离左列区域的距离相对距离右列区域的距离更短，确定左列区域对应的座位作为所述目标乘员的检测 框对应的候选座位。
在另一种可能的实施方式中，可以分别计算所述目标乘员的检测框与所述目标第二座位区域每一座位之间的交集，取交集大者，确定为所述目标乘员的检测框对应的候选座位。
示例性的，如图9，所述待绑定的目标乘员的检测框与左列区域的交集相较于右列区域的更大，则确定左列区域对应的座位为所述目标乘员的检测框对应的候选座位。
步骤S802、在所述候选座位未绑定至其他检测框的情况下，确定所述候选座位为与所述目标乘员的检测框对应的座位，并将所述候选座位对应的座位绑定至所述目标乘员的检测框。
在一种可能的实施方式中，在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
在一种可能的实施方式中，可以先分别确定目标乘员的检测框与每一所述第一座位区域的中心点；再根据中心点，将两中心点间直线距离最短的，且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
示例性的，如图10，其中图10包含多个第二座位区域，分别为第一座位区域A1001、第一座位区域B1002、第一座位区域C1003与第一座位区域D1004，所述目标乘员的检测框1005并未与以上所述的四个第一座位区域有相交区域。由于所述目标乘员的检测框的中心点与第一座位区域A中心点的直线距离最短的，且未绑定其他检测框，因此将第一座位区域A1001确定为与所述目标乘员的检测框1005对应的座位。
基于本方案提出的确定车辆内乘员的座位的方法，本公开实施例还提供了一种车辆控制方法，所述方法由电子设备执行，参见图11所示，为本公开实施例提供的一种车辆控制方法的示意图，包括以下几个步骤：
步骤S1101、基于上述实施例所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的检测框；
步骤S1102、检测目标检测框内的目标乘员做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
在实际应用中，基于上述实施例所提出的确定车辆内乘员的座位的方法，可以实现车辆的控制，其中，响应于乘员的目标动作所对应的目标指令，车辆针对于所述目标指令执行对应的目标操作。
示例性的，所述车辆控制方法可以应用于智能驾驶领域，当检测到目标乘员所对应的目标检测框与目标座位区域绑定后。当乘车人做出指示“打开车窗”的手势后，车辆根据乘车人的手势指令，以及乘车人所乘坐的位置，打开对应车门上方的车窗。例如，在发出该手势指令的乘客为副驾座位的乘客的情况下，可以打开副驾一侧车门上方的车窗，由此实现了基于乘员座位检测的针对性控制。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与确定车辆内乘员的座位的方法对应的确定车辆内乘员的座位的装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述确定车辆内乘员的座位的方法相似，因此装置的实施可以参见方法的实施。
参照图12所示，为本公开实施例提供的一种确定车辆内乘员的座位的装置的架构示意图，所述装置包括：获取部分1201、第一确定部分1202、第二确定部分1203、第三确定部分1204、第四确定部分1205、第五确定部分1206、第六确定部分1207；其中，
获取部分1201，被配置为获取车辆内的车辆图像；
第一确定部分1202，被配置为确定所述车辆图像中多个第一座位区域，每一第一座位 区域对应一个座位；
第二确定部分1203，被配置为基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
第三确定部分1204，被配置为针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；
第四确定部分1205，被配置为确定所述车辆图像中多个第二座位区域，每一第二座位区域对应一排座位或一列座位；
第五确定部分1206，被配置为针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
第六确定部分1207，被配置为根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位。
在一种可能的实施方式中，所述第一确定部分1202，还被配置为：根据所述车辆的座位分布配置信息，确定所述车辆图像中每一座位对应的第一座位区域。
在一种可能的实施方式中，所述第三确定部分1204，还被配置为：按照预设顺序遍历每一所述第一座位区域，针对遍历至的每一第一座位区域：计算所述检测框与所述第一座位区域的交叠度，作为所述检测框与所述第一座位区域的第一匹配度；在所述第一匹配度达到第一阈值、且所述第一座位区域对应的座位未绑定其他检测框的情况下，将所述第一座位区域对应的座位绑定至所述检测框，并确定所述检测框对应的座位为所述第一座位区域对应的座位。
在一种可能的实施方式中，所述车辆图像中每一所述乘员的检测框为以下至少之一：人脸检测框、人体检测框。
在一种可能的实施方式中，所述第五确定部分1206，还被配置为：针对每一目标乘员的检测框，按照预设顺序遍历所述第二座位区域，针对遍历至的每一第二座位区域：计算所述目标乘员的测框与所述第二座位区域的交叠度，作为所述第二匹配度；在所述第二匹配度达到第二阈值的情况下，确定所述第二座位区域为与目标乘员的检测框对应的目标第二座位区域。
在一种可能的实施方式中，所述第六确定部分1207，还被配置为：基于所述目标乘员的检测框与所述目标第二座位区域每一座位的位置范围，确定所述目标乘员的检测框的中心点落入的所述位置范围对应的座位为所述目标乘员的检测框对应的候选座位；在所述候选座位未绑定至其他检测框的情况下，确定所述候选座位为与所述目标乘员的检测框对应的座位，并将所述候选座位对应的座位绑定至所述目标乘员的检测框。
在一种可能的实施方式中，所述第五确定部分1206，还被配置为：针对遍历至的每一第二座位区域，在所述交叠度未达到第二阈值的情况下，确定与所述目标乘员的检测框的交叠面积最大的第二座位区域为目标第二座位区域。
在一种可能的实施方式中，所述第七确定部分1208，还被配置为：在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
关于装置中的各部分的处理流程、以及各部分之间的交互流程的描述可以参照上述方法实施例中的相关说明。
基于同一发明构思，本公开实施例中还提供了与车辆控制方法对应的车辆控制装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述确定车辆内乘员的座位的方法相似，因此装置的实施可以参见方法的实施。
参照图13所示，为本公开实施例提供的一种车辆控制装置的架构示意图，所述装置包括：第八确定部分1301、检测部分1302；其中，
第八确定部分1301，被配置为基于以上实施例所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的检测框；
检测部分1302，被配置为检测目标检测框内的目标乘员做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
关于装置中的各部分的处理流程、以及每一所述部分之间的交互流程的描述可以参照上述方法实施例中的相关说明。
在本公开实施例以及其他的实施例中，“部分”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是单元，还可以是模块也可以是非模块化的。
基于同一技术构思，本公开实施例还提供了一种计算机设备。参照图14所示，为本公开实施例提供的计算机设备1400的结构示意图，包括处理器1401、存储器1402、和总线1403。其中，存储器1402用于存储执行指令，包括内存14021和外部存储器14022；这里的内存14021也称内存储器，用于暂时存放处理器1401中的运算数据，以及与硬盘等外部存储器14022交换的数据，处理器1401通过内存14021与外部存储器14022进行数据交换，当计算机设备1400运行时，处理器1401与存储器1402之间通过总线1403通信，使得处理器1401在执行以下指令：
获取车辆内的车辆图像；
确定所述车辆图像中多个第一座位区域，每一第一座位区域对应一个座位；
基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；；
确定所述车辆图像中多个第二座位区域，每一第二座位区域对应一排座位或一列座位；
针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框确定所述目标乘员的检测框对应的座位。
或者，处理器1401也可以执行以下指令：
基于以上实施例所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的检测框；
检测目标检测框内的目标乘员做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
本公开实施例还提供一种计算机可读存储介质，所述任一计算机可读存储介质上存储有计算机程序，所述任一计算机程序被处理器运行时执行上述方法实施例中所述的确定车辆内乘员的座位的方法、车辆控制方法的步骤。其中，所述任一存储介质可以是易失性或非易失的计算机可读取存储介质。
这里需要指出的是：以上存储介质和设备实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本公开存储介质和设备实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。
上述处理器可以为目标用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(Programmable Logic Device，PLD)、现场可编程门阵列(Field Programmable Gate Array，FPGA)、中央处理器(Central Processing Unit， CPU)、控制器、微控制器、微处理器中的至少一种。可以理解地，实现上述处理器功能的电子器件还可以为其它，本公开实施例不作具体限定。
上述计算机存储介质/存储器可以是只读存储器(Read Only Memory，ROM)、可编程只读存储器(Programmable Read-Only Memory，PROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、磁性随机存取存储器(Ferromagnetic Random Access Memory，FRAM)、快闪存储器(Flash Memory)、磁表面存储器、光盘、或只读光盘(Compact Disc Read-Only Memory，CD-ROM)等存储器；也可以是包括上述存储器之一或任意组合的各种终端，如移动电话、计算机、平板设备、个人数字助理等。
本公开实施例还提供一种计算机程序产品，所述任一计算机产品承载有程序代码，所述程序代码包括的指令可用于执行上述方法实施例中所述的确定车辆内乘员的座位的方法、车辆控制方法的步骤，可参见上述方法实施例。
其中，上述计算机程序产品可以通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品体现为计算机存储介质，在另一个可选实施例中，计算机程序产品体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的工作过程，可以参考前述方法实施例中的对应过程。在本公开所提供的几个实施例中，应所述任一理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者所述任一技术方案的部分可以以软件产品的形式体现出来，所述任一计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。
工业实用性
本公开实施例提供了一种确定车辆内乘员的座位及车辆控制方法及装置，包括：获取车辆内的车辆图像；确定车辆图像中多个第一座位区域；基于车辆图像检测车辆内的乘员，确定车辆图像中每一所述乘员的检测框；针对每一乘员的检测框，基于检测框与每一所述第一座位区域的第一匹配度，确定与检测框对应的座位；确定车辆图像中多个第二座位区域；针对基于第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与目标乘员的检测框对应的目标第二座位区域；根据与目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及目标乘员的检测框确定目标乘员的检测框对应的座位。在通过第一座位区域无法确定座位的目标乘员的检测框的情况下，扩大了第一座位区域的区域范围，并基于扩大区域范围的第二座位区域，再确定目标乘员的检测框对应的座位，由此可以降低乘员与座位匹配或绑定失败的概率，提升乘员座位匹配的成功率。

Claims (21)

  1. 一种确定车辆内乘员的座位的方法，所述方法由电子设备执行，所述方法包括：
    获取车辆内的车辆图像；
    确定所述车辆图像中多个第一座位区域，每一所述第一座位区域对应一个座位；
    基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
    针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；
    确定所述车辆图像中多个第二座位区域，每一所述第二座位区域对应一排座位或一列座位；
    针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
    根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框，确定所述目标乘员的检测框对应的座位。
  2. 根据权利要求1所述的方法，其中，所述确定所述车辆图像中多个第一座位区域，包括：
    根据所述车辆的座位分布配置信息，确定所述车辆图像中每一座位对应的第一座位区域。
  3. 根据权利要求1或2所述的方法，其中，所述基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位，包括：
    按照预设顺序遍历每一所述第一座位区域，针对遍历至的每一所述第一座位区域：
    将所述检测框与所述第一座位区域的交叠度，作为所述检测框与所述第一座位区域的第一匹配度；
    在所述第一匹配度达到第一阈值、且所述第一座位区域对应的座位未绑定其他检测框的情况下，将所述第一座位区域对应的座位绑定至所述检测框，并确定所述检测框对应的座位为所述第一座位区域对应的座位。
  4. 根据权利要求1至3任一项所述的方法，其中，所述车辆图像中每一所述乘员的检测框为以下至少之一：人脸检测框、人体检测框。
  5. 根据权利要求1至4任一项所述的方法，其中，所述基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域，包括：
    按照预设顺序遍历每一所述第二座位区域，针对遍历至的每一所述第二座位区域：
    将所述目标乘员的检测框与所述第二座位区域的交叠度，作为所述目标乘员的检测框与所述第二座位区域的第二匹配度；
    在所述第二匹配度达到第二阈值的情况下，确定所述第二座位区域为与所述目标乘员的检测框对应的目标第二座位区域。
  6. 根据权利要求1至5任一项所述的方法，其中，所述根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框，确定所述目标乘员的检测框对应的座位，包括：
    基于所述目标乘员的检测框与所述目标第二座位区域中每一座位的位置范围，确定所述目标乘员的检测框的中心点落入的所述位置范围对应的座位为所述目标乘员的检测框对应的候选座位；
    在所述候选座位未绑定至其他检测框的情况下，确定所述候选座位为与所述目标乘员的检测框对应的座位，并将所述候选座位对应的座位绑定至所述目标乘员的检测框。
  7. 根据权利要求5所述的方法，其中，所述基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域，还包括：
    针对遍历至的每一所述第二座位区域，在所述交叠度未达到所述第二阈值的情况下，确定与所述目标乘员的检测框的交叠面积最大的第二座位区域为目标第二座位区域。
  8. 根据权利要求1至7任一所述的方法，其中，所述方法还包括：
    在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
  9. 一种车辆控制方法，所述方法由电子设备执行，所述方法包括：
    基于权利要求1至8任一项所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的检测框；
    检测目标检测框内的目标乘员做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
  10. 一种确定车辆内乘员的座位的装置，包括：
    获取部分，被配置为获取车辆内的车辆图像；
    第一确定部分，被配置为确定所述车辆图像中多个第一座位区域，每一所述第一座位区域对应一个座位；
    第二确定部分，被配置为基于所述车辆图像检测所述车辆内的乘员，确定所述车辆图像中每一所述乘员的检测框；
    第三确定部分，被配置为针对每一所述乘员的检测框，基于所述检测框与每一所述第一座位区域的第一匹配度，确定与所述检测框对应的座位；
    第四确定部分，被配置为确定所述车辆图像中多个第二座位区域，每一所述第二座位区域对应一排座位或一列座位；
    第五确定部分，被配置为针对基于所述第一匹配度未确定出对应的目标第一座位区域的目标乘员的检测框，基于所述目标乘员的检测框与每一所述第二座位区域的第二匹配度，确定与所述目标乘员的检测框对应的目标第二座位区域；
    第六确定部分，被配置为根据与所述目标第二座位区域对应的一排座位或一列座位中的每一座位的位置范围，以及所述目标乘员的检测框，确定所述目标乘员的检测框对应的座位。
  11. 根据权利要求10所述的装置，其中，所述第一确定部分，还被配置为：根据所述车辆的座位分布配置信息，确定所述车辆图像中每一座位对应的第一座位区域。
  12. 根据权利要求10或11所述的装置，其中，所述第三确定部分，还被配置为：按照预设顺序遍历每一所述第一座位区域，针对遍历至的每一第一座位区域：计算所述检测框与所述第一座位区域的交叠度，作为所述检测框与所述第一座位区域的第一匹配度；在所述第一匹配度达到第一阈值、且所述第一座位区域对应的座位未绑定其他检测框的情况下，将所述第一座位区域对应的座位绑定至所述检测框，并确定所述检测框对应的座位为所述第一座位区域对应的座位。
  13. 根据权利要求10至12任一项所述的装置，其中，所述车辆图像中每一所述乘员的检测框为以下至少之一：人脸检测框、人体检测框。
  14. 根据权利要求10至13任一项所述的装置，其中，所述第五确定部分，还被配置为：针对每一目标乘员的检测框，按照预设顺序遍历所述第二座位区域，针对遍历至的每一第二座位区域：计算所述目标乘员的测框与所述第二座位区域的交叠度，作为所述第二匹配度；在所述第二匹配度达到第二阈值的情况下，确定所述第二座位区域为与目标乘员的检测框对应的目标第二座位区域。
  15. 根据权利要求10至14任一项所述的装置，其中，所述第六确定部分，还被配置为：基于所述目标乘员的检测框与所述目标第二座位区域每一座位的位置范围，确定所述目标乘员的检测框的中心点落入的所述位置范围对应的座位为所述目标乘员的检测框对应的候选座位；在所述候选座位未绑定至其他检测框的情况下，确定所述候选座位为与所述目标乘员的检测框对应的座位，并将所述候选座位对应的座位绑定至所述目标乘员的检测框。
  16. 根据权利要求14所述的装置，其中，所述第五确定部分，还被配置为：针对遍历至的每一第二座位区域，在所述交叠度未达到第二阈值的情况下，确定与所述目标乘员的检测框的交叠面积最大的第二座位区域为目标第二座位区域。
  17. 根据权利要求10至16任一项所述的装置，其中，所述第七确定部分，还被配置为：在未确定与所述目标乘员的检测框对应的目标第二座位区域的情况下，将对应的第一座位区域距离所述目标乘员的检测框最近的、且未绑定其他检测框的座位确定为与所述目标乘员的检测框对应的座位。
  18. 一种车辆控制装置，其中，包括：
    第八确定部分，被配置为基于权利要求1至8任一项所述的确定车辆内乘员的座位的方法，确定每一所述座位区域对应的检测框；
    检测部分，被配置为检测目标检测框内的目标乘员做出的目标动作，基于所述目标动作以及所述目标检测框对应的座位区域进行车辆控制。
  19. 一种计算机设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当计算机设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至8任一项所述的确定车辆内乘员的座位的方法的步骤，或执行如权利要求9所述的车辆控制方法的步骤。
  20. 一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器运行时执行如权利要求1至8任一项所述的确定车辆内乘员的座位的方法的步骤，或执行如权利要求9所述的车辆控制方法的步骤。
  21. 一种计算机程序产品，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行时执行如权利要求1至8任一项所述的确定车辆内乘员的座位的方法的步骤，或执行如权利要求9所述的车辆控制方法的步骤。
PCT/CN2022/124919 2021-12-31 2022-10-12 确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品 Ceased WO2023124384A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024539629A JP7792005B2 (ja) 2021-12-31 2022-10-12 車両内の乗員の座席を決定する方法及び車両制御方法、装置、コンピュータ機器、コンピュータ可読記憶媒体並びにコンピュータプログラム
KR1020247024146A KR20240124371A (ko) 2021-12-31 2022-10-12 차량 내의 탑승자의 좌석을 결정하는 방법 및 차량 제어 방법, 장치, 전자 기기, 매체 및 프로그램 제품

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111682157.3 2021-12-31
CN202111682157.3A CN114312580B (zh) 2021-12-31 2021-12-31 确定车辆内乘员的座位及车辆控制方法及装置

Publications (1)

Publication Number Publication Date
WO2023124384A1 true WO2023124384A1 (zh) 2023-07-06

Family

ID=81023728

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/124919 Ceased WO2023124384A1 (zh) 2021-12-31 2022-10-12 确定车辆内乘员的座位及车辆控制方法、装置、电子设备、介质和程序产品

Country Status (4)

Country Link
JP (1) JP7792005B2 (zh)
KR (1) KR20240124371A (zh)
CN (1) CN114312580B (zh)
WO (1) WO2023124384A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114312580B (zh) * 2021-12-31 2024-03-22 上海商汤临港智能科技有限公司 确定车辆内乘员的座位及车辆控制方法及装置
JP7683546B2 (ja) * 2022-06-01 2025-05-27 トヨタ自動車株式会社 シートポジション推定装置、シートポジション推定方法及びシートポジション推定用コンピュータプログラム
CN115731535A (zh) * 2022-11-22 2023-03-03 北京地平线信息技术有限公司 确定车辆中目标对象的位置的方法、装置、介质和设备
CN116442869B (zh) * 2022-12-29 2026-02-27 上海洛轲智能科技有限公司 一种车辆座椅调节方法及装置
CN119018028B (zh) * 2024-10-29 2025-02-18 中国第一汽车股份有限公司 基于情绪状态的座位排列方法、装置、设备、介质及产品

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110781799A (zh) * 2019-10-22 2020-02-11 上海商汤智能科技有限公司 车舱内图像处理方法及装置
CN112208475A (zh) * 2019-07-09 2021-01-12 奥迪股份公司 用于车辆乘员的安全保护系统、车辆及相应的方法和介质
US20210008958A1 (en) * 2019-07-10 2021-01-14 Ambarella International Lp Efficient vehicle ac based on car occupancy detected by computer vision
CN112947740A (zh) * 2019-11-22 2021-06-11 深圳市超捷通讯有限公司 基于动作分析的人机交互方法、车载装置
CN114312580A (zh) * 2021-12-31 2022-04-12 上海商汤临港智能科技有限公司 确定车辆内乘员的座位及车辆控制方法及装置

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4202908B2 (ja) * 2003-12-26 2008-12-24 カルソニックカンセイ株式会社 車両用乗員検出システム
DE102015202304A1 (de) * 2015-02-10 2016-08-11 Siemens Aktiengesellschaft Verfahren zum Betreiben eines Fahrzeugs und Fahrzeug
JP2016203910A (ja) * 2015-04-27 2016-12-08 トヨタ自動車株式会社 乗員検知装置および乗員検知方法
JP6645416B2 (ja) * 2016-12-20 2020-02-14 株式会社デンソー 乗員検出装置、乗員検出システム、乗員検出方法
CN107392094A (zh) * 2017-06-14 2017-11-24 蔚来汽车有限公司 自动调整车载音响音效模式的系统和方法
JP6739672B2 (ja) * 2018-03-22 2020-08-12 三菱電機株式会社 体格推定装置および体格推定方法
CN110103820A (zh) * 2019-04-24 2019-08-09 深圳市轱辘汽车维修技术有限公司 一种检测车辆中人员的异常行为的方法、装置及终端设备
CN110414406A (zh) * 2019-07-23 2019-11-05 广汽蔚来新能源汽车科技有限公司 车内对象监管方法、装置、系统、车载终端和存储介质
CN111008566A (zh) * 2019-11-06 2020-04-14 湖北工业大学 一种基于深度学习的针对校车学生下车遗漏检测装置及方法
CN111275002A (zh) * 2020-02-18 2020-06-12 上海商汤临港智能科技有限公司 一种图像处理方法、装置及电子设备
CN111460938B (zh) * 2020-03-20 2022-04-08 南京领行科技股份有限公司 一种车辆行驶行为实时监测方法及装置
CN111738158A (zh) * 2020-06-23 2020-10-02 上海商汤临港智能科技有限公司 交通工具的控制方法及装置、电子设备和存储介质
CN113159004B (zh) * 2021-05-27 2023-03-10 苏州大学 轨道交通车厢客流估计方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112208475A (zh) * 2019-07-09 2021-01-12 奥迪股份公司 用于车辆乘员的安全保护系统、车辆及相应的方法和介质
US20210008958A1 (en) * 2019-07-10 2021-01-14 Ambarella International Lp Efficient vehicle ac based on car occupancy detected by computer vision
CN110781799A (zh) * 2019-10-22 2020-02-11 上海商汤智能科技有限公司 车舱内图像处理方法及装置
CN112947740A (zh) * 2019-11-22 2021-06-11 深圳市超捷通讯有限公司 基于动作分析的人机交互方法、车载装置
CN114312580A (zh) * 2021-12-31 2022-04-12 上海商汤临港智能科技有限公司 确定车辆内乘员的座位及车辆控制方法及装置

Also Published As

Publication number Publication date
JP7792005B2 (ja) 2025-12-24
CN114312580B (zh) 2024-03-22
CN114312580A (zh) 2022-04-12
KR20240124371A (ko) 2024-08-16
JP2025501260A (ja) 2025-01-17

Similar Documents

Publication Publication Date Title
JP7792005B2 (ja) 車両内の乗員の座席を決定する方法及び車両制御方法、装置、コンピュータ機器、コンピュータ可読記憶媒体並びにコンピュータプログラム
US10140533B1 (en) Apparatuses, systems and methods for generating data representative of vehicle occupant postures
US10223603B1 (en) Apparatuses, systems, and methods for determining when a vehicle occupant is using a mobile telephone
US10025388B2 (en) Touchless human machine interface
US12515679B2 (en) Apparatuses, systems and methods for determining distracted drivers associated with vehicle driving routes
CN103870802A (zh) 使用指谷操作车辆内的用户界面的系统和方法
US10783360B1 (en) Apparatuses, systems and methods for vehicle operator gesture recognition and transmission of related gesture data
US9990553B1 (en) Apparatuses, systems, and methods for determining degrees of risk associated with a vehicle operator
US10282624B1 (en) Apparatuses, systems, and methods for determining when a vehicle operator is texting while driving
US10943136B1 (en) Apparatuses, systems and methods for generating a vehicle driver signature
CN117218716B (zh) 一种基于dvs的汽车座舱手势识别系统及方法
US12340602B2 (en) Apparatuses, systems and methods for generating a vehicle driver signature
CN109614953A (zh) 一种基于图像识别的控制方法、车载设备及存储介质
US20220148319A1 (en) Apparatuses, systems and methods for integrating vehicle operator gesture detection within geographic maps
US20240169744A1 (en) Apparatuses, systems and methods for generation and transmission of vehicle operation mode data
US10891502B1 (en) Apparatuses, systems and methods for alleviating driver distractions
US10452933B1 (en) Apparatuses, systems and methods for generating a vehicle driver model for a particular vehicle
CN118418994A (zh) 泊车方法、装置、车辆及计算机可读取存储介质
HK40063397A (zh) 确定车辆内乘员的座位及车辆控制方法及装置
CN118552941B (zh) 一种车辆内人员落座信息检测方法、装置、设备及介质
CN120295478A (zh) 一种基于眼动追踪的自动化任务控制方法及装置、车辆
CN116610216A (zh) 用于实现人机交互的方法和设备
WO2024198839A1 (zh) 基于手势识别的车辆控制方法、装置以及车辆

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22913648

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024539629

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 20247024146

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22913648

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 11/12/2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22913648

Country of ref document: EP

Kind code of ref document: A1