WO2023124384A1 - Methods and apparatuses for determining seat of occupant inside vehicle and for vehicle control, electronic device, medium, and program product - Google Patents

Abstract

Methods and apparatuses for determining a seat of an occupant inside a vehicle, and for vehicle control. A method comprises: acquiring a vehicle image inside a vehicle; determining multiple first seat areas in the vehicle image; detecting an occupant inside the vehicle on the basis of the vehicle image and determining a detection box for each occupant in the vehicle image; for the detection box of each occupant, on the basis of a first matching degree between the detection box and each first seat area, determining a seat corresponding to the detection box; determining multiple second seat areas in the vehicle image; for a target occupant detection box for which no corresponding target first seat area has been determined on the basis of the first matching degree, determining a target second seat area corresponding to the target occupant detection box, on the basis of a second matching degree between the detection box of the target occupant and each second seat area; determining a seat corresponding to the target occupant detection box according to a position range of each seat in a row of seats or a column of seats corresponding to the target second seat area, and according to the target occupant detection box.

Description

Determining the seat of an occupant in a vehicle and vehicle control method, apparatus, electronic device, medium and program product

Cross References to Related Applications

The embodiment of the present disclosure is based on the Chinese patent application with the application number 202111682157.3, the application date is December 31, 2021, and the application name is "Determining the seat of the occupants in the vehicle and the vehicle control method and device", and requires the Chinese patent application Priority, the entire content of the Chinese patent application is hereby incorporated by reference into this disclosure.

technical field

The present disclosure relates to the technical field of intelligent driving, and relates to a method, device, electronic device, medium and program product for determining the seat of an occupant in a vehicle and controlling the vehicle.

Background technique

With the rapid development of the automobile industry, in addition to the iterative update of important parts such as the engine necessary for the automobile, many emerging technologies related to the automobile industry are also born with the rapid development of the automobile industry, including intelligent driving of vehicles technology. In the vehicle intelligent driving technology, in order to achieve the effect of allowing the vehicle to make corresponding feedback according to the actions of the person, it is necessary to bind the person and the vehicle seat in advance, and then according to the different actions of the person. The intelligent system makes corresponding judgments.

In the related art, when the person is bound to the vehicle seat, it often occurs that the person and the vehicle seat are bound incorrectly due to objective factors such as sitting posture or height. Therefore, how to correctly bind the person to the vehicle seat become an urgent problem to be solved.

Contents of the invention

Embodiments of the present disclosure at least provide a method, device, electronic device, medium, and program product for determining the seat of an occupant in a vehicle and controlling the vehicle.

In one aspect, an embodiment of the present disclosure provides a method for determining a seat of an occupant in a vehicle, including:

obtain an image of the vehicle inside the vehicle;

determining a plurality of first seat areas in the vehicle image, each of the first seat areas corresponds to a seat;

detecting occupants in the vehicle based on the vehicle image, and determining a detection frame for each occupant in the vehicle image;

For each occupant's detection frame, based on the first matching degree between the detection frame and each of the first seat areas, determine the seat corresponding to the detection frame;

determining a plurality of second seating areas in the vehicle image, each of the second seating areas corresponds to a row of seats or a row of seats;

For the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching degree between the detection frame of the target occupant and each of the second seating areas, determine the The target second seat area corresponding to the detection frame of the target occupant;

The seat corresponding to the detection frame of the target occupant is determined according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant.

In the solution provided by the embodiment of the present disclosure, if the detection frame of the target occupant of the seat cannot be determined through the first seat area, the area range of the first seat area is expanded, and based on the expanded area range of the second seat area, Then determine the seat corresponding to the detection frame of the target occupant, thereby reducing the probability of occupant and seat matching or binding failure, and improving the success rate of occupant seat matching.

In a possible implementation manner, the determining a plurality of first seat areas in the vehicle image includes:

A first seat area corresponding to each seat in the vehicle image is determined according to the seat distribution configuration information of the vehicle.

The seat distribution configuration information of different models may be different, and the first seat area can be determined more accurately through the seat distribution configuration information of the vehicle.

In a possible implementation manner, the detection frame for each occupant determines the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas, include:

Traversing each of the first seating areas in a preset order, for each of the first seating areas traversed to:

calculating the degree of overlap between the detection frame and the first seat area as the first matching degree between the detection frame and the first seat area;

When the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to other detection frames, bind the seat corresponding to the first seat area to the detection frame, And determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area.

When the first matching degree reaches the first threshold, detecting whether other detection frames are bound to the first seat area can avoid binding multiple detection frames to one seat area, thereby avoiding When the vehicle is controlled, it affects the driving of the vehicle.

In a possible implementation manner, the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.

In a possible implementation manner, the target second seat area corresponding to the detection frame of the target occupant is determined based on the second matching degree between the detection frame of the target occupant and each of the second seat areas, include:

For the detection frame of each target occupant, traverse the second seating area according to a preset order, and for each second seating area traversed to:

calculating the degree of overlap between the detection frame of the target occupant and the second seat area as the second matching degree;

In a case where the second matching degree reaches a second threshold, the second seating area is determined to be a target second seating area corresponding to the detection frame of the target occupant.

By this method, it is possible to first detect which row or column the target member is, and then perform further seat binding, thereby improving the accuracy of the binding result.

In a possible implementation manner, the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant are determined according to the location of the target occupant. Seats corresponding to the detection frame, including:

Based on the detection frame of the target occupant and the position range of each seat in the target second seat area, determine that the seat corresponding to the position range where the center point of the detection frame of the target occupant falls is the target occupant The candidate seat corresponding to the detection frame;

If the candidate seat is not bound to other detection frames, determine that the candidate seat is the seat corresponding to the detection frame of the target occupant, and bind the seat corresponding to the candidate seat to the target occupant detection frame.

In a possible implementation manner, the target second seat area corresponding to the detection frame of the target occupant is determined based on the second matching degree between the detection frame of the target occupant and each of the second seat areas, Also includes:

For each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine the second seat area with the largest overlapping area with the detection frame of the target occupant as the target second seat area.

In a possible implementation manner, the method also includes:

If the target second seat area corresponding to the detection frame of the target occupant is not determined, determine the seat corresponding to the first seat area closest to the detection frame of the target occupant and not bound to other detection frames is the seat corresponding to the detection frame of the target occupant.

On the other hand, an embodiment of the present disclosure also provides a vehicle control method, including:

Based on the method for determining the seat of an occupant in the vehicle described in the first aspect or any possible implementation manner of the first aspect, determine the user detection frame corresponding to each seat area;

A target action made by the target user within the target detection frame is detected, and vehicle control is performed based on the target action and the seat area corresponding to the target detection frame.

Based on the method for determining the seat of an occupant in the vehicle described in the above embodiment, when used for vehicle control, it can respond to the action command issued by the user, and provide targeted feedback according to the user's action and its seat, which improves the user's ability to control the vehicle. The interactive feeling when using the vehicle.

In yet another aspect, an embodiment of the present disclosure also provides a device for determining the seat of an occupant in a vehicle, including:

an acquisition part configured to acquire a vehicle image within the vehicle;

A first determining part configured to determine a plurality of first seat areas in the vehicle image, each first seat area corresponding to a seat;

a second determining section configured to detect occupants in the vehicle based on the vehicle image, and determine a detection frame for each of the occupants in the vehicle image;

The third determining part is configured to, for each occupant's detection frame, determine the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas;

A fourth determining part configured to determine a plurality of second seating areas in the vehicle image, each second seating area corresponding to a row of seats or a row of seats;

The fifth determining part is configured to, for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the detection frame of the target occupant and each of the second seating areas The second matching degree, determining the target second seat area corresponding to the detection frame of the target occupant;

The sixth determining part is configured to determine the detection frame of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area, and the detection frame of the target occupant corresponding seat.

In a possible implementation manner, the first determination part is further configured to: determine the first seat area corresponding to each seat in the vehicle image according to the seat distribution configuration information of the vehicle.

In a possible implementation manner, the third determining part is further configured to: traverse each of the first seating areas in a preset order, and for each first seating area traversed to: calculate the detection The degree of overlap between the frame and the first seating area is used as the first matching degree between the detection frame and the first seating area; when the first matching degree reaches a first threshold and the first seating area If the corresponding seat is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame, and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area .

In a possible implementation manner, the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.

In a possible implementation manner, the fifth determining part is further configured to: for each detection frame of the target occupant, traverse the second seating area in a preset order, and for each second Seat area: calculating the degree of overlap between the frame of the target occupant and the second seat area as the second matching degree; when the second matching degree reaches a second threshold, determine the second matching degree The second seat area is a target second seat area corresponding to the detection frame of the target occupant.

In a possible implementation manner, the sixth determining part is further configured to: determine the target occupant's location based on the detection frame of the target occupant and the position range of each seat in the target second seating area The seat corresponding to the position range where the center point of the detection frame falls is the candidate seat corresponding to the detection frame of the target occupant; if the candidate seat is not bound to other detection frames, determine that the candidate seat is A seat corresponding to the detection frame of the target occupant, and binding the seat corresponding to the candidate seat to the detection frame of the target occupant.

In a possible implementation manner, the fifth determining part is further configured to: for each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine the The second seat area with the largest overlapping area of the detection frames of the target occupant is the target second seat area.

In a possible implementation manner, the seventh determining part is further configured to: if the target second seat area corresponding to the detection frame of the target occupant is not determined, set the corresponding first seat area The seat closest to the detection frame of the target occupant and not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.

In yet another aspect, an embodiment of the present disclosure also provides a vehicle control device, including:

The eighth determination part is configured to determine the user detection frame corresponding to each seat area based on the method for determining the seat of the occupant in the vehicle described in the first aspect or any possible implementation manner of the first aspect;

The detection part is configured to detect the target action made by the target user within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.

In yet another aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the steps in any one of the above possible implementation manners are performed.

In yet another aspect, the embodiments of the present disclosure also provide a computer-readable storage medium, where any computer-readable storage medium stores a computer program, and any computer program executes any of the above-mentioned possibilities when run by a processor. steps in the implementation of .

In yet another aspect, an embodiment of the present disclosure further provides a computer program product, including computer readable code, when the computer readable code is run in an electronic device, the processor in the electronic device executes any one of the above-mentioned steps in a possible implementation.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

Fig. 1 shows a flow chart of a method for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure;

Fig. 2 shows a schematic diagram of the division method of the first seat area of a 4-seater vehicle provided by an embodiment of the present disclosure;

Fig. 3 shows a flow chart of determining a seat corresponding to the detection frame provided by an embodiment of the present disclosure;

Fig. 4 shows a schematic diagram of a method for calculating the degree of overlap between the detection frame and the first seat area provided by an embodiment of the present disclosure;

Figure 5(a) shows a schematic diagram of the second seating area of a 4-seater vehicle divided by row provided by an embodiment of the present disclosure;

Fig. 5(b) shows a schematic diagram of the second seat area of a 4-seater vehicle divided by column provided by an embodiment of the present disclosure;

Fig. 6 shows a flow chart of determining a target second seat area corresponding to the detection frame of the target occupant provided by an embodiment of the present disclosure;

Fig. 7(a) shows a schematic diagram of determining a target second seat area corresponding to the detection frame of the target occupant provided by an embodiment of the present disclosure;

Fig. 7(b) shows another schematic diagram of determining the target second seat area corresponding to the detection frame of the target occupant provided by the embodiment of the present disclosure;

Fig. 8 shows a flow chart of determining the seat corresponding to the detection frame of the target occupant based on the target second seat area provided by an embodiment of the present disclosure;

Fig. 9 shows a schematic diagram of determining the seat corresponding to the detection frame of the target occupant based on the second target seat area provided by an embodiment of the present disclosure;

Fig. 10 shows a detection method that distances the corresponding first seat area from the target occupant when the target second seat area corresponding to the detection frame of the target occupant is not determined according to an embodiment of the present disclosure. A schematic diagram of determining the seat closest to the frame and not bound to other detection frames as the seat corresponding to the detection frame of the target occupant;

Fig. 11 shows a flowchart of a vehicle control method provided by an embodiment of the present disclosure;

Fig. 12 shows a schematic structural diagram of a device for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure;

Fig. 13 shows a schematic structural diagram of a vehicle control device provided by an embodiment of the present disclosure;

FIG. 14 shows a schematic structural diagram of a computer device 1400 provided by an embodiment of the present disclosure.

Detailed ways

In order to enable those skilled in the art to better understand the present disclosure, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are Some embodiments of this application are not all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present disclosure.

The terms "first", "second" and the like in the specification and claims of the present disclosure and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include for these processes, Other steps or elements inherent in a method, product, or apparatus.

It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three And three or more, "and/or", is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, "A and/or B" can mean: only A exists, only B exists, and A exists at the same time and B, where A and B can be singular or plural. The character "/" can indicate that the contextual objects are an "or" relationship, which refers to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (piece) of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c ", where a, b, c can be single or multiple. The character "/" can also represent a division sign in mathematical operations, for example, a/b=a divided by b; 6/3=2. "At least one of the following" or similar expressions.

After research, it is found that in the existing disclosed technologies for binding people and vehicle seats, in the process of binding people and vehicle seats, it often occurs that the machine cannot recognize the person due to objective factors such as the person's sitting posture or height. Some situations, such as children who are small in stature, not tall enough and easy to move, will not be detected by the machine.

Moreover, in the existing published technology about the binding of people and vehicle seats, generally only one binding method is adopted in the process of binding people and vehicle seats, and only one binding is performed, and there is no relevant back-and-forth strategy , It is easy to cause misjudgment of the execution equipment, which may lead to wrong binding between the person and the vehicle seat.

Based on the above research, an embodiment of the present disclosure provides a method for determining the seat of an occupant in a vehicle. In the case where the detection frame of the target occupant of the seat cannot be determined through the first seat area, the area range of the first seat area is expanded, Based on the expanded second seat area, the seat corresponding to the detection frame of the target occupant is determined, thereby reducing the probability of occupant and seat matching or binding failure, and improving the success rate of occupant seat matching.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

To facilitate the understanding of this embodiment, a method for determining the seat of an occupant in a vehicle and a vehicle control method disclosed in the embodiment of the present disclosure are first introduced in detail. The method for determining the seat of an occupant in a vehicle provided by the embodiment of the present disclosure, The execution subject of the vehicle control method is a vehicle control device (such as a rail transit cockpit intelligent analysis box, a bus cockpit intelligent analysis box), or a server.

Referring to FIG. 1 , which is a flow chart of a method for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure, the method includes steps S101 to S107, wherein:

Step S101, acquiring a vehicle image in the vehicle;

Step S102, determining a plurality of first seat areas in the vehicle image, each of the first seat areas corresponds to a seat;

Step S103. Detect occupants in the vehicle based on the vehicle image, and determine a detection frame for each occupant in the vehicle image;

Step S104, for each occupant's detection frame, based on the first matching degree between the detection frame and each of the first seat areas, determine the seat corresponding to the detection frame;

Step S105, determining a plurality of second seat areas in the vehicle image, each of the second seat areas corresponds to a row of seats or a row of seats;

Step S106, for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching degree between the detection frame of the target occupant and each of the second seating areas , determining a target second seat area corresponding to the detection frame of the target occupant;

Step S107: Determine the seat corresponding to the detection frame of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant.

Step S101 to step S107 will be described in detail below.

For step S101,

In a possible implementation manner, step S101 may include: acquiring a vehicle image in the vehicle in response to satisfying an image acquisition trigger condition.

Wherein, the satisfying the image acquisition trigger condition and acquiring the vehicle image of the vehicle can be implemented as any one of the following scenarios:

Scenario A: The driver turns on the vehicle to start the vehicle. Among them, the vehicle can be started by igniting or electrifying. After the vehicle is started, the vehicle circuit system is connected, the vehicle is connected to the network, and the vehicle image in the vehicle is started to be acquired.

Scenario B: The driver opens the car door or triggers the car's start switch, wherein triggering the car's start switch can be done in at least one of the following ways, such as turning on the vehicle's remote control switch, inserting the vehicle's start key, or using a wearable device to touch the car body. Among them, after detecting that the driver opens the door or triggers the car start switch, the image of the vehicle inside the vehicle is acquired.

In scene C, when the vehicle is running, images in the vehicle are collected regularly according to a preset cycle, that is, images of the vehicle in the vehicle can be acquired at intervals.

In a possible implementation manner, the vehicle image is an image including the environment inside the vehicle. Exemplarily, the content of the vehicle image may include all seats in the vehicle cabin, people in the vehicle, and other pets.

In a possible implementation manner, the vehicle image may be acquired by using a driver monitoring system (driver monitoring system, DMS), and may be photographed by a DMS camera during the acquisition process. In the process of using DMS to collect vehicle images, because the situation inside the vehicle is changing dynamically in real time, it is necessary to collect vehicle images in real time during the driving process of the vehicle.

In practical applications, since the occupant's sitting position in the vehicle may change during driving, for each frame of vehicle image collected, the detection frame and seat area can be determined based on steps S102 to S107.

In a possible implementation, the method of collecting and determining the occupant's seat at time intervals may be used. After one collection, after a preset time interval, the vehicle image is collected again according to the current vehicle condition. Adopting this method can reduce the collection pressure of the DMS system and reduce resource occupation. In actual use, the preset time period can be set as small as possible, so that the DMS system can update the vehicle image in a more timely manner according to the vehicle situation in the vehicle, so that while reducing resource occupation, it can also minimize collection The error between the vehicle image and the real-time vehicle situation in the vehicle.

In a possible implementation manner, in response to satisfying an image acquisition trigger condition, the server performs an operation of acquiring vehicle images in the vehicle. Wherein, after it is detected that the vehicle is connected to a preset system, the server corresponding to the preset system can detect whether the vehicle satisfies the image acquisition trigger condition. When it is detected that the vehicle satisfies the image acquisition triggering condition, the server may send a vehicle image acquisition request to the vehicle camera or the vehicle host connected to the vehicle camera, so as to acquire the vehicle image in the vehicle.

Wherein, the vehicle accessing a preset system, for example, may refer to accessing a preset system through a vehicle-mounted system (such as a navigation system built into a vehicle); or it may be through a wearable device (such as smart watch, etc.), connected to the preset system.

In the case that the vehicle is only connected to the preset system but does not meet the image acquisition trigger condition, in order to reduce resource occupation, the server may not acquire the vehicle image in the vehicle.

In another possible implementation, the vehicle does not need to be connected to a preset system, and after the vehicle satisfies the image acquisition trigger condition, the vehicle host terminal directly acquires the vehicle image in the vehicle. In this way, the DMS system on the vehicle side can directly monitor the vehicle in real time, collect vehicle images in the vehicle, and improve the safety of the vehicle.

The above two implementations can be used in combination in specific scenarios. In the case that the vehicle cannot access the server due to extreme circumstances, for example, the car is soaked in water and the engine is turned off, the vehicle is driving on a road with no signal, etc., the built-in DMS of the vehicle can The system detects the vehicle condition in the vehicle and updates the current vehicle image; after the vehicle returns to normal condition, the server can continue to obtain the vehicle image in the vehicle.

For step S102,

After the vehicle image in the vehicle is acquired, a plurality of first seat areas in the vehicle image may be determined.

Wherein, each said first seat area is an independently seatable seat area inside the vehicle, when dividing the first seat area, each independently seatable seat inside the vehicle can be divided into a first seat area, a There are multiple first seating areas within the vehicle. Exemplarily, as shown in FIG. 2 , a way of dividing the first seating area of a 4-seater vehicle is shown in FIG. 2 .

Wherein, this 4-seater vehicle comprises 4 seats respectively, and the upper left is the driver's seat 201, the upper right is the co-driver's seat 202, the lower left and the lower right correspond to two passenger seats (including 203 and 204) respectively, and each seat can be Travel alone with at least one passenger. When dividing the first seating area, each seat can be divided into a corresponding first seating area, therefore, this 4-seater vehicle includes 4 first seating areas, namely the driver's seat area 201 and the passenger's seat area 202 , the passenger A seating area 203 , and the passenger B seating area 204 .

In a possible implementation manner, in the process of determining multiple first seat areas in the vehicle image, the first seating area corresponding to each seat in the vehicle image may be determined according to the seat distribution configuration information of the vehicle. seating area.

The seat distribution configuration information of the vehicle may exemplarily include the first seat range information of each seat inside the vehicle, the seat position arrangement and the number of seats of the vehicle, and the like.

In a possible implementation manner, the manner of determining the seat distribution configuration information of the vehicle may include: determining the seat distribution configuration information corresponding to the vehicle type information according to the vehicle type information. Wherein, the corresponding relationship between the vehicle type information and the seat distribution configuration information may be stored in advance, and the seat distribution configuration information corresponding to the vehicle type information may be searched based on the corresponding relationship after the seat distribution configuration information of the vehicle is determined.

Wherein, in the process of determining the vehicle type information of the vehicle, if the vehicle can implement a preset policy of automatically reporting the vehicle type information, the vehicle can automatically report the vehicle type information when the vehicle is connected to the preset system; If the vehicle model used by the occupant is too old to implement the preset automatic reporting strategy, the occupant can report the relevant vehicle type information by himself.

Wherein, the first seat range information may include the corresponding image coordinate range information of the first seat area in the vehicle cabin of the vehicle type, and the image coordinate range information may be fixed installed in the vehicle for collecting the above-mentioned vehicle image. After that, it is obtained by dividing the seats of the collected in-vehicle images and calculating the boundary coordinates corresponding to each of the seat areas. Based on the corresponding coordinate range information of the first seating area, each of the first seating areas may be divided from the vehicle image.

For step S103,

The detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame. In a possible implementation, when determining the detection frame of each occupant in the vehicle image In the process of frame detection, the vehicle image may be input into a pre-trained detection model to determine a detection frame corresponding to each occupant in the vehicle image.

In a possible implementation manner, the detection model used to detect the face detection frame may be a first model, the detection model used to detect the human body detection frame may be a second model, and the first model and the second The supervised data of the models may be different, the supervised data of the first model may be pre-marked human face frames, and the supervised data of the second model may be pre-marked human body frames.

Or, in another possible implementation, the detection model used to detect the face detection frame and the detection model used to detect the human body detection frame may be the same detection model, and the supervision data of the detection model includes two types, namely, pre- Annotated face frames and pre-annotated body frames.

In practical applications, step S102 and step S103 can be executed synchronously based on the order shown in FIG. 1 ; or step S102 and step S103 can be executed sequentially. This solution does not impose too many restrictions on the distinction between the above two steps.

For step S104,

The first matching degree between the above-mentioned detection frame and each of the first seating areas indicates the correlation of spatial positions, and may be based on the overlapping area, degree of overlap, etc. between the detection frame and each of the first seating areas inferred.

In a possible implementation, the detection frame of each occupant can be matched with each of the first seating areas. For example, the overlapping area between the detection frame and the first seating area can be calculated to determine the Among the plurality of seat areas within the detection frame, the seat area with the largest overlapping area with the detection frame is the seat corresponding to the detection frame, so as to determine the seat occupied by the occupant in the detection frame.

In a possible implementation manner, the above-mentioned step of determining the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas can be realized through steps S301 to S303 :

Step S301, traversing each of the first seating areas according to a preset order;

In a possible implementation manner, in the process of traversing each of the first seating areas according to a preset order, the traversal may be performed from the driver's seat first from left to right, and then from top to bottom. Exemplarily, as shown in FIG. 2, there are 4 first seating areas in FIG. The order of seat A 203 and passenger B seat 204 is traversed in sequence.

In another possible implementation manner, starting from the driver's seat, the traversal may be performed from top to bottom, and then from left to right. Exemplarily, as shown in Figure 2, there are 4 first seating areas in Figure 2, during the process of traversing each of the first seating areas, the driver's seat 201, passenger A seat 203, co-pilot The order of the passenger seat 202 and the passenger B seat 204 is traversed in sequence.

For each first seating area traversed to, perform step S302 and step S303:

Step S302, taking the degree of overlap between the detection frame and the first seat area as the first matching degree between the detection frame and the first seat area;

The degree of overlap may be, for example, an intersection-over-union (IoU), that is, a ratio of an intersection and a union between the detection frame and the first seating area. Exemplarily, as shown in FIG. 4, the first seat area in FIG. 4 is the driver's seat area 401, wherein the middle gray highlighted part is the area 403 jointly contained by the driver's seat area 401 and the detection frame 402, that is, and set area; the synthesis of the driver's seat area and all areas of the detection frame is the intersection area, and the ratio of the intersection area and the union area is obtained to obtain the overlap between the driver's seat area and the detection frame, and this overlap is used as the The first matching degree between the detection frame and the driver's seat area.

Step S303, when the first matching degree reaches the first threshold and the seat corresponding to the first seating area is not bound to other detection frames, bind the seat corresponding to the first seating area to the A frame is detected, and the seat corresponding to the detection frame is determined as the seat corresponding to the first seat area.

During the process of traversing the first seating area according to the preset order, it may be detected whether the first seating area is bound to other detection frames. A first seat area is bound to a certain detection frame, indicating that the first seat area is the seating position of the occupant corresponding to the detection frame. Usually, the first seat area cannot accommodate multiple people, so in the first matching When the degree reaches the first threshold, it is detected whether the first seat area is bound to other detection frames, which can avoid the unreasonable situation where multiple detection frames are bound to one seat area and multiple people sit on the same seat. Test results.

In a possible implementation manner, in the first seat area, the first threshold of the face detection frame and the first threshold corresponding to the human body detection frame may be different. Since there may be differences in distances and relative angles between multiple different seating areas and the cameras that collect vehicle images, the face detection frame of each seating area can correspond to a first threshold, and the human body detection frame of each seating area can be respectively corresponds to a first threshold.

In a possible implementation manner, for the first seat area currently traversed to, the first matching degree between a detection frame and the first seat area reaches the corresponding first threshold, and the seat corresponding to the first seat area If no other detection frame is bound, the seat corresponding to the first seat area is used as the seat corresponding to the detection frame, and the first seat area is bound to the first seat corresponding to its first matching degree. Threshold detection box.

In a possible implementation manner, the first matching degree calculated between the face detection frame of a certain occupant and the first seat area reaches the corresponding first threshold, but the human body detection frame of the occupant and the first seat area If the first matching degree of the area does not reach the corresponding first threshold, bind the first seat area with the face detection frame of the occupant, and determine that the seat corresponding to the face detection frame is the first seat The seat corresponding to the area, and then the face of the occupant is bound to the seat corresponding to the first seating area.

Conversely, when the human body detection frame of a certain occupant and the first matching degree calculated in the first seat area reach the corresponding first threshold, but the occupant's face detection frame and the first matching degree calculated in the first seating area If the corresponding first threshold is not reached, bind the first seat area to the human body detection frame of the occupant, determine that the seat corresponding to the human body detection frame is the seat corresponding to the first seat area, and then The occupant's human body is bound to the seat corresponding to the first seat area.

In practical applications, a double matching method can be adopted, and the seat area matching is performed on the human body detection frame and the face detection frame respectively. When the first matching degree of the area reaches the first threshold, bind the occupant's face detection frame and human body detection frame with the first seat area, and then bind the occupant with the first seat area , so as to improve the binding efficiency.

In another possible implementation, the seat area matching can be performed by only using the human body detection frame or the face detection frame, wherein, in the scheme of only using the face detection frame for detection, the face detection frame When the first matching degree with the first seat area reaches the corresponding first threshold, and the seat corresponding to the first seat area is not bound to other detection frames, bind the face detection frame to the first seat area Determine, determine that the seat corresponding to the face detection frame is the seat corresponding to the first seat area, and then bind the occupant's face with the first seat area; when the first matching degree does not exceed the corresponding matching threshold In the case of , the occupant's face is not bound to the first seat area. The method of detection using the human body detection frame is similar to the method of detection using the face detection frame.

For step S105,

The area range of the second seat area is larger than that of the first seat area. In a possible implementation manner, the second seat area is a seat area including a part of the vehicle gap, and each of the second seat areas Corresponds to a row of seats or a row of seats.

In a possible implementation manner, the second seating area may be determined based on the determined first seating area. Wherein, based on the determined first seating area and the vehicle image, some gaps in the first seating area can be supplemented, and each row or column can be determined as a second seating area; and then the second seating area can be divided according to the vehicle model information A plurality of areas within, where each second seating area corresponds to a row of seats, each second seating area can be further divided exemplarily as follows: In the case of two seats in each row of the vehicle , can be divided at 1/2, divided into the left column area and the right column area; in the case of three seats in each row of the vehicle, it can be divided at 1/3, and so on.

In another possible implementation, the second seating area can be determined directly based on the vehicle image. Among them, in the scheme of dividing according to rows, the gap interval of each row in the vehicle image can be directly removed, and each row in the vehicle is determined as a second seat area, and then the left column area and the second seat area in the second seat area can be divided. The right column area; similarly, in the scheme of dividing by column, the gap interval of each column in the vehicle image can be directly removed, and each column in the vehicle is determined as a second seat area, and then divided into the second seat area Areas listed above and areas below

Exemplarily, as shown in Figure 5(a), what Figure 5(a) shows is a schematic diagram of the second seat area of a 4-seater vehicle divided by row, wherein, Figure 5(a) contains two The second seating area is respectively the second seating area A501 and the second seating area B502; each row includes a left column area and a right column area, and each left column area and each right column area correspond to a seat respectively.

Or, exemplary, as shown in Figure 5(b), what Figure 5(b) shows is a schematic diagram of the second seat area of a 4-seater vehicle divided by columns, wherein, in Figure 5(b) Contains two second seating areas, respectively the second seating area C503 and the second seating area D504; each row includes a front row area and a rear row area, and each front row area and each rear row area correspond to a seat .

For step S106,

In a possible implementation manner, for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, the above-mentioned detection frame based on the target occupant and each of the second The second matching degree of the seat area, the step of determining the target second seat area corresponding to the detection frame of the target occupant, as shown in Figure 6, can be realized through steps S601 to S603, wherein:

Step S601, traversing the second seating areas in a preset order, for each of the second seating areas traversed;

In a possible implementation manner, in the process of traversing the second seat area according to the preset order, if there are multiple rows or rows of seats, the second seat area where the driver's seat is located may be selected according to the actual situation. Region starts and traverses row by row or column by column.

For each second seating area traversed to, the following steps S602 and S603 are performed:

Step S602, taking the degree of overlap between the detection frame of the target occupant and the second seat area as the second matching degree between the detection frame of the target occupant and the second seat area;

Step S603 , when the second matching degree reaches a second threshold, determine that the second seating area is a target second seating area corresponding to the detection frame of the target occupant.

Exemplarily, as shown in Fig. 7(a), the degree of overlap between the detection frame 701 of the target occupant and the second seat area E702, and the detection frame 701 of the target occupant and the second seat area F703 is calculated as the corresponding second matching degree, If the second matching degree corresponding to the detection frame of the target occupant and the second seat area E reaches a second threshold, the second seat area E is determined to be the target second seat area corresponding to the detection frame of the target occupant.

In a possible implementation manner, for each second seat area traversed to, if the degree of overlap does not reach the second threshold, it may be determined that the overlapping area with the detection frame of the target occupant is the largest The second seating area of is the target second seating area.

Exemplarily, as shown in FIG. 7(b), the overlapping area of the detection frame 701 of the target occupant and the second seat area G704 is larger than the overlapping area of the detection frame 701 of the target occupant and the second seat area H705, and the The second seat area G704 is determined as the target second seat area corresponding to the detection frame of the target occupant.

For step S107,

In a possible implementation manner, the target occupant is determined according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area, and the detection frame of the target occupant. The steps of the seat corresponding to the detection frame can be as shown in Figure 8, including step S801 to step S802, wherein:

Step S801, based on the detection frame of the target occupant and the position range of each seat in the target second seat area, determine that the seat corresponding to the position range where the center point of the detection frame of the target occupant falls is the The candidate seat corresponding to the detection frame of the target occupant;

When determining the position range within which the center point of the detection frame of the target occupant falls, in a possible implementation manner, the center point between the detection frame of the target occupant and the position range of each seat may be calculated For example, as shown in Figure 9, the Euclidean distance between the detection frame 901 of the target occupant and the center point 902 between the left column area and the center point 903 between the detection frame 901 of the target occupant and the right column area are calculated. Euclidean distance, that is, the straight-line distance between the detection frame of the target occupant and the two center points.

Then, the seat corresponding to the position range with the shortest straight line distance is determined as the candidate seat corresponding to the detection frame of the target occupant. As shown in Figure 9, the center point of the detection frame 901 of the target occupant to be bound is shorter than the distance from the left column area to the right column area, and the seat corresponding to the left column area is determined as the target occupant. The candidate seat corresponding to the detection box.

In another possible implementation manner, the intersection between the detection frame of the target occupant and each seat in the target second seating area may be calculated respectively, and the larger intersection is taken as the detection frame of the target occupant corresponding candidate seats.

Exemplarily, as shown in Figure 9, the intersection of the detection frame of the target occupant to be bound and the left column area is larger than that of the right column area, then it is determined that the seat corresponding to the left column area is the detection of the target occupant The candidate seats corresponding to the boxes.

Step S802, if the candidate seat is not bound to other detection frames, determine that the candidate seat is the seat corresponding to the detection frame of the target occupant, and bind the seat corresponding to the candidate seat to the The detection box of the target occupant.

In a possible implementation manner, if the target second seat area corresponding to the detection frame of the target occupant is not determined, the corresponding first seat area is closest to the detection frame of the target occupant and The seat not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.

In a possible implementation manner, the detection frame of the target occupant and the center point of each of the first seating areas may be respectively determined first; The seats of other detection frames are determined as the seats corresponding to the detection frame of the target occupant.

Exemplarily, as shown in Fig. 10 , wherein Fig. 10 includes a plurality of second seating areas, namely the first seating area A1001, the first seating area B1002, the first seating area C1003 and the first seating area D1004, the target occupant's The detection frame 1005 does not intersect with the above-mentioned four first seating areas. Since the center point of the detection frame of the target occupant has the shortest straight-line distance from the center point of the first seat area A, and no other detection frames are bound, the first seat area A1001 is determined as the detection frame of the target occupant The seat corresponding to 1005.

Based on the method for determining the seat of the occupant in the vehicle proposed by this solution, an embodiment of the disclosure also provides a vehicle control method, which is executed by an electronic device, as shown in FIG. 11 , which is a method provided by an embodiment of the disclosure. Schematic diagram of the vehicle control method, including the following steps:

Step S1101, based on the method for determining the seat of the occupant in the vehicle described in the above embodiment, determine the detection frame corresponding to each of the seat areas;

Step S1102: Detect the target action made by the target occupant within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.

In practical applications, based on the method for determining the occupant's seat in the vehicle proposed in the above embodiments, the control of the vehicle can be realized, wherein, in response to the target instruction corresponding to the occupant's target action, the vehicle executes a corresponding target operation.

Exemplarily, the vehicle control method can be applied in the field of intelligent driving, when it is detected that the target detection frame corresponding to the target occupant is bound to the target seat area. When the passenger makes a gesture indicating "open the window", the vehicle opens the window above the corresponding door according to the gesture instruction of the passenger and the position where the passenger is sitting. For example, if the passenger who issued the gesture instruction is a passenger in the passenger seat, the window above the door on the passenger side can be opened, thereby realizing targeted control based on passenger seat detection.

Those skilled in the art can understand that in the above-mentioned method of specific implementation, the writing order of each step does not imply a strict execution order and constitutes any limitation on the implementation process, and the execution order of each step should be based on its function and possible internal Logically OK.

Based on the same inventive concept, the embodiment of the present disclosure also provides a device for determining the seat of the occupant in the vehicle corresponding to the method for determining the seat of the occupant in the vehicle. The above method for determining the seat of the occupant in the vehicle is similar, so the implementation of the device can refer to the implementation of the method.

Referring to FIG. 12 , it is a schematic structural diagram of a device for determining the seat of an occupant in a vehicle provided by an embodiment of the present disclosure. The device includes: an acquisition part 1201, a first determination part 1202, a second determination part 1203, a third Determination part 1204, fourth determination part 1205, fifth determination part 1206, sixth determination part 1207; wherein,

An acquisition part 1201 configured to acquire a vehicle image in the vehicle;

The first determining part 1202 is configured to determine a plurality of first seat areas in the vehicle image, and each first seat area corresponds to a seat;

The second determining part 1203 is configured to detect occupants in the vehicle based on the vehicle image, and determine a detection frame of each occupant in the vehicle image;

The third determining part 1204 is configured to, for each occupant's detection frame, determine the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas;

The fourth determination part 1205 is configured to determine a plurality of second seating areas in the vehicle image, each second seating area corresponds to a row of seats or a row of seats;

The fifth determining part 1206 is configured to, for the detection frame of the target occupant whose corresponding target first seat area is not determined based on the first matching degree, based on the detection frame of the target occupant and each of the second seat The second matching degree of the area, determining the target second seat area corresponding to the detection frame of the target occupant;

The sixth determining part 1207 is configured to determine the detection of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant box corresponding to the seat.

In a possible implementation manner, the first determining part 1202 is further configured to: determine the first seat area corresponding to each seat in the vehicle image according to the seat distribution configuration information of the vehicle.

In a possible implementation manner, the third determining part 1204 is further configured to: traverse each of the first seating areas in a preset order, and for each first seating area traversed to: calculate the The degree of overlap between the detection frame and the first seat area is used as a first matching degree between the detection frame and the first seat area; when the first matching degree reaches a first threshold and the first seat If the seat corresponding to the area is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame, and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area seat.

In a possible implementation manner, the detection frame of each occupant in the vehicle image is at least one of the following: a face detection frame and a human body detection frame.

In a possible implementation manner, the fifth determining part 1206 is further configured to: for each target occupant's detection frame, traverse the second seat area in a preset order, and for each Second seat area: calculate the degree of overlap between the frame of the target occupant and the second seat area as the second matching degree; when the second matching degree reaches a second threshold, determine the The second seat area is a target second seat area corresponding to the detection frame of the target occupant.

In a possible implementation manner, the sixth determining part 1207 is further configured to: determine the target occupant based on the detection frame of the target occupant and the position range of each seat in the target second seating area. The seat corresponding to the position range where the center point of the detection frame falls into is the candidate seat corresponding to the detection frame of the target occupant; if the candidate seat is not bound to other detection frames, determine the candidate seat is the seat corresponding to the detection frame of the target occupant, and the seat corresponding to the candidate seat is bound to the detection frame of the target occupant.

In a possible implementation manner, the fifth determining part 1206 is further configured to: for each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine The second seat area with the largest overlapping area of the detection frames of the target occupant is the target second seat area.

In a possible implementation manner, the seventh determination part 1208 is further configured to: if the target second seat area corresponding to the detection frame of the target occupant is not determined, set the corresponding first seat area The seat whose area is closest to the detection frame of the target occupant and which is not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.

For the description of the processing flow of each part in the device and the interaction flow between each part, reference may be made to the relevant descriptions in the foregoing method embodiments.

Based on the same inventive concept, the embodiment of the present disclosure also provides a vehicle control device corresponding to the vehicle control method, because the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned method of determining the seat of the occupant in the vehicle in the embodiment of the present disclosure , so the implementation of the device can refer to the implementation of the method.

Referring to FIG. 13 , which is a schematic structural diagram of a vehicle control device provided by an embodiment of the present disclosure, the device includes: an eighth determination part 1301 and a detection part 1302; wherein,

The eighth determining part 1301 is configured to determine the detection frame corresponding to each seat area based on the method for determining the seat of the occupant in the vehicle described in the above embodiment;

The detection part 1302 is configured to detect a target action made by a target occupant within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.

For a description of the processing flow of each part in the device and the interaction flow between each of the parts, reference may be made to the relevant descriptions in the foregoing method embodiments.

In the embodiments of the present disclosure and other embodiments, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course it may also be a unit, a module or a non-modular one.

Based on the same technical idea, the embodiment of the present disclosure also provides a computer device. Referring to FIG. 14 , it is a schematic structural diagram of a computer device 1400 provided by an embodiment of the present disclosure, including a processor 1401 , a memory 1402 , and a bus 1403 . Among them, the memory 1402 is used to store execution instructions, including the memory 14021 and the external memory 14022; the memory 14021 here is also called the internal memory, and is used to temporarily store the operation data in the processor 1401 and the data exchanged with the external memory 14022 such as the hard disk, The processor 1401 exchanges data with the external memory 14022 through the memory 14021. When the computer device 1400 is running, the processor 1401 communicates with the memory 1402 through the bus 1403, so that the processor 1401 executes the following instructions:

obtain an image of the vehicle inside the vehicle;

determining a plurality of first seat areas in the vehicle image, each first seat area corresponding to a seat;

detecting occupants in the vehicle based on the vehicle image, and determining a detection frame for each occupant in the vehicle image;

For each occupant's detection frame, based on the first matching degree between the detection frame and each of the first seat areas, determine the seat corresponding to the detection frame;

determining a plurality of second seating areas in the vehicle image, each second seating area corresponding to a row of seats or a row of seats;

For the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching degree between the detection frame of the target occupant and each of the second seating areas, determine the The target second seat area corresponding to the detection frame of the target occupant;

The seat corresponding to the detection frame of the target occupant is determined according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant.

Alternatively, the processor 1401 may also execute the following instructions:

Based on the method for determining the seat of the occupant in the vehicle described in the above embodiments, determine the detection frame corresponding to each of the seat areas;

A target action made by a target occupant within the target detection frame is detected, and vehicle control is performed based on the target action and the seat area corresponding to the target detection frame.

An embodiment of the present disclosure also provides a computer-readable storage medium, any computer-readable storage medium stores a computer program, and any computer program executes the determination described in the above-mentioned method embodiments when run by a processor A method of seating an occupant in a vehicle, steps of a vehicle control method. Wherein, any one of the storage media may be a volatile or non-volatile computer-readable storage medium.

It should be pointed out here that: the descriptions of the above storage medium and device embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to those of the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

The above-mentioned processor can be an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic At least one of Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor. Understandably, the electronic device that implements the above processor function may also be other, which is not specifically limited in this embodiment of the present disclosure.

The above-mentioned computer storage medium/memory can be read-only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), Magnetic Random Access Memory (Ferromagnetic Random Access Memory, FRAM), Flash Memory (Flash Memory), Magnetic Surface Memory, CD-ROM, or CD-ROM (Compact Disc Read-Only Memory, CD-ROM) and other memories; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. .

An embodiment of the present disclosure also provides a computer program product, any of the computer products carries a program code, and the instructions included in the program code can be used to execute the method for determining the seat of an occupant in a vehicle described in the method embodiment above, For the steps of the vehicle control method, reference may be made to the foregoing method embodiments.

Wherein, the above-mentioned computer program product may be implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Those skilled in the art can clearly understand that for the convenience and brevity of description, for the working process of the above-described system and device, reference may be made to the corresponding process in the foregoing method embodiments. Among the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of any technical solution can be embodied in the form of a software product, and any computer software product is stored in A storage medium includes several instructions for enabling a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than limit them, and the protection scope of the present disclosure is not limited thereto, although referring to the aforementioned The embodiments have described the present disclosure in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in this disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be defined by the protection scope of the claims.

Industrial Applicability

An embodiment of the present disclosure provides a method and device for determining the seat of an occupant in a vehicle and controlling the vehicle, including: acquiring a vehicle image in the vehicle; determining a plurality of first seat areas in the vehicle image; detecting an occupant in the vehicle based on the vehicle image, Determining the detection frame of each occupant in the vehicle image; for the detection frame of each occupant, based on the first matching degree between the detection frame and each of the first seat areas, determining the seat corresponding to the detection frame; determining the vehicle image a plurality of second seating areas; for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching between the detection frame of the target occupant and each of the second seating areas Determine the target second seat area corresponding to the detection frame of the target occupant; determine the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant The detection box corresponds to the seat. In the case that the detection frame of the target occupant of the seat cannot be determined through the first seat area, the area range of the first seat area is expanded, and based on the expanded area range of the second seat area, the seat corresponding to the detection frame of the target occupant is determined , thereby reducing the probability of occupant and seat matching or binding failure, and improving the success rate of occupant seat matching.

Claims (21)

1. A method of determining a seat for an occupant in a vehicle, the method performed by an electronic device, the method comprising:

   obtain an image of the vehicle inside the vehicle;

   determining a plurality of first seat areas in the vehicle image, each of the first seat areas corresponds to a seat;

   detecting occupants in the vehicle based on the vehicle image, and determining a detection frame for each occupant in the vehicle image;

   For each occupant's detection frame, based on the first matching degree between the detection frame and each of the first seat areas, determine the seat corresponding to the detection frame;

   determining a plurality of second seating areas in the vehicle image, each of the second seating areas corresponds to a row of seats or a row of seats;

   For the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the second matching degree between the detection frame of the target occupant and each of the second seating areas, determine the The target second seat area corresponding to the detection frame of the target occupant;

   According to the position range of each seat in a row of seats or a row of seats corresponding to the target second seat area, and the detection frame of the target occupant, the seat corresponding to the detection frame of the target occupant is determined.
2. The method according to claim 1, wherein said determining a plurality of first seat areas in said vehicle image comprises:

   A first seat area corresponding to each seat in the vehicle image is determined according to the seat distribution configuration information of the vehicle.
3. The method according to claim 1 or 2, wherein the determining the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas comprises:

   Traversing each of the first seating areas in a preset order, for each of the first seating areas traversed to:

   Using the degree of overlap between the detection frame and the first seat area as the first matching degree between the detection frame and the first seat area;

   When the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to other detection frames, bind the seat corresponding to the first seat area to the detection frame, And determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area.
4. The method according to any one of claims 1 to 3, wherein the detection frame of each occupant in the vehicle image is at least one of the following: face detection frame, human body detection frame.
5. The method according to any one of claims 1 to 4, wherein, based on the second degree of matching between the detection frame of the target occupant and each of the second seating areas, determining the detection frame of the target occupant The corresponding target second seating areas include:

   Traversing each of the second seating areas in a preset order, for each of the second seating areas traversed to:

   using the degree of overlap between the detection frame of the target occupant and the second seat area as the second matching degree between the detection frame of the target occupant and the second seat area;

   In a case where the second matching degree reaches a second threshold, the second seating area is determined to be a target second seating area corresponding to the detection frame of the target occupant.
6. The method according to any one of claims 1 to 5, wherein the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area, and the target occupant's The detection frame is used to determine the seat corresponding to the detection frame of the target occupant, including:

   Based on the detection frame of the target occupant and the position range of each seat in the target second seat area, determine that the seat corresponding to the position range where the center point of the detection frame of the target occupant falls is the target occupant The candidate seat corresponding to the detection frame of ;

   If the candidate seat is not bound to other detection frames, determine that the candidate seat is the seat corresponding to the detection frame of the target occupant, and bind the seat corresponding to the candidate seat to the target occupant detection frame.
7. The method according to claim 5, wherein, based on the second matching degree between the detection frame of the target occupant and each of the second seating areas, the target second seat corresponding to the detection frame of the target occupant is determined. Seating area, also includes:

   For each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine that the second seat area with the largest overlapping area with the detection frame of the target occupant is Target the second seating area.
8. The method according to any one of claims 1 to 7, wherein the method further comprises:

   If the target second seat area corresponding to the detection frame of the target occupant is not determined, determine the seat corresponding to the first seat area closest to the detection frame of the target occupant and not bound to other detection frames is the seat corresponding to the detection frame of the target occupant.
9. A method for controlling a vehicle, the method being executed by an electronic device, the method comprising:

   Based on the method for determining the seat of an occupant in a vehicle according to any one of claims 1 to 8, determining a detection frame corresponding to each of the seat areas;

   A target action made by a target occupant within the target detection frame is detected, and vehicle control is performed based on the target action and the seat area corresponding to the target detection frame.
10. An apparatus for determining the seating of an occupant in a vehicle, comprising:

    an acquisition part configured to acquire a vehicle image within the vehicle;

    A first determination part configured to determine a plurality of first seat areas in the vehicle image, each of the first seat areas corresponds to a seat;

    a second determining section configured to detect occupants in the vehicle based on the vehicle image, and determine a detection frame for each of the occupants in the vehicle image;

    The third determining part is configured to, for each occupant's detection frame, determine the seat corresponding to the detection frame based on the first matching degree between the detection frame and each of the first seat areas;

    A fourth determining part configured to determine a plurality of second seating areas in the vehicle image, each of the second seating areas corresponds to a row of seats or a row of seats;

    The fifth determining part is configured to, for the detection frame of the target occupant whose corresponding target first seating area is not determined based on the first matching degree, based on the detection frame of the target occupant and each of the second seating areas The second matching degree, determining the target second seat area corresponding to the detection frame of the target occupant;

    The sixth determining part is configured to determine the detection of the target occupant according to the position range of each seat in a row of seats or a row of seats corresponding to the target second seating area and the detection frame of the target occupant. box corresponding to the seat.
11. The device according to claim 10, wherein the first determination part is further configured to: determine the first seat area corresponding to each seat in the vehicle image according to the seat distribution configuration information of the vehicle.
12. The device according to claim 10 or 11, wherein the third determining part is further configured to: traverse each of the first seating areas in a preset order, and for each first seating area traversed to: calculating the degree of overlap between the detection frame and the first seat area as a first matching degree between the detection frame and the first seat area; when the first matching degree reaches a first threshold and the If the seat corresponding to the first seat area is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame, and determine that the seat corresponding to the detection frame is the first seat Seats corresponding to the region.
13. The device according to any one of claims 10 to 12, wherein the detection frame of each occupant in the vehicle image is at least one of the following: face detection frame, human body detection frame.
14. The device according to any one of claims 10 to 13, wherein the fifth determining part is further configured to: for each target occupant's detection frame, traverse the second seat area in a preset order, for Traversing to each second seat area: calculating the degree of overlap between the frame of the target occupant and the second seat area as the second matching degree; when the second matching degree reaches a second threshold In some cases, the second seating area is determined to be the target second seating area corresponding to the detection frame of the target occupant.
15. The device according to any one of claims 10 to 14, wherein the sixth determining part is further configured to: based on the detection frame of the target occupant and the position range of each seat in the target second seating area , determine that the seat corresponding to the position range where the center point of the detection frame of the target occupant falls is the candidate seat corresponding to the detection frame of the target occupant; in the case that the candidate seat is not bound to other detection frames , determining that the candidate seat is the seat corresponding to the detection frame of the target occupant, and binding the seat corresponding to the candidate seat to the detection frame of the target occupant.
16. The device according to claim 14, wherein the fifth determining part is further configured to: for each second seat area traversed to, if the degree of overlap does not reach the second threshold, determine The second seat area with the largest overlapping area with the detection frame of the target occupant is the target second seat area.
17. The device according to any one of claims 10 to 16, wherein the seventh determination part is further configured to: if the target second seat area corresponding to the detection frame of the target occupant is not determined, The seat corresponding to the first seat area closest to the detection frame of the target occupant and not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.
18. A vehicle control device, comprising:

    The eighth determination part is configured to determine a detection frame corresponding to each of the seat areas based on the method for determining the seat of an occupant in a vehicle according to any one of claims 1 to 8;

    The detection part is configured to detect a target action made by a target occupant within the target detection frame, and perform vehicle control based on the target action and the seat area corresponding to the target detection frame.
19. A computer device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory through the bus , when the machine-readable instructions are executed by the processor, the steps of the method for determining the seat of an occupant in a vehicle according to any one of claims 1 to 8 are executed, or the vehicle control method according to claim 9 is executed A step of.
20. A computer-readable storage medium, the computer-readable storage medium is stored with a computer program, and when the computer program is executed by a processor, the method of determining the seat of an occupant in a vehicle according to any one of claims 1 to 8 is executed. method steps, or execute the steps of the vehicle control method as claimed in claim 9 .
21. A computer program product, comprising computer readable code, when the computer readable code is run in an electronic device, the processor in the electronic device executes the determination as described in any one of claims 1 to 8 The steps of the method for the seat of the occupant in the vehicle, or the steps of performing the method of controlling the vehicle as claimed in claim 9 .

Images