CN118547959A

CN118547959A - Door opening control method and device for vehicle, vehicle and computer readable storage medium

Info

Publication number: CN118547959A
Application number: CN202411009251.6A
Authority: CN
Inventors: 杨武; 钟道上; 程晓鹏; 孙小庆
Original assignee: BYD Co Ltd
Current assignee: BYD Co Ltd
Priority date: 2024-07-26
Filing date: 2024-07-26
Publication date: 2024-08-27
Anticipated expiration: 2044-07-26

Abstract

The application discloses a door opening control method and device of a vehicle, the vehicle and a computer readable storage medium, wherein a door opening mode is determined according to biological characteristic information acquired by a door opening and closing assembly; and under the condition that the door opening mode is a lotus type door opening mode, controlling the vehicle to execute preset first feedback. According to the embodiment of the application, the biological characteristic information of the user is acquired through the vehicle door opening and closing assembly, and whether the door opening mode of the user is the lotus-type door opening mode is judged according to the biological characteristic information, so that whether the user can pay attention to the vehicle condition on the road section can be determined, and then the user is given a corresponding feedback prompt, so that the accident occurrence can be reduced.

Description

Door opening control method and device for vehicle, vehicle and computer readable storage medium

Technical Field

The present application relates to the field of electronics technologies, and in particular, to a door opening control method and apparatus for a vehicle, and a computer readable storage medium.

Background

The vehicle brings traffic convenience to people and also brings traffic accidents and potential safety hazards, for example, the automobile door opened by the automobile collides with the rear vehicle, so that the rear vehicle falls to the ground in an unbalanced manner, and the cyclist is injured. The automobile driver and front passengers can see the rear view field through the rearview mirror before opening the automobile door, but the view field is limited and has visual dead angles, so that accidents are difficult to avoid. The passengers at the rear seats cannot observe the situation behind the vehicle through the rearview mirror, so that accidents are more likely to occur.

Disclosure of Invention

The embodiment of the application provides a door opening control method and device for a vehicle, the vehicle and a computer readable storage medium, which can give corresponding feedback prompt to a user according to a door opening mode of the user and can reduce accidents.

In order to achieve the above object, according to a first aspect of the present application, there is provided a door opening control method of a vehicle, comprising:

determining a door opening mode according to biological characteristic information acquired by the door opening and closing assembly;

and under the condition that the door opening mode is a lotus type door opening mode, controlling the vehicle to execute preset first feedback.

Optionally, the method further comprises:

And under the condition that the door opening mode is the non-load door opening mode, controlling the vehicle to execute a preset second feedback, wherein the preset second feedback is different from the preset first feedback.

Optionally, the biometric information includes a palmprint image, and determining the door opening mode according to the biometric information collected by the door opening and closing assembly includes:

acquiring the palmprint image of a target palm when the vehicle door opening and closing assembly is triggered;

Performing palm print recognition processing based on the palm print image to obtain a palm type of the target palm, wherein the palm type is used for indicating whether the target palm is a left palm or a right palm;

And determining the door opening mode according to the palm type.

Optionally, the palm print recognition processing based on the palm print image obtains a palm prediction result of the target palm, including:

performing global palm print recognition processing based on the palm print image to obtain a first palm type of the target palm;

And/or performing local palm print recognition processing based on the palm print image to obtain a second palm prediction result of the target palm;

and determining the palm type of the target palm based on the first palm prediction result and/or the second palm prediction result.

Optionally, the first palm prediction result includes a first probability that the target palm is a left palm and a second probability that the target palm is a right palm, the second palm prediction result includes a third probability that the target palm is a left palm and a fourth probability that the target palm is a right palm, and determining, based on the first palm prediction result and/or the second palm prediction result, a palm type of the target palm includes:

carrying out weighted summation on the first probability and the third probability to obtain left palm probability that the target palm is a left palm;

carrying out weighted summation on the second probability and the fourth probability to obtain right palm probability that the target palm is a right palm;

and determining the palm type of the target palm according to the left palm probability and the right palm probability.

Optionally, the performing a first palm print recognition process based on the palm print image to obtain the first palm prediction result of the target palm includes:

Performing palm print restoration processing on the palm print image to obtain a restored palm print image;

And carrying out palm type prediction processing based on the repaired palmprint image to obtain the first palm prediction result of the target palm.

Optionally, the palm print repairing process is performed on the palm print image to obtain a repaired palm print image, which includes:

Extracting a mask image of the palm print image, wherein the mask image is used for indicating positions of an effective palm print area and an ineffective palm print area in the palm print image;

and carrying out palm print restoration processing on the palm print image based on the mask image to obtain a restored palm print image.

Optionally, the performing palm print repairing processing on the palm print image based on the mask map to obtain a repaired palm print image includes:

extracting features of the palm print image to obtain image features of the palm print image;

Masking a feature part corresponding to an invalid region in the mask image in the image features to obtain first image features;

Masking local features corresponding to the effective areas in the mask image in the image features to obtain second image features;

and repairing the palmprint image based on the first image feature and the second image feature to obtain the repaired palmprint image.

Optionally, the repairing the palmprint image based on the first image feature and the second image feature to obtain the repaired palmprint image includes:

extracting the feature blocks of the first image features to obtain first feature blocks, and extracting the feature blocks of the second image features to obtain second feature blocks;

Repairing each second characteristic block based on the first characteristic block to obtain a third characteristic block;

and generating the repaired palmprint image based on the first feature block and the third feature block.

Optionally, repairing each of the second feature blocks based on the first feature block to obtain a third feature block, including:

Masking each first feature block according to the invalid value in each second feature block to obtain a masked first feature block corresponding to each feature block;

calculating the similarity between each second characteristic block and each first characteristic block after shielding;

carrying out weighted summation processing on the first characteristic block according to the similarity to obtain a weighted characteristic block;

And updating the second characteristic block based on the weighted characteristic block to obtain a third characteristic block.

Optionally, the performing local palm print recognition processing based on the palm print image to obtain a second palm prediction result of the target palm includes:

identifying at least one effective frame containing palmprint from the palmprint image through a region extraction model;

Carrying out palm type prediction processing on the image area corresponding to each effective frame to obtain a prediction result corresponding to each local palm print image block;

And determining a second palm prediction result of the palm print image based on the prediction result corresponding to the effective frame.

Optionally, the performing palm type prediction processing on the image area corresponding to each effective frame to obtain a prediction result corresponding to each local palm print image block includes:

Extracting a plurality of prediction frames from the palm print image, and determining at least one effective frame containing palm prints from the prediction frames based on the local palm print image in the prediction frames;

Extracting an image area corresponding to the effective frame from the palm print image, and generating the local palm print image block based on the image area;

And carrying out palm type prediction processing on each local palm print image block to obtain the prediction result corresponding to each local palm print image block.

Optionally, when the door opening mode is a load door opening mode, controlling the vehicle to execute a preset first feedback includes:

and under the condition that the door opening mode is a lotus type door opening mode and the pressure information collected by the switch assembly meets the preset pressure condition, controlling the vehicle to execute the preset first feedback.

Optionally, the controlling the vehicle to execute the preset second feedback when the door opening mode is the off-load door opening mode includes:

and under the condition that the door opening mode is an unloading door opening mode and the pressure information collected by the switch assembly meets the preset pressure condition, controlling the vehicle to execute the preset second feedback.

Optionally, the preset first feedback includes at least one of a first voice alert, a first vehicle light flashing, and a first whistle, and no feedback; the preset second feedback includes at least one of a second voice alert, a second vehicle light flashing, and a second whistle.

According to a second aspect of the present application, there is provided a door opening control apparatus of a vehicle, comprising:

the determining unit is used for determining a door opening mode according to the biological characteristic information collected by the vehicle door opening and closing assembly;

and the feedback unit is used for controlling the vehicle to execute preset first feedback under the condition that the door opening mode is a lotus-type door opening mode.

According to a third aspect of the present application there is provided a vehicle comprising a memory and a processor; the memory stores a computer program, and the processor is configured to run the computer program in the memory, so as to execute any door opening control method of the vehicle provided by the embodiment of the application.

According to a fourth aspect of the present application, there is provided a computer-readable storage medium for storing a computer program loaded by a processor to perform any one of the door opening control methods of the vehicle provided by the embodiments of the present application.

According to the embodiment of the application, the door opening mode is determined according to the biological characteristic information collected by the door opening and closing assembly; and under the condition that the door opening mode is a lotus type door opening mode, controlling the vehicle to execute preset first feedback. According to the embodiment of the application, whether the door opening mode of the user is the lotus-type door opening mode is judged through the biological characteristic information of the user collected by the door opening and closing assembly, so that whether the user can pay attention to the vehicle condition on the road section can be determined, and then a corresponding feedback prompt is given to the user, so that the accident occurrence can be reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is evident that the drawings in the following description are only some embodiments of the application and that other drawings may be obtained from these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a door opening control method of a vehicle according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a charge door opening and switching assembly provided by an embodiment of the present application;

FIG. 3is a schematic diagram of a model structure of a local recognition model according to an embodiment of the present application;

Fig. 4 is another flowchart of a door opening control method of a vehicle according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a model structure of a palm print recognition model according to an embodiment of the present application;

FIG. 6 is a schematic illustration of palmprint image restoration provided by an embodiment of the present application;

Fig. 7 is a schematic view of a door opening control apparatus of a vehicle according to an embodiment of the present application;

Fig. 8 is a schematic structural view of a vehicle according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The embodiment of the application provides a door opening control method and device of a vehicle, the vehicle and a computer readable storage medium. The door opening control apparatus of the vehicle may be integrated in the vehicle.

The following will describe in detail. The following description of the embodiments is not intended to limit the preferred embodiments.

The present embodiment will be described from the perspective of a door opening control device of a vehicle, which may be integrated in a vehicle in particular.

The embodiment of the application provides a door opening control method for a vehicle, as shown in fig. 1, and the specific flow of the door opening control method for the vehicle may be as follows:

101. And determining a door opening mode according to the biological characteristic information collected by the vehicle door opening and closing assembly.

The biometric information may include, among other things, a user's fingerprint, palm print, etc. may be representative of the physiology of the human body.

The door opening and closing assembly can be used for determining whether a user operates the door opening and closing assembly by the left hand or the right hand according to the biological characteristic information, and further whether the door opening mode of the user accords with the lotus-type door opening mode or not can be determined.

For example, if the biometric information is a fingerprint, the biometric information may be a fingerprint of at least one finger of the thumb, the index finger, the middle finger, the index finger and the tail finger of the user, and whether the finger belongs to the left hand or the right hand is judged according to the fingerprint of the finger, so as to determine whether the door opening mode of the user is the lotus door opening mode.

The biometric information may also be a palm print image from which it is determined whether the user has operated the door switch assembly with the left hand or the right hand, i.e., in one embodiment, step 101 may include steps 1011-1013, as follows:

1011. and when the vehicle door switch assembly is triggered, acquiring a palmprint image of a target palm.

The lotus-type door opening method is also called as a Holland-type door opening method, namely, when the vehicle door is opened, the upper half of the vehicle is always opened by a hand far away from the vehicle door, so that the upper half of the vehicle is also rotated inertly, the head and the shoulders naturally rotate, in the rotating process, eyes firstly observe the condition behind the vehicle through a rearview mirror, the eyes naturally see outwards and backwards after the vehicle turns, and through the action, a user can better see whether a pedestrian or a travelling crane is behind the vehicle, the vision blind area is reduced, and further a plurality of unnecessary accidents are avoided. For a user sitting on the left seat of the vehicle, the corresponding lotus-type door opening mode can be as shown in fig. 2, the door is opened by operating the switch assembly 21 by the right hand, and then the door is opened by pushing the left hand, and during the door opening process, the upper body of the user rotates, so that the surrounding vehicle condition can be observed. For a user sitting on a seat on the right side of the vehicle, the corresponding lotus-type door opening mode is that the user opens the vehicle door through a left-hand operation switch assembly and pushes the vehicle door open by the right hand, and in the door opening process, the upper body of the user rotates, so that surrounding vehicle conditions can be observed.

Therefore, in the embodiment of the application, when the user executes the door opening mode, the palm print image of the user is acquired, and whether the user uses the hand farther from the door to open the switch assembly on the door is judged through the palm print image.

For example, when a user makes a door opening mode, a palm print of a target palm on the switch assembly is acquired through a palm print sensor on the switch assembly, and a palm print image is generated.

The door opening assembly is an assembly arranged on the inner side of the car door and used for controlling the car door to be opened or closed, the car door can be pushed open after the car door is opened through the door opening assembly, a palm print sensor is arranged on the door opening assembly and can be used for collecting palm prints of a user, in an embodiment, the palm print sensor can be a capacitance sensor, the concave-convex lines of a palm can generate capacitance change when the palm print sensor is contacted, and corresponding palm print data can be generated by measuring the capacitance change.

Optionally, a pressure sensor can be further arranged in the switch component on the inner side of the car door, the pressure sensor can collect the pressure of the palm of the user on the door opening component, if the pressure is smaller, the user can consider that the car door is not opened, the indicating hand is just placed on the door opening component, and by combining the palm type of the target palm and the pressure of the target palm on the switch component, whether the user opens the car door or is about to open the car door can be accurately judged, and whether a lotus-type door opening mode is adopted.

The door opening and closing assembly is triggered by detecting an event of a door opening mode, which is an event triggered by a user acting on a door opening mode of the opening and closing assembly, wherein the door opening mode comprises operation of opening a door by operating the door opening assembly, and for different vehicles, the structures of the opening and closing assemblies arranged on the inner side of the door are different, and the operation required for opening the door is different for different opening and closing assemblies.

For example, for a switch assembly 21 as shown in fig. 2, which includes a handle portion and a connecting portion, a user holding and pulling the handle open the door, the corresponding door opening mode may be different for other configurations of door opening assemblies. For the switch assembly 21 shown in fig. 2, palm print sensors may be disposed on the inner side and the outer side of the handle, so that when the user holds the handle, palm print with a larger area on the palm can be collected by the palm print sensors on the inner side and the outer side of the handle. As shown in the switch assembly 21 of fig. 2, a user can open the door by holding and pulling the handle, so the pressure sensor can be provided on the inside of the handle.

The target palm is a palm used by a user to operate the door opening assembly.

1012. And carrying out palm print recognition processing based on the palm print image to obtain the palm type of the target palm, wherein the palm type is used for indicating whether the target palm is a left palm or a right palm.

After the palm print image is obtained through the palm print sensor, palm print recognition processing can be performed based on the palm print image, so that the palm type of the target palm is obtained, and the palm print type is one of a left palm and a right palm. When a user opens the vehicle door, the palm print recognition is adopted to open the vehicle door without limitation, and the user can open the vehicle door by adopting any gesture, so long as the palm is contacted with the switch assembly, the vehicle acquires the palm print image through the switch assembly, compared with the fingerprint recognition, the vehicle does not need to make a specified gesture, so that the user can place the finger on the fingerprint collector, and the embodiment of the application can better carry out the door opening prompt.

Since palm patterns of different palms are distributed differently, palm types corresponding to the palm pattern images can be determined through palm pattern recognition processing, and the palm pattern recognition processing can be performed in various manners, for example, palm pattern recognition processing can be performed through a palm pattern recognition model based on the palm pattern images, so as to obtain palm types of target palms, and the palm types are used for indicating whether the target palms are left palms or right palms. The palm print recognition model may be a neural network model capable of outputting a palm type corresponding to the palm print image based on the palm print image.

The final result may also be determined by combining at least one result of the palm print recognition process, so as to improve accuracy of palm print recognition, that is, in an embodiment, the step of "performing palm print recognition process based on a palm print image to obtain a palm type of a target palm" may include:

performing global palm print recognition processing based on the palm print image to obtain a first palm prediction result of the target palm;

And/or carrying out local palm print recognition processing based on the palm print image to obtain a second palm prediction result of the target palm;

The first palm prediction result comprises a first probability that the target palm obtained by the first palm print processing is a left palm and a second probability that the target palm is a right palm, and the second palm prediction result comprises a third probability that the target palm obtained by the second palm print processing is a left palm and a fourth probability that the target palm is a right palm.

The average probability of the first probability in the first palm prediction result and the third probability in the second palm prediction result can be calculated, and the average probability that the target palm is the left palm is obtained; the average probability of the second probability in the first palm prediction result and the fourth probability in the second palm prediction result can be calculated, the average probability that the target palm is the right palm is obtained, the palm type with the large average probability is obtained, and the palm type of the target palm is determined.

The palm type of the target palm may also be determined by weighted summation of the first palm prediction result and the second palm prediction result, including the first probability that the target palm is the left palm, the step of determining the palm type of the target palm based on the first palm prediction result and/or the second palm prediction result may include:

and determining the palm type of the target palm according to the left palm probability and the right palm probability. For example, weights corresponding to the first palm prediction result and the second palm prediction result may be preset, and weighted summation is performed on the first probability in the first palm prediction result and the third probability in the second palm prediction result according to the weights, so as to obtain a left palm probability that the target palm is the left palm; and carrying out weighted summation on the second probability in the first palm prediction result and the fourth probability in the second palm prediction result according to the weight to obtain right palm probability that the target palm is the right palm; and determining the palm type of the target palm with high probability, determining the left palm of the target palm if the left palm probability is greater than the right palm probability, and determining the right palm of the target palm if the right palm probability is greater than the left palm probability.

The global palm print recognition processing is different from the local palm print recognition processing, the first palm print recognition processing can recognize based on the collected whole palm print, the local palm print recognition processing can recognize based on all parts of the collected palm print, and the recognition results of all parts are integrated to determine the second palm prediction result.

In an embodiment, the palm print recognition model may include a repaired global recognition branch and a decomposed local recognition branch, where the repaired global recognition branch may perform global palm print recognition processing based on the palm print image to obtain a first palm prediction result of the target palm; and (3) carrying out partial palmprint recognition processing on the split partial recognition branches and/or based on palmprint images to obtain a second palmar prediction result of the target palms.

In an embodiment, the step of performing the local palm print recognition process based on the palm print image to obtain the palm type of the target palm may include:

The regional extraction model may use the RPN part of the Faster R-CNN network, and the classification network may use the VGG-16 network.

Specifically, the palm print image may be input into a region extraction model, and the region extraction model identifies an image region in the palm print image that includes palm print information (i.e., an effective region in the palm print image), and outputs a prediction frame that indicates the effective region as shown in fig. 3, i.e., a red frame in fig. 3.

In an example, the step of performing palm type prediction processing on the image area corresponding to each effective frame to obtain a prediction result corresponding to each partial palm print image block may include:

Extracting an image area corresponding to the effective frame from the palm print image, and generating a local palm print image block based on the image area;

and carrying out palm type prediction processing on each local palm print image block to obtain a prediction result corresponding to each local palm print image block.

For example, the palm print image may be extracted to obtain multiple candidate prediction frames, and based on the partial palm print image in the frame in the prediction frame, it is determined whether the prediction frame is an effective frame containing palm prints, where the effective frame contains more continuous palm print lines, for example, lines of one finger or half of the finger, and the like.

The image areas corresponding to the effective frames are extracted from the palm print image, the identified image areas can be used as local palm print image blocks, each palm print image block comprises a local palm print, and all the identified image areas can be converted to the same size by cutting and/or difference processing on the identified image areas, so that a batch of local palm print image blocks with the same size are obtained.

Because the palm print distribution of the left palm and the right palm is different, for example, the local palm print corresponding to the thumb in the left palm is distributed on the right side of the palm print, and the local palm print corresponding to the thumb in the right palm is distributed on the left side of the palm print, the palm type identification can be performed based on the local palm print image block corresponding to the palm print image, and the second palm prediction result of the target palm is obtained.

In an embodiment, the palm type of the local palm print module may be identified by using a classification network to obtain an identification result corresponding to each local palm print image block, where the identification result may include a probability that a local palm print in the local palm print image block belongs to a left palm and a probability that a local palm print in the local palm print image block belongs to a right palm, and the palm type of the target palm is determined according to the average probability that the local palm print module of the palm print image belongs to the left palm and the average probability that the local palm print module belongs to the left palm.

In an embodiment, the model structure of the local recognition model for palm type recognition through the classification network (i.e. the local recognition branch of the palm print recognition model may include the local recognition model) may be as shown in fig. 3, the local recognition model may include an area extraction network (may also be referred to as an area extraction model), an area classification network, a clipping and interpolation module, and a classification network, where the area extraction network is used for extracting candidate detection frames from the palm print image, i.e. the step of "extracting multiple prediction frames from the palm print image", the area classification network is used for classifying the candidate detection frames, determining an effective frame containing more continuous palm print lines, i.e. "generating local palm print image blocks based on image areas", extracting corresponding image areas from the palm print image based on the effective frame, and the clipping and interpolation module is used for transforming each image area into a local palm print image block of the same size, i.e. the step of "performing palm type prediction processing on each local palm print image block to obtain a prediction result corresponding to each local palm print image block", where the network is used for identifying the local palm print image block, and the candidate detection frames, and the effective frame is determined, i.e. the effective frame containing more continuous palm print image blocks may be extracted, and the local palm image blocks may be processed by using the RPN network.

Specifically, the palm print image may be input into the local recognition model, and the region extraction network recognizes an image region including palm print information in the palm print image (i.e., an effective region in the palm print image), and outputs a prediction frame indicating the effective region as shown in fig. 3, i.e., a red frame in fig. 3.

And extracting a plurality of effective areas from the palm print image based on the prediction frame, cutting and/or interpolating, and transforming to the same size, so that a batch of local palm print image blocks can be obtained, and inputting the batch of local palm print image blocks into a classification network for palm type recognition at the same time to obtain a recognition result corresponding to each local palm print image block.

In another embodiment, the palm type of the target palm may be determined by performing palm type recognition on the partial palm print image blocks through a pre-trained palm print recognition model, determining the probability that the partial palm print in each partial palm print image block belongs to the designated palm, for example, the probability that the partial palm print module of the palm print image belongs to the right palm, and determining the palm type of the target palm according to the average probability that the partial palm print module of the palm print image belongs to the designated palm.

The first palm recognition processing may be to perform overall recognition on all palm print information in the palm print image, that is, in an embodiment, the step of performing global palm print recognition processing based on the palm print image to obtain a first palm and second palm prediction result of the target palm includes:

and carrying out palm type prediction processing based on the repaired palmprint image to obtain a first palm prediction result of the target palm.

For example, the palm print image can be subjected to palm print restoration processing through a palm print restoration model, so that a restored palm print image is obtained, and the palm print restoration model can be used for restoring damaged palm prints, so that a neural network model of the complete palm print is obtained.

In an embodiment, the repaired global recognition branch of the palm print recognition model may include a palm print repair model, and palm print repairing is performed on the palm print image by the palm print repair model of the repaired global recognition branch to obtain a repaired palm print image; and carrying out palm type prediction processing based on the repaired palmprint image to obtain a first palm prediction result of the target palm.

Optionally, the damaged palm print in the palm print image may be repaired by using the mask image, that is, in an embodiment, the step of "performing palm print repair processing on the palm print image to obtain a repaired palm print image" includes:

The palm print image collected by the vehicle door switch assembly may be damaged, for example, a palm print portion corresponding to a thumb is lacking, and shape feature extraction can extract the shape of the palm print contained in the palm print image, so that an effective area and an ineffective area in the palm print image can be determined. The effective area is an image area containing more continuous palm print lines, and the ineffective area is an image area without more continuous palm print lines.

The shape feature of the palm print in the palm print image may be extracted by morphological operations, which may include erosion, inflation, open operation, close operation, morphological gradient operation, top hat operation, black hat operation, etc., and the morphological feature extraction may be at least one operation thereof.

For example, a mask map of the palm print image may be extracted, the mask map may indicate active and inactive areas in the palm print image, e.g., the values in the mask map contain only 0 and 1, where 0 indicates inactive areas and 1 indicates active areas.

According to the mask image, an invalid region in the palm print image can be determined, and then the invalid region can be repaired to obtain a complete palm print, and the palm type of the target palm is determined based on the repaired palm print image for recognition.

For example, image feature extraction can be performed on a palm print image to obtain image feature information of the palm print image, the image feature information is divided into a plurality of feature blocks with the same size, the palm print image is divided into a plurality of image blocks, the image blocks are in one-to-one correspondence with the feature blocks, context information in the palm print image can be extracted through a self-attention mechanism to obtain attention feature blocks corresponding to each image block, and decoding is performed based on the attention feature blocks to obtain the repaired palm print image.

Optionally, the step of repairing the palm print based on the feature block of the effective area, where the step of performing palm print repairing processing on the palm print image based on the mask map to obtain a repaired palm print image may include:

And repairing the palmprint image based on the first image feature and the second image feature to obtain a repaired palmprint image.

In an embodiment, the step of repairing the palm print image based on the first image feature and the second image feature to obtain a repaired palm print image may include:

extracting feature blocks from the first image features to obtain first feature blocks, and extracting feature blocks from the second image features to obtain second feature blocks;

and generating a repaired palmprint image based on the first feature block and the third feature block.

For example, feature extraction may be performed on a palm print image through a convolutional network to obtain image features of the palm print image.

The image features may be feature matrices, with 1 in the mask map representing the active area and 0 representing the inactive area, and the image features and mask map may be transformed to the same size, e.g., c h w. The image features and the mask map are multiplied pixel by pixel to mask the feature portion corresponding to the invalid region in the image features, and the feature block extraction is performed on the first image features after masking to obtain a first feature block, for example, the image features with the size of c×h×w may be expanded into h×w feature blocks with the size of c×3×3, and since the feature portion corresponding to the invalid region is masked, the first feature block only includes the feature portion corresponding to the valid region, and may be considered as a feature block of a known region. The specific calculation formula is as follows, wherein,As a first feature block of the set of features,In order to make the mask pattern a pattern,Representing pixel-by-pixel multiplication between features, unfold represents a feature block extraction operation, and F is the image feature of the palm print image.

Similarly, the feature portion corresponding to the invalid region in the image feature may be masked, and the feature block extraction may be performed on the second image feature after masking to obtain the second feature block, where the second feature block includes only the feature corresponding to the palm print information portion in the target image region of the palm print image because the feature portion corresponding to the valid region is masked, and the target image region is the image region corresponding to the invalid region of the mask image, so that the second feature block may be regarded as the feature block of the damaged region in the palm print image. The specific calculation formula is as follows, wherein,As a second feature block of the set of features,In order to make the mask pattern a pattern,Each pixel of the representation 1 is subtracted,Representing pixel-by-pixel multiplication between features, unfold represents a feature block extraction operation, and F is the image feature of the palm print image.

Since the second feature block may be regarded as a feature block of a damaged area in the palm print image, in combination with the context information in the palm print image, the damaged area may be repaired, specifically, for a target second feature block in the second feature block, the target second feature block and each first feature block are mapped into the same feature space, the distance between the target second feature block and each first feature block in the feature space is calculated, and the similarity between the target second feature block and each first feature block is determined based on the distance in the feature space, the closer the distance is, the higher the similarity is, the farther the distance is, and the lower the similarity is.

And carrying out normalization processing on the similarity between the target second feature block and each first feature block, mapping the similarity to between 0 and 1, and taking the normalized similarity as the weight corresponding to the first feature block. And carrying out weighted summation on the first feature blocks based on the weight corresponding to each first feature block to obtain weighted feature blocks, and updating the target second feature blocks by the weighted feature blocks to enable the target second feature blocks to be identical with the weighted feature blocks.

And executing the same processing with the second target feature block on each second feature block to obtain a third feature block corresponding to each second feature block.

And aggregating the third feature block and the first feature block to obtain a repaired palmprint image, wherein the first feature block and the third feature block are aggregated by adding the feature blocks with the same positions in the image features element by element to obtain the repaired image features.

Based on the first feature block and the third feature block, a repaired palmprint image may be generated, or the first feature block and the third feature block may be combined to obtain a combined image feature, or the repaired image feature may be corresponding to the repaired palmprint image, where a specific formula is as follows,Representing the summation of all the values in the block,Representing the characteristics of the combined image,As a first feature block of the set of features,Is the second feature block.

Optionally, local convolution processing may be performed on the original image feature to obtain a processed image feature, where the processed image feature includes a local feature of the palm print image, the combined image feature includes a contextual feature of the palm print image, and the combined image feature and the original image feature may be spliced to obtain a repaired image feature, where a specific calculation formula is as follows,Representing a splice of the dimensions of the channel,A partial convolution is represented and is shown,Representing the final output characteristics.

In an embodiment, the palm print repairing model comprises a plurality of partial convolution layers and a context effective perception module, wherein the partial convolution layers can be used for extracting features of a palm print image to obtain image features of the palm print image; the context effective perception module is used for masking a characteristic part corresponding to an invalid region in the mask image in the image characteristics to obtain first image characteristics, and extracting a characteristic block of the first image characteristics to obtain a first characteristic block; masking local features corresponding to the effective areas in the mask image in the image features to obtain second image features, and extracting feature blocks of the second image features to obtain second feature blocks; and repairing each second feature block based on the first feature block to obtain a third feature block, wherein the detailed process is referred to the related content and will not be described herein.

Because the second feature blocks correspond to the damaged areas in the palm print image, the second feature blocks contain some invalid feature values, so that feature parts corresponding to the invalid feature values in the first feature blocks can be shielded, calculation errors caused by the invalid feature values are avoided, accuracy of calculated similarity is improved, and palm print repairing effect is improved, namely in an embodiment, the step of repairing each second feature block based on the first feature blocks to obtain a third feature block includes:

carrying out weighted summation processing on the first characteristic blocks according to the similarity to obtain weighted characteristic blocks;

And updating the second feature block based on the weighted feature block to obtain a third feature block.

For example, the mask map may indicate active and inactive areas in the palm print image, the mask map may be updated, and active and inactive values in the damaged area may be determined based on differences between the repaired mask map and the original mask map.

The mask map may be updated based on the following equation, wherein,Representing the summation of all the values in the block,And representing an updated mask pattern, m being a mask block in the initial mask pattern, the mask block having the same dimensions as the feature block.

The updated mask pattern may be used as a current mask pattern, valid and invalid values in the damaged area may be determined based on a difference between the current mask pattern and the initial mask pattern, and divided into mask patterns, and a specific calculation formula may be as follows,For the mask block, M is the current mask map,In order to initiate the mask pattern,Representing inter-feature pixel-by-pixel multiplication, unfold represents a feature block extraction operation.

The masking module may indicate an effective value and an ineffective value in the damaged area, mask the first feature block based on the masking module, mask a feature portion corresponding to the ineffective value in the second feature block in the first feature block, obtain a masked first feature block, and recalculate the similarity with the second feature block based on the masked first feature block, where < and > represent an inner product,Representing a first feature blockAnd a second feature blockIs used for the degree of similarity of (c) to (c),A second feature block representing an update is presented,Representing the passing through the second feature blockCorresponding mask blockFor the first feature blockMasking is performed.

The similarity to the second feature block and the masked first feature block may be calculated based on other ways to determine similarity, such as a euclidean distance (Euclidean Distance), a manhattan distance (MANHATTAN DISTANCE), a pearson correlation coefficient (Pearson Correlation Coefficient), and the like, in addition to cosine similarity as shown in the above formula.

1013. And determining whether the door opening mode is a lotus-type door opening mode according to the palm type.

Since the charge door-opening mode requires the user to operate the door-opening assembly with a hand farther from the vehicle door, it is possible to determine whether the user's door-opening mode is the charge door-opening mode based on the palm type.

102. And under the condition that the door opening mode is a lotus type door opening mode, controlling the vehicle to execute preset first feedback.

Alternatively, the vehicle may be controlled to execute a preset second feedback when the door opening mode is the non-load door opening mode, where the preset second feedback is different from the preset first feedback.

For example, the preset first feedback includes at least one of a first voice alert, a first vehicle light flashing, and a first whistle, and no feedback;

If the door opening mode is not the lotus-type door opening mode, triggering a preset second feedback, wherein the preset second feedback comprises at least one of a second voice prompt, second vehicle light flickering and second whistling.

For example, if the door opening mode of the user is not the lotus-type door opening mode, a reminding voice, such as "please observe the coming car behind", control the whistle of the vehicle, control the light flash outside the vehicle or control the light flash inside the vehicle, can be output.

If the door opening mode of the user is the lotus type door opening mode, the encouraging voice can be output, the flashing of the lamp light in the vehicle can be controlled, or no feedback is used as feedback.

If the same kind of information is adopted in the preset first feedback and the preset second feedback, the feedback modes are different, for example, the flicker speeds of the lamplight are different, and the jerk degree of the vehicle whistle is different.

In an embodiment, a pressure sensor may be further disposed in the door inner side switch assembly, the pressure sensor may collect pressure of a palm of a user on the door opening assembly, determine whether the user opens the door according to the pressure, if the pressure is smaller, it may be considered that the user does not open the door, but just puts a hand on the door opening assembly, that is, the switch assembly also collects pressure information, where the step "in the case that the door opening mode is the load type door opening mode, the step" may include:

and under the condition that the door opening mode is a lotus type door opening mode and the pressure information meets the preset pressure condition, controlling the vehicle to execute preset first feedback.

In an embodiment, the step of controlling the vehicle to perform the preset second feedback when the door opening mode is the off-load door opening mode may include:

And under the condition that the door opening mode is the non-load door opening mode and the pressure information meets the preset pressure condition, controlling the vehicle to execute preset second feedback. Wherein the pressure information may indicate a pressure of the target palm against the switch assembly.

For example, a palm print sensor and a pressure sensor are arranged on the switch assembly, when a user makes a door opening mode, the palm print sensor on the switch assembly is used for collecting the palm print of a target palm of the operation switch assembly, generating a palm print image, and the pressure sensor is used for collecting pressure information of the target palm acting on the switch assembly.

The preset pressure condition may be that the pressure indicated by the pressure information is greater than or equal to a preset pressure threshold.

The pressure information satisfies a preset pressure condition, and the user is considered to operate the switch assembly and open or will open the door, otherwise the user is considered to indicate that the handle is placed on the switch assembly. With the switch assembly 21 as shown in fig. 2, if the pressure information satisfies the preset pressure condition, the user may be considered to pull the handle, otherwise the user may be considered to not pull the handle.

And executing feedback prompt logic for the door opening mode according to the determined result under the condition that the pressure information meets the preset pressure condition.

Optionally, the user may select whether to turn on the function of the vehicle for prompting the door opening mode of the user, and if the function is turned off, the vehicle will not perform the operations of steps 101-102 when the user performs the door opening mode.

From the above, the embodiment of the application determines the door opening mode according to the biological characteristic information collected by the door opening and closing assembly; and under the condition that the door opening mode is a lotus type door opening mode, controlling the vehicle to execute preset first feedback. According to the embodiment of the application, whether the door opening mode of the user is the lotus-type door opening mode is judged through the biological characteristic information of the user collected by the door opening and closing assembly, so that whether the user can pay attention to the vehicle condition on the road section can be determined, and then a corresponding feedback prompt is given to the user, so that the accident occurrence can be reduced.

In order to explain the door opening control method of the vehicle provided by the application, on the basis of the embodiment, the following further describes taking the example that the door opening assembly is provided with a pressure sensor and a palm print sensor, and the vehicle carries out corresponding feedback based on the acquired pressure information and the palm print image.

1. Early warning flow for opening door

The user can select whether to open the vehicle and carry out corresponding prompt function (hereinafter referred to as door opening early warning function) to the mode of opening the door of user, if the user closes this function, when the user carries out the mode of opening the door, the door can not gather user's palmprint image and pressure through switch subassembly, also can not feed back.

Under the condition that the vehicle starts the function of corresponding prompt to the door opening mode of the user, as shown in fig. 4, the vehicle triggers the feedback process, the user operates the switch assembly on the inner side of the vehicle door, the palm print sensor on the switch assembly can collect the palm print image of the target palm of the user operating the switch assembly, and the pressure sensor on the switch assembly can collect the pressure information.

Inputting the palm print image into a palm print recognition model, performing palm print recognition processing on the palm print image through the palm print recognition model, determining whether the target palm is a left palm or a right palm, and then determining whether the door opening mode of the user is a lotus door opening mode according to the preset palm type.

If the pressure of the target palm to the switch assembly is determined to be less than the preset pressure threshold based on the pressure information, feedback is not triggered.

And if the pressure of the target palm to the switch assembly is greater than or equal to the preset pressure threshold value based on the pressure information, and the door opening mode of the user is a lotus door opening mode, the vehicle performs positive feedback.

And if the pressure of the target palm to the switch assembly is greater than or equal to the preset pressure threshold value based on the pressure information, and the door opening mode of the user is not the lotus type door opening mode, carrying out negative feedback on the vehicle.

The negative feedback can be that the vehicle outputs voice to remind the user to pay attention to the coming vehicle at the rear and pay attention to the fact that the user uses the lotus-type door opening next time, the negative feedback can be used as positive feedback, the voice reminding is used for helping the user to develop the habit of opening the vehicle door through the lotus-type door opening method, and traffic accidents are reduced.

It can be understood that, for the left door of the vehicle, the corresponding preset palm type is the right palm, and for the right door of the vehicle, the corresponding preset palm type is the left palm, and when the palm print image and the pressure information are acquired through the door opening assembly, the corresponding preset palm type of the door opening assembly can also be acquired, so as to be used for judging whether the door opening of the user is the lotus-type door opening.

2. Setting of preset pressure threshold

The method comprises the steps that a proper pressure threshold can be set, early warning can be conducted before a door switch is opened, a feedback mechanism of a vehicle cannot be touched by mistake, feedback can be conducted by mistake when a user does not have a door opening intention, a pressure data sample corresponding to the door opening can be obtained, each pressure data sample can comprise the pressure of a palm collected by a pressure sensor on the switch assembly in the process from the contact of the switch assembly to the opening of the switch assembly, the maximum pressure in the door opening process is used as the pressure required by the door opening at the moment, the minimum pressure in the pressure data sample in the pressure required by the door opening is used as a preset pressure threshold, or a value smaller than the minimum pressure is determined as the preset pressure threshold, and the pressure data sample is used for judging whether the user is about to open the door or whether the intention of opening the door exists.

A reasonable pressure threshold value can be obtained based on a large number of pressure data samples, early warning can be performed in advance before a user opens a door but does not open the door yet, and the door opening intention of the user cannot be judged by mistake.

3. Model structure and training of palmprint recognition model

The model structure of the palm print recognition model may include a global recognition branch of a repair type and a local recognition branch of a decomposition type as shown in fig. 5.

(1) Repaired global identification branching

The damaged palm print image is repaired through the palm print repairing model, then the repaired palm print image containing complete palm print information is identified, and the identification result is output mainly depending on global information.

Specifically, a mask map of the palm print image may be extracted by morphological operations, the mask map having values containing only 0 and 1,0 indicating the location of an invalid region in the palm print image, and 1 indicating the location of a known region in the palm print image.

And then, inputting the mask image and the palm print image into a palm print restoration model together for restoration to obtain a restored palm print image, inputting the restored palm print image into a classification network, and outputting a recognition result through the classification network, wherein the output result can comprise the probability that the target palm is the left palm and the probability that the target palm is the right palm.

As shown in FIG. 5, the palm print restoration model comprises a plurality of partial convolution layers and a context effective perception module, and the palm print restoration module adopts a U-net type network structure which comprises multistage characteristic jump connection. The processing flow of the context effective sensing module and part of the convolution layer is shown in fig. 6.

The mask image and the palmprint image are input into a palmprint restoration model, the image features and the mask image are transformed into the same size, for example, c×h×w, through a context effective perception module, the image features and the mask image are multiplied pixel by pixel to mask the feature parts corresponding to the invalid areas in the image features, and feature block extraction is performed on the first masked image features to obtain a first feature block, for example, the image features with the size of c×h×w can be unfolded into h×w c×3×3 feature blocks. The specific calculation formula is as follows.

Similarly, the feature part corresponding to the invalid region in the image feature can be masked, and the feature block of the masked second image feature is extracted to obtain a second feature block, and a specific calculation formula is as follows.

For the first context effective perception module of the palm print restoration model, updating a mask image of the initial input palm print restoration model based on the following formula to obtain a current mask image; and the current mask pattern may be updated by the following formula and the updated mask pattern may be input to the next layer as the current mask pattern of the next layer.

The effective value and the ineffective value in the damaged area can be determined based on the difference between the updated mask pattern and the initial mask pattern, and the specific formula is as follows.

The first feature block after shielding is obtained based on the feature part corresponding to the invalid value in the second feature block in the first feature block shielded by the mask block, and the similarity with the second feature block is calculated based on the first feature block after shielding, and the specific calculation formula can be as follows.

The mask image and the palm print image are input into a palm print restoration model, and partial convolution layers in the mask image and the palm print image are used for extracting features to obtain local features, wherein the local features refer to features after partial convolution, and the partial convolution layers have the capability of extracting the local features; the context effective sensing module can extract context information, and can splice local features and context features so as to integrate the local features and the context features in the palmprint image, wherein a specific calculation formula is as follows.

(2) Split local identification branching

For example, the palm print image may be input to the region extraction network in the partial recognition branch of the split type, and the image region including the palm print information in the palm print image may be recognized by the region extraction network, and the prediction frame indicating the effective region as shown in fig. 5, that is, the red frame in fig. 5 may be output.

And extracting a plurality of effective areas from the palm print image based on the prediction frame, cutting and/or interpolating, and transforming to the same size, so that a batch of local palm print image blocks can be obtained, and inputting the batch of local palm print image blocks into a classification network for palm type recognition at the same time to obtain a recognition result corresponding to each local palm print image block. The regional extraction network may use the RPN part of the fast R-CNN network, and the classification network may use the VGG-16 network.

The recognition result may include a probability that the local palm print in the local palm print image block belongs to the left palm and a probability that the local palm print in the local palm print image block belongs to the right palm, and the palm type of the target palm is determined according to the average probability that the local palm print module of the palm print image belongs to the left palm and the average probability that the local palm print module belongs to the left palm.

(3) Model training

Training samples: the training sample for palm print recovery model training can be used for shielding the complete palm print at random positions and sizes, the shielded palm print is used as the training sample, and the complete palm print is used as a label.

For a training sample of region extraction network training, the effective region of the palmprint image sample can be marked manually.

After the palm print recovery model, the area extraction network and the classification network are trained, the repair type global recognition branch and the decomposition type local recognition branch can be finely tuned, palm print image samples after reaching a preset pressure threshold can be selected as samples, the palm print image samples correspond to sample labels, the sample labels indicate whether palm prints in the palm print images belong to left palms or right palms, and further, random cutting operation can be carried out on the palm print images, so that a sample set with rich data is constructed.

Setting a loss function; the whole palm print recognition model presents a multi-stage reasoning process, different models, networks and modules need to be provided with different loss functions, the loss functions of the area extraction network are consistent with FasterR-CNN methods, the VGG-16 model uses cross entropy as the loss function, the image restoration model jointly reconstructs loss and antagonism loss based on binary cross entropy as the loss function, the super-parameter proportion is set to be 1:1, and the discriminator uses SN-PatchGAN.

Setting training parameters; for parameter initialization of the whole model, the classification network of the repaired global recognition branches and the decomposed local recognition branches can use a VGG-16 structure, wherein parameters for feature extraction are initialized by using VGG-16 pre-trained on an ImageNet data set, and parameters of the classification network are randomly initialized from Gaussian distribution with the mean value of 0 and the standard deviation of 1.

Parameters of the area extraction network are initialized by using a fast R-CNN (R-CNN) on an RPN part pre-trained on a PASCAL VOC data set, and parameters of a palm print restoration model are randomly initialized from Gaussian distribution with a mean value of 0 and a standard deviation of 1.

In the training process, the regional extraction network and the classification model are trained by adopting SGD for parameter optimization, the momentum is set to 0.9, the weight attenuation is set to 0.0005, and the batch processing number is set to 256.

The training of the palmprint repair model adopts Adam to carry out parameter optimization, and momentum and weight attenuation are consistent with SGD. The training process of two stages is adopted, wherein in the first stage, each part is independently trained, the learning rate is set to be 0.001, and the iteration is performed for 50 ten thousand times; in the second stage, the individual parts are trained together, and the learning rate is set to 0.0001 until the whole model converges.

In order to facilitate better implementation of the door opening control method of the vehicle provided by the embodiment of the application, an embodiment also provides a door opening control device of the vehicle. The meaning of the noun is the same as that of the door opening control method of the vehicle, and specific implementation details can be referred to the description of the method embodiment.

The door opening control apparatus of a vehicle may be integrated in a vehicle, as shown in fig. 7, and may include: the determining unit 301 and the feedback unit 302 are specifically as follows:

(1) A determining unit 301, configured to determine a door opening mode according to the biometric information collected by the door opening and closing assembly.

In an embodiment, the determining unit 301 comprises an acquisition subunit, an identification subunit and a mode determining subunit.

The acquisition subunit is used for acquiring the palm print image of the target palm when the vehicle door opening and closing assembly is triggered.

And the identification subunit is used for carrying out palm print identification processing based on the palm print image to obtain the palm type of the target palm, wherein the palm type is used for indicating whether the target palm is a left palm or a right palm.

And the mode determining subunit is used for determining whether the door opening mode is a lotus-type door opening mode according to the palm type.

In an embodiment, the identifying subunit includes:

the first palm print recognition module is used for carrying out global palm print recognition processing based on the palm print image to obtain a first palm prediction result of the target palm;

And/or, the first palm print recognition module is used for carrying out local palm print recognition processing based on the palm print image to obtain a second palm prediction result of the target palm;

and the type determining module is used for determining the palm type of the target palm based on the first palm prediction result and/or the second palm prediction result.

Optionally, the first palm prediction result includes a first probability that the target palm is a left palm and a second probability that the target palm is a right palm, the second palm prediction result includes a third probability that the target palm is a left palm and a fourth probability that the target palm is a right palm, and the type determining module includes:

The first weighting submodule is used for carrying out weighted summation on the first probability and the third probability to obtain left palm probability that the target palm is a left palm;

the second weighting sub-module is used for carrying out weighted summation on the second probability and the fourth probability to obtain right palm probability that the target palm is the right palm;

And the palm type determining submodule is used for determining the palm type of the target palm according to the left palm probability and the right palm probability.

In an embodiment, the first palm print recognition module includes:

the repair sub-module is used for carrying out palm print repair treatment on the palm print image to obtain a repaired palm print image;

And the first prediction sub-module is used for carrying out palm type prediction processing based on the repaired palmprint image to obtain the first palm prediction result of the target palm.

In an embodiment, the repairing sub-module includes:

A mask extraction unit for extracting a mask image of the palm print image, wherein the mask image is used for indicating positions of an effective palm print area and an ineffective palm print area in the palm print image;

And the image restoration component is used for carrying out palm print restoration processing on the palm print image based on the mask image to obtain a restored palm print image.

In an embodiment, the image restoration component includes:

the feature extraction sub-assembly is used for extracting features of the palm print image to obtain image features of the palm print image;

the first shielding subassembly is used for shielding the characteristic part corresponding to the invalid region in the mask graph in the image characteristic to obtain a first image characteristic;

The second shielding subassembly is used for shielding local features corresponding to the effective areas in the mask image in the image features to obtain second image features;

and the palm print restoration sub-assembly is used for restoring the palm print image based on the first image characteristic and the second image characteristic to obtain the restored palm print image.

In one embodiment, the palmprint repair subassembly includes:

a first block extraction means for extracting a feature block from the first image feature to obtain a first feature block, and extracting a feature block from the second image feature to obtain a second feature block;

A second block extraction means for repairing each of the second feature blocks based on the first feature block to obtain a third feature block;

And a block repairing unit configured to generate the repaired palmprint image based on the first feature block and the third feature block.

In one embodiment, an upper block repair member includes:

An invalid value shielding sub-component, configured to shield each of the first feature blocks according to an invalid value in each of the second feature blocks, so as to obtain a first feature block after shielding corresponding to each feature block;

A calculating sub-component for calculating the similarity between each second characteristic block and each first characteristic block after shielding;

A weighting sub-component, configured to perform weighted summation processing on the first feature block according to the similarity, so as to obtain a weighted feature block;

And the updating sub-component is used for updating the second characteristic block based on the weighted characteristic block to obtain a third characteristic block.

In an embodiment, the first palm print recognition module includes:

A prediction frame extraction sub-module, configured to extract a plurality of prediction frames from the palm print image, and determine at least one valid frame including a palm print from the prediction frames based on a partial palm print image in the prediction frames;

the image block generation sub-module is used for extracting an image area corresponding to the effective frame from the palm print image and generating a local palm print image block based on the image area;

The second prediction sub-module is used for carrying out palm type prediction processing on each partial palmprint image block to obtain a prediction result corresponding to each partial palmprint image block;

And the type determining sub-module is used for determining the palm type of the palm print image based on the prediction result corresponding to each partial palm print image block of the palm print image.

(2) And a feedback unit 302, configured to control the vehicle to execute a preset first feedback when the door opening mode is a load door opening mode.

Optionally, the door opening control device of the vehicle may further include:

and the second feedback unit is used for controlling the vehicle to execute preset second feedback when the door opening mode is the non-load door opening mode, wherein the preset second feedback is different from the preset first feedback.

Optionally, the switch assembly further collects pressure information, and the feedback unit 302 is further configured to:

and controlling the vehicle to execute preset first feedback under the condition that the door opening mode is a lotus type door opening mode and the pressure information meets the preset pressure condition.

The second feedback unit is further configured to:

and under the condition that the door opening mode is an unloading door opening mode and the pressure information meets the preset pressure condition, controlling the vehicle to execute preset second feedback.

In an embodiment, the preset first feedback includes at least one of a first voice alert, a first vehicle light flashing, a first whistle, and no feedback; and if the door opening mode is not the lotus-type door opening mode, triggering negative feedback, wherein the preset second feedback comprises at least one of a second voice prompt, second vehicle lamplight flickering and second whistling.

From the above, the embodiment of the application determines the door opening mode according to the biological characteristic information collected by the door opening and closing assembly; the feedback unit 302 controls the vehicle to execute the preset first feedback when the door opening mode is the charge door opening mode. According to the embodiment of the application, whether the door opening mode of the user is the lotus-type door opening mode is judged through the biological characteristic information of the user collected by the door opening and closing assembly, so that whether the user can pay attention to the vehicle condition on the road section can be determined, and then a corresponding feedback prompt is given to the user, so that the accident occurrence can be reduced.

Control device the embodiment of the present application further provides a vehicle, as shown in fig. 8, which shows a schematic structural diagram of the vehicle according to the embodiment of the present application, specifically:

The vehicle may include one or more processing cores 'processors 1001, one or more computer-readable storage media's memory 1002, a power supply 1003, and an input unit 1004, among other components. Those skilled in the art will appreciate that the vehicle structure shown in fig. 8 is not limiting of the vehicle and may include more or fewer components than shown, or certain components may be combined, or a different arrangement of components. Wherein:

The processor 1001 is a control center of the vehicle, connects various parts of the entire vehicle using various interfaces and lines, and performs various functions of the vehicle and processes data by running or executing software programs and/or modules stored in the memory 1002 and calling data stored in the memory 1002, thereby performing overall monitoring of the vehicle. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, a computer program, and the like, and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 1001.

The memory 1002 may be used to store software programs and modules, and the processor 1001 executes various functional applications and data processing by executing the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a stored program area that may store an operating system, computer programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a stored data area; the storage data area may store data created according to the use of the vehicle, etc. In addition, memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

The vehicle further includes a power supply 1003 for powering the various components, preferably, the power supply 1003 is logically connected to the processor 1001 by a power management system, such that charge, discharge, and power consumption management functions are performed by the power management system. The power supply 1003 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The vehicle may also include an input unit 1004, which input unit 1004 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

Although not shown, the vehicle may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 1001 in the vehicle loads executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 executes the computer programs stored in the memory 1002, so as to implement various functions, as follows:

according to the embodiment of the application, the door opening mode is determined according to the biological characteristic information collected by the door opening and closing assembly;

From the above, the embodiment of the application judges whether the door opening mode of the user is the lotus-type door opening mode or not through the biological characteristic information of the user collected by the door opening and closing assembly, thereby determining whether the user can pay attention to the vehicle condition on the road section or not, and giving the user a corresponding feedback prompt, so that the accident occurrence can be reduced.

According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of the vehicle, which executes the computer instructions, causing the vehicle to perform the methods provided in the various alternative implementations of the embodiments described above.

It will be appreciated by those of ordinary skill in the art that all or part of the steps of the various methods of the above embodiments may be performed by a computer program, or by computer program control related hardware, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer-readable storage medium in which a computer program is stored, the computer program being capable of being loaded by a processor to perform any one of the door opening control methods of the vehicle provided by the embodiment of the present application.

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

Wherein the computer-readable storage medium may comprise: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.

Because the computer program stored in the computer readable storage medium can execute any vehicle door opening control method provided by the embodiment of the present application, the beneficial effects that any vehicle door opening control method provided by the embodiment of the present application can be achieved, and detailed descriptions of the foregoing embodiments are omitted herein.

The above description of the method, the device, the vehicle and the computer readable storage medium for controlling the door opening of the vehicle provided by the embodiments of the present application has been provided in detail, and specific examples are applied herein to illustrate the principles and embodiments of the present application, and the above description of the embodiments is only for helping to understand the method and the core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.

Claims

1. A door opening control method of a vehicle, characterized by comprising:

2. The method according to claim 1, wherein the method further comprises:

3. The method of claim 1, wherein the biometric information comprises a palmprint image, and wherein determining the door open mode based on the biometric information collected by the door opening and closing assembly comprises:

And determining the door opening mode according to the palm type.

4. The method according to claim 3, wherein the performing palm print recognition processing based on the palm print image to obtain the palm type of the target palm includes:

5. The method of claim 4, wherein the first palm prediction result includes a first probability that the target palm is a left palm and a second probability that the target palm is a right palm, the second palm prediction result includes a third probability that the target palm is a left palm and a fourth probability that the target palm is a right palm, and wherein determining the palm type of the target palm based on the first palm prediction result and/or the second palm prediction result includes:

6. The method of claim 4, wherein the performing the local palm print recognition process based on the palm print image to obtain the second palm prediction result of the target palm includes:

7. The method of claim 6, wherein the performing palm type prediction processing on the image area corresponding to each effective frame to obtain a prediction result corresponding to each partial palm print image block comprises:

8. The method of claim 4, wherein the performing global palm print recognition processing based on the palm print image to obtain the first palm prediction result of the target palm comprises:

9. The method of claim 8, wherein performing a palmprint repair process on the palmprint image to obtain a repaired palmprint image comprises:

10. The method of claim 9, wherein performing a palm print repair process on the palm print image based on the mask map to obtain a repaired palm print image comprises:

11. The method of claim 10, wherein the repairing the palmprint image based on the first image feature and the second image feature to obtain the repaired palmprint image comprises:

12. The method of claim 11, wherein repairing each of the second feature blocks based on the first feature blocks to obtain a third feature block comprises:

13. The method of claim 1, wherein, in the case where the door-open mode is a charge-type door-open mode, controlling the vehicle to perform a preset first feedback comprises:

and under the condition that the door opening mode is a lotus type door opening mode and the pressure information collected by the vehicle door switch assembly meets the preset pressure condition, controlling the vehicle to execute the preset first feedback.

14. The method of claim 2, wherein controlling the vehicle to perform a preset second feedback if the door-open mode is an off-load door-open mode comprises:

and under the condition that the door opening mode is an unloading door opening mode and the pressure information acquired by the door opening and closing assembly meets the preset pressure condition, controlling the vehicle to execute the preset second feedback.

15. The method of claim 2, wherein the preset first feedback comprises at least one of a first voice alert, a first vehicle light flashing, and a first whistle, and no feedback;

The preset second feedback includes at least one of a second voice alert, a second vehicle light flashing, and a second whistle.

16. A door opening control apparatus for a vehicle, comprising:

17. The apparatus of claim 16, wherein the biometric information comprises a palm print image, the determining unit comprising:

the acquisition subunit is used for acquiring the palmprint image of the target palm when the vehicle door opening and closing assembly is triggered;

The identification subunit is used for carrying out palm print identification processing based on the palm print image to obtain the palm type of the target palm, wherein the palm type is used for indicating whether the target palm is a left palm or a right palm;

and the mode determining subunit is used for determining the door opening mode according to the palm type.

18. A vehicle comprising a memory and a processor; the memory stores a computer program, and the processor is configured to execute the computer program in the memory to perform the door opening control method of the vehicle according to any one of claims 1 to 15.

19. A computer-readable storage medium storing a computer program loaded by a processor to perform the door opening control method of the vehicle according to any one of claims 1 to 15.