Disclosure of Invention
The embodiment of the invention aims to provide a vehicle-mounted refrigerator control method, a vehicle-mounted refrigerator control device, vehicle-mounted refrigerator control equipment and a storage medium, which can automatically control a vehicle-mounted refrigerator based on user behaviors and do not need a user to approach the vehicle-mounted refrigerator and manually control the vehicle-mounted refrigerator, so that user experience is improved.
In order to achieve the above object, an embodiment of the present invention provides a method for controlling a vehicle-mounted refrigerator, including:
when the preset camera shooting condition is met, controlling an in-vehicle camera to shoot the current user behavior of the in-vehicle user to obtain a user behavior image;
based on a preset user behavior recognition analysis model, recognizing the user behavior of the user behavior image to obtain recognized user behavior;
when the identified user behavior is a preset user behavior type, acquiring a preset control strategy of the vehicle-mounted refrigerator corresponding to the user behavior;
and controlling the vehicle-mounted refrigerator according to the control strategy.
As an improvement of the above scheme, the obtaining of the preset control strategy of the vehicle-mounted refrigerator corresponding to the user behavior includes:
carrying out user identification on the in-vehicle user in the user behavior image, and determining the user type of the in-vehicle user; the user types comprise new users and historical users;
when the user in the vehicle is a new user, executing the following control strategy acquisition mode: acquiring a control authority preset for a new user; finding a control strategy corresponding to the user behavior from a control strategy set mapped in advance with the control authority;
when the user in the vehicle is a historical user, executing the following control strategy acquisition mode: analyzing the character image of the historical user and acquiring a control authority preset for the historical user; finding a control strategy corresponding to the user behavior from a control strategy set mapped with the portrait and the control authority in advance; the character representation includes at least one of: personality traits, control preferences for the on-board refrigerator.
As an improvement of the above solution, the finding a control policy corresponding to the user behavior from a control policy set mapped in advance with the portrait and the control authority includes:
analyzing the behavior action amplitude of the user behavior;
determining the level of the action amplitude of the behavior in a preset action amplitude step;
and finding a control strategy corresponding to the level of the action amplitude of the user action from a control strategy set mapped in advance with the portrait and the control authority.
As an improvement of the above scheme, if the user behaviors identified in the same user behavior image have user behaviors of at least two different in-vehicle users, the obtaining of the preset control policy of the vehicle-mounted refrigerator corresponding to the user behavior when the identified user behavior is of the preset user behavior type includes:
when the at least two identified user behaviors are both preset user behavior types, judging whether the at least two user behaviors have a preset conflict relationship based on preset user behavior conflict judgment logic;
if yes, executing the following control strategy acquisition mode: performing user identification on the in-vehicle users in the user behavior image, and identifying the in-vehicle users corresponding to the at least two user behaviors; analyzing the priorities of control authorities preset by at least two in-vehicle users; acquiring a control strategy of the vehicle-mounted refrigerator corresponding to the user behavior of the user in the vehicle with the highest priority;
if not, executing the following control strategy acquisition mode: performing user identification on the in-vehicle users in the user behavior image, and identifying the in-vehicle users corresponding to the at least two user behaviors; analyzing the priorities of at least two control authorities preset by the in-vehicle users; the method comprises the steps of obtaining control strategies of the vehicle-mounted refrigerator corresponding to the user behaviors of at least two in-vehicle users, and setting execution sequences of the control strategies corresponding to the user behaviors of the at least two in-vehicle users according to the priority of the control authority of the at least two in-vehicle users.
As an improvement of the above scheme, the vehicle-mounted refrigerator comprises a refrigerator body, a shelf body arranged in the refrigerator body and used for placing articles, a driving device used for driving the shelf body to extend out of and retract into the refrigerator body, and a control device used for controlling the driving device; the control strategy then comprises a control strategy for the drive device, the control strategy for the drive device comprising at least one of:
the driving device drives the shelf body to extend out of the refrigerator body at a speed;
the driving device drives the rack body to retract to the speed of the refrigerator body.
As an improvement of the above solution, the user behavior includes a limb movement of the user; then, the recognizing the user behavior of the user behavior image based on the preset user behavior recognition analysis model includes:
preprocessing the user behavior image;
inputting the preprocessed user behavior images into a pre-trained user behavior recognition analysis model to obtain the action postures of the limbs of the user; the user behavior recognition analysis model is obtained by training based on a plurality of user behavior image training samples in advance, and the user behavior image training samples contain posture information of the body motions of the user.
As an improvement of the above scheme, the camera shooting condition includes at least one of:
the method comprises the steps of detecting a photographing voice command sent by a user, detecting that a photographing control button in a vehicle is triggered, detecting a cup touching command of the user and detecting a door opening action of the vehicle-mounted refrigerator.
As an improvement of the above, the method further comprises:
when a preset automatic door opening condition is met, controlling the door of the vehicle-mounted refrigerator to open, and controlling the driving device to drive the shelf body to extend out; wherein, the automatic opening condition of the box door meets: the current date is a preset target date, the current time is in a preset target time period, the position of the vehicle is in a preset target range, and the vehicle is in a preset gear;
and when the box door is opened, controlling a voice playing device of the vehicle to play voice information corresponding to the target date.
As an improvement of the above solution, when the door of the in-vehicle refrigerator is opened, the method further includes:
detecting scene conditions of a user in a vehicle;
acquiring a corresponding work to be played according to the scene condition;
and controlling a voice playing device in the vehicle to play the works to be played.
In order to achieve the above object, an embodiment of the present invention further provides a vehicle-mounted refrigerator control apparatus, including:
the camera starting module is used for controlling the camera in the automobile to shoot the current user behavior of the user in the automobile when the preset camera shooting condition is met, so as to obtain a user behavior image;
the user behavior recognition module is used for recognizing the user behavior of the user behavior image based on a preset user behavior recognition analysis model to obtain recognized user behavior;
the control strategy acquisition module is used for acquiring a control strategy of the vehicle-mounted refrigerator, which is preset and corresponds to the user behavior, when the identified user behavior is the preset user behavior type;
and the control module is used for controlling the vehicle-mounted refrigerator according to the control strategy.
In order to achieve the above object, an embodiment of the present invention further provides an in-vehicle refrigerator control apparatus, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, where the processor implements the in-vehicle refrigerator control method according to any one of the above embodiments when executing the computer program.
In order to achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program, and when the computer program runs, an apparatus in which the computer-readable storage medium is located is controlled to execute the method for controlling an on-vehicle refrigerator according to any of the above embodiments.
Compared with the prior art, the vehicle-mounted refrigerator control method, the vehicle-mounted refrigerator control device, the vehicle-mounted refrigerator control equipment and the storage medium disclosed by the embodiment of the invention analyze the behavior of the user in the vehicle based on the image recognition technology, acquire the control strategy of the corresponding vehicle-mounted refrigerator according to the recognized user behavior belonging to the preset user behavior type, and finally control the work of the vehicle-mounted refrigerator according to the control strategy. Therefore, the control requirement of the user in the vehicle on the vehicle-mounted refrigerator can be identified according to the user behavior, the work of the vehicle-mounted refrigerator is automatically controlled, the user does not need to approach the vehicle-mounted refrigerator and manually control and operate the vehicle-mounted refrigerator, and the user experience is improved.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a method for controlling an on-board refrigerator according to an embodiment of the present invention, where the method includes:
s1, when a preset camera shooting condition is met, controlling an in-car camera to shoot the current user behavior of a user in a car to obtain a user behavior image;
s2, based on a preset user behavior recognition analysis model, recognizing the user behavior of the user behavior image to obtain recognized user behavior;
s3, when the identified user behavior is a preset user behavior type, acquiring a preset control strategy of the vehicle-mounted refrigerator corresponding to the user behavior;
and S4, controlling the vehicle-mounted refrigerator according to the control strategy.
It is worth to be noted that the vehicle-mounted refrigerator control method according to the embodiment of the present invention is implemented by a vehicle-mounted controller in a vehicle, where the vehicle-mounted controller is a front-end device of a vehicle monitoring and management system, integrates multiple functions such as data processing and data communication, and has a strong service scheduling function and data processing capability. The vehicle-mounted controller is in communication connection with a camera in a vehicle and the vehicle-mounted refrigerator in advance. The vehicle-mounted controller analyzes the behavior of the user in the vehicle based on an image recognition technology, acquires a control strategy of the corresponding vehicle-mounted refrigerator according to the recognized user behavior belonging to the preset user behavior type, and finally controls the work of the vehicle-mounted refrigerator according to the control strategy.
Specifically, in step S1, the camera shooting condition includes at least one of:
the method comprises the steps of detecting a photographing voice command sent by a user, detecting that a photographing control button in a vehicle is triggered, detecting a cup touching command of the user and detecting a door opening action of the vehicle-mounted refrigerator.
For example, the photographing voice instruction may be triggered by any in-vehicle user, such as a "camera start" voice instruction, a "start photographing" voice instruction, and it can be understood that when the voice photographing instruction is detected, the camera starts immediately to start acquiring the user behavior of the in-vehicle user in real time, and when the photographing end voice instruction is received, the camera is turned off. When a user exists in the vehicle, starting to detect whether a cup touching instruction of the user exists, wherein the cup touching instruction is collision sound between cups or between the cups and other objects, and when the cup touching instruction is detected, the user is indicated to need to drink water at the moment, the vehicle-mounted refrigerator is possibly required to be opened, the camera is opened at the moment, and the user behavior of the user is acquired in real time; further, the camera may be turned off after a period of time after the camera is turned on.
Specifically, in step S2, the user behavior recognition analysis model may be a CNN (Convolutional neural network) model. The CNN is a feedforward neural network, comprises a convolutional layer and a pooling layer, and has wide application in the field of image recognition. Theoretically, as long as there are enough training samples, the CNN model with the corresponding recognition function can be obtained, the training samples refer to the original data marked with the recognition result, and in the user gesture recognition scene, the training samples are images marked with the gesture information of the user body movement. For example, if the body motion is a gesture, the posture information of the body motion includes a specific gesture, such as a circle, a heart, a square, or a triangle, etc. that is made by hand.
Further, in the process of training the user behavior recognition analysis model, in order to improve the recognition accuracy and the convergence rate of the model, before a training sample is input to the model to be trained, the training sample may be preprocessed, specifically, the training sample is subjected to preliminary feature extraction, so as to improve the convergence rate of the model by reducing the amount of computation.
Optionally, the user behavior comprises a limb movement of the user; then, the identifying the user behavior for the user behavior image based on the preset user behavior identification analysis model includes steps S21 to S22:
s21, preprocessing the user behavior image;
s22, inputting the preprocessed user behavior image into a pre-trained user behavior recognition analysis model to obtain the action posture of the user limb; the user behavior recognition analysis model is obtained by training in advance based on a plurality of user behavior image training samples, and the user behavior image training samples contain posture information of the body motions of the user.
It is understood that the preprocessing process may be an image processing operation such as gray processing, binarization processing, clipping, and the like, performed on the user behavior image. The user behavior may also be a facial expression of the user, e.g. a smiling face, frowning, etc. Similarly, the process of identifying and analyzing the facial expression of the user may refer to the process of identifying and analyzing the body movement of the user, which is not described in detail herein.
Specifically, in step S3, each user behavior type capable of representing the control requirement of the user for the in-vehicle refrigerator may be mapped and stored with the control policy of the in-vehicle refrigerator corresponding to the user behavior type in advance, and the mapping relationship may be stored in a preset mapping list. When the user behavior type is analyzed, the control strategy of the vehicle-mounted refrigerator corresponding to the user behavior type is searched in the mapping list, and the control function of the vehicle-mounted refrigerator, which needs to be executed currently, is determined.
Optionally, the vehicle-mounted refrigerator comprises a refrigerator body, a shelf body arranged in the refrigerator body and used for placing articles, a driving device used for driving the shelf body to extend out of and retract into the refrigerator body, and a control device used for controlling the driving device; the control strategy then comprises a control strategy for the drive device, the control strategy for the drive device comprising at least one of:
the driving device drives the shelf body to extend out of the refrigerator body at a speed;
the driving device drives the rack body to retract to the speed of the refrigerator body.
Furthermore, the vehicle-mounted refrigerator also comprises an atmosphere lamp arranged in the refrigerator body or outside the refrigerator body and a control device for controlling the operation of the atmosphere lamp; then, the control strategy may further include: control strategy for the atmosphere lamp.
Optionally, the obtaining of the preset control policy of the vehicle-mounted refrigerator corresponding to the user behavior includes steps S31 to S33:
s31, carrying out user identification on the in-vehicle user in the user behavior image, and determining the user type of the in-vehicle user; the user types comprise new users and historical users;
s32, when the user in the vehicle is a new user, executing the following control strategy acquisition mode: acquiring a control authority preset for a new user; finding a control strategy corresponding to the user behavior from a control strategy set mapped in advance with the control authority;
s33, when the user in the vehicle is a historical user, executing the following control strategy acquisition mode: analyzing the figure picture of the historical user and acquiring a control authority preset for the historical user; finding a control strategy corresponding to the user behavior from a control strategy set mapped with the portrait and the control authority in advance; the portrait representation includes at least one of: personality traits, control preferences for on-vehicle refrigerators.
Specifically, in the process of performing identification analysis on the user behavior, the type of the user in the vehicle needs to be determined. When the user in the vehicle is a new user, the control authority level of the new user is lower, more control functions cannot be executed, and specific functions which can be executed can be preset by a vehicle owner, which is not specifically limited by the invention. Further, the gender of the new user can be judged in the process, and when the new user is a female user, the control habit of the female user in other historical users can be combined to obtain a control strategy corresponding to the control habit; similarly, when the new user is a male user, the control habit of the male user in other historical users can be combined to obtain the control strategy corresponding to the new user.
When the user in the vehicle is a historical user, the control authority level of the historical user is higher, all control functions can be basically executed, and specific functions can be preset by a vehicle owner, which is not specifically limited by the invention. For example, since the historical user is a user who has previously analyzed user behavior, the historical user behavior of the historical user is stored, and the character and the control preference of the historical user for the refrigerator can be analyzed based on the historical user behavior, and the control strategy corresponding to the character and the control preference can be acquired.
Optionally, the step S33 of finding the control policy corresponding to the user behavior from the control policy set pre-mapped with the portrait and the control authority includes steps S331 to S333:
s331, analyzing the behavior action amplitude of the user behavior;
s332, determining the level of the action amplitude of the behavior in a preset action amplitude step;
s333, finding a control strategy corresponding to the level of the behavior action amplitude of the user behavior from a control strategy set mapped in advance with the portrait and the control authority.
As an example, if the user behavior is a victory "V" gesture, an included angle between two fingers of the "V" gesture is analyzed, and the included angle represents a motion amplitude of the "V" gesture. Three action amplitude steps corresponding to the "V" gesture are preset locally, for example, the three action amplitude steps are respectively: the included angle is less than 10 degrees, the included angle is more than or equal to 10 degrees and less than 20 degrees, and the included angle is more than 20 degrees. When the figure image is a control preference of the vehicle-mounted refrigerator, the figure image of the user in the vehicle corresponds to: the control preference of the vehicle-mounted refrigerator is that the pushing speed of the shelf of the vehicle-mounted refrigerator is high; the control authority of the user in the vehicle comprises the following steps: refrigeration control authority to on-vehicle refrigerator and to the release control authority of on-vehicle refrigerator's supporter. The control strategy set corresponding to the figure image and the control authority of the user in the vehicle comprises a refrigeration strategy set of the vehicle-mounted refrigerator and a quick push-out control strategy set of a commodity shelf of the vehicle-mounted refrigerator, the control strategy corresponding to the 'V' gesture of the user behavior is the quick push-out control strategy set of the commodity shelf of the vehicle-mounted refrigerator, and the push-out control strategy set comprises three quick push-out control strategies of the commodity shelf of the vehicle-mounted refrigerator and respectively corresponds to the included angle levels of the three 'V' gestures. The included angle level of the 'V' gesture is less than 10 degrees, and the corresponding push-out control strategy is that the push-out speed of the commodity shelf is 0.05 meters per second as an example; the included angle level of the 'V' gesture is greater than or equal to 10 degrees and less than 20 degrees, and the corresponding push-out control strategy is that the push-out speed of the commodity shelf is 0.08 meters per second as an example; the "V" gesture has an included angle level greater than 20 degrees, which corresponds to a push-out control strategy of 0.1 meters per second for an example push-out speed of the rack.
As another example, the user behavior is a smiling expression of the user facial expression, and a smiling radian of the smiling expression, which represents an action amplitude of the smiling expression, is analyzed. Two smile radian steps corresponding to the smile expressions are preset locally, for example, the smile radians are smaller than 40 degrees respectively, and the smile radians are equal to or larger than 40 degrees. Wherein, this in-car user's control authority includes: the control authority to the display screen of on-vehicle refrigerator and the control authority to the atmosphere lamp of on-vehicle refrigerator. The figure image comprises user characters, and the user characters of the user in the vehicle correspond to: the control strategy set corresponding to the figure image and the control authority of the user in the vehicle is provided with a control strategy set of the display screen of the vehicle-mounted refrigerator and an atmosphere lamp control strategy set of the vehicle-mounted refrigerator, the control strategy set corresponding to the smiling expression of the user is an atmosphere lamp control strategy set of the vehicle-mounted refrigerator, the atmosphere lamp control strategy set comprises atmosphere lamp control strategies of two vehicle-mounted refrigerators and corresponds to the two smiling radian steps respectively. Smile arc steps less than 40 degrees, which corresponds to an ambience light control strategy with an example of red flashing frequency of the ambience light of 5 hz per second; smile arc steps equal to or greater than 40 degrees, which corresponds to an ambience light control strategy with an exemplary flashing frequency of the red color of the ambience light of 10 hertz per second.
In the embodiment of the invention, the control strategy corresponding to the user behavior is found through the portrait and the control authority, so that the control on the vehicle-mounted refrigerator can better meet the user requirements of the user, and the use experience of the user is improved.
Optionally, if the user behaviors identified in the same user behavior image include user behaviors of at least two different in-vehicle users, step S3 includes steps S301 to S303:
s301, when the at least two recognized user behaviors are all preset user behavior types, judging whether the at least two user behaviors have a preset conflict relationship or not based on preset user behavior conflict judgment logic;
s302, if yes, executing the following control strategy acquisition mode: performing user identification on the in-vehicle users in the user behavior image, and identifying the in-vehicle users corresponding to the at least two user behaviors; analyzing the priorities of control authorities preset by at least two in-vehicle users; acquiring a control strategy of the vehicle-mounted refrigerator corresponding to the user behavior of the user in the vehicle with the highest priority;
s303, if not, executing the following control strategy acquisition mode: performing user identification on the in-vehicle users in the user behavior image, and identifying the in-vehicle users corresponding to the at least two user behaviors; analyzing the priorities of at least two control authorities preset by the in-vehicle users; the method comprises the steps of obtaining control strategies of the vehicle-mounted refrigerator corresponding to user behaviors of at least two in-vehicle users, and setting execution sequences of the control strategies corresponding to the user behaviors of the at least two in-vehicle users according to the priority of the control authority of the at least two in-vehicle users.
Specifically, the user behavior conflict logic includes a set of conflict relationships between preset user behaviors. For example, the user behavior of the OK gesture and the user behavior of the gesture for holding up the thumb are preset in a conflict relationship, wherein the control strategy of the vehicle-mounted refrigerator corresponding to the user behavior of the OK gesture is to start the refrigeration function of the vehicle-mounted refrigerator, and the control strategy of the vehicle-mounted refrigerator corresponding to the user behavior of the gesture for holding up the thumb is to close the refrigeration function of the vehicle-mounted refrigerator. For another example, the user behavior of the V gesture and the user behavior of the smiling expression have a preset conflict relationship, the control policy of the on-board refrigerator corresponding to the user behavior of the V gesture is to turn on an atmosphere lamp of the on-board refrigerator, and the control policy of the on-board refrigerator corresponding to the user behavior of the smiling expression is to turn off the atmosphere lamp of the on-board refrigerator.
As an example, when two in-vehicle users make respective user behaviors in the user behavior image captured by the in-vehicle camera, the user behavior image is identified based on a preset user behavior identification analysis model, so that the two identified user behaviors can be obtained and respectively correspond to the two in-vehicle users. At the moment, judging whether the two identified user behaviors are preset user behavior types or not; and if the two identified user behaviors are both the preset user behavior types, judging whether the two user behaviors have a preset conflict relationship or not based on a preset user behavior conflict judgment logic. If the two user behaviors are an OK gesture and a thumb-holding gesture respectively, judging that the two user behaviors have a preset conflict relationship, and identifying the in-vehicle user in the user behavior image to identify the in-vehicle user corresponding to the two user behaviors; and analyzing the priorities of the control authorities preset by the two in-vehicle users, wherein the priority of the in-vehicle user corresponding to the OK gesture is higher than the priority of the in-vehicle user holding the thumb gesture, and then acquiring the control strategy of the vehicle-mounted refrigerator corresponding to the user behavior of the in-vehicle user with the high priority, which is the OK gesture. If the two user behaviors are an OK gesture and a V gesture respectively, judging that the two user behaviors do not have a preset conflict relationship, and identifying the user in the vehicle in the user behavior image to identify the user in the vehicle corresponding to the two user behaviors; analyzing the priority of the control authority preset by the two in-vehicle users; acquiring control strategies of the vehicle-mounted refrigerator corresponding to the user behaviors of the two in-vehicle users, and setting an execution sequence of the control strategies corresponding to the user behaviors of the two in-vehicle users according to the priority of the control authority of the two in-vehicle users, for example, if the priority of the in-vehicle user corresponding to the OK gesture is higher than the priority of the in-vehicle user corresponding to the V gesture, the execution sequence of the control strategies of the two user behaviors is as follows: and executing the control strategy corresponding to the user behavior of the OK gesture, and then executing the control strategy corresponding to the user behavior of the V gesture.
The embodiment of the invention judges whether at least two user behaviors have a preset conflict relationship or not based on the preset user behavior conflict judgment logic, and only acquires the control strategy corresponding to the user behavior of the user in the vehicle with the highest priority to control the vehicle-mounted refrigerator when the preset conflict relationship is judged, so that the control logic of the vehicle-mounted refrigerator is prevented from being disordered to prevent the vehicle-mounted refrigerator from being normally controlled; and when the situation that the preset conflict relationship does not exist is judged, setting the execution sequence of the control strategies corresponding to the at least two user behaviors according to the priorities of the in-vehicle users corresponding to the at least two user behaviors, so that the normal control of the vehicle-mounted refrigerator can be realized based on the control requirements of all in-vehicle users who make the user behaviors analyzed through the user behaviors.
Further, the vehicle-mounted refrigerator control method further comprises the steps of S101-S102:
s101, when a preset automatic door opening condition is met, controlling the door of the vehicle-mounted refrigerator to be opened, and controlling the driving device to drive the shelf body to extend out; wherein, the automatic opening condition of the box door meets: the current date is a preset target date, the current time is in a preset target time period, the position of the vehicle is in a preset target range, and the vehicle is in a preset gear;
and S102, controlling a voice playing device of the vehicle to play voice information corresponding to the target date when the box door is opened.
As an example, the target date may be a particular festival, such as a day of the year, the middle of the autumn, the christmas, etc., or a particular memorial day, such as a user's birthday; the target time period can be the off-duty time of the user and can be set to five to twelve o' clock in the afternoon; the target range may be a parking lot in the home of the user; the preset gear may be a P gear, which indicates that the user has stopped the vehicle. When the box door is opened, the voice playing device is controlled to play voice information corresponding to the target date, for example, when the target date is the birthday of the user, the voice information of 'wish you happy birthday, champagne is ready and you please enjoy' can be played.
In the embodiment of the invention, the refrigerator can be automatically started according to the application scene preset by the user, and the voice information is played, so that the control on the vehicle-mounted refrigerator can better meet the user requirements of the user, and the use experience of the user is improved.
Furthermore, when the user in the vehicle opens the door of the vehicle-mounted refrigerator on a daily basis, the method further comprises steps S201 to S203:
s201, detecting scene conditions of a user in the vehicle;
s202, acquiring a corresponding work to be played according to the scene condition;
s203, controlling a voice playing device in the vehicle to play the works to be played.
In one embodiment, the scene condition is positioning information, and the work to be played is poetry. When a user opens the refrigerator door, the situation that the user needs to relax properly at the moment is indicated, the scene condition where the user is located in the vehicle is detected, and if the situation that the positioning information of the user in the vehicle is detected to be a certain scenic spot, poems related to the scenic spot are obtained and played, and the user can enjoy the poems while enjoying champagne.
In another embodiment, the scene condition is an emotional state, which may include sad emotion, cheerful emotion, mild emotion, etc.; the work to be played is music. The emotional state of the user in the vehicle can be obtained by analyzing the user picture acquired by the camera in the vehicle. When the user opens the refrigerator door, the user picture is analyzed, and when the emotional state of the user is in the sad emotion, relaxed music can be obtained, so that the function of relieving the emotion of the user is achieved.
In the embodiment of the invention, when the user opens the refrigerator door, the corresponding works can be played in combination with the scene condition of the user, so that the use experience of the user in using the vehicle-mounted refrigerator is improved.
Compared with the prior art, the vehicle-mounted refrigerator control method disclosed by the embodiment of the invention analyzes the behavior of the user in the vehicle based on the image recognition technology, acquires the corresponding control strategy of the vehicle-mounted refrigerator according to the recognized user behavior belonging to the preset user behavior type, and finally controls the work of the vehicle-mounted refrigerator according to the control strategy. Therefore, the control requirement of the user in the vehicle on the vehicle-mounted refrigerator can be identified according to the user behavior, the work of the vehicle-mounted refrigerator is automatically controlled, the user does not need to approach the vehicle-mounted refrigerator and manually control and operate the vehicle-mounted refrigerator, and the user experience is improved.
Referring to fig. 2, fig. 2 is a block diagram of a structure of an in-vehicle refrigerator control apparatus 10 according to an embodiment of the present invention, where the in-vehicle refrigerator control apparatus 10 includes:
the camera starting module 11 is used for controlling the in-vehicle camera to shoot the current user behavior of the in-vehicle user when the preset camera shooting condition is met, so as to obtain a user behavior image;
the user behavior recognition module 12 is configured to recognize a user behavior of the user behavior image based on a preset user behavior recognition analysis model, so as to obtain a recognized user behavior;
the control strategy acquisition module 13 is configured to acquire a preset control strategy of the vehicle-mounted refrigerator corresponding to the user behavior when the identified user behavior is a preset user behavior type;
and the control module 14 is used for controlling the vehicle-mounted refrigerator according to the control strategy.
It should be noted that the vehicle-mounted refrigerator control apparatus 10 according to the embodiment of the present invention is a vehicle-mounted controller in a vehicle, and the vehicle-mounted controller is a front-end device of a vehicle monitoring management system, integrates multiple functions such as data processing and data communication, and has a strong service scheduling function and a strong data processing capability. The vehicle-mounted controller is in communication connection with a camera in a vehicle and the vehicle-mounted refrigerator in advance. The vehicle-mounted refrigerator control device 10 analyzes the behavior of the user in the vehicle based on the image recognition technology, obtains the control strategy of the corresponding vehicle-mounted refrigerator according to the recognized user behavior belonging to the preset user behavior type, and finally controls the work of the vehicle-mounted refrigerator according to the control strategy.
Optionally, the camera shooting condition includes at least one of the following:
the method comprises the steps of detecting a photographing voice command sent by a user, detecting that a photographing control button in a vehicle is triggered, detecting a cup touching command of the user and detecting a door opening action of the vehicle-mounted refrigerator.
Alternatively, the user behavior recognition analysis model may be a CNN (Convolutional neural network) model. In the training process of the user behavior recognition analysis model, in order to improve the recognition accuracy and the convergence rate of the model, the training sample may be preprocessed before being input to the model to be trained, specifically, the training sample is subjected to preliminary feature extraction, so as to improve the convergence rate of the model by reducing the amount of computation.
Optionally, the user behavior comprises a limb movement of the user; then, the user behavior recognition module 12 is configured to:
preprocessing the user behavior image;
inputting the preprocessed user behavior images into a pre-trained user behavior recognition analysis model to obtain the action postures of the limbs of the user; the user behavior recognition analysis model is obtained by training based on a plurality of user behavior image training samples in advance, and the user behavior image training samples contain posture information of the body motions of the user.
It is understood that the preprocessing process may be an image processing operation such as grayscale processing, binarization processing, clipping, and the like, performed on the user behavior image. The user behavior may also be a facial expression of the user, e.g., smiling face, frown, etc. Similarly, the process of identifying and analyzing the facial expression of the user may refer to the process of identifying and analyzing the body movement of the user, which is not described in detail herein.
Optionally, the vehicle-mounted refrigerator comprises a refrigerator body, a shelf body arranged in the refrigerator body and used for placing articles, a driving device used for driving the shelf body to extend out of and retract into the refrigerator body, and a control device used for controlling the driving device; the control strategy then comprises a control strategy for the drive device comprising at least one of:
the driving device drives the shelf body to extend out of the refrigerator body at a speed;
the driving device drives the rack body to retract to the speed of the refrigerator body.
Furthermore, the vehicle-mounted refrigerator also comprises an atmosphere lamp arranged in the refrigerator body or outside the refrigerator body and a control device for controlling the operation of the atmosphere lamp; then, the control strategy may further include: a control strategy for the atmosphere lamp.
Optionally, the control policy obtaining module 13 is configured to:
carrying out user identification on the in-vehicle user in the user behavior image, and determining the user type of the in-vehicle user; the user types comprise new users and historical users;
when the user in the vehicle is a new user, executing the following control strategy acquisition mode: acquiring a control authority preset for a new user; finding a control strategy corresponding to the user behavior from a control strategy set mapped in advance with the control authority;
when the user in the vehicle is a historical user, executing the following control strategy acquisition mode: analyzing the character image of the historical user and acquiring a control authority preset for the historical user; finding a control strategy corresponding to the user behavior from a control strategy set mapped with the portrait and the control authority in advance; the character representation includes at least one of: personality traits, control preferences for the on-board refrigerator.
Optionally, the finding a control policy corresponding to the user behavior from a control policy set pre-mapped with the portrait and the control authority includes:
analyzing the behavior action amplitude of the user behavior;
determining the level of the action amplitude of the behavior in a preset action amplitude step;
and finding a control strategy corresponding to the level of the action amplitude of the user action from a control strategy set mapped in advance with the portrait and the control authority.
Optionally, if the user behavior identified in the same user behavior image includes user behaviors of at least two different in-vehicle users, the control policy obtaining module 13 is further configured to:
when the at least two recognized user behaviors are all preset user behavior types, judging whether the at least two user behaviors have a preset conflict relationship or not based on preset user behavior conflict judgment logic;
if yes, executing the following control strategy acquisition mode: performing user identification on the in-vehicle users in the user behavior image, and identifying the in-vehicle users corresponding to the at least two user behaviors; analyzing the priorities of control authorities preset by at least two in-vehicle users; acquiring a control strategy of the vehicle-mounted refrigerator corresponding to the user behavior of the user in the vehicle with the highest priority;
if not, executing the following control strategy acquisition mode: performing user identification on the in-vehicle users in the user behavior image, and identifying the in-vehicle users corresponding to the at least two user behaviors; analyzing the priorities of at least two control authorities preset by the in-vehicle users; the method comprises the steps of obtaining control strategies of the vehicle-mounted refrigerator corresponding to user behaviors of at least two in-vehicle users, and setting execution sequences of the control strategies corresponding to the user behaviors of the at least two in-vehicle users according to the priority of the control authority of the at least two in-vehicle users.
Further, the control module 14 is further configured to:
when a preset automatic door opening condition is met, controlling the door of the vehicle-mounted refrigerator to open, and controlling the driving device to drive the storage rack body to extend out; wherein, the automatic opening condition of the box door meets: the current date is a preset target date, the current time is in a preset target time period, the position of the vehicle is in a preset target range, and the vehicle is in a preset gear;
and when the box door is opened, controlling a voice playing device of the vehicle to play voice information corresponding to the target date.
Still further, the control module 14 is further configured to:
detecting scene conditions of a user in a vehicle;
acquiring a corresponding work to be played according to the scene condition;
and controlling a voice playing device in the vehicle to play the works to be played.
It should be noted that please refer to the working process of the vehicle-mounted refrigerator control method according to the above embodiment for the specific working process of each module in the vehicle-mounted refrigerator control apparatus 10, which is not described herein again.
Compared with the prior art, the vehicle-mounted refrigerator control device 10 disclosed by the embodiment of the invention analyzes the behavior of the user in the vehicle based on the image recognition technology, acquires the control strategy of the corresponding vehicle-mounted refrigerator according to the recognized user behavior belonging to the preset user behavior type, and finally controls the work of the vehicle-mounted refrigerator according to the control strategy. Therefore, the control requirement of the user in the vehicle on the vehicle-mounted refrigerator can be identified according to the user behavior, the work of the vehicle-mounted refrigerator is automatically controlled, the user does not need to approach the vehicle-mounted refrigerator and manually control and operate the vehicle-mounted refrigerator, and the user experience is improved.
Referring to fig. 3, fig. 3 is a block diagram of a structure of an in-vehicle refrigerator control apparatus 20 according to an embodiment of the present invention, where the in-vehicle refrigerator control apparatus 20 includes: a processor 21, a memory 22 and a computer program, such as a driving control program, stored in said memory and executable on said processor. The processor 21, when executing the computer program, implements the steps in the above-described in-vehicle refrigerator control method embodiment, such as steps S1 to S4 shown in fig. 1. Alternatively, the processor implements the functions of the modules in the above device embodiments when executing the computer program.
Illustratively, the computer program may be divided into one or more modules, which are stored in the memory 22 and executed by the processor 21 to accomplish the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program in the in-vehicle refrigerator control apparatus 20. For example, the computer program may be divided into a camera activation module 11, a user behavior recognition module 12, a control policy acquisition module 13, and a control module 14. For the specific working process of each module, reference may be made to the working process of the vehicle-mounted refrigerator control device 10 described in the foregoing embodiment, and details are not repeated here.
The vehicle-mounted refrigerator control device 20 may be a desktop computer, a notebook computer, a palm computer, a cloud server, or other computing devices. The in-vehicle refrigerator control apparatus 20 may include, but is not limited to, a processor 21, a memory 22. It will be understood by those skilled in the art that the schematic diagram is merely an example of an image enhancement device and does not constitute a limitation of the in-vehicle refrigerator control device 20, and may include more or less components than those shown, or combine some components, or different components, for example, the in-vehicle refrigerator control device 20 may further include an input-output device, a network access device, a bus, etc.
The Processor 21 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general processor may be a microprocessor or the processor may be any conventional processor, etc., and the processor 21 is a control center of the vehicle-mounted refrigerator control apparatus 20 and connects various parts of the entire vehicle-mounted refrigerator control apparatus 20 by various interfaces and lines.
The memory 22 may be used to store the computer programs and/or modules, and the processor 21 implements various functions of the in-vehicle refrigerator control apparatus 20 by operating or executing the computer programs and/or modules stored in the memory 22 and calling data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory 22 may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
Among them, the modules integrated by the in-vehicle refrigerator control apparatus 20 may be stored in a computer readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.