CN115658233A - Method and device for operating a driver monitoring system and vehicle - Google Patents
Method and device for operating a driver monitoring system and vehicle Download PDFInfo
- Publication number
- CN115658233A CN115658233A CN202211352744.0A CN202211352744A CN115658233A CN 115658233 A CN115658233 A CN 115658233A CN 202211352744 A CN202211352744 A CN 202211352744A CN 115658233 A CN115658233 A CN 115658233A
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- driver
- privileged
- action information
- user virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
- Stored Programmes (AREA)
Abstract
The application provides a method, a device and a vehicle for operating a driver monitoring system, wherein the method comprises the following steps: operating a privileged virtual machine and a user virtual machine based on the allocation result of the operation resources, wherein the privileged virtual machine is used for collecting the action information of a driver in the vehicle, and the operation resources comprise a core of a processor of the gateway and an operation memory of the gateway; running a driver monitoring system on a user virtual machine; and sending the action information of the driver to the user virtual machine through the privileged virtual machine so that the driver monitoring system processes the action information of the driver. The method integrates the driver monitoring system into the gateway equipment through a virtualization method, reduces the consumption of hardware in the gateway under the condition of not influencing the normal function of the driver monitoring system, and improves the processing efficiency of the driver monitoring system.
Description
Technical Field
The present application relates to the field of vehicle virtualization, and more particularly, to a method, apparatus and vehicle for operating a driver monitoring system in the field of vehicle virtualization.
Background
At present, in the vehicle industry, along with the development of internet technology and science and technology, the safety problem of a user in the process of driving a vehicle is guaranteed importantly.
For example, in the process of driving the vehicle, the Driver Monitoring System (DMS) disposed in the vehicle may be used to Monitor the Driver's actions during driving, further analyze the Driver's actions (for example, whether the Driver is tired, makes a call, or eats something that affects the driving), and determine whether the Driver has a potential safety hazard during driving the vehicle.
In a possible scenario, in the process of processing and judging the actions of the driver, the DMS processes and compares the actions of the driver collected by a camera in the vehicle based on a deep neural network model and a large number of Artificial Intelligence (AI) algorithms to obtain a judgment result.
In the above process, because a large number of algorithms and neural network models are involved in the DMS system, the DMS consumes more hardware of the system, and the computational efficiency of the system is affected.
Disclosure of Invention
The method integrates the DMS into the gateway equipment through a virtualization method, reduces the consumption of hardware in the gateway under the condition of not influencing the normal function of the DMS, and improves the processing efficiency of the DMS.
In a first aspect, there is provided a method of operating a driver monitoring system, the method being performed by a gateway operating a virtualization platform, the method comprising: operating a privileged virtual machine and a user virtual machine based on an allocation result of an operation resource, wherein the privileged virtual machine is used for collecting action information of a driver in a vehicle, and the operation resource comprises a core of a processor of the gateway and an operation memory of the gateway; running the driver monitoring system on the user virtual machine; and sending the action information of the driver to the user virtual machine through the privileged virtual machine so that the driver monitoring system processes the action information of the driver.
In the above technical solution, a scheme for virtualizing a gateway or a gateway device is provided, so as to implement a process of virtualizing a DMS. The method comprises the steps of operating the privileged virtual machine and the user virtual machine according to the operation resources respectively allocated to the privileged virtual machine and the user virtual machine, reasonably allocating gateway resources and ensuring the normal operation of the privileged virtual machine and the user virtual machine. And further, a driver monitoring system is operated in the user virtual machine, so that the effect of virtualizing the DMS is realized, and the consumption of system hardware is avoided. And finally, the action information of the driver can be sent to the user virtual machine through the privileged virtual machine, so that the action information of the driver is processed by the driver monitoring system, and the process of processing the action information of the driver is realized under the condition of not influencing the hardware loss of the system by combining the process of the DMS virtualization.
With reference to the first aspect, in some possible implementations, before executing the privileged virtual machine and the user virtual machine based on the allocation result of the operation resource, the method further includes: dividing a core of a processor of the gateway into a first core and a second core according to a preset allocation rule, and dividing an operation memory of the gateway into a first operation memory and a second operation memory, wherein the value of the first core is greater than that of the second core, and the value of the first operation memory is greater than that of the second operation memory; the first core and the first running memory are allocated to the privileged virtual machine, and the second core and the second running memory are allocated to the user virtual machine.
In the above technical solution, the operation resource of the gateway includes a core of a processor of the gateway and an operation memory of the gateway, and the first core and the second core, and the first operation memory and the second operation memory can be obtained by dividing the operation resource of the gateway device, where a value of the first core is greater than a value of the second core, and a value of the first operation memory is greater than a value of the second operation memory. The first core and the first operation memory are operation resources corresponding to the privileged virtual machine, and the second core and the second operation memory are operation resources corresponding to the user virtual machine. In the distribution process, the numerical value of the second core corresponding to the user virtual machine is controlled to be smaller than the numerical value of the first core, and the numerical value of the second running memory is controlled to be smaller than the numerical value of the first running memory. Under the condition of ensuring that the user virtual machine can run, the processing efficiency of the privileged virtual machine is not influenced, and the purpose of reasonably distributing the operation resources of the gateway equipment is achieved.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners, the running a driver monitoring system on the user virtual machine includes: in the user virtual machine, based on the driver monitoring system running in the first operating system, the numerical value of the operation resource required by the first operating system is smaller than a preset threshold value.
In one possible implementation, the first operating system is a reduced Linux operating system.
In the above technical solution, compared with the conventional Linux system, the simplified Linux system is a lightweight Linux system, can reduce the occupation of a Read-Only Memory (ROM) and a Random Access Memory (RAM), and has the characteristics of light weight and high speed, thereby not Only avoiding the occupation problem of resources, but also realizing the speed of processing the action information of the driver by the DMS.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners, the sending, by the privileged virtual machine, the action information of the driver to the user virtual machine includes: receiving a first request sent by the user virtual machine through the privileged virtual machine, wherein the first request is used for requesting to acquire action information of the driver; in response to the first request, sending, by the privileged virtual machine, action information of the driver to the user virtual machine.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners, the receiving, by the privileged virtual machine, the first request sent by the user virtual machine includes: receiving a first request sent by a front-end drive module of the user virtual machine through a rear-end drive module of the privileged virtual machine, wherein the first request comprises an identifier of the user virtual machine; and, the sending, by the privileged virtual machine, the driver's action information to the user virtual machine in response to the first request, comprising: responding to the first request, and determining an external device driver module corresponding to the user virtual machine in the privileged virtual machine according to the identification of the user virtual machine; acquiring action information of the driver, which is acquired by external equipment, through the rear-end driving module and the external equipment driving module; and sending the action information of the driver to the user virtual machine through the rear end driving module.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners, after the sending, by the privileged virtual machine, the action information of the driver to the user virtual machine, the method further includes: processing the action information of the driver through the driver monitoring system to obtain a processing result, wherein the processing result is used for indicating whether the action information of the driver has potential safety hazards; and sending the processing result to the privileged virtual machine through the user virtual machine.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners, the action information of the driver includes an image of the driver, and the processing result obtained by processing the action information of the driver by the driver monitoring system includes: comparing the image of the driver with the image in a preset image library through the driver monitoring system to obtain a processing result of the image of the driver; and sending the processing result to the privileged virtual machine through the user virtual machine, including: and sending the processing result to the privileged virtual machine through a front-end driving module.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners, in a case that the processing result indicates that the safety hazard exists in the action information of the driver, the method further includes: generating a notification instruction corresponding to the processing result by the privileged virtual machine; and sending the notification instruction to an alarm device in the vehicle through the privileged virtual machine.
In the above technical solution, when the processing result obtained after the DMS processes the motion information of the driver indicates that the motion information of the current driver may cause a danger, the driver may be reminded by generating a notification instruction corresponding to the processing result by the privileged virtual machine. The process can effectively avoid dangerous accidents, timely guarantees the safety of passengers in the automobile and improves user experience.
In summary, the present application provides a scheme for virtualizing a gateway or a gateway device, thereby implementing a process of virtualizing a DMS. The method comprises the steps of operating the privileged virtual machine and the user virtual machine according to the operation resources respectively allocated to the privileged virtual machine and the user virtual machine, reasonably allocating gateway resources and ensuring the normal operation of the privileged virtual machine and the user virtual machine. And furthermore, a driver monitoring system is operated in the user virtual machine, so that the effect of virtualizing the DMS is realized, and the consumption of system hardware is avoided. And finally, the action information of the driver can be sent to the user virtual machine through the privileged virtual machine, so that the action information of the driver is processed by the driver monitoring system, and the process of processing the action information of the driver is realized under the condition of not influencing the hardware loss of the system by combining the process of DMS virtualization.
The operation resources of the gateway comprise a core of a processor of the gateway and an operation memory of the gateway, and a first core, a second core, a first operation memory and a second operation memory can be obtained by dividing the operation resources of the gateway device, wherein the value of the first core is larger than that of the second core, and the value of the first operation memory is larger than that of the second operation memory. The first core and the first operation memory are operation resources corresponding to the privileged virtual machine, and the second core and the second operation memory are operation resources corresponding to the user virtual machine. In the distribution process, the value of the second core corresponding to the user virtual machine is controlled to be smaller than the value of the first core, and the value of the second running memory is controlled to be smaller than the value of the first running memory. Under the condition of ensuring that the user virtual machine can run, the processing efficiency of the privileged virtual machine is not influenced, and the purpose of reasonably distributing the operation resources of the gateway equipment is achieved.
When the first operating system is a simplified Linux operating system, compared with a conventional Linux system, the simplified Linux system is a lightweight Linux system, can reduce occupation of a Read-Only Memory (ROM) and a Random Access Memory (RAM), and has the characteristics of light weight and high speed, so that the problem of resource occupation is avoided, and the speed of processing the action information of the driver by the DMS is realized.
When the DMS processes the motion information of the driver and the obtained processing result indicates that the current motion information of the driver may cause a risk, the DMS may also generate a notification instruction corresponding to the processing result by the privileged virtual machine to remind the driver. The process can effectively avoid dangerous accidents, timely guarantees the safety of passengers in the vehicle, and improves user experience.
In a second aspect, there is provided an apparatus for operating a driver monitoring system, the apparatus comprising: the operation module is used for operating a privileged virtual machine and a user virtual machine based on the allocation result of the operation resources, the privileged virtual machine is used for collecting the action information of a driver in the vehicle, and the operation resources comprise a core of a processor of the gateway and an operation memory of the gateway; running the driver monitoring system on the user virtual machine; and the sending module is used for sending the state information of the driver to the user virtual machine through the privileged virtual machine so that the driver monitoring system processes the action information of the driver.
With reference to the second aspect, in some possible implementations, before running the privileged virtual machine and the user virtual machine based on the allocation result of the operation resource, the apparatus further includes: the distribution module is used for dividing the core of the processor of the gateway into a first core and a second core according to a preset distribution rule, and dividing the running memory of the gateway into a first running memory and a second running memory, wherein the numerical value of the first core is greater than the numerical value of the second core, and the numerical value of the first running memory is greater than the numerical value of the second running memory; the first core and the first running memory are allocated to the privileged virtual machine, and the second core and the second running memory are allocated to the user virtual machine.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners, the running module is specifically configured to: in the user virtual machine, based on the driver monitoring system running in the first operating system, the numerical value of the operation resource required by the first operating system is smaller than a preset threshold value.
In one possible implementation, the first operating system is a reduced Linux operating system.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners, the sending module is specifically configured to: receiving a first request sent by the user virtual machine through the privileged virtual machine, wherein the first request is used for requesting to acquire action information of the driver; in response to the first request, sending, by the privileged virtual machine, the driver's action information to the user virtual machine.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners, the sending module is further configured to: receiving a first request sent by a front-end drive module of the user virtual machine through a rear-end drive module of the privileged virtual machine, wherein the first request comprises an identifier of the user virtual machine; responding to the first request, and determining an external device driver module corresponding to the user virtual machine in the privileged virtual machine according to the identification of the user virtual machine; acquiring action information of the driver, which is acquired by external equipment, through the rear-end driving module and the external equipment driving module; and sending the action information of the driver to the user virtual machine through the rear end driving module.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners, after the sending, by the privileged virtual machine, the action information of the driver to the user virtual machine, the apparatus further includes: the first processing module is used for processing the action information of the driver through the driver monitoring system to obtain a processing result, and the processing result is used for indicating whether the action information of the driver has potential safety hazards or not; and sending the processing result to the privileged virtual machine through the user virtual machine.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners, the action information of the driver includes an image of the driver, and the first processing module is specifically configured to: comparing the image of the driver with the image in a preset image library through the driver monitoring system to obtain a processing result of the image of the driver, wherein the processing result is used for indicating whether potential safety hazards exist in the action information of the driver; and sending the processing result to the privileged virtual machine through a front-end driving module.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners, in a case that the processing result indicates that the action information of the driver has a safety hazard, the apparatus further includes: the second processing module is used for generating a notification instruction corresponding to the processing result through the privileged virtual machine; and sending the notification instruction to an alarm device in the vehicle through the privileged virtual machine.
In a third aspect, a vehicle is provided, which includes a gateway for performing the method of the first aspect or any one of the possible implementation manners of the first aspect, and the gateway includes a memory and a processor. The memory is configured to store executable program code, and the processor is configured to call and execute the executable program code from the memory, so that the gateway performs the method of the first aspect or any one of the possible implementation manners of the first aspect.
In a fourth aspect, there is provided a computer program product comprising: computer program code which, when run on a computer, causes the computer to perform the method of the first aspect or any one of the possible implementations of the first aspect.
In a fifth aspect, a computer-readable storage medium is provided, which stores computer program code, which, when run on a computer, causes the computer to perform the method of the first aspect or any one of the possible implementations of the first aspect.
Drawings
FIG. 1 is a schematic flow chart diagram of a method of operating a driver monitoring system provided by an embodiment of the present application;
FIG. 2 is a schematic diagram of a scenario of interaction between a privileged virtual machine and a user virtual machine according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of an apparatus for operating a driver monitoring system according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a vehicle according to an embodiment of the present application.
Detailed Description
The technical solution of the present application will be described in detail and clearly with reference to the accompanying drawings. In the description of the embodiments of the present application, a "/" indicates an inclusive meaning unless otherwise specified, for example, a/B may indicate a or B: "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.
In the following, the terms "first", "second" are used for descriptive purposes only and are not to be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.
Before describing the method of the embodiments of the present application, a brief description will be given of some basic concepts in the embodiments of the present application.
Xen: xen is an open-source project developed by Cambridge university computer laboratories, is open-source virtualization software which can directly run on a hardware layer, belongs to a type-I virtualization system, supports ten thousand virtualization and super virtualization, and is known to have high performance and small occupied resource. Xen only takes over directly for Central Processing Unit (CPU) and Memory, while other Input Output (IO) hardware drivers are supported by the first virtual machine (privileged domain) running on it, which can run multiple Guest operating systems (Guest OS) concurrently on the computer hardware.
Domain0: privileged virtual machines, also called privileged domains, referred to as "Dom0" for short, are guest operating systems that run on the Xen hypervisor with privileges to directly access hardware and manage other guest operating systems. All Xen virtual environments require Dom0 to be run before other virtual clients or user virtual machines can be run. Dom0 plays the role of an administrator in Xen, responsible for managing other user virtual machines.
The privileged domain has native device drivers, has direct access privileges to hardware devices, and controls and manages other user domains by interacting with the control interface provided by Xen. In the process of accessing the device, dom0 may receive an IO request sent by another DomU through the front-end driver, and call the native device driver by using the back-end driver to complete the IO request.
DomainU: the user virtual machine, also called a user domain, referred to as "DomU" for short, is a common guest operating system or a business operating system running on the Xen hypervisor, cannot directly access hardware resources (such as a memory, a hard disk, and the like), but may exist in a plurality of independent and parallel manners.
Xen separation device drive mode: after xen2.0, a separate device driver mode is introduced that establishes a front end (Frontend) device in each user domain and a back end (Backend) device in the privileged domain. All user domain operating systems send requests to the front-end equipment like using ordinary equipment, the front-end equipment sends the requests and the identity information of the user domain to the back-end equipment in the privileged domain through the IO request descriptor and the equipment channel, and the back-end equipment obtains corresponding data by calling the native equipment driver or the local equipment driver.
After the basic concept is introduced, a method for operating the driver monitoring system provided in the embodiment of the present application is described in detail below, wherein the Xen virtualization platform in the embodiment of the present application is taken as an example. A method for operating a driver monitoring system provided by an embodiment of the present application is described in detail.
FIG. 1 is a schematic flow chart of a method of operating a driver monitoring system provided by an embodiment of the present application.
Illustratively, as shown in fig. 1, the method 100 includes:
and 101, operating a privileged virtual machine and a user virtual machine based on the allocation result of the operation resources, wherein the privileged virtual machine is used for collecting the action information of a driver in the vehicle, and the operation resources comprise the core of a processor of the gateway and the operation memory of the gateway.
In the embodiment of the present application, first, a Xen virtualization technology is used to virtualize a gateway, and specifically, a vehicle network processor or a vehicle network controller (hereinafter, all may be referred to as a "processor") is used to virtualize a System on Chip (SoC) in the gateway. The model of the processor is NXP S32G3.
It should be understood that the main application scenarios in the embodiment of the present application are as follows: in the process of driving the vehicle by the driver, the action information of the driver can be collected by the camera, and further the action information of the driver can be processed by the DMS to judge whether dangerous driving actions exist in the driver.
In the above process, the DMS generally involves a large number of AI algorithms and neural network models in processing the driver's motion information. The processing of the driver's movement information therefore usually needs to be done by means of a specialized AI processor or processing unit. For example, hardware of an embedded Neural Network Processing Unit (NPU), a Vector Processing Unit (VPU), and an Intelligent Processing Unit (IPU) processes and compares driver's motion information collected by a camera in a vehicle by using a deep Neural Network model and a large number of Artificial Intelligence (AI) algorithms included in the processors, so as to obtain a result of the motion information.
However, the processor S32G3 in the gateway in the embodiment of the present application belongs to a vehicle network processor, and does not relate to professional deep learning, AI algorithm, and neural network related operations and processing. Therefore, in the embodiment of the present application, for non-AI configuration hardware, the action information of the driver can be processed by the CPU in the virtualized environment running the DMS.
It should be understood that, in the virtualization of the DMS according to the embodiment of the present application, the operation resources of the processor may be first divided according to a preset allocation rule. Because the privileged virtual machine is responsible for managing and controlling the user virtual machine, and in the embodiment of the present application, what mainly runs in the privileged virtual machine is a vehicle Open System Architecture Adaptive Platform (Autosar AP), and the user virtual machine runs the DMS. Therefore, in the process of virtualizing the processor, it is first ensured that the normal function of the Autosar AP in the privileged virtual machine can be realized without affecting the performance, and it is also ensured that the DMS in the user virtual machine can run normally and the processing speed is appropriate.
In a possible implementation manner, the operation resource of the gateway includes a core of a processor of the gateway and an operation memory of the gateway, and in the process of allocating the operation resource of the gateway, the method specifically includes:
dividing a core of a processor of a gateway into a first core and a second core according to a preset allocation rule, and dividing an operation memory of the gateway into a first operation memory and a second operation memory, wherein the value of the first core is greater than that of the second core, and the value of the first operation memory is greater than that of the second operation memory;
and allocating the first core and the first running memory to the privileged virtual machine, and allocating the second core and the second running memory to the user virtual machine.
The numerical values of the first core and the second core specifically refer to the number, for example, the number of cores of the processor of the gateway is 8, the number of the first cores is 6, and the number of the second cores is 2. The value of the first operating memory and the size of the second operating memory specifically refer to sizes, for example, the size of the first operating memory is 3GB, and the size of the second operating memory is 1GB.
The above-mentioned preset allocation principle can be understood as described in the foregoing text: the normal operation of the Autosar AP running in the privileged virtual machine needs to be ensured, the normal operation of the DMS in the user virtual machine needs to be ensured, and the processing speeds in the privileged virtual machine and the user virtual machine need to be taken into account. Based on the principle of the partition allocation, the cores and the operating memories of the processor of the gateway may be divided to obtain a first operation resource including the first core and the first operating memory, and a second operation resource including the second core and the second operating memory. The first computing resource corresponds to a privileged virtual machine, and the second computing resource corresponds to a user virtual machine. And the number of the first cores is larger than that of the second cores, and the value of the first running memory is larger than that of the second running memory.
Through the process of allocating the operation resources in advance, the first operation resources corresponding to the privileged virtual machine and the second operation resources corresponding to the user virtual machine can be obtained. The privileged virtual machine may then be run based on the first core and the first run memory, and the user virtual machine may be run based on the second core and the second run memory.
For example, since the privileged virtual machine manages and controls the other user virtual machines, in this embodiment of the present application, the privileged virtual machine may be started first by the first operation resource, and then the user virtual machine may be controlled to start by the privileged virtual machine and the second operation resource.
In the above technical solution, the operation resource of the gateway includes a core of a processor of the gateway and an operation memory of the gateway, and the first core and the second core, and the first operation memory and the second operation memory can be obtained by dividing the operation resource of the gateway device, where a value of the first core is greater than a value of the second core, and a value of the first operation memory is greater than a value of the second operation memory. The first core and the first operation memory are operation resources corresponding to the privileged virtual machine, and the second core and the second operation memory are operation resources corresponding to the user virtual machine. In the distribution process, the value of the second core corresponding to the user virtual machine is controlled to be smaller than the value of the first core, and the value of the second running memory is controlled to be smaller than the value of the first running memory. Under the condition of ensuring that the user virtual machine can run, the processing efficiency of the privileged virtual machine is not influenced, and the purpose of reasonably distributing the operation resources of the gateway equipment is achieved.
102, running a driver monitoring system on the user virtual machine.
In this embodiment of the present application, after the user virtual machine and the privileged virtual machine are started in the step 101, the DMS in this embodiment of the present application may be executed in the user virtual machine, which specifically includes:
in the user virtual machine, a driver monitoring system is operated based on a first operating system, and the calculation resource required for operating the first operating system is smaller than a preset threshold value.
The first operating system is a simplified Linux system.
In the technical scheme, compared with the traditional Linux system, the simplified Linux system is a lightweight Linux system, can reduce the occupation of the ROM and the RAM, and has the characteristics of light weight and high speed, so that the problem of resource occupation is avoided, and the speed of processing the action information of the driver by the DMS is realized.
And 103, sending the action information of the driver to the user virtual machine through the privileged virtual machine so that the driver monitoring system processes the action information of the driver.
It should be understood that, in the embodiment of the present application, the privileged virtual machine directly has a function of accessing the hardware device, and when the driver monitoring system processes the action information of the driver, the privileged virtual machine is first required to acquire the action information of the driver, and then the gateway sends the action information of the driver to the user virtual machine through the privileged virtual machine, so as to perform a subsequent processing process.
In a possible implementation manner, the gateway sends the action information of the driver to the user virtual machine through the privileged virtual machine, and the action information of the driver can be obtained by receiving an IO request sent by the user virtual machine to the privileged virtual machine. The method specifically comprises the following steps:
receiving a first request sent by a user virtual machine through a privileged virtual machine, wherein the first request is used for acquiring action information of a driver;
in response to the first request, the driver's action information is sent to the user virtual machine through the privileged virtual machine.
Specifically, during the process of data and request transmission of the privileged virtual machine and the user virtual machine, the data and request transmission is realized based on the back-end driver module in the privileged virtual machine and the front-end driver module in the user virtual machine. Receiving a first request sent by a user virtual machine through a privileged virtual machine comprises the following steps:
receiving a first request sent by a front-end drive module of a user virtual machine through a rear-end drive module of the privileged virtual machine, wherein the first request comprises an identifier of the user virtual machine;
and in response to the first request, sending, by the privileged virtual machine, the action information of the driver to the user virtual machine, including:
responding to the first request, and determining an external device driving module corresponding to the user virtual machine in the privileged virtual machine according to the identification of the user virtual machine;
acquiring action information of a driver, which is acquired by external equipment, through a rear-end driving module and the external equipment driving module;
and sending the action information of the driver to the user virtual machine through the rear end driving module.
Among them, the external device driver module is also the native device driver module described above.
Specifically, the above gateway sends the action information of the driver to the user virtual machine through the privileged virtual machine, and the specific process may be understood as follows: and the application program of the user virtual machine accesses the virtual IO equipment, the core of the user virtual machine calls a front end driving module in the user virtual machine to generate an IO request, and the IO request describes information such as the address and the length of data to be acquired and writes the information into an IO sharing ring.
The front-end driving module puts the data to be acquired into an authorization table and gives the privilege virtual machine access authority, and the front-end driving module informs the privilege virtual machine through an event channel. The Xen virtualization platform schedules the privileged virtual machine to run, checks the event, discovers a new IO request, and then calls the back-end driving module to process the new IO request. Namely, the rear end driving module takes out the IO request from the IO shared ring and analyzes the IO request. The back-end driving module determines data required to be acquired by the user virtual machine through operation of the authorization table, and calls a real device driver to execute data writing operation after preprocessing the IO request, namely, the operation of acquiring action information of a driver.
After the IO request is completed, the back-end driving module generates an IO response, puts the action information of the driver into an IO sharing ring, and informs the user of the virtual machine through the event channel mechanism again. And the Xen virtualization platform schedules the operation of the virtual machine of the user, checks the event, finds a new IO response, and calls the front-end driving module to read the IO response from the IO shared ring, so that the action information of the driver is obtained.
Further, after the gateway sends the state information of the driver to the user virtual machine through the privileged virtual machine, the driver monitoring system can process the actions of the driver, and the method specifically includes the following steps:
processing the action information of the driver through a driver monitoring system to obtain a processing result, wherein the processing result is used for indicating whether the action information of the driver has potential safety hazards;
and sending the processing result to the privileged virtual machine through the user virtual machine.
In one possible implementation manner, the action information of the driver includes an image of the driver, and the action information of the driver is processed by the driver monitoring system to obtain a processing result, including:
comparing the image of the driver with the image in the preset image library through a driver monitoring system to obtain a processing result of the image of the driver;
and sending the processing result to the privileged virtual machine through the user virtual machine, including:
and sending the processing result to the privileged virtual machine through the front-end driving module.
Specifically, the processing of the driver's action information and the sending of the processing result to the privileged virtual machine by the driver monitoring system may be understood as follows: the front-end driving equipment reads out the IO response from the IO sharing ring, and after the action information of the driver is obtained, a driver monitoring system in the user virtual machine is called to compare the image information of the driver with the image in the preset image library, and whether the action information of the driver has potential safety hazards or not is determined. For example, the images in the preset image library may include preset actions (e.g., call, eat, etc.), if the matching degree of the images of the driver with the preset actions is high, a processing result such as a matching degree value or a processing result of a voice prompt or a text prompt type may be obtained, the processing result is put into the authorization table, and then the privileged virtual machine is notified to view the processing result through an event channel mechanism.
For ease of understanding, the interaction between the privileged virtual machine and the user virtual machine is briefly described below.
Fig. 2 is a schematic view of a scenario of interaction between a privileged virtual machine and a user virtual machine according to an embodiment of the present application.
Illustratively, as shown in fig. 2, the first computing resource in the privileged virtual machine Dom0 includes a virtual memory 1 and a virtual processing module 1. The virtual processing module 1 mainly includes cores (also called cores) of 6 processors. The second computing resource in the user virtual machine includes a virtual memory 2 and a virtual processing module 2. The virtual processing module 2 includes 2 cores of processors.
The privileged virtual machine comprises a back-end driving module, and the user virtual machine comprises a front-end driving module. The Xl tool in the privileged Virtual Machine is a lightweight Virtual Machine (VM) management tool based on libxenlight, and is responsible for managing commands and data in the privileged Virtual Machine.
The privileged virtual machine may access three types of data, which are Controller Area Network (CAN) data, network (NET) data, and Universal Serial Bus (USB) data. In the embodiment of the application, the action information of the driver is mainly transmitted through the USB signal.
Further, when the user virtual machine needs to process by acquiring the action information of the driver, the user virtual machine system in the user virtual machine may send a request to the front-end driver module, and the front-end driver module sends the request to the back-end driver module in the privileged virtual machine after receiving the request. After receiving the request, the back-end driver module accesses a real device (for example, a camera inside a vehicle) through an external device driver (USB driver) and a USB interface in the privileged virtual machine to acquire or obtain the action information of the driver.
Further, the privileged virtual machine sends the action information of the driver to the user virtual machine through the rear end driving module, and then the front end driving module forwards the action information to the driver monitoring system in the user virtual machine for processing.
For example, the action information of the driver includes an image of the driver, and in the process of processing the action information of the driver through the driver monitoring system, a preset image library in the driver monitoring system may be used for comparing with a current image of the driver, and the comparison result is used as a processing result.
After the driver monitoring system completes processing of the action information of the driver, the processing result is sent to the adaptive platform in the privileged virtual machine through a network Protocol (e.g., transmission Control Protocol/Internet Protocol (TCP/IP)).
Further, when the processing result indicates that the action information of the driver has a potential safety hazard, the method further comprises the following steps:
generating a notification instruction corresponding to the processing result through the privileged virtual machine;
and sending the notification instruction to an alarm device in the vehicle through the privileged virtual machine.
Specifically, after receiving the current processing result, the adaptive platform in the privileged virtual machine may generate a notification instruction according to the current processing result, and send the notification instruction to the warning device in the vehicle. Such as a microphone or a Head Unit (HU), for voice prompting through the microphone or display prompting through the HU.
In the above technical solution, when the processing result obtained after the DMS processes the motion information of the driver indicates that the motion information of the current driver may cause a danger, the driver may be reminded by generating a notification instruction corresponding to the processing result by the privileged virtual machine. The process can effectively avoid dangerous accidents, timely guarantees the safety of passengers in the vehicle, and improves user experience.
Fig. 3 is a schematic structural diagram of an apparatus for operating a driver monitoring system according to an embodiment of the present application.
Illustratively, as shown in fig. 3, the apparatus 300 includes:
an operation module 301, configured to operate a privileged virtual machine and a user virtual machine based on an allocation result of an operation resource, where the privileged virtual machine is used to collect action information of a driver in a vehicle, and the operation resource includes a core of a processor of the gateway and an operation memory of the gateway; running the driver monitoring system on the user virtual machine;
a sending module 302, configured to send the status information of the driver to the user virtual machine through the privileged virtual machine, so that the driver monitoring system processes the action information of the driver.
Optionally, before the running of the privileged virtual machine and the user virtual machine based on the allocation result of the operation resource, the apparatus further includes: the distribution module is used for dividing the core of the processor of the gateway into a first core and a second core according to a preset distribution rule, and dividing the running memory of the gateway into a first running memory and a second running memory, wherein the numerical value of the first core is greater than that of the second core, and the numerical value of the first running memory is greater than that of the second running memory; the first kernel and the first running memory are allocated to the privileged virtual machine, and the second kernel and the second running memory are allocated to the user virtual machine.
In a possible implementation manner, the running module 301 is specifically configured to: in the user virtual machine, based on the driver monitoring system running in the first operating system, the numerical value of the operation resource required by the first operating system is smaller than a preset threshold value.
In a possible implementation manner, the sending module is specifically configured to: receiving a first request sent by the user virtual machine through the privileged virtual machine, wherein the first request is used for requesting to acquire action information of the driver; in response to the first request, sending, by the privileged virtual machine, the driver's action information to the user virtual machine.
In a possible implementation manner, the sending module is further configured to: receiving a first request sent by a front end drive module of the user virtual machine through a rear end drive module of the privileged virtual machine, wherein the first request comprises an identifier of the user virtual machine; responding to the first request, and determining an external device driver module corresponding to the user virtual machine in the privileged virtual machine according to the identification of the user virtual machine; acquiring action information of the driver, which is acquired by external equipment, through the rear-end driving module and the external equipment driving module; and sending the action information of the driver to the user virtual machine through the rear end driving module.
Optionally, after the sending the action information of the driver to the user virtual machine through the privileged virtual machine, the apparatus further includes: the first processing module is used for processing the action information of the driver through the driver monitoring system to obtain a processing result, and the processing result is used for indicating whether the action information of the driver has potential safety hazards or not; and sending the processing result to the privileged virtual machine through the user virtual machine.
In a possible implementation manner, the action information of the driver includes an image of the driver, and the first processing module is specifically configured to: comparing the image of the driver with the image in a preset image library through the driver monitoring system to obtain a processing result of the image of the driver; and sending the processing result to the privileged virtual machine through a front-end driving module.
Optionally, in a case that the processing result indicates that the action information of the driver has a safety hazard, the apparatus further includes: the second processing module is used for generating a notification instruction corresponding to the processing result through the privileged virtual machine; and sending the notification instruction to an alarm device in the vehicle through the privileged virtual machine.
Fig. 4 is a schematic structural diagram of a vehicle according to an embodiment of the present application.
Illustratively, as shown in fig. 4, the vehicle 400 includes a gateway 401: the gateway 401 includes a memory 4011 and a processor 4012, wherein the memory 4011 stores executable program code 40111, and the processor 4012 is configured to call and execute the executable program code 40111 to perform a method of operating a driver monitoring system.
In this embodiment, the gateway may be divided into functional modules according to the method example, for example, the gateway may correspond to each functional module, or two or more functions may be integrated into one processing module, and the integrated module may be implemented in a hardware form. It should be noted that the division of the modules in this embodiment is schematic, and is only a logic function division, and there may be another division manner in actual implementation.
In the case of dividing each functional module by corresponding each function, the gateway may include: a distribution module, an operation module, etc. It should be noted that all relevant contents of each step related to the above method embodiment may be referred to as a functional description of the corresponding functional module, and are not described herein again.
The gateway provided by the embodiment is used for executing the method for operating the driver monitoring system, so that the same effect as the implementation method can be achieved.
In case of an integrated unit, the gateway may comprise a processing module, a storage module. The processing module can be used for controlling and managing the action of the gateway. The memory module may be used to support the gateway in executing the mutual program code and data, etc.
Wherein a processing module may be a processor or controller that may implement or execute the various illustrative logical blocks, modules, and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, including, for example, one or more microprocessors, digital Signal Processing (DSP) and microprocessors, and the storage module may be a memory.
The present embodiment also provides a computer-readable storage medium having stored therein computer program code, which, when run on a computer, causes the computer to perform the above-mentioned related method steps to implement a method of operating a driver monitoring system in the above-mentioned embodiments.
The present embodiment also provides a computer program product, which when run on a computer causes the computer to perform the relevant steps described above, to implement a method of operating a driver monitoring system in the above embodiments.
The gateway, the computer-readable storage medium, the computer program product, or the chip provided in this embodiment are all configured to execute the corresponding method provided above, and therefore, the beneficial effects that can be achieved by the gateway may refer to the beneficial effects in the corresponding method provided above, which are not described herein again.
Through the description of the above embodiments, those skilled in the art will understand that, for convenience and simplicity of description, only the division of the above functional modules is used as an example, and in practical applications, the above function distribution may be completed by different functional modules as needed, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another apparatus, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of operating a driver monitoring system, the method being performed by a gateway operating a virtualization platform, the method comprising:
operating a privileged virtual machine and a user virtual machine based on an allocation result of computational resources, wherein the privileged virtual machine is used for collecting action information of a driver in a vehicle, and the computational resources comprise a core of a processor of the gateway and an operating memory of the gateway;
running the driver monitoring system on the user virtual machine;
and sending the action information of the driver to the user virtual machine through the privileged virtual machine so that the driver monitoring system processes the action information of the driver.
2. The method of claim 1, wherein prior to running the privileged virtual machine and the user virtual machine based on the allocation of the computational resources, the method further comprises:
dividing a core of a processor of the gateway into a first core and a second core according to a preset allocation rule, and dividing an operation memory of the gateway into a first operation memory and a second operation memory, wherein the numerical value of the first core is greater than the numerical value of the second core, and the numerical value of the first operation memory is greater than the numerical value of the second operation memory;
and allocating the first core and the first running memory to the privileged virtual machine, and allocating the second core and the second running memory to the user virtual machine.
3. The method of claim 1 or 2, wherein the running a driver monitoring system on the user virtual machine comprises:
in the user virtual machine, the driver monitoring system is operated based on a first operating system, and the numerical value of the operation resource required by the operation of the first operating system is smaller than a preset threshold value.
4. The method of claim 1 or 2, wherein said sending, by the privileged virtual machine, the driver's action information to the user virtual machine comprises:
receiving a first request sent by the user virtual machine through the privileged virtual machine, wherein the first request is used for requesting to acquire action information of the driver;
in response to the first request, sending, by the privileged virtual machine, action information of the driver to the user virtual machine.
5. The method of claim 4, wherein receiving, by the privileged virtual machine, the first request sent by the user virtual machine comprises:
receiving, by a back-end driver module of the privileged virtual machine, a first request sent by a front-end driver module of the user virtual machine, the first request including an identifier of the user virtual machine;
and said sending, by said privileged virtual machine, action information of said driver to said user virtual machine in response to said first request, comprising:
responding to the first request, and determining an external device driver module corresponding to the user virtual machine in the privileged virtual machine according to the identification of the user virtual machine;
acquiring action information of the driver, which is acquired by external equipment, through the rear end driving module and the external equipment driving module;
and sending the action information of the driver to the user virtual machine through the rear end driving module.
6. The method of claim 1 or 2, wherein after said sending the driver's action information to the user virtual machine through the privileged virtual machine, the method further comprises:
processing the action information of the driver through the driver monitoring system to obtain a processing result, wherein the processing result is used for indicating whether potential safety hazards exist in the action information of the driver;
and sending the processing result to the privileged virtual machine through the user virtual machine.
7. The method of claim 6, wherein the driver's motion information comprises an image of the driver, and wherein processing the driver's motion information by the driver monitoring system results in a processing result comprises:
comparing the image of the driver with the image in a preset image library through the driver monitoring system to obtain a processing result of the image of the driver;
and sending the processing result to the privileged virtual machine through the user virtual machine, including:
and sending the processing result to the privileged virtual machine through a front-end driving module.
8. The method according to claim 6, wherein in the case that the processing result indicates that the action information of the driver has a safety hazard, the method further comprises:
generating a notification instruction corresponding to the processing result through the privileged virtual machine;
and sending the notification instruction to an alarm device in the vehicle through the privileged virtual machine.
9. An apparatus for operating a driver monitoring system, the apparatus comprising:
the operating module is used for operating the privileged virtual machine and the user virtual machine based on the allocation result of the operating resources, the privileged virtual machine is used for collecting the action information of a driver in a vehicle, and the operating resources comprise a core of a processor of the gateway and an operating memory of the gateway; running the driver monitoring system on the user virtual machine;
and the sending module is used for sending the state information of the driver to the user virtual machine through the privileged virtual machine so that the driver monitoring system processes the action information of the driver.
10. A vehicle, characterized in that it comprises a gateway for carrying out the method according to any one of claims 1 to 8, said gateway comprising:
a memory for storing executable program code;
a processor for invoking and executing the executable program code from the memory, such that the gateway performs the method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211352744.0A CN115658233A (en) | 2022-10-31 | 2022-10-31 | Method and device for operating a driver monitoring system and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211352744.0A CN115658233A (en) | 2022-10-31 | 2022-10-31 | Method and device for operating a driver monitoring system and vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115658233A true CN115658233A (en) | 2023-01-31 |
Family
ID=84996158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211352744.0A Pending CN115658233A (en) | 2022-10-31 | 2022-10-31 | Method and device for operating a driver monitoring system and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115658233A (en) |
-
2022
- 2022-10-31 CN CN202211352744.0A patent/CN115658233A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10180843B2 (en) | Resource processing method and device for a multi-core operating system | |
| JP5989862B2 (en) | Computing device, method and machine readable storage medium for shared memory communication between secure virtual machines | |
| US10191759B2 (en) | Apparatus and method for scheduling graphics processing unit workloads from virtual machines | |
| US7971203B2 (en) | Method, apparatus and system for dynamically reassigning a physical device from one virtual machine to another | |
| JP5323897B2 (en) | Method, apparatus and system for bi-directional communication between a virtual machine monitor and an ACPI compliant guest operating system | |
| JP4921384B2 (en) | Method, apparatus and system for dynamically reallocating memory from one virtual machine to another | |
| CN107479943B (en) | Multi-operating-system operation method and device based on industrial Internet operating system | |
| US20240054006A1 (en) | Virtualization processing system, method and apparatus, and device | |
| US9354952B2 (en) | Application-driven shared device queue polling | |
| US9009702B2 (en) | Application-driven shared device queue polling in a virtualized computing environment | |
| CN106844007A (en) | A kind of virtual method and system based on spatial reuse | |
| US8880764B2 (en) | Pessimistic interrupt affinity for devices | |
| CN103744716A (en) | Dynamic interrupt balanced mapping method based on current virtual central processing unit (VCPU) scheduling state | |
| WO2023093843A1 (en) | Configuration device, scheduling device, configuration method, and scheduling method | |
| WO2018040845A1 (en) | Method and apparatus for scheduling computing resource | |
| US20220156103A1 (en) | Securing virtual machines in computer systems | |
| CN113821308A (en) | System on chip, virtual machine task processing method and device, and storage medium | |
| CN115202908A (en) | Privacy computation request response method and device based on dynamic arrangement | |
| CN114691286A (en) | Server system, virtual machine creation method and device | |
| CN116501447B (en) | Xen-based hard real-time implementation system | |
| CN115658233A (en) | Method and device for operating a driver monitoring system and vehicle | |
| EP3244311A1 (en) | Multiprocessor system and method for operating a multiprocessor system | |
| CN109634721B (en) | Method and related device for starting communication between virtual machine and host | |
| CN112667401A (en) | Resource allocation method, system and equipment | |
| WO2024087513A1 (en) | Application scenario-based data processing method and system, electronic device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |