WO2024114425A1

WO2024114425A1 - Intelligent cabin computing power sharing architecture, computing power sharing method, device and medium

Info

Publication number: WO2024114425A1
Application number: PCT/CN2023/132572
Authority: WO
Inventors: 张慧敏
Original assignee: 蔚来软件科技(上海)有限公司
Priority date: 2022-12-01
Filing date: 2023-11-20
Publication date: 2024-06-06
Also published as: CN116028239A

Abstract

The present application relates to the technical field of vehicle-machine interconnection. Specifically, provided are an intelligent cabin computing power sharing architecture, a computing power sharing method, a device and a medium, which aim to solve the problems of the performance of a chip used by a central compute cluster being relatively weak, the update period being long, upgrading being difficult and the cost being excessively high. For this purpose, the computing power sharing architecture of the present application comprises an application layer, a system layer, a service layer, a kernel layer and a hardware layer, wherein the application layer comprises a CDF and an MSF, and the application layer defines an application program and allocates the application program to the CDF and/or the MSF according to configuration parameters; the system layer provides an operation environment and application program management for the application layer; the service layer provides services for the application program in the application layer; the kernel layer provides a unified operating system kernel abstraction layer; and the hardware layer provides a communication interface. By means of the embodiments, the functions of a CDF and an MSF are rationally divided, and a componentized and service-oriented intelligent cabin computing power sharing architecture with high cohesion and low coupling is provided.

Description

Intelligent cockpit computing power sharing architecture, computing power sharing method, device and medium

This application claims priority to Chinese patent application No. 202211532272.7 filed on December 1, 2022, with the invention name “Smart cockpit computing power sharing architecture, computing power sharing method, device and medium”. The entire contents of the above Chinese patent application are incorporated into this application by reference.

Technical Field

The present application relates to the field of vehicle-machine interconnection technology, and specifically to a smart cockpit computing power sharing architecture, computing power sharing method, equipment and medium.

Background technique

The in-vehicle infotainment system is an in-vehicle infotainment system that uses a dedicated in-vehicle central processor, based on the body bus system and Internet services. It can realize a series of applications including three-dimensional navigation, real-time traffic conditions, interactive network TV, vehicle information, body control, mobile office, wireless communication, online entertainment functions and in-vehicle remote service providers, greatly improving the level of vehicle electrification, networking and intelligence.

With the gradual development of new energy vehicles, independent in-vehicle infotainment systems are gradually being integrated into the central computing-regional controller architecture. In this new vehicle electronic and electrical architecture, the central computing platform plays the role of the vehicle's "brain", responsible for the calculation tasks of the vehicle's advanced driver assistance system, smart cockpit system, and vehicle control system. Among them, the smart cockpit system is a more advanced in-vehicle infotainment system.

Because automotive standards have special requirements for chip design, production, manufacturing, packaging, testing and other aspects, the chips used in the central computing platform are weaker than consumer electronics chips of the same period, and the frequency of upgrades is much lower than that of consumer-grade chips.

Accordingly, the art needs a new technical solution to solve the above problems.

Summary of the invention

In order to overcome the above-mentioned defects, the present application is proposed to provide an intelligent cockpit computing power sharing architecture, computing power sharing method, device and medium that solve or at least partially solve the technical problems of weak chip performance, long update cycle, difficult upgrade and excessive cost used in the vehicle-mounted central computing platform.

In a first aspect, a smart cockpit computing power sharing architecture is provided, wherein the computing power sharing architecture includes an application layer, a system layer, a service layer, a kernel layer, and a hardware layer;

The application layer includes a cockpit domain function unit CDF and a multimedia system function unit MSF, and the application layer is configured to define an application program and divide the application program into the CDF and/or the MSF according to configuration parameters;

The system layer is configured to provide at least a running environment and application management for the application layer;

The service layer is configured to provide services for the application program in the application layer;

The kernel layer is configured to provide a unified operating system kernel abstraction layer;

The hardware layer is configured to provide a communication interface.

In a technical solution of the above-mentioned intelligent cockpit computing power sharing architecture, the application layer is configured to divide the application program into the CDF and/or the MSF according to the configuration parameters, including:

Divide the different applications into the CDF or the MSF for execution according to the configuration parameters;

and / or,

The same application is run in both the CDF and the MSF.

In a technical solution of the above-mentioned intelligent cockpit computing power sharing architecture, the service layer includes at least a service registration module, a service publishing module, a service discovery module, a service connection module and a service calling module;

The service registration module and the service publishing module are configured to enable the application in the CDF and the MSF to register and publish services related to the application based on the service registration module and the service publishing module respectively;

The service discovery module, the service connection module and the service invocation module are configured to enable the application programs in the MSF and the CDF to be based on the service discovery module. The module, the service connection module and the service calling module realize computing power sharing.

In a technical solution of the above-mentioned intelligent cockpit computing power sharing architecture, the hardware layer is configured to provide a communication interface for a universal serial bus, Ethernet, computer electronic components, and wireless fidelity;

and / or,

The service layer and/or the system layer further includes a security management module, a communication management module and a data management module. The security management module, the communication management module and the data management module are shared by the service layer and the system layer.

In a second aspect, a computing power sharing method according to any one of the technical solutions in the above-mentioned intelligent cockpit computing power sharing architecture is provided, the method comprising:

When the application to be called belongs to an application deployed in MSF, CDF and atomic services related to the application in MSF are mounted under the application to be called in MSF, and the atomic services are started;

Acquire data based on the CDF and send to the MSF;

Calling the service in the MSF to process the data and generate response information;

Based on the response information, the hardware device related to the CDF and/or the hardware device related to the MSF is controlled to perform corresponding actions.

In a technical solution of the above computing power sharing method, mounting the sub-services related to the application in the CDF and the MSF to the application to be called in the MSF includes:

Find the atomic service corresponding to the application in the CDF and the MSF based on the service discovery module in the service layer;

Based on the service connection module in the service layer, the atomic services in the CDF and the MSF are mounted under the application to be called in the MSF.

In a technical solution of the above computing power sharing method, 7. before mounting the atomic service related to the application in the CDF and MSF to the application to be called in the MSF, it also includes:

Determine whether the atomic services related to the CDF and the MSF and the application have been registered;

If not registered, the atomic service is registered based on the service registration module and published based on the service publishing module.

In a third aspect, a driving device is provided, comprising a driving device body and an intelligent cockpit computing power sharing architecture described in any one of the technical solutions of the above-mentioned intelligent cockpit computing power sharing architecture.

In a fourth aspect, an electronic device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the computing power sharing method described in any one of the technical solutions of the above-mentioned computing power sharing method.

In a fifth aspect, a computer-readable storage medium is provided, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the computing power sharing method described in any one of the technical solutions of the computing power sharing method.

The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

In the technical solution for implementing the present application, the computing power sharing architecture includes an application layer, a system layer, a service layer, a kernel layer and a hardware layer; the application layer includes a cockpit domain function unit CDF and a multimedia system function unit MSF, the application layer is configured to define applications and divide applications into CDF and/or MSF according to configuration parameters, the system layer is configured to provide at least a running environment and application management for the application layer, the service layer is configured to provide services for applications in the application layer, the kernel layer is configured to provide a unified operating system kernel abstraction layer, and the hardware layer is configured to provide a communication interface. Through the above implementation, the functions of CDF and MSF are reasonably divided, and a highly cohesive, low-coupling, componentized, service-oriented intelligent cockpit computing power sharing architecture is provided to meet the growing functional needs of users.

Furthermore, a computing power sharing method is provided based on the intelligent cockpit computing power sharing architecture. When the application to be called belongs to the application deployed in MSF, the atomic service related to the application in CDF and MSF is mounted under the application to be called in MSF, and the atomic service is started. Data is obtained based on CDF and sent to MSF, and the service in MSF is called to process the data and generate response information. Based on the response information, the hardware devices related to CDF and/or the hardware devices related to MSF are controlled to perform corresponding actions. This method is efficient and low-cost. Delay, safety and reliability can provide users with better services and improve user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure of the present application will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, among which:

FIG1 is a main structural block diagram of a smart cockpit computing power sharing architecture according to an embodiment of the present application;

FIG2 is a main structural block diagram of the application layer according to an embodiment of the present application;

FIG3 is a main structural block diagram of a service layer according to an embodiment of the present application;

FIG4 is a main structural block diagram of the kernel layer according to an embodiment of the present application;

FIG5 is a main structural block diagram of a hardware layer according to an embodiment of the present application;

FIG6 is a schematic diagram of a physical topology architecture of a communication interface of a hardware layer 105 according to an embodiment of the present application;

FIG7 is a schematic diagram of the main steps of a computing power sharing method according to the present application;

FIG8 is a schematic diagram of a computing power sharing framework of a multimodal artificial intelligence assistant according to the present application;

FIG. 9 is a schematic diagram of the main structure of an electronic device according to an embodiment of the present application.

List of reference numerals:

101: application layer; 102: system layer; 103: service layer; 104: kernel layer; 105: hardware layer; 301: service registration module; 302: service publishing module; 303: service discovery module; 304: service connection module; 305: service calling module; 901: processor; 902: storage device.

Detailed ways

Some embodiments of the present application are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present application and are not intended to be used for reference. It is intended to limit the scope of protection of this application.

In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and/or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and/or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

Here we first explain some terms involved in this application.

IVI: In-Vehicle Infotainment, in-vehicle infotainment system, consists of a dedicated in-vehicle central processor, based on the body bus system and Internet services. In the traditional automotive industry, IVI is used to realize entertainment functions such as three-dimensional navigation, real-time traffic conditions, IPTV, etc. However, in this application, IVI will be given a new concept, which excludes the traditional entertainment functions of general vehicles and specifically refers to the application of multimedia audio and video function units in vehicles.

CCC: Central Compute Cluster, the central computing platform of the on-board computer, including CDF.

CDF: Cockpit Domain Functional, cockpit domain functional unit, also known as smart cockpit domain controller (Cockpit Domain Controller, CDC), is a control/computing chip used for the vehicle's smart cockpit.

MSF: Multimedia system Functional, multimedia system functional unit, can realize a series of applications including 3D navigation, real-time traffic conditions, IPTV, vehicle information, body control, mobile office, wireless communication, online entertainment functions and TSP services, greatly improving the level of vehicle electrification, networking and intelligence.

Automotive grade: The functional and reliability standard requirements for electronic components used in vehicles. This includes operating temperature range, operating stability, defect rate, etc. Requirements: Chips that meet automotive-grade standards need to pass a series of certifications and obtain relevant certificates.

USB3 Gen2: USB-IF, the USB standardization organization, named the standard division of USB protocol. From USB1.0, USB2.0, USB3, USB4, etc., there are different technical names for different technical standards. USB3 Gen2 generally refers to the technical standard that adopts the third-generation USB standard protocol with a transmission rate of about 10Gbps.

HDMI: High Definition Multimedia Interface, is a fully digital video and sound transmission interface that can send uncompressed audio and video signals. It is used in set-top boxes, DVD players, personal computers, TVs, game consoles, integrated amplifiers, digital audio and TVs, etc. HDMI can send audio and video signals at the same time. Since audio and video signals use the same wire, it greatly simplifies the installation of system lines.

DP: Display Port, a digital video interface, is used to transmit image display signals from the chip to the LCD display screen. It can also transmit mixed video and audio signals. It is the same as HDMI in terms of function.

Ehternet: In-vehicle Ethernet mainly refers to the use of Ethernet local area networking technology to replace the traditional in-vehicle CAN bus to achieve joint networking of distributed control units throughout the vehicle, information exchange and control and other functions.

OS kernel: operating system kernel, which provides core job processing functions such as task scheduling, interrupt handling, and multi-tasking communication. It does not include components such as UI, file system, network communication, and multimedia.

KAL: Kernel Abstraction Layer, operating system kernel abstraction layer, is mainly used to extract the basic service functions of the operating system kernel and encapsulate them into standard interfaces for other intermediate layers to call. Using KAL can shield the specific operating system kernel function definition, allowing the bottom layer to use different OS kernels, and the upper layer only needs to follow a unified calling interface.

At present, due to the special requirements of automotive grade chips for design, production, manufacturing, packaging, testing and other aspects, the performance of chips used by CCC is weaker than consumer electronics chips of the same period, and the frequency of upgrades is much lower than that of consumer-grade chips.

In order to solve the problems of weak chip performance, long update cycle, difficult upgrade and high cost used in vehicle CCC, this application provides a smart cockpit computing power sharing architecture. The traditional IVI is physically divided into CDF and MSF, where CDF is kept in CCCBOX The MSF is separated into an independent BOX, which allows for non-compliance with automotive-grade standards.

Referring to FIG. 1 , FIG. 1 is a main structural block diagram of a smart cockpit computing power sharing architecture according to an embodiment of the present application. As shown in FIG. 1 , the smart cockpit computing power sharing architecture in the embodiment of the present application mainly includes an application layer 101, a system layer 102, a service layer 103, a kernel layer 104 and a hardware layer 105.

The application layer 101 includes CDF and MSF. The application layer is configured to define application programs and divide the application programs into CDF and/or MSF according to configuration parameters.

The system layer 102 is configured to provide at least a running environment and application management for the application layer 101 .

The service layer 103 is configured to provide services for the application programs in the application layer 101 .

Kernel layer 104 is configured to provide a unified operating system kernel abstraction layer.

The hardware layer 105 is configured to provide a communication interface.

Through the above implementation method, the functions of CDF and MSF are reasonably divided, providing a highly cohesive, low-coupling, componentized, service-oriented intelligent cockpit computing power sharing architecture to meet the users' growing functional needs.

The above-mentioned smart cockpit computing power sharing architecture is further explained below.

The application layer 101 defines a variety of flexible and scalable componentized applications, and divides the applications into CDF and/or MSF according to different configuration parameters of the applications.

In some implementations, the configuration parameters of the application program are from a pre-set static configuration file table. Those skilled in the art may set the static configuration file table according to actual conditions to perform static configuration, which is not limited here.

In other implementations, different applications can also be dynamically configured or migrated. When the existing applications in the CDF in the driving device cannot meet the needs of the user, MSF can be added and the required applications can be configured for MSF. Applications in CDF can also be migrated to MSF. For example, the artificial intelligence assistant can be deployed on CDF in the initial stage, using the perception ability of the microphone in the cockpit to realize the communication and interaction between the driving device and the user. However, as the years of use increase, the upgraded artificial intelligence assistant needs to be migrated and deployed on MSF to obtain better operating computing power to provide more services.

Furthermore, in some implementations, different applications may be divided into CDFs or MSFs for execution according to configuration parameters.

Since CDF and MSF are enhanced versions of the IVI system, they jointly implement the system functions of IVI, such as the display screen, camera, audio, 3D navigation, artificial intelligence assistant, AR, VR, games, etc. of the in-vehicle intelligent cockpit. These functions need to be reasonably divided between CDF and MSF so that the growing functional requirements of users can be reasonably realized.

Specifically, refer to Figure 2, which is a main structural block diagram of the application layer according to an embodiment of the present application. As shown in Figure 2, the applications in CDF include head-up display HUD, navigation, artificial intelligence assistant, camera, audio, microphone and display screen; the applications in MSF include multimodal artificial intelligence assistant, 3D navigation, AR, game console and 3D conference.

It should be noted that the above division of CDF and MSF applications is only a schematic illustration. As long as it does not violate the technical concept of the present application, technical personnel in this field can divide the application according to actual needs during actual application, and no limitation is made here.

In other implementations, the same application may be run in both CDF and MSF.

It should be noted that when the same application runs in both CDF and MSF, CDF and MSF will provide different services. For example, the navigation in CDF can provide positioning, two-dimensional route navigation and other services, while the navigation in MSF can provide rendered three-dimensional route navigation and other services.

The above is a further description of the application layer 101 . The following is a further description of the system layer 102 .

In some implementations of the system layer 102 , a unified standard operating environment, application management, and software development kit are provided for the application layer 101 .

In other embodiments, the system layer 102 also includes a security management module, a communication management module, and a data management module that are shared with the system layer 103 .

Furthermore, the data management module provides the application layer 101 with an atomic service library of the application program to store the atomic services of the application program; the communication management module provides the service layer 103 with basic communication capabilities, and provides system calls of different capability levels according to the actual needs of different applications to realize data communication between CDF and MSF.

The above is a further description of the system layer 102 , and the following is a further description of the service layer 103 .

In some implementations of the service layer 103, refer to FIG3, which is a main structural block diagram of the service layer according to an embodiment of the present application. As shown in FIG3, the service layer 103 mainly includes a service registration module 301, a service publishing module 302, a service discovery module 303, a service connection module 304 and a service call module 305.

Further, the service registration module 301 and the service publishing module 302 are configured to enable the applications in the CDF and the MSF to register and publish services related to the application based on the service registration module 301 and the service publishing module 302 , respectively.

The service discovery module 303 , the service connection module 304 and the service call module 305 are configured to enable the applications in the MSF and CDF to share computing power based on the service discovery module 303 , the service connection module 304 and the service call module 305 .

Specifically, CDF and MSF register the services provided based on the service registration module 301; publish the registered services based on the service publishing module 302; search for the services published by CDF and MSF based on the service discovery module 303; connect and share computing power based on the service connection module 304; and obtain service data based on the service call module 305.

In other embodiments, the service layer 103 also includes a security management module, a communication management module, and a data management module that are shared with the system layer 102 , and an announcement function for services related to storage of application programs for the application layer 101 .

The above is a further description of the service layer 103 , and the kernel layer 104 will be further described below.

In some implementations of the kernel layer 104, see Figure 4, which is a main structural block diagram of the kernel layer according to an embodiment of the present application. As shown in Figure 4, the kernel layer 104 includes a unified operating system abstraction layer KAL, a Linux operating system, a Windows operating system, an RTOS real-time operating system, a driving device driver Driver, and a vehicle network Network.

Furthermore, the kernel layer 104 is mainly used to extract the basic service functions of the operating system kernel, encapsulate them into standard interfaces for other layers to call, and use KAL to shield the specific operating system kernel function definition, allowing the use of different OS kernels. The application layer only needs to follow a unified calling interface.

The above is a further description of the kernel layer 104. Next, we will continue to explain the hardware layer 105. For further explanation.

In some implementations of the hardware layer 105, see Figure 5, which is a main structural block diagram of the hardware layer according to an embodiment of the present application. As shown in Figure 5, the communication mechanism of the hardware layer 105 includes SOC, USB, Ethernet, computer electronic component DP and Wi-Fi.

Functionally, USB is used for high-speed data interaction between CDF and MSF, such as multimedia audio and video streaming; Ethernet is used for MSF to directly connect to CCC's network port and access the Internet through CCC's 5G network; Wi-Fi provides another option for data interaction. When Wi-Fi technology reaches 10Gbps or higher in the future, Wi-Fi wireless connection can also be used instead of USB interface.

Furthermore, the hardware layer 105 defines a communication mechanism that is efficient, low-latency, secure and reliable between CDF and MSF, and supports distributed computing capabilities. Refer to Figure 6, which is a schematic diagram of the physical topology architecture of the communication interface of the hardware layer 105 according to an embodiment of the present application. As shown in Figure 6, the communication interface is a combination of USB3.0 Gen2+Ethernet+DP+Wi-Fi. This communication mechanism supports the same application layer function to achieve real-time information interaction between CDF and MSF.

Through the above-mentioned technical solution of this application, the functions of CDF and MSF are reasonably divided, providing a highly cohesive, low-coupled, componentized, service-oriented intelligent cockpit computing power sharing architecture to meet the users' growing functional needs.

Furthermore, the present application also provides a computing power sharing method based on the technical solution of the intelligent cockpit computing power sharing architecture, referring to FIG. 7, which is a schematic flow chart of the main steps of a computing power sharing method according to the present application. As shown in FIG. 7, the computing power sharing method in the embodiment of the present application mainly includes the following steps S701 to S704.

Step S701: When the application to be called is an application deployed in MSF, the atomic service related to the application in CDF and MSF is mounted under the application to be called in MSF, and the atomic service is started.

In some implementations, before executing step S701, the method further includes:

Determine whether the atomic services related to the application program in CDF and MSF have been registered.

After the CDF and MSF functions are divided, the services provided need to be registered. For example, at the service layer, CDF registers CDF display service, camera service, and CDF voice service; MSF registers the MSF display service, MSF voice service, and AI service, sets the service capabilities, and publishes them through the service publishing module of the service layer.

If CDF and MSF have not registered the service, the atomic service is registered based on the service registration module and published based on the service publishing module.

Further, step S701 is executed by following steps S7011 to S7012.

Step S7011: Search the atomic service corresponding to the application in CDF and MSF based on the service discovery module in the service layer.

In some implementations, MSF searches for available atomic services through a service discovery module.

For example, CDF's camera service and CDF voice service will provide the ability to input image data and audio data; MSF's AI service provides AI computing and decision-making capabilities; CDF display service provides the ability to display cartoon assistant images on the cockpit screen; MSF display service provides the ability to display virtual three-dimensional characters on AR glasses; CDF voice service provides the ability to play interactive voice on the cockpit audio; MSF voice service provides the ability to play interactive voice on AR glasses, etc.

Step S7012: Based on the service connection module in the service layer, the atomic services in CDF and MSF are mounted to the application to be called in MSF.

In some implementations, after MSF finds an available atomic service, it mounts the registered atomic service to the application to be called and starts the related service.

Step S702: Acquire data based on CDF and send to MSF.

In some implementations, the MSF receives voice and image data acquired from the CDF.

Step S703: Call the service in MSF to process the data and generate response information.

In some implementations, MSF first performs audio and video synchronization processing on the acquired voice and image data, and then calls the AI service of MSF to process the relevant data.

Furthermore, MSF's AI service uses an AI data processing unit to run various related algorithms such as multi-sound zone recognition algorithm, image and voice mixed recognition algorithm, semantic processing, etc. to process the data from users. These algorithms are the core functions of the AI data processing unit, and they will be reasoned by artificial intelligence to obtain recognition results and generate response information.

The response information includes display image data, response voice data, response control commands, etc. Among them, the display image data includes the display image of the cartoon image assistant and the artificial intelligence virtual three-dimensional character image; the response voice data is the response to the user's speech and feedback in the form of voice; the response control command is the control operation made in response to the user's command, such as turning on/off the air conditioner, raising/lowering the car window, turning on/off a certain application, etc.

Step S704: Control the hardware device related to the CDF and/or the hardware device related to the MSF to perform corresponding actions based on the response information.

In some implementations, based on the service call policy pre-registered by the service layer, the corresponding information will be distributed to the hardware devices of the CDF or MSF for action.

For example, the three-dimensional virtual character image will be sent by MSF to AR glasses for display; the cartoon character image will be sent to CDF through the communication channel of the hardware layer and displayed on the display screen of the vehicle cockpit.

Through the method described in the above steps S701 to S704, better services can be provided to users with high efficiency, low latency, safety and reliability, thereby improving the user experience.

It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present application.

Furthermore, the present application also provides a driving device. In an embodiment of a driving device according to the present application, the driving device may include a driving device body and the smart cockpit computing power sharing architecture described in the above-mentioned smart cockpit computing power sharing architecture embodiment.

The following uses a smart cockpit computing power sharing framework of a multimodal artificial intelligence assistant to demonstrate how computing power sharing is achieved between the CDF and MSF of the driving device. Refer to Figure 8, which is a schematic diagram of a computing power sharing framework of a multimodal artificial intelligence assistant according to the present application.

The AI assistant is a human-computer interaction system installed on the driving device. It acts like a driving assistant to help passengers control the functions of the driving device. The basic AI assistant can interact through voice.

Furthermore, the multimodal AI assistant is an advanced version of the human-computer interaction system. In addition to voice, it can also collect user image information through sensors such as cameras and even in-car radars, recognize the user's mouth shape, expression, body, gestures, etc. Through the multimodal perception system, it can also Good listening and understanding of user commands, emotional appeals, etc. may even be called the core and soul of a smart cockpit.

In terms of input data, AI assistants require voice data from microphones, while multimodal AI assistants require voice data from microphones and image data from cameras. In terms of computing power, AI assistants only recognize and analyze user voices and make relevant responses. Multimodal AI assistants need to comprehensively analyze image and voice data for recognition and reasoning. They not only need to identify which passenger in the cabin is interacting, but also need to be able to recognize and understand the passenger's language, demeanor, movements and other implicit information. These requirements determine that multimodal voice assistants need more powerful AI computing power to meet the requirements.

The CDF and MSF of the driving device both run a SOA-based software system, which is based on the intelligent cockpit computing power sharing architecture designed in this application. From a functional perspective, the artificial intelligence assistant application can be deployed on the CDF, which relies on the microphone device in the cockpit. The multimodal artificial intelligence assistant needs to be deployed on the MSF because it requires higher computing power. The voice data and image video data it needs to input come from the CDF and are transmitted through the communication management module.

Specifically, after the functions are divided, the underlying services need to be defined. At the service layer, CDF registers CDF display service, camera service, and CDF voice service, sets the service capabilities, and publishes them through the service publishing module; MSF registers MSF display service, MSF voice service, and AI service, and also publishes them through the service publishing module of the service layer.

Now we choose to use the multimodal artificial intelligence assistant function, which will be deployed on MSF. It first searches for available atomic services through the service discovery module. CDF's camera service and CDF voice service will provide the ability to input image data and audio data; MSF's AI service provides AI computing and decision-making capabilities; CDF display service provides the ability to display cartoon assistant images on the cockpit screen; MSF display service provides the ability to display virtual three-dimensional characters on AR glasses; CDF voice service provides the ability to play interactive voice on the cockpit audio; MSF voice service provides the ability to play interactive voice on AR glasses.

Furthermore, CDF and MSF mount the registered atomic services to the multimodal artificial intelligence assistant application of MSF through the service connection module and start related services.

The multimodal artificial intelligence assistant application first obtains the voice and image data from CDF through the service call module, and then processes the audio and video synchronization of these data, and then calls the AI service of MSF to process the relevant data.

Specifically, MSF's AI service calls the underlying AI data processing unit to run various related algorithms such as multi-sound zone recognition algorithm, image and voice mixed recognition algorithm, semantic processing, etc. to process the data from users. These algorithms are the core functions of the multimodal AI assistant, and they will be inferred by AI to obtain recognition results.

After the recognition result is obtained, relevant response information is generated according to the recognition result, including: display image data, response voice data, response control command, etc. Among them, the display image data is the display image of the cartoon image assistant, or the artificial intelligence virtual three-dimensional character image; the response voice data is the response given to the user's speech, and feedback is given in the form of voice; the response control command is the control operation made in response to the user's command, such as turning on/off the air conditioner, raising/lowering the car window, turning on/off a certain application, etc.

Furthermore, according to the pre-registered service call strategy, these response messages will be distributed to the hardware devices of CDF or MSF to perform corresponding actions. For example, the 3D virtual character image will be sent by MSF to AR glasses for display; while the cartoon character image will be sent to CDF through the underlying communication channel and displayed on the display screen of the vehicle cockpit.

The above is the content of realizing computing power sharing between MSF and CDF using artificial intelligence assistant as an example.

It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

Furthermore, the present application also provides an electronic device. Refer to FIG. 9, which is a schematic diagram of the main structure of an electronic device embodiment according to the present application. As shown in FIG. 9, the electronic device in the embodiment of the present application mainly includes a processor 901 and a storage device 902. The storage device 902 can be configured to store a program for executing the computing power sharing method of the above method embodiment. The processor 901 can be configured to execute the program in the storage device 902, which includes But it is not limited to the program of the computing power sharing method of the above method embodiment. For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the method part of the embodiment of the present application.

In some possible implementations of the present application, the electronic device may include multiple processors 901 and multiple storage devices 902. The program for executing the computing power sharing method of the above method embodiment can be divided into multiple subprograms, and each subprogram can be loaded and run by the processor 901 to execute different steps of the computing power sharing method of the above method embodiment. Specifically, each subprogram can be stored in different storage devices 902, and each processor 901 can be configured to execute the program in one or more storage devices 902 to jointly implement the computing power sharing method of the above method embodiment, that is, each processor 901 executes different steps of the computing power sharing method of the above method embodiment to jointly implement the computing power sharing method of the above method embodiment.

The above-mentioned multiple processors 901 may be processors deployed on the same device. For example, the above-mentioned electronic device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors 901 may be processors configured on the high-performance device. In addition, the above-mentioned multiple processors 901 may also be processors deployed on different devices. For example, the above-mentioned electronic device may be a server cluster, and the above-mentioned multiple processors 901 may be processors on different servers in the server cluster; the above-mentioned electronic device may be a driving device cluster, and the above-mentioned multiple processors 901 may be processors on different driving devices in the driving device cluster.

Furthermore, the present application also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium may be configured to store a program for executing the computing power sharing method of the above-mentioned method embodiment, and the program may be loaded and run by the processor to implement the above-mentioned computing power sharing method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Technicians can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of this application.

Claims

A smart cockpit computing power sharing architecture, characterized in that the computing power sharing architecture includes an application layer, a system layer, a service layer, a kernel layer and a hardware layer;

The application layer includes a cockpit domain function unit CDF and a multimedia system function unit MSF, and the application layer is configured to define an application program and divide the application program into the CDF and/or the MSF according to configuration parameters;

The system layer is configured to provide at least a running environment and application management for the application layer;

The service layer is configured to provide services for the application program in the application layer;

The kernel layer is configured to provide a unified operating system kernel abstraction layer;

The hardware layer is configured to provide a communication interface.
The intelligent cockpit computing power sharing architecture according to claim 1, characterized in that the application layer is configured to divide the application into the CDF and/or the MSF according to configuration parameters, including:

Divide the different applications into the CDF or the MSF for execution according to the configuration parameters;

and / or,

The same application is run in both the CDF and the MSF.
The intelligent cockpit computing power sharing architecture according to claim 1 is characterized in that the service layer at least includes a service registration module, a service publishing module, a service discovery module, a service connection module and a service calling module;

The service registration module and the service publishing module are configured to enable the application in the CDF and the MSF to register and publish services related to the application based on the service registration module and the service publishing module respectively;

The service discovery module, the service connection module and the service call module are configured to enable the applications in the MSF and the CDF to share computing power based on the service discovery module, the service connection module and the service call module.
The intelligent cockpit computing power sharing architecture according to claim 1 is characterized in that the hardware layer is configured to provide a communication interface for a universal serial bus, Ethernet, computer electronic components, and wireless fidelity;

and / or,

The service layer and/or the system layer further includes a security management module, a communication management module and a data management module. The security management module, the communication management module and the data management module are shared by the service layer and the system layer.
A computing power sharing method based on the intelligent cockpit computing power sharing architecture according to any one of claims 1 to 4, characterized in that the method comprises:

When the application to be called belongs to an application deployed in MSF, CDF and atomic services related to the application in MSF are mounted under the application to be called in MSF, and the atomic services are started;

Acquire data based on the CDF and send to the MSF;

Calling the service in the MSF to process the data and generate response information;

Based on the response information, the hardware device related to the CDF and/or the hardware device related to the MSF is controlled to perform corresponding actions.
A computing power sharing method based on claim 5, characterized in that mounting the sub-services related to the application in the CDF and the MSF to the application to be called in the MSF comprises:

Find the atomic service corresponding to the application in the CDF and the MSF based on the service discovery module in the service layer;

Based on the service connection module in the service layer, the atomic services in the CDF and the MSF are mounted under the application to be called in the MSF.
The computing power sharing method according to claim 5 is characterized in that before mounting the atomic service related to the application in the CDF and the MSF to the application to be called in the MSF, it also includes:

Determine whether the atomic services related to the CDF and the MSF and the application have been registered;

If not registered, the atomic service is registered based on the service registration module and published based on the service publishing module.
A driving device, characterized in that the driving device comprises a driving device body and the smart cockpit computing power sharing architecture described in any one of claims 1 to 5.
An electronic device comprises a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and is characterized in that the program codes are suitable for being loaded and run by the processor to execute the computing power sharing method described in any one of claims 5 to 7.
A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are suitable for being loaded and run by a processor to execute the computing power sharing method described in any one of claims 5 to 7.