WO2024061137A1 - 车载座舱感知设备及其控制方法 - Google Patents

车载座舱感知设备及其控制方法 Download PDF

Info

Publication number
WO2024061137A1
WO2024061137A1 PCT/CN2023/119234 CN2023119234W WO2024061137A1 WO 2024061137 A1 WO2024061137 A1 WO 2024061137A1 CN 2023119234 W CN2023119234 W CN 2023119234W WO 2024061137 A1 WO2024061137 A1 WO 2024061137A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
visual perception
image
processing chip
processed
Prior art date
Application number
PCT/CN2023/119234
Other languages
English (en)
French (fr)
Inventor
王骏也
郭其江
王进
Original Assignee
虹软科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 虹软科技股份有限公司 filed Critical 虹软科技股份有限公司
Publication of WO2024061137A1 publication Critical patent/WO2024061137A1/zh

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems

Definitions

  • This application relates to but is not limited to the field of vehicle sensing technology.
  • In-cabin monitoring is a new application that has emerged in recent years. It improves the safety of drivers and passengers by monitoring the entire interior environment in real time.
  • the first method it not only requires higher computing power of the vehicle computer chip, but also increases the hardware cost of the vehicle computer.
  • the visual perception system of each vehicle model needs to be customized and developed; while for the second type, It also increases the cost of the vehicle hardware, and setting up the ECU separately will increase the complexity of the vehicle installation and wiring harness layout.
  • the disclosed embodiments provide a vehicle-mounted cockpit sensing device and a control method thereof, a storage medium, and an electronic device, so as to at least solve the technical problems of high vehicle hardware cost and high installation complexity caused by integrating the cockpit visual perception system into the vehicle system or ECU.
  • a vehicle-mounted cockpit perception device in which an image acquisition device, an image sensor, a visual perception processing chip, and a vehicle-mounted processor are integrated, wherein the image acquisition device is used to acquire an image to be processed; the input end of the image sensor is connected to the output end of the image acquisition device, for receiving the image to be processed and converting the image to be processed into an image signal to be processed; the input end of the visual perception processing chip is connected to the output end of the image sensor, for receiving the image signal to be processed and performing visual perception processing on the image signal to be processed to obtain a perception result; the input end of the vehicle-mounted processor is connected to the output end of the visual perception processing chip, for receiving the perception result.
  • performing visual perception processing on the image signal to be processed to obtain a perception result includes: reading pre-stored object feature points; extracting image feature points of the image signal to be processed; and comparing the result. The object feature points and the image feature points are combined to obtain the perception result.
  • the method before extracting the image feature points of the image signal to be processed, the method further includes: Perform image signal processing debugging on the image signal to be processed.
  • the vehicle cabin sensing device is also integrated with: a memory, wherein the memory is connected to the visual perception processing chip and configured to store the object feature points.
  • the vehicle-mounted cabin sensing device is also integrated with: one or more vehicle-mounted modules, wherein the input end of the one or more vehicle-mounted modules is connected to the output end of the vehicle-mounted processor, and is configured to To receive the sensing result transmitted by the on-board processor.
  • the vehicle cabin sensing device is also integrated with: a serializer, wherein the input end of the serializer is connected to the output end of the visual perception processing chip, and the serializer The output end is connected to the input end of the one or more vehicle-mounted modules, and is configured to receive the image signal to be processed and output the image signal to be processed to the one or more vehicle-mounted modules.
  • the vehicle cockpit sensing device is also integrated with: a power supply, wherein the output end of the power supply is connected to the input end of the visual perception processing chip and is configured to provide power to the visual perception processing chip. .
  • the vehicle-mounted cockpit sensing device is further integrated with: a power converter, wherein the input end of the power converter is connected to the output end of the power supply, and the output end of the power converter is connected to the input end of the image acquisition device and the image sensor, and is configured to convert the power supply to power the image acquisition device and the image sensor.
  • the vehicle-mounted processor includes: a system fault register configured to monitor the working status of the visual perception processing chip, and update and store the visual perception processing chip fault information if the visual perception processing chip fails. ;
  • the power supply fault register is configured to monitor the working status of the power supply, and update and store the power supply fault information if the power supply fails.
  • the vehicle-mounted processor sends the visual perception processing chip fault information and the power supply fault information to the one or more vehicle-mounted modules according to a fault diagnosis protocol.
  • a method for controlling a vehicle cabin sensing device including: collecting images to be processed; integrating the images to be processed into image signals to be processed; and processing the image signals to be processed. Perform visual perception processing to obtain a perception result; transmit the perception result to the on-vehicle processor.
  • the method for controlling a vehicle-mounted cabin sensing device further includes: receiving a service upgrade request initiated by one or more vehicle-mounted modules; responding to the service upgrade request, sending a service upgrade package to the vehicle-mounted processor;
  • the vehicle-mounted processor parses the service upgrade package and stores part of the service upgrade firmware;
  • the vehicle-mounted processor sends the service upgrade firmware of the visual perception processing chip to the visual perception processing chip;
  • the visual perception processing chip The service upgrade firmware is saved in the memory; the visual perception processing chip performs service upgrade, and after the upgrade is completed, sends the first upgrade result to the vehicle-mounted processor; the vehicle-mounted processor performs service upgrade, and in After the upgrade is completed, the second upgrade result is sent to the one or more vehicle-mounted modules.
  • a computer-readable storage medium which stores computer-executable instructions.
  • the computer-executable instructions are used to perform control of the vehicle cabin sensing device provided by any embodiment of the present disclosure. method.
  • the image to be processed collected by the image acquisition device is converted into an image signal to be processed through an image sensor, and is transmitted to the visual perception processing chip, and then the visual perception processing chip performs visual perception processing on the image signal to be processed to obtain a perception result. , and transmit the sensing results to the vehicle-mounted processor, and transmit the sensing results to other vehicle-mounted modules through the vehicle-mounted processor.
  • the disclosed embodiment integrates the image acquisition function and the image processing function in the vehicle cockpit sensing device. While realizing cockpit sensing and monitoring, it also supports fault diagnosis and remote system upgrade. It also has strong car model adaptability and strong multi-platform compatibility. Reduce the cost of vehicle hardware, thereby solving the problem of integrating the cockpit visual perception system in related technologies. In the vehicle system or ECU, there are technical issues such as increased vehicle hardware costs and high vehicle installation complexity.
  • Figure 1 is a hardware block diagram of a vehicle cockpit sensing device according to an embodiment of the present disclosure
  • FIG. 2 is a hardware block diagram of another vehicle cabin sensing device according to an embodiment of the present disclosure.
  • Figure 3 is a flow chart of a control method of a vehicle cabin sensing device according to an embodiment of the present disclosure
  • FIG. 4 is a hardware block diagram of a vehicle cockpit sensing device that implements DMS system functions according to an embodiment of the present disclosure
  • FIG. 5 is a hardware block diagram of a vehicle cabin sensing device that implements OMS system functions according to an embodiment of the present disclosure.
  • Cockpit monitoring mainly includes DMS and OMS systems.
  • DMS Driver Monitor System
  • a Driver Monitor System refers to a system that monitors the driver's fatigue status and dangerous driving behavior around the clock while driving.
  • DMS functions include fatigue detection (for example, yawning, closing eyes), dangerous action detection (smoking, talking on the phone), distraction detection, and gesture recognition.
  • OMS Occupancy Monitoring System
  • OMS Occupancy Monitoring System
  • OMS functions include rear-seat left-behind detection, left-behind pet detection and left-behind child detection.
  • LVDS Low-Voltage Differential Signaling
  • the core is the use of extremely low voltage swing for high-speed differential transmission of data. , can realize point-to-point or point-to-multipoint connection, and its transmission medium can be copper PCB connection or balanced cable.
  • UART Universal Asynchronous Receiver/Transmitter
  • the I2C bus is a simple, bidirectional, two-wire synchronous serial bus that requires only two wires to transmit information between devices connected to the bus.
  • MIPI Mobile Industry Processor Interface
  • Serial Peripheral Interface Serial Peripheral Interface
  • SPI Serial Peripheral Interface
  • Controller Area Network Controller Area Network, referred to as CAN.
  • Over-the-air download technology Over The Air, referred to as OTA, enables remote management of software through mobile communication interfaces.
  • MCU Microcontroller Unit
  • This application proposes a vehicle-mounted intelligent cockpit visual perception device, through which one or more of the driver's face recognition function, DMS function and OMS function can be realized.
  • the device meets functional safety requirements such as real-time fault diagnosis, and can perform OTA upgrades to the system through the CAN bus.
  • the present application will be described in detail below with reference to various embodiments.
  • FIG. 1 is a hardware block diagram of a vehicle-mounted cabin sensing device according to an embodiment of the present disclosure.
  • the vehicle-mounted cabin sensing device 100 integrates an image acquisition device 101, an image sensor 102, a visual perception processing chip 103, and a vehicle-mounted processor 104.
  • the vehicle cabin sensing device is a camera
  • the image acquisition device 101 is the lens of the camera.
  • the image acquisition device 101 is configured to acquire images to be processed.
  • the vehicle cockpit sensing device 100 in some exemplary embodiments may first complete self-detection and initialize the image sensor 102 after being powered on.
  • the input end of the image sensor 102 is connected to the output end of the image acquisition device 101.
  • the image acquisition device 101 transmits the image to be processed to the image sensor 102.
  • the image sensor 102 is configured to convert the image to be processed into an image signal to be processed.
  • the image sensor 102 supports a variety of vehicle-standard sensors including, but not limited to, 100W and 200W pixels from OV, ON and other manufacturers.
  • the image acquisition device 101 and the image sensor 102 constitute a camera module or a sensing module, and the camera module collects image signals to be processed.
  • the image signals collected by the camera module in some exemplary embodiments include but are not limited to: RGB image signals, gray and white images and depth image signals, for each type of vehicle (the vehicle type in this embodiment includes but is not limited to: car ( Such as pure electric vehicles, hybrid vehicles, fuel vehicles), trucks, commercial vehicles, buses, etc.) shall prevail.
  • the input end of the visual perception processing chip 103 is connected to the output end of the image sensor 102.
  • the image sensor 102 transmits the image signal to be processed to the visual perception processing chip 103.
  • the visual perception processing chip 103 performs visual perception processing on the image signal to be processed to obtain a perception result. .
  • the input terminal of the visual perception processing chip 103 communicates with the output terminal of the image sensor 102.
  • the first type of cable includes but is not limited to: FPC (Flexible printed board, flexible circuit board) cable.
  • the FPC cable includes but is not limited to MIPI transmission lines, clock signal lines, control signal lines, I2C line.
  • the image sensor 102 may also output clock signals and control signals to the visual perception processing chip 103 through the first type of cable.
  • the input terminal of the vehicle-mounted processor 104 is connected to the output terminal of the visual perception processing chip 103, and the vehicle-mounted processor 104 receives the perception results transmitted by the visual perception processing chip 103.
  • the input end of the vehicle-mounted processor 104 and the output end of the visual perception processing chip 103 are connected through a first type bus, and the first type bus includes but is not limited to: a UART bus.
  • FIG. 2 is a hardware block diagram of another vehicle-mounted cabin sensing device according to an embodiment of the present disclosure.
  • the vehicle-mounted cabin sensing device 200 integrates an image acquisition device 201, an image sensor 202, a visual perception processing chip 203 and a vehicle-mounted processor 204.
  • the image The acquisition device 201 is configured to acquire images to be processed.
  • the input end of the image sensor 202 is connected to the output end of the image acquisition device 201.
  • the image acquisition device 201 transmits the image to be processed to the image sensor 202, and the image sensor 202 converts the image to be processed into an image signal to be processed.
  • the input end of the visual perception processing chip 203 is connected to the output end of the image sensor 202.
  • the image sensor 202 transmits the image signal to be processed to the visual perception processing chip 203.
  • the visual perception processing chip 203 performs visual perception processing on the image signal to be processed to obtain a perception result. .
  • the input terminal of the vehicle-mounted processor 204 is connected to the output terminal of the visual perception processing chip 203, and the vehicle-mounted processor 204 receives the perception results transmitted by the visual perception processing chip 203.
  • the visual perception processing chip 203 performs visual perception processing on the image signal to be processed to obtain a perception result, including: reading pre-stored object feature points; extracting image feature points of the image signal to be processed; and comparing object feature points. and image feature points to obtain the perception results.
  • performing image signal processing debugging on the image signal to be processed includes: the visual perception processing chip 203 performs image perception processing (Image Signal Process, ISP) debugging on the image signal to be processed, obtains the debugged image signal, and then performs visual perception algorithm processing on the debugged image signal to obtain the perception result, wherein the visual perception algorithm processing is performed in the neural network processing unit (Neural Networks Process Units, NPU) of the visual perception processing chip 203.
  • image perception processing Image Signal Process, ISP
  • NPU neural network processing unit
  • the vehicle cabin sensing device 200 is also integrated with: one or more vehicle-mounted modules, wherein the input end of the one or more vehicle-mounted modules is connected to the output end of the vehicle-mounted processor 204 and is configured to receive vehicle-mounted processing. The perception result transmitted by the device.
  • vehicle-mounted module in this embodiment may be but is not limited to: a vehicle engine or an ECU (Electronic Control Unit).
  • the vehicle-mounted module is configured to remind drivers and passengers or trigger other driving/riding assistance functions based on the received sensing results. For example, triggering functions such as sound reminders.
  • the vehicle-mounted processor 204 is also configured to receive CAN bus messages; obtain new reminder messages based on the perception results transmitted from the visual perception processing chip 203 and the CAN bus messages, and send them via the CAN bus Transmitted to the vehicle-mounted module;
  • the CAN bus message includes one or more of the following: vehicle speed, steering information;
  • the new reminder message includes: sensing results and prompt information.
  • the vehicle-mounted processor 204 is further configured to obtain a service upgrade package, parse and obtain the service firmware and local service firmware of the visual perception chip, and send the service firmware of the visual perception chip to the visual perception chip.
  • the processing chip 203 obtains the first upgrade result returned by the visual perception processing chip 203; completes the local firmware upgrade according to the local service firmware;
  • the service upgrade package comes from the one or more vehicle-mounted modules.
  • the vehicle-mounted processor 204 is further configured to determine a second upgrade result based on the local firmware upgrade result and the first upgrade result and send it to the one or more vehicle-mounted modules.
  • the vehicle cabin sensing device 200 is also integrated with: a serializer 205, wherein the input end of the serializer 205 is connected to the output end of the visual perception processing chip 203, and is configured to receive the image signal to be processed, And the output end of the serializer is connected to the input end of one or more vehicle-mounted modules, and is configured to output the image signal to be processed to the one or more vehicle-mounted modules.
  • connection between the serializer 205 and the visual perception processing chip 203 can be implemented through various interfaces, for example, through the MIPI interface.
  • the serializer 205 and one or more vehicle modules may be connected via a second type bus, and the vehicle processor 204 and one or more vehicle modules may be connected via a third type bus.
  • the second type bus includes: an LVDS bus
  • the third type bus includes: a CAN bus.
  • the vehicle cabin sensing device is also integrated with: memory 206, where the memory 206 is connected to the visual perception processing chip 203 and is configured to store object feature points.
  • the types of the schematically illustrated serializer 205 and memory 206 can be selected by oneself.
  • the serializer 205 can be various types of serializers from TI (Texas Instruments) or Maxim (Maxim), or other Serializer for MIPI interface.
  • the vehicle cockpit sensing device 200 is also integrated with: a power supply 207 , wherein the output end of the power supply 207 is connected to the power input end of the visual perception processing chip 203 and configured to provide power to the visual perception processing chip 203 .
  • the vehicle cabin sensing device 200 is also integrated with: a power converter 208, wherein the input end of the power converter 208 is connected to the output end of the power supply 207, and the output end of the power converter 208 is connected to the image acquisition device.
  • the power input terminals of the device 201 and the image sensor 202 are connected, and are configured to convert the power output from the power supply 207 to provide power to the image acquisition device 201 and the image sensor 202 .
  • the on-board processor 204 is connected to the ignition switch through a fourth type bus, where the fourth type bus includes: an ACC bus, that is, an adaptive cruise control bus.
  • the vehicle cockpit sensing device 200 is also integrated with: a system fault register 210, which is configured to monitor the working status of the visual perception processing chip 203. If the visual perception processing chip 203 fails, update and store the visual perception chip fault. Information; the power failure register 211 is set to monitor the working status of the power supply 207. If the power supply 207 fails, update and store the power failure information.
  • the power failure register 211 determines whether the power supply 207 fails based on the working status, generates power failure information, and displays the power failure information.
  • system fault register 210 and power fault register 211 are provided in the on-board processor 204.
  • the vehicle-mounted processor 204 receives a fault alarm from the visual perception processing chip 203, the information of the system fault register 210 stored internally in the vehicle-mounted processor 204 is updated. At the same time, the vehicle-mounted processor 204 regularly monitors the working status of the power supply 207. After discovering a power failure, the vehicle-mounted processor 204 updates the power failure information stored inside the vehicle-mounted processor 204. information in register 211.
  • the vehicle-mounted processor 204 may send the visual perception chip fault information and the power supply fault information to one or more vehicle-mounted modules through the third type bus according to the fault diagnosis protocol.
  • the vehicle-mounted cockpit perception device 200 by integrating the image acquisition device 201 and the visual perception processing chip 203 in the vehicle-mounted cockpit perception device 200, while realizing cockpit perception and monitoring, it supports fault diagnosis and remote system upgrades, making the vehicle model adaptable and multi-platform compatible, reducing the hardware cost of the whole vehicle, thereby solving the technical problems of integrating the cockpit visual perception system into the vehicle system or ECU in the related technology, increasing the hardware cost of the whole vehicle, and high complexity of the whole vehicle installation.
  • the vehicle-mounted cockpit perception device 200 is independent of the vehicle system or ECU, which can simplify its installation complexity.
  • the present application also provides a method for controlling a vehicle cabin sensing device. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the steps in the flow chart A logical order is shown, but in some cases the steps shown or described may be performed in a different order than herein.
  • Figure 3 is a flow chart of an optional control method for vehicle cockpit sensing equipment provided by an embodiment of the present disclosure. The method includes the following steps:
  • Step S302 collect images to be processed
  • Step S304 convert the image to be processed into an image signal to be processed
  • Step S306 Perform visual perception processing on the image signal to be processed to obtain a perception result
  • Step S308 transmit the sensing results to the vehicle-mounted processor.
  • the image to be processed can be collected first through the image acquisition device, and then the image to be processed is converted into the image signal to be processed through the image sensor, and the image signal to be processed is subjected to visual perception processing through the visual perception processing chip to obtain the perception result, and
  • the sensing results are transmitted to the onboard processor.
  • the vehicle-mounted cockpit sensing device integrates image acquisition function and visual perception processing function. While realizing cockpit sensing and monitoring, it supports fault diagnosis and remote system upgrade. At the same time, it has strong car model adaptability and is multi-platform. It has strong compatibility and reduces the cost of vehicle hardware, thereby solving the technical problems of integrating the cockpit visual perception system into the vehicle system or ECU, which increases the cost of vehicle hardware and makes vehicle installation more complex.
  • the step of collecting images to be processed includes: after the vehicle-mounted cabin sensing device is powered on, controlling the image sensor of the vehicle-mounted cabin sensing device to initialize; after the image sensor completes the initialization, controlling the vehicle-mounted image acquisition device to collect the scheduled images inside the vehicle images of spatial regions.
  • the step of performing visual perception processing on the image signal to be processed to obtain the perception result includes: reading the object feature points pre-stored in the memory through the visual perception processing chip; extracting the image signal to be processed through the visual perception processing chip The image feature points in the object are compared with the image feature points to obtain the perception result.
  • the object feature points may include facial feature points of the user object, such as facial feature points, facial prominent feature points, etc.
  • the control method further includes: receiving a service upgrade request initiated by one or more vehicle-mounted modules; responding to the service upgrade request, sending the service upgrade package to the vehicle-mounted processor; the vehicle-mounted processor parses the service upgrade package and stores part of the service firmware; the vehicle-mounted processor sends the service firmware of the visual perception chip to the visual perception processing chip; the visual perception processing chip saves the service firmware to its on-chip memory; the visual perception processing chip performs a service upgrade, and after the upgrade is completed, sends the first upgrade result to the vehicle-mounted processor; the vehicle-mounted processor performs a preset service upgrade, and after the upgrade is completed, sends the second upgrade result to one or more vehicle-mounted modules.
  • the vehicle-mounted processor executes the locally stored part of the service firmware and performs a local firmware upgrade, that is, the local preset service upgrade is completed.
  • the service upgrade request includes a service upgrade package.
  • control method further includes:
  • the vehicle-mounted processor obtains the service upgrade package, analyzes and obtains the service firmware and local service firmware of the visual perception chip; sends the service firmware of the visual perception chip to the visual perception processing chip, and obtains the service firmware returned by the visual perception processing chip.
  • the first upgrade result complete the local firmware upgrade according to the local service firmware
  • the service upgrade package comes from the one or more vehicle-mounted modules.
  • the visual perception processing chip saves the service firmware to its on-chip memory; the visual perception processing chip performs service upgrades, and after the upgrade is completed, sends the first upgrade result to the on-board processor;
  • control method further includes: the vehicle-mounted processor determines a second upgrade result based on the local firmware upgrade result and the first upgrade result and sends it to the one or more vehicle-mounted modules.
  • the image sensor and the image acquisition device are integrated into a sensing module.
  • the first system DMS system.
  • the vehicle cabin sensing device is integrated in the DMS system.
  • the DMS system can realize driver face recognition and DMS functions.
  • DMS functions include but are not limited to: fatigue detection (yawning, closing eyes), dangerous action detection (smoking, talking on the phone), distraction detection, and gesture recognition.
  • FIG. 4 is a hardware block diagram of an optional cabin sensing device according to an embodiment of the present disclosure, applied in a DMS system.
  • the vehicle cabin sensing device includes: a DMS system sensing module and a DMS visual sensing processing board, where,
  • the DMS system sensing module at least includes: an image acquisition device and an image sensor. After collecting the image to be processed through the image acquisition device, the image sensor integrates the collected image to be processed into an image signal to be processed, and transmits it to the DMS visual perception processing board. .
  • the image acquisition device may be the lens of a DMS camera.
  • the DMS system sensing module and the DMS visual perception processing board are integrated in the DMS camera.
  • the vehicle cabin sensing device is a DMS camera.
  • the DMS visual perception processing board includes: visual perception processing chip, on-board processor, serializer, memory and power supply.
  • the visual perception processing chip is connected to the image sensor in the DMS system sensing module through the FPC cable and through the MIPI line. It is connected to the serializer, connected to the on-board processor through the UART bus, and connected to the memory through the serial peripheral interface SPI.
  • the visual perception processing chip After receiving the image signal to be processed, the visual perception processing chip performs visual perception processing on the image signal to be processed, obtains the perception result, and transmits the perception result to the vehicle-mounted processor through the UART bus.
  • the vehicle-mounted processor transmits the perception result to the vehicle through the CAN bus. machine or other vehicle ECU.
  • the serializer is connected to the vehicle computer or ECU through the LVDS bus, and the ignition switch and the on-board processor are connected through the key-controlled ACC bus.
  • electric energy is provided through the car level to the power supply in the DMS visual perception processing board.
  • the power supply supplies power to the visual perception processing chip.
  • the electric energy in the power supply is converted by the power converter to power the DMS system sensing module.
  • the above-mentioned vehicle cockpit sensing equipment includes two parts: the DMS visual perception processing board and the DMS system sensing module.
  • the DMS visual perception processing board includes: visual perception processing chip, on-board processor MCU, serializer, memory and power supply.
  • the DMS system sensing module includes an image sensor and an image acquisition device.
  • the DMS system sensing module transmits the MIPI signal, RST control signal and MCLK clock signal that transmits the image to the DMS visual perception processing board through the FPC cable.
  • the DMS visual perception processing board After receiving the image data from the DMS system sensing module, performs visual perception algorithm processing on the image on the visual perception processing chip to obtain the perception results, and transmits the perception results to the vehicle-mounted processor through the UART bus.
  • the processor transmits the sensing results to the vehicle computer or other vehicle ECUs through the CAN bus.
  • the visual perception processing chip in this embodiment transmits the image signal to the serializer through MIPI, and the serializer transmits the image signal to the vehicle machine or other vehicle-mounted ECU through LVDS.
  • the workflow includes:
  • the vehicle cockpit sensing equipment is powered on and starts self-test. At the same time, the image sensor is initialized;
  • control the sensing module to collect images of a predetermined space area inside the vehicle to obtain the image signal to be processed
  • the MIPI signal, RST control signal and MCLK clock signal of the transmitted image are transmitted to the DMS visual perception processing board through the FPC cable, and the visual perception processing chip performs ISP (image signal processing) debugging on the image signal;
  • the visual perception processing chip transmits the perception results to the vehicle-mounted processor through the UART bus, and at the same time, outputs the image signal to the serializer through MIPI, and then the serializer outputs the image signal to the machine or other vehicle-mounted ECU through LVDS;
  • the vehicle-mounted processor After the vehicle-mounted processor receives the sensing results, it transmits the sensing results to the vehicle computer or other vehicle-mounted ECUs through the CAN bus.
  • the user Before using the face recognition function of the vehicle-mounted DMS system, the user needs to perform a face registration operation.
  • the user can upload the face image to the system according to the system prompts or through active upload or active input, and the system will recognize the face image. , obtain the facial features (including facial features, multiple facial point features, etc.), and save the facial features to the memory.
  • the face image signals collected by the DMS system sensing module can be transmitted to the DMS visual perception processing board.
  • the visual perception processing chip extracts facial features, compares the extracted features with the facial features stored in the memory, and sends the comparison results, that is, the perception results to the on-board processor, and then the on-board processor can Sent to the vehicle computer or other vehicle-mounted ECU through CAN protocol.
  • detection feature data required to implement functions such as fatigue detection (yawning, closing eyes), dangerous action detection (smoking, talking on the phone), distraction detection, gesture recognition, etc. are also pre-stored in the memory.
  • the following is a schematic illustration of the fault diagnosis operation of the vehicle-mounted cockpit sensing device.
  • the visual perception processing chip regularly monitors the working status of each hardware in the DMS system. If the working status of any hardware indicates that it is faulty, it will send a fault alarm to the on-board processor through UART;
  • the system fault register in the vehicle-mounted processor updates and stores the visual perception chip fault information.
  • the on-board processor regularly monitors the working status of the power supply. If the working status of the power supply indicates a power failure, the power failure register in the on-board processor updates and stores the power failure information.
  • the vehicle-mounted processor sends the visual perception chip fault information and power supply fault information to the vehicle engine or other vehicle-mounted ECU through the CAN bus according to the fault diagnosis protocol.
  • the vehicle computer or other vehicle-mounted ECU initiates an OTA upgrade request and sends the OTA upgrade package to the vehicle through the CAN bus. loaded into the processor;
  • the on-board processor parses the OTA upgrade package and stores part of the OTA firmware into the internal storage of the on-board processor;
  • the on-board processor sends the OTA firmware of the visual perception processing chip to the visual perception processing chip;
  • the visual perception processing chip saves its OTA firmware to the internal memory of the visual perception processing chip
  • the visual perception processing chip performs an OTA upgrade. After the upgrade is completed, the upgrade result (i.e., the first upgrade result) is sent to the vehicle-mounted processor through the UART bus;
  • the vehicle-mounted processor performs an OTA upgrade. After the upgrade is completed, the upgrade result (ie, the second upgrade result) is sent to the vehicle computer or other vehicle-mounted ECU.
  • the second system OMS system.
  • the vehicle cabin sensing device is integrated in the OMS system.
  • the OMS system can implement OMS functions, which include but are not limited to: detection of objects left behind in the rear row, detection of pets left behind, and detection of children left behind.
  • FIG. 5 is a hardware block diagram of an optional cockpit sensing device according to an embodiment of the present disclosure.
  • the cockpit sensing device includes: an OMS system sensing module and an OMS visual perception processing board, where,
  • the OMS system sensing module at least includes: an image acquisition device and an image sensor. After the image acquisition device acquires the image to be processed, the image sensor integrates the acquired image to be processed into an image signal to be processed and transmits it to the OMS visual perception processing board.
  • the image acquisition device may be the lens of an OMS camera.
  • the OMS system sensing module and the OMS visual perception processing board are integrated in the OMS camera.
  • the vehicle cabin sensing device is an OMS camera.
  • the OMS visual perception processing board includes: visual perception processing chip, on-board processor, serializer, memory and power supply.
  • the visual perception processing chip is connected to the image sensor in the OMS system sensing module through the FPC cable and through the MIPI line. It is connected to the serializer, connected to the on-board processor through the UART bus, and connected to the memory through the serial peripheral interface SPI.
  • the visual perception processing chip After receiving the image signal to be processed, the visual perception processing chip performs visual perception processing on the image signal to be processed, obtains the perception result, and transmits the perception result to the vehicle-mounted processor through the UART bus.
  • the vehicle-mounted processor transmits the perception result to the vehicle through the CAN bus. machine or other vehicle ECU.
  • the serializer is connected to the vehicle computer or ECU through the LVDS bus, and the ignition switch and the on-board processor are connected through the key-controlled ACC bus.
  • electric energy is provided through the car level to the power supply in the OMS visual perception processing board.
  • the power supply supplies power to the visual perception processing chip.
  • the electric energy in the power supply is converted by the power converter to power the OMS system sensing module.
  • the above-mentioned cockpit sensing equipment includes two parts: the OMS visual perception processing board and the OMS system sensing module.
  • the OMS visual perception processing board includes: visual perception processing chip, on-board processor MCU, serializer, memory and power supply.
  • the OMS system sensing module includes an image sensor and an image acquisition device.
  • the OMS system sensing module transmits the MIPI signal, RST control signal and MCLK clock signal that transmits the image to the OMS visual perception processing board through the FPC cable.
  • the OMS visual perception processing board After receiving the image data from the OMS system sensing module, performs visual perception algorithm processing on the image on the visual perception processing chip to obtain the perception results, and transmits the perception results to the vehicle-mounted processor through the UART bus.
  • the processor transmits the sensing results to the vehicle computer or other vehicle ECUs through the CAN bus.
  • the visual perception processing chip in this embodiment transmits the image to the serializer through MIPI, and the serializer transmits the image to the vehicle computer or other vehicle-mounted ECU through LVDS.
  • the OMS visual perception processing board is directly powered by the car level, and the power supply The converter converts the electric energy provided by the car level and supplies power to the OMS system sensing module.
  • the workflow includes:
  • the vehicle cockpit sensing equipment is powered on and starts self-test. At the same time, the image sensor is initialized;
  • control the sensing module to collect images of a predetermined space area inside the vehicle to obtain the image signal to be processed
  • the MIPI signal, RST control signal and MCLK clock signal that transmits the image are transmitted to the OMS visual perception processing board through the FPC cable.
  • the visual perception processing chip performs ISP (Image Signal Processing) debugging on the image signal;
  • the visual perception processing chip transmits the perception results to the on-board processor through the UART bus. At the same time, it outputs the image signal to the serializer through MIPI, and then the serializer outputs the image signal to the machine or other on-board ECU through LVDS;
  • the vehicle-mounted processor After the vehicle-mounted processor receives the sensing results, it transmits the sensing results to the vehicle computer or other vehicle-mounted ECUs through the CAN bus.
  • the following is a schematic illustration of the face registration and face recognition process of the OMS system.
  • users Before using the face recognition function of the vehicle-mounted OMS system, users need to perform face registration operations. Users can upload face images to the system according to system prompts or through active upload or active input, and the system will recognize the face images. , obtain the facial features (including facial features, multiple facial point features, etc.), and save the facial features to the memory.
  • the system's face recognition function can be used.
  • the face images collected by the OMS system sensing module can be transmitted to the OMS visual perception processing board, and the face images collected by the OMS system sensing module can be transmitted through the visual perception processing board.
  • the perception processing chip extracts facial features, compares the extracted features with the facial features stored in the memory, and sends the comparison result, that is, the perception result to the on-board processor, and then the on-board processor can pass the perception result through
  • the CAN protocol is sent to the vehicle computer or other vehicle ECUs.
  • the detection characteristic data required to implement functions such as detection of objects left behind in the rear seat, detection of left pets, detection of left behind children, and detection of passenger movements are also stored in the memory in advance.
  • the visual perception processing chip regularly monitors the working status of each hardware in the OMS system. If the working status of a hardware indicates that it has failed, a fault alarm is sent to the on-board processor via UART;
  • the system fault register in the vehicle-mounted processor updates and stores the visual perception chip fault information.
  • the on-board processor regularly monitors the working status of the power supply. If the working status of the power supply indicates that the power supply has failed, the power supply failure register in the on-board processor updates and stores the power supply failure information.
  • the vehicle-mounted processor sends the visual perception chip fault information and power supply fault information to the vehicle engine or other vehicle-mounted ECU through the CAN bus according to the fault diagnosis protocol.
  • the vehicle computer or other vehicle-mounted ECU initiates an OTA upgrade request and sends the OTA upgrade package to the vehicle-mounted processor through the CAN bus;
  • the on-board processor parses the OTA upgrade package and stores part of the OTA firmware into the internal storage of the on-board processor;
  • the on-board processor sends the OTA firmware of the visual perception processing chip to the visual perception processing chip;
  • the visual perception processing chip saves its OTA firmware to the internal memory of the visual perception processing chip
  • the visual perception processing chip performs an OTA upgrade. After the upgrade is completed, the upgrade result (i.e., the first upgrade result) is sent to the vehicle-mounted processor through the UART bus;
  • the vehicle-mounted processor performs an OTA upgrade. After the upgrade is completed, the upgrade result (ie, the second upgrade result) is sent to the vehicle computer or other vehicle-mounted ECU.
  • the DMS system and the OMS system are integrated and share the same sensing module (including an image sensor and an image acquisition device), that is, the same sensing module serves as the DMS system sensing module and also as the DMS system sensing module.
  • the OMS system sensing module shares the same visual perception processing board, that is, the same visual perception processing board serves as the DMS visual perception processing board and also serves as the OMS visual perception processing board, and has pre-stored the characteristics that need to be compared. All required characteristics.
  • the vehicle cockpit sensing equipment includes: sensing module and visual perception processing board to realize DMS system functions and OMS system functions.
  • the vehicle cabin sensing device is a camera arranged in the vehicle. That is, based on the sensing module (image sensor and image acquisition device) included in the camera, a visual perception processing board is integrated.
  • the disclosed technical content can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units may be a logical functional division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or may be Integrated into another system, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the units or modules may be in electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above integrated units can be implemented in the form of hardware or software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solution of the embodiments of the present disclosure is essentially or contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage
  • the medium includes several instructions to cause a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

一种车载座舱感知设备及其控制方法，以及执行该方法的计算机可读存储介质。车载座舱感知设备（100）中集成有图像采集装置（101），图像传感器（102），视觉感知处理芯片（103）和车载处理器（104）。其中，图像采集装置（101）设置为采集待处理图像；图像传感器（102）的输入端与图像采集装置（101）的输出端连接，设置为接收待处理图像并将待处理图像转换为待处理图像信号；视觉感知处理芯片（103）的输入端与图像传感器（102）的输出端连接，设置为接收待处理图像信号并对待处理图像信号进行视觉感知处理，得到感知结果；车载处理器（104）的输入端与视觉感知处理芯片（103）的输出端连接，设置为接收感知结果。该车载座舱感知设备克服了将座舱视觉感知系统集成在车机或ECU中所引起的整体硬件成本高，安装复杂的问题。

Description

车载座舱感知设备及其控制方法
本申请要求于2022年09月19日提交中国专利局，申请号为202211137970.7，发明名称为“座舱感知设备及其控制方法”的中国专利申请的优先权，其内容应理解为通过引用的方式并入本申请中。
技术领域
本申请涉及但不限于车辆感知技术领域。
背景技术
随着汽车智能化技术的不断普及，汽车内部开始设置智能座舱，汽车智能座舱通过智能座舱内饰和座舱电子的联动，实现人、路、车的智能交互，是重新定义人车关系的智能车载产品。汽车驾驶舱是最直观体现汽车科技的载体，伴随个性化消费和高端消费趋势的持续加强，以智能化和科技感为代表的座舱电子可满足消费者对智慧化驾驶配置的需求。
座舱监控(In-cabin monitoring)是近年来兴起的一种新应用，通过实时监控整个车内环境来提高驾驶及乘客的安全。在一些技术方案中存在两种座舱监控方案：第一种，将座舱视觉感知系统放在车机系统中集成；第二种，将座舱视觉感知系统集成在专用的ECU中，但是这两种集成方式存在明显的弊端：对于第一种，不仅对车机芯片计算能力要求较高，且增加了车机的硬件成本，每个车型车机的视觉感知系统需要定制开发；而对于第二种，同样增加了整车硬件成本，且单独设置ECU会增加整车安装和线束布置的复杂度。
针对上述的问题，目前尚未提出有效的解决方案。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本公开实施例提供了一种车载座舱感知设备及其控制方法、存储介质、电子设备，以至少解决将座舱视觉感知系统集成于车机系统或ECU中造成的整车硬件成本高，安装复杂度高的技术问题。
根据本公开实施例的一个方面，提供了一种车载座舱感知设备，所述车载座舱感知设备中集成有图像采集装置，图像传感器，视觉感知处理芯片，车载处理器，其中，所述图像采集装置用于采集待处理图像；所述图像传感器的输入端与所述图像采集装置的输出端连接，用于接收所述待处理图像并将所述待处理图像转换为待处理图像信号；所述视觉感知处理芯片的输入端与所述图像传感器的输出端连接，用于接收所述待处理图像信号并对所述待处理图像信号进行视觉感知处理，得到感知结果；所述车载处理器的输入端与所述视觉感知处理芯片的输出端连接，用于接收所述感知结果。
一些示例性实施例中，所述对所述待处理图像信号进行视觉感知处理，得到感知结果，包括：读取预先存储的对象特征点；提取所述待处理图像信号的图像特征点；对比所述对象特征点与所述图像特征点，得到所述感知结果。
一些示例性实施例中，在所述提取所述待处理图像信号的图像特征点之前，还包括： 对所述待处理图像信号进行图像信号处理调试。
一些示例性实施例中，所述车载座舱感知设备中还集成有：存储器，其中，所述存储器与所述视觉感知处理芯片连接，设置为存储所述对象特征点。
一些示例性实施例中，所述车载座舱感知设备中还集成有：一个或多个车载模块，其中，所述一个或多个车载模块的输入端与所述车载处理器的输出端连接，设置为接收所述车载处理器传输的所述感知结果。
一些示例性实施例中，所述车载座舱感知设备中还集成有：串化器，其中，所述串化器的输入端与所述视觉感知处理芯片的输出端连接，且所述串化器的输出端与所述一个或多个车载模块的输入端连接，设置为接收所述待处理图像信号，并将所述待处理图像信号输出至所述一个或多个车载模块。
一些示例性实施例中，所述车载座舱感知设备中还集成有：电源，其中，所述电源的输出端与所述视觉感知处理芯片的输入端连接，设置为为所述视觉感知处理芯片供电。
一些示例性实施例中，所述车载座舱感知设备中还集成有：电源转换器，其中，所述电源转换器的输入端与所述电源的输出端连接，且所述电源转换器的输出端与所述图像采集装置和所述图像传感器的输入端连接，设置为对所述电源进行转换，为所述图像采集装置和所述图像传感器供电。
一些示例性实施例中，所述车载处理器包括：系统故障寄存器，设置为监控所述视觉感知处理芯片的工作状态，若所述视觉感知处理芯片出现故障，更新并存储视觉感知处理芯片故障信息；电源故障寄存器，设置为监控所述电源的工作状态，若所述电源出现故障，更新并存储电源故障信息。
一些示例性实施例中，所述车载处理器根据故障诊断协议将所述视觉感知处理芯片故障信息和所述电源故障信息发送至所述一个或多个车载模块。
根据本公开实施例的另一方面，还提供了一种车载座舱感知设备的控制方法，包括：采集待处理图像；将所述待处理图像集成为待处理图像信号；对所述待处理图像信号进行视觉感知处理，得到感知结果；将所述感知结果传输至车载处理器。
一些示例性实施例中，车载座舱感知设备的控制方法还包括：接收一个或多个车载模块发起的服务升级请求；响应所述服务升级请求，将服务升级包发送至所述车载处理器；所述车载处理器对所述服务升级包进行解析，并存储部分服务升级固件；所述车载处理器将视觉感知处理芯片的服务升级固件发送至所述视觉感知处理芯片；所述视觉感知处理芯片将所述服务升级固件保存至存储器中；所述视觉感知处理芯片进行服务升级，并在升级完毕后，将第一升级结果发送至所述车载处理器；所述车载处理器进行服务升级，并在升级完毕后，将第二升级结果发送至所述一个或多个车载模块。
根据本公开实施例的另一方面，还提供一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令用于执行本公开任一实施例提供的车载座舱感知设备的控制方法。
在本公开实施例中，通过图像传感器将图像采集装置采集的待处理图像转换为待处理图像信号，传输至视觉感知处理芯片，然后视觉感知处理芯片对待处理图像信号进行视觉感知处理，得到感知结果，并将感知结果传输至车载处理器，通过车载处理器将感知结果传输至其它车载模块。本公开实施例在车载座舱感知设备中集成图像采集功能与图像处理功能，在实现座舱感知与监控的同时，支持故障诊断和远程系统升级，同时车型适配性强，多平台的兼容性强，降低整车硬件成本，从而解决相关技术中将座舱视觉感知系统集成在 车机系统或者ECU中，增加整车硬件成本，整车安装复杂度高的技术问题。
在阅读并理解了附图和详细描述后，可以明白其他方面。
附图概述
此处所说明的附图用来提供对本公开实施例的进一步理解，构成本申请的一部分，本公开的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是根据本公开实施例的一种车载座舱感知设备的硬件框图；
图2是根据本公开实施例的另一种车载座舱感知设备的硬件框图；
图3是根据本公开实施例的一种车载座舱感知设备的控制方法的流程图；
图4是根据本公开实施例的一种实现DMS系统功能的车载座舱感知设备的硬件框图；
图5是根据本公开实施例的一种实现OMS系统功能的车载座舱感知设备的硬件框图。
详述
为了使本技术领域的人员更好地理解本申请方案，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分的实施例，而不是全部的实施例。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
为便于本领域技术人员理解本公开实施例，下面对各实施例中涉及的部分术语或者名词进行解释：
座舱监控主要包括DMS和OMS系统。
驾驶员监控系统，Driver Monitor System，简称DMS，是指驾驶员行驶过程中，全天候监测驾驶员的疲劳状态、危险驾驶行为的系统。DMS功能包括疲劳检测(例如，打哈欠、闭眼)，危险动作检测(抽烟、打电话)，分心检测，手势识别。
乘客监控系统，Occupancy Monitoring System，简称OMS，是指检测座舱内其他人的感知数据来进一步提高安全性的系统。基于计算机视觉、深度学习和人工智能技术的不断融合，座舱监控也是完整自动驾驶能力的必要配置之一。OMS功能包括后排遗留物检测、遗留宠物检测和遗留儿童检测。
低电压差分信号，Low-Voltage Differential Signaling，简称LVDS，是一种低功耗、低误码率、低串扰和低辐射的差分信号技术，核心是采用极低的电压摆幅高速差动传输数据，可以实现点对点或一点对多点的连接，其传输介质可以是铜质的PCB连线，或者是平衡电缆。
自适应巡航控制，Adaptive Cruise Control，简称ACC。
通用异步收发传输器，Universal Asynchronous Receiver/Transmitter，简称UART，将要传输的资料在串行通信与并行通信之间加以转换。
I2C总线，一种简单、双向二线制同步串行总线，只需要两根线即可在连接于总线上的器件之间传送信息。
移动产业处理器接口，Mobile Industry Processor Interface，简称MIPI，是MIPI联盟发起的为移动应用处理器制定的开放标准和一个规范。
串行外设接口，Serial Peripheral Interface，简称SPI，是一种高速的、全双工、同步的通信总线。
控制器局域网络，Controller Area Network，简称CAN。
空中下载技术，Over The Air，简称OTA，通过移动通信的接口实现对软件进行远程管理。
微控制单元，Microcontroller Unit，简称MCU。
本申请提出一种车载智能座舱视觉感知设备，通过该设备可以实现驾驶员人脸识别功能、DMS功能和OMS功能中的一项或多项。同时，该设备满足实时故障诊断等功能安全要求，并且可通过CAN总线对系统做OTA升级。下面结合各个实施例对本申请进行详细说明。
图1是根据本公开实施例的一种车载座舱感知设备的硬件框图，该车载座舱感知设备100中集成有图像采集装置101，图像传感器102，视觉感知处理芯片103，车载处理器104。
一些示例性实施例中，车载座舱感知设备是摄像头，图像采集装置101为该摄像头的镜头。
图像采集装置101设置为采集待处理图像。
一些示例性实施例中的车载座舱感知设备100可以在上电后先完成自检测，并对图像传感器102进行初始化。
图像传感器102的输入端与图像采集装置101的输出端连接，图像采集装置101将待处理图像传输至图像传感器102，图像传感器102设置为将待处理图像转换为待处理图像信号。
一些示例性实施例中，图像传感器102支持包括但不限于OV、ON等厂家100W和200W像素的多款车规传感器。
一些示例性实施例中，图像采集装置101和图像传感器102构成摄像头模组或传感模组，摄像头模组采集得到待处理的图像信号。
一些示例性实施例通过摄像头模组采集的图像信号包括但不限于：RGB图像信号、灰白图像和深度图像信号，以每一种类型车辆(本实施例的车辆类型包括但不限于：小轿车(如纯电动车、混合动力车、燃油车)、卡车、商务车、公交车等)的采集需求为准。
视觉感知处理芯片103的输入端与图像传感器102的输出端连接，图像传感器102将待处理图像信号传输至视觉感知处理芯片103，视觉感知处理芯片103对待处理图像信号进行视觉感知处理，得到感知结果。
一些示例性实施例中，视觉感知处理芯片103的输入端与图像传感器102的输出端通 过第一类型排线连接，第一类型排线包括但不限于：FPC(Flexible printed board，柔性线路板)排线，该FPC排线包括但不限于MIPI传输线、时钟信号线、控制信号线、I2C线。
一些示例性实施例中，图像传感器102还可通过第一类型排线向视觉感知处理芯片103输出时钟信号和控制信号。
车载处理器104的输入端与视觉感知处理芯片103的输出端连接，车载处理器104接收视觉感知处理芯片103传输的感知结果。
一些示例性实施例中，车载处理器104的输入端与视觉感知处理芯片103的输出端通过第一类型总线连接，第一类型总线包括但不限于：UART总线。
图2是根据本公开实施例的另一种车载座舱感知设备的硬件框图，该车载座舱感知设备200中集成有图像采集装置201，图像传感器202，视觉感知处理芯片203和车载处理器204，图像采集装置201设置为采集待处理图像。
图像传感器202的输入端与图像采集装置201的输出端连接，图像采集装置201将待处理图像传输至图像传感器202，图像传感器202将待处理图像转换为待处理图像信号。
视觉感知处理芯片203的输入端与图像传感器202的输出端连接，图像传感器202将待处理图像信号传输至视觉感知处理芯片203，视觉感知处理芯片203对待处理图像信号进行视觉感知处理，得到感知结果。
车载处理器204的输入端与视觉感知处理芯片203的输出端连接，车载处理器204接收视觉感知处理芯片203传输的感知结果。
一些示例性实施例中，视觉感知处理芯片203对待处理图像信号进行视觉感知处理，得到感知结果，包括：读取预先存储的对象特征点；提取待处理图像信号的图像特征点；对比对象特征点与图像特征点，得到感知结果。
一些示例性实施例中，在提取待处理图像信号的图像特征点之前，还包括：对待处理图像信号进行图像信号处理调试。一些示例性实施例中，对待处理图像信号进行图像信号处理调试，包括：视觉感知处理芯片203对待处理图像信号进行图像感知处理(Image Signal Process，ISP)调试，获得调试后的图像信号，然后对调试后的图像信号进行视觉感知算法处理，获取感知结果，其中，视觉感知算法处理在视觉感知处理芯片203的神经网络处理单元(Neural Networks Process Units，NPU)中进行。
一些示例性实施例中，车载座舱感知设备200中还集成有：一个或多个车载模块，其中，一个或多个车载模块的输入端与车载处理器204的输出端连接，设置为接收车载处理器传输的感知结果。
需要说明的是，本实施例中的车载模块可以为但不限于：车机或ECU(Electronic Control Unit，电子控制单元)。
一些示例性实施例中，所述车载模块设置为，根据接收到的感知结果进行驾乘人员提醒，或触发其他驾驶/乘坐辅助功能。例如，触发声音提醒等功能。
一些示例性实施例中，所述车载处理器204还设置为，接收CAN总线消息；根据来自视觉感知处理芯片203传输的感知结果和所述CAN总线消息，得到新的提醒消息，并通过CAN总线传输至所述车载模块；
其中，所述CAN总线消息包括以下一项或多项内容：车速、转向信息；
所述新的提醒消息包括：感知结果和提示信息。
一些示例性实施例中，所述车载处理器204还设置为，获取服务升级包，解析得到视觉感知芯片的服务固件和本地服务固件；将所述视觉感知芯片的服务固件发送至所述视觉感知处理芯片203，获得所述视觉感知处理芯片203返回的第一升级结果；根据所述本地服务固件完成本地固件升级；
其中，所述服务升级包来自所述一个或多个车载模块。
一些示例性实施例中，所述车载处理器204还设置为，根据本地固件升级结果和所述第一升级结果，确定第二升级结果并发送至所述一个或多个车载模块。
一些示例性实施例中，车载座舱感知设备200中还集成有：串化器205，其中，串化器205的输入端与视觉感知处理芯片203的输出端连接，设置为接收待处理图像信号，且串化器的输出端与一个或多个车载模块的输入端连接，设置为将待处理图像信号输出至一个或多个车载模块。
需要说明的是，一些示例性实施例中，串化器205与视觉感知处理芯片203的连接可以通过多种接口实现，例如，通过MIPI接口实现。
一些示例性实施例中，串化器205与一个或多个车载模块之间可以通过第二类型总线连接，车载处理器204与一个或多个车载模块之间通过第三类型总线连接。可选的，第二类型总线包括：LVDS总线，第三类型总线包括：CAN总线。
一些示例性实施例中，车载座舱感知设备中还集成有：存储器206，其中，存储器206与视觉感知处理芯片203连接，设置为存储对象特征点。本实施例中，示意说明的串化器205和存储器206的类型可以自行选取，例如，串化器205可用TI(德州仪器)或Maxim(美信)的多种型号的串化器、或者其他的MIPI接口的串化器。
一些示例性实施例中，车载座舱感知设备200中还集成有：电源207，其中，电源207的输出端与视觉感知处理芯片203的电源输入端连接，设置为为视觉感知处理芯片203供电。
一些示例性实施例中，车载座舱感知设备200中还集成有：电源转换器208，其中，电源转换器208的输入端与电源207的输出端连接，且电源转换器208的输出端与图像采集装置201和图像传感器202的电源输入端连接，设置为对电源207输出的电源进行转换，为图像采集装置201和图像传感器202供电。
一些示例性实施例中，车载处理器204通过第四类型总线与点火开关连接，其中，该第四类型总线包括：ACC总线，即自适应巡航控制总线。
一些示例性实施例中，车载座舱感知设备200中还集成有：系统故障寄存器210，设置为监控视觉感知处理芯片203的工作状态，若视觉感知处理芯片203出现故障，更新并存储视觉感知芯片故障信息；电源故障寄存器211，设置为监控电源207的工作状态，若电源207出现故障，更新并存储电源故障信息。
一些示例性实施例中，电源故障寄存器211在按照预定时间周期分析电源207的工作状态后，基于工作状态确定电源207是否出现故障，生成电源故障信息，并展示该电源故障信息。
一些示例性实施例中，上述的系统故障寄存器210和电源故障寄存器211设置在车载处理器204中。
一些示例性实施例中，当车载处理器204收到视觉感知处理芯片203的故障报警后，更新车载处理器204内部存储的系统故障寄存器210的信息。同时，车载处理器204定期监控电源207的工作状态，发现电源故障后，更新车载处理器204内部存储的电源故障寄 存器211的信息。
一些示例性实施例中，车载处理器204可以按照故障诊断协议将视觉感知芯片故障信息和电源故障信息通过第三类型总线发送至一个或多个车载模块。
一些示例性实施例中，通过车载座舱感知设备200中集成图像采集装置201与视觉感知处理芯片203，在实现座舱感知与监控的同时，支持故障诊断和远程系统升级，使得车型适配性强，多平台的兼容性强，降低整车硬件成本，从而解决相关技术中将座舱视觉感知系统集成在车机系统或者ECU中，增加整车硬件成本，整车安装复杂度高的技术问题。可以理解，一些示例性实施例中，车载座舱感知设备200独立于车机系统或者ECU，能够简化其安装复杂度。
本申请还提供一种车载座舱感知设备的控制方法，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
图3是本公开实施例提供一种可选的车载座舱感知设备的控制方法的流程图，该方法包括如下步骤：
步骤S302，采集待处理图像；
步骤S304，将待处理图像转换为待处理图像信号；
步骤S306，对待处理图像信号进行视觉感知处理，得到感知结果；
步骤S308，将感知结果传输至车载处理器。
通过上述步骤，可以先通过图像采集装置采集待处理图像，然后通过图像传感器将待处理图像转换为待处理图像信号，通过视觉感知处理芯片对待处理图像信号进行视觉感知处理，得到感知结果，并将感知结果传输至车载处理器。在本公开实施例中，车载座舱感知设备中集成了图像采集功能与视觉感知处理功能，在实现座舱感知与监控的同时，支持故障诊断和远程系统升级，同时车型适配性强，多平台的兼容性强，降低整车硬件成本，从而解决相关技术中将座舱视觉感知系统集成在车机系统或者ECU中，增加整车硬件成本，整车安装复杂度高的技术问题。
一些示例性实施例中，采集待处理图像的步骤，包括：车载座舱感知设备上电后，控制车载座舱感知设备的图像传感器初始化；在图像传感器完成初始化后，控制车载图像采集装置采集车辆内部预定空间区域的图像。
一些示例性实施例中，对待处理图像信号进行视觉感知处理，得到感知结果的步骤，包括：通过视觉感知处理芯片读取存储器中预先存储的对象特征点；通过视觉感知处理芯片提取待处理图像信号中的图像特征点；对比对象特征点与图像特征点，得到感知结果。
一些示例性实施例中，对象特征点可以包括用户对象的人脸特征点，如五官特征点，面部显著特征点等。
一些示例性实施例中，控制方法还包括：接收一个或多个车载模块发起的服务升级请求；响应服务升级请求，将服务升级包发送至车载处理器中；车载处理器对服务升级包进行解析，并存储部分服务固件；车载处理器将视觉感知芯片的服务固件发送至视觉感知处理芯片；视觉感知处理芯片将服务固件保存至其片内存储器中；视觉感知处理芯片进行服务升级，并在升级完毕后，将第一升级结果发送至车载处理器；车载处理器进行预设服务升级，并在升级完毕后，将第二升级结果发送至一个或多个车载模块。其中，车载处理器执行本地存储的所述部分服务固件，进行本地固件升级，即完成了本地的预设服务升级。
一些示例性实施例中，服务升级请求中包括服务升级包。
一些示例性实施例中，控制方法还包括：
所述车载处理器获取服务升级包，解析得到视觉感知芯片的服务固件和本地服务固件；将所述视觉感知芯片的服务固件发送至所述视觉感知处理芯片，获得所述视觉感知处理芯片返回的第一升级结果；根据所述本地服务固件完成本地固件升级；
其中，所述服务升级包来自所述一个或多个车载模块。视觉感知处理芯片将服务固件保存至其片内存储器中；视觉感知处理芯片进行服务升级，并在升级完毕后，将第一升级结果发送至车载处理器；
一些示例性实施例中，控制方法还包括：所述车载处理器根据本地固件升级结果和所述第一升级结果，确定第二升级结果并发送至所述一个或多个车载模块。
下面结合另一种可选的实施例来说明本申请方案。
下面分别结合车载智能座舱相关的两种系统(DMS系统和OMS系统)对本公开实施例进行详细说明。在本实施例中，将图像传感器和图像采集装置都集成在一个传感模组内。
第一种系统：DMS系统，在本实施例中，车载座舱感知设备集成在DMS系统中。DMS系统可实现驾驶员人脸识别和DMS功能，DMS功能包括但不限于：疲劳检测(打哈欠、闭眼)、危险动作检测(抽烟、打电话)、分心检测、手势识别。
图4是根据本公开实施例的一种可选的座舱感知设备的硬件框图，应用于DMS系统中，该车载座舱感知设备包括：DMS系统传感模组和DMS视觉感知处理板，其中，
DMS系统传感模组，至少包括：图像采集装置和图像传感器，通过图像采集装置采集待处理图像后，通过图像传感器将采集的待处理图像集成为待处理图像信号，传输至DMS视觉感知处理板。
一些示例性实施例中，图像采集装置可以是DMS摄像头的镜头，DMS系统传感模组与DMS视觉感知处理板集成在DMS摄像头中，此时，所述车载座舱感知设备为DMS摄像头。
DMS视觉感知处理板，包括：视觉感知处理芯片、车载处理器、串化器、存储器和电源，视觉感知处理芯片通过FPC排线与DMS系统传感模组中的图像传感器连接，并通过MIPI线与串化器连接，通过UART总线与车载处理器连接，通过串行外设接口SPI与存储器连接。
视觉感知处理芯片在接收到待处理图像信号后，对待处理图像信号进行视觉感知处理，得到感知结果，将感知结果通过UART总线传输至车载处理器，车载处理器将感知结果通过CAN总线传输至车机或者其它车载ECU。
在图4中，串化器与车机或者ECU之间通过LVDS总线连接，点火开关与车载处理器之间通过钥匙控制的ACC总线连接。
图4中，通过汽车电平提供电能给DMS视觉感知处理板中的电源，电源为视觉感知处理芯片供电，同时，电源中的电能经由电源转换器的转换，为DMS系统传感模组供电。
上述车载座舱感知设备包括DMS视觉感知处理板和DMS系统传感模组两部分。DMS视觉感知处理板包括：视觉感知处理芯片、车载处理器MCU、串化器、存储器和电源。DMS系统传感模组包括图像传感器和图像采集装置。DMS系统传感模组通过FPC排线将传输图像的MIPI信号、RST控制信号和MCLK时钟信号传输至DMS视觉感知处理板。 DMS视觉感知处理板接收到DMS系统传感模组的图像数据后，在视觉感知处理芯片上对图像进行视觉感知算法处理，得到感知结果，并将感知结果通过UART总线传输至车载处理器，车载处理器通过CAN总线将感知结果传输至车机或其他车载ECU。
本实施例中的视觉感知处理芯片将图像信号通过MIPI传输至串化器，串化器将图像信号通过LVDS传输至车机或其他车载ECU。
下面对该车载座舱感知设备的硬件系统的工作流程进行示意性说明，工作流程包括：
1.车载座舱感知设备上电，开始自检，同时，图像传感器进行初始化；
2.在图像传感器完成初始化后，控制传感模组采集车辆内部预定空间区域的图像，得到待处理的图像信号；
3.通过FPC排线将传输图像的MIPI信号、RST控制信号和MCLK时钟信号传输至DMS视觉感知处理板，视觉感知处理芯片对图像信号进行ISP(图像信号处理)调试；
4.对调试后的图像信号进行视觉感知算法处理，得到感知结果；
5.视觉感知处理芯片将感知结果通过UART总线传输至车载处理器，同时，将图像信号通过MIPI输出至串化器，而后串化器通过LVDS将图像信号输出至机或其他车载ECU；
6.车载处理器收到感知结果后，通过CAN总线将感知结果传输至车机或其他车载ECU。
下面对DMS系统的人脸注册和人脸识别流程进行示意性说明。
用户在使用车载DMS系统的人脸识别功能之前，需要进行人脸注册操作，用户可以按照系统提示或者主动上传、主动输入的方式，将人脸图像上传至系统中，系统对人脸图像进行识别，获取人脸特征(包括：五官特征、脸部多个点位特征等)，并将人脸特征保存至存储器中。
在完成人脸注册后，即可使用系统的人脸识别功能，对于需要进行人脸识别的用户，可以提供DMS系统传感模组采集的人脸图像信号传输至DMS视觉感知处理板中，通过视觉感知处理芯片提取人脸特征，将所提取的特征与保存在存储器中的人脸特征进行比对，将比对结果，即，感知结果发送至车载处理器，然后车载处理器可以将感知结果通过CAN协议发给车机或者其它车载ECU等。
一些示例性实施例中，实现疲劳检测(打哈欠、闭眼)、危险动作检测(抽烟、打电话)、分心检测、手势识别等功能所需的检测特征数据，也预先存储在存储器中。
下面对车载座舱感知设备的故障诊断操作进行示意性说明。
1.视觉感知处理芯片定期监控DMS系统中每一个硬件的工作状态，若有硬件的工作状态指示其出现故障，则通过UART发送故障报警至车载处理器；
2.车载处理器在收到故障报警后，车载处理器中的系统故障寄存器更新并存储视觉感知芯片故障信息。
同时，车载处理器定期监控电源的工作状态，若电源的工作状态指示电源出现故障，车载处理器中的电源故障寄存器更新并存储电源故障信息。
3.车载处理器根据故障诊断协议将视觉感知芯片故障信息和电源故障信息通过CAN总线发至车机或者其它车载ECU。
下面对车载座舱感知设备的OTA升级进行示意性说明。
1.车机或其他车载ECU发起OTA升级请求，并通过CAN总线将OTA升级包发到车 载处理器中；
2.车载处理器对OTA升级包进行解析，并将部分OTA固件存储至车载处理器的内部存储中；
3.车载处理器将视觉感知处理芯片的OTA固件发给至视觉感知处理芯片；
4.视觉感知处理芯片将其OTA固件保存至视觉感知处理芯片的内部存储器中；
5.视觉感知处理芯片进行OTA升级，升级完毕后，将其升级结果(即，第一升级结果)通过UART总线发送至车载处理器；
6.车载处理器进行OTA升级，升级完毕后，将其升级结果(即，第二升级结果)发送至车机或其它车载ECU。
第二种系统：OMS系统，在本实施例中，车载座舱感知设备集成在OMS系统中。OMS系统可实现OMS功能，OMS功能包括但不限于：后排遗留物检测、遗留宠物检测和遗留儿童检测。
图5是根据本公开实施例的一种可选的座舱感知设备的硬件框图，该座舱感知设备包括：OMS系统传感模组和OMS视觉感知处理板，其中，
OMS系统传感模组，至少包括：图像采集装置和图像传感器，通过图像采集装置采集待处理图像后，通过图像传感器将采集的待处理图像集成为待处理图像信号，传输至OMS视觉感知处理板。
一些示例性实施例中，图像采集装置可以是OMS摄像头的镜头，OMS系统传感模组与OMS视觉感知处理板集成在OMS摄像头中，此时，所述车载座舱感知设备为OMS摄像头。
OMS视觉感知处理板，包括：视觉感知处理芯片、车载处理器、串化器、存储器和电源，视觉感知处理芯片通过FPC排线与OMS系统传感模组中的图像传感器连接，并通过MIPI线与串化器连接，通过UART总线与车载处理器连接，通过串行外设接口SPI与存储器连接。
视觉感知处理芯片在接收到待处理图像信号后，对待处理图像信号进行视觉感知处理，得到感知结果，将感知结果通过UART总线传输至车载处理器，车载处理器将感知结果通过CAN总线传输至车机或者其它车载ECU。
在图5中，串化器与车机或者ECU之间通过LVDS总线连接，点火开关与车载处理器之间通过钥匙控制的ACC总线连接。
图5中，通过汽车电平提供电能给OMS视觉感知处理板中的电源，电源为视觉感知处理芯片供电，同时，电源中的电能经由电源转换器的转换，为OMS系统传感模组供电。
上述座舱感知设备包括OMS视觉感知处理板和OMS系统传感模组两部分。OMS视觉感知处理板包括：视觉感知处理芯片、车载处理器MCU、串化器、存储器和电源。OMS系统传感模组包括图像传感器和图像采集装置。OMS系统传感模组通过FPC排线将传输图像的MIPI信号、RST控制信号和MCLK时钟信号传输至OMS视觉感知处理板。OMS视觉感知处理板接收到OMS系统传感模组的图像数据后，在视觉感知处理芯片上对图像进行视觉感知算法处理，得到感知结果，并将感知结果通过UART总线传输至车载处理器，车载处理器通过CAN总线将感知结果传输至车机或其他车载ECU。
本实施例中的视觉感知处理芯片将图像通过MIPI传输至串化器，串化器将图像通过LVDS传输至车机或其他车载ECU。OMS视觉感知处理板直接由汽车电平供电，电源转 换器对汽车电平提供的电能进行转换，为OMS系统传感模组供电。
下面对该车载OMS系统的硬件系统的工作流程进行示意性说明，工作流程包括：
1.车载座舱感知设备上电，开始自检，同时，图像传感器进行初始化；
2.在图像传感器完成初始化后，控制传感模组采集车辆内部预定空间区域的图像，得到待处理的图像信号；
3.通过FPC排线将传输图像的MIPI信号、RST控制信号和MCLK时钟信号传输至OMS视觉感知处理板，视觉感知处理芯片对图像信号进行ISP(图像信号处理)调试；
4.对调试后的图像信息进行视觉感知算法处理，得到感知结果；
5.视觉感知处理芯片将感知结果通过UART总线传输至车载处理器，同时，将图像信号通过MIPI输出至串化器，而后串化器通过LVDS将图像信号输出至机或其他车载ECU；
6.车载处理器收到感知结果后，通过CAN总线将感知结果传输至车机或其他车载ECU。
下面对OMS系统的人脸注册和人脸识别流程进行示意性说明。
用户在使用车载OMS系统的人脸识别功能之前，需要进行人脸注册操作，用户可以按照系统提示或者主动上传、主动输入的方式，将人脸图像上传至系统中，系统对人脸图像进行识别，获取人脸特征(包括：五官特征、脸部多个点位特征等)，并将人脸特征保存至存储器中。
在完成人脸注册后，即可使用系统的人脸识别功能，对于需要进行人脸识别的用户，可以提供OMS系统传感模组采集的人脸图像传输至OMS视觉感知处理板中，通过视觉感知处理芯片提取人脸特征，将所提取的特征与保存在存储器中的人脸特征进行比对，将比对结果，即，感知结果发送至车载处理器，然后车载处理器可以将感知结果通过CAN协议发给车机或者其它车载ECU等。
一些示例性实施例中，实现后排遗留物检测、遗留宠物检测、遗留儿童检测和乘客动作检测等功能所需的检测特征数据，也预先存储在存储器中。
下面对OMS系统的故障诊断操作进行示意性说明。
1.视觉感知处理芯片定期监控OMS系统中每一个硬件的工作状态，若有硬件的工作状态指示其出现故障，则通过UART发送故障报警至车载处理器；
2.车载处理器在收到故障报警后，车载处理器中的系统故障寄存器更新并存储视觉感知芯片故障信息。
同时，车载处理器定期监控电源的工作状态，若电源的工作状态指示电源出现故障，车载处理器中的电源故障寄存器更新并存储电源故障信息。
3.车载处理器根据故障诊断协议将视觉感知芯片故障信息和电源故障信息通过CAN总线发至车机或者其它车载ECU。
下面对OMS系统的OTA升级进行示意性说明。
1.车机或其他车载ECU发起OTA升级请求，并通过CAN总线将OTA升级包发到车载处理器中；
2.车载处理器对OTA升级包进行解析，并将部分OTA固件存储至车载处理器的内部存储中；
3.车载处理器将视觉感知处理芯片的OTA固件发给至视觉感知处理芯片；
4.视觉感知处理芯片将其OTA固件保存至视觉感知处理芯片的内部存储器中；
5.视觉感知处理芯片进行OTA升级，升级完毕后，将其升级结果(即，第一升级结果)通过UART总线发送至车载处理器；
6.车载处理器进行OTA升级，升级完毕后，将其升级结果(即，第二升级结果)发送至车机或其它车载ECU。
一些示例性实施中，所述DMS系统和OMS系统集成一体，共用同一传感模组(包括图像传感器和图像采集装置)，即同一传感模组作为所述DMS系统传感模组，也作为所述OMS系统传感模组；共用同一视觉感知处理板，即同一视觉感知处理板作为所述DMS视觉感知处理板，也作为所述OMS视觉感知处理板，预先存储了需要进行特征比对的全部所需特征。
此时，车载座舱感知设备包括：传感模组和视觉感知处理板，以实现DMS系统功能和OMS系统功能。一些示例性实施例中，所述车载座舱感知设备为布设于车内的摄像头。即在摄像头包括的传感模组(图像传感器和图像采集装置)的基础上，集成了视觉感知处理板。
上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。
在本公开的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，可以为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (14)

  1. 一种车载座舱感知设备，，所述车载座舱感知设备中集成有图像采集装置，图像传感器，视觉感知处理芯片，车载处理器，其中，
    所述图像采集装置设置为采集待处理图像；
    所述图像传感器的输入端与所述图像采集装置的输出端连接，设置为接收所述待处理图像并将所述待处理图像转换为待处理图像信号；
    所述视觉感知处理芯片的输入端与所述图像传感器的输出端连接，设置为接收所述待处理图像信号并对所述待处理图像信号进行视觉感知处理，得到感知结果；
    所述车载处理器的输入端与所述视觉感知处理芯片的输出端连接，设置为接收所述感知结果。
  2. 如权利要求1所述的车载座舱感知设备，其中，对所述待处理图像信号进行视觉感知处理，得到感知结果，包括：
    读取预先存储的对象特征点；
    提取所述待处理图像信号的图像特征点；
    对比所述对象特征点与所述图像特征点，得到所述感知结果。
  3. 如权利要求2所述的车载座舱感知设备，其中，在提取所述待处理图像信号的图像特征点之前，还包括：
    对所述待处理图像信号进行图像信号处理调试。
  4. 如权利要求2所述的车载座舱感知设备，其中，所述车载座舱感知设备中还集成有：存储器，
    其中，所述存储器与所述视觉感知处理芯片连接，设置为存储所述对象特征点。
  5. 如权利要求1至4中任一项所述的车载座舱感知设备，其中，所述车载座舱感知设备中还集成有：一个或多个车载模块，
    其中，所述一个或多个车载模块的输入端与所述车载处理器的输出端连接，设置为接收所述感知结果。
  6. 如权利要求5所述的车载座舱感知设备，其中，所述车载座舱感知设备中还集成有：串化器，
    其中，所述串化器的输入端与所述视觉感知处理芯片的输出端连接，且所述串化器的输出端与所述一个或多个车载模块的输入端连接，设置为接收所述待处理图像信号，并将所述待处理图像信号输出至所述一个或多个车载模块。
  7. 如权利要求5所述的车载座舱感知设备，其中，所述车载座舱感知设备中还集成有：电源，
    其中，所述电源的输出端与所述视觉感知处理芯片的输入端连接，设置为为所述视觉感知处理芯片供电。
  8. 如权利要求7所述的车载座舱感知设备，其中，所述车载座舱感知设备中还集成有：电源转换器，
    其中，所述电源转换器的输入端与所述电源的输出端连接，且所述电源转换器的输出端与所述图像采集装置和所述图像传感器的输入端连接，设置为对所述电源进行转换，为 所述图像采集装置和所述图像传感器供电。
  9. 如权利要求7所述的车载座舱感知设备，其中，所述车载处理器包括：
    系统故障寄存器，设置为监控所述视觉感知处理芯片的工作状态，在所述视觉感知处理芯片出现故障的情况下，更新并存储视觉感知处理芯片故障信息；
    电源故障寄存器，设置为监控所述电源的工作状态，在所述电源出现故障的情况下，更新并存储电源故障信息。
  10. 如权利要求9所述的车载座舱感知设备，其中，所述车载处理器还设置为，根据故障诊断协议将所述视觉感知处理芯片故障信息和所述电源故障信息发送至所述一个或多个车载模块。
  11. 如权利要求5所述的车载座舱感知设备，其中，
    所述车载处理器还设置为，获取服务升级包，解析得到视觉感知芯片的服务固件和本地服务固件；将所述视觉感知芯片的服务固件发送至所述视觉感知处理芯片，获得所述视觉感知处理芯片返回的第一升级结果；根据所述本地服务固件完成本地固件升级；
    其中，所述服务升级包来自所述一个或多个车载模块。
  12. 一种车载座舱感知设备的控制方法，包括：
    采集待处理图像；
    将所述待处理图像转换为待处理图像信号；
    对所述待处理图像信号进行视觉感知处理，得到感知结果；
    将所述感知结果传输至车载处理器。
  13. 如权利要求12所述的车载座舱感知设备的控制方法，还包括：
    接收一个或多个车载模块发起的服务升级请求；
    响应所述服务升级请求，将服务升级包发送至所述车载处理器；
    所述车载处理器对所述服务升级包进行解析，并存储部分服务升级固件；
    所述车载处理器将视觉感知处理芯片的服务升级固件发送至所述视觉感知处理芯片；
    所述视觉感知处理芯片将所述服务升级固件保存至存储器中；
    所述视觉感知处理芯片进行服务升级，并在升级完毕后，将第一升级结果发送至所述车载处理器；
    所述车载处理器进行服务升级，并在升级完毕后，将第二升级结果发送至所述一个或多个车载模块。
  14. 一种计算机可读存储介质，存储有计算机可执行指令，所述计算机可执行指令用于执行权利要求12至13任一项的方法。
PCT/CN2023/119234 2022-09-19 2023-09-15 车载座舱感知设备及其控制方法 WO2024061137A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211137970.7 2022-09-19
CN202211137970.7A CN115447508A (zh) 2022-09-19 2022-09-19 车载座舱感知设备及其控制方法

Publications (1)

Publication Number Publication Date
WO2024061137A1 true WO2024061137A1 (zh) 2024-03-28

Family

ID=84304956

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/119234 WO2024061137A1 (zh) 2022-09-19 2023-09-15 车载座舱感知设备及其控制方法

Country Status (2)

Country Link
CN (1) CN115447508A (zh)
WO (1) WO2024061137A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115447508A (zh) * 2022-09-19 2022-12-09 虹软科技股份有限公司 车载座舱感知设备及其控制方法

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105966405A (zh) * 2015-03-13 2016-09-28 哈曼国际工业有限公司 驾驶员分心检测系统
US20190283672A1 (en) * 2018-03-19 2019-09-19 Honda Motor Co., Ltd. System and method to control a vehicle interface for human perception optimization
CN209640714U (zh) * 2018-12-13 2019-11-15 汉能移动能源控股集团有限公司 智能集成控制装置、车辆控制/道路交通监测系统和飞行器
CN110562271A (zh) * 2018-06-05 2019-12-13 上海博泰悦臻网络技术服务有限公司 基于车辆智能座舱的人脸识别装置、车辆智能座舱及车辆
CN112351915A (zh) * 2020-04-24 2021-02-09 上海商汤临港智能科技有限公司 车辆和车舱域控制器
US20210211614A1 (en) * 2019-07-11 2021-07-08 Chris Pritchard Systems and methods for providing real-time surveillance in automobiles
CN215944529U (zh) * 2021-09-01 2022-03-04 诺博汽车科技有限公司 座舱系统和车辆
CN114332831A (zh) * 2021-12-29 2022-04-12 天瀚科技(吴江)有限公司 一种汽车座舱的智能视觉监控系统
CN115447508A (zh) * 2022-09-19 2022-12-09 虹软科技股份有限公司 车载座舱感知设备及其控制方法

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105966405A (zh) * 2015-03-13 2016-09-28 哈曼国际工业有限公司 驾驶员分心检测系统
US20190283672A1 (en) * 2018-03-19 2019-09-19 Honda Motor Co., Ltd. System and method to control a vehicle interface for human perception optimization
CN110562271A (zh) * 2018-06-05 2019-12-13 上海博泰悦臻网络技术服务有限公司 基于车辆智能座舱的人脸识别装置、车辆智能座舱及车辆
CN209640714U (zh) * 2018-12-13 2019-11-15 汉能移动能源控股集团有限公司 智能集成控制装置、车辆控制/道路交通监测系统和飞行器
US20210211614A1 (en) * 2019-07-11 2021-07-08 Chris Pritchard Systems and methods for providing real-time surveillance in automobiles
CN112351915A (zh) * 2020-04-24 2021-02-09 上海商汤临港智能科技有限公司 车辆和车舱域控制器
CN215944529U (zh) * 2021-09-01 2022-03-04 诺博汽车科技有限公司 座舱系统和车辆
CN114332831A (zh) * 2021-12-29 2022-04-12 天瀚科技(吴江)有限公司 一种汽车座舱的智能视觉监控系统
CN115447508A (zh) * 2022-09-19 2022-12-09 虹软科技股份有限公司 车载座舱感知设备及其控制方法

Also Published As

Publication number Publication date
CN115447508A (zh) 2022-12-09

Similar Documents

Publication Publication Date Title
CN109995631A (zh) 具有域控制功能的车载控制系统及方法
US9836904B2 (en) Key fob dongle
CN101895394B (zh) 与obd诊断座进行数据交换的实现方法及其装置
WO2024061137A1 (zh) 车载座舱感知设备及其控制方法
CN106230678A (zh) 基于车载网关控制器的信息处理方法及网关控制器
CN105320520A (zh) 用于将数据功能集成在用于车辆的运动控制系统中的方法
EP4310668A1 (en) Method and apparatus for upgrading terminal
CN110562271A (zh) 基于车辆智能座舱的人脸识别装置、车辆智能座舱及车辆
CN114461070A (zh) 用于实现车载虚拟现实的系统、方法和车辆
CN114332831A (zh) 一种汽车座舱的智能视觉监控系统
CN213399299U (zh) 一种仪表域控制器系统
CN110493449A (zh) 车辆及其基于乘车人数的隐私策略实时设置方法
CN106347242B (zh) 一种汽车分离娱乐系统的唤醒方法及其装置
KR101866725B1 (ko) 차량 전자 모듈의 통신방법 및 이를 포함하는 차량
CN208360161U (zh) 基于车辆智能座舱的人脸识别装置、车辆智能座舱及车辆
CN111347976B (zh) 车载显示系统及车辆
CN208360117U (zh) 基于车辆智能座舱的指纹识别装置、车辆智能座舱及车辆
CN113206861A (zh) 信息处理装置、信息处理方法和记录介质
CN209821620U (zh) 一种智能座舱域控制器
CN210310244U (zh) 车辆间的信息交互系统和汽车
CN219268923U (zh) 车载视觉控制器
CN208360056U (zh) 基于车辆智能座舱的360环视装置、车辆智能座舱及车辆
CN112153522A (zh) 车载音箱控制系统及车载音箱
CN108016381B (zh) 一种汽车电子系统及其程序更新方法
JP2022080346A (ja) 車両用制御システム、車両用制御システムの異常検知方法及び異常検知プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23867426

Country of ref document: EP

Kind code of ref document: A1