WO2022199330A1 - 视频采集设备、车辆、车舱检测及同步曝光方法 - Google Patents

视频采集设备、车辆、车舱检测及同步曝光方法 Download PDF

Info

Publication number
WO2022199330A1
WO2022199330A1 PCT/CN2022/078297 CN2022078297W WO2022199330A1 WO 2022199330 A1 WO2022199330 A1 WO 2022199330A1 CN 2022078297 W CN2022078297 W CN 2022078297W WO 2022199330 A1 WO2022199330 A1 WO 2022199330A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
instruction
synchronous exposure
chip
chips
Prior art date
Application number
PCT/CN2022/078297
Other languages
English (en)
French (fr)
Inventor
孙牵宇
许亮
张宣彪
王勇
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2022199330A1 publication Critical patent/WO2022199330A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present disclosure relates to the field of data processing, and in particular, to a video acquisition device, a vehicle, and a vehicle cabin detection and synchronous exposure method.
  • the application of intelligent vehicles is more and more extensive.
  • the functions it can perform include but are not limited to detecting fatigue, distracted driving, and dangerous actions.
  • Multiple cameras can be deployed at different positions on the vehicle to collect video streams and analyze the DMS function based on the collected video streams.
  • the present disclosure provides a video acquisition device, a vehicle, and a vehicle cabin detection and synchronous exposure method.
  • a video capture device comprising: at least two cameras for capturing video streams; at least two chips connected to the at least two cameras, wherein at least two of the cameras are The chips are respectively connected to different cameras, and the chips are connected by hard wires and transmit synchronous exposure instructions. Exposure command for synchronized exposure.
  • At least two of the chips include a first chip and at least one second chip; the first chip is configured to receive the synchronous exposure instruction, and send the synchronous exposure instruction to each the second chip, and send the synchronous exposure command to the camera connected to the first chip; each of the second chips is used for receiving the synchronous exposure command sent by the first chip through a hard wire, and Send the synchronous exposure instruction to the cameras connected to each of the second chips.
  • the chip is provided with a deserializer
  • the camera is provided with a serializer matching the deserializer
  • the serializer is used for sending the video stream captured by the camera to the deserializer through the coaxial cable; and/or the deserializer is used for transmitting the synchronous exposure command through the coaxial cable to the serializer.
  • it further includes: a controller, configured to generate a synchronous exposure instruction; a switch connected with the controller and each of the chips, configured to send the said controller generated by the controller to the first chip Synchronized exposure command.
  • the switch and each of the chips are connected through a network cable, and the switch and the controller are connected through a network cable.
  • the controller is further configured to generate an associated control instruction, wherein the associated control instruction is used to instruct the camera to perform a predetermined control operation other than the synchronous exposure operation; the switch is also configured to use is used to broadcast the associated control instruction to each of the chips through a network cable; each of the chips is also used to receive the associated control instruction broadcast by the switch through a network cable, and control the camera connected to each of the chips to execute the The operation corresponding to the associated control instruction.
  • the associated control instruction includes at least one of the following: a video data synchronous upload instruction, a fill light switch control instruction, and a working mode control instruction of the combined camera, wherein the working mode control instruction is used for Indicates that each camera in the combination camera is on or off.
  • the device is an in-vehicle video capture device.
  • a vehicle including the video capture device according to any one of the first aspect.
  • the at least two cameras are installed in at least one of the following positions: interior rearview mirror, exterior rearview mirror, central control screen, steering column, A-pillar, steering wheel, the surrounding area of the gear handle, Car air conditioner, car audio, car door; and/or the at least two cameras include at least two of a normal color mode RGB camera, an infrared IR camera, a time-of-flight TOF camera, and a combination camera, consisting of an RGB camera, an IR camera, and a TOF camera At least two of them make up the combination camera.
  • the camera includes a combined camera composed of at least two of an RGB camera, an IR camera and a TOF camera, and the camera is configured to receive a working mode control instruction sent by a connected chip, and to The own working mode is switched to the working mode indicated by the working mode control instruction, wherein the working mode control instruction is used to instruct each camera in the combination camera to be turned on or off.
  • a vehicle cabin detection method wherein the vehicle cabin includes the video capture device according to any one of the first aspect; the method includes: acquiring data captured by at least two cameras at least two channels of video data; according to the at least two channels of video data, state detection is performed on the state of the people in the vehicle cabin.
  • the performing state detection on the state of the persons in the vehicle cabin according to the at least two channels of video data includes: estimating the head of the driver based on the at least two channels of video data Posture and/or gaze area.
  • the estimating the driver's head posture and/or the gaze area based on the at least two channels of video data includes: determining the vehicle cabin based on the at least two channels of the video data Correspondence between at least two of the cameras in the cabin and the deflection angles of the driver's head and/or eyes; according to the at least two of the cameras in the cabin and the driver's head and/or Or the correspondence between the deflection angles of the eyes, to determine the driver's head posture and/or the gaze area.
  • the correspondence between the at least two cameras and the deflection angles of the driver's head and eyes, respectively, is determined through a pre-trained neural network; the method further includes: acquiring the In the state where the driver deflects at least one of the head and the eyes, the image captured by any camera in the at least two cameras in the cabin is used as a sample image; the driver marked on the sample image is used as the sample image. At least one of the head deflection angle and the eye deflection angle is supervised, and the neural network is trained.
  • a synchronous exposure method comprising: receiving a synchronous exposure instruction generated by a controller and sent through a switch; sending the synchronous exposure instruction to at least one second chip by hard wire, and sent to the camera connected to itself through a coaxial line, so that the camera connected to each of the second chips and the camera connected to itself perform synchronous exposure according to the synchronous exposure instruction.
  • the method further includes: receiving an associated control instruction generated by the controller and broadcast through the switch, wherein the associated control instruction is used to instruct the camera to perform operations other than synchronous exposure other predetermined control operations; control the camera connected to itself to perform corresponding operations according to the associated control instruction.
  • the associated control instruction includes at least one of the following: a video data synchronous upload instruction, a fill light switch control instruction, and a working mode control instruction of the combined camera, wherein the working mode control instruction is used for Indicates that each camera in the combination camera is on or off.
  • a synchronous exposure method comprising: receiving a synchronous exposure command sent by a first chip through a hard wire; sending the synchronous exposure command to a target connected to itself through a coaxial wire a camera, so as to control the target camera and the camera connected with the first chip through a coaxial line to expose synchronously.
  • a computer-readable storage medium where the storage medium stores a computer program, and the computer program is used to execute any of the methods described in the third aspect above.
  • a camera module where the camera module includes the camera in the video capture device according to any one of the above-mentioned first aspects, the camera module includes: a camera, a camera with The camera sensor connected to the camera is used for receiving the synchronous exposure instruction sent by the chip through the coaxial cable, and performing exposure according to the synchronous exposure instruction.
  • the method further includes: a serializer connected to the camera and the camera sensor, configured to send video data included in the captured video stream to the chip through the coaxial cable.
  • the camera sensor is further configured to receive an associated control instruction sent by the chip through the coaxial cable, and control execution of a predetermined control operation corresponding to the associated control instruction based on the associated control instruction .
  • a controller of a vehicle including: a human-computer interaction module for acquiring a trigger instruction for triggering a function of an on-board monitoring system of the vehicle; an instruction generation module for triggering an instruction to generate a synchronous exposure instruction; an instruction sending module, configured to send the synchronous exposure instruction to a switch, so that the switch sends the synchronous exposure instruction to a first chip among the at least two chips, the first The chip sends the simultaneous exposure command to at least one second chip of at least two of the chips by hard wire.
  • each chip can send a synchronous exposure instruction to the cameras connected to it, so that each camera performs synchronous exposure according to the synchronous exposure instruction, which reduces the error of synchronous exposure of multiple cameras, and has high usability.
  • FIG. 1 is a schematic structural diagram of a video capture device according to an exemplary embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of another video capture device according to an exemplary embodiment of the present disclosure.
  • FIG. 3 is a schematic structural diagram of another video capture device according to an exemplary embodiment of the present disclosure.
  • FIG. 4 is a schematic diagram of a scene where a camera is installed in a vehicle according to an exemplary embodiment of the present disclosure
  • FIG. 5 is a flowchart of a vehicle cabin detection method according to an exemplary embodiment of the present disclosure
  • FIG. 6 is a flowchart of another vehicle cabin detection method according to an exemplary embodiment of the present disclosure.
  • FIG. 7 is a flowchart of a synchronous exposure method according to an exemplary embodiment of the present disclosure.
  • FIG. 8 is a flowchart of another synchronous exposure method according to an exemplary embodiment of the present disclosure.
  • FIG. 9 is a flowchart of another synchronous exposure method according to an exemplary embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of a camera module according to an exemplary embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of another camera module according to an exemplary embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of a controller of a vehicle according to an exemplary embodiment of the present disclosure.
  • first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure.
  • word “if” as used herein may be interpreted as "at the time of” or "when” or "in response to determining.”
  • the present disclosure provides a video capture device, such as shown in FIG. 1 , including:
  • At least two chips 102 connected to the at least two cameras wherein the at least two chips 102 are respectively connected to different cameras 101, and the chips 102 are connected by hard wires and transmit synchronous exposure instructions, and
  • the chip 102 is configured to send a synchronous exposure instruction to the respective connected cameras 101, so that each of the cameras 101 performs synchronous exposure according to the synchronous exposure instruction.
  • each chip 102 may adopt, but is not limited to, a System Basic Chip (SBC).
  • SBC System Basic Chip
  • Each chip 102 can be connected to different cameras 101 respectively, and the cameras 101 include but are not limited to ordinary color mode (Red Green Blue, RGB) cameras, or infrared (InfraRed, IR) cameras, or RGB-IR cameras. Or a combined camera obtained by combining a separate RGB camera and an IR camera can jointly realize the function of the RGB-IR camera.
  • the camera 101 needs to support an external trigger function, that is, the camera 101 can be triggered by a control command sent by the connected chip 102 , and the control command includes but is not limited to a synchronous exposure command.
  • each chip can send a synchronous exposure command to each connected camera, so that each camera can perform synchronous exposure synchronously, which reduces the error of synchronous exposure.
  • the error of synchronous exposure can reach the microsecond level, and the usability is high.
  • the at least two chips 102 may include a first chip 102-1 and at least one second chip 102-2, such as shown in FIG. 2 .
  • the first chip 102-1 may be a preset master chip in each chip.
  • the first chip 102-1 can be configured to receive a synchronous exposure instruction, and send the synchronous exposure instruction to each of the second chips 102-2 and the camera 101 connected to the first chip 102-1 through a hard wire.
  • Each second chip 102-2 is configured to receive the synchronous exposure instruction sent by the first chip 102-1 through a hard wire, and send the synchronous exposure instruction to the camera 101 connected to each second chip 102-2.
  • At least two chips may be connected by hard wires.
  • the first chip 102-1 sends a synchronous exposure command to each second chip 102-2 through a hard wire, which is faster than sending commands between chips through a network cable, so that different cameras connected to each chip can be connected to each other.
  • the error of synchronous exposure is reduced to microsecond level.
  • the first chip sends a synchronous exposure command to each of the second chips through a hard wire, and each chip sends the synchronous exposure command to the cameras connected to each chip, thereby reducing the synchronous exposure error of each camera, and high usability.
  • each chip 102 is provided with a deserializer
  • the camera 101 connected to each chip 102 is provided with a serializer matching the deserializer
  • the serializer and the deserializer are connected through the same
  • the axis is connected.
  • the serializer may be configured to send the video stream collected by the camera to the deserializer through a coaxial cable
  • the deserializer may be configured to transmit the synchronous exposure instruction to the serializer through the coaxial cable.
  • a first interface may be set on the serializer of the camera 101, and the first interface may be, but not limited to, a general-purpose input/output (General-Purpose Input/Output) Output, GPIO) interface.
  • a second interface corresponding to the first interface may be set on the deserializer of the chip 102, and the second interface may also adopt but is not limited to a GPIO interface.
  • the first interface and the second interface are connected by a coaxial cable, so that the transmission speed of instructions or data is faster.
  • a deserializer can be set on the chip, so that the synchronous exposure command can be transmitted to the serializer set on the camera, and the serializer can be used to send the video stream collected by the camera to the deserializer of the chip,
  • the serializer and the deserializer are connected by a coaxial cable, which improves the transmission speed of data and synchronous exposure commands, and can also reduce the error of synchronous exposure of each camera.
  • the device may further include:
  • controller 103 configured to generate a synchronous exposure instruction
  • the switch 104 connected to the controller 103 and each chip 102 is configured to send the synchronous exposure instruction generated by the controller 103 to the first chip 102 - 1 of the at least two chips 102 .
  • the switch 104 and each chip 102 may be connected through a network cable, and the switch 104 and the controller 103 may be connected through a network cable, thereby reducing the cost of the video capture device.
  • the controller 103 may also be used to generate associated control instructions.
  • the association control instruction is used to instruct the camera to perform a predetermined control operation other than the synchronous exposure operation.
  • the switch 104 may also be used to broadcast the association control instruction to each chip 102 through a network cable.
  • Each chip 102 receives the associated control instruction broadcasted by the switch 104 through the network cable, and controls the camera 101 connected to each chip 102 to execute the operation corresponding to the associated control instruction.
  • the associated control instruction includes, but is not limited to, at least one of the following:
  • a control command can be generated by the controller, and the control command includes the above-mentioned synchronous exposure command and/or associated control command, and the associated control command can be broadcast to each chip through the switch, so as to control the camera connected to each of the chips to execute.
  • the operation corresponding to the associated control instruction realizes controlling multiple cameras to synchronously execute the operation corresponding to the associated control instruction, and has high usability.
  • the above-mentioned video capture device may be a vehicle-mounted video capture device. That is, the video streams inside and/or outside the vehicle are collected by cameras deployed in various positions of the vehicle. Through at least two chips, each camera connected to each chip can be controlled to perform synchronous exposure, reducing the synchronous exposure error of each camera and reducing the number of cameras. exposure interference.
  • the sensor of the camera involved in the present disclosure needs to support the function of synchronously uploading video data, and the frame rate needs to support external triggering, that is, the chip is allowed to trigger the camera to perform video at a preset frame rate by issuing an instruction Stream acquisition, in addition, it is also necessary to allow the chip to control the fill light, camera working mode, etc.
  • a camera parameter obtained by combining an RGB camera and an IR camera is provided, as shown in Table 1, for example.
  • the parameters corresponding to any one of the at least two chips may be as shown in Table 2, for example.
  • the pin names used to realize the parallel interface function of image digital transmission can be DVP DIN11 ⁇ DIN0, in addition, the power supply parameters can be 12V/100mA, and other parameters can be related to the table.
  • the parameters in 3 are the same and will not be repeated here.
  • the corresponding relationship between the SBC internal deserializer MAX9286 and the interface of the processor S32V234 can be as shown in Table 4:
  • the pin PIN4 is located on the deserializer, and the pin PC10 is located on the processor S32v234 of the SBC chip for triggering the pin PIN4.
  • the present disclosure also provides a vehicle including the above-mentioned video capture device.
  • At least two cameras can be installed in at least one of the following positions: interior rearview mirror, exterior rearview mirror, central control screen, steering column, A-pillar, steering wheel, the surrounding area of the gear handle, car air conditioner, car audio , door, such as shown in Figure 4.
  • the at least two cameras include at least two of a normal color mode RGB camera, an infrared IR camera, and a Time Of Flight (TOF) camera, or the at least two cameras include a RGB camera, an IR camera, and a TOF camera.
  • a combination camera consisting of at least two.
  • the camera on the vehicle is configured to receive the working mode control instruction sent by the connected chip, and switch the working mode to the working mode indicated by the working mode control instruction, wherein the working mode control The instruction is used to instruct each camera in the combination camera to be turned on or off.
  • each camera can be triggered on the vehicle to perform synchronous exposure, or to switch the working mode, which is convenient for the driver to set the working mode of the cameras as required.
  • the present disclosure also provides a vehicle cabin detection method, where the vehicle cabin includes the video capture device provided in any of the above embodiments, for example, as shown in FIG. 5 , the method may include the following steps:
  • step 201 at least two channels of video data collected by at least two of the cameras are acquired.
  • At least two cameras may be installed in at least one of the following positions: interior rear-view mirror, exterior rear-view mirror, central control panel, steering column, A-pillar, steering wheel, the surrounding area of the gear handle, and vehicle air conditioner , car audio, door.
  • step 202 state detection is performed on the state of the persons in the vehicle cabin according to the at least two channels of the video data.
  • the on-board monitoring function can be implemented by performing state detection on the state of the personnel in the vehicle cabin, and the on-board monitoring function includes but is not limited to the DMS function and/or the Occupant Monitor System (Occupant Monitor System, OMS) function.
  • Occupant Monitor System Occupant Monitor System
  • At least two channels of video data can be collected through at least two cameras in the vehicle cabin, so as to perform state detection on the state of the people in the vehicle cabin, so as to realize the vehicle-mounted monitoring function, and at the same time improve the accuracy and timeliness of the vehicle-mounted monitoring, High availability.
  • the personnel in the vehicle cabin may include a driver, and based on at least two channels of video data, the driver's head posture and/or gaze area may be estimated.
  • the process of estimating the driver's head posture and/or the gaze area of the line of sight may include:
  • step 301 the correspondence between at least two cameras in the vehicle cabin and the deflection angles of the driver's head and/or eyes is determined based on at least two channels of the video data.
  • step 302 according to the correspondence between the at least two cameras in the cabin and the deflection angles of the driver's head and/or eyes, determine the driver's head posture and/or Gaze area.
  • the above-mentioned corresponding relationship may be determined according to the video streams collected by at least two cameras, and based on the above-mentioned corresponding relationship, the driver's head pose and/or the driver's head pose may be determined.
  • the gaze area can be used to analyze the DMS function. It can be seen that the smaller the synchronization error of the video streams collected by multiple cameras is, the more accurate the final analysis result of the DMS function is.
  • the correspondence between the at least two cameras and the deflection angles of the driver's head and eyes, respectively may be determined through a pre-trained neural network.
  • the driver sits in the driver's seat, deflects at least one part of the head and eyes, and looks at any one of the at least two cameras in the cabin, and the image captured by the camera Can be used as a sample image.
  • At least one of the driver's head deflection angle (HeadPose) and the eye deflection angle (Gaze) marked on the sample image is used as supervision, and the sample image and the video stream collected by other cameras are used as training data to train the preset neural network. , so that a correspondence between the camera and the deflection angle of the driver's head and/or eyes can be established.
  • a correspondence between the camera and the deflection angle of the driver's head and/or eyes can be established for each of the at least two cameras in the cabin.
  • the correspondence between the at least two cameras in the cabin and the deflection angles of the driver's head and/or eyes may be predetermined, and the driver's Head posture and/or gaze area, improving the accuracy of in-vehicle monitoring systems.
  • An embodiment of the present disclosure further provides a synchronous exposure method, as shown in FIG. 7 , which shows a synchronous exposure method according to an exemplary embodiment, and the method can be used for a first chip among at least two chips. , including the following steps:
  • step 401 a synchronous exposure instruction generated by the controller and sent through the switch is received.
  • the synchronous exposure instruction may be generated by the controller, and then sent to the first chip through the switch, and the first chip receives the synchronous exposure instruction.
  • step 402 the synchronous exposure instruction is sent to at least one second chip through hard wire, and sent to the camera connected to itself through a coaxial wire, so that the camera connected to each of the second chips and the camera connected to itself The connected camera performs synchronous exposure according to the synchronous exposure instruction.
  • the first chip can send the synchronous exposure instruction to at least one second chip among the at least two chips by hard wire, so that the transmission speed of the synchronous exposure instruction is faster.
  • the synchronous exposure instruction can be sent to the cameras connected to each second chip through a coaxial cable.
  • the first chip can also send the synchronous exposure instruction to the camera connected to the first chip through a coaxial cable. Therefore, the camera connected to each of the second chips and the camera connected to itself can perform synchronous exposure according to the synchronous exposure instruction.
  • the purpose of synchronizing exposure of at least two cameras in the middle of the video stream acquisition process is achieved, and the error of synchronizing exposure is reduced.
  • the above method may further include:
  • step 403 an associated control instruction generated by the controller and broadcast through the switch is received.
  • the associated control instruction is used to instruct the camera to perform a predetermined control operation other than the synchronous exposure operation, and may include at least one of the following: a video data synchronization upload instruction, a fill light switch control instruction, The working mode control instruction of the combined camera, wherein the working mode control instruction is used to instruct each camera in the combined camera to be turned on or off.
  • step 404 the connected camera is controlled to perform a corresponding operation according to the associated control instruction.
  • the first chip when the associated control instruction includes a video data synchronous upload instruction, can control the camera connected to itself to upload video data.
  • the first chip can control the camera connected to itself to turn on or off the fill light.
  • one of the at least two cameras may be triggered to turn on the fill light, and the other cameras may be controlled to turn off the fill light.
  • the camera that turns on the fill light includes, but is not limited to, an IR camera.
  • IR camera if multiple IR cameras turn on the fill lights at the same time, exposure interference will be caused. Therefore, the fill light of only one IR camera can be turned on, and the fill lights of other IR cameras can be turned off, thereby avoiding multiple cameras. exposure interference.
  • the first chip can control the camera connected to itself to switch the working mode.
  • the first chip is connected with an RGB camera and an IR camera, and the target working mode is the IR mode, then the chip can turn off the RGB camera, initialize the IR camera, and put the IR camera in the working mode.
  • the connected camera may be controlled to perform the corresponding operation in the case of receiving the associated control instruction.
  • the purpose of controlling the camera through external trigger is realized, and the usability is high.
  • Embodiments of the present disclosure also provide a synchronous exposure method, for example, as shown in FIG. 9 .
  • FIG. 9 shows a synchronous exposure method according to an exemplary embodiment, and the method can be used for a second chip among at least two chips. , including the following steps:
  • step 501 a synchronous exposure instruction sent by the first chip through a hard wire is received.
  • step 502 the synchronous exposure instruction is sent to a target camera through a coaxial cable, so as to control the synchronous exposure of the target camera and the camera connected to the first chip through a coaxial cable.
  • the camera connected to the first chip and the camera connected to each second chip can perform synchronous exposure according to the synchronous exposure instruction, thereby reducing the error of synchronous exposure.
  • the second chip may also receive an associated control instruction generated by the controller and broadcast through a switch, wherein the associated control instruction is used to instruct the camera to perform a predetermined control operation other than the synchronous exposure operation .
  • the second chip can control the camera connected to itself to perform corresponding operations according to the associated control instruction.
  • the purpose of controlling the camera through an external trigger is achieved, and the usability is high.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored in the storage medium, and the computer program is used to execute any of the above-mentioned vehicle cabin detection methods or synchronous exposure methods.
  • embodiments of the present disclosure provide a computer program product, comprising computer-readable code, when the computer-readable code is executed on a device, the processor in the device executes any of the above implementations. Examples provide instructions for the cabin detection method or the simultaneous exposure method.
  • the embodiments of the present disclosure further provide another computer program product for storing computer-readable instructions, and when the instructions are executed, cause the computer to execute the vehicle cabin detection method or synchronization provided by any of the foregoing embodiments The operation of the exposure method.
  • the computer program product can be specifically implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • a software development kit Software Development Kit, SDK
  • An embodiment of the present disclosure also provides a camera module, as shown in FIG. 10 , including:
  • the camera sensor 602 connected to the camera 601 is used to receive the synchronous exposure instruction sent by the chip through the coaxial cable, and perform exposure according to the synchronous exposure instruction.
  • the camera module can perform exposure based on the external synchronous exposure instruction, so as to realize the purpose of performing the corresponding operation according to the external trigger instruction, and the usability is high.
  • the camera module further includes:
  • the serializer 603 connected to the camera and the camera sensor is configured to send the video data included in the video stream to the chip through the coaxial cable.
  • the camera sensor 602 is further configured to receive the associated control instruction sent by the chip through the coaxial cable, and control the execution of the corresponding operation based on the associated control instruction, wherein the associated control instruction is used for The camera is instructed to perform a predetermined control operation other than the synchronized exposure operation.
  • An embodiment of the present disclosure also provides a vehicle controller, such as shown in FIG. 12 , including:
  • an instruction generation module 702 configured to generate a synchronous exposure instruction based on the trigger instruction
  • the instruction sending module 703 is configured to send the synchronous exposure instruction to the switch, so that the switch sends the synchronous exposure instruction to the first chip in the at least two chips, and the first chip sends the synchronous exposure instruction to the first chip through hard wires. Simultaneous exposure instructions are sent to at least one second chip of at least two of the chips.
  • the person in the vehicle can send a trigger instruction to trigger the function of the on-board monitoring system.
  • the vehicle controller obtains the trigger instruction, it generates a synchronous exposure instruction, sends the synchronous exposure instruction to the switch, and the interactive machine sends it to the first synchronous exposure instruction.
  • a chip, the first chip sends the simultaneous exposure command to at least one second chip of at least two of the chips through hard wires. Therefore, at least two cameras can be controlled to be exposed synchronously during video stream collection, thereby improving the accuracy and timeliness of the vehicle monitoring function.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

本公开提供了一种视频采集设备、车辆、车舱检测及同步曝光方法，其中，所述视频采集设备包括：用于采集视频流的至少两个相机；与所述至少两个相机连接的至少两个芯片，其中，至少两个所述芯片分别连接不同的相机，各所述芯片之间通过硬线连接及传输同步曝光指令，各所述芯片用于向各自连接的相机发送同步曝光指令，以使各所述相机根据所述同步曝光指令进行同步曝光。

Description

视频采集设备、车辆、车舱检测及同步曝光方法
相关申请的交叉引用
本公开要求于2021年3月26日提交的、申请号为2021103286585的中国专利公开的优先权，该中国专利公开的全部内容以引用的方式并入本文中。
技术领域
本公开涉及数据处理领域，尤其涉及一种视频采集设备、车辆、车舱检测及同步曝光方法。
背景技术
目前，智能车辆的应用越来越广泛。在智能车辆上，以驾驶员监控系统(Driver Monitor System，DMS)功能为例，其可以进行的功能包括但不限于检测疲劳，分心驾驶，危险动作等。在车辆上的不同位置可以部署多个相机，从而进行视频流采集，基于采集到的视频流进行DMS功能的分析。
发明内容
本公开提供了一种视频采集设备、车辆、车舱检测及同步曝光方法。
根据本公开实施例的第一方面，提供一种视频采集设备，包括：用于采集视频流的至少两个相机；与所述至少两个相机连接的至少两个芯片，其中，至少两个所述芯片分别连接不同的相机，各所述芯片之间通过硬线连接及传输同步曝光指令，各所述芯片用于向各自连接的相机发送同步曝光指令，以使各所述相机根据所述同步曝光指令进行同步曝光。
在一些可选实施例中，至少两个所述芯片包括第一芯片和至少一个第二芯片；所述第一芯片用于接收所述同步曝光指令，通过硬线发送所述同步曝光指令给各所述第二芯片，并发送所述同步曝光指令给与所述第一芯片连接的相机；各所述第二芯片用于通过硬线接收所述第一芯片发送的所述同步曝光指令，并发送所述同步曝光指令给与各所述第二芯片连接的相机。
在一些可选实施例中，所述芯片上设置有解串器，所述相机设置有与所述解串器匹配的串化器；所述串化器与所述解串器之间通过同轴线进行连接；所述串化器用于将相机采集到的视频流通过同轴线发送到所述解串器；和/或所述解串器用于通过同轴线将所述同步曝光指令传输至所述串化器。
在一些可选实施例中，还包括：控制器，用于生成同步曝光指令；与所述控制器和各所述芯片连接的交换机，用于向第一芯片发送所述控制器生成的所述同步曝光指令。
在一些可选实施例中，所述交换机与各所述芯片之间通过网线进行连接，所述交换机与控制器之间通过网线进行连接。
在一些可选实施例中，所述控制器还用于生成关联控制指令，其中，所述关联控制指令用于指示所述相机执行除了同步曝光操作之外的预定控制操作；所述交换机还用于通过网线将所述关联控制指令广播给各所述芯片；各所述芯片还用于通过网线接收所述交换机广播的所述关联控制指令，并控制与各所述芯片连接的相机执行所述关联控制指令对应的操作。
在一些可选实施例中，所述关联控制指令包括以下至少一项：视频数据同步上传指令、补光灯开关控制指令、组合相机的工作模式控制指令，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。
在一些可选实施例中，所述设备为车载视频采集设备。
根据本公开实施例的第二方面，提供一种车辆，包括如第一方面任一所述的视频采集设备。
在一些可选实施例中，所述至少两个相机安装于以下至少一个位置：车内后视镜、车外后视镜、中控屏、转向柱、A柱、方向盘、档把周边区域、车载空调、车载音响、车门；和/或所述至少两个相机包括普通彩色模式RGB相机、红外IR相机、飞行时间TOF相机、组合相机中的至少两个，由RGB相机、IR相机和TOF相机中的至少两个组成所述组合相机。
在一些可选实施例中，所述相机包括由RGB相机、IR相机和TOF相机中的至少两个组成的组合相机，以及所述相机用于接收连接的芯片发送的工作模式控制指令，并将自身的工作模式切换到所述工作模式控制指令所指示的工作模式，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。
根据本公开实施例的第三方面，提供一种车舱检测方法，所述车舱包括如第一方面任一所述的视频采集设备；所述方法包括：获取由至少两个所述相机采集的至少两路视频数据；根据至少两路所述视频数据，对所述车舱内的人员的状态进行状态检测。
在一些可选实施例中，所述根据至少两路所述视频数据，对所述车舱内的人员的状态进行状态检测，包括：基于所述至少两路视频数据，估计驾驶员的头部姿态和/或视线 注视区域。
在一些可选实施例中，所述基于所述至少两路视频数据，估计所述驾驶员的头部姿态和/或视线注视区域，包括：基于至少两路所述视频数据确定所述车舱内的至少两个所述相机与所述驾驶员头部和/或眼睛的偏转角度之间的对应关系；根据所述车舱内的至少两个所述相机与所述驾驶员头部和/或眼睛的偏转角度之间的对应关系，确定所述驾驶员的头部姿态和/或视线注视区域。
在一些可选实施例中，所述至少两个所述相机分别与所述驾驶员头部、眼睛的偏转角度之间的对应关系通过预先训练的神经网络确定；所述方法还包括：获取所述驾驶员偏转头部和眼睛中的至少一个部位的状态下，所述车舱内的至少两个相机中的任意相机拍摄的图像作为样本图像；以所述样本图像上标注的所述驾驶员的头部偏转角度和眼睛偏转角度中的至少一项为监督，对所述神经网络进行训练。
根据本公开实施例的第四方面，提供一种同步曝光方法，包括：接收由控制器生成并通过交换机发送的同步曝光指令；将所述同步曝光指令通过硬线发送给至少一个第二芯片，和通过同轴线发送给与自身连接的相机，以使与各所述第二芯片连接的相机和与自身连接的相机根据所述同步曝光指令进行同步曝光。
在一些可选实施例中，所述方法还包括：接收由所述控制器生成并通过所述交换机广播的关联控制指令，其中，所述关联控制指令用于指示所述相机执行除了同步曝光操作之外的预定控制操作；根据所述关联控制指令，控制与自身连接的相机执行对应的操作。
在一些可选实施例中，所述关联控制指令包括以下至少一项：视频数据同步上传指令、补光灯开关控制指令、组合相机的工作模式控制指令，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。
根据本公开实施例的第五方面，提供一种同步曝光方法，包括：接收由第一芯片通过硬线发送的同步曝光指令；将所述同步曝光指令通过同轴线发送给与自身连接的目标相机，以控制所述目标相机和与所述第一芯片通过同轴线连接的相机同步曝光。
根据本公开实施例的第六方面，提供一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行上述第三方面任一所述的方法。
根据本公开实施例的第七方面，提供一种相机模组，所述相机模组包括如上述第一方面任一所述的视频采集设备中的相机，所述相机模组包括：摄像头，用于进行视频流 采集；与所述摄像头连接的相机传感器，用于通过同轴线接收芯片发送的同步曝光指令，并根据所述同步曝光指令进行曝光。
在一些可选实施例中，还包括：与所述摄像头和所述相机传感器连接的串化器，用于将采集到的视频流包括的视频数据通过所述同轴线发送给所述芯片。
在一些可选实施例中，所述相机传感器还用于通过所述同轴线接收所述芯片发送的关联控制指令，并基于所述关联控制指令控制执行所述关联控制指令对应的预定控制操作。
根据本公开实施例的第八方面，提供一种车辆的控制器，包括：人机交互模块，用于获取触发所述车辆的车载监控系统功能的触发指令；指令生成模块，用于基于所述触发指令，生成同步曝光指令；指令发送模块，用于将所述同步曝光指令发送给交换机，以使所述交换机发送所述同步曝光指令给至少两个芯片中的第一芯片，所述第一芯片通过硬线将所述同步曝光指令发送给至少两个所述芯片中的至少一个第二芯片。
本公开的实施例提供的技术方案可以包括以下有益效果：
本公开实施例中，可以通过各芯片向各自连接的相机发送同步曝光指令，使得各相机根据同步曝光指令进行同步曝光，降低了多个相机同步曝光的误差，可用性高。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
图1是本公开根据一示例性实施例示出的一种视频采集设备结构示意图；
图2是本公开根据一示例性实施例示出的另一种视频采集设备结构示意图；
图3是本公开根据一示例性实施例示出的另一种视频采集设备结构示意图；
图4是本公开根据一示例性实施例示出的一种相机安装在车辆内的场景示意图；
图5是本公开根据一示例性实施例示出的一种车舱检测方法流程图；
图6是本公开根据一示例性实施例示出的另一种车舱检测方法流程图；
图7是本公开根据一示例性实施例示出的一种同步曝光方法流程图；
图8是本公开根据一示例性实施例示出的另一种同步曝光方法流程图；
图9是本公开根据一示例性实施例示出的另一种同步曝光方法流程图；
图10是本公开根据一示例性实施例示出的一种相机模组结构示意图；
图11是本公开根据一示例性实施例示出的另一种相机模组结构示意图；
图12是本公开根据一示例性实施例示出的一种车辆的控制器的结构示意图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
在本公开运行的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所运行的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中运行的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所运行的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
本公开提供了一种视频采集设备，例如图1所示，包括：
用于采集视频流的至少两个相机101；
与所述至少两个相机连接的至少两个芯片102，其中，至少两个所述芯片102分别连接不同的相机101，各所述芯片102之间通过硬线连接及传输同步曝光指令，各所述芯片102用于向各自连接的相机101发送同步曝光指令，以使各所述相机101根据所述同步曝光指令进行同步曝光。
在本公开实施例中，各芯片102可以采用但不限于系统基本芯片(System Basic Chip，SBC)。各芯片102可以分别连接不同的相机101，相机101包括但不限于普通的色彩模式(Red Green Blue，RGB)相机，或红外(InfraRed，IR)相机，或RGB-IR相机。 或者可以通过单独的RGB相机和IR相机组合后得到的组合相机，共同实现RGB-IR相机的功能。在本公开实施例中的相机101需要支持外部触发功能，即可以由所连接的芯片102发送的控制指令进行触发，该控制指令包括但不限于同步曝光指令。
上述实施例中，可以通过各芯片向各自连接的相机发送同步曝光指令，使得各相机同步进行同步曝光，降低了同步曝光的误差，其同步曝光的误差可达到微秒级，可用性高。
在一些可选实施例中，在图1所示的设备中，至少两个芯片102可以包括第一芯片102-1和至少一个第二芯片102-2，例如图2所示。第一芯片102-1可以是各芯片中预先设置的主控(master)芯片。
第一芯片102-1可用于接收同步曝光指令，并通过硬线将该同步曝光指令发送给各第二芯片102-2以及与第一芯片102-1连接的相机101。
各第二芯片102-2用于通过硬线接收第一芯片102-1发送的同步曝光指令，并发送该同步曝光指令给与各第二芯片102-2连接的相机101。
在本公开实施例中，为了减小各相机同步曝光的误差，可以将至少两个芯片两两之间通过硬线进行连接。
这样第一芯片102-1通过硬线发送同步曝光指令给各第二芯片102-2，相较于芯片之间通过网线发送指令的发送速度更快，从而可以让各芯片分别连接的不同相机之间同步曝光的误差缩小到微秒级。
上述实施例中，第一芯片通过硬线发送同步曝光指令给各第二芯片，各芯片将同步曝光指令发送给各芯片分别连接的相机，从而降低了各相机同步曝光的误差，可用性高。
在一些可选实施例中，各芯片102上设置有解串器，与各芯片102分别连接的相机101设置有与解串器匹配的串化器，串化器与解串器之间通过同轴线进行连接。其中，串化器可以用于将相机采集到的视频流通过同轴线发送到所述解串器，解串器用于通过同轴线将所述同步曝光指令传输至所述串化器。
在本公开实施例中，为实现解串器与串化器的匹配，可以在相机101的串化器上设置第一接口，第一接口可以采用但不限于通用输入输出(General-Purpose Input/Output，GPIO)接口。芯片102的解串器上可以设置与第一接口对应的一个第二接口，第二接口也可以采用但不限于GPIO接口。第一接口与所述第二接口之间通过同轴线进行连接，使得指令或数据的传输速度更快。上述实施例中，可以在芯片上设置解串器，从而可以 将同步曝光指令传输至相机上设置的串化器，串化器可以用于将相机采集的视频流发送到芯片的解串器，串化器和解串器之间通过同轴线连接，提高了数据和同步曝光指令的传输速度，同样可以降低各相机同步曝光的误差。
在一些可选实施例中，例如图3所示，该设备还可以包括：
控制器103，用于生成同步曝光指令；
与所述控制器103和各芯片102连接的交换机104，用于向至少两个所述芯片102中的第一芯片102-1发送所述控制器103生成的所述同步曝光指令。
可选地，交换机104与各芯片102之间可以通过网线进行连接，交换机104与控制器103之间可以通过网线进行连接，降低视频采集设备的成本。在一些可选实施例中，控制器103还可以用于生成关联控制指令。其中，关联控制指令用于指示所述相机执行除了同步曝光操作之外的预定控制操作，相应地，交换机104还可以用于通过网线将关联控制指令广播给各芯片102。
各芯片102通过网线接收交换机104广播的关联控制指令，并控制与各芯片102连接的相机101执行关联控制指令对应的操作。
在本公开实施例中，关联控制指令包括但不限于以下至少一项：
视频数据同步上传指令、补光灯开关控制指令、组合相机的工作模式控制指令，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。上述实施例中，可以通过控制器生成控制指令，控制指令包括上述的同步曝光指令和/或关联控制指令，关联控制指令可以通过交换机广播给各芯片，从而控制与各所述芯片连接的相机执行所述关联控制指令对应的操作，实现了控制多个相机同步执行关联控制指令对应的操作，可用性高。
在一些可选实施例中，上述的视频采集设备可以为车载视频采集设备。即通过部署在车辆各个位置的相机采集车内和/或车外的视频流，通过至少两个芯片，可以控制各芯片连接的各相机进行同步曝光，减少各相机的同步曝光误差，减少各相机曝光干扰。在一些可选实施例中，本公开中涉及到的相机的传感器需要支持视频数据同步上传的功能，帧率需要支持外部触发，即允许芯片通过下发指令触发相机以预设的帧率进行视频流采集，另外，还需要允许芯片对补光灯、相机工作模式等进行控制。
在一个示例中，提供了一种由RGB相机和IR相机组合得到的相机的参数，例如表1所示。
表1
Figure PCTCN2022078297-appb-000001
在一个示例中，至少两个芯片中的任意一个芯片对应的参数，可以例如表2所示。
表2
Figure PCTCN2022078297-appb-000002
在一个示例中，RGB相机的串化器MAX96705的接口与对应的参数例如表3所示：
表3
Figure PCTCN2022078297-appb-000003
IR相机的串化器MAX96705的接口与对应的参数中，用于实现图像数字传输并行接口功能的管脚名称可以是DVP DIN11～DIN0，另外，电源参数可以是12V/100mA，其他参数可以与表3中的参数一致，在此不再赘述。
在一个示例中，SBC内部解串器MAX9286与处理器S32V234的接口之间的对应关系可以例如表4所示：
表4
Figure PCTCN2022078297-appb-000004
其中，管脚PIN4位于解串器上，管脚PC10位于SBC芯片的处理器S32v234上，用于触发管脚PIN4。
上述实施例中，仅是对至少两个芯片、至少两个相机以及涉及到的接口参数进行举例说明，实际应用中，如果采用本公开提供的视频采集设备，其采用的硬件设备仅是参数上的替换，都应属于本公开的保护范围。
本公开还提供了一种车辆，该车辆包括上述视频采集设备。
可选地，至少两个相机可以安装于以下至少一个位置：车内后视镜、车外后视镜、中控屏、转向柱、A柱、方向盘、档把周边区域、车载空调、车载音响、车门，例如图 4所示。
至少两个相机包括普通彩色模式RGB相机、红外IR相机、飞行时间(Time Of Flight,TOF)相机中的至少两个，或者所述至少两个相机包括由RGB相机、IR相机和TOF相机中的至少两个组成的组合相机。
在一些可选实施例中，车辆上的相机用于接收连接的芯片发送的工作模式控制指令，并将工作模式切换到所述工作模式控制指令所指示的工作模式，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。
上述实施例中，可以在车辆上，触发各相机进行同步曝光，或进行工作模式切换，便于驾驶人员根据需要对相机进行工作模式设置。
本公开还提供了一种车舱检测方法，车舱包括上述任一实施例提供的视频采集设备，例如图5所示，该方法可以包括以下步骤：
在步骤201中，获取由至少两个所述相机采集的至少两路视频数据。
在本公开实施例中，至少两个相机可以安装在以下至少一个位置：车内后视镜、车外后视镜、中控屏、转向柱、A柱、方向盘、档把周边区域、车载空调、车载音响、车门。
在步骤202中，根据至少两路所述视频数据，对所述车舱内的人员的状态进行状态检测。
在本公开实施例中，通过对车舱内人员的状态进行状态检测，可以实现车载监控功能，车载监控功能包括但不限于DMS功能和/或乘客监控系统(Occupant Monitor System，OMS)功能。
上述实施例中，可以通过车舱内的至少两个相机采集至少两路视频数据，从而对车舱内人员状态进行状态检测，实现车载监控功能的同时，提高车载监控的准确性、时效性，可用性高。
在一些可选实施例中，车舱内的人员可以包括驾驶员，基于至少两路视频数据，可以估计驾驶员的头部姿态和/或视线注视区域。
在本公开实施例中，例如图6所示，基于至少两路视频数据，估计驾驶员的头部姿态和/或视线注视区域的过程可以包括：
在步骤301中，基于至少两路所述视频数据确定所述车舱内的至少两个所述相机与 所述驾驶员头部和/或眼睛的偏转角度之间的对应关系。
在步骤302中，根据所述车舱内的至少两个所述相机与所述驾驶员头部和/或眼睛的偏转角度之间的对应关系，确定所述驾驶员的头部姿态和/或视线注视区域。
在本公开实施例中，在驾驶员进入车舱后，可以根据至少两个相机采集到的视频流确定上述对应关系，并基于上述对应关系，确定驾驶员的头部姿态(headpose)和/或视线(Gaze)注视区域，进而可以进行DMS功能分析。可以看出，多个相机采集的视频流同步误差越小，最终得到的DMS功能的分析结果越准确。
在一些可选实施例中，至少两个所述相机分别与所述驾驶员头部、眼睛的偏转角度之间的对应关系可以通过预先训练的神经网络确定。
在本公开实施例中，驾驶员坐在主驾位置上，偏转头部和眼睛中的至少一个部位的状态下，注视车舱内至少两个相机中的任意一个相机，该相机拍摄到的图像可以作为样本图像。样本图像上标注的驾驶员头部偏转角度(HeadPose)和眼睛偏转角度(Gaze)中的至少一项作为监督，将样本图像和其他相机采集的视频流作为训练数据，对预设神经网络进行训练，从而可以建立该相机与驾驶员头部和/或眼睛的偏转角度之间的对应关系。采用同样的方式，可以针对车舱内至少两个相机中的每个相机，建立该相机与驾驶员头部和/或眼睛的偏转角度之间的对应关系。
上述实施例中，可以预先确定车舱内的至少两个所述相机与所述驾驶员头部和/或眼睛的偏转角度之间的对应关系，后续可以基于该对应关系，快速确定驾驶员的头部姿态和/或视线注视区域，提高了车载监控系统的准确性。
本公开实施例还提供了一种同步曝光方法，例如图7所示，图7是根据一示例性实施例示出的一种同步曝光方法，该方法可以用于至少两个芯片中的第一芯片，包括以下步骤：
在步骤401中，接收由控制器生成并通过交换机发送的同步曝光指令。
在本公开实施例中，可以由控制器生成同步曝光指令，然后通过交换机发送给第一芯片，第一芯片接收该同步曝光指令。
在步骤402中，将所述同步曝光指令通过硬线发送给至少一个第二芯片，和通过同轴线发送给与自身连接的相机，以使与各所述第二芯片连接的相机和与自身连接的相机根据所述同步曝光指令进行同步曝光。
在本公开实施例中，第一芯片可以将该同步曝光指令通过硬线发送给至少两个芯片中的至少一个第二芯片，使得同步曝光指令的传输速度更快。至少一个第二芯片接收后，可以通过同轴线将同步曝光指令发送给与各第二芯片连接的相机。另外，第一芯片还可以通过同轴线将同步曝光指令发送给与第一芯片连接的相机。从而可以让各所述第二芯片连接的相机和与自身连接的相机根据所述同步曝光指令进行同步曝光。
上述实施例中，实现了让至少两个相机在视频流采集过程中间同步曝光的目的，减少了同步曝光的误差。
在一些可选实施例中，例如图8所示，上述方法还可以包括：
在步骤403中，接收由所述控制器生成并通过所述交换机广播的关联控制指令。
在本公开实施例中，所述关联控制指令用于指示所述相机执行除了同步曝光操作之外的预定控制操作，可以包括以下至少一项：视频数据同步上传指令、补光灯开关控制指令、组合相机的工作模式控制指令，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。
在步骤404中，根据所述关联控制指令，控制连接的相机执行对应的操作。
在本公开实施例中，在关联控制指令包括视频数据同步上传指令的情况下，第一芯片可以控制与自身连接的相机进行视频数据上传。
在关联控制指令包括补光灯开关控制指令的情况下，第一芯片可以控制与自身连接的相机打开或关闭补光灯。在一个可能地实现方式中，可以触发至少两个相机中的一个打开补光灯，以及控制其他相机关闭补光灯。其中，打开补光灯的相机包括但不限于IR相机。在本公开实施例中，如果多个IR相机同时打开补光灯，会造成曝光干扰，因此，可以只打开一个IR相机的补光灯，关闭其他IR相机的补光灯，从而避免多个相机曝光干扰。
在关联控制指令包括工作模式控制指令的情况下，第一芯片可以控制与自身连接的相机切换工作模式。在一个可能地实现方式中，第一芯片与一个RGB相机和一个IR相机连接，目标工作模式为IR模式，那么该芯片可以关闭RGB相机，初始化IR相机，让IR相机处于工作模式。
上述实施例中，可以在接收到关联控制指令的情况下，控制连接的相机执行对应的操作。实现了对相机可以通过外部触发进行控制的目的，可用性高。
本公开实施例还提供了一种同步曝光方法，例如图9所示，图9是根据一示例性实施例示出的一种同步曝光方法，该方法可以用于至少两个芯片中的第二芯片，包括以下步骤：
在步骤501中，接收由第一芯片通过硬线发送的同步曝光指令。
在步骤502中，将所述同步曝光指令通过同轴线发送给目标相机，以控制所述目标相机和与所述第一芯片通过同轴线连接的相机同步曝光。
上述实施例中，可以让第一芯片连接的相机和与各第二芯片连接的相机根据所述同步曝光指令进行同步曝光，降低了同步曝光的误差。
在一些可选实施例中，第二芯片还可以接收由控制器生成并通过交换机广播的关联控制指令，其中所述关联控制指令用于指示所述相机执行除了同步曝光操作之外的预定控制操作。第二芯片可以根据该关联控制指令，控制与自身连接的相机执行对应的操作。
上述实施例中，实现了对相机可以通过外部触发进行控制的目的，可用性高。
本公开实施例还提供了一种计算机可读存储介质，存储介质存储有计算机程序，计算机程序用于执行上述任一所述的车舱检测方法或同步曝光方法。
在一些可选实施例中，本公开实施例提供了一种计算机程序产品，包括计算机可读代码，当计算机可读代码在设备上运行时，设备中的处理器执行用于实现如上任一实施例提供的车舱检测方法或同步曝光方法的指令。
在一些可选实施例中，本公开实施例还提供了另一种计算机程序产品，用于存储计算机可读指令，指令被执行时使得计算机执行上述任一实施例提供的车舱检测方法或同步曝光方法的操作。
该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
在本公开实施例中还提供了一种相机模组，例如图10所示，包括：
摄像头601，用于进行视频流采集；
与摄像头601连接的相机传感器602，用于通过同轴线接收芯片发送的同步曝光 指令，并根据所述同步曝光指令进行曝光。
上述实施例中，相机模组可以基于外部同步曝光指令，进行曝光，实现了相机模组根据外部触发指令执行相应操作的目的，可用性高。
在一些可选实施例中，例如图11所示，该相机模组还包括：
与所述摄像头和所述相机传感器连接的串化器603，用于将所述视频流包括的视频数据通过所述同轴线发送给所述芯片。
在上述实施例中，相机传感器602还用于通过所述同轴线接收所述芯片发送的关联控制指令，并基于所述关联控制指令控制执行对应的操作，其中，所述关联控制指令用于指示所述相机执行除了同步曝光操作之外的预定控制操作。
在本公开实施例中还提供了一种车辆的控制器，例如图12所示，包括：
人机交互模块701，用于获取触发所述车辆的车载监控系统功能的触发指令；
指令生成模块702，用于基于所述触发指令，生成同步曝光指令；
指令发送模块703，用于将所述同步曝光指令发送给交换机，以使所述交换机发送所述同步曝光指令给至少两个芯片中的第一芯片，所述第一芯片通过硬线将所述同步曝光指令发送给至少两个所述芯片中的至少一个第二芯片。
上述实施例中，车内人员可以通过发出触发车载监控系统功能的触发指令，车辆控制器获取到该触发指令后，生成同步曝光指令，将该同步曝光指令发送给交换机，交互机再发送给第一芯片，所述第一芯片通过硬线将所述同步曝光指令发送给至少两个所述芯片中的至少一个第二芯片。从而可以控制至少两个相机在进行视频流采集时同步曝光，提高车载监控功能的准确性、时效性。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或者惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。
以上所述仅为本公开的较佳实施例而已，并不用以限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开保护的范围之内。

Claims (24)

  1. 一种视频采集设备，包括：
    用于采集视频流的至少两个相机；
    与所述至少两个相机连接的至少两个芯片，其中，至少两个所述芯片分别连接不同的相机，各所述芯片之间通过硬线连接及传输同步曝光指令，各所述芯片用于向各自连接的相机发送同步曝光指令，以使各所述相机根据所述同步曝光指令进行同步曝光。
  2. 根据权利要求1所述的设备，其特征在于，至少两个所述芯片包括第一芯片和至少一个第二芯片；
    所述第一芯片用于接收所述同步曝光指令，通过硬线发送所述同步曝光指令给各所述第二芯片，并发送所述同步曝光指令给与所述第一芯片连接的相机；
    各所述第二芯片用于通过硬线接收所述第一芯片发送的所述同步曝光指令，并发送所述同步曝光指令给与各所述第二芯片连接的相机。
  3. 根据权利要求1或2所述的设备，其特征在于，
    所述芯片上设置有解串器，所述相机设置有与所述解串器匹配的串化器；
    所述串化器与所述解串器之间通过同轴线进行连接；
    所述串化器用于将所述相机采集到的视频流通过同轴线发送到所述解串器；
    所述解串器用于通过同轴线将所述同步曝光指令传输至所述串化器。
  4. 根据权利要求2所述的设备，还包括：
    控制器，用于生成所述同步曝光指令；
    与所述控制器和各所述芯片连接的交换机，用于向所述第一芯片发送所述控制器生成的所述同步曝光指令。
  5. 根据权利要求4所述的设备，其特征在于，
    所述交换机与各所述芯片之间通过网线进行连接；
    所述交换机与所述控制器之间通过网线进行连接。
  6. 根据权利要求5所述的设备，其特征在于，
    所述控制器还用于生成关联控制指令；
    所述交换机还用于通过网线将所述关联控制指令广播给各所述芯片；
    各所述芯片还用于通过网线接收所述交换机广播的所述关联控制指令，并控制与各所述芯片连接的相机执行所述关联控制指令对应的预定控制操作。
  7. 根据权利要求6所述的设备，其特征在于，所述关联控制指令包括以下至少一项：
    视频数据同步上传指令；
    补光灯开关控制指令；
    组合相机的工作模式控制指令，用于指示所述组合相机中的各相机的开启或关闭。
  8. 根据权利要求1至7任一所述的设备，其特征在于，所述设备为车载视频采集设备。
  9. 一种车辆，其特征在于，包括如权利要求1至8任一所述的视频采集设备。
  10. 根据权利要求9所述的车辆，其特征在于，
    所述至少两个相机安装于以下至少一个位置：车内后视镜、车外后视镜、中控屏、转向柱、A柱、方向盘、档把周边区域、车载空调、车载音响、车门；
    所述至少两个相机包括普通彩色模式RGB相机、红外IR相机、飞行时间TOF相机、组合相机中的至少两个，由RGB相机、IR相机和TOF相机中的至少两个组成所述组合相机。
  11. 根据权利要求9或10所述的车辆，其特征在于，所述相机用于接收连接的芯片发送的工作模式控制指令，并将自身的工作模式切换到所述工作模式控制指令所指示的工作模式，其中，所述工作模式控制指令用于指示所述组合相机中的各相机的开启或关闭。
  12. 一种车舱检测方法，所述车舱包括如权利要求1至8任一所述的视频采集设备；所述方法包括：
    获取由至少两个所述相机采集的至少两路视频数据；
    根据至少两路所述视频数据，对所述车舱内的人员的状态进行状态检测。
  13. 根据权利要求12所述的方法，其特征在于，所述根据至少两路所述视频数据，对所述车舱内的人员的状态进行状态检测，包括：
    基于所述至少两路视频数据，估计驾驶员的头部姿态和/或视线注视区域。
  14. 根据权利要求13所述的方法，其特征在于，所述基于所述至少两路视频数据，估计所述驾驶员的头部姿态和/或视线注视区域，包括：
    基于至少两路所述视频数据确定所述车舱内的至少两个所述相机与所述驾驶员头部和/或眼睛的偏转角度之间的对应关系；
    根据所述车舱内的至少两个所述相机与所述驾驶员头部和/或眼睛的偏转角度之间的对应关系，确定所述驾驶员的头部姿态和/或视线注视区域。
  15. 根据权利要求13所述的方法，其特征在于，所述至少两个所述相机分别与所述驾驶员头部、眼睛的偏转角度之间的对应关系通过预先如下训练的神经网络确定：
    获取所述驾驶员偏转头部和眼睛中的至少一个部位的状态下，所述车舱内的至少两个相机中的任意相机拍摄的图像作为样本图像；
    以所述样本图像上标注的所述驾驶员的头部偏转角度和眼睛偏转角度中的至少一项为监督，对所述神经网络进行训练。
  16. 一种同步曝光方法，其特征在于，包括：
    接收由控制器生成并通过交换机发送的同步曝光指令；
    将所述同步曝光指令通过硬线发送给至少一个第二芯片，和通过同轴线发送给与自身连接的相机，以使与各所述第二芯片连接的相机和与自身连接的相机根据所述同步曝光指令进行同步曝光。
  17. 根据权利要求16所述的方法，还包括：
    接收由所述控制器生成并通过所述交换机广播的关联控制指令；
    根据所述关联控制指令，控制与自身连接的相机执行所述关联控制指令对应的预定控制操作。
  18. 根据权利要求17所述的方法，其特征在于，所述关联控制指令包括以下至少一项：
    视频数据同步上传指令；
    补光灯开关控制指令；
    组合相机的工作模式控制指令，用于指示所述组合相机中的各相机的开启或关闭。
  19. 一种同步曝光方法，其特征在于，包括：
    接收由第一芯片通过硬线发送的同步曝光指令；
    将所述同步曝光指令通过同轴线发送给与自身连接的目标相机，以控制所述目标相机和与所述第一芯片通过同轴线连接的相机同步曝光。
  20. 一种计算机可读存储介质，其特征在于，所述存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求12至19任一所述的方法。
  21. 一种相机模组，其特征在于，所述相机模组包括如权利要求1至8任一所述的视频采集设备中的相机，所述相机模组包括：
    摄像头，用于进行视频流采集；
    与所述摄像头连接的相机传感器，用于通过同轴线接收芯片发送的同步曝光指令，并根据所述同步曝光指令进行曝光。
  22. 根据权利要求21所述的相机模组，还包括：
    与所述摄像头和所述相机传感器连接的串化器，用于将采集到的视频流包括的视频 数据通过所述同轴线发送给所述芯片。
  23. 根据权利要求21或22所述的相机模组，其特征在于，所述相机传感器还用于：
    通过所述同轴线接收所述芯片发送的关联控制指令，并
    基于所述关联控制指令控制执行所述关联控制指令对应的预定控制操作。
  24. 一种车辆的控制器，其特征在于，包括：
    人机交互模块，用于获取触发所述车辆的车载监控系统功能的触发指令；
    指令生成模块，用于基于所述触发指令，生成同步曝光指令；
    指令发送模块，用于将所述同步曝光指令发送给交换机，以使所述交换机发送所述同步曝光指令给至少两个芯片中的第一芯片，所述第一芯片通过硬线将所述同步曝光指令发送给至少两个所述芯片中的至少一个第二芯片。
PCT/CN2022/078297 2021-03-26 2022-02-28 视频采集设备、车辆、车舱检测及同步曝光方法 WO2022199330A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110328658.5 2021-03-26
CN202110328658.5A CN112954231A (zh) 2021-03-26 2021-03-26 视频采集设备、车辆、车舱检测及同步曝光方法

Publications (1)

Publication Number Publication Date
WO2022199330A1 true WO2022199330A1 (zh) 2022-09-29

Family

ID=76226987

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/078297 WO2022199330A1 (zh) 2021-03-26 2022-02-28 视频采集设备、车辆、车舱检测及同步曝光方法

Country Status (2)

Country Link
CN (1) CN112954231A (zh)
WO (1) WO2022199330A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115988331A (zh) * 2022-12-09 2023-04-18 浙江华锐捷技术有限公司 一种曝光控制方法、装置、设备及介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954231A (zh) * 2021-03-26 2021-06-11 上海商汤临港智能科技有限公司 视频采集设备、车辆、车舱检测及同步曝光方法
CN114158156B (zh) * 2022-02-10 2022-05-03 深圳佑驾创新科技有限公司 一种补光灯的pwm调节方法、装置、设备及存储介质
CN116723406A (zh) * 2022-02-28 2023-09-08 比亚迪股份有限公司 图像数据处理方法、装置、计算机设备及存储介质
CN114598797A (zh) * 2022-03-07 2022-06-07 合众新能源汽车有限公司 驾驶员监控系统与舱内监控系统共用补光灯的系统和方法
CN114827404B (zh) * 2022-04-07 2024-03-05 安徽蔚来智驾科技有限公司 车载图像采集系统、控制方法、车辆和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040017486A1 (en) * 2002-07-24 2004-01-29 Cooper Alan Neal Digital camera synchronization
CN101505434A (zh) * 2009-03-12 2009-08-12 浙江大学 具有全局同步功能的高分辨率智能网络摄像机阵列系统
CN110312056A (zh) * 2019-06-10 2019-10-08 青岛小鸟看看科技有限公司 一种同步曝光方法和图像采集设备
CN111832373A (zh) * 2019-05-28 2020-10-27 北京伟景智能科技有限公司 一种基于多目视觉的汽车驾驶姿态检测方法
CN112153306A (zh) * 2020-09-30 2020-12-29 深圳市商汤科技有限公司 图像采集系统、方法、装置、电子设备及可穿戴设备
CN112954231A (zh) * 2021-03-26 2021-06-11 上海商汤临港智能科技有限公司 视频采集设备、车辆、车舱检测及同步曝光方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN216057289U (zh) * 2021-03-26 2022-03-15 上海商汤临港智能科技有限公司 车舱影像数据采集系统及车辆

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040017486A1 (en) * 2002-07-24 2004-01-29 Cooper Alan Neal Digital camera synchronization
CN101505434A (zh) * 2009-03-12 2009-08-12 浙江大学 具有全局同步功能的高分辨率智能网络摄像机阵列系统
CN111832373A (zh) * 2019-05-28 2020-10-27 北京伟景智能科技有限公司 一种基于多目视觉的汽车驾驶姿态检测方法
CN110312056A (zh) * 2019-06-10 2019-10-08 青岛小鸟看看科技有限公司 一种同步曝光方法和图像采集设备
CN112153306A (zh) * 2020-09-30 2020-12-29 深圳市商汤科技有限公司 图像采集系统、方法、装置、电子设备及可穿戴设备
CN112954231A (zh) * 2021-03-26 2021-06-11 上海商汤临港智能科技有限公司 视频采集设备、车辆、车舱检测及同步曝光方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115988331A (zh) * 2022-12-09 2023-04-18 浙江华锐捷技术有限公司 一种曝光控制方法、装置、设备及介质
CN115988331B (zh) * 2022-12-09 2024-10-08 浙江华锐捷技术有限公司 一种曝光控制方法、装置、设备及介质

Also Published As

Publication number Publication date
CN112954231A (zh) 2021-06-11

Similar Documents

Publication Publication Date Title
WO2022199330A1 (zh) 视频采集设备、车辆、车舱检测及同步曝光方法
US10850693B1 (en) Determining comfort settings in vehicles using computer vision
US11713600B1 (en) Child safety lock
US11833966B2 (en) Switchable display during parking maneuvers
WO2019205642A1 (zh) 基于情绪识别的安抚方法、装置、系统、计算机设备以及计算机可读存储介质
US10345806B2 (en) Autonomous driving system and method for same
JPWO2018051809A1 (ja) 撮像装置、及び、電子機器
US11891002B1 (en) Determining comfort settings in vehicles using computer vision
WO2021139749A1 (zh) 用于采集图像的方法、电子设备、计算机存储介质和车辆
CN113542529B (zh) 用于dms和oms的940nm led闪光同步
US11119633B2 (en) Information processing device and method
CN111762184B (zh) 车辆控制系统以及车辆
WO2022170804A1 (zh) 一种控制方法及装置
US20200282909A1 (en) Vehicle imaging system and method for a parking solution
CN110070058A (zh) 一种车载手势识别装置及系统
CN216057289U (zh) 车舱影像数据采集系统及车辆
WO2023221998A1 (zh) 车载流媒体显示系统和方法
WO2020137398A1 (ja) 操作制御装置、撮像装置、操作制御方法
US20230093198A1 (en) Vehicle video conferencing system
KR101816570B1 (ko) 차량용 디스플레이 장치
CN113561988A (zh) 一种基于视线追踪的语音控制方法、汽车及可读存储介质
WO2018142969A1 (ja) 表示制御装置および方法、並びにプログラム
CN217562054U (zh) 一种基于体态手势识别拍照的汽车
CN114449167B (zh) 基于体态识别的汽车拍照方法、系统、车辆及存储介质
CN218383692U (zh) 一种投影式中控系统和车辆

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22773992

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27/02/2024)