CN113163171A

CN113163171A - Image processing system, intelligent cabin system, vehicle and image processing method

Info

Publication number: CN113163171A
Application number: CN202110445961.3A
Authority: CN
Inventors: 胡旦
Original assignee: Qianhai Qijian Technology Shenzhen Co ltd
Current assignee: Qianhai Qijian Technology Shenzhen Co ltd
Priority date: 2021-04-23
Filing date: 2021-04-23
Publication date: 2021-07-23

Abstract

The application provides an image processing system, an intelligent cabin system, a vehicle and an image processing method, wherein the image processing system comprises: the system-on-chip comprises a system-on-chip, a camera serial interface and a camera module, wherein the system-on-chip is provided with the camera serial interface; the acquisition equipment is connected with the camera serial interface and is used for acquiring the data of the surrounding environment of the automobile; a first operating system and a second operating system running on the system-on-chip, wherein the time length required for starting the first operating system is less than the time length required for starting the second operating system; the first operating system is used for acquiring and processing images acquired by the acquisition equipment so as to provide a first type of function for the automobile, wherein the first type of function comprises a panoramic image function; the second operating system is used for acquiring and processing the images acquired by the acquisition equipment to provide a second type of functions for the automobile, and the second type of functions comprise a hard disk video recording function. The difficulty of realizing each function in the intelligent cabin can be reduced.

Description

Image processing system, intelligent cabin system, vehicle and image processing method

Technical Field

The application relates to the technical field of intelligent cabins of automobiles, in particular to an image processing system, an intelligent cabin system, a vehicle and an image processing method.

Background

In the technology of intelligent cabins of automobiles, functions in the cabins are becoming more and more abundant, and for example, functions such as panoramic View Monitor (AVM), Digital Video Recorders (DVR), and the like are commonly used in automobiles.

In the current functional application, a common implementation manner is as shown in fig. 1, wherein in the panoramic image function, the navigation end is implemented by a System On Chip (SOC) and a software operating System (in the example shown in fig. 1, the software operating System is an android operating System), the panoramic image box end needs another SOC and software operating System, the panoramic image box end is responsible for image capture and image processing, after the image is collected by the camera, the image is processed by the image deserializing Chip and then transmitted to the System on Chip, and then the image is output to an image deserializing chip through an interface, the image deserializing chip is connected with an image deserializing chip at the navigation end through a link, a deserializing image signal of the image deserializing chip at the navigation end is input to a system level chip at the navigation end through a serial interface, and the system level chip at the navigation end is responsible for outputting and displaying the image on a display screen. The driving recording function also adopts a similar implementation mode to the panoramic image function. The implementation method needs a relatively large number of hardware structures and is relatively complex to implement.

Disclosure of Invention

The application aims to provide an image processing system, an intelligent cabin system, a vehicle and an image processing method, and can solve the problem that the difficulty in realizing hardware structures of all functions in the existing intelligent cabin is high.

In a first aspect, the present invention provides an image processing system comprising:

the system-on-chip comprises a system-on-chip, a camera serial interface and a camera module, wherein the system-on-chip is provided with the camera serial interface;

the acquisition equipment is connected with the camera serial interface and used for acquiring the data of the surrounding environment of the automobile;

a first operating system and a second operating system running on the system-on-chip, wherein the time length required for starting the first operating system is less than the time length required for starting the second operating system;

the first operating system is used for acquiring and processing images acquired by the acquisition equipment so as to provide a first type of function for the automobile, wherein the first type of function comprises a panoramic image function;

the second operating system is used for acquiring and processing the images acquired by the acquisition equipment to provide a second type of functions for the automobile, wherein the second type of functions comprise a hard disk video recording function.

In an alternative embodiment, the method further comprises:

and the deserializing chip is connected between the camera serial interface and the acquisition equipment.

In an alternative embodiment, a plurality of camera serial interfaces are arranged on the system level chip;

the number of the deserializing chips is the same as that of the camera serial interfaces;

each deserializing chip is connected to one or more acquisition devices.

In the above embodiment, a plurality of camera serial interfaces are provided to connect different capturing devices for different functions.

In an alternative embodiment, the plurality of camera serial interfaces comprises: a first camera serial interface and a third camera serial interface;

a plurality of the deserializing chips comprise: a first deserializing chip and a third deserializing chip;

the first deserializing chip is connected with the first camera serial interface and is used for being connected with one or more first acquisition devices, and the first acquisition devices are used for acquiring images required by the hard disk video recording function;

and the third deserializing chip is connected with the serial interface of the third camera and is used for being connected with one or more third acquisition devices, and the third acquisition devices are used for acquiring images required by the panoramic image function.

In an optional embodiment, a spare link channel of the input acquisition device is provided on the first deserializing chip, and is used for connecting the spare acquisition device.

In an optional embodiment, the second type of function further includes: a driver state monitoring function, an in-vehicle monitoring function and a night vision function; the plurality of camera serial interfaces include: a second camera serial interface and a fourth camera serial interface;

a plurality of the deserializing chips comprise: a second deserializing chip and a fourth deserializing chip;

the second deserializing chip is connected with the second camera serial interface and is used for being connected with one or more second acquisition devices, and the second acquisition devices are used for acquiring images required by a driver state monitoring function and images required by an in-vehicle monitoring function;

and the fourth deserializing chip is connected with the serial interface of the fourth camera and is used for being connected with one or more fourth acquisition devices, and the fourth acquisition devices are used for acquiring images required by the night vision function.

In the above embodiment, different camera serial interfaces and deserializing chips are used for connecting and acquiring image data with different requirements, so that image data with better applicability can be provided for different functions, and the accuracy of realizing the functions can be improved.

In an optional embodiment, a spare link channel of the input acquisition device is provided on the second deserializing chip, and is used for connecting the spare acquisition device.

In the above embodiment, by adding the connection channel of the standby acquisition device, when the function of camera input needs to be added, the camera input can be added through the connection channel, so that the camera input can be realized without changing the structure of the whole system, and the expansibility of the image processing system is better.

In a second aspect, the present invention provides an intelligent cockpit system comprising:

the image processing system of any of the preceding embodiments;

and the display screen is connected with the system-on-chip and used for displaying the processing result of the first operating system or the second operating system.

In an alternative embodiment, the display screen comprises: one or more of a dashboard screen, a central control screen, an entertainment navigation screen, and a secondary driver screen.

In a third aspect, the present invention provides a vehicle comprising: the intelligent cabin system of the previous embodiment.

In a fourth aspect, the present invention provides an image processing method applied to the image processing system described in any one of the foregoing embodiments, the method including:

acquiring environmental data of a designated area through the acquisition equipment;

processing the environmental data using the first operating system or the second operating system.

The beneficial effects of the embodiment of the application are that: the panoramic image and the driving recording function of the intelligent cabin do not need to be additionally provided with the panoramic image box and the driving recording module, only one system level chip can be used, a peripheral circuit connecting circuit of the panoramic image box and the driving recording module is omitted, interactive hardware communication between each external module and a navigation end system is omitted, and the difficulty of realizing each function in the intelligent cabin is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a schematic block diagram of a prior art intelligent cabin.

Fig. 2 is a block diagram of an image processing system according to an embodiment of the present application.

Fig. 3 is a block schematic diagram of an intelligent cabin system provided in an embodiment of the present application.

Fig. 4 is a flowchart of an image processing method according to an embodiment of the present application.

Description of the main element symbols: 110-system on a chip; 111-a first operating system; 112-a second operating system; 121-a first acquisition device; 122-a second acquisition device; 123-a third acquisition device; 124-a fourth acquisition device; 131-a first deserializing chip; 132-a second deserializing chip; 133-a third deserializing chip; 134-fourth deserializer chip; 210-instrument panel; 220-a central control screen; 230-a navigation screen; 240-copilot screen.

Detailed Description

The technical solution in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.

Example one

An embodiment of the present application provides an image processing system, as shown in fig. 2, the image processing system in the embodiment includes: a system-on-chip 110 and an acquisition device.

The image processing system in this embodiment may be installed in a vehicle for implementing some of the image processing functions required for the vehicle.

In this embodiment, the system on chip 110 is provided with a camera serial interface.

Alternatively, the Camera Serial Interface may be an MIPI CSI (Mobile Industry Processor Camera Serial Interface) Interface.

The acquisition equipment is connected with the camera serial interface and used for acquiring the data of the surrounding environment of the automobile.

The acquisition device is used for acquiring environmental data around or in the automobile.

Alternatively, the acquisition device may be an image acquisition device, such as a camera.

In this embodiment, the system on chip 110 may have two first operating systems 111 and two second operating systems 112 running thereon.

Wherein the time length required for starting the first operating system 111 is less than the time length required for starting the second operating system 112.

Illustratively, the first operating system 111 may be a QNX Hypervisor system, and the second operating system 112 may be an Android operating system. The time length required by starting the QNX Hypervisor system is shorter than that of an Android operating system. In some examples, the QNX Hypervisor operating system may be booted within two seconds, and the Android operating system may require more than ten seconds to boot.

Alternatively, the second operating system 112 may run on a virtual machine.

Since the time required for starting the first operating system 111 is short, some time-limited first-class functions can be implemented by the first operating system 111. Such as a panoramic image function.

Since the time required for booting the second operating system 112 is relatively long, some second-class functions with low time requirements can be implemented by the second operating system 112. Such as a drive recording function.

Referring again to fig. 2, the image processing system may further include: and (5) deserializing the chip.

The deserializing chip is connected between the camera serial interface and the acquisition equipment.

Illustratively, the deserialized chip connected acquisition device may be connected to the acquisition device via an FPD-LINK.

Alternatively, the number of deserializing chips may be matched according to the number of camera serial interfaces provided on the system-on-chip 110.

In one embodiment, it is desirable to provide a vehicle with a number of functions that require the use of image recognition. A plurality of camera serial interfaces may be provided on the system-on-chip 110.

In this embodiment, each camera serial interface may be connected to a deserializing chip.

Each deserializer chip may be connected to one or more acquisition device connections based on different needs. In the example shown in fig. 2, multiple acquisition devices are connected to each deserializer chip.

In one embodiment, the functions to be implemented by the vehicle equipped with the image processing system include a driving recording function and a panoramic image function. The system-on-chip 110 of the image processing system may then be provided with a first camera serial interface and a third camera serial interface.

The image processing system may include a plurality of said deserializing chips including: a first deserializer chip 131 connected to the first camera serial interface and a third deserializer chip 133 connected to the third camera serial interface.

The first deserializing chip 131 is used for connecting with one or more first acquisition devices 121, and the first acquisition devices 121 are used for acquiring images required by the driving recording function.

Optionally, the first deserializing chip 131 may be connected to at least two first collecting devices 121, wherein one first collecting device 121 is used for collecting image data in front of the vehicle, and the other first collecting device 121 is used for collecting image data behind the vehicle. Of course, more first capture devices 121 may be provided to capture image data of various directions of the vehicle based on different needs.

In the example shown in fig. 2 or 3, the image processing system may comprise three first acquisition devices 121, respectively acquisition device a, acquisition device b and acquisition device c.

In this embodiment, the image acquired by the first acquisition device 121 may be processed by the second operating system 112 after being processed by the first deserializer chip 131.

For example, an application program for processing the image captured by the first capturing device 121 may be used in the second operating system 112. For example, the image captured by the first capture device 121 may be used as a driving record for recording events occurring around the vehicle. The application program may store the image acquired by the first acquisition device 121 in a storage space built in the vehicle, or an external Secure Digital Memory Card (SD).

Optionally, the first deserializing chip 131 may also include a spare link channel for the input camera. In some optional embodiments, when it is necessary to add another function based on image acquisition, a standby acquisition device may be connected to the idle link channel of the input camera.

For example, after the first capturing device 121 captures the image, the image may be transmitted to the second operating system 112 via the first deserializing chip 131 for processing.

The third deserializing chip 133 is configured to be connected to one or more third capturing devices 123, where the third capturing devices 123 are configured to capture images required by the panoramic image function.

In the example shown in fig. 2 or 3, the image processing system may comprise four third acquisition devices 123, denoted acquisition device g, acquisition device h, acquisition device i and acquisition device j, respectively.

Optionally, the third deserializing chip 133 may be connected to a plurality of third acquisition devices 123. Illustratively, the third capturing device 123 captures image data of each orientation of the vehicle to stitch the images of each orientation to obtain a panoramic image of the surroundings of the vehicle.

Alternatively, the third capturing device 123 may be a fisheye camera.

To facilitate the acquisition of image data of various orientations of the vehicle, the third acquisition device 123 may be installed at various orientations of the vehicle. For example, the respective third collecting apparatuses 123 may be installed at four corners of the housing of the vehicle.

For example, after the images are acquired by the third acquisition devices 123 and processed by the third deserializing chip 133, the obtained images are spliced by the first operating system 111 to obtain a panoramic image around the vehicle.

Optionally, an application program for implementing the panoramic image function may be run on the first operating system 111. The application program may perform stitching processing on the images acquired by the third acquisition device 123 to obtain a panoramic image around the vehicle. Illustratively, the application program implementing the panoramic image function may also perform distortion processing, correction processing, and the like on the image captured by the third capture device 123. The spliced panoramic image around the vehicle can be more real through distortion processing and correction processing, so that the condition around the vehicle can be better reflected.

In this embodiment, the second function further includes: a driver state monitoring function, an in-vehicle monitoring function and a night vision function. The plurality of camera serial interfaces include: a second camera serial interface and a fourth camera serial interface.

The image processing system may further comprise a second deserializer chip 132 connected to the second camera serial interface and a fourth deserializer chip 134 connected to the fourth camera serial interface.

The second deserializing chip 132 is configured to be connected to one or more second collecting devices 122, and the second collecting devices 122 are configured to collect images required by the driver state monitoring function and images required by the in-vehicle monitoring function.

In the example shown in fig. 2 or 3, the image processing system may comprise three second acquisition devices 122, respectively acquisition device d, acquisition device e and acquisition device f.

In this embodiment, the second collecting device 122 is used for collecting data inside the vehicle, and the second collecting device 122 may be installed inside the vehicle.

For example, the second capturing device 122 for assisting the realization of the driver state monitoring function may be installed at a position where the face image of the driver can be captured. For example, the second capture device 122 for assisting in implementing the driver state monitoring function may be mounted on the upper edge of the windshield of the vehicle.

For example, the second collecting apparatus 122 for assisting the in-vehicle monitoring function may be installed at a position where more angles in the vehicle can be collected. The second collection device 122 for assisting in-vehicle monitoring functions may be mounted on top of the vehicle interior.

In this embodiment, an application program for recognizing the internal environment of the vehicle may be run in the second operating system 112. The application program is used to identify the images captured by the second capture device 122 to determine whether a potential risk factor exists for the current vehicle. For example, a facial image of the driver may be identified by the application to determine whether dangerous driving is present. For example, if the driver is absent in the vehicle, the application program can recognize that an infant or the like is present.

The fourth deserializer chip 134 is configured to be coupled to one or more fourth capture devices 124, where the fourth capture devices 124 are configured to capture images for night vision functionality.

In the example shown in fig. 2 or 3, the image processing system may comprise four third acquisition devices 123, denoted acquisition device k, acquisition device l, acquisition device m and acquisition device n, respectively.

Optionally, the second function may further include gesture recognition. An image capture device for implementing gesture recognition may also be included in the fourth capture device 124.

Optionally, the second type of function may further include blind spot detection. An image capturing device for detecting blind areas may also be included in the fourth capturing device 124.

Optionally, an application program for image recognition required for implementing night vision function may also be run in the second operating system 112. The application may enable the nighttime images captured by the fourth capture device 124 to be identified to determine whether a driving hazard exists in the currently identified nighttime environment.

Optionally, an application program for image recognition required for implementing the gesture recognition function may also be run in the second operating system 112. The application may implement the fourth capturing device 124 to capture the image of the designated area for recognition to determine whether there is a traffic guidance gesture in the currently captured designated area.

Optionally, an application program for image recognition required for implementing the blind spot detection function may also be run in the second operating system 112. The application program may identify the blind area image collected by the fourth collecting device 124 to determine whether a driving hazard exists in the currently identified blind area.

Alternatively, the image recognition required for implementing the night vision function, the image recognition required for the gesture recognition function, and the image recognition required for the blind spot detection function may be implemented by three different applications, or may be integrated in one application, or may integrate any two kinds of recognition in one application, and implement the other recognition by another application.

In this embodiment, the fourth deserializing chip 134 may be connected to a plurality of fourth collecting devices 124, wherein one fourth collecting device 124 is used for collecting images at night, another fourth collecting device 124 is used for collecting images in a designated area of the vehicle, and another or two fourth collecting devices 124 are used for collecting images of blind areas of the vehicle.

Optionally, the second deserializing chip 132 may also include an idle link channel for the input camera. In some optional embodiments, when it is necessary to add another function based on image acquisition, the camera may be connected to the idle input camera in the link channel.

Example two

Please refer to fig. 3, which is a block diagram of an intelligent cabin system according to an embodiment of the present application. The intelligent cabin system in this embodiment may include: an image processing system and a display screen.

The display screen may be connected to the system on chip 110 of the image processing system, and is used for displaying the processing result of the first operating system 111 or the second operating system 112.

The image processing system provided in this embodiment is similar to the image processing system provided in the first embodiment, and other details of the image processing system in this embodiment may refer to the description of the image processing system in the first embodiment, and are not described herein again.

Optionally, the display screen may include: one or more of instrument panel 210, center control panel 220, entertainment navigation panel 230, and copilot panel 240 (the smart cockpit system shown in fig. 3 shows four display panels).

In this embodiment, the panoramic image obtained after the processing by the first operating system 111 may be displayed on the instrument panel 210 or the central control panel 220.

In one embodiment, the images processed by the second operating system 112 may be displayed on the navigation screen 230 or the secondary screen 240.

EXAMPLE III

The embodiment of the application also provides a vehicle. The vehicle in the embodiment may include: intelligent cabin system.

The intelligent cabin system provided in this embodiment is similar to the intelligent cabin system provided in the second embodiment, and other details of the intelligent cabin system in this embodiment may refer to the description of the intelligent cabin system in the second embodiment, and are not described herein again.

The vehicle provided by the embodiment can be a four-wheel automobile or a two-wheel automobile.

The image processing system, intelligence passenger cabin system, vehicle that this application embodiment provided, realize that the panoramic image of intelligence passenger cabin and driving recording function need not additionally add panoramic image box and driving recording module, can only use a system level chip 110, saved the peripheral circuit interconnecting link of panoramic image box and driving recording module, saved interactive hardware communication between each external module and navigation end system, reduce the degree of difficulty that each function in the intelligence passenger cabin realized.

Furthermore, as the AVM & DMS box and the DVR module are not required to be additionally added, the hardware costs of the SOC, the peripheral circuit, the connecting line, the host structure and the like of the AVM & DMS box and the DVR module are saved.

Further, for a function which needs to be processed quickly, the first operating system 111 is adopted, and the starting speed of the first operating system 111 is high, so that the processing efficiency of the function which has a requirement on time limit can be improved, and the requirement on quick processing is met, meanwhile, the first operating system 111 and the second operating system 112 can control each path of camera shooting input, and thus, an input image can be guaranteed to be displayed in any display screen. Therefore, the intelligent cabin system can realize the functions of night vision, gesture recognition, in-vehicle monitoring, blind area detection and the like which need camera shooting assistance besides the functions of panoramic images, driver state monitoring systems and driving records.

Example four

The embodiment of the present application provides an image processing method, and the image processing method in the embodiment is applied to the image processing system provided in the first embodiment.

Exemplarily, as shown in fig. 4, the method in the present embodiment may include the following steps.

And step 302, collecting environmental data of the designated area through the collecting device.

The environmental data collected by the collecting device can be pictures or videos.

In this embodiment, the acquired image may be stored in the buffer area.

Step 304, processing the environment data by using the first operating system or the second operating system.

In this embodiment, after the first operating system or the second operating system processes the environment data, the processed data may be sent to a display screen for display.

Optionally, before executing each step in the image processing method provided by the embodiment of the present application, the method may further include: step 301, an image processing system is initialized.

Illustratively, initialization processing may be performed for each camera serial interface, each acquisition device, and an application program running in the first operating system and the second operating system.

Optionally, the device information of each connected acquisition device may be bound with application programs in the first operating system and the second operating system, so that each application program processes an image acquired by the corresponding acquisition device.

And configuring corresponding application programs for each acquisition device through initialization. Different applications may be used to process images acquired by different acquisition devices.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing system, characterized in that the image processing system comprises:

the second operating system is used for acquiring images acquired by the acquisition equipment and processing the images to provide a second function for the automobile, wherein the second function comprises a driving recording function.

2. The image processing system according to claim 1, further comprising:

3. The image processing system of claim 2, wherein a plurality of camera serial interfaces are provided on the system-on-chip;

each deserializing chip is connected to one or more acquisition devices.

4. The image processing system of claim 2, wherein the camera serial interface comprises: a first camera serial interface and a third camera serial interface;

the first deserializing chip is connected with the first camera serial interface and is used for being connected with one or more first acquisition devices, and the first acquisition devices are used for acquiring images required by the driving recording function;

5. The image processing system of claim 4, wherein a link channel of a standby input acquisition device is disposed on the first deserializing chip for connecting with a standby acquisition device.

6. The image processing system of claim 2, wherein the second class of functions further comprises: a driver state monitoring function, an in-vehicle monitoring function and a night vision function; the camera serial interface includes: a second camera serial interface and a fourth camera serial interface;

7. An intelligent cabin system, comprising:

the image processing system of any one of claims 1-6;

8. The intelligent cabin system of claim 7, wherein the display screen comprises: one or more of a dashboard screen, a central control screen, an entertainment navigation screen, and a secondary driver screen.

9. A vehicle, characterized by comprising: the intelligent cabin system of claim 8.

10. An image processing method applied to the image processing system according to any one of claims 1 to 7, the method comprising: