CN110780934B

CN110780934B - Deployment method and device of vehicle-mounted image processing system

Info

Publication number: CN110780934B
Application number: CN201911013808.2A
Authority: CN
Inventors: 曾彬
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2019-10-23
Filing date: 2019-10-23
Publication date: 2024-03-12
Anticipated expiration: 2039-10-23
Also published as: CN110780934A

Abstract

The embodiment of the disclosure provides a deployment method and device of a vehicle-mounted image processing system, wherein the method comprises the following steps: acquiring system attributes of a vehicle-mounted operation system of a vehicle to be deployed with a vehicle-mounted image processing system; acquiring mapping relations between system attributes of different vehicle-mounted operating systems and acquisition paths of different camera drivers; determining an acquisition path of a camera driver corresponding to the system attribute of the vehicle according to the mapping relation; invoking a camera driver of the vehicle-mounted camera stored in the vehicle-mounted operating system according to an acquisition path of the camera driver in a state that the vehicle-mounted camera is connected with the vehicle; and establishing the interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through the camera drive corresponding to the vehicle-mounted camera.

Description

Deployment method and device of vehicle-mounted image processing system

Technical Field

The present disclosure relates to image processing technologies, and in particular, to a deployment method and apparatus for a vehicle-mounted image processing system.

Background

Product intellectualization, electronization and interconnection are continuously developed and popularized, and a plurality of more and more intelligent man-machine interaction modes are developed to meet the demands of people for pursuing individuation and fashion. For example, control of the functions of the device may be performed based on images captured by the camera. When a certain vehicle-mounted image processing system is to perform equipment function control based on images acquired by a camera, communication interaction connection is required to be established between the vehicle-mounted image processing system and the camera, the vehicle-mounted image processing system can acquire the images acquired by the camera, and further control is performed according to the images.

Disclosure of Invention

In view of this, the embodiments of the present disclosure at least provide a deployment method and apparatus for a vehicle-mounted image processing system.

In a first aspect, there is provided a deployment method of an in-vehicle image processing system, the method including:

acquiring system attributes of a vehicle-mounted operation system of a vehicle to be deployed with a vehicle-mounted image processing system;

acquiring mapping relations between system attributes of different vehicle-mounted operating systems and acquisition paths of different camera drivers;

determining an acquisition path of a camera driver corresponding to the system attribute of the vehicle according to the mapping relation;

invoking a camera driver of the vehicle-mounted camera stored in the vehicle-mounted operating system according to an acquisition path of the camera driver in a state that the vehicle-mounted camera is connected with the vehicle;

and establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through the camera drive corresponding to the vehicle-mounted camera.

In combination with any one of the embodiments of the present disclosure, the invoking the camera driver of the vehicle-mounted camera stored in the vehicle-mounted operating system according to the acquisition path of the camera driver includes: calling the camera driver according to a plurality of calling interfaces arranged in the acquisition path of the camera driver; the call interface includes at least one of: the camera management system comprises a camera parameter setting interface for setting camera parameters of the camera, an image setting interface for setting parameters of images shot by the camera and a camera management interface for managing the camera.

In combination with any of the embodiments of the present disclosure, the different camera-driven acquisition paths include: and the calling interfaces are at least partially different and are used for operating the camera.

In combination with any one of the embodiments of the present disclosure, the invoking the camera driver of the vehicle-mounted camera stored in the vehicle-mounted operating system according to the acquisition path of the camera driver includes: and calling the camera driver stored in the Linux layer of the Android system architecture according to a calling mode set in the acquisition path based on the Android system architecture of the vehicle-mounted operating system.

In combination with any one of the embodiments of the present disclosure, after the acquiring the system attribute of the in-vehicle operating system of the vehicle in which the in-vehicle image processing system is to be deployed, the method further includes: a drive loading module of a third party tool library stored in the vehicle-mounted operating system is called, and a camera driver of the vehicle-mounted camera is loaded through the drive loading module, wherein an access path for accessing the camera driver in the vehicle-mounted operating system is provided in the drive loading module, and the access path provides a call interface for directly accessing the camera driver of a Linux layer by the vehicle-mounted image processing system without passing through an intermediate layer of an Android operating system architecture; loading a camera driver corresponding to the vehicle-mounted camera according to the access path; and establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through camera driving corresponding to the vehicle-mounted camera.

In combination with any one of the embodiments of the present disclosure, the vehicle-mounted camera is a USB camera connected to the vehicle through a USB interface; before the calling the driver loading module of the third party tool library stored in the vehicle-mounted operating system, the method further comprises: and determining to use a USB camera connected with the vehicle through a USB interface according to the system attribute.

In combination with any one of the embodiments of the present disclosure, the determining, according to the system attribute, to use a USB camera connected to the vehicle through a USB interface includes: according to the system attribute, if the use priority of the USB cameras connected with the USB interface is higher than that of the cameras connected with the non-USB interface, the USB cameras are used; or if the feedback information of call failure is received when the camera drive of the vehicle-mounted camera connected with the vehicle through the non-USB interface is called, determining to use the USB camera.

In combination with any one of the embodiments of the present disclosure, the determining, according to the mapping relationship, an acquisition path of a camera driver corresponding to a system attribute of the vehicle includes: acquiring at least two acquisition paths of the camera driver corresponding to the system attribute according to the mapping relation; and determining one acquisition path from the at least two acquisition paths as a target acquisition path to be used.

In combination with any one of the embodiments of the present disclosure, after the establishing of the interaction among the vehicle-mounted image processing system, the vehicle-mounted camera, and the vehicle-mounted operating system through the camera driver corresponding to the vehicle-mounted camera, the method further includes: the vehicle-mounted image processing system receives gesture images acquired by the camera; the vehicle-mounted image processing system performs gesture recognition processing on the gesture image to obtain a gesture recognition result; and the vehicle-mounted image processing system transmits a control instruction corresponding to the gesture recognition result to a controlled module in the vehicle through the vehicle-mounted operating system, so that the controlled module executes an operation corresponding to the control instruction.

In combination with any one of the embodiments of the present disclosure, after the establishing of the interaction among the vehicle-mounted image processing system, the vehicle-mounted camera, and the vehicle-mounted operating system through the camera driver corresponding to the vehicle-mounted camera, the method further includes: the vehicle-mounted image processing system receives the face image of the driver acquired by the camera; the vehicle-mounted image processing system performs image recognition processing on the face image of the driver to obtain fatigue driving information; and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the fatigue driving information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, after the establishing of the interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera, and the vehicle-mounted operating system through the camera driver corresponding to the vehicle-mounted camera, the method further includes: the vehicle-mounted image processing system receives the face image of the driver acquired by the camera; the vehicle-mounted image processing system performs image recognition processing on the face image of the driver to obtain distracted driving information; and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the distraction driving information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, after the establishing of the interaction among the vehicle-mounted image processing system, the vehicle-mounted camera, and the vehicle-mounted operating system through the camera driver corresponding to the vehicle-mounted camera, the method further includes: the vehicle-mounted image processing system receives the driver action image acquired by the camera; the vehicle-mounted image processing system performs image recognition processing on the driver action image to obtain dangerous action information; and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the dangerous action information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, after the establishing of the interaction among the vehicle-mounted image processing system, the vehicle-mounted camera, and the vehicle-mounted operating system through the camera driver corresponding to the vehicle-mounted camera, the method further includes: the vehicle-mounted image processing system receives the face image of the driver acquired by the camera; the vehicle-mounted image processing system performs image processing on the face image of the driver acquired by the camera to obtain regional fixation information; and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the regional fixation information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, the vehicle-mounted response module is a vehicle-mounted display module, and the sending, by the vehicle-mounted operation system, of prompt indication information to the vehicle-mounted response module, so that the vehicle-mounted response module performs a prompt operation according to the prompt indication information, including: and sending display parameter adjustment information or display content adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system so that the vehicle-mounted display module carries out prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, the sending, by the vehicle-mounted operating system, display parameter adjustment information to the vehicle-mounted display module, so that the vehicle-mounted display module performs a prompting operation according to the prompting indication information, including: and sending interface brightness adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module adjusts the brightness of the image display interface according to the interface brightness adjustment information.

In combination with any one of the embodiments of the present disclosure, the sending, by the vehicle-mounted operating system, display content adjustment information to the vehicle-mounted display module, so that the vehicle-mounted display module performs a prompting operation according to the prompting indication information, including: and sending digital person display control information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module displays the digital person on a vehicle-mounted image display interface according to the digital person display control information, or controls the digital person to output interactive feedback information.

In a second aspect, there is provided a deployment apparatus of an in-vehicle image processing system, the apparatus comprising:

the attribute acquisition module is used for acquiring system attributes of a vehicle-mounted operation system of a vehicle to which the vehicle-mounted image processing system is to be deployed;

The mapping acquisition module is used for acquiring mapping relations between system attributes of different vehicle-mounted operating systems and acquisition paths of different camera drivers;

the path determining module is used for determining a camera driving acquisition path corresponding to the system attribute of the vehicle according to the mapping relation;

the drive loading module is used for calling the camera drive of the vehicle-mounted camera stored in the vehicle-mounted operating system according to the acquisition path of the camera drive in a state that the vehicle-mounted camera is connected with the vehicle;

the connection establishment module is used for establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through the camera drive corresponding to the vehicle-mounted camera.

In combination with any one of the embodiments of the present disclosure, the driving loading module is specifically configured to: calling the camera driver according to a plurality of calling interfaces arranged in the acquisition path of the camera driver; the call interface includes at least one of: the camera management system comprises a camera parameter setting interface for setting camera parameters of the camera, an image setting interface for setting parameters of images shot by the camera and a camera management interface for managing the camera.

In combination with any one of the embodiments of the present disclosure, the driving loading module is specifically configured to: and calling the camera driver stored in the Linux layer of the Android system architecture according to a calling mode set in the acquisition path based on the Android system architecture of the vehicle-mounted operating system.

In combination with any one of the embodiments of the present disclosure, the path determining module is further configured to invoke a driver loading module of a third party tool library stored in the vehicle-mounted operating system, and load a camera driver of the vehicle-mounted camera through the driver loading module, where an access path for accessing the camera driver in the vehicle-mounted operating system is provided in the driver loading module, and the access path provides a call interface for the vehicle-mounted image processing system to directly access a camera driver of a Linux layer without passing through an intermediate layer of an Android operating system architecture; the drive loading module is used for loading the camera drive corresponding to the vehicle-mounted camera according to the access path; the connection establishment module is used for establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through camera driving corresponding to the vehicle-mounted camera.

In combination with any one of the embodiments of the present disclosure, the path determining module is further configured to determine, before the invoking the driver loading module of the third party tool library stored in the vehicle-mounted operating system, to use a USB camera connected to the vehicle through a USB interface according to the system attribute.

In combination with any one of the embodiments of the present disclosure, the path determining module, when configured to determine, according to the system attribute, to use a USB camera connected to the vehicle through a USB interface, includes: according to the system attribute, if the use priority of the USB cameras connected with the USB interface is higher than that of the cameras connected with the non-USB interface, the USB cameras are used; or if the feedback information of call failure is received when the camera drive of the vehicle-mounted camera connected with the vehicle through the non-USB interface is called, determining to use the USB camera.

In combination with any one of the embodiments of the present disclosure, the path determining module, when configured to determine, according to the mapping relationship, an acquisition path of a camera driver corresponding to a system attribute of the vehicle, includes: acquiring at least two acquisition paths of the camera driver corresponding to the system attribute according to the mapping relation; and determining one acquisition path from the at least two acquisition paths as a target acquisition path to be used.

In combination with any of the embodiments of the present disclosure, the apparatus further comprises: the image processing module is used for receiving the gesture image acquired by the camera; performing gesture recognition processing on the gesture image to obtain a gesture recognition result; and transmitting a control instruction corresponding to the gesture recognition result to a controlled module in the vehicle through the vehicle-mounted operating system, so that the controlled module executes an operation corresponding to the control instruction.

With reference to any one of the embodiments of the present disclosure, the image processing module is further configured to receive a face image of the driver acquired by the camera; performing image recognition processing on the face image of the driver to obtain fatigue driving information; based on the fatigue driving information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

With reference to any one of the embodiments of the present disclosure, the image processing module is configured to receive a face image of a driver acquired by the camera; performing image recognition processing on the face image of the driver to obtain distracted driving information; based on the distraction driving information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

With reference to any one of the embodiments of the present disclosure, the image processing module is configured to receive a driver action image acquired by the camera; performing image recognition processing on the driver action image to obtain dangerous action information; based on the dangerous action information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

With reference to any one of the embodiments of the present disclosure, the image processing module is configured to receive a face image of a driver acquired by the camera; performing image processing on the face image of the driver acquired by the camera to obtain regional fixation information; and based on the regional fixation information, sending prompt indication information to a vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, the image processing module is specifically configured to send, by using the vehicle-mounted operating system, display parameter adjustment information or display content adjustment information to a vehicle-mounted display module, so that the vehicle-mounted display module performs a prompt operation according to the prompt indication information.

In combination with any one of the embodiments of the present disclosure, the image processing module is specifically configured to: and sending interface brightness adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module adjusts the brightness of an image display interface according to the interface brightness adjustment information.

In combination with any one of the embodiments of the present disclosure, the image processing module is specifically configured to: and sending digital person display control information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module displays the digital person on a vehicle-mounted image display interface according to the digital person display control information, or controls the digital person to output interactive feedback information.

In a third aspect, an electronic device is provided, the device comprising a memory for storing computer instructions executable on the processor for implementing the method of any of the embodiments of the disclosure when the computer instructions are executed.

In a fourth aspect, a computer readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements a method according to any of the embodiments of the present disclosure.

According to the deployment method and device for the vehicle-mounted image processing system, the paths matched with the current attribute are acquired from the acquisition paths driven by the cameras according to the system attribute, so that the camera driving of the vehicle-mounted camera can be accessed according to the paths, the vehicle-mounted image processing system can be deployed as soon as possible, and the deployment efficiency of the vehicle-mounted image processing system is improved.

Drawings

In order to more clearly illustrate the technical solutions of one or more embodiments of the present disclosure or related technologies, the following description will briefly describe the drawings that are required to be used in the embodiments or related technology descriptions, and it is apparent that the drawings in the following description are only some embodiments described in one or more embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.

FIG. 1 illustrates a flow chart of a deployment method of an in-vehicle image processing system provided by at least one embodiment of the present disclosure;

FIG. 2 illustrates a flow chart of a deployment method of an in-vehicle image processing system provided by at least one embodiment of the present disclosure;

FIG. 3 illustrates an illustration of acquiring an image of a face of a driver provided in at least one embodiment of the present disclosure;

FIG. 4 illustrates an illustration of one type of gathering driver actions provided by at least one embodiment of the present disclosure;

FIG. 5 illustrates a block diagram of a deployment apparatus of an in-vehicle image processing system provided by at least one embodiment of the present disclosure;

FIG. 6 illustrates a block diagram of a deployment apparatus of another in-vehicle image processing system provided by at least one embodiment of the present disclosure;

fig. 7 illustrates an electronic device provided by at least one embodiment of the present disclosure.

Detailed Description

In order that those skilled in the art will better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in one or more embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, not all embodiments. All other embodiments, which may be made by one of ordinary skill in the art based on one or more embodiments of the present disclosure without inventive faculty, are intended to be within the scope of the present disclosure.

Fig. 1 provides a deployment method of an in-vehicle image processing system, which can be executed by a deployment apparatus of the in-vehicle image processing system. As shown in fig. 1, the method may include the following processes:

in step 100, system attributes of an in-vehicle operating system of a vehicle in which an in-vehicle image processing system is to be deployed are acquired.

A vehicle-mounted operating system can be installed on the vehicle so as to install a plurality of different applications through the vehicle-mounted operating system, thereby realizing the multifunctional experience of the vehicle. For example, an in-vehicle image processing system may be installed by the in-vehicle operating system.

The vehicle-mounted operating system can be a vehicle-mounted Android operating system. In this step, the obtained system attribute may include, for example: a system version of the current vehicle-mounted operating system. The version of the system used by the different vehicle devices may be different, from which it is known which vehicle device, or which model of vehicle device, is. By way of example, based on the read system attributes, it may be determined that the current system is a version of the on-board operating system installed on a vehicle of a certain model.

The vehicle-mounted image processing system to be deployed may be an image processing system mounted on a vehicle through the above-described vehicle-mounted operating system. The present embodiment does not limit the functions of the image processing system, for example, the image processing system may be an automatic driving assistance system or a cabin control system disposed on a vehicle, and may collect images of a driver or images of the interior of the vehicle through a camera disposed on the vehicle, so as to assist the vehicle to perform certain intelligent driving, safety monitoring, provide a new man-machine interaction mode, or improve riding experience, and so on. For example, the method can assist in monitoring whether a driver is tired or not, and assist in intelligently controlling functional components such as windows, media players, vehicle-mounted display equipment and the like in the vehicle.

The deployment apparatus of the present embodiment may be a separate component from the in-vehicle image processing system, or may be one of the modules as the in-vehicle image processing system, or the like. This embodiment is not limited thereto.

In step 102, a mapping relationship between system attributes of different vehicle-mounted operating systems and acquisition paths of different camera drivers is acquired.

Different vehicle-mounted operating systems can use different camera-driven acquisition paths, for example, the different camera-driven acquisition paths can provide different API calling modes for calling the camera drive for a vehicle-mounted image processing system layer. Illustratively, the S1 system (one of the onboard operating systems, identified herein for the moment by S1) may access the camera driver in one API call, and the S2 system (the other onboard operating system, identified herein for the moment by S2) may access the camera driver in another API call.

In this embodiment, the mapping relationship between the acquisition paths of the plurality of camera drivers and the system attribute of the vehicle-mounted operating system may be stored in advance. For example, if the vehicle-mounted operating system of the vehicle is the system S1, the acquisition path of the camera drive to be used is P1 (one of the acquisition paths, here denoted by P1 for short); if the on-board operation system of the vehicle is the system S2, the acquisition path of the camera drive to be used is P2.

The acquisition path of the camera driver can be provided with various calling interfaces, and the calling interfaces are used for operating the camera. The call interface may include at least one of:

1) The camera parameter setting interface is used for setting the camera parameters of the camera, for example, the parameters of the camera can be set through the interface.

2) And the image setting interface is used for setting parameters of the image shot by the camera. For example, the size and format of the image collected by the camera can be set through the interface.

3) And the camera management interface is used for carrying out management operation on the camera. For example, the camera can be opened, previewed, closed and the like through the interface.

In a specific implementation, the call interface included in the acquisition path is not limited to the above interface. Different vehicle-mounted operating systems can adapt to different camera-driven acquisition paths.

In step 104, according to the mapping relationship, determining a camera driving acquisition path corresponding to the system attribute of the vehicle.

In this step, the acquisition path of the camera driver corresponding to the matching may be determined as the acquisition path to be used according to the system attribute acquired in step 100 and the mapping relationship acquired in step 102.

In step 106, in a state that the vehicle-mounted camera is connected to the vehicle, invoking the camera drive of the vehicle-mounted camera stored in the vehicle-mounted operating system according to the acquisition path of the camera drive.

In this step, the vehicle-mounted camera may be, for example, a camera connected to the vehicle via a non-USB interface, which is equivalent to a camera carried by the vehicle, i.e., the camera is fixedly connected to the vehicle, and is not connected to the vehicle via a USB interface in a plug and play manner; alternatively, the vehicle-mounted camera may be a USB camera connected to the vehicle via a USB interface.

And (4) invoking a camera driver corresponding to the vehicle-mounted camera stored in the vehicle-mounted operating system according to the acquired path acquired in the step (104). For example, the camera management interface set in the acquisition path P1 invokes the camera driver to perform operations such as opening and closing on the camera.

In actual implementation, when the camera driver stored in the vehicle-mounted operating system is called according to the acquisition path of the camera driver, the camera driver stored in the Linux layer of the Android system architecture can still be called according to a calling mode set in the acquisition path based on the Android system architecture of the vehicle-mounted operating system.

In step 108, the interaction connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operation system is established through the camera drive corresponding to the vehicle-mounted camera.

In the step, the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operation system can be built by calling the camera driver. The interactive connection means that the vehicle-mounted image processing system and the vehicle-mounted camera can perform interactive communication through the vehicle-mounted operation system, for example, the vehicle-mounted image processing system can control the vehicle-mounted camera to collect images through the vehicle-mounted operation system and receive the images collected by the vehicle-mounted camera to perform image processing.

According to the deployment method of the vehicle-mounted image processing system, the paths matched with the current attribute are acquired from the acquisition paths driven by the cameras according to the system attribute, so that the camera driving of the vehicle-mounted camera can be accessed according to the paths, the vehicle-mounted image processing system can be deployed as soon as possible, and the deployment efficiency of the vehicle-mounted image processing system is improved. For example, different vehicle-mounted operation systems of different vehicle-mounted equipment on the market are quite different, the different vehicle-mounted operation systems can access the camera driver through different calling interfaces, if the vehicle-mounted image processing systems are to be deployed on the different vehicle-mounted equipment, the vehicle-mounted image processing systems are required to be developed in a targeted manner aiming at different vehicles so as to adapt to the driving access modes of the different systems respectively, the cost is increased, the engineering period is prolonged, and the efficiency of deploying the image processing systems is lower; the method of the embodiment can automatically acquire the matched driving path according to the system attribute, quickly realize driving access and vehicle-mounted image processing system deployment, does not need to independently develop the vehicle-mounted image processing system aiming at different operation systems, reduces development cost, improves universality of the vehicle-mounted image processing system, can realize quick deployment of the same vehicle-mounted image processing system on vehicles of different vehicle types or different operation systems of the same vehicle type, and meets diversified practical application requirements such as post-loading deployment of the vehicle-mounted image processing system or temporary deployment of function demonstration of the vehicle-mounted image processing system.

In an optional example of the disclosure, the vehicle-mounted image processing system expects to use a camera which is already configured in a standard way on the vehicle, or when the camera is accessed through a special interface of the camera on the vehicle, the corresponding driving loading path is automatically matched according to the vehicle-mounted operating system, so as to accelerate deployment of the vehicle-mounted image processing system.

In another optional example of the disclosure, if the vehicle-mounted image processing system expects to use other types of cameras, for example, a camera that is additionally connected to the vehicle is used, or a camera that is connected to the vehicle through a non-camera-specific interface (such as a camera that is connected to a universal USB interface, which may be called a USB camera), the vehicle-mounted image processing system may load a camera driver of the camera by adopting the technical solution provided in the disclosure, and an interactive connection between the vehicle-mounted image processing system and the camera is established through the camera driver of the camera.

Fig. 2 is a deployment method of an on-vehicle image processing system according to another embodiment of the present disclosure, which may be used in a deployment scenario of the on-vehicle image processing system when other types of cameras described above are to be used (e.g., cameras that are additionally accessed by a vehicle, or cameras that are accessed by a non-camera-specific interface on the vehicle are used). The method may include a process wherein the same steps as the process shown in fig. 1 may be described in connection with the process of fig. 1:

In step 200, system attributes of an in-vehicle operating system of a vehicle in which an in-vehicle image processing system is to be deployed are acquired.

In practical implementation, system attributes of different vehicle-mounted operating systems can be set, and different camera use priorities are correspondingly provided. For example, it is assumed that two types of in-vehicle cameras are provided on a vehicle, which are temporarily called a first camera, such as a camera that is already configured in a standard manner on the vehicle, or a camera that is accessed through a camera-specific interface on the vehicle, and a second camera, such as a USB camera that is connected with a USB interface. The first camera may be used preferentially by the vehicle-mounted operating system that may set certain system attributes and the second camera may be used preferentially by the vehicle-mounted operating system that may set other system attributes.

In this step, after the system attribute of the vehicle-mounted operating system is obtained, if it is determined that the use priority of the second camera is higher than that of the first camera, it is determined that the camera driver of the second camera is used to establish the interactive connection between the second camera and the vehicle-mounted image processing system, and the subsequent steps 202 to 206 are continuously executed.

Or if it is determined that the first camera is to be preferentially used according to the system attribute, but when the camera driver of the first camera is called, feedback information of call failure is received, it is determined that the second camera is used, and steps 202 to 206 are continuously executed to complete loading of the camera driver of the second camera.

Of course, if the call of the first camera is normal, it may be performed according to the flow of fig. 1 without performing steps 202 to 206 of the present embodiment.

In step 202, a driver loading module of a third party tool library stored in the vehicle-mounted operating system is called, and a camera driver of a camera is loaded through the driver loading module.

The driver loading module provides an access path for accessing the camera driver in the vehicle-mounted operating system, and the access path provides a call interface for directly accessing the camera driver of the Linux layer by the vehicle-mounted image processing system without passing through the middle layer of the Android operating system architecture.

In this step, the driver loading module in the third party tool library may include, but is not limited to, a libuvccamera tool package, where the tool package may bypass an intermediate layer of the Android system framework, for example, a hardware abstraction layer, an application framework layer, and the like, and directly access a camera driver of the underlying Linux layer.

For example, the call interface provided by the driver loading module may also include: an interface for carrying out format coding on images acquired by a camera, an interface for carrying out enumeration, data stream reading and other operations on the camera, an interface for acquiring parameters such as the size, the frame interval and the like of video frames acquired by the camera, and the like. The vehicle-mounted image processing system can call the interfaces provided by the drive loading module to directly access the camera drive of the Linux layer.

For example, taking the example that the second camera is a USB camera, the driver loading module may perform device enumeration, and identify identification information of the USB camera. The camera identification information may include, for example: the manufacturer identifier VID (Vendor ID) and the product identifier PID (Product ID) of the USB camera acquire the USB camera through device enumeration, and call the camera driver of the USB camera according to the call interface.

In step 204, a camera driver corresponding to the camera is loaded according to the access path.

For example, the in-vehicle image processing system may invoke these interfaces provided by a driver loading module that provides another method of indirectly accessing the camera driver of the Linux layer, through which the camera driver corresponding to the second camera may be loaded.

In step 206, an interactive connection among the vehicle-mounted image processing system, the camera and the vehicle-mounted operation system is established through a camera driver corresponding to the camera.

In the step, the interaction connection among the vehicle-mounted image processing system, the second camera and the vehicle-mounted operating system can be established through the camera driving of the second camera. For example, the vehicle-mounted image processing system can control the second camera to acquire an image through the vehicle-mounted operation system, and receive the image acquired by the second camera for image processing.

According to the deployment method of the vehicle-mounted image processing system, the camera driver of the camera is loaded according to the driver loading module, so that the influence of differences in calling modes of different vehicle-mounted operating systems of different vehicle-mounted equipment is avoided, the vehicle-mounted image processing system does not need to be independently developed aiming at different operating systems, development cost is reduced, universality of the vehicle-mounted image processing system is improved, rapid deployment of the same vehicle-mounted image processing system on vehicles of different vehicle types or different operating systems of the same vehicle type can be realized, and diversified practical application requirements such as post-loading deployment of the vehicle-mounted image processing system or temporary deployment of function demonstration of the vehicle-mounted image processing system are met.

In addition, in other embodiments, when the acquisition path of the camera driver corresponding to the system attribute is acquired according to the mapping relationship between the system attribute and the acquisition path of the camera driver, it is also possible to acquire at least two acquisition paths corresponding to the same system attribute.

In this case, one of the at least two acquisition paths may be determined as the target acquisition path to be used. Ways to determine the target acquisition path include, but are not limited to, the following:

for example, one default path may be selected from the at least two acquisition paths.

For example, one acquisition path may be selected randomly from at least two acquisition paths, and the camera driver may be loaded according to the acquisition paths to establish the interactive connection between the camera and the vehicle-mounted image processing system or the vehicle-mounted operating system. Then, acquiring a performance parameter of the image acquired by the camera driven based on the acquisition path, for example, acquiring a frame rate of the video frame acquired by the camera, and if the frame rate does not reach a preset frame rate condition, switching to another path of the at least two acquisition paths.

For another example, when the interaction between the camera and the vehicle-mounted image processing system and the vehicle-mounted operating system is established based on each of the at least two acquisition paths, the performance parameters of the camera for acquiring the image, such as the frame rate of the video frames acquired by the camera, may be acquired respectively. And comparing which of the at least two acquisition paths has better performance parameters, and selecting the acquisition path with better performance parameters.

After the camera is called to drive the enabling camera, the vehicle-mounted image processing system can acquire images acquired by the camera, and perform function control after performing image processing on the images. The following examples are examples of the in-vehicle image processing system performing function control based on the camera-captured image, but it is understood that the embodiments are not limited to these examples.

In one example, the vehicle-mounted image processing system may perform gesture recognition processing on the gesture image based on the gesture image acquired by the camera, so as to obtain a gesture recognition result. And the vehicle-mounted image processing system transmits a control instruction corresponding to the gesture recognition result to a controlled module in the vehicle through the vehicle-mounted operating system, so that the controlled module executes an operation corresponding to the control instruction.

For example, the controlled module may be a control module of a music player. The driver can make gesture images, such as V gestures, fist gestures and the like, the camera collects the gesture images, the vehicle-mounted image processing system can conduct gesture recognition according to the gesture images and obtain control instructions corresponding to the gestures, and the control instructions can be transmitted to a control module of the music player through the vehicle-mounted operation system so that music playing of the music player can be controlled. For example, controlling a music player to start playing music, or controlling the volume adjustment of the player, etc.

In another example, the camera may capture an image of the face of the driver, such as the captured image shown in FIG. 3, including an image of the face of the driver. The vehicle-mounted image processing system can perform image processing on the face image to obtain fatigue driving information, wherein the fatigue driving information is used for indicating that a driver is in fatigue driving.

For example, the facial image may be input into a pre-trained image processing neural network that may output whether the driver corresponding to the facial image is driving fatigue. For example, if the image processing neural network determines that the driver is in the closed eye or yawned according to the image recognition and recognizes that the driver does the above-mentioned action for a certain period of time, the image processing neural network can determine that the driver is driving fatigue and output the driving fatigue information.

If the driver is identified to be in fatigue driving, the vehicle-mounted image processing system can send prompt indication information to the vehicle-mounted response module based on the fatigue driving information so as to prompt the fatigue driving. The vehicle-mounted response module may be a vehicle-mounted display module (e.g., a vehicle-mounted display interface), or may be a non-display module, such as a voice playing module. The prompt indication information is an instruction for controlling the vehicle-mounted response module to send out fatigue driving prompts. For example, the prompt indication information can indicate the voice playing module to carry out voice fatigue driving prompt on the driver; or, the indication information of fatigue driving can be indicated to be displayed on the display interface of the vehicle. For example, a red alert frame may be sprung out of the interface shown in fig. 3.

In yet another example, the camera may capture an image of the face of the driver, such as the captured image shown in FIG. 3, including an image of the face of the driver. The vehicle-mounted image processing system can perform image processing on the face image to obtain distraction driving information, wherein the distraction driving information is used for indicating that a driver is not attentive when driving.

For example, the facial image may be input into a pre-trained image processing neural network that may output whether the driver corresponding to the facial image is inattentive while driving. For example, if the image processing neural network determines from image recognition that the driver's eyes are not looking ahead, are looking in the upward/downward/left/right direction, and recognize that the driver does the above-described action for a certain period of time in succession, it can be determined that the driver is distracting.

If it is identified that the driver is driving in a distraction manner, the vehicle-mounted image processing system may send prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the distraction driving information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In yet another example, the camera may capture an image of the driver's action, such as the captured image shown in FIG. 4, which may be taken of what the driver did. For example, the driver in fig. 4 is making a call, and in other examples, the driver may also take other actions such as smoking, drinking water, etc., which may affect the driving safety during driving, which is a dangerous action during driving. The vehicle-mounted image processing system can perform image processing on the acquired image to obtain dangerous action information, wherein the dangerous action information is used for indicating that a driver is doing dangerous actions when driving.

For example, the driver action image collected by the camera may be input into a pre-trained image processing neural network, and the neural network may output whether the driver corresponding to the image is doing dangerous actions while driving. For example, if the image processing neural network determines from the image recognition that the driver is making a call and recognizes that the driver does the above-described action for a certain period of time, it may be determined that the driver is doing a dangerous action.

If the driver is recognized to be doing dangerous actions, the vehicle-mounted image processing system can send prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the dangerous action information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information. In this embodiment, the vehicle-mounted response module may be used to prompt the dangerous action affecting driving, for example, "please do not call" may be displayed in the interface shown in fig. 4.

In yet another example, as illustrated in fig. 3 or 4, the in-vehicle image processing system may display the driver image captured by the camera, such as the driver face image, through an image display interface. The present example may also detect a gaze area of the driver, for example, during normal driving, the driver should gaze the central control display screen, and the vehicle-mounted image processing system may perform image processing on the face image of the driver acquired by the camera to obtain area gaze information, where the area gaze information may indicate whether the driver is gazing at a target area, and the target area is, for example, the central control display screen.

The vehicle-mounted image processing system can send prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the regional fixation information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

For example, the vehicle-mounted image processing system may send display parameter adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module performs a prompt operation according to the display parameter adjustment information. The display parameter adjusting information can be interface brightness adjusting information, the vehicle-mounted display module can adjust the brightness of an image display interface according to the interface brightness adjusting information, and the image display interface can be used for displaying the face image of the driver acquired by the camera.

If the region fixation information indicates that the driver is fixation on the target region, the brightness of the image display interface is enhanced. For example, in the interface of fig. 3, the driver may find that the screen brightness of the interface may be enhanced. And if the region-gazing information indicates that the driver does not gaze at the target region, the in-vehicle image processing system may decrease the brightness of the image display interface. For example, in the interface of fig. 3, the driver may find that the screen brightness of the interface may decrease.

In addition, not only is the example of gazing region detection, in the above-mentioned multiple examples of fatigue detection, distraction driving detection and the like, the vehicle-mounted operation system can send interface brightness adjustment information to the vehicle-mounted display module, and the brightness of the image display interface is adjusted to prompt different conditions of fatigue driving, distraction driving and the like.

In other examples, the vehicle-mounted image processing system may further send digital person display control information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module displays the digital person on the vehicle-mounted image display interface according to the digital person display control information, or controls the digital person to output interactive feedback information. Thus, the conditions such as fatigue driving, distraction driving and the like can be prompted by the digital person.

For example, the digital person may be an avatar generated by software, and the digital person may be displayed on an in-vehicle display device, such as a center control display screen or an in-vehicle tablet device. The interactive feedback information output by the digital person includes at least one of: voice feedback information, expression feedback information and action feedback information.

For example, a digital person may be displayed when the gaze direction of the person in the vehicle is directed to the in-vehicle display module. And displaying the digital person when the gazing area of the person in the vehicle is at least partially overlapped with the setting area of the vehicle-mounted display module. When the sight and fixation direction of the person in the vehicle points to the vehicle-mounted display device again or the fixation area and the setting area of the vehicle-mounted display module are overlapped at least partially again, the digital person can send out the language, expression or action of what needs me to do.

In another example, for fatigue driving, a digital person may be controlled to output corresponding interactive feedback information. For example, a digital person may output "I call you for a song bar", "do not need to rest" speech, expression or motion to alleviate fatigue.

In another example, for the case of distracted driving, the digital person may be controlled to output voices, expressions or actions such as "focus", "done well, keep on" and the like.

In another example, in the case that the driver makes dangerous actions affecting driving, the digital person may be controlled to output voices, expressions or actions such as "please not extend the body out of the window", "please look ahead".

In other examples, whether or not a different function is enabled may be displayed by a status icon, or the detection result may be indicated by the lighting of an icon. For example, if the icon corresponding to the fatigue detection is displayed in a lighted manner, the vehicle-mounted image processing system detects whether the driver is in fatigue driving or not according to the image acquired by the camera at the moment; for another example, if the region is gazed at the corresponding icon, it indicates that the driver is not gazing at the region at the moment, so as to warn the driver.

Fig. 5 provides a deployment apparatus of an in-vehicle image processing system, which may perform the deployment method of the in-vehicle image processing system of any of the embodiments of the present disclosure. As shown in fig. 5, the apparatus may include: an attribute acquisition module 51, a map acquisition module 52, a path determination module 53, a drive loading module 54, and a connection establishment module 55.

The attribute obtaining module 51 is configured to obtain a system attribute of an on-board operating system of a vehicle in which the on-board image processing system is to be deployed. For example, the in-vehicle image processing system to be deployed may be an image processing system mounted on a vehicle by the in-vehicle operating system described above. The present embodiment does not limit the functions of the image processing system, for example, the image processing system may be an automatic driving assistance system deployed on a vehicle, and may collect an image of a driver or an image of the inside of the vehicle through a camera provided on the vehicle, to assist the vehicle in performing certain intelligent driving or safety monitoring.

The mapping obtaining module 52 is configured to obtain mapping relationships between system attributes of different vehicle-mounted operating systems and obtaining paths of different camera drivers.

And the path determining module 53 is configured to determine, according to the mapping relationship, an acquisition path of a camera driver corresponding to a system attribute of the vehicle.

Different vehicle-mounted operating systems can use different camera-driven acquisition paths, for example, the different camera-driven acquisition paths can provide different API calling modes for calling the camera drive for a vehicle-mounted image processing system layer. Illustratively, the S1 system may access the camera driver using one API call and the S2 system may access the camera driver using another API call.

The drive loading module 54 is configured to invoke a camera drive of the vehicle-mounted camera stored in the vehicle-mounted operating system according to an acquisition path of the camera drive in a state where the vehicle-mounted camera is connected to the vehicle;

and the connection establishing module 55 is used for establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through the camera drive corresponding to the vehicle-mounted camera.

According to the deployment device of the vehicle-mounted image processing system, the paths matched with the current attribute are acquired from the acquisition paths driven by the cameras according to the system attribute, so that the camera driving of the vehicle-mounted camera can be accessed according to the paths, and the vehicle-mounted image processing system can be deployed as soon as possible.

In one example, the driver loading module 54 is specifically configured to: calling the camera driver according to a plurality of calling interfaces arranged in the acquisition path of the camera driver; the call interface includes at least one of: the camera management system comprises a camera parameter setting interface for setting camera parameters of the camera, an image setting interface for setting parameters of images shot by the camera and a camera management interface for managing the camera.

In one example, the different camera driven acquisition paths include: and the calling interfaces are at least partially different and are used for operating the camera.

In one example, the driver loading module 54 is specifically configured to: and calling the camera driver stored in the Linux layer of the Android system architecture according to a calling mode set in the acquisition path based on the Android system architecture of the vehicle-mounted operating system.

In one example, the path determining module 53 is further configured to determine, when the vehicle-mounted camera is a USB camera connected to the vehicle through a USB interface, to use the USB camera connected to the vehicle through the USB interface according to the system attribute; a drive loading module of a third-party tool library stored in the vehicle-mounted operating system is called, and a camera driver of the USB camera is loaded through the drive loading module, wherein an access path for accessing the camera driver in the vehicle-mounted operating system is provided in the drive loading module, and the access path provides a call interface for directly accessing the camera driver of a Linux layer by the vehicle-mounted image processing system without passing through an intermediate layer of an Android operating system architecture;

the driver loading module 54 is configured to load a camera driver corresponding to the USB camera according to the access path;

the connection establishing module 55 is configured to establish an interactive connection between the vehicle-mounted image processing system, the USB camera, and the vehicle-mounted operating system through a camera driver corresponding to the USB camera.

The camera driver of the USB camera is loaded according to the driver loading module, so that the influence of different vehicle-mounted operating systems on different calling modes is avoided, the method can be widely applied to various vehicle-mounted operating systems, and the deployment speed of a vehicle-mounted image processing system is accelerated.

In one example, the path determining module 53, when configured to determine to use a USB camera connected to the vehicle via a USB interface according to the system attribute, includes: according to the system attribute, if the use priority of the USB cameras connected with the USB interface is higher than that of the cameras connected with the non-USB interface, the USB cameras are used; or if the feedback information of call failure is received when the camera driver of the camera connected with the vehicle through the non-USB interface is called, determining to use the USB camera.

In one example, the path determining module 53, when determining, according to the mapping relationship, an acquisition path of a camera driver corresponding to a system attribute of the vehicle, includes: acquiring at least two acquisition paths of the camera driver corresponding to the system attribute according to the mapping relation; and determining one acquisition path from the at least two acquisition paths as a target acquisition path to be used.

In one example, as shown in fig. 6, the apparatus further includes:

the image processing module 56 is used for receiving the gesture image acquired by the camera; performing gesture recognition processing on the gesture image to obtain a gesture recognition result; and transmitting a control instruction corresponding to the gesture recognition result to a controlled module in the vehicle through the vehicle-mounted operating system, so that the controlled module executes an operation corresponding to the control instruction.

In one example, the image processing module 56 is further configured to receive an image of the face of the driver acquired by the camera; performing image recognition processing on the face image of the driver to obtain fatigue driving information; and based on the fatigue driving information, sending prompt indication information to a vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In one example, the image processing module 56 is configured to receive the image of the face of the driver acquired by the camera; performing image recognition processing on the face image of the driver to obtain distracted driving information; based on the distraction driving information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In one example, the image processing module 56 is configured to receive the driver action image acquired by the camera; performing image recognition processing on the driver action image to obtain dangerous action information; based on the dangerous action information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

For example, the driver is making a call, and in other examples, the driver may also be in other actions such as smoking, drinking water, etc., which may affect the driving safety during driving, which is a dangerous action during driving. The vehicle-mounted image processing system can perform image processing on the acquired image to obtain dangerous action information, wherein the dangerous action information is used for indicating that a driver is doing dangerous actions when driving.

In one example, the image processing module 56 is configured to receive the image of the face of the driver acquired by the camera; performing image processing on the face image of the driver acquired by the camera to obtain regional fixation information; and based on the regional fixation information, sending prompt indication information to a vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

In one example, the image processing module 56 is specifically configured to send, through the vehicle-mounted operating system, display parameter adjustment information or display content adjustment information to the vehicle-mounted display module, so that the vehicle-mounted display module performs a prompt operation according to the prompt indication information.

In one example, the image processing module 56, when configured to send the display parameter adjustment information to the in-vehicle display module through the in-vehicle operating system, is specifically configured to: and sending interface brightness adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module adjusts the brightness of an image display interface according to the interface brightness adjustment information.

In one example, the image processing module 56 is specifically configured to, when configured to send display content adjustment information to the in-vehicle display module through the in-vehicle operating system: and sending digital person display control information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module displays the digital person on a vehicle-mounted image display interface according to the digital person display control information, or controls the digital person to output interactive feedback information.

The disclosed embodiments also provide an electronic device, as shown in fig. 7, the device comprising a memory 71 and a processor 72, the memory 71 for storing computer instructions executable on the processor 72, the processor 72 for implementing the method of any of the embodiments of the disclosure when executing the computer instructions.

The disclosed embodiments also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of the embodiments of the disclosure.

One skilled in the art will appreciate that one or more embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program may be stored, which when executed by a processor, implements the steps of the neural network training method for word recognition described in any embodiment of the present disclosure, and/or implements the steps of the word recognition method described in any embodiment of the present disclosure. Wherein the term "and/or" means at least one of the two, e.g. "multiple and/or B" includes three schemes: many, B, and "many and B".

The various embodiments in this disclosure are described in a progressive manner, and identical and similar parts of the various embodiments are all referred to each other, and each embodiment is mainly described as different from other embodiments. In particular, for data processing apparatus embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.

The foregoing has described certain embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Embodiments of the subject matter and functional operations described in this disclosure may be implemented in the following: digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this disclosure and structural equivalents thereof, or a combination of one or more of them. Embodiments of the subject matter described in this disclosure can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions may be encoded on a manually-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode and transmit information to suitable receiver apparatus for execution by data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

The processes and logic flows described in this disclosure can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., a FPG multi (field programmable gate array) or multi SIC (application specific integrated circuit).

Computers suitable for executing computer programs include, for example, general purpose and/or special purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read only memory and/or a random access memory. The essential elements of a computer include a central processing unit for carrying out or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks, etc. However, a computer does not have to have such a device. Furthermore, the computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PD multislot), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) flash drive, to name a few.

Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disk or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

Although this disclosure contains many specific implementation details, these should not be construed as limiting the scope of any disclosure or the scope of what is claimed, but rather as primarily describing features of particular embodiments of the particular disclosure. Certain features that are described in this disclosure in the context of separate embodiments can also be implemented in combination in a single embodiment. On the other hand, the various features described in the individual embodiments may also be implemented separately in the various embodiments or in any suitable subcombination. Furthermore, although features may be acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Furthermore, the processes depicted in the accompanying drawings are not necessarily required to be in the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

The foregoing description of the preferred embodiment(s) of the present disclosure is merely intended to illustrate the embodiment(s) of the present disclosure, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the embodiment(s) of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A deployment method of an on-vehicle image processing system, the method comprising:

establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through the camera drive corresponding to the vehicle-mounted camera;

wherein after the system attribute of the vehicle-mounted operation system of the vehicle to which the vehicle-mounted image processing system is to be deployed is acquired, the method further comprises:

a drive loading module of a third party tool library stored in the vehicle-mounted operating system is called, and a camera driver of the vehicle-mounted camera is loaded through the drive loading module, wherein an access path for accessing the camera driver in the vehicle-mounted operating system is provided in the drive loading module, and the access path provides a call interface for directly accessing the camera driver of a Linux layer by the vehicle-mounted image processing system without passing through an intermediate layer of an Android operating system architecture;

loading a camera driver corresponding to the vehicle-mounted camera according to the access path;

and establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through camera driving corresponding to the vehicle-mounted camera.

2. The method of claim 1, wherein invoking the camera driver of the onboard camera stored in the onboard operating system according to the acquisition path of the camera driver comprises:

calling the camera driver according to a plurality of calling interfaces arranged in the acquisition path of the camera driver; the call interface includes at least one of: the camera management system comprises a camera parameter setting interface for setting camera parameters of the camera, an image setting interface for setting parameters of images shot by the camera and a camera management interface for managing the camera.

3. A method according to claim 1 or 2, characterized in that,

the different camera driving acquisition paths comprise: and the calling interfaces are at least partially different and are used for operating the camera.

4. The method of claim 1, wherein invoking the camera driver of the onboard camera stored in the onboard operating system according to the acquisition path of the camera driver comprises:

and calling the camera driver stored in the Linux layer of the Android system architecture according to a calling mode set in the acquisition path based on the Android system architecture of the vehicle-mounted operating system.

5. The method of claim 1, wherein the onboard camera is a USB camera connected to the vehicle with a USB interface;

before the calling the driver loading module of the third party tool library stored in the vehicle-mounted operating system, the method further comprises: and determining to use a USB camera connected with the vehicle through a USB interface according to the system attribute.

6. The method of claim 5, wherein determining to use a USB camera connected to the vehicle with a USB interface based on the system attributes comprises:

according to the system attribute, if the use priority of the USB cameras connected with the USB interface is higher than that of the cameras connected with the non-USB interface, the USB cameras are used;

or if the feedback information of call failure is received when the camera drive of the vehicle-mounted camera connected with the vehicle through the non-USB interface is called, determining to use the USB camera.

7. The method according to claim 1, wherein determining a camera-driven acquisition path corresponding to a system attribute of the vehicle according to the mapping relation includes:

acquiring at least two acquisition paths of the camera driver corresponding to the system attribute according to the mapping relation;

And determining one acquisition path from the at least two acquisition paths as a target acquisition path to be used.

8. The method according to any one of claims 1 to 7, wherein after the establishing of the interactive connection among the in-vehicle image processing system, the in-vehicle camera, and the in-vehicle operating system by the camera drive corresponding to the in-vehicle camera, the method further comprises:

the vehicle-mounted image processing system receives gesture images acquired by the camera;

the vehicle-mounted image processing system performs gesture recognition processing on the gesture image to obtain a gesture recognition result;

and the vehicle-mounted image processing system transmits a control instruction corresponding to the gesture recognition result to a controlled module in the vehicle through the vehicle-mounted operating system, so that the controlled module executes an operation corresponding to the control instruction.

9. The method according to any one of claims 1 to 7, wherein after the establishing of the interactive connection among the in-vehicle image processing system, the in-vehicle camera, and the in-vehicle operating system by the camera drive corresponding to the in-vehicle camera, the method further comprises:

The vehicle-mounted image processing system receives the face image of the driver acquired by the camera;

the vehicle-mounted image processing system performs image recognition processing on the face image of the driver to obtain fatigue driving information;

and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the fatigue driving information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

10. The method according to any one of claims 1 to 7, wherein after the establishing of the interactive connection among the in-vehicle image processing system, the in-vehicle camera, and the in-vehicle operating system by the camera drive corresponding to the in-vehicle camera, the method further comprises:

the vehicle-mounted image processing system performs image recognition processing on the face image of the driver to obtain distracted driving information;

and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the distraction driving information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

11. The method according to any one of claims 1 to 7, wherein after the establishing of the interactive connection among the in-vehicle image processing system, the in-vehicle camera, and the in-vehicle operating system by the camera drive corresponding to the in-vehicle camera, the method further comprises:

the vehicle-mounted image processing system receives the driver action image acquired by the camera;

the vehicle-mounted image processing system performs image recognition processing on the driver action image to obtain dangerous action information;

and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the dangerous action information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

12. The method according to any one of claims 1 to 7, wherein after the establishing of the interactive connection among the in-vehicle image processing system, the in-vehicle camera, and the in-vehicle operating system by the camera drive corresponding to the in-vehicle camera, the method further comprises:

The vehicle-mounted image processing system performs image processing on the face image of the driver acquired by the camera to obtain regional fixation information;

and the vehicle-mounted image processing system sends prompt indication information to the vehicle-mounted response module through the vehicle-mounted operation system based on the regional fixation information, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

13. The method according to any one of claims 9 to 12, wherein the vehicle-mounted response module is a vehicle-mounted display module, and the sending, by the vehicle-mounted operating system, prompt instruction information to the vehicle-mounted response module, so that the vehicle-mounted response module performs a prompt operation according to the prompt instruction information, includes:

and sending display parameter adjustment information or display content adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system so that the vehicle-mounted display module carries out prompt operation according to the prompt indication information.

14. The method according to claim 13, wherein the sending, by the vehicle-mounted operation system, display parameter adjustment information to the vehicle-mounted display module to cause the vehicle-mounted display module to perform a prompting operation according to the prompting instruction information includes:

And sending interface brightness adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module adjusts the brightness of an image display interface according to the interface brightness adjustment information.

15. The method according to claim 13, wherein the sending, by the vehicle-mounted operating system, display content adjustment information to the vehicle-mounted display module to cause the vehicle-mounted display module to perform a prompting operation according to the prompting instruction information includes:

and sending digital person display control information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module displays the digital person on a vehicle-mounted image display interface according to the digital person display control information, or controls the digital person to output interactive feedback information.

16. A deployment apparatus of an in-vehicle image processing system, characterized by comprising:

the connection establishment module is used for establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through the camera drive corresponding to the vehicle-mounted camera;

the path determining module is further used for calling a drive loading module of a third party tool library stored in the vehicle-mounted operating system, and loading a camera driver of the vehicle-mounted camera through the drive loading module, wherein an access path for accessing the camera driver in the vehicle-mounted operating system is provided in the drive loading module, and the access path provides a calling interface for the vehicle-mounted image processing system to directly access the camera driver of the Linux layer without passing through an intermediate layer of an Android operating system architecture;

the drive loading module is used for loading the camera drive corresponding to the vehicle-mounted camera according to the access path;

The connection establishment module is used for establishing interactive connection among the vehicle-mounted image processing system, the vehicle-mounted camera and the vehicle-mounted operating system through camera driving corresponding to the vehicle-mounted camera.

17. The apparatus of claim 16, wherein the device comprises a plurality of sensors,

the drive loading module is specifically configured to: calling the camera driver according to a plurality of calling interfaces arranged in the acquisition path of the camera driver; the call interface includes at least one of: the camera management system comprises a camera parameter setting interface for setting camera parameters of the camera, an image setting interface for setting parameters of images shot by the camera and a camera management interface for managing the camera.

18. The apparatus of claim 16 or 17, wherein the different camera-driven acquisition paths comprise: and the calling interfaces are at least partially different and are used for operating the camera.

19. The apparatus of claim 16, wherein the device comprises a plurality of sensors,

the drive loading module is specifically configured to: and calling the camera driver stored in the Linux layer of the Android system architecture according to a calling mode set in the acquisition path based on the Android system architecture of the vehicle-mounted operating system.

20. The apparatus of claim 16, wherein the device comprises a plurality of sensors,

the path determining module is further configured to determine, according to the system attribute, to use a USB camera connected to the vehicle through a USB interface, before the driver loading module of the third party tool library stored in the vehicle-mounted operating system is invoked.

21. The apparatus of claim 20, wherein the device comprises a plurality of sensors,

the path determining module, when determining to use a USB camera connected to the vehicle through a USB interface according to the system attribute, includes: according to the system attribute, if the use priority of the USB cameras connected with the USB interface is higher than that of the cameras connected with the non-USB interface, the USB cameras are used; or if the feedback information of call failure is received when the camera drive of the vehicle-mounted camera connected with the vehicle through the non-USB interface is called, determining to use the USB camera.

22. The apparatus of claim 16, wherein the device comprises a plurality of sensors,

the path determining module, when determining an acquisition path of a camera driver corresponding to a system attribute of the vehicle according to the mapping relationship, includes: acquiring at least two acquisition paths of the camera driver corresponding to the system attribute according to the mapping relation; and determining one acquisition path from the at least two acquisition paths as a target acquisition path to be used.

23. The apparatus according to any one of claims 16 to 22, further comprising:

the image processing module is used for receiving the gesture image acquired by the camera; performing gesture recognition processing on the gesture image to obtain a gesture recognition result; and transmitting a control instruction corresponding to the gesture recognition result to a controlled module in the vehicle through the vehicle-mounted operating system, so that the controlled module executes an operation corresponding to the control instruction.

24. The apparatus of claim 23, wherein the device comprises a plurality of sensors,

the image processing module is also used for receiving the face image of the driver acquired by the camera; performing image recognition processing on the face image of the driver to obtain fatigue driving information; and based on the fatigue driving information, sending prompt indication information to a vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

25. The apparatus of claim 23, wherein the device comprises a plurality of sensors,

the image processing module is used for receiving the face image of the driver acquired by the camera; performing image recognition processing on the face image of the driver to obtain distracted driving information; based on the distraction driving information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

26. The apparatus of claim 23, wherein the device comprises a plurality of sensors,

the image processing module is used for receiving the driver action image acquired by the camera; performing image recognition processing on the driver action image to obtain dangerous action information; based on the dangerous action information, prompt indication information is sent to the vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

27. The apparatus of claim 23, wherein the device comprises a plurality of sensors,

the image processing module is used for receiving the face image of the driver acquired by the camera; performing image processing on the face image of the driver acquired by the camera to obtain regional fixation information; and based on the regional fixation information, sending prompt indication information to a vehicle-mounted response module through the vehicle-mounted operation system, so that the vehicle-mounted response module carries out prompt operation according to the prompt indication information.

28. The apparatus according to any one of claims 24 to 27, wherein,

the image processing module is specifically configured to send display parameter adjustment information or display content adjustment information to the vehicle-mounted display module through the vehicle-mounted operating system, so that the vehicle-mounted display module performs a prompt operation according to the prompt indication information.

29. The apparatus of claim 28, wherein the device comprises a plurality of sensors,

the image processing module is specifically configured to, when configured to send display parameter adjustment information to the vehicle-mounted display module through the vehicle-mounted operating system: and sending interface brightness adjustment information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module adjusts the brightness of an image display interface according to the interface brightness adjustment information.

30. The apparatus of claim 29, wherein the device comprises a plurality of sensors,

the image processing module is specifically configured to, when configured to send display content adjustment information to the vehicle-mounted display module through the vehicle-mounted operating system: and sending digital person display control information to the vehicle-mounted display module through the vehicle-mounted operation system, so that the vehicle-mounted display module displays the digital person on a vehicle-mounted image display interface according to the digital person display control information, or controls the digital person to output interactive feedback information.

31. An electronic device comprising a memory, a processor for storing computer instructions executable on the processor for implementing the method of any one of claims 1 to 15 when the computer instructions are executed.

32. A computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements the method of any of claims 1 to 15.