CN112861638A

CN112861638A - Screen projection method and device

Info

Publication number: CN112861638A
Application number: CN202110048469.2A
Authority: CN
Inventors: 薛志荣; 邓园; 李宏汀; 叶军涛; 黄雪妍; 刘祖齐
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-01-14
Filing date: 2021-01-14
Publication date: 2021-05-28

Abstract

The embodiment of the application discloses a screen projection method and a screen projection device, which can be applied to the field of automobiles, and the method comprises the following steps: the terminal device (or vehicle) obtains a plurality of screen projection conditions of a plurality of display modules projected on the vehicle, one display module corresponds to one screen projection condition, the screen projection condition can be set according to the running information of the vehicle (such as the driving speed of the vehicle, the type of the traffic scene where the vehicle is located, and the like), or according to the characteristic information of the application program (such as the application type of the first application program running on the terminal equipment, the preset use frequency of the first application program and the like), when the first screen projection condition is met, screen projection information of the first application program is projected to the first display module corresponding to the first screen projection condition, so that the screen projection information on the terminal equipment is reasonably distributed to different display modules of the vehicle, the driver can quickly acquire the information on the terminal equipment, the probability of distraction of the driver is reduced, the interference of the screen projection information on the driving of the driver is reduced, and the driving safety is improved.

Description

Screen projection method and device

Technical Field

The application relates to the field of screen projection display, in particular to a screen projection method and device.

Background

With the improvement of the overall economic level of the society and the rapid development of automobile technology, in the field of automobile consumption, people change from traditional consumption meeting automobile function use to novel consumption pursuing automobile life experience. The number of screens (also called display modules) in the car is getting more and more, and the number of physical buttons is getting less and less, and most of the information and functions will be carried on the car central control screen, as shown in the sub-diagram (a) of fig. 1. In addition, the rapid development of intelligent terminal devices (such as mobile phones) also enables people to be more and more away from terminal devices such as mobile phones, and people inevitably encounter situations requiring terminal devices in the process of driving automobiles, for example, the sub-diagram (b) in fig. 1 shows that a driver uses a mobile phone to navigate, but the driver is not allowed to use the terminal devices such as mobile phones in the process of driving, which brings danger to driving.

Therefore, the existing solution is to directly project the information to be displayed on the terminal device such as a mobile phone to the central control screen of the automobile, and the user can directly obtain the related information through the information displayed on the central control screen.

However, the central control screen is not the optimal viewing area for the driver under any circumstances, and the driver is required to turn his head and shift his sight line when viewing the central control screen, so that frequent viewing is likely to distract the driver, and the driver is in danger during driving.

Disclosure of Invention

The embodiment of the application provides a screen projection method and device, screen projection conditions (such as driving speed of a vehicle, types of traffic scenes where the vehicle is located, application types of application programs running on terminal equipment, preset use frequency and the like) which need to be met when a display module on the vehicle is projected are obtained, when the screen projection conditions are met, screen projection information of the application programs running on the terminal equipment is projected to corresponding display modules, so that screen projection information on the terminal equipment is reasonably distributed to different display modules on the vehicle, a driver can quickly obtain information on the terminal equipment, the probability of distraction of the driver is reduced, the interference of the screen projection information on the driving of the driver is reduced, and the driving safety is improved.

Based on this, the embodiment of the present application provides the following technical solutions:

in a first aspect, an embodiment of the present application first provides a screen projection method, which may be used in the field of automobile consumption, and the method includes: first, the terminal device (or the target vehicle) acquires a first screen projection condition that a first display module on the target vehicle is projected on, where the first screen projection condition may be related to driving information of the target vehicle or related to feature information of an application program to be projected on the target vehicle, and is not limited herein. In particular, since the driving information of the vehicle may be characterized by various parameters, for example, the driving speed of the vehicle, the type of traffic scene in which the vehicle is located, the number of passengers in the vehicle, the fatigue state of the driver of the vehicle, the distance between the vehicle and the obstacle, and the like. Therefore, in the embodiment of the present application, the first screen projecting condition may be set according to the driving information of the target vehicle, and in this case, when the first screen projecting condition is satisfied, the method includes: when the running information of the target vehicle meets the first screen projection condition; the first screen projection condition can be set based on the running information of the target vehicle and can also be set according to the characteristic information of the application program to be projected. In this case, when the first screen projecting condition is satisfied, the method includes: and when the characteristic information of the first screen projection application program to be projected on the terminal equipment meets the first screen projection condition. And when the first screen projection condition is met and the communication connection between the terminal equipment and the vehicle is established, projecting screen projection information of a first screen application program to be projected on the terminal equipment to the first display module. For example, the judgment manner that the first screen projection condition is met may be that the characteristic information of the first screen application to be projected on the terminal device meets the first screen projection condition, or that the driving information of the target vehicle meets the first screen projection condition, which is not limited herein. In this embodiment, the first display module is one of a plurality of display modules on the target vehicle, and different display modules on the target vehicle have different screen projection conditions, for example, if there are 2 different display modules on the target vehicle a, which are respectively a dashboard screen and a central control screen, then there are 2 screen projection conditions for each display module, which are respectively screen projection condition 1 and screen projection condition 2.

It should be noted here that the application program to be screen-launched may refer to a screen-launched application program on a terminal device that establishes a communication connection with a target vehicle, but if a plurality of terminal devices are all connected to the target vehicle, a terminal device that is established earlier at the time may be selected as the terminal device to be screen-launched, and the screen-launched application program on the terminal device may be referred to as the screen-launched application program; the terminal device set as the default may also be selected as the terminal device to be subjected to screen projection, and a user may also select which terminal device is to be subjected to screen projection according to actual requirements, which is not limited herein. It should also be noted that the application to be projected on the terminal device may refer to an application that displays an application interface on a display interface of the terminal device, or may refer to an application that runs in the background of the terminal device (i.e., the application interface is not displayed on the display interface of the terminal device), and is not limited herein.

In the above embodiment of the application, by obtaining a first screen projection condition that needs to be met when a first display module on a vehicle is projected, when the first screen projection condition is met (for example, the driving speed of the vehicle, the type of a traffic scene where the vehicle is located, the application type of an application program to be projected on a terminal device, a preset use frequency and the like), screen projection information of the application program to be projected on the terminal device is projected to a corresponding first display module, so that screen projection information on the terminal device is reasonably distributed to a corresponding display module on the vehicle based on the screen projection condition, a driver can quickly obtain information on the terminal device, the probability of distraction of the driver is reduced, interference of the screen projection information on the driving of the driver is reduced, and the driving safety is improved.

In a possible design of the first aspect, a specific manner for the terminal device (or the target vehicle) to obtain the first screen projection condition that the first display module on the target vehicle is projected on may be: the method comprises the steps that a terminal device (or a target vehicle) obtains a first screen projection condition of a first display module projected from a plurality of screen projection conditions stored in the terminal device (or the target vehicle) in advance, can obtain the plurality of screen projection conditions at the same time, and can also obtain the plurality of screen projection conditions according to a preset sequence for comparison; the screen projection condition information may also be received by a cross-device, for example, the terminal device acquires the screen projection condition information from the target vehicle, or the target vehicle acquires the screen projection condition information from the terminal device of the owner. The screen projection condition can be set by default or set by the user according to the driving habit or other requirements. Thus, the term obtaining as used herein may also refer to accepting input settings from a user.

In a possible design of the first aspect, a specific manner for the terminal device (or the target vehicle) to obtain the first screen projection condition that the first display module on the target vehicle is projected on may be: in the embodiment of the present application, one display module corresponds to one screen projecting condition, for example, if there are 3 different display modules on the target vehicle a, which are respectively a dashboard screen, a steering wheel screen and a central control screen, then there are 3 screen projecting conditions for each display module, which are respectively screen projecting condition 1, screen projecting condition 2 and screen projecting condition 3. After the terminal device (or the target vehicle) acquires the screen projection conditions of the screen projected by the display modules on the target vehicle, when the screen projection conditions (which can be called as first screen projection conditions) of the screen projected by a certain display module (which can be called as a first display module) on the target vehicle are met, and under the condition that the terminal device establishes communication connection with the vehicle, screen projection information of a first screen application program to be projected on the terminal device is projected to the first display module.

In the above embodiment of the application, through obtaining the screen projection condition that needs to be met when each display module on the vehicle is projected, when the screen projection condition is met, then projecting the display information of the first screen application to be projected on the terminal device to the corresponding display module, thereby realizing the rational distribution of the screen projection information on the terminal device to different display modules on the vehicle, so that the driver can quickly obtain the information on the terminal device, the probability of driver distraction is reduced, the interference of the screen projection information to the driver driving is reduced, and the driving safety is improved.

In a possible embodiment of the first aspect, the conditions to be met for the projection are different when the driving information of the vehicle is represented by different parameters. In one case, the travel information of the target vehicle may include: the driving speed of the target vehicle. The first screen projecting condition includes the driving speed of the target vehicle being within a first driving speed range. Since different driving speeds are essentially aimed at driving safety, one principle that should be followed is: in the case of low safety, the screen projection information should be displayed on a display module that is most easily visible and visible to the driver with the least cost, for example, when the driving speed is high, and the driver needs to have higher attention, the screen projection information can be projected to a screen (e.g., a steering wheel screen, a head-up display (HUD), etc.) that can be seen by the driver with the least attention, and when the driving speed is slow, the screen projection information can be projected to a screen (e.g., a central control screen) that can be seen by the driver with a little head twist because the driver can not need so high attention due to the slow speed.

In the above embodiment of the application, the screen projection information of the application program to be screen projected on the terminal device is selected to be screen projected on different display modules based on different driving speeds, so that the driving safety is improved.

In one possible design of the first aspect, the travel information of the target vehicle may include: the type of traffic scene the target vehicle is in. The first screen projection condition comprises that the type of the traffic scene where the target vehicle is located belongs to the first traffic scene type. It should be noted that, in the embodiment of the present application, a traffic scene refers to a scene formed by communication participants (such as vehicles, pedestrians, and the like) and objects (such as traffic signs, obstacle signs, and the like) on a traffic network. The traffic scene can be divided into different types of traffic scenes according to preset division rules, for example, one division mode can be that the traffic scenes are divided according to traffic signs, and the traffic scenes can be divided into crossroads, T-shaped intersections, traffic light intersections (displaying red lights), turning intersections, straight roads and the like; one type of division may also be based on road conditions and surrounding traffic participants, and may divide traffic scene types into "freeway", "mountain road", "country road", "urban road", and so on. In the embodiment of the application, there is no limitation on what principle the traffic scene types are divided based on. Similarly, since the different traffic scene types that the vehicle is in are also substantially directed to driving safety, one principle that should be followed is: under the condition of low safety, the screen projection information is presented on a display module which can be seen and is most easily seen by a driver with the least cost, for example, when the vehicle is at an intersection with a bright red light, the driver stops the vehicle and waits for the change of the red light into a street light, at the moment, the driver can not need so high attention, the screen projection information can be projected to a screen (such as a central control screen) which can be seen by the driver by slightly twisting the head, when the vehicle is at the intersection, the intersection is generally a high-incidence area of a traffic accident, and at the moment, the driver needs to have sufficiently high attention, so the screen projection information can be projected to a screen (such as a steering wheel screen, a HUD and the like) which can be seen by the driver with the least attention.

In the embodiment of the application, the screen projection information of the application program to be screen projected on the terminal equipment is selected to be screen projected on different display modules based on different traffic scene types of the vehicle, so that the driving safety is improved.

In one possible design of the first aspect, the travel information of the target vehicle may include: the number of passengers in the target vehicle. The first screen projection condition includes the number of passengers in the target vehicle being within a first range of passenger numbers. It should be noted that in the embodiment of the present application, the number of passengers in the vehicle is not included in the driver, for example, when only the driver is in the vehicle, the number of passengers in the vehicle is 0. The essential point of the difference of the number of passengers in the vehicle is privacy and commonality of screen projection information, when an application program to be screen projected on a terminal device belongs to an application program with high privacy (such as WeChat), in order to avoid other passengers from seeing the screen projection information, the application program with high privacy can be screen-projected to a dashboard screen or a steering wheel screen which is only seen by a driver, and when the application program to be screen projected on the terminal device belongs to an application program with low privacy (such as Tencent video), the application program with low privacy can be screen-projected to a central control screen which can be seen by both the driver and the passengers in operation in consideration of the requirement of common use.

In the embodiment of the application, based on the consideration of privacy and commonality of the application programs, the number of passengers in the vehicle is different, screen projection information of the application programs to be projected on the terminal equipment can be selected to be projected on different display modules, and user experience is improved.

In one possible design of the first aspect, the travel information of the target vehicle may include: fatigue state of the target vehicle driver. The first screen projecting condition includes that the fatigue state of the driver of the target vehicle belongs to a first fatigue state. It should be noted that, in the embodiment of the present application, the fatigue state of the driver of the vehicle may be calculated from human body state information detected by a sensor module disposed in the vehicle, or may be calculated from human body state information detected by a wearable device (e.g., a smart band, a smart watch, etc.) worn by the driver. In some embodiments of the present application, the fatigue status of the driver may be classified according to the number of display modules of the vehicle, for example, assuming that there are 3 display modules on the vehicle, the fatigue status of the driver may be classified into 3 grades of "fatigue", "sub-fatigue", and "mental"; assuming that there are 2 display modules on the vehicle, the fatigue state of the driver can be divided into 2 grades of fatigue and spirit, and the corresponding driving behavior can be known to be safe or not based on different fatigue states of the driver. The different driving speeds are essentially aimed at driving safety, and therefore one principle that should be followed is: when the fatigue state of the driver is at a high fatigue level, which means that the driving behavior is not safe at the time, the screen projection information should be presented on a display module which can be seen and is most easily seen by the driver with minimal cost, for example, when the fatigue state of the driver is "fatigue", and the driver is required to use less energy and attention for driving the vehicle, the screen projection information can be projected to a screen (such as a steering wheel screen, a HUD, and the like) which can be seen by the driver with minimal attention, and when the fatigue state of the driver is "mental", which indicates that the driver is full at the time, the screen projection information can be projected to a screen (such as a central control screen) which can be seen by the driver with slight head twist.

In the above embodiment of the application, the screen projection information of the application program to be projected on the terminal device is selected to be projected on different display modules based on different fatigue states of the driver, so that the driving safety is improved.

In one possible design of the first aspect, the travel information of the target vehicle may include: the distance between the target vehicle and the obstacle. The first screen projecting condition includes that the distance between the target vehicle and the obstacle is within a first distance range. It should be noted that, in the embodiment of the present application, the distance between the vehicle and the obstacle may be calculated based on the perception information collected by the sensor (e.g., laser radar, camera, etc.) disposed on the vehicle, and according to the collected perception information, the distance between the vehicle and each obstacle and the orientation and position of the obstacle relative to the vehicle may be obtained. The difference in distance between the vehicle and the obstacle is also substantial for driving safety, and therefore one principle that should be followed is: when the distance between the vehicle and the obstacle is shorter, the reaction time left for the driver is shorter, the risk that the vehicle collides with the obstacle is higher, the driving environment of the vehicle is unsafe, the driver needs to pay attention to careful driving at the moment so as not to collide with the obstacle, and screen projection information can be projected to a screen (such as a steering wheel screen, a HUD and the like) which can be seen by the driver with the least attention; when the distance between the vehicle and the obstacle is longer, the driving environment of the vehicle is safer, and the screen projection information can be projected to a screen (such as a central control screen) which can be seen by a driver by slightly twisting his head.

In the above embodiment of the application, the screen projection information of the application program to be projected on the terminal device is selected to be projected to different display modules based on different distances between the vehicle and the obstacle, so that the probability of collision between the vehicle and the obstacle is reduced, and the driving safety is improved.

In a possible design of the first aspect, the feature information of the application to be screen-projected may include an application type to which the application to be screen-projected belongs, that is, the screen-projecting condition is that the specific application type corresponds to the specific display module, in this case, the meeting of the first screen-projecting condition that the first display module is screen-projected on the vehicle by the first screen-to-be-projected application on the terminal device includes: the first screen projection condition includes that the application type of the application program is a first application type (e.g., social class), and meanwhile, the first screen application to be projected on the terminal device belongs to the first application type, which may also be referred to as that the application type to which the first screen application to be projected on the terminal device belongs to an application type required in the first screen projection condition.

In the above embodiments of the present application, it is stated that the screen projection conditions are divided based on the application types to which the application programs belong, and different screen projection conditions correspond to different application types, so that the application programs belonging to different application types can correspondingly project screens to respective corresponding display modules, thereby realizing differential screen projection from the perspective of the application types and improving user experience.

In a possible design of the first aspect, the characteristic information of the application may include usage frequency information of the application, that is, the screen projection condition is that a specific usage frequency range of the application on the terminal device corresponds to a specific display module, and the usage frequency information of the application on the terminal device may be obtained from a function of "using a mobile phone healthily" of the terminal device, where the first screen projection condition that the first display module on the vehicle is projected by the first screen to be projected by the first screen application on the terminal device includes: the first screen projection condition includes that the use frequency information of the application program is in a first use frequency range (e.g., a high use frequency range), and meanwhile, the use frequency of the first application program to be projected on the terminal device is in the first use frequency range, which may also be referred to as that the use frequency of the first application program to be projected on the terminal device is in the use frequency range required in the first screen projection condition. In the embodiments of the present application, one principle to follow is: the more frequently used application's screen projection information is presented on the display module that the user needs to see with minimal cost or is most easily seen. For example: the screen projection information of the application program with high use frequency is presented on a steering wheel screen or a HUD of the vehicle, and the screen projection information of the application program with low use frequency is presented on a central control screen of the vehicle.

In the foregoing embodiment of the present application, it is stated that the screen projecting conditions are divided based on the use frequency information of the application program on the terminal device, and different screen projecting conditions correspond to different use frequency ranges, so that the application programs with different use frequencies can project screens to respective corresponding display modules, thereby achieving differentiated screen projection from the use frequency of the application programs, and the screen projecting information of the application program with higher use frequency is presented on the display module that the user can see or can see most easily with the least cost, thereby indirectly improving the driving attention of the user.

In a possible design of the first aspect, the feature information of the application may include both an application type to which the application belongs and usage frequency information of the application, in which case, the meeting of the first screen projection condition that the first display module is projected on the vehicle by the first screen projection application on the terminal device includes: the application type of the first screen-projection application program on the terminal device belongs to the application type required in the first screen projection condition, and the use frequency information of the first screen-projection application program on the terminal device is in the use frequency range required in the first screen projection condition.

In the above embodiment of the application, the single factors (for example, the application type is a single factor, and the use frequency information of the application program is a single factor) are combined with each other to obtain the division result based on the combined factor, so that the conditions required to be met by screen projection are more diversified, and the user experience is improved.

In a possible design of the first aspect, the frequency information used by the application on the terminal device may be characterized as follows: 1) the usage duration of the application program on the terminal device may be, for example, a total duration spent by the user when the application program runs on the terminal device since the application program is downloaded, or a duration spent by the user when the application program runs on the terminal device within a time period of one day, one week, or one month, and the specific calculation manner of the usage duration is not limited herein; 2) the number of times that the application program is started to run on the terminal device may be, for example, the number of times that the application program is started to run on the terminal device since the application program is downloaded by the user, or the number of times that the application program is started to run on the terminal device within a time period of one day, one week, or one month, and the specific way of calculating the number of times that the application program is started to run is not limited herein.

In the above embodiments of the present application, several characterization ways of using frequency information by an application on a terminal device are specifically set forth, and selectivity and flexibility are provided.

In a possible design of the first aspect, the application types to which the application programs belong may be divided in a plurality of ways, wherein one way of division may be to classify the application types into a social class, a travel navigation class, a video and audio class, a tool class, an information class, a game class, a travel accommodation class, a shopping class, an education class, a sports health class, and a financial management class, and any application program may be necessarily classified into one of the application types. This classification is based on that when the terminal device downloads the application programs from an application market, an App Store, etc. (different manufacturers of the terminal device, and different names may be used, and the application programs are not limited here), the application market already classifies the application programs (that is, the classification information determined by the application market label, and each application program downloaded from the application market has its own classification), that is, the above-mentioned classification.

In the above embodiments of the present application, a partition manner of an application type is specifically set forth, and the partition manner is easy to obtain, can cover most of application programs, and has realizability.

In a possible design of the first aspect, one application may simultaneously satisfy the feature information of the applications required in the plurality of screen projection conditions. For example, the application type "social class" (e.g., "WeChat", "QQ", "short message", etc.) to which the application belongs may be the application type required in the screen projection condition 1, or may be the application type required in the screen projection condition 2. That is to say, when the first screen-to-be-projected application on the terminal device simultaneously satisfies a first screen-projecting condition that the first display module on the vehicle is projected on the screen and a second screen-projecting condition that the second display module on the vehicle is projected on the screen, in this case, the screen-projecting information of the first screen-to-be-projected application can be simultaneously projected on the first display module and the second display module, and the first display module and the second display module are two different display modules on the vehicle. It should be noted that, in the embodiment of the present application, the precondition that the screen projection information of the first screen application to be projected is projected to the first display module and the second display module at the same time is that neither the first display module nor the second display module is occupied by the screen projection.

In the foregoing embodiment of the present application, it is specifically stated that when a certain application program meets the screen projection condition that a plurality of display modules are projected on a screen, the application program is projected on the plurality of display modules, and a user selects to view screen projection information on which display module, so that flexibility is provided.

In a possible design of the first aspect, one application may simultaneously satisfy the feature information of the applications required in the plurality of screen projection conditions. That is to say, when the first screen-projection application program to be projected on the terminal device simultaneously satisfies a first screen-projection condition that the first display module is projected on the vehicle and a second screen-projection condition that the second display module is projected on the vehicle, in this case, the terminal device may also divide the screen-projection information of the first screen-projection application program to be projected into first screen-projection sub-information and second screen-projection sub-information according to a preset manner, where the first display module and the second display module are two different ones of the plurality of display modules on the vehicle, and then, respectively project the first screen-projection sub-information and the second screen-projection sub-information to the first display module and the second display module.

In the foregoing embodiment of the present application, it is specifically stated that when a certain application program meets the screen projection condition that a plurality of display modules are projected with a screen, the screen projection information of the application program is correspondingly split into a plurality of screen projection sub-information, one screen projection sub-information is correspondingly projected onto one display module, and a user selects to view the screen projection sub-information on which display module by himself, so that flexibility is achieved.

In a possible design of the first aspect, one application may simultaneously satisfy the feature information of the applications required in the plurality of screen projection conditions. That is to say, when the first screen-projection application program to be screen-projected on the terminal device simultaneously satisfies a first screen-projection condition that the first display module on the vehicle is screen-projected and a second screen-projection condition that the second display module on the vehicle is screen-projected, then under the condition that the screen-projection priority of the first display module is higher than that of the second display module, screen-projection information of the first screen-projection application program to be screen-projected is only screen-projected to the first display module with the higher priority.

In the foregoing embodiments of the present application, it is specifically stated that when a certain application program meets a screen projection condition that a plurality of display modules are projected on a screen, selective screen projection may be further performed based on priority, and screen projection information is preferentially projected on a display module with a high priority. The mode of projecting the screen according to the priority ensures that the user can watch the screen projection information in an optimum mode in real time, and the influence of dispersed attention of the user is reduced to the maximum extent.

In a possible design of the first aspect, the screen projection information of the first application to be screen projected is projected to the second display module only when the first display module with the higher priority is occupied by screen projection.

In the above embodiments of the present application, it is specifically stated that only when a display module with a high priority is occupied by screen projection, the screen is projected onto a display module with a next priority, so that it is ensured that the user can view screen projection information in an optimum manner in real time.

In a possible design of the first aspect, the screen projection information of the first application to be screen projected may be an entire application interface of the first application or may be a part of the application interface of the first application to be screen projected.

In the above embodiments of the present application, it is described which components of the screen projection information may be, and flexibility is provided based on which part or parts of the display interface the user needs to selectively project.

In one possible design of the first aspect, the display module on the target vehicle may include at least: a center control screen, a dashboard screen, a steering wheel screen, a HUD, a front windshield screen (e.g., which may include a left view screen, a right view screen, etc. of the vehicle), a rear windshield screen, a copilot screen, a window screen (e.g., a left window screen, a right window screen, etc.), a sunroof screen, etc., or, a display module on the vehicle may further include: any surface area inside/outside the vehicle, such as a vehicle body, a vehicle door, a sun visor inside the vehicle, a skylight and the like, is wrapped and coated with a material with a display function (such as a flexible display screen). That is to say, in the embodiment of the present application, any area or device in the vehicle where the terminal device can project a screen may be referred to as a display module on the vehicle according to the embodiment of the present application.

In the above embodiments of the present application, several specific expressions of the display module of the vehicle are described, and the display module has universality and flexibility.

A second aspect of the embodiments of the present application further provides a display information switching method, where the method includes: the method comprises the steps that firstly, the vehicle can obtain running information of the vehicle, then, the vehicle judges whether the running information of the vehicle accords with a first running condition, when the running information of the vehicle does not accord with the first running condition, screen projection information of a first application program displayed on a first display module is indicated to be switched to a second display module to be displayed, wherein the first display module and the second display module are two different display modules on the vehicle respectively, and the first running condition is a condition which needs to be met when the first display module is projected.

In the above embodiment of the application, after the screen projection information on the terminal device has been projected to the first display module at the vehicle end, the vehicle can further determine, by combining with the driving information of the vehicle itself, whether the first screen projection condition corresponding to the first display module meets the driving condition of the vehicle itself, and if not, the screen projection information needs to be switched to other display modules (for example, the second display module) on the vehicle for display, so that flexibility is provided, and the user experience is improved.

It should be noted that, in some embodiments of the present application, the information displayed on the first display module may be not only the screen-projection information of the first application, but also other information, for example, an operation interface of an application installed in the vehicle itself, or a part of the operation interface, which may be collectively referred to as target information, and in this case, the information displayed on the first display module is not screen-projection information, in which case, one way to implement the screen-cutting is: the method comprises the steps that a vehicle obtains driving information of the vehicle, whether the driving information of the vehicle is in accordance with a first driving condition or not is judged, when the driving information of the vehicle is not in accordance with the first driving condition, target information displayed on a first display module is switched to a second display module to be displayed, wherein the first display module and the second display module are two different display modules on the vehicle respectively, and the first driving condition is a condition which needs to be met when the first display module is projected.

In a possible design of the second aspect, when the driving information of the vehicle does not meet the first driving condition, a specific manner of instructing to switch the screen projection information of the first application program displayed on the first display module to be displayed on the second display module may be: when the running information of the vehicle is not accordant with the first running condition and the running information of the vehicle is accordant with the second running condition, switching screen projection information of the first application program displayed on the first display module to the second display module to be displayed, wherein the second running condition is a condition which needs to be met when the second display module is projected.

In the above embodiment of the present application, after the screen projection information on the terminal device has been projected to the first display module at the vehicle end, the vehicle can further judge whether screen switching display is needed in different driving scenes by combining the driving information of the vehicle, only the first screen projection condition corresponding to the first display module is not accordant with the current driving condition of the vehicle, and the other display modules (e.g., the second display module) indicate that the screen projection information of the first application program displayed on the first display module is switched to the second display module to be displayed on the second display module on the premise that the current driving condition of the vehicle is met, that is, the dynamic switching can be carried out between different display modules by combining the driving state of the vehicle, so that the user can watch the screen projection information in the best mode under any driving condition, and the user experience is improved.

In one possible design of the second aspect, the driving information of the vehicle may include: the driving speed of the vehicle, in which case the travel information of the vehicle does not comply with the first travel condition and the travel information of the vehicle complies with the second travel condition, includes: the first driving condition includes a driving speed being within a first driving speed range, and the second driving condition includes a driving speed being within a second driving speed range; the driving speed of the vehicle is not within the first driving speed range, and the driving speed of the vehicle is within the second driving speed range. It should be noted that, since the driving safety is substantially aimed at by the different driving speeds, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect specifically, which is not described herein.

In the above embodiment of the application, based on the driving speed of the vehicle, the screen projection information of the application program running on the terminal device is switched to the display module corresponding to the driving condition that meets the driving speed of the vehicle for display, so that the display information on the terminal device is reasonably distributed to different display modules on the vehicle according to the real-time driving speed of the vehicle, and the driving safety is improved.

In one possible design of the second aspect, the driving information of the vehicle may include: the type of traffic scene the vehicle is in. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition comprises that the traffic scene type belongs to a first traffic scene type, and the second driving condition comprises that the traffic scene type belongs to a second traffic scene type; the traffic scene type of the vehicle does not belong to the first traffic scene type, and the traffic scene type of the vehicle belongs to the second traffic scene type. It should be noted that, in the embodiment of the present application, no limitation is made on what principle the traffic scene types are divided based on, and specific reference may be made to the description of the first aspect, which is not described herein again; in addition, since the traffic scene type of the vehicle is substantially specific to driving safety, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not repeated herein.

In the embodiment of the application, the screen projection information of the application program running on the terminal device is switched to the display module corresponding to the driving condition according with the traffic scene type of the vehicle based on the different traffic scene types of the vehicle, so that the display information on the terminal device is reasonably distributed to the different display modules on the vehicle according to the real-time traffic scene type of the vehicle, and the driving safety is improved.

In one possible design of the second aspect, the driving information of the vehicle may include: number of passengers in the vehicle. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition includes that the number of passengers is within a first passenger number range, and the second driving condition includes that the number of passengers is within a second passenger number range; the number of passengers in the vehicle is not within the first range of passenger numbers and the number of passengers in the vehicle is within the second range of passenger numbers. It should be noted that in the embodiment of the present application, the number of passengers in the vehicle is not included in the driver, for example, when only the driver is in the vehicle, the number of passengers in the vehicle is 0. The essential point of the difference of the number of passengers in the vehicle is privacy and commonality of screen projection information, when an application program running on a terminal device belongs to an application program with high privacy (such as WeChat), in order to avoid other passengers from seeing screen projection information, the application program with high privacy can be projected to a dashboard screen or a steering wheel screen only seen by a driver, and when the application program running on the terminal device belongs to an application program with low privacy (such as Tencent video), the application program with low privacy can be projected to a central control screen seen by both the driver and the passengers in consideration of the requirement of common use.

In the above embodiment of the application, based on the consideration of privacy and commonality of the application program, the number of passengers in the vehicle is different, the screen projection information of the application program running on the terminal device can be switched to the display module corresponding to the driving condition in accordance with the number of passengers in the vehicle for display, so that the display information on the terminal device can be reasonably distributed to different display modules on the vehicle according to the number of passengers in the vehicle, and the user experience is improved.

In one possible design of the second aspect, the driving information of the vehicle may include: fatigue state of the vehicle driver. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition includes that the fatigue state of the driver belongs to a first fatigue state, and the second driving condition includes that the fatigue state of the driver belongs to a second fatigue state; the fatigue state of the driver of the vehicle does not belong to the first fatigue state, and the fatigue state of the driver of the vehicle belongs to the second fatigue state. It should be noted that, in the embodiment of the present application, the fatigue state of the driver of the vehicle may be calculated from human body state information detected by a sensor module disposed in the vehicle, or may be calculated from human body state information detected by a wearable device (e.g., a smart band, a smart watch, etc.) worn by the driver. In some embodiments of the present application, the fatigue status of the driver may be classified according to the number of display modules of the vehicle, for example, assuming that there are 3 display modules on the vehicle, the fatigue status of the driver may be classified into 3 grades of "fatigue", "sub-fatigue", and "mental"; assuming that there are 2 display modules on the vehicle, the fatigue state of the driver can be divided into 2 grades of fatigue and spirit, and the corresponding driving behavior can be known to be safe or not based on different fatigue states of the driver. Since the driving speed is substantially directed to driving safety, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not described herein.

In the above embodiment of the application, based on different fatigue states of the driver, the screen projection information of the application running on the terminal device is switched to the display module corresponding to the driving condition that meets the fatigue state of the driver of the vehicle for display, so that the display information on the terminal device is reasonably distributed to different display modules on the vehicle according to the fatigue state of the driver of the vehicle, and the driving safety is improved.

In one possible design of the second aspect, the driving information of the vehicle may include: the distance between the vehicle and the obstacle. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition includes a distance between the vehicle and the obstacle being within a first distance range, and the second driving condition includes a distance between the vehicle and the obstacle being within a second distance range; the distance between the vehicle and an obstacle is not within the first distance range and the distance between the vehicle and an obstacle is within the second distance range. It should be noted that, in the embodiment of the present application, the distance between the vehicle and the obstacle may be calculated based on the perception information collected by the sensor (e.g., laser radar, camera, etc.) disposed on the vehicle, and according to the collected perception information, the distance between the vehicle and each obstacle and the orientation and position of the obstacle relative to the vehicle may be obtained. The difference in the distance between the vehicle and the obstacle is also for driving safety, and therefore, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not described herein.

In the above embodiment of the application, the screen projection information of the application running on the terminal device is switched to the display module corresponding to the driving condition corresponding to the distance between the vehicle and the obstacle based on the different distances between the vehicle and the obstacle, so that the display information on the terminal device is reasonably distributed to the different display modules on the vehicle according to the distance between the vehicle and the obstacle, the probability of collision between the vehicle and the obstacle is reduced, and the driving safety is improved.

In a possible design of the second aspect, the process that the vehicle switches the screen projection information to the second display module for displaying may specifically be: the method comprises the steps that a vehicle generates a switching instruction (which can be called as a first switching instruction) firstly, the first switching instruction is used for instructing the vehicle to switch screen projection information to a second display module, and then the vehicle switches the screen projection information to the second display module for displaying according to the first switching instruction.

In the above embodiments of the present application, how to perform screen-cut display is specifically described, and the realizability is provided.

In a possible design of the second aspect, the screen projection information of the first application may be the entire application interface of the first application or a part of the application interface of the first application.

In one possible design of the second aspect, the display module on the vehicle may include at least: a center control screen, a dashboard screen, a steering wheel screen, a HUD, a front windshield screen (e.g., which may include a left view screen, a right view screen, etc. of the vehicle), a rear windshield screen, a copilot screen, a window screen (e.g., a left window screen, a right window screen, etc.), a sunroof screen, etc., or, a display module on the vehicle may further include: any surface area inside/outside the vehicle, such as a vehicle body, a vehicle door, a sun visor inside the vehicle, a skylight and the like, is wrapped and coated with a material with a display function (such as a flexible display screen). That is to say, in the embodiment of the present application, any area or device in the vehicle where the terminal device can project a screen may be referred to as a display module on the vehicle according to the embodiment of the present application.

The third aspect of the embodiments of the present application further provides a screen projection method, including: the terminal device firstly sends a first instruction to the vehicle, the first instruction is used for instructing a first display module of the vehicle to display screen projection information of a first application program, in addition, the terminal device also periodically receives driving information of the vehicle sent by the vehicle, when the running information of the vehicle is not accordant with the first running condition and the running information of the vehicle is accordant with the second running condition, a second instruction is sent to the vehicle, the second instructions are for instructing a second display module of the vehicle to display information of the first application (e.g., a running interface of the first application, or, a first portion of the running interface), the first driving condition is a condition which needs to be met when the first display module is projected, the second driving condition is a condition which needs to be met when the second display module is projected, and the first display module and the second display module are two different display modules on the vehicle.

In the above embodiment of the application, after the screen projection information on the terminal device has been projected to the display module at the vehicle end, the terminal device may further determine whether screen cutting display needs to be performed in different driving scenes by combining with the driving information of the vehicle itself, that is, dynamic switching may be performed between different display modules by combining with the driving state of the vehicle itself by the terminal device, so that the user can view the screen projection information in an optimal manner under any driving condition, thereby improving user experience.

In one possible design of the third aspect, the travel information of the vehicle may include: the driving speed of the vehicle, in which case the travel information of the vehicle does not comply with the first travel condition and the travel information of the vehicle complies with the second travel condition, includes: the first driving condition includes a driving speed being within a first driving speed range, and the second driving condition includes a driving speed being within a second driving speed range; the driving speed of the vehicle is not within the first driving speed range, and the driving speed of the vehicle is within the second driving speed range. Since the driving safety is substantially aimed at by different driving speeds, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not described herein.

In one possible design of the third aspect, the travel information of the vehicle may include: the type of traffic scene the vehicle is in. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition comprises that the traffic scene type belongs to a first traffic scene type, and the second driving condition comprises that the traffic scene type belongs to a second traffic scene type; the traffic scene type of the vehicle does not belong to the first traffic scene type, and the traffic scene type of the vehicle belongs to the second traffic scene type. It should be noted that, in the embodiment of the present application, no limitation is made on what principle the traffic scene types are divided based on, and specific reference may be made to the description of the first aspect, which is not described herein again; in addition, since the traffic scene type of the vehicle is substantially specific to driving safety, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not repeated herein.

In one possible design of the third aspect, the travel information of the vehicle may include: number of passengers in the vehicle. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition includes that the number of passengers is within a first passenger number range, and the second driving condition includes that the number of passengers is within a second passenger number range; the number of passengers in the vehicle is not within the first range of passenger numbers and the number of passengers in the vehicle is within the second range of passenger numbers. It should be noted that in the embodiment of the present application, the number of passengers in the vehicle is not included in the driver, for example, when only the driver is in the vehicle, the number of passengers in the vehicle is 0. The essential point of the difference of the number of passengers in the vehicle is privacy and commonality of screen projection information, when an application program running on a terminal device belongs to an application program with high privacy (such as WeChat), in order to avoid other passengers from seeing screen projection information, the application program with high privacy can be projected to a dashboard screen or a steering wheel screen only seen by a driver, and when the application program running on the terminal device belongs to an application program with low privacy (such as Tencent video), and considering the requirement of common use, the application program with low privacy can be projected to a central control screen seen by both the driver and the passengers during running.

In the embodiment of the application, based on the consideration of privacy and commonality of the application programs, the number of passengers in the vehicle is different, the screen projection information of the application program running on the terminal equipment can be selected to be switched to the display module corresponding to the driving condition which is in accordance with the number of passengers in the vehicle for display, so that the display information on the terminal equipment is reasonably distributed to different display modules on the vehicle according to the number of passengers in the vehicle, and the user experience is improved.

In one possible design of the third aspect, the travel information of the vehicle may include: fatigue state of the vehicle driver. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition includes that the fatigue state of the driver belongs to a first fatigue state, and the second driving condition includes that the fatigue state of the driver belongs to a second fatigue state; the fatigue state of the driver of the vehicle does not belong to the first fatigue state, and the fatigue state of the driver of the vehicle belongs to the second fatigue state. It should be noted that, in the embodiment of the present application, the fatigue state of the driver of the vehicle may be calculated from human body state information detected by a sensor module disposed in the vehicle, or may be calculated from human body state information detected by a wearable device (e.g., a smart band, a smart watch, etc.) worn by the driver. In some embodiments of the present application, the fatigue states of the drivers may be classified, the driver may be classified according to the number of the vehicle display modules, and whether the corresponding driving behavior is safe or not may be known based on the different fatigue states of the drivers. Since the driving speed is substantially directed to driving safety, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not described herein.

In one possible design of the third aspect, the travel information of the vehicle may include: the distance between the vehicle and the obstacle. In this case, the matching of the travel information of the vehicle with the first travel condition and the matching of the travel information of the vehicle with the second travel condition includes: the first driving condition includes a distance between the vehicle and the obstacle being within a first distance range, and the second driving condition includes a distance between the vehicle and the obstacle being within a second distance range; the distance between the vehicle and an obstacle is not within the first distance range and the distance between the vehicle and an obstacle is within the second distance range. It should be noted that, in the embodiment of the present application, the distance between the vehicle and the obstacle may be calculated based on the perception information collected by the sensor (e.g., laser radar, camera, etc.) disposed on the vehicle, and according to the collected perception information, the distance between the vehicle and each obstacle and the orientation and position of the obstacle relative to the vehicle may be obtained. The difference in the distance between the vehicle and the obstacle is also for driving safety, and therefore, the principle to be followed is similar to that of the first aspect, and please refer to the description of the first aspect, which is not described herein.

In a possible design of the third aspect, the screen projection information of the first application may be the entire application interface of the first application or may be a part of the application interface of the first application.

In one possible design of the third aspect, the display module on the vehicle may include at least: a center control screen, a dashboard screen, a steering wheel screen, a head-up display (HUD), a front windshield screen (e.g., which may include a left view screen, a right view screen, etc. of the vehicle), a rear windshield screen, a copilot screen, a window screen (e.g., a left window screen, a right window screen, etc.), a sunroof screen, etc., or alternatively, a display module on the vehicle may further include: any surface area inside/outside the vehicle, such as a vehicle body, a vehicle door, a sun visor inside the vehicle, a skylight and the like, is wrapped and coated with a material with a display function (such as a flexible display screen). That is to say, in the embodiment of the present application, any area or device in the vehicle where the terminal device can project a screen may be referred to as a display module on the vehicle according to the embodiment of the present application.

A fourth aspect of embodiments of the present application provides an apparatus having functionality to implement the method of the first aspect or any one of the possible implementation manners of the first aspect. The function can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above. In some embodiments of the present application, the apparatus may be a terminal device or a vehicle, and is not limited herein.

A fifth aspect of embodiments of the present application provides an apparatus having functionality to implement the method of any one of the second aspect or possible implementation manners of the second aspect. The function can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above. In some embodiments of the present application, the device may be a vehicle.

A sixth aspect of the embodiments of the present application further provides an apparatus having a function of implementing the method according to any one of the above third aspect and the third aspect. The function can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above. In some embodiments of the present application, the apparatus may be a terminal device.

A seventh aspect of the present embodiment provides an apparatus, which may include a memory, a processor, and a bus system, where the memory is configured to store a program, and the processor is configured to call the program stored in the memory to execute the method according to the first aspect or any one of the possible implementations of the first aspect of the present embodiment, or the processor is configured to call the program stored in the memory to execute the method according to the second aspect or any one of the possible implementations of the second aspect of the present embodiment, or the processor is configured to call the program stored in the memory to execute the method according to any one of the possible implementations of the third aspect or the third aspect of the present embodiment.

An eighth aspect of the present application provides a computer-readable storage medium, which stores instructions that, when executed on a computer, enable the computer to perform the method of the first aspect or any one of the possible implementations of the first aspect, or enable the computer to perform the method of the second aspect or any one of the possible implementations of the second aspect, or enable the computer to perform the method of any one of the possible implementations of the third aspect.

A ninth aspect of embodiments of the present application provides a computer program, which when run on a computer, causes the computer to perform the method of any one of the above-mentioned first aspect or first possible implementation manner, or causes the computer to perform the method of any one of the above-mentioned second aspect or second possible implementation manner, or causes the computer to perform the method of any one of the above-mentioned third aspect or third possible implementation manner.

Drawings

FIG. 1 is a schematic view of a driver using a utility screen and terminal equipment in a vehicle;

fig. 2 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present application;

fig. 3 is a schematic diagram of a software structure of a terminal device according to an embodiment of the present application;

fig. 4 is a block diagram of several exemplary terminal devices provided in the embodiments of the present application;

fig. 5 is a block diagram of a foldable mobile phone provided in an embodiment of the present application in an unfolded state;

fig. 6 is a block diagram of a folding mobile phone provided in an embodiment of the present application in a folded state;

FIG. 7 is a schematic illustration of various display modules within a vehicle according to an embodiment of the present application;

FIG. 8 is a schematic flowchart of a screen projection method according to an embodiment of the present application;

Fig. 9 is a schematic diagram of one manner in which a terminal device obtains multiple screen projection conditions for multiple display modules projected on a vehicle according to an embodiment of the present application;

fig. 10 is a schematic diagram illustrating that a user selects to acquire identification information corresponding to a display module of a target vehicle on a terminal device according to an embodiment of the present application;

fig. 11 is a schematic diagram of another way for acquiring, by a terminal device according to an embodiment of the present application, a plurality of screen projection conditions for a plurality of display modules projected on a vehicle;

fig. 12 is another schematic diagram illustrating that a user selects to acquire identification information corresponding to a display module of a target vehicle on a terminal device according to an embodiment of the present application;

fig. 13 is a schematic diagram illustrating adding a name of an application running on a terminal device to a target screen-casting condition according to an embodiment of the present application;

fig. 14 is a schematic diagram illustrating that a terminal device adds an application name running on the terminal device to a target screen-casting condition in a voice prompt manner according to the embodiment of the present application;

fig. 15 is a schematic view of a screen projection method provided in an embodiment of the present application;

FIG. 16 is a schematic view of another screen projection method provided by an embodiment of the present application;

FIG. 17 is a schematic view of another screen projection method provided in the embodiments of the present application;

FIG. 18 is a schematic view of another screen projection method provided in the embodiments of the present application;

FIG. 19 is a schematic view of another screen projection method provided by an embodiment of the present application;

FIG. 20 is a schematic diagram illustrating the relationship between various screen projection conditions provided by an embodiment of the present application;

FIG. 21 is another schematic diagram illustrating the relationship between various screen projection conditions provided by an embodiment of the present application;

FIG. 22 is another illustration of the relationship between various screen projection conditions provided by an embodiment of the present application;

FIG. 23 is a schematic view of another screen projection method provided in the embodiments of the present application;

FIG. 24 is a schematic view of another screen projection method provided in the embodiments of the present application;

FIG. 25 is a schematic view of another screen projection method provided in the embodiments of the present application;

fig. 26 is another schematic flow chart of a display information switching method according to an embodiment of the present application;

FIG. 27 is a schematic view of another screen projection method provided in the embodiments of the present application;

fig. 28 is a schematic view of a traffic scene at an intersection according to an embodiment of the present application;

FIG. 29 is a schematic view of a vehicle traveling on a highway spaced from surrounding obstacles according to an exemplary embodiment of the present disclosure;

fig. 30 is another schematic flow chart of a display information switching method according to an embodiment of the present application;

FIG. 31 is a schematic structural diagram of an apparatus according to an embodiment of the present application;

FIG. 32 is a schematic view of another embodiment of the apparatus of the present application;

FIG. 33 is a schematic view of another embodiment of the apparatus of the present application;

fig. 34 is a schematic structural diagram of a vehicle according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely descriptive of the various embodiments of the application and how objects of the same nature can be distinguished. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

The screen projection method provided by the embodiment of the application can be applied to various terminal devices, and various different types of terminal devices can be used to implement the terminal device provided by the application, for example, the terminal device in the application may include but is not limited to: smart phones (e.g., cell phones), laptop computers (laptops), Personal Computers (PCs), tablet computers, ultrabooks, wearable devices (e.g., smartband, smartwatch, smart glasses, Head Mounted Display (HMD), etc.), Augmented Reality (AR) devices, Mixed Reality (MR) devices, cellular phones (cellular phones), Personal Digital Assistants (PDAs), digital broadcast terminals, etc. Of course, in the following embodiments, the specific form of the terminal device is not limited at all.

The structure of the terminal device provided in the present application is exemplified below by taking a specific structure as an example, only by way of non-limiting example. Referring specifically to fig. 2, fig. 2 shows a schematic structural diagram of the terminal device 100.

The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a Subscriber Identification Module (SIM) card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the terminal device 100. In other embodiments of the present application, terminal device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The I2C interface is a bi-directional synchronous serial bus that includes a serial data line (SDA) and a Serial Clock Line (SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces, respectively. For example: the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through an I2C bus interface to implement the touch function of the terminal device 100.

The I2S interface may be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may communicate audio signals to the wireless communication module 160 via the I2S interface, enabling answering of calls via a bluetooth headset.

The PCM interface may also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled by a PCM bus interface. In some embodiments, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through a bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 with the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In some embodiments, the audio module 170 may transmit the audio signal to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a bluetooth headset.

MIPI interfaces may be used to connect processor 110 with peripheral devices such as display screen 194, camera 193, and the like. The MIPI interface includes a Camera Serial Interface (CSI), a Display Serial Interface (DSI), and the like. In some embodiments, processor 110 and camera 193 communicate through a CSI interface to implement the capture function of terminal device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the terminal device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In some embodiments, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the terminal device 100, and may also be used to transmit data between the terminal device 100 and a peripheral device. And the earphone can also be used for connecting an earphone and playing audio through the earphone. The interface may also be used to connect other terminal devices, such as AR devices and the like.

It should be understood that the interface connection relationship between the modules according to the embodiment of the present invention is only an exemplary illustration, and does not limit the structure of the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the terminal device 100. The charging management module 140 may also supply power to the terminal device through the power management module 141 while charging the battery 142.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, and battery state of health (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The

antennas

1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in terminal device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied on the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional modules, independent of the processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the terminal device 100, including Wireless Local Area Networks (WLANs), such as wireless fidelity (Wi-Fi) networks, Bluetooth (BT), Global Navigation Satellite Systems (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150 and the antenna 2 is coupled to the wireless communication module 160 so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), code division multiple access (code division multiple access, CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), Long Term Evolution (LTE), LTE, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).

The terminal device 100 implements a display function by the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, video, and the like. The display screen 194 includes a display panel. The display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), and the like. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The terminal device 100 may implement a shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the terminal device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform fourier transform or the like on the frequency point energy.

Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record video in a plurality of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. The NPU can implement applications such as intelligent recognition of the terminal device 100, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to extend the storage capability of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 121 may be used to store computer-executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like. The storage data area may store data (such as audio data, a phonebook, etc.) created during use of the terminal device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like. The processor 110 executes various functional applications of the terminal device 100 and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The terminal device 100 may implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called a "horn", is used to convert the audio electrical signal into an acoustic signal. The terminal device 100 can listen to music through the speaker 170A, or listen to a handsfree call.

The receiver 170B, also called "earpiece", is used to convert the electrical audio signal into an acoustic signal. When the terminal device 100 answers a call or voice information, it is possible to answer a voice by bringing the receiver 170B close to the human ear.

The microphone 170C, also referred to as a "microphone," is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can input a voice signal to the microphone 170C by speaking the user's mouth near the microphone 170C. The terminal device 100 may be provided with at least one microphone 170C. In other embodiments, the terminal device 100 may be provided with two microphones 170C, which may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the terminal device 100 may further include three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The headphone interface 170D is used to connect a wired headphone. The headset interface 170D may be the USB interface 130, or may be an Open Mobile Terminal Platform (OMTP) standard interface of 3.5mm, or a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used for sensing a pressure signal, and converting the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a wide variety, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure from the change in the capacitance. When a touch operation is applied to the display screen 194, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The terminal device 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In some embodiments, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.

The gyro sensor 180B may be used to determine the motion attitude of the terminal device 100. In some embodiments, the angular velocity of terminal device 100 about three axes (i.e., x, y, z axes) may be determined by gyroscope sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. Illustratively, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the terminal device 100, calculates the distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the terminal device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates an altitude from the barometric pressure measured by the barometric pressure sensor 180C, and assists in positioning and navigation.

The magnetic sensor 180D includes a hall sensor. The terminal device 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the terminal device 100 is a folder, the terminal device 100 may detect the opening and closing of the folder according to the magnetic sensor 180D. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.

The acceleration sensor 180E can detect the magnitude of acceleration of the terminal device 100 in various directions (generally, three axes). The magnitude and direction of gravity can be detected when the terminal device 100 is stationary. The method can also be used for recognizing the posture of the terminal equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.

A distance sensor 180F for measuring a distance. The terminal device 100 may measure the distance by infrared or laser. In some embodiments, shooting a scene, the terminal device 100 may range using the distance sensor 180F to achieve fast focus.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light to the outside through the light emitting diode. The terminal device 100 detects infrared reflected light from a nearby object using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can utilize the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear for talking, so as to automatically turn off the screen to achieve the purpose of saving power. The proximity light sensor 180G may also be used in a holster mode, a pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the ambient light level. The terminal device 100 may adaptively adjust the brightness of the display screen 194 according to the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket, in order to prevent accidental touches.

The fingerprint sensor 180H is used to collect a fingerprint. The terminal device 100 can utilize the collected fingerprint characteristics to realize fingerprint unlocking, access to an application lock, fingerprint photographing, fingerprint incoming call answering and the like.

The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 executes a temperature processing policy using the temperature detected by the temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the terminal device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the terminal device 100 heats the battery 142 when the temperature is below another threshold to avoid the terminal device 100 being abnormally shut down due to low temperature. In other embodiments, when the temperature is lower than a further threshold, the terminal device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K is also called a "touch device". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on the surface of the terminal device 100, different from the position of the display screen 194.

The bone conduction sensor 180M may acquire a vibration signal. In some embodiments, the bone conduction sensor 180M may acquire a vibration signal of the human vocal part vibrating the bone mass. The bone conduction sensor 180M may also contact the human pulse to receive the blood pressure pulsation signal. In some embodiments, the bone conduction sensor 180M may also be disposed in a headset, integrated into a bone conduction headset. The audio module 170 may analyze a voice signal based on the vibration signal of the bone mass vibrated by the sound part acquired by the bone conduction sensor 180M, so as to implement a voice function. The application processor can analyze heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power-on key, a volume key, and the like. The keys 190 may be mechanical keys. Or may be touch keys. The terminal device 100 may receive a key input, and generate a key signal input related to user setting and function control of the terminal device 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration cues, as well as for touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also respond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenarios (e.g., time alert, received message, alarm clock, game, etc.) may also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be brought into and out of contact with the terminal device 100 by being inserted into the SIM card interface 195 or being pulled out of the SIM card interface 195. The terminal device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. The same SIM card interface 195 can be inserted with multiple cards at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as communication and data communication. In some embodiments, the terminal device 100 employs eSIM, namely: an embedded SIM card. The eSIM card may be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

The software system of the terminal device 100 may adopt a hierarchical architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes an Android system with a layered architecture as an example, and exemplarily illustrates a software structure of the terminal device 100. Referring to fig. 3, fig. 3 is a schematic diagram of a software structure of a terminal device 100 according to an embodiment of the present application.

The layered architecture divides the software into a plurality of layers, each layer has clear roles and division of labor, and the layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are an application layer, an application framework layer, an Android Runtime (Android Runtime) system library, and a kernel layer from top to bottom.

The application layer may include a series of application packages, that is, the application described in this embodiment, as shown in fig. 3, the application packages may include: camera, gallery, calendar, talk, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions. As shown in FIG. 3, the application framework layers may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

The window manager is used for managing window programs, and can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and answered, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views, for example, may include a display interface of a short message notification icon, and may also include a view for displaying text and a view for displaying pictures.

The telephone manager is used to provide communication functions of the terminal device 100, such as management of call states (including connection, hang-up, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so forth.

The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text in the top status bar of the system, such as a notification of a background running application, or a notification that appears in the form of a dialog window on the screen, such as prompting a text message in the status bar, sounding a prompt tone, vibrating the terminal device, flashing an indicator light, etc.

The Android Runtime comprises a core library and a virtual machine, and is responsible for scheduling and managing an Android system. The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application program layer and the application program framework layer run in the virtual machine, the virtual machine executes java files of the application program layer and the application program framework layer into binary files, and the virtual machine is used for executing the functions of management of object life cycles, stack management, thread management, safety and exception management, garbage recovery and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), media libraries (media libraries), three-dimensional graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording in a variety of commonly used audio and video formats, as well as still image files, and the like. The media library may support a variety of audio-video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, composition, layer processing and the like. The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software, and at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The following exemplarily illustrates the workflow of the software and hardware of the terminal device 100 in connection with capturing a photo scene:

when the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, a time stamp of the touch operation, and other information). The raw input events are stored at the kernel layer. And the application program framework layer acquires the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the example that the touch operation is a touch click operation and the control corresponding to the click operation is a control of a camera application icon, an application program camera calls an interface of an application framework layer to start a camera application, and then starts a camera drive by calling a kernel layer, and captures a still image or a video by a camera 193.

Since the screen projection method provided by the embodiment of the present application can be applied to various types of terminal devices, the following takes several block diagrams of typical terminal devices as examples to illustrate the terminal device provided by the embodiment of the present application, as shown in fig. 4, (a) a sub-diagram and (b) a sub-diagram of fig. 4 respectively illustrate block diagrams of a smart phone and a smart watch, and both a hardware structure of the terminal device corresponding to fig. 2 and a software structure corresponding to fig. 3 can be mounted on the block diagram of the terminal device of fig. 4, so as to implement the screen projection method provided by the embodiment of the present application.

In addition, the screen projection method provided by the embodiment of the present application may be applied to various foldable terminal devices, such as a foldable mobile phone, a foldable personal computer, a foldable tablet computer, and foldable glasses, and the foldable mobile phone is taken as an example below to illustrate the foldable terminal device provided by the embodiment of the present application, as shown in fig. 5, fig. 5 illustrates a block diagram of the smart foldable mobile phone in an unfolded state, a screen of the foldable mobile phone is a flexible screen, and a user can view videos and images through the flexible screen and input touch events on the flexible screen, and after using the foldable mobile phone, the user can fold the foldable mobile phone to facilitate carrying, as shown in fig. 6, fig. 6 illustrates a folded state of the foldable mobile phone obtained by folding the foldable mobile phone through a rotating device.

It should be noted that fig. 5 and 6 illustrate one of the unfolding and folding modes, i.e. folding or unfolding by the rotating device handling the central shaft of the folding handset, in some embodiments of the present application, the rotating device may be at any position of the folding handset, and the folding device may be more than one, and the folding mode is not limited in this application.

The screen projection method provided by the embodiment of the application is to project screen projection information of an application program running on a terminal device to a display module on a vehicle (such as a common vehicle, an intelligent vehicle, an automatic driving vehicle, an internet vehicle and the like) meeting corresponding screen projection conditions based on screen projection conditions, because the display modules on the vehicle are different due to factors such as vehicles produced by different vehicle enterprises and different vehicle types, in the embodiment of the application, the display module on the vehicle can be display equipment installed or placed at any position of the vehicle, as shown in fig. 7, fig. 7 shows an example that various display modules are concentrated on one vehicle, for convenience of understanding, fig. 7 shows the display modules deployed on the vehicle from different view angles, and the display modules on the vehicle at least can include: a center control screen 101, a dashboard screen 102, a steering wheel screen 103, a head-up display (HUD) 104, a front windshield screen 105 (which may include, for example, a left view screen 1051, a right view screen 1052, etc. of the vehicle), a rear windshield screen 106, a copilot screen 107, a window screen (e.g., a left window screen 108, a right window screen 109, etc.), a sunroof screen 110, etc., or the display module on the vehicle may further include: any surface area inside/outside the vehicle, such as a vehicle body, a vehicle door, a sun visor inside the vehicle, a skylight and the like, is wrapped and coated with a material with a display function (such as a flexible display screen). That is to say, in the embodiment of the present application, any area or device in the vehicle where the terminal device can project a screen may be referred to as a display module on the vehicle according to the embodiment of the present application.

In addition, it should be further noted that, in some embodiments of the present application, the display interface corresponding to the display module on the vehicle may be an interface of any shape, for example: rectangular (as shown in fig. 7 for some display modules), square, circular, oval, trapezoid, and any irregular shape, and the display interface shape of the display module is not limited herein.

In some embodiments of the present application, the vehicle may be a car (as shown in fig. 7), a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a recreational vehicle, a playground vehicle, construction equipment, an electric train, a golf cart, a train, etc., and the examples of the present application are not particularly limited. Therefore, the display module on the vehicle according to the embodiment of the present application may be a display module on any one of the above-mentioned types of vehicles.

After the description of the terminal device and the display module on the vehicle described in the embodiment of the present application is completed, the screen projection method provided in the embodiment of the present application is described below, it should be noted that the following embodiments can be performed based on the terminal device described in fig. 2 to fig. 6, specifically refer to fig. 8, and fig. 8 is a schematic flow diagram of the screen projection method provided in the embodiment of the present application, and specifically includes the following steps:

801. The method comprises the steps of obtaining a first screen projection condition of a first display module projected on a vehicle, wherein the first screen projection condition is related to running information of a target vehicle, or the first screen projection condition is related to characteristic information of an application program to be projected.

First, the terminal device obtains a first screen projecting condition that a first display module on the vehicle is projected on, where the first screen projecting condition is related to driving information of a target vehicle (that is, the first screen projecting condition may be set according to the driving information of the target vehicle), or the first screen projecting condition is related to feature information of an application program to be projected on (that is, the first screen projecting condition may also be set according to the feature information of the application program to be projected on), and the details here is not limited.

It should be noted that different display modules on the vehicle have different screen projection conditions, and therefore, in some embodiments of the present application, a specific way for the terminal device to obtain the first screen projection condition that the first display module on the target vehicle is projected may be: the terminal device may first acquire a plurality of screen projecting conditions in which a plurality of display modules on a vehicle are projected, where one display module corresponds to one screen projecting condition, for example, as shown in table 1, if there are 3 different display modules on a target vehicle a, which are respectively a dashboard screen, a steering wheel screen, and a central control screen, there is a corresponding screen projecting condition for each display module, and there are 3 screen projecting conditions in total, which are respectively screen projecting condition 1, screen projecting condition 2, and screen projecting condition 3, and the screen projecting conditions are set according to feature information of an application program, for example, different screen projecting conditions may be formed by different division results obtained by dividing the feature information of the application program.

TABLE 1 correspondence between display modules and screen projection conditions on target vehicle A

It should be noted that, in some embodiments of the present application, the terminal device may obtain all screen projection conditions that all display modules on the vehicle are projected on, as shown in table 1 above, the terminal device may obtain 3 screen projection conditions that each of 3 display modules on the target vehicle a corresponds to; the terminal device may also obtain partial screen projection conditions (at least 2) for the partial display modules projected on the vehicle, and as shown in table 1, the terminal device may only obtain 2 screen projection conditions (e.g., a center control screen and an instrument panel screen on the designated target vehicle a) corresponding to any 2 display modules or designated 2 display modules on the target vehicle a. For convenience of understanding, in the following embodiments of the present application, all screen projection conditions that the terminal device acquires that all display modules on the vehicle are projected are exemplified.

It should be noted that, in some embodiments of the present application, the terminal device obtains the screen projecting conditions that the plurality of display modules on the vehicle are projected on, may be obtained based on screen projecting conditions corresponding to various display modules stored in advance on the terminal device, the display modules may exist on vehicles of the same vehicle type, or may exist on vehicles of different vehicle types, the vehicle types are different, and the number and types of the display modules on the vehicle may also be different, specifically, as shown in table 2, the screen projecting conditions that each correspond to each other are made based only on the types of the different display modules and positions on the vehicle, regardless of which vehicle the display modules are on.

For example, suppose that the vehicle B has 3 display modules, which are respectively a dashboard screen, a steering wheel screen, and a central control screen; the vehicle C is provided with 5 display modules which are respectively an instrument panel screen, a steering wheel screen, a central control screen, a HUD and a copilot screen; the vehicle D is provided with 6 display modules which are respectively an instrument panel screen, a steering wheel screen, a central control screen, a left window screen, a right window screen and a skylight screen; vehicle E has 8 display modules, which are a dashboard screen, a steering wheel screen, a center control screen, a HUD, a front windshield screen, a rear windshield screen, a left view screen, a right view screen, and so on. Then, according to the corresponding relationship in table 2, the screen projection conditions corresponding to the display modules on the respective vehicles can be obtained. For example: assuming that the terminal device stores the correspondence between the display modules and the screen projecting conditions shown in table 2, the terminal device can acquire that the screen projecting conditions corresponding to the 3 display modules, namely, the dashboard screen, the steering wheel screen, and the center control screen, on the vehicle B are screen projecting condition 1, screen projecting condition 2, and screen projecting condition 3.

TABLE 2 correspondence between types of different display modules and screen projection conditions

It should be noted that, since the field of vehicle human-computer interfaces is experiencing the technical trend of automobile design, and the automatic driving system is developing rapidly, in the above table 2, all display modules possibly existing on the vehicle are not exhausted, and when a new type of display module appears, the user can update the corresponding relationship of the table 2 as required, so that the corresponding relationship between the types of different display modules in the table 2 and the screen projection condition is as perfect as possible.

It should be further noted that, in some embodiments of the present application, the correspondence between the types of the different display modules and the screen projection conditions illustrated in table 2 is not constant, and in practical applications, a user may adjust the correspondence between the display modules and the screen projection conditions according to driving habits, screen viewing habits, and the like.

It should be further noted that, in the embodiment of the present application, there are several implementation manners for the terminal device to obtain the multiple screen projection conditions that the multiple display modules on the vehicle are projected on, including but not limited to:

A. the terminal equipment receives the identification information of the display modules sent by the vehicle, and obtains a plurality of screen projection conditions of a plurality of display modules projected on the vehicle based on the identification information.

In the embodiment of the application, a communication connection needs to be established between the terminal device and the vehicle, and under the condition that the communication connection is successfully established, the terminal device receives identification information of a plurality of display modules sent by the vehicle, wherein each display module in the plurality of display modules corresponds to one identification information, and the identification information is used for uniquely representing the corresponding display module, so that the terminal device can identify the corresponding display module based on each received identification information, and because the terminal device stores the corresponding relationship between the type of the display module and the screen projection condition (as shown in table 2), the terminal device can acquire a plurality of screen projection conditions of the plurality of display modules projected on the vehicle based on the corresponding relationship.

It should be noted that, in some embodiments of the present application, the identification information of the display module may be directly a name of the display module, such as "center control screen", "steering wheel screen", "dashboard screen", and the like, described in table 2 above; the identification information of the display module may also be a self-named identifier, for example, the central control screen, the steering wheel screen, the dashboard screen, and the like described in table 2 may be named as "screen projection mark 1", "screen projection mark 2", "screen projection mark 3", and the like, and may be specifically shown in table 3; the identification information of the display module may also be self-defined letter identification (e.g., a, b, c, etc.), number identification (e.g., 11, 22, 33, etc.), special identifier (e.g., @ and &, #, etc.), and the identification information may be referred to as the identification information of the display module according to the embodiment of the present application as long as the identification has the function of uniquely identifying the corresponding display module.

TABLE 3 corresponding relationship between different display modules and identification information and screen projection conditions

For convenience of understanding, a manner that the terminal device obtains a plurality of screen projection conditions for a plurality of display modules projected on a vehicle is described below by taking the terminal device as a mobile phone as an example, please refer to fig. 9: first, the corresponding relations (may be referred to as mapping relation table 1) between the different types of display modules corresponding to table 3 and the identification information and the screen projecting conditions are stored in the mobile phone, and when the mobile phone carried by the user establishes a communication connection with the vehicle a, the communication connection may be a wired connection mode (e.g., through a USB connection) or a wireless connection mode (e.g., through wifi, Ultra Wide Band (UWB), bluetooth, etc.), which is not limited herein. Assuming that the vehicle a is deployed with 3 display modules, which are a central control screen, a steering wheel screen, and an instrument panel screen, respectively, the mobile phone receives a "screen projection mark 1", a "screen projection mark 2", and a "screen projection mark 3" sent by the vehicle a (identification information corresponding to each display module is pre-stored on the vehicle a). Then, the mobile phone searches the mapping relation table 1 based on the received "screen projection mark 1", "screen projection mark 2", and "screen projection mark 3", so as to obtain "screen projection condition 1", "screen projection condition 2", and "screen projection condition 3", which correspond to the "screen projection mark 1", "screen projection mark 2", and "screen projection mark 3", respectively.

In some embodiments of the present application, the mobile phone may also receive the center control screen, the steering wheel screen, and the dashboard screen sent by the vehicle a, and then the mobile phone recognizes the corresponding identification information based on the center control screen, the steering wheel screen, and the dashboard screen. That is, in the embodiment of the present application, the identification information may be recognized by the vehicle a or by a mobile phone, which is not limited herein.

It should be noted that, in some embodiments of the present application, when the terminal device connects to the vehicle for the first time in a wired or wireless manner, the identification information corresponding to the plurality of display modules on the vehicle is obtained from the vehicle and stored in the storage module of the terminal device, and if the subsequent terminal device establishes communication connection with the vehicle, the vehicle may not need to send the identification information corresponding to the plurality of display modules on the vehicle to the terminal device.

It should be further noted that, in some embodiments of the present application, identification information corresponding to display modules of more than one vehicle may be stored in the terminal device (for example, identification information corresponding to display modules on respective vehicles sent by different vehicles received at different times), and in this case, the user may select, on the terminal device, which identification information corresponds to the display module of the vehicle to obtain. As shown in fig. 10, taking a terminal device as a mobile phone as an example, assuming that identification information corresponding to a display module of a vehicle a, identification information corresponding to a display module of a vehicle B, and identification information corresponding to a display module of a vehicle C are stored on the mobile phone, if a user carries the mobile phone to prepare to drive the vehicle a, the user can select to acquire the identification information corresponding to the display module of the vehicle a from the mobile phone by himself or herself under the condition that the mobile phone has not established a communication connection with the vehicle a. It should be noted that fig. 10 is only an exemplary operation manner, and in a specific application process, there may be a plurality of operation manners for selecting and acquiring the identification information corresponding to the display module of the vehicle a from the mobile phone, which is not limited herein.

B. The terminal equipment obtains the vehicle model to which the vehicle belongs, and obtains a plurality of screen projection conditions for projecting screens of a plurality of display modules on the vehicle based on the vehicle model.

In the embodiment of the application, the terminal device may also obtain the vehicle model to which the vehicle belongs, and since the types and the number of the display modules deployed on the vehicle corresponding to each vehicle model are the same, the vehicle model to which the vehicle belongs, that is, which display modules the corresponding vehicle has, is known, and then a plurality of screen projection conditions on which the plurality of display modules on the vehicle are projected are obtained based on the vehicle model.

For the convenience of understanding, the following description still takes the terminal device as a mobile phone as an example, and please refer to fig. 11 for describing a manner that the terminal device obtains a plurality of screen projection conditions for a plurality of display modules projected on a vehicle: first, the corresponding relationship between the different types of display modules corresponding to the above table 3 and the identification information and the screen projection condition (which may be referred to as a mapping relationship table 1) is stored in the mobile phone, and in addition, the terminal device has a corresponding relationship between the vehicle model and the display module (which may be referred to as a mapping relationship table 2) in addition to the mapping relationship table, for example, table 4 shows the specific type of the display module of some vehicle models in the mapping relationship table 2, when the mobile phone carried by the user establishes a communication connection with the vehicle B, the mobile phone may be in a wired connection manner (e.g., through a USB connection), or may be in a wireless connection manner (e.g., through wifi, Ultra Wide Band (UWB), bluetooth, etc.), and the specific description is not limited herein. The mobile phone can receive the vehicle model of the vehicle B sent by the vehicle B, and supposing that the vehicle model of the vehicle B is Volvo XC60, the mobile phone finds out that the vehicle of the Volvo XC60 model from the mapping relation table 2 is provided with 2 display modules, namely a central control screen and an instrument panel screen, and the corresponding identification information is screen projection condition 1 and screen projection condition 3 respectively, and then the mobile phone obtains the screen projection condition 1 and the screen projection condition 3 respectively corresponding to the screen projection mark 1 and the screen projection mark 3 based on the mapping relation table 1.

TABLE 4 correspondence between vehicle model and display module type

Similarly, when a new vehicle model appears, the corresponding relationship between the vehicle model and the display module shown in table 4 can be updated at any time according to the user's needs, and similarly, when a certain existing vehicle model is updated with the display module on the corresponding vehicle (e.g., the number of display modules is increased, a new display module is added, the display module is replaced, etc.), the corresponding relationship in table 4 can also be updated and maintained in time.

It should be noted that, in some embodiments of the present application, when the terminal device connects to the vehicle for the first time in a wired or wireless manner, the vehicle model to which the vehicle belongs is obtained from the vehicle and stored in the storage module of the terminal device, and if the subsequent terminal device establishes communication connection with the vehicle, the vehicle may not need to send the vehicle model to which the vehicle belongs to the terminal device.

It should also be noted that in some embodiments of the present application, the vehicle model of more than one vehicle may be stored in the terminal device (for example, the vehicle model to which the respective vehicle sent by different vehicles received at different times belongs), and in this case, the user may select to obtain the vehicle model to which the vehicle belongs on the terminal device. As shown in fig. 12, taking a terminal device as an example of a mobile phone, assuming that the mobile phone stores a vehicle model to which a vehicle a belongs, a vehicle model to which a vehicle B belongs, and a vehicle model to which a vehicle C belongs, if a user carries the mobile phone to prepare to drive the vehicle B, the user can select to acquire the vehicle model to which the vehicle a belongs from the mobile phone by himself or herself under the condition that the mobile phone does not establish a communication connection with the vehicle B. It should be noted that fig. 12 is only an exemplary operation manner, and during a specific application process, there may be a plurality of operation manners for selecting and acquiring the vehicle model to which the vehicle a belongs from the mobile phone, which is not limited herein.

It should be noted that, in the embodiment of the present application, one display module corresponds to one screen projection condition, the screen projection condition may be set according to driving information of a vehicle (for example, a driving speed of the vehicle, a type of a traffic scene where the vehicle is located, a number of passengers in the vehicle, a fatigue state of a driver of the vehicle, and the like), the screen projection condition may also be set according to characteristic information of an application program (for example, an application type of the application program to be screen projected on a terminal device, a preset use frequency, and the like), and a specific setting manner of the screen projection condition is not limited herein. The following are described separately.

Firstly, setting screen projection conditions according to characteristic information of application program

First, an implementation manner in which a screen projection condition is set according to feature information of an application program is described. Different screen projection conditions are formed by different division results obtained by dividing the characteristic information of the application program. However, the characteristic information of the application includes various expressions, such as: the application type of the application program, the use frequency of the application program on the terminal device, the power consumption of the application program when the application program runs on the terminal device, and the like, different division results can be obtained by performing different divisions based on different characteristic information of the application program, and the division results constitute different screen projection conditions described in the embodiment of the present application. For convenience of understanding, in the following embodiments of the present application, 3 display modules on a vehicle are taken as examples, where the 3 display modules are respectively a "central control screen", a "steering wheel screen", and a "dashboard screen", and identification information of the corresponding display module is respectively "screen projection mark 1", "screen projection mark 2", and "screen projection mark 3", for illustration:

A. The characteristic information of the application program is the name of the application program, namely the screen projection condition is that the specific application program corresponds to the specific display module.

In the embodiment of the application, the screen projecting conditions are divided based on application names, different screen projecting conditions correspond to different application names, for example, as indicated in table 5, a corresponding relationship between each type of display module and the corresponding screen projecting condition is indicated, for example, if the name of the application to be projected on the terminal device is "WeChat", it can be known that "WeChat" belongs to the application name required in screen projecting condition 1 according to each screen projecting condition divided based on the application name in table 5, and the display module corresponding to screen projecting condition 1 is a dashboard screen, so that when the terminal device establishes a communication connection with the vehicle, the terminal device can project screen projecting information of the application "WeChat" onto a dashboard screen of the vehicle; similarly, if the name of the application program to be projected on the terminal device is "navigation", the navigation belongs to the application program name required in the screen projecting condition 3 according to each screen projecting condition obtained by dividing based on the application program name in the table 5, and the display module corresponding to the screen projecting condition 3 is a central control screen, so that after the terminal device establishes communication connection with the vehicle, the terminal device can project screen projecting information of the application program "navigation" to the central control screen of the vehicle. Similarly, which screen projection condition the application program to be projected on the terminal device satisfies is determined based on each screen projection condition obtained by dividing the application program name, and if one screen projection condition is satisfied, screen projection information of the application program is projected onto a display module corresponding to the screen projection condition, which is not described herein in detail.

It should be noted that table 5 is only an illustration of dividing based on application names to obtain each screen projection condition, and in an actual application process, there may be more dividing manners, for example, the application "WeChat" may be divided into the screen projection condition 2, and the details are not limited herein.

TABLE 5 correspondence between different types of display modules and corresponding screen projection conditions

Display module	Identification information	Screen projection condition	The screen projection condition comprises the name of the application program
				Instrument panel screen	Screen projection mark 1	Screen projection condition 1	WeChat, telephone, Welink,……
Steering wheel screen	Screen projection mark 2	Screen projection condition 2	Network cloud music, QQ music, … …
				Central control screen	Screen projection mark 3	Screen projection condition 3	Navigation, video conference, Tengchong video, Royal, … …
……	……	……	……

It should be noted that, in some embodiments of the present application, the user may modify the corresponding screen projection condition according to needs, for example, the names of the application programs "WeChat", "telephone", and "Welink" indicated in the above table 5 correspond to that screen projection information can be projected onto the dashboard screen, and if a new application program, such as "QQ", is installed on the terminal device, the user may add the new application program into the screen projection condition 1. Or after the terminal device establishes communication connection with the vehicle, the terminal device detects that an application program running on the terminal device does not belong to the application program name in any screen projection condition, at this time, the terminal device can prompt a user to confirm which display module on the vehicle is connected through interface display or voice (that is, the user can set the display module by himself in a manual mode), and after the user confirms connection, the terminal device stores the application program name into the screen projection condition corresponding to the display module so as to call the display module next time.

For the sake of understanding, fig. 13 is taken as an example to illustrate how to add the name of the application to be screened on the terminal device to a certain screen-screening condition, as shown in fig. 13, an application "QQ" is running on the mobile phone, and the mobile phone detects that the QQ does not belong to any kind of screen projection condition in the stored mapping relation table 1, at the moment, the mobile phone reminds the user to confirm the QQ to be projected to which display module in 3 display modules (namely, a center control screen, a steering wheel screen and a dashboard screen) on the vehicle in a display interface mode, and if the user confirms that the screen is projected to the dashboard screen, the mobile phone automatically saves the application program name QQ in the screen projection condition 1 corresponding to the steering wheel screen, therefore, if the connection between the mobile phone and the vehicle is established next time and the application program runs on the mobile phone, the mobile phone can automatically project the screen projection information of the QQ to the corresponding instrument panel screen. It should be noted that fig. 13 illustrates that the mobile phone reminds the user of confirmation through the text information displayed on the display interface, in some embodiments of the present application, the user may also be reminded of confirmation through a voice prompt manner, as shown in fig. 14, fig. 14 illustrates how the user adds the name of the application program to be screen-launched on the terminal device into a certain screen-launching condition through a voice prompt manner (e.g., mobile phone voice play "QQ does not have a correspondence with the vehicle display module, please add"), which is not described herein in detail.

It should be noted that fig. 13 and 14 are only two schematic operation manners, and in a specific application process, a user may be reminded to confirm in various manners of displaying an interface, which is not limited herein.

B. The characteristic information of the application program is the application type of the application program, namely the screen projection condition is that the specific application type corresponds to the specific display module.

In the embodiment of the present application, the screen-casting conditions are divided based on the application types to which the applications belong, and different screen-casting conditions correspond to different application types, in an implementation manner of the present application, when the terminal device downloads the applications from an "application market", "App Store", and the like (different manufacturers of the terminal device, different names may be different, and no limitation is made herein) of the terminal device, the application markets already classify the applications (i.e. the classification information determined by the application market tag, each application downloaded from the application market belongs to its own classification), for example, the application types may be classified into a social class, a travel navigation class, a video and audio class, a tool class, an information class, a game class, a travel accommodation class, a shopping class, an education class, a sports health class, a financial finance class according to the classification tag of the application market, any application must be able to be classified into one of the above application types. For example, when a social application program such as "WeChat", "QQ", "short message" is run on the terminal device, it can be known that "WeChat", "QQ", "short message" and the like belong to the application types required in the screen projecting condition 1 according to the screen projecting conditions obtained by dividing the application type to which the application program belongs in Table 6, and the display module corresponding to the screen projecting condition 1 is the dashboard screen, so that, after the terminal device establishes a communication connection with the vehicle, the terminal device can project the screen projecting information of the application program such as "WeChat", "QQ", "short message" and the like to the dashboard screen of the vehicle; similarly, if the application program to be screen-projected on the terminal device is "popular comment", the application type of the application program belongs to "information type", the "popular comment" belongs to the application type required in the screen-projecting condition 3, which is known according to the screen-projecting conditions divided based on the application type to which the application program belongs in table 6, and the display module corresponding to the screen-projecting condition 3 is a central control screen, so that the terminal device can screen-project the screen-projecting information of the application program "popular comment" to the central control screen of the vehicle after the terminal device establishes communication connection with the vehicle. Similarly, which screen projection condition the application program to be projected on the terminal device satisfies is determined based on each screen projection condition obtained by dividing the application type to which the application program belongs, and if one screen projection condition is satisfied, screen projection information of the application program is projected onto a display module corresponding to the screen projection condition, which is not described herein in detail.

TABLE 6 correspondence between different types of display modules and corresponding screen projection conditions

It should be noted that table 6 is only an illustration of dividing based on application types to obtain various screen projection conditions, and in the actual application process, there may be more dividing manners, for example, the dividing manners may be directly classified as: the first type, the second type, the third type and … … are only required to ensure that each application program only belongs to one of the application types, and the application is not limited in how to divide the application types.

It should be further noted that, in table 6 above, each screen projection condition includes at least 2 application types, and in practical applications, the number of application types included in the screen projection condition is not limited, for example, the screen projection condition 1 may include only social application programs, and table 6 is only used for illustration and not for limitation of the embodiment of the present application.

It should be noted that, in some embodiments of the present application, a user may modify a corresponding screen projection condition according to needs, for example, it is indicated in table 6 that all applications of which application types are social are projected onto a dashboard screen of a vehicle, and the user may also select to project the social applications onto other screens according to usage habits, viewing habits, driving habits, and the like, which is not limited herein.

It should be further noted that, in the embodiment of the present application, when the terminal device downloads a new application from the application market, the terminal device may automatically classify the application according to the classification of the application by the application market, and at this time, the user does not need to classify the application by himself or herself, for example, if a new application, such as "QQ", is installed on the terminal device, and it is known that "QQ" belongs to the social class according to the classification of the application market. Or after the terminal device establishes communication connection with the vehicle, the terminal device detects that a certain application program running on the terminal device belongs to the social class; if the application type described in the embodiment of the present application is a user-defined application type, when downloading a new application program, a user may classify the newly downloaded application program by himself or herself, and update the corresponding relationship between the different types of display modules and the corresponding screen-projecting conditions, where the updating manner is the manner type corresponding to fig. 13 and 14, and details thereof are not repeated here.

C. The characteristic information of the application program is the use frequency range of the application program, namely the screen projection condition is that the specific use frequency range of the application program on the terminal equipment corresponds to the specific display module.

In the embodiment of the application, the screen projection conditions are divided based on the use frequency range of the application program on the terminal device, different screen projection conditions correspond to different use frequency ranges, and the use frequency of the application program on the terminal device can be acquired from similar functions of the terminal device, such as 'healthy use of a mobile phone'.

In the embodiments of the present application, one principle to follow is: the more frequently used application's screen projection information is presented on the display module that the user needs to see with minimal cost or is most easily seen. For example: the screen projection information of the application program with high use frequency is presented on a steering wheel screen of the vehicle, and the screen projection information of the application program with low use frequency is presented on a central control screen of the vehicle.

It should be noted that, in some embodiments of the present application, the frequency range used by the application program on the terminal device may be characterized as follows: 1) the usage duration of the application program on the terminal device may be, for example, a total duration spent by the user when the application program runs on the terminal device since the application program is downloaded, or a duration spent by the user when the application program runs on the terminal device within a time period of one day, one week, or one month, and the specific calculation manner of the usage duration is not limited herein; 2) the number of times that the application program is started to run on the terminal device may be, for example, the number of times that the application program is started to run on the terminal device since the application program is downloaded by the user, or the number of times that the application program is started to run on the terminal device within a time period of one day, one week, or one month, and the specific way of calculating the number of times that the application program is started to run is not limited herein.

Taking the time length spent by the application program in one week as an example, the corresponding relationship between each type of display module and the corresponding screen projection condition is shown, please refer to table 7, the usage frequency is classified into different levels, for example, into "high", "medium", and "low", each usage frequency level corresponds to a usage frequency range (i.e., usage time length range), wherein the usage time length in one week is represented by t, and the unit of h is hour. For example, when the running time of the application program "WeChat" to be projected on the terminal device is 48 hours within one week of the time from the moment, according to each screen projection condition obtained by dividing the screen projection condition based on the use frequency range of the application program on the terminal device in table 7, the "WeChat" is in the range of high use frequency in screen projection condition 2, and the display module corresponding to screen projection condition 2 is a steering wheel screen, so that after the terminal device establishes communication connection with the vehicle, the terminal device can project the screen projection information of the application program "WeChat" to the dashboard screen of the vehicle; similarly, if the running time of the application program "royal glory" to be projected on the terminal device within one week of the time from the moment is 5h, the "royal glory" is known to be in the range of the low use frequency in the screen projection condition 3 according to each screen projection condition obtained by dividing the use frequency range of the application program on the terminal device in the table 7, and the display module corresponding to the screen projection condition 3 is a central control screen, so that the terminal device can project the screen projection information of the application program "royal glory" to the central control screen of the vehicle after the terminal device is in communication connection with the vehicle. Similarly, which screen projection condition the application program to be projected on the terminal device satisfies is determined based on each screen projection condition obtained by dividing the use frequency of the application program on the terminal device, and if one screen projection condition is satisfied, screen projection information of the application program is projected onto a display module corresponding to the screen projection condition, which is not described herein in detail.

TABLE 7 correspondence between different types of display modules and corresponding screen projection conditions

Display module	Identification information	Screen projection condition	The screen projection condition comprises a use frequency range (use duration)
				Instrument panel screen	Screen projection mark 1	Screen projection condition 1	The frequency of use: t ∈ [20h, + ∞ -]
Steering wheel screen	Screen projection mark 2	Screen projection condition 2	High use frequency: t is in the range of 10h, 20h]
				Central control screen	Screen projection mark 3	Screen projection condition 3	Low use frequency: t is belonged to 0h, 10h]
……	……	……	……

It should be noted that in some embodiments of the present application, the usage frequency range may be divided according to the number of display module types, and 3 display modules are illustrated in table 7, so the usage frequency is divided into 3 ranges, which correspond to 3 levels of "high", "medium", and "low" usage frequencies, one display module for each range, and if there are n display modules on the vehicle, the usage frequency may be divided into n ranges, one usage frequency level for each range, and one display module for each range.

It should be noted that, in some embodiments of the present application, the user may modify the corresponding screen projection condition by himself or herself as needed, for example, it is indicated in table 7 that all the applications with high usage frequency are projected on the steering wheel screen of the vehicle, and the user may also select to project the applications with high usage frequency on other screens according to usage habits, viewing habits, driving habits, and the like, which is not limited herein.

D. The characteristic information of the application program comprises an application type and a use frequency range to which the application program belongs, namely, the screen projection condition is that the specific use frequency range of the application program of the specific application type on the terminal equipment corresponds to the specific display module.

In the above a, B, and C modes, the characteristic information of the application program is divided based on the single factors of the application program (e.g., application name, application type, usage frequency range of the application program on the terminal device, etc.), in this embodiment, the single factors may be combined with each other to obtain the division result based on the combined factors, the screen projection condition is constructed based on the application type to which the application program belongs and the usage frequency range of the application program on the terminal device, as shown in table 8, the corresponding relationship between each type of display module and the corresponding screen projection condition is shown, for example, when the duration of the application program "QQ" to be projected on the terminal device runs on the terminal device within one week from the time of the application "QQ" on the terminal device is 17h, and the application type of the "QQ" is known to be social based on the classification of the application market at the time of downloading, according to the screen projecting conditions obtained by dividing the application program based on the use frequency range of the application program on the terminal device and the application type of the application program in table 8, it can be known that "QQ" is in the use frequency range in the screen projecting condition 1 and is social, and the display module corresponding to the screen projecting condition 1 is the dashboard screen, so that after the terminal device establishes communication connection with the vehicle, the terminal device can project the screen projecting information of the application program "QQ" to the dashboard screen of the vehicle; similarly, if the running time of the application program "royal glory" to be projected on the terminal device within one week of the time is 5h, and the application type of "royal glory" can be known as the game type according to the classification of the "royal glory" by the application market at the time of downloading, the "royal glory" can be known to be in the range of low use frequency in the projection condition 3 and be the game type according to each projection condition obtained by dividing based on the use frequency range of the application program on the terminal device and the application type to which the application program belongs in the table 8, and the display module corresponding to the projection condition 3 is the central control screen, so that after the terminal device and the vehicle are in communication connection, the terminal device can project the projection information of the application program "royal glory" to the central control screen of the vehicle. Similarly, which screen projection condition the application program to be projected on the terminal device satisfies is determined based on the use frequency of the application program on the terminal device and each screen projection condition obtained by dividing the application type to which the application program belongs, and if one screen projection condition is satisfied, screen projection information of the application program is projected onto a display module corresponding to the screen projection condition, which is not described herein in detail.

TABLE 8 correspondence between different types of display modules and corresponding screen projection conditions

Similarly, it should be noted that table 8 is only an illustration of obtaining each screen projection condition by dividing based on the frequency of use of the application program on the terminal device and the application type to which the application program belongs, and in the actual application process, there may be more dividing manners, for example, the application types may be directly classified as: first class, second class, third class, … …, and the frequency of use range can also be divided according to the number of display module types, 3 display modules are illustrated in table 8, so the frequency of use is divided into 3 ranges, corresponding to "high", "medium", and "low" 3 levels of frequency of use, each range corresponding to one display module, if there are n display modules on the vehicle, similarly, the frequency of use can be divided into n ranges, each range corresponding to one level of frequency of use, and each range corresponding to one display module. In short, in the embodiment of the present application, it is only necessary to ensure that each application program belongs to only one of the application types, and the specific application does not limit how the application types are divided. It should be further noted that, in table 8 above, each screen projection condition includes at least 2 application types, and in practical applications, the number of application types included in the screen projection condition is not limited, for example, the screen projection condition 1 may include only social applications, and table 8 is only used for illustration and not for limitation of the embodiment of the present application.

It should be noted that, in some embodiments of the present application, a user may modify a corresponding screen projection condition according to needs, for example, it is indicated in table 8 that all applications whose application types are social applications and whose usage frequency on a terminal device is a medium level are projected onto a dashboard screen of a vehicle, and the user may also select to project the social applications onto other screens according to usage habits, viewing habits, driving habits, and the like, which is not limited herein.

It should be further noted that table 8 is only an illustration of a division result based on a combination factor obtained by combining two single factors (an application type and a usage frequency range of an application program on a terminal device) with each other, and in some embodiments of the present application, the division result based on a combination factor obtained by combining any two or more other factors with each other may also be used, and details are not repeated herein.

Secondly, setting screen projection conditions according to the driving information of the vehicle

Next, an implementation in which the screen projecting condition is set according to the travel information of the vehicle will be described. In the embodiment of the present application, the driving information of the vehicle may be characterized by a plurality of parameters, for example, the driving speed of the vehicle, the type of traffic scene in which the vehicle is located (e.g., highway, country road, mountain road, etc.), the number of passengers in the vehicle (e.g., 1 passenger in a copilot, 2 passengers in a back row, etc.), the fatigue state of the driver of the vehicle (e.g., fatigue, no fatigue, which may be calculated from the human body state information detected by a sensor module disposed in the vehicle or from the human body state information detected by a wearable device worn on the driver), the distance between the vehicle and an obstacle (e.g., how far away the obstacle is from the vehicle to determine whether the driving environment is safe or unsafe), and the like. Therefore, in the embodiment of the present application, the screen projection condition may be set according to the driving information of the vehicle, in which case, when the first screen projection condition that the first display module is projected on the target vehicle is satisfied, the method includes: and when the running information of the target vehicle meets the first screen projection condition. When the driving information is represented by different parameters, conditions to be met when the screen information is projected are also different, which is described below, for convenience of understanding, in the following embodiments of the present application, 3 display modules on the vehicle are all used, and the 3 display modules are respectively illustrated as a "central control screen", a "steering wheel screen", and a "dashboard screen", for example:

A. The running information of the vehicle is a driving speed of the vehicle.

In the embodiment of the present application, when the characterizing parameter of the driving information is "driving speed", the first screen projecting condition includes: the driving speed of the target vehicle is within a first driving speed range. Specifically, refer to the method a in step 2602 and table 10, which are not described herein.

Since different driving speeds are essentially aimed at driving safety, one principle that should be followed is: in the case of low safety, the screen projection information should be displayed on a display module that is most easily visible and can be seen by the driver with the least cost, for example, when the driving speed is high, and the driver is required to have higher attention, the screen projection information can be projected to a screen (such as a steering wheel screen, a HUD, and the like) that can be seen by the driver with the least attention, and when the driving speed is low, the screen projection information can be projected to a screen (such as a central control screen) that can be seen by the driver with a little head twist due to the low speed because the driver can not need the high attention.

B. The driving information of the vehicle is the type of the traffic scene where the vehicle is located.

In the embodiment of the application, when the characterization parameter of the driving information is "the type of the traffic scene where the vehicle is located", the first screen projecting condition includes: the traffic scene type in which the target vehicle is located belongs to a first traffic scene type. For details, reference may be made to the following B mode in step 2602 and table 11, which are not described herein again.

Similarly, since the different traffic scene types that the vehicle is in are also substantially directed to driving safety, one principle that should be followed is: under the condition of low safety, the screen projection information is presented on a display module which can be seen and is most easily seen by a driver with the least cost, for example, when the vehicle is at an intersection with a bright red light, the driver stops the vehicle and waits for the change of the red light into a street light, at the moment, the driver can not need so high attention, the screen projection information can be projected to a screen (such as a central control screen) which can be seen by the driver by slightly twisting the head, when the vehicle is at the intersection, the intersection is generally a high-incidence area of a traffic accident, and at the moment, the driver needs to have sufficiently high attention, so the screen projection information can be projected to a screen (such as a steering wheel screen, a HUD and the like) which can be seen by the driver with the least attention.

C. The driving information of the vehicle is the number of passengers in the vehicle.

In the embodiment of the present application, when the characterizing parameter of the driving information is "the number of passengers in the vehicle", the first screen projecting condition includes: the number of passengers in the target vehicle is within the first passenger number range. The essential reason for the difference in the number of passengers in the vehicle is privacy and commonality of the screen projection information, which can be specifically referred to the C mode in step 2602, and tables 12 and 13, which are not described herein again.

D. The running information of the vehicle is a fatigue state of a driver of the vehicle.

In the embodiment of the present application, when the characterizing parameter of the driving information is "fatigue state of the vehicle driver", the first screen projecting condition includes: the fatigue state of the driver of the target vehicle belongs to the first fatigue state. Specifically, refer to the following D mode in step 2602 and table 14, which are not described herein.

E. The travel information of the vehicle is a distance between the vehicle and the obstacle.

In the embodiment of the present application, when the characteristic parameter of the driving information is "a distance between the vehicle and an obstacle", the first screen projecting condition includes: the distance between the target vehicle and the obstacle is within a first range of distances. Specifically, reference may be made to the mode E in step 2602, and tables 15 and 16, which are not described herein again.

802. And when the first screen projection condition is met, projecting screen projection information of a first screen application program to be projected on the terminal equipment to a first display module.

After the terminal device obtains a plurality of screen projection conditions that a plurality of display modules on a vehicle are projected, when characteristic information of a screen to be projected application program (which can be called as a first screen to be projected application program) on the terminal device meets a screen projection condition (which can be called as a first screen projection condition) that a certain display module (which can be called as a first display module) on the vehicle is projected, or running information of the vehicle meets the first screen projection condition that the first display module on the vehicle is projected, the terminal device projects screen projection information of the first screen to be projected application program to the first display module.

For convenience of understanding, as shown in fig. 15, by way of example, taking a terminal device as a mobile phone, assuming that feature information of an application is an application type to which the application belongs (i.e., the above-mentioned B mode), and assuming that an application running on the mobile phone is "QQ", as can be seen from the above-mentioned table 6, the "QQ" belongs to the application type required in the screen projection condition 1, and a display module corresponding to the screen projection condition 1 is a dashboard screen, that is, the application "QQ" running on the mobile phone satisfies the screen projection condition 1 that the dashboard screen is projected on the vehicle, then in a case where the mobile phone establishes a communication connection with the vehicle, the mobile phone projects screen projection information of the application "QQ" onto the dashboard screen of the vehicle.

It should be noted that, in some embodiments of the present application, the screen projection information of the first application to be screen projected may be an entire application interface of the first application to be screen projected, or may be a part of the application interface of the first application to be screen projected. For ease of understanding, fig. 16 and 17 are illustrated as an example: assuming that the first screen-to-be-projected application is an "Tencent video" and the "Tencent video" satisfies the screen projecting condition that the central control screen on the vehicle is projected, if the "Tencent video" is playing a certain video, fig. 16 illustrates that the whole application program interface presented by the first screen-to-be-projected application on the display interface of the mobile phone is projected onto the central control screen of the vehicle; fig. 17 illustrates that the first screen-to-be-projected application projects a part of the application interface (e.g., only the video playing part) presented on the display interface of the mobile phone onto the center control screen of the vehicle.

It should be noted that, in the embodiment of the present application, since the size of the display interface of the terminal device is different from the size of different display modules on the vehicle, the vehicle needs to be adapted to the screen size during the screen projection process, for example, the screen size is changed from 6.1 inches of the mobile phone to 12 inches, or even 17 inches, of the center control screen of the on-vehicle display screen.

It should be further noted that, in some embodiments of the present application, there may be more than one application to be screen-projected on the terminal device, that is, there are multiple first application to be screen-projected, where only screen-projection information of one application is displayed on a display interface on the terminal device, and another application is run in the background, in this case, if multiple first application to be screen-projected that are simultaneously run on the terminal device respectively satisfy screen-projection conditions that different display modules on the vehicle are screen-projected, there may be different screen-projection manners, including but not limited to the following:

in the mode 1, screen projection information of a plurality of first screen projection application programs to be projected is respectively projected to display modules meeting screen projection conditions.

For example, as shown in fig. 18, assuming that an application "QQ" and "know" are running on the mobile phone, and assuming that the feature information of the application is the application type to which the application belongs (i.e., the above-mentioned B mode), it can be known from the above-mentioned table 6 that the application "QQ" belongs to the application type required in the screen projecting condition 1, the display module corresponding to the screen projecting condition 1 is a dashboard screen, the application "know" belongs to the application type required in the screen projecting condition 3, and the display module corresponding to the screen projecting condition 3 is a central control screen, that is, the "QQ" and "know" respectively satisfy the screen projecting conditions of different display modules, in this case, the mobile phone can project the screen projecting information of the "QQ" and the screen projecting information of the "know" to the dashboard screen and the central control screen respectively.

And 2, selecting the first screen to be projected application program which is displayed on the display interface of the terminal equipment for screen projection. For example, as shown in fig. 19, assuming that an application "QQ" and "know" are running on the mobile phone, and assuming that the feature information of the application is the application type (i.e., the above-mentioned B mode) to which the application belongs, it can be known from the above-mentioned table 6 that the application "QQ" belongs to the application type required in the screen projecting condition 1, the display module corresponding to the screen projecting condition 1 is a dashboard screen, the application "know" belongs to the application type required in the screen projecting condition 3, and the display module corresponding to the screen projecting condition 3 is a central control screen, but the application "know" runs in the background of the mobile phone, and the "QQ" is being displayed on the display interface of the mobile phone, in this case, the mobile phone can project the screen projecting information of the "QQ" to the dashboard screen; if the user switches the application program 'know' to the display interface of the mobile phone for displaying at a certain time, the mobile phone also synchronously projects the 'know' screen projection information to the central control screen.

It should be noted that, in the above embodiments of the present application, there is no intersection between different screen projection conditions, for example, table 5 is taken as an example for description: in table 5, the different applications can only be the application names required in one of the screen-projection conditions, and as shown in fig. 20, there is no intersection between the applications required in the respective screen-projection conditions.

However, in some embodiments of the present application, one application may simultaneously satisfy the feature information of the application required in the plurality of screen-projection conditions. For example, as shown in fig. 21, the application "WeChat" may be both the application required in screen projection Condition 1 and the application required in screen projection Condition 2; the application "QQ" may be an application required in the screen-projection condition 1, an application required in the screen-projection condition 3, or the like. For another example, as shown in fig. 22, the application type "social class" (e.g., "WeChat", "QQ", "short message", etc.) to which the application program belongs may be the application type required in the screen projection condition 1, the application type required in the screen projection condition 2, or the like. Similarly, the frequency range of the application program used on the terminal device may be the use range required in the screen projecting condition 1 or the use range required in the screen projecting condition 2, similar to fig. 21 and 22, and is not described herein again. In summary, when a certain application program meets the screen projection conditions that a plurality of display modules are projected on, in this embodiment of the application, there may be a plurality of screen projection modes, including but not limited to the following ones, for convenience of understanding, the following description is given by taking as an example that the application program simultaneously meets the screen projection conditions that 2 display modules are projected on, where the 2 display modules are respectively a first display module and a second display module, and the met screen projection conditions are respectively a first screen projection condition and a second screen projection condition:

A. And screen projection information of the application program to be screen projected on the terminal equipment is projected to the first display module and the second display module.

In the embodiment of the application, when a first screen projection application program to be projected on a terminal device simultaneously meets a first screen projection condition that a first display module on a vehicle is projected on a screen and a second screen projection condition that a second display module on the vehicle is projected on the screen, and under the condition that the terminal device is in communication connection with the vehicle, the terminal device simultaneously projects screen projection information of the first screen projection application program to the first display module and the second display module of the vehicle, wherein the first display module and the second display module are two different display modules on the vehicle.

It should be noted that, in this embodiment of the application, a premise that the terminal device simultaneously projects screen projection information of the first screen application to be projected onto the first display module and the second display module is that neither the first display module nor the second display module is occupied by screen projection, and if any one of the first display module and the second display module is occupied by screen projection, the screen projection information of the first screen application to be projected can only be projected onto the display module that is not occupied by screen projection.

For convenience of understanding, the following description takes a terminal device as a mobile phone as an example, please refer to fig. 23, and assume that a first screen application program to be projected on the mobile phone is "QQ", and it can be known from the mapping relation table 1 that the application program "QQ" simultaneously satisfies the screen projection conditions that a center control screen and a steering wheel screen on a vehicle are projected, so that the mobile phone can simultaneously project screen projection information of "QQ" to the center control screen and the steering wheel screen under the condition that neither the center control screen nor the steering wheel screen is occupied by the screen projection. In the embodiment of the present application, if any one of the center control screen and the steering wheel screen is occupied by screen projection before screen projection, for example, assuming that the center control screen is occupied by screen projection, then the screen projection information of "QQ" can be projected only to the display module that is not occupied by screen projection, for example, only to the steering wheel screen.

B. The screen projection information of the application program to be screen projected on the terminal equipment is divided into first screen projection sub-information and second screen projection sub-information according to a preset mode, the first screen projection sub-information is projected to a first display module, and the second screen projection sub-information is projected to a second display module.

In the embodiment of the application, when a first screen projection application program to be projected on a terminal device simultaneously meets a first screen projection condition that a first display module on a vehicle is projected on the screen and a second screen projection condition that a second display module on the vehicle is projected on the screen, and under the condition that the terminal device establishes communication connection with the vehicle, the screen projection information of the first screen projection application program to be projected is divided into first screen projection sub-information and second screen projection sub-information by the terminal device according to a preset mode, the first display module and the second display module are two different display modules on the vehicle, and then the terminal device respectively projects the first screen projection sub-information and the second screen projection sub-information to the first display module and the second display module.

It should be noted that, in some embodiments of the present application, the terminal device divides the screen projection information of the first application program to be screen projected into the first screen projection sub-information and the second screen projection sub-information according to a preset mode, and there may be multiple dividing modes, including but not limited to the following:

a. dividing an application program interface of a first screen application program to be projected into a first part and a second part, wherein the first part is first screen projection sub information, and the second part is second screen projection sub information.

For convenience of understanding, a terminal device is taken as a mobile phone for illustration, please refer to fig. 24, an application program "Tencent video" is displayed on a display interface of the mobile phone, and assuming that the application program "Tencent video" simultaneously satisfies screen projection conditions of a central control screen and a steering wheel screen, a video picture of the "Tencent video" being played can be projected to the central control screen (i.e., a first screen projection sub-information is projected to a first display module) and a text part of the "Tencent video" is projected to the steering wheel screen (i.e., a second screen projection sub-information is projected to a second display module) according to a preset mode. It should be noted that, in the embodiment of the present application, fig. 24 is only illustrated by taking the screen projection information as the entire application interface of the application program as an example, and in some embodiments of the present application, the screen projection information may also be a part of the application interface of the application program. Specifically, reference may be made to the embodiment portions corresponding to fig. 16 and fig. 17, which are not described herein again.

b. And dividing program interfaces corresponding to a first function module and a second function module of the first application program to be screen-projected into first screen-projecting sub information and second screen-projecting sub information.

For convenience of understanding, the following description still takes the terminal device as a mobile phone as an example, please refer to fig. 25 and table 9, assuming that the first application program to be screen-projected on the mobile phone is "QQ", and the program interface of "QQ" is divided into "message pushing module" and "message list module" according to the function of the function module, assuming that the application program "QQ" simultaneously satisfies the screen-projecting conditions of the central control screen and the steering wheel screen, the program interface corresponding to the "message pushing module" of "QQ" can be projected to the steering wheel screen of the vehicle (i.e., the first screen-projecting sub-information is projected to the first display module), and the program interface corresponding to the "message list module" of "QQ" can be projected to the central control screen of the vehicle (i.e., the second screen-projecting sub-information is projected to the second display module) according to a preset manner.

Table 9, correspondence between screen projection sub-information divided based on different function modules of the application and the display module

Application program	Functional module	Identification information	Display module
				QQ	Message pushing module	Screen projection mark 1	Instrument panel screen
QQ	Message list module	Screen projection mark 2	Central control screen
				……	……	……	……

It should be noted that, in some embodiments of the present application, the functions may be divided according to the usage frequency and the interaction behavior of different function modules, for example, the screen-projecting sub-information of the program interface corresponding to the function module with high-frequency display and low-frequency interaction is displayed on the display module that is beneficial for the driver to read at the most comfortable angle, for example, the screen-projecting sub-information corresponding to the message pushing module is displayed on the sub-dashboard screen; and displaying the screen-casting sub-information of the program interface corresponding to the functional module (such as list sliding and interaction) with a certain interaction process on a display module which is easiest to operate by both hands of a driver, for example, displaying the screen-casting sub-information corresponding to the message list module on a steering wheel screen or a central control screen. The screen projection sub-information corresponding to different functional modules of the application program is projected to different display modules on the vehicle, so that the interference on the driving of a driver is reduced, and the driving safety is improved.

It should be noted that, in this embodiment of the present application, fig. 25 is only an example, and in some embodiments of the present application, different application programs may be divided into different functional modules, which may be divided into 2 functional modules, or divided into more than 2 functional modules (each functional module corresponds to one program interface), which is not described herein in detail.

C. The terminal equipment judges the priority of the first display module and the priority of the second display module, and screens the screen projection information of the application program to be screen projected on the terminal equipment to the display module with the high priority.

In the embodiment of the application, when a first screen to be projected application program on a terminal device simultaneously meets a first screen projecting condition that a first display module on a vehicle is projected on the screen and a second screen projecting condition that a second display module on the vehicle is projected on the screen, and under the condition that the terminal device establishes communication connection with the vehicle, the terminal device judges whether the first display module with high priority is occupied by the projected screen or not according to a preset priority, wherein the first display module and the second display module are two different display modules on the vehicle; if the first display module is not occupied by screen projection, the terminal device projects screen projection information of the first screen application program to be projected to the first display module preferentially, and only under the condition that the first display module is occupied by screen projection, the terminal device projects screen projection information of the first screen application program to be projected to the second display module.

For convenience of understanding, the following description will be made by taking a terminal device as a mobile phone as an example: the application program 'WeChat' is displayed on a display interface of the mobile phone, the application program 'WeChat' meets the screen projecting condition that the center control screen and the steering wheel screen are projected on the same screen, the mobile phone judges the priority of the center control screen and the steering wheel screen by the application program 'WeChat', if the priority of the center control screen is higher than that of the steering wheel screen, the mobile phone further determines whether the center control screen is occupied by other application programs (for example, whether videos are played), if the center control screen is not occupied by the screen projection, the mobile phone projects the screen projection information of the 'WeChat' to the center control screen, and if the center control screen is determined to be projected by other application programs (for example, videos are played), the mobile phone projects the screen projection information of the 'WeChat' to the steering wheel screen (on the premise that the steering wheel screen is not occupied by other application programs).

In the above embodiments of the present application, the following is specifically set forth: through obtaining the screen projection condition (for example, the screen application program to be projected on the terminal equipment belongs to a certain application type, the preset use frequency is reached, and the like) which needs to be met when each display module on the vehicle is projected, when the screen projection condition is met, the display information of the screen application program to be projected on the terminal equipment is projected to the corresponding display module, so that the display information on the terminal equipment is reasonably distributed to different display modules on the vehicle, a driver can quickly obtain the information on the terminal equipment, the probability of driver attention dispersion is reduced, the interference of the screen projection information on the driving of the driver is reduced, and the driving safety is improved.

It should be noted that the main execution body of the screen projection method described in the embodiment corresponding to fig. 8 is a terminal device, and the screen projection process is as follows: the method comprises the steps that the terminal equipment firstly obtains a plurality of screen projecting conditions that a plurality of display modules of a vehicle are projected on, then the terminal equipment judges whether running information of the vehicle (or characteristic information of an application program to be projected on the terminal equipment) meets a first screen projecting condition, and when the running information meets the first screen projecting condition, the terminal equipment projects screen projecting information of the application program to be projected on the terminal equipment onto a first display module corresponding to the first screen projecting condition for displaying. That is, it is the terminal device that performs the screen projection. However, in some embodiments of the present application, the execution subject of the screen projection method may also be the vehicle itself, in which case, the screen projection process is as follows: the method comprises the steps that a target vehicle acquires a plurality of screen projection conditions of a plurality of display modules projected by the target vehicle, whether the running information (or the characteristic information of an application program to be projected on the terminal device) of the target vehicle meets a first screen projection condition is judged by the target vehicle, and when the first screen projection condition is met, the target vehicle automatically projects the received screen projection information of the application program to be projected on the terminal device onto a first display module corresponding to the first screen projection condition for displaying. That is, it is the target vehicle itself to select which display module on itself the screen projection information is projected to.

In the embodiments shown in fig. 8 to fig. 25, how the terminal device projects the screen projection information of the application program to be projected on the terminal device onto the display modules of the vehicle (e.g., a normal vehicle, an intelligent vehicle, an auto-driven vehicle, an internet vehicle, etc.) meeting the corresponding screen projection conditions based on the screen projection conditions is described, where each display module on the vehicle corresponds to one screen projection condition. In some embodiments of the application, after the terminal device finishes screen projection based on the screen projection modes shown in fig. 8 to fig. 25, it may be further determined, by combining the driving information of the vehicle itself, whether screen-cutting display is required in different driving scenes, that is, switching between different display modules may be performed by combining the driving state of the vehicle itself. Referring to fig. 26, specifically, referring to fig. 26, fig. 26 is another schematic flow chart of the display information switching method provided in the embodiment of the present application, which specifically includes the following steps:

2601. the driving information of the vehicle is acquired.

First, the vehicle acquires its own travel information. In the embodiment of the present application, the driving condition of the vehicle may be characterized by a plurality of parameters, for example, the driving speed of the vehicle, the type of traffic scene in which the vehicle is located (e.g., highway, country road, mountain road, etc.), the number of passengers in the vehicle (e.g., 1 passenger in a copilot, 2 passengers in a back row, etc.), the fatigue state of the driver of the vehicle (e.g., fatigue, no fatigue, which may be calculated from human body state information detected by a sensor module disposed in the vehicle or from human body state information detected by a wearable device worn on the driver), the distance between the vehicle and an obstacle (e.g., how far away the obstacle is from the vehicle to determine whether the driving environment is safe or unsafe), and the like.

It should be noted that, in the embodiment of the present application, the vehicle may acquire its own driving information periodically, for example, the acquisition period may be set to 1 minute, that is, the vehicle may acquire its own driving information every 1 minute; the vehicle may obtain the running information of itself only when the running information changes, for example, if the running information is characterized by the driving speed, when the variation range of the driving speed exceeds a preset value (for example, the preset value is set to be 20km/h), if the driving speed is accelerated from 30km/h to 60km/h, when the driving speed is accelerated from 30km/h to 50km/h (that is, the preset value is 20km/h), the vehicle may be triggered to obtain the running information of itself. The specific application does not limit how the driving information of the vehicle itself is acquired.

2602. When the running information of the vehicle does not accord with the first running condition, switching screen projection information of the first application program displayed on the first display module to the second display module to be displayed, wherein the first running condition is a condition which needs to be met when the first display module is projected.

The vehicle may further obtain a first driving condition that a first display module displaying screen projection information needs to meet, where the first display module is one of a plurality of display modules on the vehicle, and it should be noted that in this embodiment of the application, the screen projection information of the first application may be screen projection information that the terminal device projects an application running on the terminal device to the first display module when the terminal device establishes a communication connection with the vehicle, or may be an operation interface of an application installed in the vehicle itself, or may be a part of the operation interface, and may be collectively referred to as target information.

If the first display module on the vehicle is displaying the screen projection information of the first application program running on the terminal device, the vehicle also acquires a first running condition which needs to be met by the first display module.

For example, taking a terminal device as a mobile phone for illustration, please refer to table 10, assuming that there are 3 display modules in total on a vehicle, which are respectively a "central control screen", a "steering wheel screen", and a "dashboard screen", and the driving information is represented by a parameter "driving speed", assuming that the driving condition to be satisfied when the central control screen is projected is "driving speed < 30 km/h", the driving condition to be satisfied when the dashboard screen is projected is "driving speed 30km/h ≦ 60 km/h", and the driving condition to be satisfied when the steering wheel screen is projected is "driving speed 60km/h ≦ 100 km/h" (i.e. the driving speed is considered to be too fast above 100km/h, and is not suitable for screen projection). Assuming that the first display module of the vehicle is a central control screen and the screen projection information of the application program "QQ" running on the mobile phone is being displayed on the central control screen, the vehicle will acquire the driving condition to be satisfied when the central control screen is projected, for example, as shown in table 10, the driving condition to be satisfied is "driving speed < 30 km/h".

Watch 10, driving conditions to be met when different display modules are projected (driving speed as an example)

Display module	Driving conditions to be met (Driving speed)
		Central control screen	The driving speed is less than 30km/h
Instrument panel screen	The driving speed is less than or equal to 30km/h and less than 60km/h
		Steering wheel screen	The driving speed is less than or equal to 60km/h and less than 100km/h
……	……

It should be further noted that, in some embodiments of the present application, the screen projection information of the first application may be an entire application interface of the first application or a part of the application interface of the first application. As shown in fig. 16 and 17, detailed description thereof is omitted.

It should be noted that, in the embodiment of the present application, since the size of the display interface of the terminal device is different from the size of different display modules on the vehicle, the vehicle needs to be adapted to the screen size during the screen projection process, for example, the screen size is changed from 6.1 inches of a mobile phone to 12 inches, or even 17 inches, of a central control screen of an on-vehicle display.

The method comprises the steps that after the vehicle acquires the running information of the vehicle and a first running condition, whether the running information of the vehicle is in accordance with the first running condition or not is further judged, and under the condition that the running information of the vehicle is not in accordance with the first running condition, the vehicle switches screen projection information to a second display module corresponding to a second running condition in accordance with the first running condition to display, wherein the second display module is another display module in a plurality of display modules on the vehicle.

In some embodiments of the present application, a process that the vehicle switches the screen projection information to the second display module corresponding to the second driving condition that is met by the first driving condition for displaying may specifically be: the vehicle generates a switching instruction (which can be called as a first switching instruction) firstly, the first switching instruction is used for instructing the vehicle to switch the screen projection information to the second display module, and then the vehicle switches the screen projection information to the second display module corresponding to the second running condition which is accordant with the running information of the vehicle according to the first switching instruction to display.

In some embodiments of the present application, the information displayed on the first display module may be not only the screen projection information of the first application program, but also other information, for example, an operation interface of an application program installed in the vehicle itself, or a part of the operation interface, which may be collectively referred to as target information, and in this case, the information displayed on the first display module is not screen projection information, in which case, one way to implement the screen cutting is: the method comprises the steps that a vehicle obtains driving information of the vehicle, whether the driving information of the vehicle accords with a first driving condition or not is judged, when the driving information of the vehicle does not accord with the first driving condition and the driving information of the vehicle accords with a second driving condition, target information displayed on a first display module is switched to a second display module to be displayed, wherein the first display module and the second display module are two different display modules on the vehicle respectively, the first driving condition is a condition which needs to be met when the first display module is projected on a screen, and the second driving condition is a condition which needs to be met when the second display module is projected on the screen.

It should be noted that, in the embodiment of the present application, since the driving information of the vehicle may be characterized by a plurality of parameters, for example, the driving speed of the vehicle, the type of traffic scene where the vehicle is located, the number of passengers in the vehicle, the fatigue state of the driver of the vehicle, the distance between the vehicle and the obstacle, and the like, which are described above. When the driving information is represented by different parameters, conditions to be met by switching display are different, and the following description is made respectively, for convenience of understanding, in the following embodiments of the present application, 3 display modules on a vehicle are all taken, and the 3 display modules are respectively exemplified as a "central control screen", a "steering wheel screen", and a "dashboard screen", for example:

A. the running information of the vehicle is a driving speed of the vehicle.

In the embodiment of the present application, when the characterizing parameter of the running information is "driving speed", then the running information of the vehicle does not conform to the first running condition, and the running information of the vehicle conforms to the second running condition is: the first driving condition includes a driving speed being within a first driving speed range, and the second driving condition includes a driving speed being within a second driving speed range; the driving speed of the vehicle is not within the first driving speed range, and the driving speed of the vehicle is within the second driving speed range.

For ease of understanding, the following is illustrated by way of example in FIG. 27: assuming that the vehicle has 3 display modules, namely, a central control screen, a steering wheel screen and an instrument panel screen, and the driving information is represented by a parameter of driving speed, specifically, as shown in table 10, assuming that the driving condition to be met when the central control screen is projected is that the driving speed is less than 30km/h, the driving condition to be met when the instrument panel screen is projected is that the driving speed is less than or equal to 30km/h and less than 60km/h, and the driving condition to be met when the steering wheel screen is projected is that the driving speed is less than or equal to 60km/h and less than 100km/h (i.e., the driving speed is considered to be too fast when the speed is higher than 100km/h, and the vehicle is not suitable for screen projection). In the embodiment of the present application, it is assumed that the first display module of the vehicle is a central control screen, and the screen projection information of the application program "QQ" running on the mobile phone is being displayed on the central control screen, then the vehicle will obtain the driving condition that needs to be met when the central control screen is projected, for example, as shown in table 10, the driving condition that needs to be met is "driving speed < 30 km/h". When the driving speed of the vehicle is less than 30km/h, the screen projection information of the QQ is continuously projected on the central control screen, when the driving speed of the vehicle is 55km/h, the vehicle judges that the driving speed is not accordant with the driving speed required in the driving conditions (namely the driving speed is less than 30 km/h), the vehicle further determines that the driving speed is 55km/h and is in the range of the driving speed being less than or equal to 30km/h and less than 60km/h, and the table 10 shows that the driving condition of the driving speed being less than or equal to 30km/h and less than 60km/h is the driving condition which needs to be met when the screen of the instrument panel screen is projected, so the vehicle can switch the screen projection information of the QQ to the instrument panel screen for displaying.

It should be noted that the division of fig. 27 based on the driving speed range is only an illustration, and the user can set the driving conditions (i.e. the "driving speed" parameter is configurable) to be satisfied by each display module on the vehicle according to the driving habits, the screen viewing habits, etc., which is not limited herein.

In the above embodiment of the present application, the screen projection information of the application running on the terminal device is selected to be projected onto different display modules based on different driving speeds, where the different driving speeds substantially aim at driving safety and are not high in safety, and the screen projection information should be presented on the display module which is visible and most easily visible by the driver with the least cost.

In the embodiment of the present application, when the characterizing parameter of the driving information is "the type of the traffic scene where the vehicle is located", the driving information of the vehicle does not conform to the first driving condition, and the driving information of the vehicle conforms to the second driving condition, that is: the first driving condition comprises that the traffic scene type belongs to a first traffic scene type, and the second driving condition comprises that the traffic scene type belongs to a second traffic scene type; the traffic scene type of the vehicle does not belong to the first traffic scene type, and the traffic scene type of the vehicle belongs to the second traffic scene type.

It should be noted that, in the embodiment of the present application, a traffic scene refers to a scene formed by communication participants (such as vehicles, pedestrians, and the like) and objects (such as traffic signs, obstacle signs, and the like) on a traffic network. As shown in fig. 28, fig. 28 is a typical traffic scene of an intersection, in which traffic participants are various vehicles (pedestrians are not shown) and traffic lights, street lamps, zebra crossings, etc. (i.e., objects). It should be noted that the traffic scene can be divided into different types of traffic scenes according to a preset division rule, for example, one division manner can be that the traffic scenes are divided according to traffic signs, and the traffic scenes can be divided into "crossroads", "T-shaped intersections", "traffic light intersections (displaying red lights)", "turning intersections", "straight roads", and the like; one type of division may also be based on road conditions and surrounding traffic participants, and may divide traffic scene types into "freeway", "mountain road", "country road", "urban road", and so on. In the embodiment of the application, there is no limitation on what principle the traffic scene types are divided based on.

For ease of understanding, the following examples are given for illustration: suppose that the vehicle has 3 display modules, namely a central control screen, a steering wheel screen and an instrument panel screen, and the driving information is characterized by a parameter "traffic scene type where the vehicle is located", as shown in table 11 specifically, assuming that the traffic scene type is divided according to the traffic identification, and the traffic scene types are divided into crossroads, T-shaped crossroads, traffic light crossroads (displaying red light), turning crossroads and straight roads, and assuming that the driving conditions to be met when the central control screen is projected are that the traffic scene types of the vehicles are traffic light intersections (displaying red lights) and straight roads, the driving conditions to be met when the dashboard screen is projected are that the traffic scene types of the vehicles are crossroads and T-shaped intersections, and the driving conditions to be met when the steering wheel screen is projected are that the traffic scene types of the vehicles are turning intersections. In this embodiment of the application, it is assumed that the first display module of the vehicle is a central control screen, and the screen projection information of the application program "WeChat" running on the mobile phone is being displayed on the central control screen, then the vehicle will obtain the driving condition that needs to be met when the central control screen is projected, for example, as shown in table 11, the driving condition that needs to be met is that the traffic scene type where the vehicle is located is "traffic light intersection (displaying red light)" or "straight road". When a driver drives a vehicle (or an automatic driving vehicle drives by itself) to a turning intersection, the vehicle judges that the vehicle is located at the turning intersection by the self-navigation positioning module and/or the sensor module, then the vehicle can further judge that the traffic scene where the vehicle is located does not accord with the traffic scene required in the driving conditions (namely, a traffic light intersection (displaying red light) or a straight road), and the table 10 can know that the vehicle further determines that the traffic scene where the vehicle is located belongs to the traffic scene type (namely, the turning intersection) which needs to be met when a steering wheel screen is projected, so that the vehicle can switch the projection screen information of the WeChat application program to the steering wheel screen for displaying.

TABLE 11 running conditions to be satisfied when different display modules are projected (traffic scene type of vehicle is an example)

Display module	Driving conditions to be satisfied (type of traffic scene where vehicle is located)
		Central control screen	Traffic light crossing (displaying red light) and straight road
Instrument panel screen	Crossroad, T-shaped crossing
		Steering wheel screen	Turning intersection
……	……

It should be noted that the above division based on the traffic scene type of the vehicle is only an illustration, and a user may set the driving conditions that each display module on the vehicle needs to satisfy (that is, "the traffic scene type of the vehicle" parameter is configurable) according to driving habits, screen watching habits, and the like, which is not limited herein.

In the above embodiment of the application, the screen projection information of the application running on the terminal device is selected to be projected onto different display modules based on different traffic scene types of the vehicle, where the different traffic scene types of the vehicle are substantially aimed at driving safety, and the situation that the safety is not high is selected, the screen projection information should be presented on the display module which can be seen most easily and can be seen with least cost for the driver, for example, when the vehicle is at an intersection which lights a red light, the driver has stopped the vehicle at a waiting time for the road lamp to change from the red light to the road lamp, and at this time, the driver can not need so high attention, the screen projection information can be projected onto a screen (such as a central control screen) which can be seen by the driver by slightly twisting the head, when the vehicle is at a turn, since the turn intersection is generally a traffic accident high-occurring area, at this time, the driver needs to have sufficiently high attention, the screen projection information may be projected to a screen (e.g., a steering wheel screen) that the driver can see with minimal attention.

In the embodiment of the present application, when the characterizing parameter of the running information is "the number of passengers in the vehicle", then the running information of the vehicle does not conform to the first running condition, and the running information of the vehicle conforms to the second running condition is: the first driving condition includes that the number of passengers is within a first passenger number range, and the second driving condition includes that the number of passengers is within a second passenger number range; the number of passengers in the vehicle is not within the first range of passenger numbers and the number of passengers in the vehicle is within the second range of passenger numbers.

It should be noted that in the embodiment of the present application, the number of passengers in the vehicle is not included in the driver, for example, when only the driver is in the vehicle, the number of passengers in the vehicle is 0.

For ease of understanding, the following examples are given for illustration: it is assumed that the vehicle has 3 display modules in total, which are respectively a "central control screen", a "steering wheel screen", and a "dashboard screen", and the driving information is represented by a parameter "number of passengers in the vehicle", specifically as shown in table 12, it is assumed that the driving condition to be satisfied when the central control screen is projected is that the number of passengers in the vehicle is greater than or equal to 1 (for example, a passenger is riding on a passenger car), and the driving condition to be satisfied when the dashboard screen is projected is that the number of passengers in the vehicle is 0, and the steering wheel screen is not projected. In the embodiment of the present application, it is assumed that the first display module of the vehicle is a central control screen, and the screen projection information of the application program "WeChat" running on the mobile phone is being displayed on the central control screen, then the vehicle will obtain the driving condition that needs to be met when the central control screen is projected, for example, as shown in table 12, the driving condition that needs to be met is that the number of passengers in the vehicle is greater than or equal to 1, if a passenger sits on a passenger seat in a passenger seat, and when the passenger seat in the passenger seat gets off the vehicle at a certain time, only one driver (i.e., the number of passengers in the vehicle is 0) in the vehicle, then the vehicle will determine that the number of passengers in the vehicle does not match the number of passengers in the vehicle (i.e., "red light display (or" straight road ") required in the driving condition, as can be known from table 12, the vehicle further determines that the number of passengers in the vehicle belongs to the range of passengers that, therefore, the vehicle switches the screen projection information of the application program 'WeChat' to the dashboard screen for displaying.

TABLE 12 running conditions to be satisfied when different display modules are projected (number of passengers in vehicle as an example)

Display module	Required driving conditions (number of passengers in vehicle)
		Central control screen	The number of passengers in the vehicle is more than or equal to 1
Instrument panel screen	Number of passengers in vehicle is 0
		Steering wheel screen	Non-projection screen
……	……

It should be noted that, in the embodiment of the present application, whether there is a passenger in the vehicle, several passengers, and what seat the passenger is in the vehicle (e.g., whether there is a passenger in the secondary driver) may be obtained based on a camera mounted in the vehicle (i.e., based on image recognition), a sensor device mounted on the seat, and the like, which is not described herein in detail.

It should be noted that, in some embodiments of the present application, the number of passengers in the vehicle may also determine, in combination with other parameters, to which display module the screen projection information of the application running on the terminal device is projected, for example, the privacy degree of the application, such as "WeChat", "QQ", and other programs belong to an application with high privacy, and when the number of passengers in the vehicle is greater than or equal to 1, that is, there are other people in the vehicle besides the driver, such an application with high privacy may project a screen to a dashboard screen or a steering wheel screen that is only seen by the driver when the vehicle runs; and the application programs such as 'flight news video' and 'QQ music' belong to application programs with low privacy, when the number of passengers in the vehicle is more than or equal to 1, the application programs with low privacy can be projected to a central control screen and the like which can be seen by both a driver and the passengers when the application programs are operated in consideration of the requirement of common use. Referring to table 13 in detail, table 13 is only an exemplary manner, and in some embodiments of the present application, it may be determined at will which application programs or application types belong to application programs with high privacy (i.e., the degree of privacy is configurable), which is not limited herein.

Watch 13, driving conditions to be satisfied when different display modules are projected (privacy is considered)

It should be noted that the above tables 12 and 13 are only illustrated based on the division of the number of passengers in the vehicle, and the user can set the driving conditions (that is, "the number of passengers in the vehicle" parameter may be configured) that each display module on the vehicle needs to meet by himself or herself according to the driving habit, the habit of viewing the screen, and the like.

In the above embodiment of the application, screen projection information of an application running on a terminal device is selected to be projected onto different display modules based on different numbers of passengers in a vehicle, the different numbers of passengers in the vehicle substantially aim at privacy and commonality of the screen projection information, when the application running on the terminal device belongs to an application with high privacy (e.g., WeChat), in order to avoid other passengers from seeing the screen projection information, the application with high privacy can be projected onto a dashboard screen or a steering wheel screen which is only seen by a driver, and when the application running on the terminal device belongs to an application with low privacy (e.g., flight video), considering the demand of common use, the application with low privacy can be projected onto a central control screen which can be seen by both the driver and the passengers during running.

In the embodiment of the present application, when the characterizing parameter of the running information is "fatigue state of the driver of the vehicle", then the running information of the vehicle does not conform to the first running condition, and the running information of the vehicle conforms to the second running condition is: the first driving condition includes that the fatigue state of the driver belongs to a first fatigue state, and the second driving condition includes that the fatigue state of the driver belongs to a second fatigue state; the fatigue state of the driver of the vehicle does not belong to the first fatigue state, and the fatigue state of the driver of the vehicle belongs to the second fatigue state.

It should be noted that, in the embodiment of the present application, the fatigue state of the driver of the vehicle may be calculated from human body state information detected by a sensor module disposed in the vehicle, or may be calculated from human body state information detected by a wearable device (e.g., a smart band, a smart watch, etc.) worn by the driver. In some embodiments of the present application, the fatigue status of the driver may be classified according to the number of display modules of the vehicle, for example, assuming that there are 3 display modules on the vehicle, the fatigue status of the driver may be classified into 3 grades of "fatigue", "sub-fatigue", and "mental"; assuming that there are 2 display modules on the vehicle, the fatigue state of the driver can be divided into 2 grades of fatigue and spirit, and the corresponding driving behavior can be known to be safe or not based on different fatigue states of the driver.

For ease of understanding, the following examples are given for illustration: it is assumed that there are 3 display modules in total on the vehicle, which are respectively a "central control screen", a "steering wheel screen", and a "dashboard screen", and the driving information is represented by a parameter "driver fatigue state", specifically, as shown in table 14, it is assumed that the driving condition to be satisfied when the central control screen is projected is the driver fatigue state is "mental", the driving condition to be satisfied when the dashboard screen is projected is the driver fatigue state is "sub-fatigue", and the driving condition to be satisfied when the steering wheel screen is projected is the driver fatigue state is "fatigue". In the embodiment of the present application, it is assumed that the first display module of the vehicle is a central control screen, and the screen projection information of the application program "QQ" running on the mobile phone is being displayed on the central control screen, then the vehicle will obtain the driving condition to be satisfied when the central control screen is projected, for example, as shown in table 14, and the driving condition to be satisfied is that the fatigue state of the driver is "mental". When the sensor module disposed in the vehicle calculates that the fatigue state of the driver is "mental" (obtained by periodic calculation) based on the detected human body state information of the driver, or when the wearable device (e.g., smart bracelet, smart watch, etc.) worn on the driver calculates that the fatigue state of the driver is "mental" (obtained by periodic calculation) based on the detected human body state information of the driver, the screen projection information of "QQ" is continuously projected on the central control screen, and in a certain detection period, when the fatigue state of the driver is calculated to be "sub-fatigue", the vehicle determines that the fatigue state of the driver does not conform to the fatigue state (i.e., "mental") of the driver required in the driving condition, as can be known from table 14, the fatigue state of the driver is the fatigue state of the driver required when the dashboard screen is projected, therefore, the vehicle switches the screen projection information of "QQ" to the dashboard screen for display.

Watch 14, driving conditions to be met when different display modules are projected (fatigue state of driver as an example)

Display module	Driving conditions to be satisfied (fatigue state of driver)
		Central control screen	Fatigue grade is "spirit"
Instrument panel screen	Fatigue grade is sub-fatigue "
		Steering wheel screen	Fatigue grade is "fatigue"
……	……

It should be noted that the above table 14 is only illustrated based on the division of the fatigue state of the driver, and the user can set the driving conditions to be satisfied by each display module on the vehicle (i.e. the "fatigue state of the driver" parameter is configurable) according to the driving habits, the screen viewing habits, and the like, which is not limited herein. For example, in some embodiments of the present application, if the fatigue state of the driver is "fatigue", the driving behavior of the driver may be considered unsafe, the screen projection information of the application running on the terminal device is not projected, and the vehicle may further remind the driver to stop for a rest through the voice module.

In the above embodiment of the present application, the screen projection information of the application running on the terminal device is selected to be projected onto different display modules based on different fatigue states of the driver, different driving speeds substantially aim at driving safety, when the fatigue state of the driver is at a high fatigue level, which means that the safety of the driving behavior at the time is not high, the screen projection information should be presented on the display module which can be seen and is most easily seen by the driver with the least cost, for example, when the fatigue state of the driver is "fatigue", which requires the driver to use less energy and attention for driving the vehicle, the screen projection information can be projected onto a screen (for example, a steering wheel screen) which can be seen by the driver with the least attention, when the fatigue state of the driver is "mental", which means that the driver is full of energy at the time, the screen projection information can be projected onto a screen which can be seen by the driver with a little head twist (for example, a center control screen).

In the embodiment of the present application, when the characterizing parameter of the running information is "distance between the vehicle and the obstacle", then the running information of the vehicle does not conform to the first running condition, and the running information of the vehicle conforms to the second running condition is: the first driving condition includes a distance between the vehicle and the obstacle being within a first distance range, and the second driving condition includes a distance between the vehicle and the obstacle being within a second distance range; the distance between the vehicle and an obstacle is not within the first distance range and the distance between the vehicle and an obstacle is within the second distance range.

It should be noted that, in the embodiment of the present application, the distance between the vehicle and the obstacle may be calculated based on the perception information collected by the sensor (e.g., laser radar, camera, etc.) disposed on the vehicle, and according to the collected perception information, the distance between the vehicle and each obstacle and the orientation and position of the obstacle relative to the vehicle may be obtained. As shown in fig. 29, the traffic scene in which the vehicle illustrated in fig. 29 travels on the expressway, where the area enclosed by the dotted line indicates the safe driving area of the host vehicle, and no obstacle (e.g., surrounding vehicle, pedestrian, roadblock, etc.) exists within the area, indicating that the driving environment of the host vehicle is safe, and in fig. 29, the distances between the surrounding vehicle and the host vehicle at the front, rear, left, and right of the host vehicle are 23m, 21m, 2m, and 3m, respectively, and are all outside the safe area, indicating that the driving environment of the host vehicle is safe. If an obstacle appears in the safe area during the driving process of the self-vehicle, the driving environment of the self-vehicle is unsafe, and the driver needs to concentrate on careful driving so as to avoid collision with the obstacle. The closer the obstacle is to the vehicle, the shorter the time left for the driver to react is, and the greater the risk of collision between the vehicle and the obstacle is, so in this application embodiment, the screen projection information of the application running on the terminal device can be selected to be projected onto different display modules based on different distances between the vehicle and the obstacle.

It should be noted that in some embodiments of the present application, the distance between the vehicle and the obstacle may be the distance between the vehicle and a plurality of obstacles, and the more obstacles, the more the driver needs to concentrate on driving; the distance between the vehicle and the obstacle can also be the distance between the vehicle and the nearest obstacle, and if the obstacle closest to the vehicle is located outside the safety area of the vehicle, it indicates that other obstacles are also far away from the vehicle, and the driving environment of the vehicle is safe at this time.

For ease of understanding, the following examples are given for illustration: suppose that the vehicle has 3 display modules in total, namely a central control screen, a steering wheel screen and an instrument panel screen, and the driving information is represented by a parameter "distance between the vehicle and the obstacle", specifically as shown in table 15, wherein h is used for representing the distance between the vehicle and the obstacle, the driving condition to be met when the central control screen is projected is assumed to be "h is greater than or equal to 20 m", the driving condition to be met when the instrument panel screen is projected is assumed to be "15 m is less than or equal to h less than 20 m", and the driving condition to be met when the steering wheel screen is projected is assumed to be "h less than 15 m". In the embodiment of the present application, it is assumed that the first display module of the vehicle is a central control screen, and the screen projection information of the application program "QQ" running on the mobile phone is being displayed on the central control screen, then the vehicle will obtain the driving condition that needs to be met when the central control screen is projected, for example, as shown in table 15, the driving condition that needs to be met is "h ≧ 20 m". When the sensor module deployed in the vehicle periodically calculates that h is more than or equal to 20m, the screen projection information of the QQ is continuously projected on the central control screen, in a certain calculation period, when the calculated value is that h is more than or equal to 15m and less than 20m, the vehicle judges that the distance between the vehicle and the obstacle is not in accordance with the distance between the vehicle and the obstacle required in the driving condition (namely, h is more than or equal to 20 m), and the table 15 shows that the fatigue state of the driver is the distance between the vehicle and the obstacle when the dashboard screen is projected, so that the vehicle can switch the screen projection information of the QQ to the dashboard screen for display.

Watch 15, driving conditions to be satisfied when different display modules are projected (distance between vehicle and obstacle is taken as an example)

Display module	The driving condition to be satisfied (distance h between vehicle and obstacle)
		Central control screen	h≥20m
Instrument panel screen	15m≤h＜20m
		Steering wheel screen	h＜15m
……	……

It should be noted that, in some embodiments of the present application, the distance between the vehicle and the obstacle may be further subdivided according to the orientation of the obstacle located in the vehicle, the obstacle is located in different orientations of the vehicle, and the distance requirement is different, for example, with reference to fig. 29, when the obstacle is located at the left or right of the vehicle, the distance requirement between the vehicle and the obstacle may be smaller (e.g., 1.5m), when the obstacle is located at the front or rear of the vehicle, because the vehicle may be faster in driving, there may be a braking distance, so the distance requirement between the vehicle and the obstacle may be larger (e.g., 20m), which may be referred to specifically as table 16, and table 16 is only one way, in some embodiments of the present application, a distance threshold value between the vehicle and the obstacle when the screen cut display is set by the user himself or herself, the details are not limited herein.

Watch 16, driving conditions to be satisfied when different display modules are projected (considering the orientation between the vehicle and the obstacle)

It should be noted that the division based on the distance h between the vehicle and the obstacle in the above tables 15 and 16 is only an illustration, and the user can set the driving condition (that is, the parameter "distance between the vehicle and the obstacle" is configurable) that each display module on the vehicle needs to satisfy according to the driving habit, the habit of watching the screen, and the like, and the specific details are not limited herein.

In the above embodiment of the application, the screen projection information of the application running on the terminal device is selected to be projected on different display modules based on different distances between the vehicle and the obstacle, the different distances between the vehicle and the obstacle are also substantially aimed at driving safety, when the distance between the vehicle and the obstacle is shorter, the reaction time left for the driver is shorter, the risk of collision between the vehicle and the obstacle is higher, the driving environment of the vehicle is unsafe, the driver needs to concentrate on the attention to drive so as to avoid collision with the obstacle, and the screen projection information can be projected on a screen (such as a steering wheel screen) which can be seen by the driver with the least attention; when the distance between the vehicle and the obstacle is longer, the driving environment of the vehicle is safer, and the screen projection information can be projected to a screen (such as a central control screen) which can be seen by a driver by slightly twisting his head.

In the above modes a to E, the description is performed based on the vehicle driving information as a single parameter (e.g., "driving speed", "traffic scene type", "number of passengers in the vehicle", "fatigue state of driver", "distance between vehicle and obstacle", etc.), in some embodiments of the present application, the single parameters may be combined with each other to obtain a screen-cut display mode based on the combined parameter, and the screen-cut display process of the combined parameter is similar to that of the above modes a to E, and is not repeated here.

In the embodiments shown in fig. 26 to fig. 29, it is described how, after the terminal device finishes screen projection of the application program to be screen projected on the terminal device (for example, based on the screen projection manner shown in fig. 8 to fig. 25), the vehicle determines whether screen-cut display needs to be performed in different driving scenes in combination with the driving information of the vehicle itself, the screen projection method provided in the embodiment corresponding to fig. 26 is that the vehicle implements a screen-cut display function, in some embodiments of the present application, the terminal device may also implement a screen-cut display function, and another display information switching method provided in the embodiment of the present application is described below, specifically referring to fig. 30, where fig. 30 is another flow diagram of the display information switching method provided in the embodiment of the present application, and specifically includes the following steps:

3001. The terminal equipment sends a first instruction to the vehicle, wherein the first instruction is used for indicating screen projection information of the first application program to be displayed on a first display module of the vehicle.

First, the terminal device sends a first instruction to the vehicle, where the first instruction is used to instruct to display screen projection information of a first application program on a first display module of the vehicle, and the first application program may be an application program running on the terminal device.

3002. The terminal device acquires the traveling information of the vehicle.

After the vehicle acquires the running information of the vehicle, the acquired running information of the vehicle is further sent to the terminal device, and if the vehicle periodically acquires the running information of the vehicle, the terminal device also periodically receives the running information of the vehicle sent by the vehicle.

3003. When the running information of the vehicle is not in accordance with the first running condition and the running information of the vehicle is in accordance with the second running condition, the terminal device sends a second instruction to the vehicle, wherein the second instruction is used for indicating a second display module of the vehicle to display the information of the first application program, the first running condition is a condition which needs to be met when the first display module is projected on a screen, and the second running condition is a condition which needs to be met when the second display module is projected on the screen.

The method comprises the steps that the terminal equipment obtains a first driving condition which needs to be met by a first display module displaying screen projection information, the first display module is one of a plurality of display modules on a vehicle, and the screen projection information is screen projection information of screen projection of a first application program running on the terminal equipment to the first display module by the terminal equipment under the condition that the terminal equipment is in communication connection with the vehicle.

If the first display module on the vehicle is displaying the screen projection information of the first application program running on the terminal equipment, the terminal equipment also acquires the first running condition which needs to be met by the first display module.

The terminal device further determines whether the driving information of the vehicle is in accordance with the first driving condition after acquiring the driving information of the vehicle and the first driving condition, and sends a second instruction to the vehicle when the driving information of the vehicle is not in accordance with the first driving condition, wherein the second instruction is used for switching the information of the first application program (such as an operation interface of the first application program or a part of the operation interface) to a second display module corresponding to a second driving condition in accordance with the first driving condition for display, and the second display module is another one of a plurality of display modules on the vehicle.

It should be noted that, in some embodiments of the present application, the process of switching the screen projection information by the terminal device to display on the second display module corresponding to the second driving condition that meets the first driving condition specifically includes, but is not limited to, the following manners:

in the mode 1, the terminal equipment firstly stops screen projection to the first display module on the vehicle, and then projects screen projection information to the second display module corresponding to the second driving condition which is in accordance with the driving information of the vehicle. That is, the screen projection to the first display module is terminated, and the screen projection to the second display module is resumed.

In the mode 2, the terminal device generates a switching instruction (which may be referred to as a second switching instruction) and sends the second switching instruction to the vehicle, the second switching instruction is used for instructing the vehicle to switch the screen projection information to a second display module corresponding to a second driving condition which is in accordance with the driving information of the vehicle for display, and after receiving the second switching instruction, the vehicle switches the screen projection information to the second display module according to the second switching instruction.

It should be noted that, in the embodiment of the present application, since the driving information of the vehicle may be characterized by a plurality of parameters, for example, the driving speed of the vehicle, the type of traffic scene where the vehicle is located, the number of passengers in the vehicle, the fatigue state of the driver of the vehicle, the distance between the vehicle and the obstacle, and the like, described above. When the driving information is represented by different parameters, the conditions to be satisfied by switching the display are different, and the following description is separately given:

A. the running information of the vehicle is a driving speed of the vehicle.

For a detailed process, please refer to the method a in step 2602, which is not described herein.

For a detailed process, please refer to the method B in step 2602, which is not described herein.

For a detailed process, please refer to the method C in step 2602, which is not described herein.

For a detailed process, please refer to the method D in step 2602, which is not described herein.

For a specific process, please refer to the manner E in step 2602, which is not described herein.

3004. And the vehicle displays the screen projection information of the first application program on the second display module according to the second instruction.

And when the vehicle receives the second instruction, displaying the screen projection information of the first application program on the second display module according to the second instruction.

Similarly, in the modes a to E, the description is performed based on the driving information of the vehicle as a single parameter (e.g., "driving speed", "traffic scene type", "number of passengers in the vehicle", "fatigue state of driver", "distance between vehicle and obstacle", etc.), in some embodiments of the present application, the single parameters may be combined with each other to obtain a screen-cut display mode based on the combined parameter, and the screen-cut display process of the combined parameter is similar to the single parameter in the modes a to E, which is not described herein in detail.

On the basis of the embodiment corresponding to fig. 8, in order to better implement the above-mentioned solution of the embodiment of the present application, a related apparatus for implementing the above-mentioned solution is further provided below, specifically referring to fig. 31, where fig. 31 is a schematic structural diagram of the apparatus provided in the embodiment of the present application, and the apparatus may be a terminal device or a vehicle, and is not limited herein. The apparatus 3100 comprises: an obtaining module 3101 and a screen projecting module 3102, wherein the obtaining module 3101 is configured to obtain a first screen projecting condition of a first display module projected on a vehicle, where the first screen projecting condition may be related to driving information of a target vehicle or related to characteristic information of an application program to be projected, and is not limited herein; the screen projecting module 3102 is configured to project screen projecting information of a first screen application to be projected on the terminal device to a first display module when a first screen projecting condition is met, where the first display module is one of a plurality of display modules on a vehicle, and different display modules on the vehicle have different screen projecting conditions. Specifically, in a case where the first screen projecting condition is set according to the traveling information of the vehicle, satisfying the first screen projecting condition includes: when the driving information of the vehicle meets the first screen projection condition; under the condition that the first screen projection condition is set according to the characteristic information of the application program to be projected, when the first screen projection condition is met, the method comprises the following steps: and when the characteristic information of the first screen projection application program to be projected on the terminal equipment meets the first screen projection condition.

In one possible design, the obtaining module 3101 is specifically configured to: acquiring a plurality of screen projection conditions of a plurality of display modules projected on the vehicle, wherein one display module corresponds to one screen projection condition, and the screen projection condition is set according to characteristic information of an application program to be projected, for example, if a target vehicle A is provided with 3 different display modules which are respectively a dashboard screen, a steering wheel screen and a central control screen, each display module has one corresponding screen projection condition, the total number of the screen projection conditions is 3, namely, a screen projection condition 1, a screen projection condition 2 and a screen projection condition 3; the screen projection module 3102 is configured to, when a screen projection condition (which may be referred to as a first screen projection condition) that a certain display module (which may be referred to as a first display module) on the vehicle is projected is met by a screen application to be projected (which may be referred to as a first screen projection application) on the terminal device, and when the terminal device establishes a communication connection with the vehicle, project screen projection information of the first screen application to be projected to the first display module.

In the above embodiment of the present application, the obtaining module 3101 obtains the screen projection conditions that need to be met when each display module on the vehicle is projected (for example, the application program to be projected on the terminal device belongs to a certain application type, a preset use frequency is reached, and the like), and when the screen projection conditions are met, the screen projection module 3102 projects the display information of the application program to be projected on the terminal device to the corresponding display module, so that the display information on the terminal device is reasonably distributed to different display modules on the vehicle, a driver can quickly obtain the information on the terminal device, the probability of distraction of the driver is reduced, the interference of the screen projection information on the driver driving is reduced, and the driving safety is improved.

In one possible design, the screen projecting condition may be set according to the driving information of the vehicle, in which case, when the first screen projecting condition that the first display module on the target vehicle is projected is satisfied, the screen projecting condition includes: and when the running information of the target vehicle meets the first screen projection condition.

In one possible embodiment, the conditions to be met by the screen projection are different when the driving information of the vehicle is represented by different parameters. In one case, the running information of the vehicle may include: the driving speed of the vehicle. The first screen projecting condition includes the driving speed of the target vehicle being within a first driving speed range.

In one possible design, the driving information of the vehicle may include: the type of traffic scene the vehicle is in. The first screen projection condition comprises that the type of the traffic scene where the target vehicle is located belongs to the first traffic scene type.

In one possible design, the driving information of the vehicle may include: number of passengers in the vehicle. The first screen projection condition includes the number of passengers in the target vehicle being within a first range of passenger numbers.

In one possible design, the driving information of the vehicle may include: fatigue state of the vehicle driver. The first screen projecting condition includes that the fatigue state of the driver of the target vehicle belongs to a first fatigue state.

In one possible design, the driving information of the vehicle may include: the distance between the vehicle and the obstacle. The first screen projecting condition includes that the distance between the target vehicle and the obstacle is within a first distance range.

In one possible design of the first aspect, the screen projecting condition may be set based on characteristic information of the application program in addition to the traveling information of the vehicle. In this case, when a first screen projection condition that the first display module on the target vehicle is projected is satisfied, the method includes: and when the characteristic information of the first screen projection application program to be projected on the terminal equipment meets the first screen projection condition.

In a possible design, the feature information of the application may include an application type to which the application belongs, that is, the screen projection condition is that the specific application type corresponds to the specific display module, in this case, the meeting of the first screen projection condition of the first display module projected on the vehicle by the first screen application to be projected on the terminal device includes: the first screen projection condition includes that the application type of the application program is a first application type (e.g., social class), and meanwhile, the first application program to be projected on the terminal device belongs to the first application type, which may also be referred to as an application type of the application program being projected on the terminal device, which belongs to the application type required in the first screen projection condition.

In a possible design, the characteristic information of the application may include usage frequency information of the application, that is, the screen projection condition is that a specific usage frequency range of the application on the terminal device corresponds to a specific display module, and the usage frequency of the application on the terminal device may be obtained from functions such as "healthy use of mobile phone" of the terminal device, in which case, the meeting of the first screen projection condition of the first display module projected on the vehicle by the first screen application to be projected on the terminal device includes: the first screen projection condition comprises that the use frequency information of the application program is in a first use frequency range (such as a high use frequency range), and meanwhile, the use frequency of the first screen to be projected application program on the terminal equipment is in the first use frequency range.

In a possible design, the feature information of the application program may include both an application type to which the application program belongs and usage frequency information of the application program, in which case, the meeting of the first screen projection condition that the first display module on the vehicle is projected by the first screen to be projected on the terminal device includes: the application type of the first screen projection application program to be projected on the terminal device belongs to the application type required in the first screen projection condition, and the use frequency of the first application program on the terminal device is within the use frequency range required in the first screen projection condition.

In the above embodiment of the application, the single factors (for example, the application type is a single factor, and the application program use frequency range is a single factor) are combined with each other to obtain the division result based on the combined factors, so that the conditions required to be met by screen projection are more diversified, and the user experience is improved.

In one possible design, the frequency range used by the application on the terminal device may be characterized as follows: 1) the use duration of the application program on the terminal equipment; 2) and starting the running times of the application program on the terminal equipment.

In one possible design, one application may simultaneously satisfy the characteristic information of the applications required in the plurality of screen projection conditions. For example, the application type "social class" (e.g., "WeChat", "QQ", "short message", etc.) to which the application belongs may be the application type required in the screen projection condition 1, or may be the application type required in the screen projection condition 2. That is, when the first screen-projection-awaiting application on the terminal device simultaneously satisfies a first screen projection condition that the first display module on the vehicle is projected and a second screen projection condition that the second display module on the vehicle is projected, in this case, the screen projection module 3102 may be configured to: and simultaneously projecting screen projection information of the first application program to a first display module and a second display module, wherein the first display module and the second display module are two different display modules on the vehicle. It should be noted that, in the embodiment of the present application, the premise that the terminal device simultaneously projects the screen projection information of the first screen application to be projected to the first display module and the second display module is that neither the first display module nor the second display module is occupied by screen projection.

In the above embodiments of the present application, it is specifically stated that when a certain application program meets the screen projection condition that a plurality of display modules are projected on a screen, the screen projection module 3102 projects the application program on the plurality of display modules, and a user selects to view screen projection information on which display module, so that flexibility is provided.

In one possible design, one application may simultaneously satisfy the characteristic information of the applications required in the plurality of screen projection conditions. That is, when the first screen-projection-awaiting application on the terminal device simultaneously satisfies the first screen-projection condition that the first display module on the vehicle is projected and the second screen-projection condition that the second display module on the vehicle is projected, then the screen-projection module 3102 may also be configured to: the screen projection information of the first application program is divided into first screen projection sub-information and second screen projection sub-information according to a preset mode, the first display module and the second display module are two different display modules on the vehicle, and then the terminal device respectively projects the first screen projection sub-information and the second screen projection sub-information to the first display module and the second display module.

In the foregoing embodiment of the present application, it is specifically stated that when a certain application program meets the screen projection condition that a plurality of display modules are projected on a screen, the screen projection module 3102 correspondingly splits screen projection information of the application program into a plurality of screen projection sub-information, one screen projection sub-information is projected on one display module, and a user selects to view the screen projection sub-information on which display module by himself, so that flexibility is achieved.

In one possible design, one application may simultaneously satisfy the characteristic information of the applications required in the plurality of screen projection conditions. That is, when the first screen-projection-awaiting application program on the terminal device simultaneously satisfies the first screen-projection condition that the first display module on the vehicle is projected and the second screen-projection condition that the second display module on the vehicle is projected, then the screen-projection module 3102 may be further configured to: judging whether a first display module with a high priority is occupied by screen projection according to a preset priority, wherein the first display module and the second display module are two different display modules in a plurality of display modules on a vehicle; if the first display module is not occupied by screen projection, the terminal device projects screen projection information of the first application program to the first display module preferentially, and only under the condition that the first display module is occupied by screen projection, the terminal device projects screen projection information of the first application program to the second display module.

In the above embodiments of the present application, it is specifically stated that when a certain application program meets the screen projection condition that a plurality of display modules are projected on a screen, the screen projection module 3102 may further selectively project the screen based on the priority, preferentially project screen projection information to a display module with a higher priority, and project the screen to a display module with a next priority only when the display module with the higher priority is occupied by the screen projection, so that the screen projection mode according to the priority ensures that the user can view screen projection information in an optimum mode in real time, and the influence of the dispersed attention of the user is reduced to the greatest extent.

It should be noted that, the contents of information interaction, execution process, and the like between the modules/units in the device 3100 provided in fig. 31 are based on the same concept as the method embodiment corresponding to fig. 8 in the present application, and specific contents may refer to the description in the foregoing method embodiment in the present application, and are not described herein again.

An embodiment of the present application further provides a device, where the device may be a vehicle, specifically referring to fig. 32, where fig. 32 is a schematic structural diagram of the device provided in the embodiment of the present application, and the device 3200 includes: the system comprises an acquisition module 3201 and a screen cutting module 3202, wherein the acquisition module 3201 is used for acquiring the running information of the vehicle; the screen switching module 3202 is configured to switch screen projection information of the first application program displayed on the first display module to the second display module to be displayed when the driving information of the vehicle does not conform to the first screen projection condition, where the first screen projection condition is a condition that needs to be met when the first display module is projected, and the first display module and the second display module are two different display modules in the plurality of display modules on the vehicle.

In one possible design, the screen cropping module 3202 is specifically configured to: when the running information of the vehicle is not accordant with the first running condition and the running information of the vehicle is accordant with the second running condition, switching screen projection information of the first application program displayed on the first display module to the second display module to be displayed, wherein the second running condition is a condition which needs to be met when the second display module is projected.

In the above embodiment of the present application, after the screen projection information on the terminal device has been projected to the first display module at the vehicle end, the vehicle can further judge whether screen switching display is needed in different driving scenes by combining the driving information of the vehicle, only the first screen projection condition corresponding to the first display module is not accordant with the current driving condition of the vehicle, and the other display modules (e.g., the second display module) and the current driving condition of the vehicle are met, the screen-switching module 3202 instructs to switch the screen-projecting information of the first application program displayed on the first display module to the second display module for display, that is, the dynamic switching can be carried out between different display modules by combining the driving state of the vehicle, so that the user can watch the screen projection information in the best mode under any driving condition, and the user experience is improved.

In one possible design, the driving information of the vehicle may include: the driving speed of the vehicle, in which case the driving information of the vehicle does not comply with the first screen projecting condition, and the complying of the driving information of the vehicle with the second screen projecting condition, includes: the first screen projection condition comprises that the driving speed is within a first driving speed range, and the second screen projection condition comprises that the driving speed is within a second driving speed range; the driving speed of the vehicle is not within the first driving speed range, and the driving speed of the vehicle is within the second driving speed range.

In the above embodiment of the application, the screen projection information of the application program running on the terminal device is selected to be projected to different display modules based on different driving speeds, so that the driving safety is improved.

In one possible design, the driving information of the vehicle may include: the type of traffic scene the vehicle is in. In this case, the step of determining that the driving information of the vehicle does not comply with the first screen projecting condition and the step of determining that the driving information of the vehicle complies with the second screen projecting condition includes: the first screen projection condition comprises that the traffic scene type belongs to a first traffic scene type, and the second screen projection condition comprises that the traffic scene type belongs to a second traffic scene type; the traffic scene type of the vehicle does not belong to the first traffic scene type, and the traffic scene type of the vehicle belongs to the second traffic scene type.

In the embodiment of the application, the screen projection information of the application program running on the terminal device is selected to be projected to different display modules based on different traffic scene types of the vehicle, so that the driving safety is improved.

In one possible design, the driving information of the vehicle may include: number of passengers in the vehicle. In this case, the step of determining that the driving information of the vehicle does not comply with the first screen projecting condition and the step of determining that the driving information of the vehicle complies with the second screen projecting condition includes: the first screen projection condition comprises that the number of passengers is within a first passenger number range, and the second screen projection condition comprises that the number of passengers is within a second passenger number range; the number of passengers in the vehicle is not within the first range of passenger numbers and the number of passengers in the vehicle is within the second range of passenger numbers.

In the above embodiment of the application, based on the consideration of privacy and commonality of the application programs, the number of passengers in the vehicle is different, and screen projection information of the application programs running on the terminal device can be selected to be projected onto different display modules, so that user experience is improved.

In one possible design, the driving information of the vehicle may include: fatigue state of the vehicle driver. In this case, the step of determining that the driving information of the vehicle does not comply with the first screen projecting condition and the step of determining that the driving information of the vehicle complies with the second screen projecting condition includes: the first screen projection condition comprises that the fatigue state of the driver belongs to a first fatigue state, and the second screen projection condition comprises that the fatigue state of the driver belongs to a second fatigue state; the fatigue state of the driver of the vehicle does not belong to the first fatigue state, and the fatigue state of the driver of the vehicle belongs to the second fatigue state.

In the above embodiment of the application, the screen projection information of the application program running on the terminal device is selected to be projected to different display modules based on different fatigue states of the driver, so that the driving safety is improved.

In one possible design, the driving information of the vehicle may include: the distance between the vehicle and the obstacle. In this case, the step of determining that the driving information of the vehicle does not comply with the first screen projecting condition and the step of determining that the driving information of the vehicle complies with the second screen projecting condition includes: the first screen projection condition comprises that the distance between the vehicle and the obstacle is within a first distance range, and the second screen projection condition comprises that the distance between the vehicle and the obstacle is within a second distance range; the distance between the vehicle and an obstacle is not within the first distance range and the distance between the vehicle and an obstacle is within the second distance range.

In the above embodiment of the application, the screen projection information of the application program running on the terminal device is selected to be projected to different display modules based on different distances between the vehicle and the obstacle, so that the probability of collision between the vehicle and the obstacle is reduced, and the driving safety is improved.

In one possible design, the screen cropping module 3202 may be configured to: firstly, a switching instruction (which may be called a first switching instruction) is generated, the first switching instruction is used for instructing the vehicle to switch the screen projection information to the second display module, and then the screen projection information is switched to the second display module corresponding to the second screen projection condition which accords with the driving information of the vehicle according to the first switching instruction to be displayed.

In the above embodiments of the present application, how the screen cutting module 3202 performs screen cutting display is specifically described, and the realizability is provided.

It should be noted that, the information interaction, the execution process, and the like between the modules/units in the device 3200 provided in fig. 32 are based on the same concept as the method embodiment corresponding to fig. 26 in the present application, and specific contents may refer to the description in the foregoing method embodiment in the present application, and are not described herein again.

An apparatus is also provided in this embodiment of the present application, where the apparatus may be a terminal device, specifically refer to fig. 33, and fig. 33 is a schematic structural diagram of the terminal device provided in this embodiment of the present application, and the apparatus 3300 includes: the system comprises a first sending module 3301, an obtaining module 3302 and a second sending module 3303, wherein the first sending module 3301 is used for sending a first instruction to a vehicle, and the first instruction is used for indicating a first display module of the vehicle to display screen projection information of a first application program; an obtaining module 3302, configured to obtain driving information of the vehicle; the second sending module 3303 is configured to send a second instruction to the vehicle when the driving information of the vehicle does not conform to the first screen projecting condition and the driving information of the vehicle conforms to the second screen projecting condition, where the second instruction is used to instruct a second display module of the vehicle to display information of the first application program, the first screen projecting condition is a condition that needs to be met when the first display module is projected, the second screen projecting condition is a condition that needs to be met when the second display module is projected, and the first display module and the second display module are two different display modules in the plurality of display modules on the vehicle.

In the above embodiment of the application, after the screen projection information on the terminal device is projected to the display module at the vehicle end, the terminal device can also be used for judging whether screen cutting display needs to be performed in different driving scenes by combining with the driving information of the vehicle, that is, dynamic switching can be performed between different display modules by combining with the driving state of the vehicle by the terminal device, so that a user can view the screen projection information in an optimal mode under any driving information, and user experience is improved.

It should be noted that, the contents of information interaction, execution process, and the like between the modules/units in the apparatus 3300 provided in fig. 33 are based on the same concept as the method embodiment corresponding to fig. 30 in the present application, and specific contents may refer to the description in the foregoing method embodiment in the present application, and are not described herein again.

It should be further noted that, in the embodiments of the present application, the embodiments corresponding to fig. 8 and fig. 30 can be implemented based on the terminal devices corresponding to fig. 2 to fig. 6, and the structure of the terminal device can specifically refer to the descriptions in the terminal devices corresponding to fig. 2 to fig. 6 shown in the foregoing description of the present application, which is not repeated herein.

Referring to fig. 34, fig. 34 is a schematic structural diagram of a vehicle according to an embodiment of the present application, and for convenience of description, only portions related to the embodiment of the present application are shown, and details of the vehicle are not disclosed, and refer to a method portion of the embodiment of the present application. The vehicle 3400 may be disposed with corresponding modules in the vehicle 3200 described in the embodiment corresponding to fig. 32, to implement the functions of the vehicle in the embodiment corresponding to fig. 32, specifically, the vehicle 3400 is implemented by one or more servers, and the vehicle 3400 may have a relatively large difference due to different configurations or performances, and may include one or more processors 3422, a memory 3432, one or more storage media 3430 (e.g., one or more mass storage devices) for storing the application programs 3442 or the data 3444. Specifically, the processor 3422 may be a Central Processing Unit (CPU), an embedded microcontroller, an AI processor, etc., and the type of the processor 3422 is not limited herein, wherein the memory 3432 and the storage medium 3430 may be a transient storage or a persistent storage. The program stored on the storage medium 3430 may include one or more modules (not shown), each of which may include a sequence of instructions for operating on the exercise device. Still further, the processor 3422 may be configured to communicate with the storage medium 3430 to execute a series of instruction operations in the storage medium 3430 on the vehicle 3400, for example, the processor 3422 may call the series of instruction operations in the storage medium 3430 to perform the following steps: compiling a source code to obtain a control flow graph corresponding to the source code, segmenting the obtained control flow graph to obtain a plurality of segmented sub-graphs, recompiling the obtained sub-graphs to obtain a plurality of linear execution sequences, simulating and executing the linear execution sequences in sequence, and inserting at least one operator between the linear execution sequences according to semantics corresponding to the linear execution sequences to connect the linear execution sequences into a computation graph.

The vehicle 3400 may also include one or more power supplies 3426, one or more wired or wireless network interfaces 3450, one or more input-output interfaces 3458, and/or one or more operating systems 3441, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so forth.

In this embodiment of the application, the steps performed by the deep learning framework in the foregoing various processing methods related to the control flow graph may be performed on the processor by invoking relevant codes stored on the storage medium based on the structure shown in the vehicle in fig. 34, which is not described herein in detail.

It should be noted that the above-described embodiments of the apparatus are merely schematic, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiments of the apparatus provided in the present application, the connection relationship between the modules indicates that there is a communication connection therebetween, and may be implemented as one or more communication buses or signal lines.

Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus necessary general-purpose hardware, and certainly can also be implemented by special-purpose hardware including special-purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose components and the like. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions may be various, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, the implementation of a software program is more preferable. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk of a computer, and includes instructions for enabling a computer device (which may be a personal computer, an exercise device, or a network device) to execute the methods described in the embodiments of the present application.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, training device, or data center to another website site, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a training device, a data center, etc., that incorporates one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

Claims

1. A screen projection method, comprising:

acquiring a first screen projection condition of a first display module projected on a target vehicle, wherein the first screen projection condition is related to running information of the target vehicle, or the first screen projection condition is related to characteristic information of an application program to be projected;

and when the first screen projection condition is met, projecting screen projection information of a first screen application program to be projected on the terminal equipment to the first display module, wherein the first display module is one of a plurality of display modules on the target vehicle, and different display modules on the target vehicle have different screen projection conditions.

2. The method according to claim 1, wherein the obtaining of the first screen projection condition for the first display module projected on the target vehicle includes:

the method comprises the steps of obtaining a plurality of screen projection conditions of a plurality of display modules projected on the target vehicle, wherein one display module corresponds to one screen projection condition, and the first screen projection condition is one of the plurality of screen projection conditions.

3. The method according to any one of claims 1-2, wherein the travel information of the target vehicle includes:

a driving speed of the target vehicle;

The first screen projecting condition includes a driving speed of the target vehicle being within a first driving speed range.

4. The method according to any one of claims 1-3, wherein the travel information of the target vehicle includes:

the type of traffic scene in which the target vehicle is located;

the first screen projection condition comprises that the type of the traffic scene where the target vehicle is located belongs to a first traffic scene type.

5. The method according to any one of claims 1-4, wherein the travel information of the target vehicle includes:

a number of passengers within the target vehicle;

the first screen projection condition includes a number of passengers within the target vehicle being within a first range of passenger numbers.

6. The method according to any one of claims 1-5, wherein the travel information of the target vehicle includes:

a fatigue state of a driver of the target vehicle;

the first screen projecting condition comprises that the fatigue state of the driver of the target vehicle belongs to a first fatigue state, and the fatigue state of the driver of the target vehicle is obtained by a sensor module.

7. The method according to any one of claims 1-6, wherein the travel information of the target vehicle includes:

A distance between the target vehicle and an obstacle;

the first screen projecting condition includes a distance between the target vehicle and an obstacle being within a first distance range.

8. The method according to any one of claims 1-7, wherein the feature information of the application to be screened comprises:

the application type of the application program to be projected;

the first screen projection condition comprises that the application type of the application program to be projected is a first application type;

the when the first screen projecting condition is met comprises the following steps:

and when the application type of the first screen application to be projected is the first application type.

9. The method according to any one of claims 1-7, wherein the feature information of the application to be screened comprises:

the using frequency information of the application program to be projected;

the first screen projection condition comprises that the use frequency information of the application program to be projected is in a first use frequency range;

and when the use frequency information of the first screen application to be projected on the terminal equipment is in the first use frequency range.

10. The method according to any one of claims 1-9, further comprising:

And when the first screen projection condition and the second screen projection condition are met simultaneously, the screen projection information of the first screen application program to be projected is projected to the first display module and the second display module simultaneously, the second screen projection condition is a screen projection condition corresponding to the second display module, and the first display module and the second display module are two different display modules.

11. The method according to any one of claims 1-9, further comprising:

when the first screen projection condition and the second screen projection condition are met simultaneously, dividing screen projection information of the first screen application program to be projected into first screen projection sub information and second screen projection sub information, wherein the second screen projection condition is a screen projection condition corresponding to a second display module, and the first display module and the second display module are two different display modules;

and respectively projecting the first screen projection sub-information and the second screen projection sub-information to the first display module and the second display module.

12. The method according to any one of claims 1-9, further comprising:

when the first screen projection condition and the second screen projection condition are met simultaneously, screen projection information of the first screen application program to be projected is projected to only the first display module, the second screen projection condition is a screen projection condition corresponding to the second display module, the first display module and the second display module are two different display modules, and the first display module is a display module with higher screen projection priority than the second display module.

13. A display information switching method, which is applied to a vehicle, includes:

acquiring running information of a target vehicle;

when the running information of the target vehicle does not accord with a first running condition, switching screen projection information of a first application program displayed on a first display module to a second display module to be displayed, wherein the first display module and the second display module are display modules on the target vehicle, and the first running condition is a condition which needs to be met when the first display module is projected.

14. The method according to claim 13, wherein when the driving information of the target vehicle does not conform to the first driving condition, switching the screen projection information of the first application program displayed on the first display module to the second display module for displaying, specifically:

when the running information of the target vehicle is not in accordance with the first running condition and the running information of the target vehicle is in accordance with a second running condition, switching screen projection information of the first application program displayed on the first display module to the second display module to be displayed, wherein the second running condition is a condition which needs to be met when the second display module is projected.

15. The method according to claim 14, wherein the travel information of the target vehicle includes:

a driving speed of the target vehicle;

the first travel condition includes a driving speed of the vehicle being within a first driving speed range, and the second travel condition includes the driving speed of the vehicle being within a second driving speed range;

the running information of the target vehicle does not accord with the first running condition, and the running information of the target vehicle accords with the second running condition, and the running information of the target vehicle comprises:

the driving speed of the target vehicle is not within the first driving speed range, and the driving speed of the target vehicle is within the second driving speed range.

16. The method according to any one of claims 14-15, wherein the travel information of the target vehicle includes:

the type of traffic scene in which the target vehicle is located;

the first driving condition comprises that the traffic scene type belongs to a first traffic scene type, and the second driving condition comprises that the traffic scene type belongs to a second traffic scene type;

The traffic scene type in which the target vehicle is located does not belong to the first traffic scene type, and the traffic scene type in which the target vehicle is located belongs to the second traffic scene type.

17. The method according to any one of claims 14-16, wherein the travel information of the target vehicle includes:

a number of passengers within the target vehicle;

the first driving condition includes that the number of passengers is within a first passenger number range, and the second driving condition includes that the number of passengers is within a second passenger number range;

the number of passengers in the target vehicle is not within the first range of passenger numbers and the number of passengers in the target vehicle is within the second range of passenger numbers.

18. The method according to any one of claims 14-17, wherein the travel information of the target vehicle includes:

a fatigue state of a driver of the target vehicle, the fatigue state of the driver of the target vehicle being obtained by a sensor module;

The first driving condition includes that a fatigue state of the driver belongs to a first fatigue state, and the second driving condition includes that the fatigue state of the driver belongs to a second fatigue state;

the fatigue state of the driver of the target vehicle does not belong to the first fatigue state, and the fatigue state of the driver of the target vehicle belongs to the second fatigue state.

19. The method according to any one of claims 14-18, wherein the travel information of the target vehicle includes:

a distance between the target vehicle and an obstacle;

the first driving condition includes a distance between the vehicle and the obstacle being within a first distance range, and the second driving condition includes a distance between the vehicle and the obstacle being within a second distance range;

the distance between the target vehicle and an obstacle is not within the first distance range, and the distance between the target vehicle and an obstacle is within the second distance range.

20. An apparatus, comprising:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a first screen projection condition of a first display module projected on a target vehicle, and the first screen projection condition is related to running information of the target vehicle or is related to characteristic information of an application program to be projected;

and the screen projection module is used for projecting screen projection information of a first screen application program to be projected on the terminal equipment to the first display module when the first screen projection condition is met, the first display module is one of a plurality of display modules on the target vehicle, and different display modules on the target vehicle have different screen projection conditions.

21. The apparatus of claim 20, wherein the obtaining module is specifically configured to:

22. The apparatus of any one of claims 20-21, wherein the screen projecting module is further configured to:

23. The apparatus of any one of claims 20-21, wherein the screen projecting module is further configured to:

24. The apparatus of any one of claims 20-21, wherein the screen projecting module is further configured to:

25. An apparatus, comprising:

the acquisition module is used for acquiring the running information of the target vehicle;

the screen switching module is used for switching screen projection information of a first application program displayed on a first display module to a second display module to be displayed when the driving information of the target vehicle does not accord with a first driving condition, the first display module and the second display module are display modules on the target vehicle, and the first driving condition is a condition which needs to be met when the first display module is projected.

26. The device of claim 25, wherein the screen-switching module is specifically configured to:

when the running information of the target vehicle is not in accordance with the first running condition and the running information of the target vehicle is in accordance with the second running condition, switching screen projection information of the first application program displayed on the first display module to the second display module to be displayed, wherein the second running condition is a condition which needs to be met when the second display module is projected.

27. The apparatus according to claim 26, wherein the travel information of the target vehicle includes:

A driving speed of the target vehicle;

28. The apparatus according to any one of claims 26 to 27, wherein the travel information of the target vehicle includes:

the type of traffic scene in which the target vehicle is located;

29. An apparatus comprising a processor and a memory, the processor coupled with the memory,

the memory is used for storing programs;

the processor to execute a program in the memory to cause the apparatus to perform the method of any of claims 1-19.

30. A computer-readable storage medium comprising a program which, when run on a computer, causes the computer to perform the method of any one of claims 1-19.

31. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1-19.