WO2024067052A1

WO2024067052A1 - Screen mirroring display method, electronic device, and system

Info

Publication number: WO2024067052A1
Application number: PCT/CN2023/118085
Authority: WO
Inventors: 饶刚; 庄効谚; 邢海峰; 李自由; 卢恒惠
Original assignee: 华为技术有限公司
Priority date: 2022-09-30
Filing date: 2023-09-11
Publication date: 2024-04-04
Also published as: CN117850715A

Abstract

The present application relates to the technical field of terminals, and provides a screen mirroring display method, an electronic system, and a system. In the present application, an electronic device adaptively adjusts screen mirroring content of an application according to vehicle-mounted terminal information, so that the application has better universality, and the application development difficulty is reduced while ensuring the screen mirroring effect. The method comprises: an electronic device acquires first information sent by a vehicle-mounted terminal, wherein the first information comprises vehicle-mounted terminal information; then, after receiving a screen mirroring request, the electronic device determines screen mirroring data according to first interface information of a first application indicated by the screen mirroring request and the vehicle-mounted terminal information; the electronic device sends the screen mirroring data to the vehicle-mounted terminal, so that the vehicle-mounted terminal displays a screen mirroring interface according to the screen mirroring data.

Description

Screen projection display method, electronic device and system

This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on September 30, 2022, with application number 202211214674.2 and invention name “Screen projection display method, electronic device and system”, all contents of which are incorporated by reference in this application.

Technical Field

The embodiments of the present application relate to the field of terminal technology, and in particular to a screen projection display method, electronic device and system.

Background technique

With the development of terminal technology, in-vehicle terminals support screen projection technology. Electronic devices (such as mobile phones, tablets, etc.) can respond to user operations and project applications in electronic devices to the in-vehicle terminal for display. In this way, users can continue to use application functions on the in-vehicle terminal.

For example, a user enters a car while listening to music through a mobile phone music application. In response to the user's operation, the mobile phone establishes a connection with the vehicle terminal, and the user can continue to listen to the unfinished music previously played through the mobile phone through the vehicle terminal.

In the above-mentioned process of screen projection on the vehicle terminal, the application in the electronic device must first support screen projection to the vehicle terminal for display before screen projection can be realized. However, since there are many manufacturers of vehicle terminals and different manufacturers produce different models of vehicle terminals, the developer of the same application also needs to develop multiple types of applications that are suitable for different models of vehicle terminals, and the application development is difficult. As a result, the number of screen projection applications is limited, and the vehicle terminal application ecosystem is not perfect, which cannot meet the user's demand for the use of vehicle terminals.

Summary of the invention

In order to solve the above technical problems, the embodiments of the present application provide a screen projection display method, electronic device and system. The technical solution provided by the embodiments of the present application is that the electronic device adaptively adjusts the application screen projection content according to the vehicle terminal information, so that the application has better universality, while ensuring the screen projection effect, reducing the difficulty of application development.

In order to achieve the above technical objectives, the embodiments of the present application provide the following technical solutions:

In a first aspect, a screen projection display method is provided. The method includes: an electronic device obtains first information sent by a vehicle-mounted terminal, and the first information includes vehicle-mounted terminal information. The electronic device receives a screen projection request. In response to the screen projection request, the electronic device determines the first interface information of the first application indicated by the screen projection request. The electronic device determines the screen projection data based on the vehicle-mounted terminal information and the first interface information. The screen projection data is sent to the vehicle-mounted terminal, and the screen projection data is used to enable the vehicle-mounted terminal to display the screen projection interface.

In some examples, the first application is installed in the electronic device, and the first application is installed or not installed in the vehicle terminal. During the screen projection process, the electronic device can display the first interface of the first application and determine the first interface information of the first interface. Alternatively, during the screen projection process, the electronic device does not display the first application, and the electronic device determines the first interface information and the projection data in the background of the first application.

In this way, the electronic device can adaptively adjust the display of the application interface to be projected based on the vehicle terminal information and the identified display information of the application interface to be projected, so as to improve the application projection display effect and enhance the user experience.

According to the first aspect, the in-vehicle terminal information includes application layout information.

According to the first aspect, or any implementation of the first aspect above, the application layout information includes desktop layout information of the vehicle-mounted terminal.

In some examples, application layout information is used to represent the display of the vehicle terminal display screen.

For example, the vehicle terminal displays a desktop, and the display content on the desktop includes the function icons of the vehicle terminal (such as seat adjustment map icons, air conditioning control icons, etc.), and may also include icons of vehicle terminal applications. Then, the application layout information may include desktop layout information, such as information on multiple icons displayed, icon location information, etc.

For another example, the vehicle terminal displays an application interface, and the application layout information includes the layout information of the displayed application interface. The application layout information is used by subsequent electronic devices to re-layout the interface display content of the application to be projected and the display content of the vehicle terminal.

In this way, during the screen projection process, the electronic device can adaptively adjust the screen projection content of the application according to the content that the vehicle terminal needs to display fixedly (such as the content corresponding to the application layout information). This ensures that during the screen projection process, the vehicle terminal can still display the fixed content to meet the user's usage needs.

According to the first aspect, or any implementation method of the first aspect above, the vehicle-mounted terminal information also includes display screen size information, and the method also includes: in response to a screen projection request, determining whether the display status of the electronic device and the vehicle-mounted terminal is the same or different based on the display screen size information, and the display status is used to indicate horizontal screen display or vertical screen display.

In some examples, the display screen size information is used to indicate the size of the display area corresponding to the display screen of the vehicle terminal, such as length, width and other information, where the display area is used for display on the vehicle terminal.

In some examples, the electronic device first determines the display status of the electronic device and the vehicle terminal to determine the method of obtaining the projection data. For example, if the electronic device display screen and the vehicle terminal display screen are both horizontal or vertical, the electronic device can directly scale the application display interface to meet the display requirements of the vehicle terminal display screen. If the electronic device display screen is vertical and the vehicle terminal display screen is horizontal, or the electronic device display screen is horizontal and the vehicle terminal display screen is vertical. Then, the electronic device needs to rearrange the display content on the application interface to achieve a better display effect on the vehicle terminal display screen.

According to the first aspect, or any implementation of the first aspect above, determining whether the display states of the electronic device and the vehicle-mounted terminal are the same or different based on the display screen size information includes: determining a first aspect ratio of the display screen of the electronic device, and determining a second aspect ratio of the display screen of the vehicle-mounted terminal based on the display screen size information. When the absolute value of the difference between the first aspect ratio and the second aspect ratio is less than or equal to a first threshold, the display states of the electronic device and the vehicle-mounted terminal are the same. When the absolute value of the difference between the first aspect ratio and the second aspect ratio is greater than the first threshold, the display states of the electronic device and the vehicle-mounted terminal are different.

In some examples, the display screen of the vehicle terminal can be rotated to switch between horizontal and vertical display. Then, in the process of determining the projection data, the electronic device needs to determine whether the current display state of the vehicle terminal is horizontal or vertical display. Afterwards, the electronic device can determine whether the display state of the electronic device is the same as the current display state of the vehicle terminal. Optionally, the horizontal dimension and the vertical dimension indicated in the display screen size information sent by the vehicle terminal correspond to the horizontal dimension and the vertical dimension of the vehicle terminal in the current display state. For example, after the vehicle terminal rotates the display screen, the vehicle terminal switches the original horizontal dimension and the original vertical dimension, determines the new horizontal dimension (ie, the original vertical dimension) and the vertical dimension (ie, the original horizontal dimension) to generate new display screen size information and send it to the electronic device.

In some examples, the electronic device can also switch the display state by rotating the display screen. Then, when the electronic device starts the application to determine the projection data, it needs to determine whether the current display state of the electronic device is horizontal or vertical display. Similarly, the electronic device determines the horizontal and vertical dimensions of the display area during the current display process to determine the aspect ratio. Afterwards, the electronic device can determine whether the display state of the electronic device is the same as the current display state of the vehicle-mounted terminal. Alternatively, the electronic device determines whether the display state of the corresponding electronic device is horizontal or vertical display based on the application background data. Afterwards, the electronic device can determine whether the display state of the electronic device is the same as the current display state of the vehicle-mounted terminal.

According to the first aspect, or any implementation of the first aspect above, the display states of the electronic device and the vehicle-mounted terminal are different, and in response to a screen projection request, determining the first interface information of the first application indicated by the screen projection request includes: determining the first interface of the first application to be projected in response to the screen projection request. Dividing the functional area of the first interface and determining the first interface information.

In some examples, when determining the projection data, the electronic device determines that the display states of the electronic device and the vehicle-mounted terminal are different, and the application interface display content needs to be re-layouted. Therefore, the electronic device can also perform structural division on the layout of the projection interface.

For example, the electronic device divides the functional areas on the screen to be projected, and divides each functional area into a waterfall structure. Optionally, the functional area identification process includes, for example, identifying a tabulator (TAB) page, a search bar, a banner page, a grid bar, a title bar, a player, etc. included in the screen to be projected.

According to the first aspect, or any one of the implementations of the first aspect above, the projection data is determined according to the vehicle terminal information and the first interface information, including: according to the first interface information, the display screen size information, and the application layout information, the first display content corresponding to the functional area after the division and the second display content corresponding to the application layout information are re-layouted to generate the second interface information. The projection data is generated according to the second interface information.

According to the first aspect, or any implementation of the first aspect above, the first display content corresponding to the functional area and the second display content corresponding to the application layout information are re-layouted according to the first interface information, the display screen size information, and the application layout information to generate the second interface information, including: according to the functional area, the display screen size information, and the application layout information, the display area corresponding to the display screen of the vehicle terminal is divided into a first display area and a second display area, the first display area is used to display the first display content, and the second display area is used to display the second display content. According to the first display area, the first display content is re-layouted, and according to the second display area, the second display content is re-layouted. According to the re-layouted first display content and second display content, the second interface information is determined.

In some examples, the electronic device can divide the adaptation interface placement area (i.e., the first display area) on the screen to be projected and the in-vehicle application function placement area (i.e., the first display area) based on the in-vehicle terminal information (e.g., display screen size information, application layout information) and the application interface information of the electronic device. The adapter interface placement area is used to display the application interface of the projection screen (i.e. the first display content), and the in-vehicle application function placement area is used to display the content that the in-vehicle terminal needs to display first, such as desktop icons (i.e. the second display content).

Optionally, the screen to be projected may only include the adaptation interface placement area, and the vehicle-mounted terminal may subsequently display the projected application interface in full screen.

Optionally, a division template of the screen to be projected interface of the vehicle terminal is pre-configured, and the second display content to be displayed on the vehicle terminal and the first display content of the screen to be projected are mixed and arranged and optimized to determine the adaptation interface placement area and the vehicle application function placement area.

According to the first aspect, or any implementation of the first aspect above, the display status of the electronic device and the vehicle-mounted terminal is the same, and the projection data is determined according to the vehicle-mounted terminal information and the first interface information, including: determining the first display area for projecting and displaying the first application according to the display screen size information and the application layout information, the first display area being the display area other than the second display content corresponding to the application layout information. According to the first display area, the first interface of the first application corresponding to the first interface information is enlarged or reduced to generate the third interface information. The projection data is determined according to the third interface information and the application layout information.

In this way, the electronic device first determines the display status of the electronic device and the vehicle-mounted terminal. When it is determined that the display status is the same, the projection data can be directly determined by scaling the application interface to be projected, thereby effectively improving the projection efficiency under the same display status.

According to the first aspect, or any implementation method of the first aspect above, the vehicle-mounted terminal information also includes application information of the vehicle-mounted terminal, and the second display content includes function icons corresponding to the application layout information and/or commonly used application icons of the vehicle-mounted terminal, the function icons are function icons displayed by the vehicle-mounted terminal before screen projection and/or function icons used to control electronic devices, and the commonly used application icons are icons of commonly used applications of the vehicle-mounted terminal determined by the electronic device based on the application information.

In some examples, application information is used to represent information about applications installed in the vehicle-mounted terminal, such as application usage information, and the electronic device can subsequently determine the commonly used applications of the vehicle-mounted terminal based on the application usage information.

For example, the screen projection data determined by the electronic device may include icons of commonly used applications of the vehicle terminal. In this way, the vehicle terminal can display icons of commonly used applications during the subsequent screen projection process, which is convenient for users to use.

In some embodiments, the electronic device may also determine the electronic device function icon to be displayed on the vehicle terminal during the process of generating the projection data. Afterwards, the projection data sent by the electronic device to the vehicle terminal includes the electronic device function icon. Subsequently, the vehicle terminal includes the electronic device function icon in the projection interface displayed according to the projection data.

In this way, the user can control the electronic device through the electronic device function icon on the vehicle terminal, thereby improving the user experience.

According to the first aspect, or any implementation of the first aspect above, the vehicle terminal information also includes a display screen control method and hardware information of the vehicle terminal, the display screen control method is used to indicate whether the vehicle terminal is a touch screen or a non-touch screen, and before sending the projection data to the vehicle terminal, the method also includes: determining the operation response mode corresponding to the first control in the projection data. A binding relationship is generated based on the first control, the operation response mode, the hardware information, and the display screen control method, and the binding relationship is used to indicate the first response of the vehicle terminal after detecting the user's operation on the first control displayed in the projection interface.

In some examples, the control method of the display screen is used to indicate whether the display screen is a touch screen or non-touch screen, key or knob controlled information, etc. Hardware information includes, for example, information of hardware devices such as microphones, speakers, loudspeakers, cameras, etc. installed in the vehicle terminal.

According to the first aspect, or any implementation of the first aspect above, the method further includes: sending the binding relationship to the vehicle-mounted terminal.

In this way, the display content of the projected application interface is bound to the corresponding operation response, while ensuring the display effect, the interactive function of the application interface is retained to ensure the user experience.

According to the first aspect, or any implementation of the first aspect above, the screen projection request is a screen projection request determined after the electronic device receives the user's screen projection operation, or the screen projection request is a screen projection request sent by the vehicle-mounted terminal.

For example, when the electronic device is displaying the first interface of the first application, if it detects that the user clicks the screen projection control, it can be determined that the user instructs to project the currently displayed first interface to the vehicle terminal for display. Before the application is projected, the vehicle terminal displays the desktop or other applications, which are vehicle terminal applications or projection applications.

For another example, after establishing a communication connection with an electronic device, the vehicle terminal can obtain information about applications installed in the electronic device, and display corresponding application icons based on the information. When the vehicle terminal detects the user's operation on an application icon, it can send a screen projection request to the electronic device to request the electronic device to project the first application corresponding to the application icon, thereby determining that the screen needs to be projected to the vehicle terminal for display. The first application is shown.

Optionally, the first application is, for example, a video application, a music application, a navigation application, or the like.

In the second aspect, an embodiment of the present application provides a screen projection display method, which is applied to a vehicle-mounted terminal. The method includes: sending first information to an electronic device, the first information including vehicle-mounted terminal information. Receive projection data sent by the electronic device, the projection data is the electronic device responding to the received projection request, and the first interface information of the first application indicated by the projection request and the vehicle-mounted terminal information are determined. According to the projection data, display the projection interface.

According to the second aspect, the in-vehicle terminal information includes application layout information.

According to the second aspect, or any implementation of the second aspect, the application layout information includes desktop layout information of the vehicle-mounted terminal.

According to the second aspect, or any implementation of the second aspect above, the vehicle terminal information also includes display screen size information, and the display screen size information is used for the electronic device to determine the display status of the vehicle terminal, and the display status is used to indicate horizontal screen display or vertical screen display.

According to the second aspect, or any implementation of the second aspect above, the method also includes: receiving a binding relationship sent by an electronic device, the binding relationship is used to indicate a first response of the vehicle-mounted terminal after detecting a user operation on a first control displayed on the projection interface; the vehicle-mounted terminal information also includes a display screen control method and hardware information of the vehicle-mounted terminal, and the display screen control method and hardware information are used to enable the electronic device to generate a binding relationship.

According to the second aspect, or any implementation of the second aspect above, before receiving the projection data sent by the electronic device, the method also includes: sending a projection request to the electronic device in response to the user's projection operation.

The technical effects corresponding to the second aspect and any implementation method of the second aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any implementation method of the first aspect, and will not be repeated here.

In the third aspect, an embodiment of the present application provides an electronic device. The electronic device includes: a processor and a memory, the memory is coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the electronic device executes: obtain the first information sent by the vehicle-mounted terminal, the first information includes the vehicle-mounted terminal information. Receive a screen projection request. In response to the screen projection request, determine the first interface information of the first application that the screen projection request indicates is projected. Determine the screen projection data based on the vehicle-mounted terminal information and the first interface information. Send the screen projection data to the vehicle-mounted terminal, and the screen projection data is used to enable the vehicle-mounted terminal to display the screen projection interface.

According to the third aspect, the in-vehicle terminal information includes application layout information.

According to the third aspect, or any implementation of the third aspect, the application layout information includes desktop layout information of the vehicle-mounted terminal.

According to the third aspect, or any implementation method of the third aspect above, the vehicle-mounted terminal information also includes display screen size information. When the processor reads computer-readable instructions from the memory, it also enables the electronic device to perform the following operations: in response to the screen projection request, determine whether the display status of the electronic device and the vehicle-mounted terminal is the same or different based on the display screen size information, and the display status is used to indicate horizontal screen display or vertical screen display.

According to the third aspect, or any implementation of the third aspect above, determining whether the display states of the electronic device and the vehicle-mounted terminal are the same or different based on the display screen size information includes: determining a first aspect ratio of the display screen of the electronic device, and determining a second aspect ratio of the display screen of the vehicle-mounted terminal based on the display screen size information. When the absolute value of the difference between the first aspect ratio and the second aspect ratio is less than or equal to a first threshold, the display states of the electronic device and the vehicle-mounted terminal are the same. When the absolute value of the difference between the first aspect ratio and the second aspect ratio is greater than the first threshold, the display states of the electronic device and the vehicle-mounted terminal are different.

According to the third aspect, or any implementation of the third aspect above, the display states of the electronic device and the vehicle-mounted terminal are different, and in response to a screen projection request, determining the first interface information of the first application indicated by the screen projection request includes: determining the first interface of the first application to be projected in response to the screen projection request. Dividing the functional area of the first interface and determining the first interface information.

According to the third aspect, or any one of the implementations of the third aspect, the projection data is determined according to the vehicle terminal information and the first interface information, including: according to the first interface information, the display screen size information, and the application layout information, the first display content corresponding to the functional area after the division and the second display content corresponding to the application layout information are re-layouted to generate the second interface information. The projection data is generated according to the second interface information.

According to the third aspect, or any implementation of the third aspect, according to the first interface information, the display screen size information, and the application layout information, the first display content corresponding to the functional area and the second display content corresponding to the application layout information are re-layouted. The second interface information is generated by: dividing the display area corresponding to the display screen of the vehicle terminal into a first display area and a second display area according to the functional area, the display screen size information, and the application layout information, wherein the first display area is used to display the first display content and the second display area is used to display the second display content. The first display content is re-layouted according to the first display area, and the second display content is re-layouted according to the second display area. The second interface information is determined according to the re-layouted first display content and the second display content.

According to the third aspect, or any implementation of the third aspect above, the display status of the electronic device and the vehicle-mounted terminal is the same, and the projection data is determined according to the vehicle-mounted terminal information and the first interface information, including: determining the first display area for projecting and displaying the first application according to the display screen size information and the application layout information, the first display area being the display area other than the second display content corresponding to the application layout information. According to the first display area, the first interface of the first application corresponding to the first interface information is enlarged or reduced to generate the third interface information. The projection data is determined according to the third interface information and the application layout information.

According to the third aspect, or any implementation method of the third aspect above, the vehicle-mounted terminal information also includes application information of the vehicle-mounted terminal, and the second display content includes function icons corresponding to the application layout information and/or commonly used application icons of the vehicle-mounted terminal, the function icons are function icons displayed by the vehicle-mounted terminal before screen projection and/or function icons used to control electronic devices, and the commonly used application icons are icons of commonly used applications of the vehicle-mounted terminal determined by the electronic device based on the application information.

According to the third aspect, or any implementation of the third aspect above, the vehicle terminal information also includes a display screen control method and hardware information of the vehicle terminal. The display screen control method is used to indicate whether the vehicle terminal is a touch screen or a non-touch screen. When the processor reads computer-readable instructions from the memory, it also causes the electronic device to perform the following operations: determine the operation response mode corresponding to the first control in the projection data. A binding relationship is generated based on the first control, the operation response mode, the hardware information, and the display screen control method. The binding relationship is used to indicate the first response of the vehicle terminal after detecting the user's operation on the first control displayed in the projection interface.

According to the third aspect, or any implementation of the third aspect above, when the processor reads the computer-readable instructions from the memory, the electronic device is also caused to perform the following operation: sending the binding relationship to the vehicle-mounted terminal.

According to the third aspect, or any implementation of the third aspect above, the screen projection request is a screen projection request determined after the electronic device receives the user's screen projection operation, or the screen projection request is a screen projection request sent by the vehicle-mounted terminal.

The technical effects corresponding to the third aspect and any one of the implementation methods of the third aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods of the first aspect, and will not be repeated here.

In a fourth aspect, an embodiment of the present application provides a vehicle-mounted terminal. The vehicle-mounted terminal includes: a processor, a memory, and a display screen, the memory and the display screen are coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the vehicle-mounted terminal executes: sending first information to the electronic device, the first information includes vehicle-mounted terminal information. Receive the projection data sent by the electronic device, the projection data is the electronic device responding to the received projection request, according to the first interface information of the first application indicated by the projection request and the vehicle-mounted terminal information. According to the projection data, display the projection interface.

According to a fourth aspect, the in-vehicle terminal information includes application layout information.

According to the fourth aspect, or any implementation of the fourth aspect, the application layout information includes desktop layout information of the vehicle-mounted terminal.

According to the fourth aspect, or any implementation of the fourth aspect above, the vehicle-mounted terminal information also includes display screen size information, and the display screen size information is used to enable the electronic device to determine the display status of the vehicle-mounted terminal, and the display status is used to indicate horizontal screen display or vertical screen display.

According to the fourth aspect, or any implementation method of the fourth aspect above, when the processor reads computer-readable instructions from the memory, the vehicle-mounted terminal also performs the following operations: receiving a binding relationship sent by an electronic device, the binding relationship is used to indicate the first response of the vehicle-mounted terminal after detecting the user's operation on the first control displayed on the projection interface; the vehicle-mounted terminal information also includes the display screen control method and hardware information of the vehicle-mounted terminal, and the display screen control method and hardware information are used to enable the electronic device to generate a binding relationship.

According to the fourth aspect, or any implementation of the fourth aspect above, when the processor reads computer-readable instructions from the memory, it also enables the vehicle-mounted terminal to perform the following operations: in response to the user's screen projection operation, send a screen projection request to the electronic device.

The technical effects corresponding to the fourth aspect and any one of the implementation methods of the fourth aspect can be found in the technical effects corresponding to the above-mentioned second aspect and any one of the implementation methods of the second aspect, and will not be repeated here.

In a fifth aspect, an embodiment of the present application provides a screen projection display system, which includes an electronic device and a vehicle-mounted terminal. The vehicle-mounted terminal is used to send a first information to the electronic device, and the first information includes vehicle-mounted terminal information. The electronic device is used to receive a screen projection request. The electronic device is also used to determine, in response to the screen projection request, the first interface information of the first application that the screen projection request indicates to be projected. The electronic device is also used to determine the projection data based on the first interface information and the vehicle-mounted terminal information in the received first information. The electronic device is also used to send the projection data to the vehicle-mounted terminal. The vehicle-mounted terminal is also used to display the projection interface based on the received projection data.

According to the fifth aspect, the in-vehicle terminal information includes application layout information.

According to the fifth aspect, or any implementation of the fifth aspect, the application layout information includes desktop layout information of the vehicle-mounted terminal.

According to the fifth aspect, or any implementation of the fifth aspect above, the vehicle terminal information also includes display screen size information. The electronic device is also used to respond to the screen projection request and determine, based on the display screen size information, whether the display states of the electronic device and the vehicle terminal are the same or different, and the display state is used to indicate horizontal screen display or vertical screen display.

According to the fifth aspect, or any implementation of the fifth aspect above, when the display states of the electronic device and the vehicle-mounted terminal are different, the electronic device is further used to determine the first interface of the first application to be projected in response to the screen projection request. The electronic device is also used to divide the functional area of the first interface and determine the first interface information.

According to the fifth aspect, or any implementation of the fifth aspect above, the electronic device is further used to re-layout the first display content corresponding to the divided functional area and the second display content corresponding to the application layout information according to the first interface information, the display screen size information, and the application layout information, and generate the second interface information. The electronic device is also used to generate the projection data according to the second interface information.

According to the fifth aspect, or any implementation of the fifth aspect above, when the display status of the electronic device and the vehicle-mounted terminal are the same, the electronic device is also used to determine the first display area for projecting and displaying the first application based on the display screen size information and the application layout information, and the first display area is the display area other than the second display content corresponding to the application layout information. The electronic device is also used to enlarge or reduce the first interface of the first application corresponding to the first interface information based on the first display area to generate third interface information. The electronic device is also used to determine the projection data based on the third interface information and the application layout information.

The technical effects corresponding to the fifth aspect and any one of the implementation methods of the fifth aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods of the first aspect, and will not be repeated here.

In a sixth aspect, an embodiment of the present application provides an electronic device having the function of implementing the screen projection display method described in the first aspect and any possible implementation thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

The technical effects corresponding to the sixth aspect and any one of the implementation methods of the sixth aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods of the first aspect, and will not be repeated here.

In a seventh aspect, an embodiment of the present application provides a vehicle-mounted terminal, which has the function of implementing the screen projection display method described in the second aspect and any possible implementation thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

The technical effects corresponding to the seventh aspect and any one of the implementation methods of the seventh aspect can be found in the technical effects corresponding to the above-mentioned second aspect and any one of the implementation methods of the second aspect, and will not be repeated here.

In an eighth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect.

The technical effects corresponding to the eighth aspect and any one of the implementation methods of the eighth aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods of the first aspect, and will not be repeated here.

In a ninth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by a vehicle-mounted terminal, the vehicle-mounted terminal executes the method of the second aspect or any one of the implementations of the second aspect.

The technical effects corresponding to the ninth aspect and any one of the implementation methods of the ninth aspect can be found in the technical effects corresponding to the above-mentioned second aspect and any one of the implementation methods of the second aspect, and will not be repeated here.

In a tenth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, Enable the electronic device to execute the method of the first aspect or any one of the embodiments of the first aspect.

The technical effects corresponding to the tenth aspect and any one of the implementation methods in the tenth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods in the first aspect, and will not be repeated here.

In the eleventh aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a vehicle-mounted terminal, the vehicle-mounted terminal executes the method of the second aspect or any one of the implementations of the second aspect.

The technical effects corresponding to the eleventh aspect and any one of the implementation methods in the eleventh aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any one of the implementation methods in the second aspect, and will not be repeated here.

In the twelfth aspect, an embodiment of the present application provides a circuit system, the circuit system includes a processing circuit, and the processing circuit is configured to execute the method of the first aspect or any one of the embodiments of the first aspect; or, the processing circuit is configured to execute the method of the second aspect or any one of the embodiments of the second aspect.

The technical effects corresponding to the twelfth aspect and any one of the implementation methods of the twelfth aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods of the first aspect, and will not be repeated here.

In the thirteenth aspect, an embodiment of the present application provides a chip system, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method of the first aspect or any one of the embodiments of the first aspect; or, the at least one processor executes the method of the second aspect or any one of the embodiments of the second aspect.

The technical effects corresponding to the thirteenth aspect and any one of the implementation methods of the thirteenth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any one of the implementation methods of the first aspect, and will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a communication system to which a screen projection display method is applied according to an embodiment of the present application;

FIG2 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

FIG3A is a schematic diagram of an in-car scene provided in an embodiment of the present application;

FIG3B is a schematic diagram of a display screen of a vehicle terminal provided in an embodiment of the present application;

FIG4 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

FIG5 is a schematic diagram of a screen projection scenario provided in an embodiment of the present application;

FIG6 is a second schematic diagram of a screen projection scenario provided in an embodiment of the present application;

FIG. 7 is a second schematic flow chart of the screen projection display method provided in an embodiment of the present application;

FIG8 is a third schematic diagram of a screen projection scenario provided in an embodiment of the present application;

FIG9 is a fourth schematic diagram of a screen projection scenario provided in an embodiment of the present application;

FIG10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of the structure of the vehicle-mounted terminal provided in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application are described below in conjunction with the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

In the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments should not be interpreted as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific way.

In some scenarios, electronic devices (such as mobile phones, tablets, etc.) can establish a screen projection connection with the vehicle-mounted terminal and project the application to the vehicle-mounted terminal display to enrich the functions of the vehicle-mounted terminal and meet the user's usage needs.

For example, when a user is playing music on a mobile phone, he or she enters the car. In response to the user's operation, the mobile phone establishes a screen projection connection with the in-vehicle terminal, and the in-vehicle terminal can continue to play the music that the mobile phone has not finished playing.

For example, during parking breaks, the vehicle-mounted terminal receives the projection data sent by the mobile phone and displays the projected video based on the projection data, so that users can get a better video playback experience on the vehicle-mounted terminal.

In some examples, a screen projection connection is established between the electronic device and the vehicle-mounted terminal. The vehicle-mounted terminal can receive the screen projection data sent by the electronic device through the screen projection connection, and the screen projection data is the data of the application in the electronic device. The vehicle-mounted terminal displays the corresponding application interface based on the screen projection data. In this way, the user can operate the electronic device and the vehicle-mounted terminal through the vehicle-mounted terminal to meet the user's usage needs. Among them, the screen projection process includes, for example, push screen projection and mirror screen projection. During the push screen projection process, the display content of the vehicle-mounted terminal and the electronic device may not be synchronized. During the mirror screen projection process, the display content of the vehicle-mounted terminal and the electronic device is synchronized.

In other examples, after the vehicle terminal establishes a connection with the electronic device, the application list information of the electronic device can be obtained. Then, the vehicle terminal can display the corresponding application icon according to the application list information. In response to the user's operation on the application icon, the vehicle terminal can request the electronic device to obtain the application data corresponding to the application icon. Then, the vehicle terminal displays the corresponding application interface according to the obtained application data.

In the above example screen projection scenario, application developers need to develop applications that are suitable for vehicle terminals. Otherwise, screen projection cannot be achieved. Or, the screen projection effect is poor, and only small window projection can be achieved, which cannot fully reflect the large-screen display effect of the vehicle terminal display.

Optionally, during the application development process, the developer can create a display template based on a preset display template (such as Functions, etc.), develop application types adapted to different vehicle terminals according to the needs of different vehicle terminals, such as display screen size, system requirements, etc. In this way, subsequent users can project application content to the vehicle terminal display through electronic devices.

However, there are many types of in-vehicle terminals at present. During the development process of the same application, it is necessary to develop multiple types of applications to adapt to different types of in-vehicle terminals, which makes application development difficult. As a result, the number of screen-casting applications is limited, and the in-vehicle terminal application ecosystem is not perfect, which cannot meet the user's demand for in-vehicle terminals.

Therefore, the embodiment of the present application provides a screen projection display method, during the screen projection process, the electronic device can adaptively adjust the application screen projection content according to the display information of the vehicle terminal, such as the size of the vehicle terminal display screen, the desktop display content, etc., to achieve screen projection. In this way, the electronic device application has better universality, while ensuring the screen projection effect, reducing the difficulty of application development.

Fig. 1 is a schematic diagram of a communication system for the screen projection display method provided in an embodiment of the present application. As shown in Fig. 1, the communication system includes an electronic device 100 and a vehicle terminal 200.

Optionally, the electronic device 100 may be, for example, a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a wearable device, an artificial intelligence (AI) device, or other terminal device with a screen projection function. The operating system installed in the electronic device 100 includes but is not limited to: Or other operating systems. This application does not limit the specific type of the electronic device 100 or the installed operating system.

Optionally, the vehicle-mounted terminal 200 may be a vehicle, automobile, vehicle computer, or other terminal. The operating system installed in the vehicle-mounted terminal 200 includes but is not limited to Or other operating systems. The vehicle terminal 200 may not be installed with an operating system. This application does not limit the specific type of the vehicle terminal 200, whether an operating system is installed, and the installed operating system.

In some embodiments, a communication connection is established between the electronic device 100 and the vehicle terminal 200, and the communication connection can be a wired connection or a wireless communication connection. Optionally, the electronic device and the vehicle terminal 200 can establish a wired connection through a universal serial bus (USB) interface. The wireless communication technology for establishing a wireless communication connection includes but is not limited to at least one of the following: Bluetooth (BT) (for example, traditional Bluetooth or low energy (BLE) Bluetooth), wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), ultra-wideband (UWB), etc.

In some examples, after the electronic device 100 and the vehicle-mounted terminal 200 establish a communication connection, screen projection data can be transmitted through the communication connection to achieve screen projection.

FIG. 2 is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

As shown in Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (SIM) interface, and/or a universal serial bus (USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor through the I2C interface, so that the processor 110 communicates with the touch sensor through the I2C bus interface to realize the touch function of the electronic device 100.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it can also charge the battery 142 through the power management module 130. Block 141 provides power to the electronic device.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

The mobile communication module 150 can provide solutions for wireless communications including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) and the like applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering processing, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of the signal, amplify the signal, and convert it into electromagnetic waves for radiation through the antenna 2.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and/or a satellite based augmentation system (SBAS).

The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display. The display 194 may be a liquid crystal display (LCD), for example, an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-oled, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The electronic device 100 can use the audio module 170, such as music playing, recording, etc. The audio module 170 can include a speaker, a receiver, a microphone, a headphone interface, and an application processor to implement audio functions.

The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

A touch sensor is also called a "touch control device". The touch sensor can be arranged on the display screen 194. The touch sensor and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor can also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.

The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 may be an indicator light, which may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

In some embodiments, the electronic device 100 can communicate with the vehicle terminal 200 through the wireless communication module 160 or the USB interface 130. Establish a communication connection. After that, the electronic device 100 can obtain the display screen size information of the vehicle terminal 200. The processor 110 of the electronic device 100 can adaptively adjust the display of the screen interface to be projected according to the display screen size information of the vehicle terminal 200, determine the projection data, and send the projection data to the vehicle terminal 200 through the wireless communication module 160. After receiving the projection data, the vehicle terminal 200 can also display the corresponding application interface on the display screen according to the projection data.

Optionally, the electronic device 100 and the vehicle terminal 200 in the embodiment of the present application can be implemented by different devices, and the different devices can have the same, similar or somewhat different hardware structures. For example, the vehicle terminal can also have the hardware structure shown in FIG.

It is understandable that the vehicle terminal 200 may include more or fewer components than the hardware structure shown in Figure 2, or combine some components, or split some components, or arrange the components differently. For example, the hardware structure of the vehicle terminal may not include the mobile communication module 150, etc.

Optionally, the vehicle terminal 200 may be configured with one or more display screens. As shown in FIG3A , the vehicle terminal 200 includes a central control screen 21, a display screen 22 in front of the driver, a display screen 23 in front of the co-pilot seat, or may also include a display screen in front of the rear passenger seat, etc.

In some examples, the different display screens of the vehicle-mounted terminal 200 may be touch screens or non-touch screens. The touch screen may receive a user's touch operation on the display screen. In addition, the touch screen and non-touch screen may also receive a user's operation on a button in the vehicle-mounted terminal.

For example, as shown in FIG3A , the steering wheel 23 of the vehicle terminal 200 is provided with a square control key 231, and the user can control the display of the display screen of the vehicle terminal 200 through the square control key 231. For example, to switch to play the next music, the display screen can display the corresponding interface.

For another example, as shown in FIG3B , the display screen 24 is a non-touch screen. The user can control the display on the display screen 24 through the knob 241 and the button 242 .

In some examples, the electronic device 100 can obtain information such as the size of the display screen of the vehicle terminal 200 to be projected, whether it is a touch screen, button or knob control, etc.

The following takes the electronic device 100 as an electronic device and the vehicle-mounted terminal 200 as a vehicle-mounted terminal as an example to introduce the screen projection display method provided in the embodiment of the present application in detail.

Exemplarily, Fig. 4 is a flow chart of a screen projection display method provided in an embodiment of the present application. As shown in Fig. 4, the method includes the following steps.

S401: The electronic device and the vehicle-mounted terminal establish a communication connection.

The communication connection may be a wired connection or a wireless connection, and the communication connection may be used for screen projection. The electronic device and the vehicle-mounted terminal are configured with a screen projection function, and the screen projection function is turned on to allow screen projection.

For example, electronic devices and vehicle-mounted terminals establish screen projection connections through Bluetooth, vehicle-mounted Wi-Fi networks, USB connections, etc.

S402: The vehicle-mounted terminal sends vehicle-mounted terminal information to the electronic device.

The vehicle-mounted terminal information includes, for example, one or more of the display screen size information, display screen control method, application layout information, application information, and hardware information of the vehicle-mounted terminal.

Among them, the display screen size information is used to indicate the size of the display area corresponding to the display screen of the vehicle terminal, such as length, width and other information, wherein the display area is used for the display of the vehicle terminal. The control method of the display screen is used to indicate whether the display screen is a touch screen or a non-touch screen, a button or a knob control information, etc. The application layout information is used to indicate the display of the vehicle terminal display screen. For example, the vehicle terminal displays the desktop, and the display content on the desktop includes the function icons of the vehicle terminal (such as seat adjustment map icons, air conditioning control icons, etc.), and may also include icons of vehicle terminal applications. Then, the application layout information may include desktop layout information, such as information including multiple icons displayed, icon location information, etc. For another example, the vehicle terminal displays the application interface, and the application layout information includes the layout information of the displayed application interface. The application layout information is used for subsequent electronic devices to re-layout the interface display content of the application to be projected and the display content of the vehicle terminal. The application information is used to indicate the information of the applications installed in the vehicle terminal, such as application usage information, etc., and the subsequent electronic devices can determine the commonly used applications of the vehicle terminal based on the application usage information. Hardware information, for example, includes information on hardware devices such as microphones (mic), speakers, loudspeakers, cameras, etc. installed in the vehicle terminal.

Exemplarily, as shown in FIG5 , the electronic device 100 and the vehicle terminal 200 establish a communication connection. As shown in the reference numeral 51 , some icons are displayed on the desktop of the vehicle terminal display screen, and the application layout information includes the display information of these icons. The icons may include icons that are fixedly displayed on the desktop, such as navigation icons, seat adjustment icons, air conditioning control icons and other functional icons. Alternatively, the icons may also include icons of commonly used applications of the vehicle terminal.

It should be understood that the icons that are fixedly displayed on the desktop are used to indicate that these icons need to be displayed on the desktop to facilitate users to use the car. The icon can be moved according to the user's operation or according to the projection data.

In some embodiments, when the vehicle terminal is equipped with multiple display screens, the vehicle terminal information may correspond to the display screen to be projected, or to all the display screens of the vehicle terminal. Subsequently, before projecting the screen, the electronic device may first determine the vehicle terminal information corresponding to the display screen to be projected, so as to generate corresponding projection data and realize the projection.

In some embodiments, after the vehicle-mounted terminal and the electronic device establish a communication connection, the determined vehicle-mounted terminal information may be sent to the electronic device. Correspondingly, the electronic device receives the vehicle-mounted terminal information sent by the vehicle-mounted terminal.

In some embodiments, step S402 is an optional step. After the electronic device establishes a communication connection with the vehicle terminal for the first time and obtains the vehicle terminal information, the vehicle terminal information can be saved. After that, after the electronic device establishes a communication connection with the vehicle terminal and determines that screen projection is required, the corresponding screen projection data can be directly determined based on the saved vehicle terminal information without having to obtain the vehicle terminal information again.

In some examples, the vehicle terminal information may change. After the vehicle terminal establishes a communication connection with the electronic device, the vehicle terminal may send the updated vehicle terminal information to the electronic device. For example, the vehicle terminal information includes the application layout information of the vehicle terminal. In response to user operations, the application layout information changes, such as adding an icon of a newly downloaded application. Then, the vehicle terminal may update the vehicle terminal information and send the updated vehicle terminal information to the electronic device.

S403: The electronic device receives a screen projection request.

The screen projection request may be a screen projection request determined by the electronic device after receiving the screen projection operation of the user. Alternatively, the screen projection request may also be a screen projection request sent by the vehicle-mounted terminal and received by the electronic device.

In some embodiments, the electronic device detects a screen projection operation, or receives a screen projection request sent by the vehicle terminal, and determines that the application needs to be projected to the vehicle terminal for display. Optionally, the electronic device has the application to be projected installed, and the vehicle terminal may or may not have the application to be projected installed. The application to be projected is the first application installed in the electronic device, and the interface of the first application to be projected is the first interface of the first application.

For example, when the electronic device is displaying the application interface of the first application, if it detects that the user clicks the screen projection control, it can determine that the user instructs to project the currently displayed application interface to the vehicle terminal for display. Before the application is projected, the vehicle terminal displays the desktop or other applications, which are vehicle terminal applications or projection applications.

For another example, after establishing a communication connection with an electronic device, the vehicle terminal can obtain information about applications installed in the electronic device, and display the corresponding application icon based on the information. When the vehicle terminal detects the user's operation on the application icon, it can send a screen projection request to the electronic device to request the electronic device to project the first application corresponding to the application icon, and then determine that the first application needs to be projected and displayed on the vehicle terminal.

In some embodiments, step S403 is an optional step. After the electronic device and the vehicle terminal establish a communication connection, the electronic device is directly triggered to project the screen to the vehicle terminal, that is, the following step S404 is triggered. If the electronic device determines that a communication connection is established with the vehicle terminal, the running application can be directly projected to the vehicle terminal for display.

In some embodiments, the present application does not limit the execution order of step S402 and step S403. For example, after determining that the user instructs to project the screen, the electronic device may request the vehicle terminal to obtain the vehicle terminal information, that is, first execute step S403 and then execute step S402.

S404: The electronic device determines the projection data according to the vehicle terminal information and the application interface information.

In some embodiments, after determining the application to be projected to the vehicle-mounted terminal, the electronic device can determine the corresponding projection data based on the vehicle-mounted terminal information and the application interface information of the application to be projected.

For example, the size, aspect ratio, and resolution of the electronic device and the vehicle terminal display screen are not uniform, so during the screen projection process, the electronic device needs to re-determine the projection data that matches the vehicle terminal display screen. For example, the electronic device can adjust the display size of the application interface to be projected based on the display size information of the vehicle terminal included in the vehicle terminal information to determine the corresponding projection data.

In some embodiments, the display screen of the vehicle terminal can be horizontal or vertical. As shown in Figure 5, the long side of the display screen of the vehicle terminal 200 is the bottom side, and the short side is the side side, and the display screen of the vehicle terminal 200 is displayed in horizontal mode. As shown in Figure 6, the long side of the display screen of the vehicle terminal 200 is the side side, and the short side is the bottom side, and the display screen of the vehicle terminal 200 is displayed in vertical mode. Similarly, the display screen of the electronic device can also be divided into horizontal and vertical screens. If the display screen of the electronic device and the display screen of the vehicle terminal are both horizontal or both vertical, then the electronic device can directly scale the application display interface to meet the display requirements of the display screen of the vehicle terminal. If the display screen of the electronic device is vertical and the display screen of the vehicle terminal is horizontal, or the display screen of the electronic device is horizontal and the display screen of the vehicle terminal is vertical. Then, the electronic device The display content on the application interface needs to be rearranged to achieve better display effects on the vehicle terminal display screen.

Therefore, when determining the projection data, the electronic device needs to determine whether it is necessary to rearrange the application interface display content.

Optionally, as shown in FIG. 7 , step S404 may include steps S701 to S705 .

S701: The electronic device determines that the display states of the electronic device and the vehicle-mounted terminal are different. If yes, execute step S702; if not, execute step S705.

In some embodiments, the electronic device can determine whether the electronic device and the vehicle-mounted terminal are in the same display state by the aspect ratio of the display screen (i.e., the ratio of the bottom edge to the side edge length), and the display state includes horizontal screen display and vertical screen display.

For example, the electronic device determines the aspect ratio of the electronic device display screen, and determines the aspect ratio of the vehicle terminal display screen based on the acquired vehicle terminal information. Afterwards, the electronic device determines the absolute value of the difference between the two aspect ratios. If the absolute value of the difference is less than or equal to a preset threshold, then the electronic device can determine that the electronic device and the vehicle terminal are in the same display state, such as both are displayed in horizontal screens, or both are displayed in vertical screens. If the absolute value of the difference is greater than the preset threshold, then the electronic device can determine that the electronic device and the vehicle terminal are not in the same display state.

For another example, the display screen size information in the vehicle-mounted terminal information includes information for indicating the display state of the vehicle-mounted terminal. Then, after acquiring the vehicle-mounted terminal information, the electronic device can directly determine the display state of the vehicle-mounted terminal. Afterwards, the electronic device can determine the display state of the electronic device according to the display screen size of the electronic device. Afterwards, the electronic device can determine whether the display state of the electronic device is the same as the display state of the vehicle-mounted terminal.

In some embodiments, the electronic device determines that the display states of the electronic device and the vehicle terminal are different, and can re-layout the application interface display content according to the vehicle terminal information, and generate corresponding projection data, that is, execute the following steps S702-S704. Alternatively, the electronic device determines that the display states of the electronic device and the vehicle terminal are the same, and can scale the application interface display content according to the vehicle terminal information, and generate corresponding projection data, that is, execute the following step S705.

S702: The electronic device obtains application interface information.

In some embodiments, the electronic device can obtain interface information of the application's screen projection interface, such as layout data and control data, through control tree recognition or AI recognition.

The control data includes the position, size, color and other data of the control. The electronic device draws the control according to the control data to display the control visible to the user. The layout data is used to represent the layout format and layout strategy. The layout data includes the control data of multiple controls and the layout information of the control data.

In some examples, the electronic device can obtain a description file (such as an XML file, etc.) of the current screen to be projected, or a screenshot of the screen. Afterwards, the electronic device can identify controls such as text, images, and icons on the screen to be projected based on the description file or the screenshot, through control tree recognition or AI recognition, etc., and the control can be used as a human-computer interaction interface to receive user operations on the application.

In addition, the electronic device can also structurally divide the layout of the screen to be projected. For example, the electronic device divides the functional areas on the screen to be projected, and divides each functional area into a waterfall structure. Optionally, the identification process of the functional area includes, for example, identifying the tabulator (TAB) page, search bar, banner page, grid bar, title bar, player, etc. included in the screen to be projected.

For example, as shown in FIG5 , the electronic device 100 (such as a mobile phone) is in a vertical display mode, and the vehicle-mounted terminal 200 is in a horizontal display mode. The electronic device 100 determines the application interface information of the application to be projected. Among them, the functional area includes a top tab page 52, a search bar 53, a banner page 54, a grid bar 55, a grid bar 56, and a bottom function tab page 57. Different functional areas may include one or more controls.

It should be understood that the interface shown in Figure 5 is used to illustrate the application interface information of the application to be projected. During the projection process, the electronic device does not need to display the application interface, and the electronic device can determine the projection data in the application background. Optionally, during the projection process, the electronic device can display or not display the application.

In some embodiments, there is some unnecessary information on the screen to be projected, such as advertising information, unnecessary tags, etc., which may interfere with the recognition of the application interface information. Therefore, when the electronic device obtains the application interface information, it can first remove the unnecessary information.

For example, if an application interface displays advertising information, the advertising information can be displayed floating on the application interface or embedded in the application interface. When the electronic device identifies the application interface information through control tree recognition or AI recognition, the display position of the advertising information may cause abnormal recognition of control data and layout data, resulting in abnormal final projection display.

Optionally, the electronic device removes non-essential information from the control tree and resource files during the process of obtaining the application interface information. If the application interface information finally obtained by the electronic device does not include non-essential information, then the corresponding subsequent application interface displayed on the vehicle terminal will not include non-essential information.

S703. The electronic device rearranges the application interface according to the vehicle terminal information and the application interface information.

In some embodiments, after the electronic device obtains the application interface information, it can rearrange the application interface according to the vehicle terminal information.

In some examples, the electronic device can divide the adaptation interface placement area and the vehicle-mounted application function placement area on the screen to be projected according to the vehicle-mounted terminal information (such as display screen size information) and the application interface information of the electronic device. Among them, the adaptation interface placement area is used to display the application interface of the projection screen, and the vehicle-mounted application function placement area is used to display the icons that the vehicle-mounted terminal needs to display on the desktop first. Optionally, the screen to be projected may also only include the adaptation interface placement area, and the subsequent vehicle-mounted terminal can display the projected application interface in full screen. Optionally, the vehicle-mounted application function placement area can be displayed in a floating manner on the adaptation interface placement area, such as the icon of the vehicle-mounted terminal can be displayed in a floating manner on the projection application interface. For example, the electronic device matches the display area required to be occupied by the application interface corresponding to the display application interface information on the vehicle terminal display screen according to the application interface information, divides the display area into the video interface placement area, and then divides the remaining display area into the vehicle-mounted application function placement area.

For example, as shown in Figure 5, the electronic device 100 determines that the icons that need to be displayed on the desktop of the vehicle terminal include the fixed icons in the dotted box shown by the reference numeral 51, based on the application layout information in the vehicle terminal information of the vehicle terminal 200. The electronic device can determine whether it is necessary to adjust the display area of these fixed icons based on the acquired application interface information and display screen size information. If no adjustment is required, as shown in Figure 5, the electronic device determines that the interface to be projected includes an adaptation interface placement area 58 and a vehicle application function placement area 51. If adjustment is required, the electronic device can adjust the display position of the fixed icons of the vehicle terminal to achieve a better application projection effect.

For another example, the electronic device determines the commonly used applications of the vehicle terminal based on the application information in the vehicle terminal information, and can display the icons of the commonly used applications on the desktop first. In addition, the vehicle terminal can determine the fixed icons to be displayed based on the application layout information in the vehicle terminal information. Afterwards, the electronic device can divide the display area according to the commonly used application icons, fixed icons, display screen size information, and application interface information, determine the adaptation interface placement area and the vehicle application function placement area, and re-layout the application interface.

Optionally, a partition template of the screen to be projected of the vehicle terminal is pre-configured, and the fixed icons to be displayed on the vehicle terminal and the content of the screen to be projected are mixed and arranged to optimize, so as to determine the placement area of the adaptation interface and the placement area of the vehicle application function. Optionally, different numbers of fixed icons can correspond to different templates, so that the fixed icons can be displayed in the edge display area of the vehicle terminal, such as distributed display on both sides of the display screen, or distributed display on one side of the display, to ensure the application projection effect.

In some embodiments, when the electronic device is determining the projection data, the vehicle terminal may display the desktop or application. Then, if the vehicle terminal displays the application, the application layout information in the vehicle terminal information includes the display information of the application. Then, when the electronic device is re-layouting the application interface according to the vehicle terminal information and the application interface information, the divided adaptation interface placement area is used to display the projection application interface, and the vehicle application function placement area is used to display the display content of the application displayed by the vehicle terminal before this projection. That is, the electronic device re-layouts the display content of the two applications, the projection application and the original display application of the vehicle terminal, to achieve the subsequent determination of the projection data.

In some embodiments, different functional areas of the application interface are configured with different adaptation templates. The electronic device rearranges the controls and layout according to the determined adaptation interface placement area, the information of the functional area in the application interface information, and the adaptation template. Re-layout the application interface and generate corresponding re-layout interface information. The re-layout interface information includes new control data and layout data. Optionally, the new interface data and layout data can be saved and transmitted in a corresponding file format, such as a re-layout interface partition file and an adaptive adaptation layout description file. Among them, the re-layout interface partition file includes, for example, adjusted functional area information, such as location coordinate information, etc. The adaptive adaptation layout description file includes resource information of the functional area, such as control data and other information used for subsequent interface rendering.

For example, as shown in FIG5 , the display state of the electronic device 100 is a vertical screen display, and the display state of the vehicle-mounted terminal 200 is a horizontal screen display. In the process of adapting the top label tab bar 52 of the electronic device 100, it is necessary to rearrange the text of the label tab bar 52 according to the horizontal screen design concept of the display screen of the vehicle-mounted terminal 200, and wrap the line after the determined control data and layout data meet the width constraint. In the process of adapting the banner page 54 of the electronic device 100, it is adapted to a wider horizontal screen display, and the determined control data and layout data can display the banner page hidden on the electronic device 100 side.

For another example, when the electronic device is playing a video in a video application, in response to the user's instruction to cast the screen, it determines that a screen casting request has been received. In addition, the electronic device determines that the display state of the electronic device is portrait display, and the display state of the vehicle terminal is landscape display. Then, in the process of re-layouting the application interface, the electronic device may determine that some playback control controls in the video application are not displayed due to the size of the electronic device's display screen, and the application interface information after re-layout may include information about these playback control controls. The vehicle terminal may then display these playback control controls. That is, the content of the screen casting application displayed by the vehicle terminal may be different from the content of the screen casting application displayed by the electronic device.

S704. The electronic device generates screen projection data according to the rearranged application interface information.

In some embodiments, after the electronic device re-layouts the application interface and determines the application interface information after the re-layout, it can draw and render the corresponding control data and layout data according to the re-layout interface partition file and the adaptive adaptation layout description file to generate projection data, which is used for the subsequent vehicle-mounted terminal to display the projection interface.

The above describes the scenarios where the interface needs to be re-layouted during screen projection. The following describes the scenarios where the interface does not need to be re-layouted during screen projection.

S705. The electronic device matches the application interface to the display area size and generates corresponding screen projection data.

In some embodiments, the electronic device determines that the display status of the electronic device and the vehicle-mounted terminal is the same. Then, the electronic device can directly divide the adaptation interface placement area and the vehicle-mounted application function placement area according to the display screen size information, application layout information and other information of the vehicle-mounted terminal in the vehicle-mounted terminal information. Afterwards, the electronic device scales the size of the interface to be projected according to the size information of the adaptation interface placement area so that the application interface matches the size of the adaptation interface placement area (ie, the display area). The electronic device draws and renders the corresponding interface based on the scaled projection interface information to generate projection data.

Optionally, the electronic device can realize screen projection by determining that only the screen to be projected needs to be scaled. Then, the electronic device can divide the area of the original vehicle terminal used to display fixed icons into the vehicle application function placement area, and the area outside the area used to display fixed icons is the adaptation interface placement area.

Exemplarily, as shown in FIG6 , the electronic device 100 determines that the display states of the electronic device 100 and the vehicle-mounted terminal 200 are both displayed in portrait mode. Based on the acquired vehicle-mounted terminal information, the electronic device 100 can determine that the display area of the display screen of the vehicle-mounted terminal 200 includes an adaptation interface placement area 62 and a vehicle-mounted application function placement area 61. Afterwards, the electronic device 100 can adaptively scale the application interface to be projected based on the size information of the adaptation interface placement area 62.

Alternatively, the electronic device can adjust the display position of the area used by the original vehicle terminal to display fixed icons, and divide the adjusted display position into the vehicle application function placement area. Similarly, the area outside the area used to display fixed icons is the adaptation interface placement area. Among them, since the electronic device does not recognize the controls in the interface to be projected and does not perform layout division, the electronic device will not perform mixed arrangement optimization on the fixed icons and the controls in the interface to be projected. In the process of dividing the vehicle application function placement area and the adaptation interface placement area, the vehicle application function placement area can be divided above or below, to the left or to the right of the adaptation interface placement area to meet display requirements.

In some embodiments, after the electronic device determines that screen projection is required, it is not necessary to execute the above step S701, but directly execute Step S702: That is, after determining that screen projection is required, the electronic device directly generates corresponding screen projection data according to the method for determining screen projection data when the display states of the electronic device and the vehicle-mounted terminal are different (such as steps S701 to S704).

Through the above method, the electronic device can determine the projection data, and then through the following steps, the electronic device can instruct the vehicle terminal to display the projection interface according to the projection data.

S405. The electronic device sends the screen projection data to the vehicle-mounted terminal.

In some embodiments, after determining the projection data, the electronic device may send the projection data to the vehicle-mounted terminal. Correspondingly, the vehicle-mounted terminal receives the projection data sent by the electronic device.

S406. The vehicle-mounted terminal displays the screen projection interface according to the screen projection data.

In some embodiments, after receiving the projection data, the vehicle-mounted terminal can display the corresponding application projection interface according to the projection data.

Exemplarily, corresponding to the scene in which the electronic device and the vehicle terminal display states are different as shown in FIG5, the vehicle terminal 200 may display the interface shown in FIG8 after receiving the projection data. Among them, as shown by reference numeral 81, the display content of the projection application interface displayed by the vehicle terminal 200 is different from the display content of the electronic device 100. The display content indicated by reference numeral 81 is more compatible with the large-screen display of the vehicle terminal 200 display screen, and the display content is richer, which effectively improves the projection display effect.

In addition, as shown in FIG5 , the vehicle terminal 200 displays the fixed icon of the vehicle terminal at the lower edge display position of the display screen. During the application screen projection process, as shown in FIG8 , the electronic device 100 adaptively adjusts the position of the vehicle application function placement area for displaying fixed icons, such as displaying the fixed icons at the left edge display position and the lower edge display position of the display screen, respectively, to ensure the application screen projection effect in the adaptation interface placement area.

As another example, corresponding to the scenario in which the electronic device and the vehicle-mounted terminal have the same display status as shown in Figure 6, the vehicle-mounted terminal 200 may display the interface shown in Figure 9 after receiving the projection data. Among them, as shown in Figure 6, the electronic device 100 has determined the adaptation interface placement area 62 and the vehicle-mounted application function placement area 61, and generated corresponding projection data. Then, as shown in Figure 9, the vehicle-mounted terminal 200 displays the application projection interface in the corresponding adaptation interface placement area. For example, the vehicle-mounted terminal 200 still displays the fixed icon at the display position on the left edge of the display screen, and displays the enlarged application interface at the display position on the right side of the display screen.

In this way, the electronic device can adaptively adjust the display of the application interface to be projected according to the vehicle terminal information to match the vehicle terminal display screen and improve the application projection display effect.

In addition, during the screen projection process, the electronic device can adaptively adjust the interface information of the screen projection application according to the content (such as icons, etc.) that the vehicle terminal needs to display fixedly, thereby ensuring that during the screen projection process, the vehicle terminal can still display the fixed content to meet the user's functional usage needs.

Moreover, through the adaptive screen projection display of electronic devices, developers no longer have to develop different types of the same application based on the vehicle terminal model, effectively reducing the difficulty of application development.

In some embodiments, the projection data may also include audio data and other data, so that the vehicle-mounted terminal can play corresponding audio while displaying the application projection interface.

For example, if the screen projection application is a music application, the audio played by the music application in the electronic device is the local audio data of the electronic device. Then, the electronic device may also include audio data in the screen projection data sent to the vehicle terminal to ensure that the vehicle terminal can play the corresponding audio when displaying the screen projection interface of the music application.

In some embodiments, the vehicle terminal may receive user operations on the projection application during the process of displaying the application projection interface. Therefore, the electronic device can bind the operation response mode of the control corresponding to the operation event on the electronic device interface with the operation event on the vehicle terminal to generate a binding relationship. In this way, the vehicle terminal can support interactive operations on the content displayed on the display screen, and after receiving the user's operation on the content displayed on the display screen, the corresponding operation response can be triggered.

In some examples, the vehicle terminal encapsulates information such as the position information of the control operated by the user into an operation event and sends it to the electronic device to determine the operation to be performed. And/or, the vehicle terminal directly determines the operation response of the control to be performed based on the binding relationship.

Optionally, after detecting a user operation, the vehicle terminal may generate a corresponding operation event, which may include information such as the position information of the control operated by the user. In addition, the vehicle terminal may send the operation event to the electronic device. Afterwards, the electronic device determines the operation response mode of the control corresponding to the operation event based on the received operation event and the pre-generated binding relationship, so as to execute or instruct the vehicle terminal to execute the corresponding operation.

Optionally, the electronic device sends the binding relationship to the vehicle-mounted terminal. In this way, after detecting the user operation, the vehicle-mounted terminal can determine the control operated by the user according to the binding relationship.

For example, the display screen of the vehicle terminal is a touch screen. After generating the projection data (such as the above step S704 or step S705), the electronic device can bind the controls on the application interface with the corresponding functions to generate a binding relationship.

For another example, the display screen of the vehicle terminal is a non-touch screen, or the display screen of the vehicle terminal can be controlled by a button or a knob. After generating the projection data, the electronic device can bind the virtual selection of the button or knob with the control selection on the application interface to generate a binding relationship.

For example, the electronic device determines that the display screen of the vehicle terminal is a non-touch screen and supports knob control according to the vehicle terminal information obtained through the above step S402. Then, after the electronic device determines the projection data through the above step S704 or step S705, it can divide the application interface, number the controls therein according to the matrix grid, match the rotation of the knob to the forward, backward, selection and other operations, and correspond to the controls with the corresponding numbers to generate a binding relationship.

Optionally, in the above step S402, the vehicle terminal information obtained by the electronic device also includes the hardware information of the vehicle terminal. Then, the electronic device can virtualize the hardware resources of the vehicle terminal on the electronic device side, so that in the process of determining the binding relationship, the response mode of the control can also be adjusted to correspond to the corresponding virtual hardware resources, which is used to indicate the hardware operation of the vehicle terminal, such as turning on the speaker and camera of the vehicle terminal.

In some embodiments, an application for screen projection and display of an electronic device may or may not be installed in the vehicle terminal. Then, in the case where the vehicle terminal is installed with an application for screen projection and display of an electronic device, the vehicle terminal may determine whether the display frame rate meets the frame rate threshold requirement when the application is started and displayed through the chip of the vehicle terminal in the process of displaying the application interface according to the received screen projection data. If satisfied, the vehicle terminal can start the application and instruct the electronic device not to project and display the application, that is, switch the application to the vehicle terminal, thereby improving the application operation efficiency. If not satisfied, the vehicle terminal can continue to display the screen projection application according to the received screen projection data, so that the screen projection application runs on the electronic device.

In some embodiments, the electronic device may also determine the electronic device function icon to be displayed on the vehicle terminal during the process of generating the projection data. Afterwards, the projection data sent by the electronic device to the vehicle terminal includes the electronic device function icon. Subsequently, the vehicle terminal includes the electronic device function icon in the projection interface displayed according to the projection data, and the user can control the electronic device through the electronic device function icon on the vehicle terminal.

For example, when the vehicle terminal detects the user's operation on the electronic device function icon during the display of the projection interface, it can send an indication message to the electronic device, and the indication message can carry information about the electronic device function icon operated by the user. Afterwards, the electronic device can execute the function corresponding to the electronic device function icon operated by the user according to the indication message.

The above is a detailed description of the screen projection display method provided by the embodiment of the present application in combination with Figures 4 to 9. The following is a detailed description of the electronic device provided by the embodiment of the present application in combination with Figure 10, and a detailed description of the vehicle-mounted terminal provided by the embodiment of the present application in combination with Figure 11.

In a possible design, Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 10, the electronic device 1000 may include: a transceiver unit 1001 and a processing unit 1002. The electronic device 1000 may be used to implement the functions of the electronic device involved in the above method embodiment.

Optionally, the transceiver unit 1001 is used to support the electronic device 1000 to execute S401, S402 and S405 in FIG. 4 .

Optionally, the processing unit 1002 is used to support the electronic device 1000 to execute S403 and S404 in Figure 4; and/or to support the electronic device 1000 to execute S701-S705 in Figure 7.

Among them, the transceiver unit may include a receiving unit and a sending unit, which may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and/or functions of each unit in the electronic device 1000 are respectively to implement the corresponding process of the projection display method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit. For the sake of brevity, it will not be repeated here.

Optionally, the electronic device 1000 shown in FIG10 may further include a storage unit (not shown in FIG10 ), in which a program or instruction is stored. When the transceiver unit 1001 and the processing unit 1002 execute the program or instruction, the electronic device 1000 shown in FIG10 may execute the screen projection display method described in the above method embodiment.

The technical effects of the electronic device 1000 shown in FIG. 10 may refer to the technical effects of the screen projection display method described in the above method embodiment, and will not be repeated here.

In addition to being in the form of electronic device 1000, the technical solution provided by the present application may also be a functional unit or chip in the electronic device, or a device used in conjunction with the electronic device.

In a possible design, FIG11 is a schematic diagram of the structure of the vehicle terminal provided in an embodiment of the present application. As shown in FIG11, the vehicle terminal 1100 may include: a transceiver unit 1101, a processing unit 1102, and a display unit 1103. The vehicle terminal 1100 may be used to implement the functions of the vehicle terminal involved in the above method embodiment.

Optionally, the transceiver unit 1101 is used to support the vehicle terminal 1100 to execute S401, S402 and S405 in Figure 4.

Optionally, the processing unit 1102 is used to support the vehicle terminal 1100 to execute S406 in Figure 4.

Optionally, the display unit 1103 is used to support the vehicle-mounted terminal 1100 to execute S406 in FIG. 4 .

Among them, the transceiver unit may include a receiving unit and a sending unit, which may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and/or functions of each unit in the vehicle terminal 1100 are respectively to implement the corresponding process of the projection display method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit. For the sake of brevity, it will not be repeated here.

Optionally, the vehicle-mounted terminal 1100 shown in FIG11 may further include a storage unit (not shown in FIG11 ), in which a program or instruction is stored. When the transceiver unit 1101, the processing unit 1102, and the display unit 1103 execute the program or instruction, the vehicle-mounted terminal 1100 shown in FIG11 may execute the screen projection display method described in the above method embodiment.

The technical effects of the vehicle-mounted terminal 1100 shown in FIG. 11 can refer to the technical effects of the screen projection display method described in the above method embodiment, and will not be repeated here.

In addition to being in the form of the vehicle-mounted terminal 1100, the technical solution provided by the present application may also be a functional unit or chip in the vehicle-mounted terminal, or a device used in conjunction with the vehicle-mounted terminal.

An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned related steps to implement the screen projection display method in the above-mentioned embodiment.

An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the screen projection display method in the above-mentioned embodiment.

In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the screen projection display method in the above-mentioned method embodiments.

The apparatus, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute The corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

The steps of the method or algorithm described in conjunction with the disclosed content of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, read-only compact disks (CD-ROMs) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

Computer-readable storage media include, but are not limited to, any of the following: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store program codes.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

A screen projection display method, characterized in that it is applied to an electronic device, and the method comprises:

Acquire first information sent by the vehicle-mounted terminal, where the first information includes vehicle-mounted terminal information;

Receive screen projection request;

In response to the screen projection request, determining first interface information of a first application indicated by the screen projection request for screen projection;

Determining screen projection data according to the in-vehicle terminal information and the first interface information;

The screen projection data is sent to the vehicle-mounted terminal, and the screen projection data is used to enable the vehicle-mounted terminal to display a screen projection interface.
The method according to claim 1 is characterized in that the vehicle-mounted terminal information includes application layout information.
The method according to claim 2 is characterized in that the application layout information includes desktop layout information of the vehicle-mounted terminal.
The method according to claim 2 or 3, characterized in that the vehicle-mounted terminal information also includes display screen size information, and the method further includes:

In response to the screen projection request, it is determined based on the display screen size information whether the display status of the electronic device and the vehicle-mounted terminal is the same or different, and the display status is used to indicate horizontal screen display or vertical screen display.
The method according to claim 4, characterized in that the determining, based on the display screen size information, whether the display states of the electronic device and the vehicle-mounted terminal are the same or different comprises:

Determine a first aspect ratio of a display screen of the electronic device, and determine a second aspect ratio of a display screen of the vehicle-mounted terminal according to the display screen size information;

When the absolute value of the difference between the first aspect ratio and the second aspect ratio is less than or equal to a first threshold, the display states of the electronic device and the vehicle-mounted terminal are the same;

When the absolute value of the difference between the first aspect ratio and the second aspect ratio is greater than the first threshold, the display states of the electronic device and the vehicle-mounted terminal are different.
The method according to claim 4 or 5, characterized in that the display states of the electronic device and the vehicle-mounted terminal are different, and the determining, in response to the screen projection request, the first interface information of the first application indicated by the screen projection request for projection comprises:

In response to the screen projection request, determining a first interface of the first application to be projected;

The first interface is divided into functional areas, and the first interface information is determined.
The method according to claim 6, characterized in that the step of determining the projection data according to the vehicle terminal information and the first interface information comprises:

Generate second interface information by re-layouting the first display content corresponding to the divided functional area and the second display content corresponding to the application layout information according to the first interface information, the display screen size information, and the application layout information;

Generate the screen projection data according to the second interface information.
The method according to claim 7, characterized in that the step of re-layouting the first display content corresponding to the functional area and the second display content corresponding to the application layout information according to the first interface information, the display screen size information, and the application layout information to generate the second interface information comprises:

According to the functional area, the display screen size information, and the application layout information, the display area corresponding to the display screen of the vehicle terminal is divided into a first display area and a second display area, the first display area is used to display the first display content, and the second display area is used to display the second display content;

Rearranging the first display content according to the first display area, and rearranging the second display content according to the second display area;

The second interface information is determined according to the first display content and the second display content after the rearrangement.
The method according to claim 4 or 5, characterized in that the display status of the electronic device and the vehicle-mounted terminal are the same, and the determining of the screen projection data according to the vehicle-mounted terminal information and the first interface information comprises:

Determine, according to the display screen size information and the application layout information, a first display area for projecting and displaying the first application, wherein the first display area is a display area other than a second display content corresponding to the application layout information;

According to the first display area, enlarging or reducing the first interface of the first application corresponding to the first interface information to generate third interface information;

The screen projection data is determined according to the third interface information and the application layout information.
The method according to claim 8 or 9 is characterized in that the vehicle-mounted terminal information also includes application information of the vehicle-mounted terminal, the second display content includes function icons corresponding to the application layout information and/or commonly used application icons of the vehicle-mounted terminal, the function icons are function icons displayed by the vehicle-mounted terminal before screen projection and/or function icons used to control the electronic device, and the commonly used application icons are icons of commonly used applications of the vehicle-mounted terminal determined by the electronic device based on the application information.
The method according to any one of claims 1 to 10 is characterized in that the vehicle-mounted terminal information also includes a display screen control method and hardware information of the vehicle-mounted terminal, the display screen control method is used to indicate whether the vehicle-mounted terminal is a touch screen or a non-touch screen, and before sending the projection data to the vehicle-mounted terminal, the method further includes:

Determine an operation response mode corresponding to a first control in the projection data;

A binding relationship is generated based on the first control, the operation response mode, the hardware information, and the display screen control method, and the binding relationship is used to indicate the first response of the vehicle-mounted terminal after detecting the user's operation on the first control displayed in the projection interface.
The method according to claim 11, characterized in that the method further comprises:

The binding relationship is sent to the vehicle-mounted terminal.
The method according to any one of claims 1-12 is characterized in that the screen projection request is a screen projection request determined after the electronic device receives the user's screen projection operation, or the screen projection request is a screen projection request sent by the vehicle-mounted terminal.
A screen projection display method, characterized in that it is applied to a vehicle-mounted terminal, and the method comprises:

Sending first information to the electronic device, wherein the first information includes vehicle-mounted terminal information;

Receiving screen projection data sent by the electronic device, the screen projection data being data determined by the electronic device in response to the received screen projection request, according to first interface information of the first application indicated by the screen projection request and the in-vehicle terminal information;

The screen projection interface is displayed according to the screen projection data.
The method according to claim 14 is characterized in that the vehicle-mounted terminal information includes application layout information.
The method according to claim 15 is characterized in that the application layout information includes desktop layout information of the vehicle-mounted terminal.
According to the method described in any one of claims 14-16, it is characterized in that the vehicle-mounted terminal information also includes display screen size information, and the display screen size information is used to enable the electronic device to determine the display status of the vehicle-mounted terminal, and the display status is used to indicate horizontal screen display or vertical screen display.
The method according to any one of claims 14 to 17, characterized in that the method further comprises:

Receive the binding relationship sent by the electronic device, wherein the binding relationship is used to indicate the first response of the vehicle-mounted terminal after detecting the user's operation on the first control displayed on the projection interface; the vehicle-mounted terminal information also includes the display screen control method and hardware information of the vehicle-mounted terminal, and the display screen control method and the hardware information are used to enable the electronic device to generate the binding relationship.
The method according to any one of claims 14 to 18, characterized in that before receiving the screen projection data sent by the electronic device, the method further comprises:

In response to the user's screen projection operation, the screen projection request is sent to the electronic device.
A screen projection display system, characterized in that the system includes an electronic device and a vehicle-mounted terminal;

The vehicle-mounted terminal is used to send first information to the electronic device, wherein the first information includes vehicle-mounted terminal information;

The electronic device is used to receive a screen projection request;

The electronic device is further configured to determine, in response to the screen projection request, first interface information of a first application that the screen projection request indicates is to be projected;

The electronic device is further used to determine the screen projection data according to the first interface information and the vehicle terminal information in the received first information;

The electronic device is further used to send the screen projection data to the vehicle-mounted terminal;

The vehicle-mounted terminal is also used to display a screen projection interface based on the received screen projection data.
The system according to claim 20 is characterized in that the vehicle-mounted terminal information includes application layout information.
The system according to claim 21, characterized in that the application layout information includes the desktop of the vehicle terminal Layout information.
The system according to claim 21 or 22, characterized in that the vehicle terminal information also includes display screen size information;

The electronic device is also used to respond to the screen projection request and determine whether the display status of the electronic device and the vehicle-mounted terminal is the same or different according to the display screen size information, and the display status is used to indicate horizontal screen display or vertical screen display.
The system according to claim 23, characterized in that, when the display states of the electronic device and the vehicle-mounted terminal are different,

The electronic device is further configured to determine, in response to the screen projection request, a first interface of the first application to be projected;

The electronic device is further used to divide the functional areas of the first interface and determine the first interface information.
The system according to claim 24, characterized in that

The electronic device is further configured to re-layout the first display content corresponding to the divided functional area and the second display content corresponding to the application layout information according to the first interface information, the display screen size information, and the application layout information, to generate the second interface information;

The electronic device is also used to generate the screen projection data according to the second interface information.
The system according to claim 23, characterized in that, when the display states of the electronic device and the vehicle-mounted terminal are the same,

The electronic device is further used to determine, based on the display screen size information and the application layout information, a first display area for projecting and displaying the first application, wherein the first display area is a display area other than a second display content corresponding to the application layout information;

The electronic device is further configured to enlarge or reduce the first interface of the first application corresponding to the first interface information according to the first display area to generate third interface information;

The electronic device is also used to determine the screen projection data based on the third interface information and the application layout information.
An electronic device, characterized in that it includes: a processor and a memory, wherein the memory is coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 13.
A vehicle-mounted terminal, characterized in that it includes: a processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program codes, and the computer program codes include computer instructions. When the processor reads the computer instructions from the memory, the vehicle-mounted terminal executes the method described in any one of claims 14 to 19.
A computer-readable storage medium, characterized in that the computer-readable storage medium includes a computer program, and when the computer program is run on an electronic device, the electronic device executes the method as described in any one of claims 1 to 13.
A computer-readable storage medium, characterized in that the computer-readable storage medium includes a computer program, and when the computer program is run on a vehicle-mounted terminal, the vehicle-mounted terminal executes the method as described in any one of claims 14-19.
A computer program product, characterized in that when the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 13.
A computer program product, characterized in that when the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 14 to 19.