CN113590059A

CN113590059A - Screen projection method and mobile terminal

Info

Publication number: CN113590059A
Application number: CN202010361177.XA
Authority: CN
Inventors: 修平; 刘涛; 王双
Original assignee: Hisense Mobile Communications Technology Co Ltd
Current assignee: Hisense Mobile Communications Technology Co Ltd
Priority date: 2020-04-30
Filing date: 2020-04-30
Publication date: 2021-11-02

Abstract

The application discloses a screen projection method and a mobile terminal, which are used for projecting contents displayed on a current screen to other screen windows of the same terminal. After receiving a screen projection instruction triggered by a user in a source screen window, a mobile terminal determines a target screen window except the source screen window from at least two screen windows displayed on a display screen of the mobile terminal, and projects multimedia content currently displayed on the source screen window to the target screen window. According to the embodiment of the invention, other screen windows of the mobile terminal except the source screen window can be used as screen windows for receiving screen projection, and after the multimedia content currently displayed by the source screen window is projected to the target screen window, the screen projection function among different screen windows of the same mobile terminal is realized. A method for flexibly using a plurality of screen windows in the mobile terminal is provided for a user, and user experience is improved.

Description

Screen projection method and mobile terminal

Technical Field

The invention relates to the technical field of terminal display, in particular to a screen projection method and a mobile terminal.

Background

The screen projection refers to projecting an interface displayed on a screen of one device onto screens of other devices, and is widely applied to smart homes or office scenes, and a user can project content displayed on the screen of the current device onto the other devices to obtain a better watching or operating effect.

With the popularization of mobile terminals, the functions of mobile terminals are increasing, and a mobile terminal with a split screen function or multiple screens supports multiple screens to be displayed simultaneously, so that a better use experience is provided for a user, but currently, when the user operates a mobile terminal with multiple screen windows, the user can only operate different screen windows respectively, when the user uses one screen window of the mobile terminal to watch multimedia content, if the user wants to use other screen windows of the mobile terminal to show the display content of the current screen window, the user can only operate application programs on other screen windows to show the multimedia content again, and there is no scheme capable of projecting the content displayed on the current screen window to other screen windows of the same terminal.

Disclosure of Invention

The exemplary embodiment of the invention provides a screen projection method and a mobile terminal, which are used for projecting contents displayed on a current screen to other screens of the same terminal.

According to a first aspect of the exemplary embodiments there is provided a communication terminal comprising a display and a processor:

the display screen is configured to display at least two screen windows, and a screen projection instruction triggered in the screen windows by a user is detected;

the processor is configured to determine a target screen window except a source screen window from the at least two screen windows displayed by the display screen after receiving a screen projection instruction triggered by a user in the source screen window; and projecting the multimedia content currently displayed by the source screen window to the target screen window.

In the above embodiment, in response to a screen-casting instruction triggered by a user in a source screen window, a mobile terminal determines a target screen window other than a screen window from at least two screen windows of the mobile terminal, and casts multimedia content currently displayed in the source screen window to the target screen window, because the embodiment of the present invention can determine other screen windows of the mobile terminal other than the source screen window as the target screen window, and can use other screen windows of the mobile terminal as screen windows for accepting screen casting, after casting the multimedia content currently displayed in the source screen window to the target screen window, the screen casting function between different screen windows of the same mobile terminal is realized, the user can continue to use the source screen window to perform other operations after screen casting, and can continue to watch the multimedia content cast in the target screen window, thereby providing a method for flexibly using a plurality of screen windows in the mobile terminal for the user, and the user experience is improved.

In some exemplary embodiments, the treatment appliance is configured to:

scanning available screen windows to determine an available screen window set; wherein a screen window in the available screen window set is another screen window except the source screen window in the at least two screen windows displayed by the display screen;

and responding to an instruction of a user for selecting a screen window from the available screen window set, and taking the screen window selected by the user as the target screen window.

In the above embodiment, the mobile terminal may use other screen windows of the mobile terminal itself except the source screen window as the available screen windows, so that the media content displayed on the source screen window can be displayed on other screen windows of the mobile terminal, and screen projection between different screen windows in the same mobile terminal is realized.

In some exemplary embodiments, there is one DLNA application per screen window;

the treatment appliance is configured to:

and scanning the DLNA application, and generating the available screen window set according to the scanned screen window corresponding to the DLNA application of the mobile terminal.

In the above embodiment, because the screen window corresponding to the DLNA application of the mobile terminal can be scanned in the embodiment of the present invention, other screen windows in the mobile terminal can be used as target screen windows for accepting screen projection.

In some exemplary embodiments, the treatment appliance is configured to:

acquiring configuration information corresponding to the scanned DLNA application;

aiming at any one scanned DLNA application, judging whether the DLNA application is the DLAN application of the mobile terminal according to the configuration information corresponding to the DLNA application; judging whether the DLAN application is matched with the multimedia content or not according to the configuration information corresponding to the DLNA application;

and composing the screen windows corresponding to the DLAN application matched with the multimedia content in the scanned DLAN applications of the mobile terminal into the available screen window set.

In the above embodiment, the mobile terminal may determine whether the scanned DLNA application is the DLNA application of the mobile terminal, and may determine whether the screen window corresponding to the DLNA application is matched with the multimedia content, so that other screen windows that belong to the mobile terminal and can meet the service required by the screen projection at this time may be used as available screen windows, so that when the user selects the target screen window to be projected, other screen windows of the mobile terminal itself, except for the source screen window, may be selected for screen projection, thereby improving user experience.

In some exemplary embodiments, the treatment appliance is configured to:

and transmitting the multimedia content to the DLNA application corresponding to the target screen window through the DLNA application corresponding to the source screen window, and displaying the multimedia content in the target screen window.

In the above embodiment, the DLNA application corresponding to the source screen window can transmit the multimedia content to the DLNA application corresponding to the target screen window, so that the DLNA application corresponding to the target screen window is displayed according to the received multimedia content, and the screen projection process of different screen windows of the same mobile terminal is completed.

According to a second aspect of the exemplary embodiments there is provided a mobile terminal comprising a processor and two display screens;

the display screen is configured to detect a user-triggered screen projection instruction;

the processor is configured to receive a screen projection instruction triggered by a user through a currently used display screen, and project multimedia content displayed in the currently used display screen to another display screen.

In some exemplary embodiments, there is one DLNA application for each display screen;

the processor is further configured to:

scanning the DLNA application and determining the DLNA application corresponding to the other display screen;

and transmitting the multimedia content to the DLNA application corresponding to the other display screen through the DLNA application corresponding to the currently used display screen, and displaying the multimedia content in the other display screen.

According to a third aspect of the exemplary embodiments, there is provided a screen projection method, including:

after receiving a screen projection instruction triggered in a source screen window by a user, a mobile terminal determines a target screen window except the source screen window from at least two screen windows displayed on a display screen of the mobile terminal;

and the mobile terminal projects the multimedia content currently displayed by the source screen window to the target screen window.

In some exemplary embodiments, the determining a target screen window other than the source screen window from at least two screen windows displayed on a display screen of the mobile terminal includes:

the mobile terminal scans available screen windows and determines an available screen window set; wherein a screen window in the available screen window set is another screen window except the source screen window in the at least two screen windows displayed by the display screen;

and responding to an instruction of a user for selecting a screen window from the available screen window set, wherein the screen window selected by the user is taken as the target screen window by the mobile terminal.

In some exemplary embodiments, there is one DLNA application per screen window;

the mobile terminal scans available screen windows and determines an available screen window set, including:

and the mobile terminal scans the DLNA application and generates the available screen window set according to the scanned screen window corresponding to the DLNA application of the mobile terminal.

In some exemplary embodiments, the generating the set of available screen windows according to the scanned screen windows corresponding to the DLNA application of the mobile terminal includes:

the mobile terminal acquires configuration information corresponding to the scanned DLNA application;

aiming at any one scanned DLNA application, the mobile terminal judges whether the DLNA application is the DLAN application of the mobile terminal according to the configuration information corresponding to the DLNA application; judging whether the DLAN application is matched with the multimedia content or not according to the configuration information corresponding to the DLNA application;

and the mobile terminal makes the screen windows corresponding to the DLAN application matched with the multimedia content in the scanned DLAN applications of the mobile terminal into the available screen window set.

In some exemplary embodiments, the method for the mobile terminal to project the multimedia content currently displayed in the source screen window to the target screen window includes:

and the mobile terminal transmits the multimedia content to the DLNA application corresponding to the target screen window through the DLNA application corresponding to the source screen window, and displays the multimedia content in the target screen window.

According to a fourth aspect of the exemplary embodiments, there is provided a screen projection method applied to a mobile terminal including two display screens, including:

the method comprises the steps that a mobile terminal receives a screen projection instruction triggered by a user through a currently used display screen;

and the mobile terminal casts the multimedia content displayed in the currently used display screen to another display screen.

the mobile terminal screens the multimedia content displayed in the currently used display screen to another display screen, and the method comprises the following steps:

the mobile terminal scans the DLNA application and determines the DLNA application corresponding to the other display screen;

and the mobile terminal transmits the multimedia content to the DLNA application corresponding to the other display screen through the DLNA application corresponding to the currently used display screen, and displays the multimedia content in the other display screen.

According to a fifth aspect of exemplary embodiments, there is provided a screen projection apparatus configured to perform the screen projection method as described in the third aspect above.

According to a sixth aspect of the exemplary embodiments, there is provided a screen projection apparatus configured to perform the screen projection method according to the fourth aspect.

According to a seventh aspect of the exemplary embodiments, there is provided a computer storage medium having stored therein computer program instructions, which when run on a computer, cause the computer to perform the screen projection method according to the third or fourth aspect.

On the basis of the common knowledge in the field, the above preferred conditions can be combined randomly to obtain the preferred embodiments of the application.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating a software architecture of a mobile terminal according to an embodiment of the present invention;

FIG. 3 is a schematic diagram illustrating a user interface of a mobile terminal according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating a method for screen projection between different screen windows according to an embodiment of the present invention;

fig. 5 is a schematic diagram illustrating a split screen function of a mobile terminal according to an embodiment of the present invention;

fig. 6 is a diagram illustrating a picture-in-picture mode of a mobile terminal according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a user-triggered screen-projection instruction provided by an embodiment of the present invention;

FIG. 8 is a diagram illustrating a user selecting a target screen window from a set of available screen windows provided by an embodiment of the invention;

FIG. 9 is a diagram illustrating a screen-dropping command triggered in a first split-screen window in the split-screen mode according to an embodiment of the present invention;

FIG. 10 illustrates a schematic view of an embodiment of the present invention in a split-screen mode for projecting into a second split-screen window;

FIG. 11 is a flowchart illustrating a screen projection method between different screen windows according to an embodiment of the present invention;

FIG. 12 is a flowchart illustrating a screen projection method between different display screens of the same mobile terminal according to an embodiment of the present invention;

fig. 13 is a schematic diagram illustrating a dual-screen terminal according to an embodiment of the present invention;

fig. 14a is a schematic diagram illustrating that a folding screen terminal according to an embodiment of the present invention uses two screens respectively;

fig. 14b is a schematic diagram illustrating a folded screen terminal according to an embodiment of the present invention using a full screen;

fig. 15 is a schematic diagram illustrating that an available display screen is shown as a back screen in a source display screen in a dual-screen terminal according to an embodiment of the present invention;

FIG. 16 is a diagram illustrating an embodiment of the present invention triggering a screen-cast command in one of the folding screens;

FIG. 17 illustrates a schematic view of an embodiment of the present invention projecting a screen in a folded screen to another display screen;

fig. 18 is a flowchart illustrating a complete screen projection method between different display screens of the same mobile terminal according to an embodiment of the present invention;

fig. 19 is a schematic structural diagram illustrating a first mobile terminal according to an embodiment of the present invention;

fig. 20 is a schematic structural diagram schematically illustrating a first screen projection device provided by an embodiment of the invention;

fig. 21 is a schematic structural diagram illustrating a second mobile terminal according to an embodiment of the present invention;

fig. 22 is a schematic structural diagram schematically illustrating a second screen projection device provided by an embodiment of the invention.

Detailed Description

The technical solution in the embodiments of the present application will be described in detail and removed with reference to the accompanying drawings. In the description of the embodiments herein, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.

In the following, the terms "first", "second" are used for descriptive purposes only and are not to be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of embodiments of the application, unless stated otherwise, "plurality" means two or more.

Some terms appearing herein are explained below:

1. in the embodiment of the invention, the term "Digital Living Network Alliance (DLNA)" aims to solve the interconnection and intercommunication between a wireless Network and a wired Network, such as a personal computer, a consumer appliance and a mobile device, so that unlimited sharing and growth of Digital media and content services are possible.

3. The term "screen projection" in the embodiment of the present invention refers to that one device synchronizes the content displayed on the device to another device for playing, viewing or operating in a screen projection manner.

Fig. 1 shows a schematic configuration of a mobile terminal 100.

The following describes an embodiment specifically by taking the mobile terminal 100 as an example. It should be understood that the mobile terminal 100 shown in fig. 1 is merely an example, and that the mobile terminal 100 may have more or fewer components than shown in fig. 1, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

A block diagram of a hardware configuration of a mobile terminal 100 according to an exemplary embodiment is exemplarily shown in fig. 1. As shown in fig. 1, the mobile terminal 100 includes: a Radio Frequency (RF) circuit 110, a memory 120, a display unit 130, a camera 140, a sensor 150, an audio circuit 160, a Wireless Fidelity (Wi-Fi) module 170, a processor 180, a bluetooth module 181, and a power supply 190.

The RF circuit 110 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink data of a base station and then send the downlink data to the processor 180 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

The memory 120 may be used to store software programs and data. The processor 180 performs various functions of the mobile terminal 100 and data processing by executing software programs or data stored in the memory 120. The memory 120 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 120 stores an operating system that enables the mobile terminal 100 to operate. The memory 120 may store an operating system and various application programs, and may also store codes for performing the methods described in the embodiments of the present application.

The display unit 130 may be used to receive input numeric or character information, generate signal input related to user settings and function control of the communication terminal 100, and specifically, the display unit 130 may include a touch screen 1301 disposed on the front side of the communication terminal 100, and may collect touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like;

the display unit 130 may also be used to display a Graphical User Interface (GUI) of information input by or provided to the user and various menus of the terminal 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front surface of the communication terminal 100. The display 132 may be a color liquid crystal display, and may be configured in the form of a liquid crystal display, a light emitting diode, or the like. The display unit 130 may be used to display various graphical user interfaces described herein.

The touch screen 131 may cover the display screen 132, or the touch screen 131 and the display screen 132 may be integrated to implement the input and output functions of the communication terminal 100, and after the integration, the touch screen may be referred to as a touch display screen for short. The display unit 130 in the present application can display the application program and the corresponding operation steps, and can be used as a source screen in the implementation of the present application.

When the mobile terminal is a mobile terminal having two screens in the present application, the display unit 140 may be further included and may be configured to receive input numeric or character information and generate signal input related to user setting and function control of the communication terminal 100, and specifically, the display unit 140 may include a touch screen 141 disposed on a rear surface of the communication terminal 100 or connected to the touch screen 131 in the display unit 130 and may collect touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like.

The display unit 140 may also be used to display information input by the user or information provided to the user and a GUI for various menus of the terminal 100. Specifically, the display unit 140 may include a display screen 142 disposed at the rear of the communication terminal 100 or connected to the display screen 132 in the display unit 130. The display screen 142 may be a color liquid crystal screen, and may be configured in the form of a liquid crystal display, a light emitting diode, or the like. The display unit 140 may be used to display various graphical user interfaces described herein.

The touch screen 141 may cover the display screen 142, or the touch screen 141 and the display screen 142 may be integrated to implement the input and output functions of the communication terminal 100, and the integrated touch screen may be referred to as a touch display screen for short. The display unit 140 in the present application may display the application programs and the corresponding operation steps, and may be used as other screens besides the source screen in the implementation of the present application.

Camera 150 may be used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing elements convert the light signals into electrical signals which are then passed to processor 190 for conversion into digital image signals.

The mobile terminal 100 may further include at least one sensor 160, such as an acceleration sensor 161, a distance sensor 162, a fingerprint sensor 163, a temperature sensor 164. The mobile terminal 100 may also be configured with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, motion sensor, and the like.

The audio circuitry 170, speaker 171, microphone 172 may provide an audio interface between a user and the mobile terminal 100. The audio circuit 170 may transmit the electrical signal converted from the received audio data to the speaker 171, and convert the electrical signal into a sound signal by the speaker 171 for output. The mobile terminal 100 may also be provided with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 172 converts the collected sound signals into electrical signals, converts the electrical signals into audio data after being received by the audio circuit 170, and outputs the audio data to the RF circuit 110 to be transmitted to, for example, another terminal or outputs the audio data to the memory 120 for further processing. In the present application, the microphone 172 may capture the voice of the user.

Wi-Fi belongs to a short-distance wireless transmission technology, and the mobile terminal 100 may help a user to receive and transmit e-mails, browse webpages, access streaming media, and the like through the Wi-Fi module 180, which provides a wireless broadband internet access for the user.

The processor 190 is a control center of the mobile terminal 100, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the mobile terminal 100 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. In some embodiments, processor 190 may include one or more processing units; processor 190 may also integrate an application processor, which primarily handles operating systems, user interfaces, and applications, etc., with a baseband processor, which primarily handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into processor 190. In the present application, the processor 190 may run an operating system, an application program, a user interface display, and a touch response, and the processing method described in the embodiments of the present application. Further, processor 190 is coupled to display unit 130.

And the Bluetooth module 191 is used for performing information interaction with other Bluetooth devices with Bluetooth modules through a Bluetooth protocol. For example, the mobile terminal 100 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) also equipped with a bluetooth module through the bluetooth module 191, so as to perform data interaction.

The mobile terminal 100 also includes a power supply 1100 (such as a battery) that powers the various components. The power supply may be logically coupled to processor 190 through a power management system to manage charging, discharging, and power consumption functions through the power management system. The mobile terminal 100 may also be configured with power buttons for powering the terminal on and off, and locking the screen.

Fig. 2 is a block diagram of a software configuration of the mobile terminal 100 according to the embodiment of the present invention.

The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, an application layer, an application framework layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom.

The application layer may include a series of application packages.

As shown in fig. 2, the application package may include applications such as camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 2, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide a communication function of the mobile terminal 100. Such as management of call status (including on, off, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top status bar of the system, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, text information is prompted in the status bar, a prompt tone is given, the mobile terminal vibrates, an indicator light flashes, and the like.

The Android Runtime comprises a core library and a virtual machine. The Android runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. And executing java files of the application program layer and the application program framework layer into a binary file by the virtual machine. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), Media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, and the like.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The following describes exemplary workflow of the mobile terminal 100 software and hardware in connection with capturing a photo scene.

When the touch screen 131 receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, a time stamp of the touch operation, and other information). The raw input events are stored at the kernel layer. And the application program framework layer acquires the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and taking a control corresponding to the click operation as a control of a camera application icon as an example, the camera application calls an interface of an application framework layer, starts the camera application, further starts a camera drive by calling a kernel layer, and captures a still image or a video through the camera 140.

The mobile terminal 100 in the embodiment of the present application may be a mobile phone, a tablet computer, a wearable device, a notebook computer, a television, and the like.

Fig. 3 is a schematic diagram illustrating a user interface on a mobile terminal (e.g., mobile terminal 100 of fig. 1). In some implementations, a user can open a corresponding application by touching an application icon on the user interface, or can open a corresponding folder by touching a folder icon on the user interface.

Digital Living Network Alliance (DLNA) aims to solve the interconnection and interworking between wireless networks and wired networks including personal computers, consumer electronics, and mobile devices, so that unlimited sharing and growth of Digital media and content services are possible.

The DLNA screen projection technology comprises the following parts:

1. a Digital Media Server (DMS), a provider of Media content, which provides content playback for a Digital Media player or/and a Digital Media renderer, and the DMS may control the provided Media content. Typical devices are such as digital set-top boxes, mobile terminals, music players, etc.

2. Digital Media Players (DMP), which can search for Media content from a DMS and play the Media content, typically include: smart televisions, and the like.

3. A Digital Media Renderer (DMR) can play Media content on a DMS, which is different from a DMP in that the DMR has only a Media playing function, and the DMP also has a function of browsing and searching for Media, such as: display, audio amplifier.

4. A Digital Media Controller (DMC) may search the DMS contents, establish a connection between the DMS and DMR, and control the playing of Media, and typical devices are: and a remote controller.

The DLNA screen projection system comprises control equipment, controlled equipment and services; the control equipment is also called a control point and is equipment for triggering screen projection, and the control equipment controls various operations in DLNA screen projection, such as video playing, video stopping and other operations; controlled equipment is also called equipment, is equipment for providing screen projection service, and can be internet of things equipment, such as: household appliances, television boxes, etc. The service refers to a service supported or provided by the controlled device, such as supporting: control service, event service, presentation service.

In the DLNA screen projection process, DMS is on the control device, and DMP or DMR is on the controlled device.

The DLNA screen projection technology uses Universal Plug and Play (UPnP) protocol for communication transmission.

The DLNA screen projection process includes device discovery, device control, and device events.

After a user operates a control device to trigger a screen-casting instruction, the control device is added to a local area network as a control point, and searches for available devices in the local area network through a Simple Service Discovery Protocol (SSDP), wherein the SSDP specifically includes a mode in which the control device discovers a Service on the network, and related information of the Service, and also includes Service contents that the control device itself can provide.

When a user selects one device from the searched available device list to be used as a controlled device, the controlled device is controlled by the control device through Simple Object Access Protocol (SOAP), the SOAP defines how to use EXtensible Markup Language (xml) and HyperText Transfer Protocol (HTTP) to execute remote procedure call, and the SOAP defines how to send a command message to the controlled device, and how to send a response message to the control device after receiving the command message.

In the service running time, the variable value in the service changes or the mode state changes, an event is generated, the event service provider can multicast the event to the whole network, the control device can subscribe the event information to the event server in advance, and the event which is interested by the control device is timely and accurately unicast-transmitted. In the process of controlling the controlled device to perform screen projection by the control device, the control device transmits subscription information to the controlled device and receives information returned by the device through a Generic Event Notification Architecture (GENA), where GENA defines how the control device transmits the subscription information and receives information when the control device wants to monitor the status of a certain service state variable of the controlled device.

However, the current DLNA only supports screen projection between different devices, and with the popularization of mobile terminals, a mobile terminal with a split screen function or a picture-in-picture function or with multiple screens provides a better use experience for users, but currently, when a user operates a mobile terminal with multiple screen windows, the user can only operate different screen windows respectively, and when the user uses one screen window of the mobile terminal to watch multimedia content, if the user wants to use another screen of the mobile terminal to show the display content of the current screen window, the user can only operate application programs on other screen windows to show the multimedia content again, and there is no method capable of projecting the content displayed on the current screen window to other screen windows of the same terminal.

Based on the above problems, the present invention provides a screen projection method applying the DLNA screen projection technology to different screen windows or different display screens of the same mobile terminal, and the screen projection method between different screen windows and the screen projection method between different display screens of the same mobile terminal are respectively introduced as follows:

screen projection method between different screen windows

As shown in fig. 4, a first screen projection method provided for the embodiment of the present invention includes the following steps:

step S401, after receiving a screen projection instruction triggered in a source screen window by a user, a mobile terminal determines a target screen window except the source screen window from at least two screen windows displayed on a display screen of the mobile terminal;

step S402, the mobile terminal projects the multimedia content currently displayed in the source screen window to the target screen window.

The screen projecting method provided by the embodiment of the invention responds to a screen projecting instruction triggered by a user in a source screen window, the mobile terminal determines a target screen window except for the screen window from at least two corresponding screen windows of the mobile terminal and projects the multimedia content currently displayed by the source screen window to the target screen window, because the invention can determine other screen windows except for the source screen window of the mobile terminal as the target screen window and can take other screen windows of the mobile terminal as the screen window for receiving screen projection, after projecting the multimedia content currently displayed by the source screen window to the target screen window, the screen projecting function between different screen windows of the same mobile terminal is realized, the user can continue to use the source screen window to perform other operations after screen projection, and simultaneously can continue to watch the projected multimedia content in the target screen window, a method for flexibly using a plurality of screen windows in the mobile terminal is provided for a user, and user experience is improved.

In the embodiment of the invention, different screen windows can be a full screen window and a floating small window under a picture-in-picture mode of the mobile terminal, or split screen windows in a screen under a split screen mode;

it should be noted that different screen windows of the mobile terminal may be different screen windows located on the same display screen of the mobile terminal, or may also be screen windows on different display screens of the mobile terminal.

The following describes screen windows in a picture-in-picture mode and a split screen mode of a mobile terminal according to an embodiment of the present invention with reference to the accompanying drawings:

1. split screen window of mobile terminal in split screen mode

The mobile terminal with the screen splitting function can divide a display screen into two or more screen splitting windows, and a user can operate in the two or more screen splitting windows after screen splitting; for example, as shown in the screen splitting schematic diagram of the mobile terminal shown in fig. 5, after the screen of the mobile terminal is split, the screen is divided into a first split screen window and a second split screen window, and both the first split screen window and the second split screen window support normal operation and display; both the first split-screen window and the second split-screen window can be used as screen windows in the embodiment of the invention.

2. Full screen window and floating small window under picture-in-picture mode of mobile terminal

The picture-in-picture mode of the mobile terminal is that one or more floating small windows can be displayed on a full-screen window of the mobile terminal, the floating small windows cover a part of the full-screen window, a user can check or operate an application program in the floating small windows, and meanwhile, the user can operate the area of the full-screen window which is not covered by the floating small windows. For example, as shown in fig. 6, it is a schematic diagram of a picture-in-picture mode of a mobile terminal, including a full-screen window and a floating small window located above the full-screen window; both a full screen window and a floating small window can be used as the screen window in the embodiment of the invention.

According to the introduction of the DLNA technology, it can be known that in the embodiment of the present invention, multimedia content in a source screen window of a mobile terminal needs to be projected to other screen windows of the same mobile terminal, and how to use other screen windows of the same terminal as screen windows capable of receiving screen projection is critical.

It should be noted that, in the embodiment of the present invention, the DLNA application may be a DLNA application program on the mobile terminal, or may be a DLNA function module on the mobile terminal.

In response to an instruction of projecting the multimedia content currently displayed in a source screen window triggered by a user in the source screen window, the mobile terminal determines a target screen window except the source screen window from at least two corresponding screen windows;

the multimedia content currently displayed by the source screen window can be multimedia content in a third-party video playing application program supporting DLNA screen projection, such as video in the video playing application program; content presented in other applications that support DLNA screen projection, such as text content in a viewer application, may also be presented.

An optional implementation manner is that the user triggers a command for projecting the multimedia content currently displayed in the source window by clicking a screen projection button in the third-party application, for example, as shown in fig. 7, the user clicks a button corresponding to screen projection in the third-party application to trigger the screen projection command.

After a screen projection instruction triggered by a user is monitored, scanning available screen windows by a DLNA application corresponding to a third-party application program, and determining an available screen window set; and the screen windows in the available screen window set are other screen windows of the mobile terminal except for the source screen window.

It should be noted that the available screen window set may further include screen windows of other scanned available devices, for example, screen windows of a television receiving screen projection and a tablet device, where the screen windows of the other available devices may belong to the same available screen window set as other screen windows of the mobile terminal itself except for the source screen window, or may respectively belong to different available screen window sets.

In an alternative embodiment, the set of available screen windows is determined according to the following:

the mobile terminal scans the DLNA application and generates an available screen window set according to the screen window corresponding to the scanned DLNA application of the mobile terminal;

specifically, the mobile terminal acquires configuration information corresponding to the scanned DLNA application; aiming at any one scanned DLNA application, judging whether the DLNA application is the DLAN application of the mobile terminal according to the configuration information corresponding to the DLNA application; judging whether the DLAN application is matched with the multimedia content or not according to the configuration information corresponding to the DLNA application; and forming an available screen window set by screen windows corresponding to the DLAN application matched with the multimedia content in the DLAN applications of the scanned mobile terminal.

In an optional implementation manner, the configuration information corresponding to the DLNA application includes device information of a screen window corresponding to the DLNA application and service information of the screen window corresponding to the DLNA application, where the device information includes resolution of the screen window corresponding to the DLNA application, an IP address of a mobile terminal to which the DLNA application belongs, interface information of a DLNA screen projection, and the like; the service information includes services that can be provided by a screen window corresponding to the DLNA application, for example, service information supporting video playback, audio playback, and the like.

In specific implementation, the mobile terminal may obtain the scanned configuration information corresponding to the DLNA application according to the following manner:

mode 1, the mobile terminal actively acquires the configuration information corresponding to the DLNA application

And the mobile terminal sends the search request information through the UDP port, and after receiving the search request information, the DLNA application returns response information containing the configuration information corresponding to the DLNA application to the mobile terminal.

Mode 2, the mobile terminal passively acquires the configuration information corresponding to the DLNA application

The DLNA application periodically broadcasts the configuration information corresponding to the DLNA application, and the mobile terminal receives the configuration information broadcasted by the DLNA application and passively acquires the configuration information corresponding to the DLNA application.

After the mobile terminal acquires the configuration information corresponding to the DLNA application, judging whether a screen window corresponding to the DLNA application is an available screen window or not according to the configuration information corresponding to the DLNA application;

specifically, the mobile terminal judges whether the screen window corresponding to the DLNA application is another screen window of the mobile terminal according to the IP address included in the device information in the configuration information corresponding to the DLNA application, and determines that the screen window corresponding to the DLNA application is another screen window of the mobile terminal when the IP address is consistent with the IP address of the mobile terminal;

the mobile terminal judges whether the DLNA application is matched with the multimedia content according to the equipment information and the service information in the configuration information corresponding to the DLNA application, for example, whether the service which can be provided by the DLNA application meets the requirement of screen projection at this time is judged according to the resolution and the service information in the configuration information corresponding to the DLNA application, the multimedia content of the screen projection at this time is assumed to be video and the resolution is 1080p, if the resolution in the configuration information corresponding to the DLNA application is 1080p and video screen projection is supported, the screen window corresponding to the DLNA application is determined to be capable of providing the service required by screen projection, and the DLAN application is matched with the multimedia content.

And forming an available screen window set by screen windows corresponding to the DLAN application matched with the multimedia content in the DLAN applications of the scanned mobile terminal.

And displaying the scanned available screen window set in the source screen window, and taking the screen window selected by the user as a target screen window in response to an instruction of the user for selecting the screen window from the available screen window set. For example, as shown in fig. 8, a list of available screen windows corresponding to the set of available screen windows is presented in the source screen window, from which the user selects the screen window as the target screen window.

And the mobile terminal transmits the multimedia content to the DLNA application corresponding to the target screen window through the DLNA application corresponding to the source screen window.

In an optional implementation manner, the DLNA application corresponding to the source screen window transmits multimedia data corresponding to the multimedia content to the digital media server DMS, and the digital media server DMS transmits the multimedia data to the DLNA application corresponding to the target screen window.

And displaying the DLNA application corresponding to the target screen window in the target screen window according to the received multimedia content.

For example, as shown in fig. 9, taking a first split-screen window and a second split-screen window in a split-screen mode as an example, the first split-screen window is playing a video, responding to a screen-casting instruction triggered by a user at the first split-screen window, selecting to cast the video currently being played in the first split-screen window to the second split-screen window, playing the video in the second split-screen window after screen casting as shown in fig. 10, after screen casting, the first split-screen window may display a home page, and the user may perform an operation by touching an application icon in the first split-screen window.

As shown in fig. 11, a flowchart of a complete screen projection method between different screen windows according to an embodiment of the present invention includes the following steps:

step S1101, responding to an instruction of a user to screen-cast the multimedia content currently displayed in a source screen window, wherein the instruction is triggered in the source screen window by the user, and the mobile terminal scans available screen windows to acquire configuration information corresponding to the scanned DLNA application;

step S1102, the mobile terminal makes screen windows corresponding to DLAN applications matched with multimedia contents in the scanned DLAN applications of the mobile terminal form an available screen window set;

step S1103, responding to an instruction of a user for selecting a screen window from the available screen window set, and taking the screen window selected by the user as a target screen window;

step S1104, the mobile terminal transmits the multimedia content currently displayed on the source screen window to the DLNA application corresponding to the target screen window;

and step S1105, the DLNA application corresponding to the target screen window displays in the target screen window according to the received multimedia content.

Screen projection method between different display screens of same mobile terminal

As shown in fig. 12, a screen projection method between different display screens of the same mobile terminal according to an embodiment of the present invention is applied to a mobile terminal including two display screens, and includes the following steps:

step S1201, the mobile terminal receives a screen projection instruction triggered by a user through a currently used display screen;

and step S1202, the mobile terminal casts the multimedia content displayed in the currently used display screen to another display screen.

In the embodiment of the invention, different display screens of the same mobile terminal can be a main screen and a back screen of a double-screen terminal, or a first display screen and a second display screen of a folding screen terminal; the following describes a main screen and a back screen of a dual-screen terminal and a first display screen and a second display screen of a folding screen terminal respectively in an embodiment of the present invention with reference to the accompanying drawings:

1. main screen and back screen of double-screen terminal

The dual-screen terminal is a mobile terminal with two display screens, the two display screens are a main screen and a back screen of the mobile terminal respectively, and a user can operate or view pages on the main screen and the back screen respectively; for example, as shown in fig. 13, a mobile terminal having a main screen and a back screen may be used as the display screen in the embodiment of the present invention.

2. First display screen and second display screen of folding screen terminal

The folding screen terminal is a mobile terminal having a flexible screen capable of being folded by 360 degrees, the folding screen terminal can use two folded display screens respectively, or can use a complete display screen after being unfolded, for example, when a user uses the folding screen terminal, the user can use a first display screen and a second display screen shown in fig. 14a respectively, or can use a complete display screen shown in fig. 14b, and both the first display screen and the second display screen can be used as display screens in the embodiment of the present invention.

For convenience of description, a display currently used by a user is referred to as a source display and another display projected to is referred to as a target display in the following description.

The embodiment of the invention installs a DLNA application in the mobile terminal in advance, and binds the DLNA application with other display screens in the mobile terminal except for the source display screen, so that the other display screens have corresponding DLNA applications, and the other display screens can be used as a virtual device in DLNA screen projection and added into a local area network, so that the other display screens of the same mobile terminal can be used as target display screens capable of receiving screen projection.

In response to an instruction of projecting the multimedia content currently displayed on a source display screen triggered by a user in the source display screen, the mobile terminal determines a target display screen except the source display screen from two display screens corresponding to the mobile terminal;

the multimedia content currently displayed by the source display screen can be multimedia content in a third-party video playing application program supporting DLNA screen projection, such as video in the video playing application program; content presented in other applications that support DLNA screen projection, such as text content in a viewer application, may also be presented.

In an alternative embodiment, the user triggers an instruction to screen-cast the multimedia content currently displayed on the source display screen by clicking a screen-casting button in the third-party application program.

After a screen projection instruction triggered by a user is monitored, scanning an available display screen by a DLNA application corresponding to a third-party application program to determine the available display screen;

in an alternative embodiment, the available display is determined according to the following:

the mobile terminal scans the DLNA application and takes a display screen corresponding to the scanned DLNA application of the mobile terminal as an available display screen;

specifically, the mobile terminal acquires configuration information corresponding to the scanned DLNA application; aiming at any one scanned DLNA application, judging whether the DLNA application is the DLAN application of the mobile terminal according to the configuration information corresponding to the DLNA application; judging whether the DLAN application is matched with the multimedia content or not according to the configuration information corresponding to the DLNA application; and taking the display screen corresponding to the DLAN application matched with the multimedia content in the DLAN applications of the scanned mobile terminal as an available display screen.

In an optional implementation manner, the configuration information corresponding to the DLNA application includes device information of a display screen corresponding to the DLNA application and service information of the display screen corresponding to the DLNA application, where the device information includes resolution of the display screen corresponding to the DLNA application, an IP address of a mobile terminal to which the DLNA application belongs, interface information of a DLNA screen projection, and the like; the service information includes services that can be provided by a display screen corresponding to the DLNA application, for example, service information supporting video playing, audio playing, and the like.

After the mobile terminal acquires the configuration information corresponding to the DLNA application, judging whether a display screen corresponding to the DLNA application is an available display screen according to the configuration information corresponding to the DLNA application;

specifically, the mobile terminal judges whether the display screen corresponding to the DLNA application is another display screen of the mobile terminal according to the IP address included in the device information in the configuration information corresponding to the DLNA application, and determines that the display screen corresponding to the DLNA application is another display screen of the mobile terminal when the IP address is consistent with the IP address of the mobile terminal;

the mobile terminal judges whether the DLNA application is matched with the multimedia content according to the equipment information and the service information in the configuration information corresponding to the DLNA application, for example, whether the service which can be provided by the DLNA application meets the requirement of screen projection at this time is judged according to the resolution and the service information in the configuration information corresponding to the DLNA application, the multimedia content of the screen projection at this time is assumed to be video and the resolution is 1080p, if the resolution in the configuration information corresponding to the DLNA application is 1080p and video screen projection is supported, the display screen corresponding to the DLNA application is determined to be capable of providing the service required by screen projection, and the DLAN application is matched with the multimedia content.

And taking the display screen corresponding to the DLAN application matched with the multimedia content in the DLAN applications of the scanned mobile terminal as an available display screen.

The scanned available display screen is displayed in the source display screen, and in response to an instruction of selecting the available display screen by the user, the available display screen is used as the target display screen, for example, as shown in fig. 15, in the dual-screen terminal, the available display screen is displayed as a back screen in the source display screen, and the user selects the back screen as the target display screen.

And the mobile terminal transmits the multimedia content to the DLNA application corresponding to the target display screen through the DLNA application corresponding to the source display screen.

In an optional implementation manner, the DLNA application corresponding to the source display screen transmits multimedia data corresponding to the multimedia content to the digital media server DMS, and the digital media server DMS transmits the multimedia data to the DLNA application corresponding to the target display screen.

And displaying the DLNA application corresponding to the target display screen in the target display screen according to the received multimedia content.

For example, taking a folding screen as an example, when a video is played in one display screen of the folding screens shown in fig. 16, a user triggers a screen projection instruction to project the video displayed in the current display screen to the other display screen, as shown in fig. 17, after the screen projection, the current display screen may display a home page, and the user may touch an application image in the current display screen to trigger an operation.

It should be noted that the screen projecting method between different screen windows provided in the embodiment of the present invention may also be applied to a mobile terminal having multiple display screens, and after a user triggers a screen projecting instruction, the mobile terminal having multiple display screens may also use a split screen window or a floating small window on a current display screen or other display screens of the same mobile terminal as an available window, that is, in the screen projecting method provided in the embodiment of the present invention, any combination scheme belongs to the protection scope of the present invention.

As shown in fig. 18, a flowchart of a complete screen projection method between different display screens of the same mobile terminal according to an embodiment of the present invention includes the following steps:

step S1801, responding to a screen projection instruction triggered by a user in a source display screen, and scanning an available display screen by the mobile terminal to acquire configuration information corresponding to scanned DLNA application;

step S1802, the mobile terminal takes a display screen corresponding to the DLAN application matched with the multimedia content in the scanned DLAN application of the mobile terminal as an available display screen;

step S1803, responding to an instruction of selecting an available display screen by a user, and taking the available display screen as a target display screen;

step S1804, the mobile terminal transmits the multimedia content displayed by the source display screen to the DLNA application corresponding to the target display screen;

and step S1805, displaying the DLNA application corresponding to the target display screen in the target display screen according to the received multimedia content.

Based on the same inventive concept, the embodiment of the present invention further provides a mobile terminal, and as the principle of solving the problem of the mobile terminal is similar to the screen projection method between different screen windows of the embodiment of the present invention, the implementation of the mobile terminal may refer to the implementation of the method, and repeated parts are not described again.

As shown in fig. 19, an embodiment of the present invention provides a mobile terminal including a display 1901 and a processor 1902.

The display screen 1901 is configured to display at least two screen windows, and detect a screen-projection instruction triggered in a screen window by a user;

the processor 1902 is configured to determine, after receiving a screen-projection instruction triggered by a user in a source screen window, a target screen window other than the source screen window from the at least two screen windows displayed by the display screen 1901; and projecting the multimedia content currently displayed by the source screen window to the target screen window.

In some exemplary embodiments, the processor 1902 is specifically configured to:

In some exemplary embodiments, there is one DLNA application per screen window;

the processor 1902 is specifically configured to:

As shown in fig. 20, the first screen projection apparatus according to the embodiment of the present invention includes:

a determining module 2001, configured to determine, after receiving a screen-projecting instruction triggered by a user in a source screen window, a target screen window other than the source screen window from at least two screen windows displayed by the display screen;

a first screen projection module 2002 configured to project the multimedia content currently displayed by the source screen window to the target screen window.

In some exemplary embodiments, the determination module 2001 is specifically configured to:

In some exemplary embodiments, there is one DLNA application per screen window;

the determination module 2001 is specifically configured to:

In some exemplary embodiments, the first screen projection module 2002 is specifically configured to:

Based on the same inventive concept, the embodiment of the present invention further provides a mobile terminal, and as the principle of solving the problem of the mobile terminal is similar to the screen projection method between different display screens of the same mobile terminal in the embodiment of the present invention, the implementation of the mobile terminal may refer to the implementation of the method, and repeated details are not repeated.

As shown in fig. 21, an embodiment of the present invention provides a mobile terminal, which includes a processor 2101, a display 2102 and a display 2103;

the display 2102 and the display 2103 are configured to detect a user-triggered screen-projection instruction;

the processor 2101 is further configured to:

As shown in fig. 22, the second screen projection apparatus according to the embodiment of the present invention includes:

a receiving module 2201 configured to receive a screen projection instruction triggered by a user through a currently used display screen;

a second screen projection module 2202 configured to project the multimedia content displayed in the currently used display screen to another display screen.

the second screen projection module 2202 is further configured to:

Since the mobile terminal and the computer storage medium in the embodiment of the present invention may be applied to the processing method, reference may also be made to the method embodiment for obtaining technical effects, and details of the embodiment of the present invention are not described herein again.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims

1. A mobile terminal, comprising a display and a processor:

2. The mobile terminal of claim 1, wherein the processor is configured to:

3. The mobile terminal of claim 2, wherein each screen window corresponds to one DLNA application;

the treatment appliance is configured to:

4. The mobile terminal of claim 3, wherein the processor is configured to:

5. The mobile terminal of claim 3, wherein the processor is configured to:

6. A mobile terminal is characterized by comprising a processor and two display screens;

7. The mobile terminal of claim 1, wherein each display screen corresponds to a DLNA application;

the processor is further configured to:

8. A method of projecting a screen, the method comprising:

9. A screen projection method is applied to a mobile terminal comprising two display screens, and comprises the following steps:

10. A computer storage medium having computer program instructions stored therein which, when run on a computer, cause the computer to perform the method of claim 8 or 9.