CN117992004A - Electronic equipment and message interaction method based on spaced delivery - Google Patents

Abstract

The application provides an electronic device and a message interaction method based on spaced delivery, which mainly relate to the technical field of computers, and the electronic device comprises: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs a space-free delivery system service and a space-free delivery daemon, the space-free delivery system service is positioned in a system partition in an application layer in a software system architecture, and the space-free delivery process is positioned in a developer partition in the application layer in the software system architecture; establishing a first socket connection between the space-based delivery system service and a space-based delivery daemon; an air drop daemon configured to: receiving a control signal of media content and sending the control signal to an air-spaced drop system service; an air space delivery system service configured to: and controlling the media content put in the space based on the control signal. The message communication among the modules is completed through socket connection, so that the interface implementation mode is more universal, the implementation is simpler, and the occupied memory is smaller.

Description

Electronic equipment and message interaction method based on spaced delivery

Technical Field

The embodiment of the application relates to the technical field of computers. And more particularly, to an electronic device and a method of message interaction based on spaced apart delivery.

Background

The air gap play (AirPlay) is a play technology added in iOS4.2 and OS X Mountain Lion, and may transfer iOS and OS X Mountain Lion or updated files (including video, photos, video and mirror images) to electronic devices supporting air gap play for play. The electronic device wants to support the alternate play requires integrating an AirPlay software development kit (software development kit, SDK) within the software system of the electronic system, the core protocol stack portion of which is an application that can run on a LINUX system compatible with the portable operating system interface (Portable Operating SYSTEM INTERFACE, POSIX).

The Daemon (Daemon) protocol stack in AIRPLAY SDK and the AirPlay native application (Native Application) must be located in a developer (Vendor) partition because of the need to interact with the trusted execution environment (trusted execution environment, TEE) portion. Whereas PAL programs need to be located in a System (System) partition because of the need to interface with the Framework standard interface. Because of isolation between the Vendor partition and the System partition, android requires communication by using a hardware abstraction layer interface definition language (HAL INTERFACE definition language, HIDL) interface, and the scheme is complex, needs more interfaces to be realized, and occupies high memory.

Disclosure of Invention

The application provides an electronic device and a message interaction method based on spaced delivery, and the message communication among modules is completed through socket connection.

The technical scheme provided by the embodiment of the application is as follows:

in a first aspect, an embodiment of the present application provides an electronic device, including: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs a space-air drop system service and a space-air drop daemon, and the space-air drop system service is positioned in a system partition in an application layer in a software system architecture of the electronic equipment; the space-air drop process is positioned in a developer partition in an application layer in the software system architecture; a first socket connection is established between the space-division drop system service and the space-division drop daemon; a display screen configured to: playing media pictures put in at intervals; an audio player configured to: playing media audio put in a spaced mode; an air drop daemon configured to: receiving a control signal of media content; transmitting a control signal to the space-air drop system service through a first socket connection; an air space delivery system service configured to: and controlling media content put in the space based on the control signal, wherein the media content comprises a media picture and/or a media sound.

In a second aspect, an embodiment of the present application provides a method for message interaction based on spaced apart delivery, where the method is applied to an electronic device, and the electronic device includes: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs a space-air drop system service and a space-air drop daemon, and the space-air drop system service is positioned in a system partition in an application layer in a software system architecture of the electronic equipment; the space-air drop process is positioned in a developer partition in an application layer in the software system architecture; establishing a first socket connection between the space-based delivery system service and a space-based delivery daemon; the method comprises the steps of; receiving a control signal of media content through an air separation drop daemon, and sending the control signal to an air separation drop system service through a first socket connection; controlling media content put in a space based on a control signal through a space-based put system service, wherein the media content comprises a media picture and/or a media sound; controlling media content put in a space based on the control signal through a space-free put system service, wherein the media content comprises a media picture and/or a media sound; playing the media pictures put in at intervals through a display screen; and/or the audio player plays the media audio put at intervals.

In a third aspect, embodiments of the present application provide a computer readable storage medium, on which a computer program is stored, which when executed by a computing device, causes the computing device to implement a method for message interaction based on spaced apart delivery provided by any embodiment of the first or second aspects.

In a fourth aspect, embodiments of the present application provide a computer program product, which when run on a computer causes the computer to implement the method of spaced apart delivery based message interaction provided by any of the embodiments of the first or second aspects.

As can be seen from the above technical solutions, the embodiment of the present application provides an electronic device and a method for interaction of messages based on spaced delivery, where the electronic device includes: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs a space-free delivery system service and a space-free delivery daemon, the space-free delivery system service is positioned in a system partition in an application layer in a software system architecture of the electronic equipment, and the space-free delivery process is positioned in a developer partition in the application layer in the software system architecture; establishing a first socket connection between the space-based delivery system service and a space-based delivery daemon; a display screen configured to: playing media pictures put in at intervals; an audio player configured to: playing media audio put in a spaced mode; an air drop daemon configured to: receiving a control signal of media content; transmitting a control signal to the space-air drop system service through a first socket connection; an air space delivery system service configured to: and controlling media content put in the space based on the control signal, wherein the media content comprises a media picture and/or a media sound. Because the space-air drop system service is connected with the space-air drop daemon through the socket, the message communication among the modules is completed by sending the json format message character strings, and the interface implementation mode is more universal, simpler to implement and smaller in occupied memory.

Drawings

In order to more clearly illustrate the embodiments of the present application or the implementation of the related art, the drawings that are required for the embodiments or the related art description will be briefly described, and it is apparent that the drawings in the following description are some embodiments of the present application and that other drawings may be obtained according to these drawings for a person having ordinary skill in the art.

FIG. 1 is a schematic diagram of a AIRPLAY SDK architecture;

Fig. 2 is a scene structure diagram of a message interaction method based on spaced delivery according to an embodiment of the present application;

Fig. 3 is a schematic structural diagram of a message interaction device based on spaced delivery according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 5 illustrates a software configuration diagram in a display device 200 according to some embodiments;

Fig. 6 is a schematic diagram of a software system architecture of an electronic device according to an embodiment of the present application;

fig. 7 is a schematic diagram of a software system architecture of another electronic device according to an embodiment of the present application;

Fig. 8 is a schematic diagram of a software system architecture of still another electronic device according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a software system architecture of another electronic device according to an embodiment of the present application;

fig. 10 is a data flow chart of audio push provided in an embodiment of the present application;

FIG. 11 is a data flow chart of video push provided in an embodiment of the present application;

FIG. 12 is a data flow diagram of mirror push provided by an embodiment of the present application;

FIG. 13 is a diagram of a space-free drop input selection interface provided by an embodiment of the present application;

FIG. 14 is a diagram of a network inspection prompt interface provided by an embodiment of the present application;

FIG. 15 is an interface diagram of a setup network according to an embodiment of the present application;

FIG. 16 is a view of a barrier interface for air drop provided by an embodiment of the present application;

FIG. 17 is an interface diagram of a space-free drop daemon setup provided by an embodiment of the present application;

Fig. 18 is a flow chart of a message interaction method based on spaced delivery according to an embodiment of the present application.

Detailed Description

For the purposes of making the objects and embodiments of the present application more apparent, an exemplary embodiment of the present application will be described in detail below with reference to the accompanying drawings in which exemplary embodiments of the present application are illustrated, it being apparent that the exemplary embodiments described are only some, but not all, of the embodiments of the present application.

It should be noted that the brief description of the terminology in the present application is for the purpose of facilitating understanding of the embodiments described below only and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

AirPlay is a playback technique incorporated in iOS4.2 and OS X Mountain Lion, and can transfer iOS and OS X Mountain Lion or updated versions of files (including video, photos, videos, and images) to electronic devices supporting spaced playback for playback. The electronic device wants to support the alternate play requires integration AIRPLAY SDK within the software system of the electronic system, the core protocol stack portion of the SDK being an application that can run on the LINUX system of POSIX.

The software system architecture of the current electronic device generally adopts an Android (Android) + Trebel architecture, and the architecture separates hardware-specific driving software (Vendor Implementation) written by a chip manufacturer from an Android Framework layer (Framework) for more convenient updating of Android system versions.

FIG. 1 is a schematic diagram of an architecture of AIRPLAY SDK and a host operating system; as shown in fig. 1, the AIRPLAY SDK architecture mainly includes a trusted execution environment (trusted execution environment, TEE) and a rich execution environment (rich execution environment, REE), wherein the TEE is a secure executable environment and isolated from other software running in the main operating system. A TEE is typically a small, stand-alone operating system that runs on special hardware protection provided by a System On Chip (SOC). Wherein AIRPLAY SDK is deployed in the TEE. The REE is a main operating system of the electronic equipment and runs outside the TEE, wherein the Android system in the scheme is deployed in the REE.

AirPlay and Smart home platform eidolon virtual drive (Homkit Daemon), airPlay native application (Native Application), and multicast Domain name resolution protocol (mDNS) are included in AIRPLAY SDK.

In the AirPlay & Homkit Daemon, AIRPLAY NATIVE procedure, an AirPlay service is published in the network, the client connection is processed and the AirPlay protocol stack message is forwarded to AIRPLAY NATIVE Application, which is responsible for processing Audio buffer (Audio buffer) data, etc. Control the lifecycle of the AirPlay application through a PAL application programming interface (Application Programming Interface, API) (API), such as: start, stop, go back to the background, etc.

AIRPLAY NATIVE Application is responsible for Video (Video) playback, spaced-apart drop mirror (AirPlay Mirroring) and upper User Interface (UI) rendering based on the HTTP-based streaming media transport protocol (HTTP LIVE STREAMING, HLS) uniform resource location system (uniform resource locator, URL). AIRPLAY NATIVE Application consists essentially of AirPlay JavaScript App and APPLE NATIVE APP. APPLE NATIVE APP includes Luna libraries and physical abstraction layer (platform abstraction layer, PAL) layer implementations of Luna libraries. AIRPLAY APP is controlled by AirPlay & Homkit Daemon.

The Luna library is a node. Js environment based javascript runtime environment and open graphics library (Open Graphics Library, openGL) drawing engine. The Luna library comprises a JavaScript application running environment independent of a platform, mainly runs developed JavaScript programs, and realizes unified AirPlay UI and function realization on multiple platforms through various services and applications in the Luna library.

MDNS is a device discovery protocol based on IP/TCP networks.

PAL layer, platform layer implementation for AIRPLAY SDK interfaces.

The PTP protocol defines a method for achieving synchronization between network devices by transmitting time synchronization messages over the network. This protocol can achieve sub-microsecond time synchronization accuracy.

The TEE includes: MFI Trusted Application, SHARED KEY Store, fairPlay Trusted Application and clock synchronization protocol (Precision Time Protocol, PTP).

Wherein MFI Trusted Application is responsible for managing access to the keying material, handling authorization verification from AirPlay Daemon. SHARED KEY Store is responsible for securely storing the pairing Key, sharing keys with AirPlay Daemon at HomeKit. The paired keys must be stored through a secure storage environment provided by the TEE. FairPlay Trusted Application is responsible for content key agreement, secure playback of video. From the above structure, the Daemon protocol stack and AIRPLAY NATIVE Application in AIRPLAY SDK must be located in a developer (Vendor) partition because of the need to interact with the TEE part. Whereas PAL programs need to be located in a System (System) partition because of the need to interface with the Framework standard interface. Because of isolation between the Vendor partition and the System partition, android requires communication by using a hardware abstraction layer interface definition language (HAL INTERFACE definition language, HIDL) interface, and the scheme is complex, needs more interfaces to be realized, and occupies high memory.

The protocol stack procedure in AIRPLAY SDK requires the use of a PTP protocol that listens to UDP ports 319 and 320 during use. In the Android System, only the user group with the user identification (User Identification, UID) smaller than 10000 has the authority to monitor the ports, such as the super user authority (ROOT) user group (uid=0) or the System (System) user group (uid=1000), and since the UID is smaller than 10000, the user group has the authority to monitor the ports 319 and 320. The Android system will prohibit the system user group process (UID greater than 10000) from listening to any TCP/UDP port. But the user rights group is very dangerous in the system and can cause serious system security problems if misused or hijacked by a hacker. Therefore, the Android system does not allow the processes of the external open port to have user group permission, and test items in the Android system compatibility test (compatibility test suite, CTS) can scan all processes in the electronic equipment to check the security of the user group permission. Therefore, the existing AIRPLAY SDK architecture cannot pass the CTS test and cannot meet the authentication requirement of the Android system.

In order to solve the technical problems, the embodiment of the application provides electronic equipment and a message interaction method based on spaced delivery, and the technical scheme of the application is described in detail below with reference to the drawings and the embodiment.

Fig. 2 is a schematic diagram of an operation scenario between an electronic device and a control device in an embodiment. As shown in fig. 2, a user may operate the electronic device 200 through the smart device 300 or the control apparatus 100.

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the electronic device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the electronic device 200 is controlled by a wireless or wired mode. The user may control the electronic device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc.

In some embodiments, the smart device 300 (e.g., mobile terminal, tablet, computer, notebook, etc.) may also be used to control the electronic device 200. For example, the electronic device 200 is controlled using an application running on a smart device.

In some embodiments, the electronic device may not receive instructions using the smart device or control device described above, but rather receive control of the user by touch or gesture, or the like.

In some embodiments, the electronic device 200 may further perform control in a manner other than the control apparatus 100 and the smart device 300, for example, the module configured inside the electronic device 200 for obtaining the voice command may directly receive the voice command control of the user, or the voice command control of the user may also be received through a voice control device set outside the electronic device 200.

In some embodiments, the electronic device 200 is also in data communication with a server 400. Electronic device 200 may be permitted to communicate over a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various content and interactions to the electronic device 200. The server 400 may be a cluster, or may be multiple clusters, and may include one or more types of servers.

Fig. 3 exemplarily shows a block diagram of a configuration of the control apparatus 100 in accordance with an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control device 100 may receive an input operation instruction from a user, and convert the operation instruction into an instruction recognizable and responsive to the electronic apparatus 200, and may perform an interaction between the user and the electronic apparatus 200.

As shown in fig. 4, the electronic apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

The display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal from the controller output, displaying video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

The display 260 may be a liquid crystal display, an OLED display, a projection device, or a projection screen.

The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The electronic apparatus 200 may establish transmission and reception of control signals and data signals with the external control apparatus 100 or the server 400 through the communicator 220.

A user interface, which may be used to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; either the detector 230 comprises an image collector, such as a camera, which may be used to collect external environmental scenes, user attributes or user interaction gestures, or the detector 230 comprises a sound collector, such as a microphone or the like, for receiving external sounds.

The external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, or the like. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

The modem 210 receives broadcast television signals through a wired or wireless reception manner, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

The controller 250 controls the operation of the electronic device and responds to the user's operations by various software control programs stored on the memory. The controller 250 controls the overall operation of the electronic device 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments the controller includes at least one of a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), first to nth interfaces for input/output, a communication Bus (Bus), and the like.

The user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Or the user may input the user command by inputting a specific sound or gesture, the user input interface recognizes the sound or gesture through the sensor, and receives the user input command.

A "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user, which enables conversion between an internal form of information and a user-acceptable form. A commonly used presentation form of a user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a graphically displayed user interface that is related to computer operations. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

Referring to FIG. 5, in some embodiments, the system is divided into four layers, from top to bottom, an application layer (referred to as an "application layer"), an application framework layer (Application Framework) layer (referred to as a "framework layer"), a An Zhuoyun row layer (Android runtime) and a system library layer (referred to as a "system runtime layer"), and a kernel layer, respectively.

In some embodiments, at least one application program is running in the application program layer, and these application programs may be a Window (Window) program of an operating system, a system setting program, a clock program, or the like; or may be an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an application programming interface (application programming interface, API) and programming framework for the application. The application framework layer includes a number of predefined functions. The application framework layer corresponds to a processing center that decides to let the applications in the application layer act. Through the API interface, the application program can access the resources in the system and acquire the services of the system in the execution.

As shown in fig. 5, the application framework layer in the embodiment of the present application includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an activity manager (ACTIVITY MANAGER) is used to interact with all activities running in the system; a Location Manager (Location Manager) is used to provide system services or applications with access to system Location services; a package manager (PACKAGE MANAGER) for retrieving various information about the application packages currently installed on the device; a notification manager (Notification Manager) for controlling the display and clearing of notification messages; a Window Manager (Window Manager) is used to manage bracketing icons, windows, toolbars, wallpaper, and desktop components on the user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the individual applications as well as the usual navigation rollback functions, such as controlling the exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of the display screen, judging whether a status bar exists or not, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window to display, dithering display, distorting display, etc.), etc.

In some embodiments, the system runtime layer provides support for the upper layer, the framework layer, and when the framework layer is in use, the android operating system runs the C/C++ libraries contained in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 5, the kernel layer contains at least one of the following drivers: audio drive, display drive, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (e.g., fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive, etc.

An embodiment of the present application provides an electronic device, including: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs a space-free delivery system service and a space-free delivery daemon, the space-free delivery system service is positioned in a system partition in an application layer in a software system architecture of the electronic equipment, and the space-free delivery process is positioned in a developer partition in the application layer in the software system architecture; establishing a first socket connection between the space-based delivery system service and a space-based delivery daemon; a display screen configured to: playing media pictures put in at intervals; an audio player configured to: playing media audio put in a spaced mode; an air drop daemon configured to: receiving a control signal of media content; transmitting a control signal to the space-air drop system service through a first socket connection; an air space delivery system service configured to: and controlling media content put in the space based on the control signal, wherein the media content comprises a media picture and/or a media sound.

As shown in fig. 6, an embodiment of the present application provides a mechanism schematic of a software system architecture of an electronic device; as shown in fig. 6, the software system architecture of the electronic device mainly includes a system (system) partition of an application layer and a developer (vendor) partition of the application layer, wherein a space-division drop system service (AIRPLAYSYSTEMSERVICE) is deployed in the system partition, and a space-division drop daemon (AirPlayDaemon) is deployed in the vendor partition. The space-air drop system service is connected with the space-air drop daemon through a first socket.

Wherein the socket connection is established over the TCP protocol. socket connections are generally divided into two types: short and long connections. Short connection means that the two communication parties establish connection and immediately disconnect after completing data transmission, namely, once connection is established and disconnected after each transmission is completed, and the communication process is similar to call. The long connection means that after one connection is established, data transmission can be performed for a plurality of times until the data transmission is completed or one party is disconnected. In socket connection, the efficiency of long connection is higher than that of short connection, because the short connection needs to be connected and disconnected every time data is transmitted, and the long connection can be connected in a multiplexing mode, so that the time for establishing and disconnecting connection is reduced, and the efficiency is improved. The selection of long connection and short connection should be performed according to the service needs, and if the service needs to frequently transmit a large amount of data, the long connection should be selected; if the traffic needs to transmit a small amount of data or needs to make and break connections frequently, a short link should be selected. According to the actual situation, the first socket connection between the air separation drop system service and the air separation drop daemon can be long connection or short connection, and the embodiment of the application is not limited in detail.

The air-separation system service is a non-system process, the assigned user identification (User Identification, UID) is more than 10000, and the corresponding Internet protocol (Internet Protocol, IP) address of the air-separation system service is: 127.0.0.1. the air separation system service monitors a transmission control protocol (Transmission Control Protocol, TCP) port 12345, and is used for receiving a control signal sent by AirPlayDaemon, completing control of media content of air separation according to the control signal, and further completing Android system layer implementation of AirPlay and HomeKit event messages. Further, PAL is deployed in the air space drop system service.

The media content may be audio and/or video. The media content is put into the electronic equipment by the air space throwing equipment, so that the media content on the air space throwing equipment can be played on the electronic equipment. The air-separation throwing device can be a device with an air-separation playing technology, and specifically, an operating system of the air-separation throwing device is ios4.2 and OS X Mountain Lion.

The air space drop daemon corresponds to AirPlay & Homkit Daemon in the SDK, and the control of the life cycle of AIRPLAY APP and the sending of HomeKit event messages are realized by establishing a first socket connection with the air space drop system service to perform message interaction.

In one possible implementation, the space drop daemon is configured to: and receiving a control signal sent by the air-spaced throwing equipment through the set port. The control signal may be understood as a control signal for media content, for example: a signal for adjusting the volume of the media content, a signal for adjusting the playing progress of the media content, an opening signal or a closing signal of spaced-apart delivery, and the like.

The control signal can be input by a user at the air-spaced throwing device. For example: the user can adjust the playing progress of the media content on the mobile phone, generate an adjusting signal of the playing progress, send the adjusting signal to the space-air drop daemon, and send the adjusting signal of the playing progress to the space-air drop system service through the first socket connection after the space-air drop daemon receives the adjusting signal of the playing progress, and the space-air drop system service adjusts the playing progress of the media content based on the adjusting signal of the playing progress.

In another possible implementation manner, the air separation drop daemon receives a life cycle signal sent by the air separation drop device, and sends the life cycle signal to an air separation drop system service through a first socket connection, and the air separation drop system service receives the life cycle signal sent by the air separation drop daemon; the lifecycle of the spaced-apart drop application is controlled based on the lifecycle signal.

The life cycle signals comprise a start signal, a stop signal, a background entering signal and the like of the space-air drop application. After the air separation throwing device performs life cycle related operation, a life cycle signal is generated by a user, the life cycle signal is sent to an air separation throwing daemon, and after the air separation throwing daemon receives the life cycle signal related to an air separation throwing system, the life cycle signal is sent to an air separation throwing system service through a first socket connection, so that the air separation throwing system service controls the life cycle of an air separation throwing application program based on a life cycle control signal.

The mobile phone generates a background entering signal and sends the background entering signal to the air space throwing daemon, and the air space throwing daemon sends the background entering signal to the air space throwing system service through a first socket connection, so that the air space throwing system service controls the air space throwing application program to run in the background based on the background entering signal.

In one possible implementation, the space drop daemon is configured to: monitoring a first TCP port, wherein the first TCP port is a standard protocol port for air-spaced drop; monitoring a first user datagram protocol (User Datagram Protocol, UDP) port, the first UDP port being a multicast DNS device discovery protocol port.

The first TCP port is 7000, and the port is an AirPlay protocol standard port; the first UDP port is 5353, which is an mDNS device discovery protocol standard port.

On the basis of the above embodiment, the embodiment of the present application further optimizes the software system architecture, as shown in fig. 7, where the optimized software system architecture further includes a spaced android application package (Android application package, APK), where the spaced android application package is located in a developer partition in an application layer in the software system architecture; establishing a second socket connection between the spaced throwing APK and the spaced throwing daemon; establishing broadcast communication connection between the spaced throwing APK and the spaced throwing system service; an air drop daemon configured to: receiving media content sent by the space-air drop equipment; sending media content to the air-spaced drop APK through the second socket connection; an alternate delivery APK configured to: and controlling the electronic equipment to play the media content.

The spaced throwing APK is a non-system process, the assigned UID is the same as the spaced throwing daemon, a random port of the websocket can be opened for communication with the spaced throwing daemon, and AIRPLAY NATIVE Application in the official SDK corresponds to the spaced throwing daemon. Responsible for Video (Video) playback and UI rendering.

When the media content is video data or mirror image delivery, the spaced delivery APK draws the received video picture or mirror image picture on a display screen of the electronic equipment.

After the mirror image space drop operation is executed, the mobile phone sends mirror image data of the video to a space drop daemon, and after the space drop daemon receives the mirror image data, the mirror image data is sent to a space drop APK through a second socket connection, and the space drop APK performs video playing and UI drawing based on the received mirror image data.

Further, the air drop daemon is configured to: and receiving a control signal sent by the spaced-apart delivery APK through a second socket connection.

The processing flow of the control signal is consistent with the processing flow of the control signal sent by the air-spaced drop equipment, and the difference between the processing flow of the control signal and the processing flow of the control signal is that the control signal is input by a user through electronic equipment; for example: in the case where the electronic device is a television, the control signal may be a volume adjustment signal, a play progress signal, or the like, which is input by a user through a remote control of the television or a knob of the television.

For example: the control signal may be input by a user via an input means of the electronic device. For example: the user can adjust the volume of the media content on the electronic equipment, generate a volume adjustment signal of the media content, send the volume adjustment signal to the spaced throwing APK, send the volume adjustment signal to the spaced throwing daemon through the second socket connection, and send the volume adjustment signal to the spaced throwing system service through the first socket connection after the spaced throwing daemon receives the volume adjustment signal of the media content, wherein the spaced throwing system service adjusts the volume of the media content based on the volume adjustment signal.

In one possible implementation, a broadcast connection is established between the air-spaced drop APK and an air-spaced drop system service; an air space delivery system service configured to: receiving a life cycle signal sent by the spaced-apart delivery APK through a broadcast connection; the lifecycle of the spaced-apart drop application is controlled based on the lifecycle signal.

As shown in fig. 7, a broadcast connection is established between the air-spaced delivering APK and the air-spaced delivering system service, wherein the life cycle signal includes a start signal, a stop signal, an enter background signal, and the like of the air-spaced delivering application. After the spaced throwing APK receives a life cycle signal related to the spaced throwing system, the life cycle signal is sent to the spaced throwing system service in a broadcasting mode, so that the spaced throwing system service controls the life cycle of the spaced throwing application program based on the life cycle control signal.

The method includes the steps that a user inputs a space-air drop stopping operation on the electronic equipment, the space-air drop APK receives a space-air drop stopping signal, and after the space-air drop APK receives the space-air drop stopping signal, the space-air drop stopping signal is sent to space-air drop system service in a broadcasting mode, so that the space-air drop system service finishes running of a space-air drop application program based on the space-air drop stopping signal.

On the basis of the above embodiment, the embodiment of the present application further optimizes the software system architecture, as shown in fig. 8, where the optimized software system architecture is further configured with a PTP unit, where the PTP unit is located in a hardware abstraction layer HAL in the software system architecture, the UID of the PTP unit is a first UID value, the first UID value is smaller than a set value, and a hardware abstraction layer interface definition language (HAL INTERFACE definition language, HIDL) connection is established between the clock synchronization protocol unit and the space-air drop system service.

The PTP unit provides services in the HAL in the form of a PTP agent interface. And establishing HIDL connection between the PTP unit and the space-time drop system service.

The PTP unit system process is deployed on the HAL, the first UID value is 2920, and is used for implementing PTP protocol in a butt joint mode, monitoring UDP ports 319 and 320, and communicating with AirPlayDaemon through an Android standard HIDL interface, so that butt joint of PTP protocol and AirPlayDaemon protocol stack is implemented. UID 2920 is an open special UID, and using this UID to monitor UDP 319, 320 meets the security requirements that can meet CTS.

In one possible implementation, the PTP unit is configured to: clock synchronization information of the air-spaced drop equipment is obtained through the second UDP port and the third UDP port; the clock synchronization information is sent to the space-time drop system service; an air space delivery system service configured to: and controlling the playing progress of the media content on the electronic equipment to be consistent with the playing progress on the spaced-apart throwing equipment based on the clock synchronization information.

Specifically, the PTP unit acquires clock synchronization information of media content of the air drop device through UDP ports 319, 320; the PTP unit sends clock synchronization information to the space-time drop system service through HIDL connection; the space-time drop system service controls the playing progress of the media content on the electronic equipment to be consistent with the playing progress on the space-time drop equipment based on the clock synchronization information.

On the basis of the embodiment, the embodiment of the application further optimizes the software system architecture, and as shown in fig. 9, the optimized software system architecture is further provided with a PTP unit, wherein the PTP unit is deployed in the spaced air drop daemon; the UID of the space-air drop daemon is a second UID value, wherein the second UID value is smaller than a set value, and the space-air drop daemon is set as a system process.

The scheme is different from the scheme in that the PTP unit is realized in the space-time drop daemon, the space-time drop daemon is set as a system process in the scheme, UID is 1020, and IP address is: 127.0.0.2, monitor TCP ports: 12345, corresponding to air play & Homkit Daemon in the SDK, through message interaction with the first socket established by the air-space drop system service, to control the lifecycle of the air play application and send HomeKit event messages.

Due to UID:1020 is a special user group of a native mDNSC equipment discovery protocol, so that a monitoring port meets the safety requirement of CTS when the air drop daemon uses the UID, and the scheme can also meet the technical specification requirement of AIRPLAY SDK.

In one possible implementation, the PTP unit is configured to: clock synchronization information of the air-spaced drop equipment is obtained through the second UDP port and the third UDP port; the clock synchronization information is sent to the space-time drop system service; an air space delivery system service configured to: and controlling the playing progress of the media content on the electronic equipment to be consistent with the playing progress on the spaced-apart throwing equipment based on the clock synchronization information.

Specifically, a PTP unit in the air-separation drop daemon acquires clock synchronization information of media content of the air-separation drop device through UDP ports 319, 320; the space-time drop daemon transmits clock synchronization information monitored by the PTP unit to space-time drop system service through first socket connection; the space-time drop system service controls the playing progress of the media content on the electronic equipment to be consistent with the playing progress on the space-time drop equipment based on the clock synchronization information.

Further, as shown in fig. 8 or 9, the software architecture further includes: TEE, including: MFI Trusted Application, SHARED KEY Store, fairPlay Trusted Application.

Wherein MFI Trusted Application is responsible for managing access to the keying material, handling authorization verification from AirPlay Daemon. SHARED KEY Store is responsible for securely storing the pairing Key, sharing keys with AirPlay Daemon at HomeKit. The paired keys must be stored through a secure storage environment provided by the TEE. FairPlay Trusted Application is responsible for content key agreement, secure playback of video.

In one possible implementation, when the media content transmitted by the air drop device is audio data, the audio data is transmitted in the form of audio buffered data.

As shown in fig. 10, when the air-spaced throwing device performs Audio (Audio) data pushing, the air-spaced throwing device directly sends Audio Buffer (Audio Buffer) data to the air-spaced throwing daemon, the air-spaced throwing daemon forwards the Audio Buffer data to the air-spaced throwing APK, and the air-spaced throwing APK performs upper layer UI drawing based on the Audio Buffer data. The air-drop device and the air-drop daemon front Clock synchronization (Clock sync) module are used for synchronizing the simultaneous playing of audio.

In one possible implementation manner, when the media content sent by the air space delivery device is video data, the video data is sent in a Uniform Resource Locator (URL) address; the URL address is used for indicating the spaced apart putting APK to acquire video data from the URL address.

As shown in fig. 11, when the air-separation drop device performs Video (Video) pushing, the air-separation drop device sends a URL address of the Video to the air-separation drop daemon, and then the air-separation drop daemon forwards the URL address to the air-separation drop APK, and the air-separation drop APK completes media resource downloading based on the URL address, and performs Video playing and upper layer UI drawing according to the downloaded media resource.

In one possible implementation, when the media content transmitted by the air drop device is mirrored data, the audio data is transmitted in the form of audio buffered and video buffered data.

As shown in fig. 12, when the space-saving device performs mirror image (Mirroring) data pushing, the space-saving device directly sends audio buffer data and video buffer data to the space-saving daemon, and then the space-saving daemon is responsible for processing the audio buffer data and forwarding the video data to a space-saving APK, which is responsible for upper UI rendering.

FIG. 13 is a diagram of a space-free drop input selection interface provided by an embodiment of the present application; FIG. 14 is a diagram of a network inspection prompt interface provided by an embodiment of the present application; FIG. 15 is an interface diagram of a setup network according to an embodiment of the present application; FIG. 16 is a view of a barrier interface for air drop provided by an embodiment of the present application; FIG. 17 is an interface diagram of a space-free drop daemon setup provided by an embodiment of the present application; fig. 13-17 are interface flows for opening the air drop function in the electronic device according to the embodiment of the present application, as shown in fig. 13, when a user wants to use the air drop function, an input signal source of the electronic device may be set to be in an air drop in the user interface shown in fig. 13, the user interface of the electronic device jumps to a network detection prompt interface shown in fig. 14 to prompt the user to check whether the network connection is normal, and in the case that the network connection is normal, the user interface jumps to a setting network interface shown in fig. 15, and in this page, the electronic device is set to be a TV for turning on the android system. After the setting is successful, the method jumps to the air separation drop shielding interface of fig. 16, in which the user can enter the air separation drop daemon setting interface shown in fig. 17, in which the air separation drop is set to an on state. Thus, after the setting of the air separation throwing function of the electronic equipment is finished, the media content transmitted by the air separation throwing equipment can be received and the media content thrown at intervals can be played.

The embodiment of the application provides a message interaction method based on spaced throwing, which is applied to the electronic equipment, and comprises the following steps: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs a space-free drop system service and a space-free drop daemon, the space-free drop system service is positioned in a system partition in an application layer in a software system architecture of electronic equipment, and a physical abstraction layer PAL is deployed in the space-free drop system service; the space-air drop process is positioned in a developer partition in an application layer in the software system architecture; and a first socket connection is established between the space-air drop system service and the space-air drop daemon.

As shown in fig. 18, the message interaction method based on spaced apart delivery provided by the embodiment of the application mainly includes the following steps:

s101, receiving a control signal of media content through an air space drop daemon, and sending the control signal to an air space drop system service through a first socket connection.

S102, controlling media content which is put in the air space based on the control signal through the air space putting system service, wherein the media content comprises a media picture and/or a media sound.

S103, playing media pictures put in at intervals through a display screen; and/or the audio player plays the media audio put at intervals.

In one possible implementation manner, the processor is also provided with an air-spaced application package APK, and the air-spaced application APK is located in a developer partition in an application layer in the software system architecture; establishing a second socket connection between the spaced throwing APK and the spaced throwing daemon; establishing broadcast communication connection between the spaced throwing APK and the spaced throwing system service;

The method further comprises the steps of: the air separation drop daemon receives media content sent by the air separation drop equipment and sends the media content to the air separation drop APK through the second socket connection; and the spaced-apart delivery APK controls the electronic equipment to play the media content.

In one possible implementation, the method further includes: the air separation drop daemon receives a control signal sent by an air separation drop APK through second socket connection; or receiving a control signal sent by the air-spaced throwing equipment through the set port.

In one possible implementation, when the media content transmitted by the air-spaced delivery device is audio data, the audio data is transmitted in the form of audio buffered data; when the media content transmitted by the air space drop device is video data, the video data is transmitted by a uniform resource locator URL address; the URL address is used for indicating the air-spaced drop APK to acquire video data from the URL address; when the media content transmitted by the spaced-apart air drop device is mirror image data, the audio data is transmitted in the form of audio buffer and video buffer data.

In one possible implementation, a broadcast connection is established between the air-spaced drop APK and an air-spaced drop system service; the method further comprises the steps of: the service of the air-spaced throwing system receives a life cycle signal sent by the air-spaced throwing APK through broadcast connection; the lifecycle of the spaced-apart drop application is controlled based on the lifecycle signal.

In one possible implementation, the method further includes: an air drop daemon configured to: monitoring a first Transmission Control Protocol (TCP) port, wherein the first TCP port is an air-spaced drop standard protocol port; and monitoring a first User Datagram Protocol (UDP) port, wherein the first UDP port is a multicast DNS equipment discovery protocol port.

In one possible implementation manner, the processor is further provided with a clock synchronization protocol unit, the clock synchronization protocol unit is located in a hardware abstraction layer HAL in the software system architecture, a user identification UID of the clock synchronization protocol unit is a first UID value, the first UID value is smaller than a set value, and hardware abstraction layer interface definition language HIDL connection is established between the clock synchronization protocol unit and the space-air drop system service.

In one possible implementation, the clock synchronization protocol unit is deployed in an air drop daemon; the UID of the space-air drop daemon is a second UID value, wherein the second UID value is smaller than a set value, and the space-air drop daemon is set as a system process.

In one possible implementation, the method further includes: the clock synchronization protocol unit acquires clock synchronization information of the air-spaced drop equipment through the second UDP port and the third UDP port; the clock synchronization information is sent to the space-time drop system service; the space-time drop system service controls the playing progress of the media content on the electronic equipment to be consistent with the playing progress on the space-time drop equipment based on the clock synchronization information.

In some embodiments, embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, which when executed by a computing device, causes the computing device to implement the method for spaced apart delivery based message interaction of any of the embodiments described above.

In some embodiments, embodiments of the application provide a computer program product which, when run on a computer, causes the computer to implement the method of spaced apart delivery based message interaction of any of the embodiments of the first or second aspects.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. The illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims (10)

1. An electronic device, comprising: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs an air-separation system service and an air-separation daemon, and the air-separation system service is positioned in a system partition in an application layer in a software system architecture of the electronic equipment; the air separation drop process is located in a developer partition in an application layer in the software system architecture; establishing a first socket connection between the space-time throwing system service and the space-time throwing daemon;

The display screen is configured to: playing media pictures put in at intervals;

the audio player is configured to: playing media audio put in a spaced mode;

the air space drop daemon is configured to: receiving a control signal of media content;

The control signal is sent to the space-time throwing system service through the first socket connection;

The air space drop system service is configured to: and controlling media content put in the space based on the control signal, wherein the media content comprises a media picture and/or a media sound.

2. The electronic device of claim 1, wherein the processor further has an air drop application package APK running therein, the air drop APK being located in a developer partition in an application layer in the software system architecture; establishing a second socket connection between the spaced throwing APK and the spaced throwing daemon;

the air space drop daemon is configured to: receiving media content sent by the space-air drop equipment;

sending media content to the air-spaced drop APK through the second socket connection;

The spaced apart impression APK is configured to: and controlling the electronic equipment to play the media content.

3. The electronic device of claim 2, wherein the air drop daemon is configured to: receiving a control signal sent by the spaced apart delivery APK through the second socket connection; or alternatively

And receiving a control signal sent by the air-spaced throwing equipment through the set port.

4. The electronic device of claim 3, wherein when the media content transmitted by the air drop device is audio data, the audio data is transmitted in the form of audio buffered data;

When the media content sent by the air space drop device is video data, the video data is sent by a uniform resource locator URL address; the URL address is used for indicating the spaced apart putting APK to acquire video data from the URL address;

when the media content sent by the air-spaced drop device is mirror image data, the audio data are sent in the form of data of audio buffer and video buffer.

5. The electronic device of claim 2, wherein a broadcast connection is established between the spaced apart delivery APK and the spaced apart delivery system service;

The air space drop system service is configured to: receiving a life cycle signal sent by the spaced apart delivery APK through the broadcast connection;

And controlling the life cycle of the spaced air drop application program based on the life cycle signal.

6. The electronic device of claim 1, wherein the air space drop daemon is configured to:

monitoring a first Transmission Control Protocol (TCP) port, wherein the first TCP port is an air-spaced drop standard protocol port;

and monitoring a first User Datagram Protocol (UDP) port, wherein the first UDP port is a multicast DNS equipment discovery protocol port.

7. The electronic device of claim 2, wherein a clock synchronization protocol unit is further running in the processor, the clock synchronization protocol unit is located in a hardware abstraction layer HAL in the software system architecture, a user identification UID of the clock synchronization protocol unit is a first UID value, the first UID value is smaller than a set value, and a hardware abstraction layer interface definition language HIDL connection is established between the clock synchronization protocol unit and the space-time drop system service.

8. The electronic device of claim 2, wherein a clock synchronization protocol unit is deployed in the air drop daemon; the UID of the air separation drop daemon is a second UID value, wherein the second UID value is smaller than a set value, and the air separation drop daemon is set as a system process.

9. The electronic device according to claim 7 or 8, characterized in that,

The clock synchronization protocol unit is configured to: clock synchronization information of the air-spaced drop equipment is obtained through the second UDP port and the third UDP port;

the clock synchronization information is sent to the space-time drop system service;

The air space drop system service is configured to: and controlling the playing progress of the media content on the electronic equipment to be consistent with the playing progress on the air-spaced throwing equipment based on the clock synchronization information.

10. The message interaction method based on the spaced delivery is characterized by being applied to electronic equipment, and the electronic equipment comprises the following steps: the system comprises a processor, a display screen and/or an audio player, wherein the processor runs an air-separation system service and an air-separation daemon, and the air-separation system service is positioned in a system partition in an application layer in a software system architecture of the electronic equipment; the air separation drop process is located in a developer partition in an application layer in the software system architecture; establishing a first socket connection between the space-time throwing system service and the space-time throwing daemon; the method comprises the following steps;

receiving a control signal of media content through the air separation drop daemon, and sending the control signal to an air separation drop system service through the first socket connection;

controlling media content put in a space based on the control signal through the space-time put system service, wherein the media content comprises a media picture and/or media sound;

playing media pictures put in at intervals through the display screen; and/or the audio player plays the media audio put at intervals.