CN117271170B

CN117271170B - Activity event processing method and related equipment

Info

Publication number: CN117271170B
Application number: CN202311569296.4A
Authority: CN
Inventors: 李国英
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-11-23
Filing date: 2023-11-23
Publication date: 2024-04-16
Anticipated expiration: 2043-11-23
Also published as: CN117271170A

Abstract

The embodiment of the application provides an activity event processing method and related equipment, wherein the method comprises the following steps: initializing an activity recognition module and a plurality of physical sensors of the electronic device; establishing communication connection between the activity recognition module and clients in a plurality of hardware devices of the electronic device; the method comprises the steps that clients in a plurality of hardware devices respectively send subscription requests to an activity recognition module, and the activity recognition module responds to the subscription requests to create an activity recognition instance corresponding to each hardware device; the activity recognition module acquires sensing data of a plurality of physical sensors and determines an activity recognition result according to the sensing data; if the activity recognition result is matched with the at least one activity recognition instance, reporting the activity recognition result to a client corresponding to the at least one activity recognition instance matched with the activity recognition result. The embodiment of the application respectively creates the activity recognition examples for different hardware devices without reporting the activity event to the client sides of the different hardware devices at the same time, thereby effectively reducing the system power consumption.

Description

Activity event processing method and related equipment

Technical Field

The present application relates to the field of terminal technologies, and in particular, to a method and related device for processing an activity event.

Background

With the development of terminal technology, intelligent terminal devices such as smart phones and tablet computers can provide an AR (Activity Recognition ) function, the AR function can recognize different types of user activities, such as stillness, walking, running, riding an elevator, riding a high-speed rail, riding a subway and the like, and a client can subscribe to an AR service, so that when an AR module detects a user activity event, the client is informed to perform corresponding service processing. However, when detecting a user activity event, the AR module typically notifies all clients subscribed to the AR service, and as different clients may operate in different hardware devices, it is necessary to wake up different hardware devices to process the activity event, which results in higher system power consumption and waste of processing resources.

Disclosure of Invention

In view of the foregoing, it is necessary to provide an activity event processing method and related device, so as to solve the problem that when the above-mentioned AR module detects a user activity event, it notifies all clients subscribing to the AR service to perform activity event processing, resulting in higher system power consumption and waste of processing resources.

In a first aspect, the present application provides an activity event processing method, applied to an electronic device, the method including: initializing an activity recognition module and a plurality of physical sensors of the electronic device; establishing communication connection between the activity recognition module and clients in a plurality of hardware devices of the electronic device; the method comprises the steps that clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, and the activity recognition module responds to the subscription requests to create activity recognition examples corresponding to each hardware device; the activity recognition module acquires sensing data of the plurality of physical sensors and determines an activity recognition result according to the sensing data; and if the activity recognition result is matched with at least one activity recognition instance, reporting the activity recognition result to a client corresponding to the at least one activity recognition instance matched with the activity recognition result.

Through the technical scheme, the activity recognition module can respectively establish activity recognition examples for different hardware devices, each activity recognition example processes the activity event report of the corresponding hardware device, the number of times of waking up the hardware device is reduced, the system power consumption is reduced, and the waste of processing resources is avoided.

In one possible implementation, the method further includes: initializing an activity recognition module of the electronic device includes: and creating the activity recognition module through a coprocessor of the electronic equipment, and configuring parameters of the activity recognition module, wherein the parameters of the activity recognition module comprise the name of the activity recognition module, the type of the activity to be recognized, the data type and the unique identifier.

By the technical scheme, the activity recognition module is initialized, and parameters of the activity recognition module are configured, so that the activity recognition module timely detects the activities of the user.

In one possible implementation, the method further includes: initializing the plurality of physical sensors of the electronic device includes: acquiring a sensor manager, and acquiring a physical sensor list through the sensor manager; and acquiring the object of each physical sensor in the physical sensor list, and issuing the available identification of the corresponding physical sensor through the object of each physical sensor.

By adopting the technical scheme, the physical sensor is initialized, so that the physical sensor can timely detect the sensing data related to the activity recognition, and the activity state is accurately determined.

In one possible implementation, the method further includes: the activity recognition module registers a listening availability status with the plurality of physical sensors.

Through the technical scheme, the activity recognition module monitors the available state of the physical sensor, so that when the physical sensor is available, sensing data detected by the physical sensor are timely obtained, and the activity state is analyzed according to the sensing data.

In one possible implementation, the method further includes: the activity recognition module obtains an instance of a sensor manager, registers a sensor listener for each physical sensor, and registers the sensor listener to the sensor manager, thereby listening for available states of the plurality of physical sensors through the sensor listener.

Through the technical scheme, the sensor monitor is adopted, so that the activity recognition module can timely acquire the available state of the physical sensor.

In one possible implementation, the method further includes: the establishing a communication connection between the activity recognition module and a client in a plurality of hardware devices of the electronic device includes: the clients of the plurality of hardware devices send the data types and the unique identifiers of the activity recognition modules to the activity recognition modules, and request to establish communication connection with the activity recognition modules; the activity recognition module verifies the data type and the unique identifier sent by the client; if the data type sent by the client is the same as the data type of the activity identification module, and the unique identifier sent by the client is the same as the unique identifier of the activity identification module, determining that the data type and the unique identifier sent by the client pass verification, and establishing communication connection between the activity identification module and the client.

Through the technical scheme, the communication connection between the hardware equipment and the movable identification module is verified according to the data type and the unique identifier of the movable identification module, so that the normal communication connection between the hardware equipment and the movable identification module is ensured, and further, the normal message transmission between the hardware equipment and the movable identification module is ensured.

In one possible implementation, the method further includes: the plurality of hardware devices comprise an application processor, clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, and the activity recognition module responds to the subscription requests, and the creating of the activity recognition instance corresponding to each hardware device comprises the following steps: the client of the application processor registers an event callback monitor in an activity recognition management service in a hardware abstraction layer of the electronic equipment; the client of the application processor issues subscription parameters to the activity recognition management service; the activity recognition management service generates a subscription request according to the subscription parameters and issues the subscription request to the activity recognition module; and the activity recognition module creates an activity recognition instance corresponding to the application processor according to the subscription request.

Through the technical scheme, the application processor subscribes the activity state in cooperation with the activity recognition management service in the hardware abstraction layer, and the activity recognition module creates an activity recognition instance for the application processor, so that only the activity event which needs to be processed by the application processor can wake up the application processor for processing, the system power consumption is effectively reduced, and the processing resources are saved.

In one possible implementation, the method further includes: the activity recognition management service encapsulates the subscription parameters and the subscription instructions to generate the subscription request, the subscription request is issued to a sensor center of a coprocessor of the electronic device, and the sensor center sends the subscription request to the activity recognition module.

Through the technical scheme, the clients in the application processor issue subscription requests through the hardware abstraction layer, so that the hardware abstraction layer is responsible for message transfer of a plurality of clients, and the message transfer efficiency between the application program layer and the activity recognition module in the sensor center layer is improved.

In one possible implementation, the method further includes: the plurality of hardware devices comprise a sensor center, clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, and the activity recognition module responds to the subscription requests, and the creating of the activity recognition instance corresponding to each hardware device comprises the following steps: the client of the sensor center registers an event callback monitor in the activity recognition module; the client of the sensor center issues a subscription request to the activity recognition module, wherein the subscription request comprises subscription parameters and subscription instructions; and the activity recognition module responds to the subscription instruction, and creates an activity recognition instance corresponding to the sensor center according to the subscription parameter.

Through the technical scheme, the client side of the sensor center subscribes the activity state, and the activity recognition module creates an activity recognition instance for the sensor center, so that only the activity event to be processed by the sensor center can inform the sensor center to process, the system power consumption is effectively reduced, and the processing resources are saved.

In one possible implementation, the method further includes: the plurality of hardware devices comprise modems, clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, the activity recognition module responds to the subscription requests, and the creating of the activity recognition instance corresponding to each hardware device comprises the following steps: the client of the modem registers an event callback monitor in the activity recognition module; the client of the modem issues a subscription request to the activity recognition module, wherein the subscription request comprises subscription parameters and subscription instructions; and the activity recognition module responds to the subscription instruction and creates an activity recognition instance corresponding to the modem according to the subscription parameter.

Through the technical scheme, the client of the modem subscribes the activity state, and the activity recognition module creates an activity recognition instance for the modem, so that only the activity event to be processed by the modem can inform the modem to process, the system power consumption is effectively reduced, and the processing resources are saved.

In one possible implementation, the subscription parameters include client information, activity type, and activity status.

Through the technical scheme, the client information, the activity type and the activity state required by the client are used as subscription parameters, so that subsequent activity events can be reported to the corresponding client, the number of times of waking up hardware equipment is reduced, the system power consumption is effectively reduced, and processing resources are saved.

In one possible implementation, the method further includes: if the activity type in the activity recognition result is the same as the activity type in at least one subscription parameter, and the activity state in the activity recognition result is the same as the activity state in the at least one subscription parameter, determining that the activity recognition result is matched with the at least one subscription parameter, and further determining that the activity recognition result is matched with at least one activity recognition instance corresponding to the client information in the at least one subscription parameter.

Through the technical scheme, after the activity recognition result is determined, the activity recognition instance corresponding to the activity recognition result can be accurately determined, so that the activity event can be reported through the corresponding activity recognition instance in time.

In one possible implementation, the method further includes: the activity recognition module obtains sensing data of the plurality of physical sensors, and determines an activity recognition result according to the sensing data, including: the activity recognition module determines sensing data required by the client according to subscription parameters of each client, and registers a data acquisition request to a physical sensor for detecting the sensing data; after registering a data acquisition request, the physical sensor sends the detected sensing data to the activity recognition module; the activity recognition module receives the sensing data and sends the sensing data to an activity recognition algorithm module; the activity recognition algorithm module obtains an activity recognition result based on the sensing data and returns the activity recognition result to the activity recognition module.

Through the technical scheme, the activity recognition algorithm module can intelligently analyze and obtain the activity recognition result according to the sensing data, so that the accuracy of the activity recognition result is improved.

In a second aspect, the present application provides an electronic device comprising a memory and a processor: wherein the memory is used for storing program instructions; the processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, cause the electronic device to execute the activity event processing method described above.

In a third aspect, the present application provides a chip coupled to a memory in an electronic device, the chip being configured to control a processor of the electronic device to perform the above-described activity event processing method.

In a fourth aspect, the present application provides a computer storage medium storing program instructions that, when executed on an electronic device, cause a processor of the electronic device to perform the above-described activity event processing method.

In addition, the technical effects of the second aspect to the fourth aspect may be referred to in the description related to the method designed in the method section, and are not repeated here.

Drawings

Fig. 1 is a schematic diagram of an AR service architecture according to an embodiment of the present application.

Fig. 2 is a schematic diagram of an activity event processing system in the related art.

Fig. 3 is a software architecture diagram of an electronic device according to an embodiment of the present application.

Fig. 4 is a flowchart of an activity event processing method according to an embodiment of the present application.

FIG. 5 is a flow chart of an example of activity recognition corresponding to a subscription request for creating an application processor, according to one embodiment of the present application.

FIG. 6 is a flow chart of an example of activity recognition corresponding to a subscription request to create a sensor hub, according to one embodiment of the present application.

Fig. 7 is a flowchart of an activity recognition example corresponding to a subscription request for creating a modem according to an embodiment of the present application.

FIG. 8 is a schematic diagram of an activity event handling system according to an embodiment of the present application.

Fig. 9 is a partial processing timing diagram of an activity event processing method according to an embodiment of the present application.

FIG. 10 is a timing diagram illustrating another portion of an activity event processing method according to an embodiment of the present application.

Fig. 11 is a hardware architecture diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more features. In describing embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "e.g." in an embodiment of the present application should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It is to be understood that, unless otherwise indicated, a "/" means or. For example, A/B may represent A or B. The "and/or" in the present application is merely one association relationship describing the association object, indicating that three relationships may exist. For example, a and/or B may represent: a exists alone, A and B exist simultaneously, and B exists alone. "at least one" means one or more. "plurality" means two or more than two. For example, at least one of a, b or c may represent: seven cases of a, b, c, a and b, a and c, b and c, a, b and c. The following embodiments and features of the embodiments may be combined with each other without conflict.

With the development of terminal technology, intelligent terminal devices such as smart phones and tablet computers can provide an AR (Activity Recognition) function, different types of user activities such as resting, walking, running, riding, elevator, high-speed rail, subway and the like can be identified through the AR function, and clients can subscribe to AR services, so that when an AR module detects a user activity event, the clients are notified to perform corresponding service processing. For example, sports health applications assist in the updating of pedometer steps by subscribing to the activity status of a user's walking, running, riding, etc.; the weather clock application refreshes weather display when the AR module detects that the user is in a motion state; the information recommendation application displays a flight information card on a user interface popup window when the AR module detects that a user is in a riding state; the wireless communication application controls the network of the electronic equipment to intelligently switch between WLAN (Wireless Local Area Network ) and cellular mobile data when the AR module detects that the user is in an active state such as walking, running, taking an elevator and the like, so that the network intelligent connection experience of the user is improved; the compass application performs calibration of the magnetometer upon detection of an active state of the user, walking, running, hand-holding, etc.

However, when the AR module detects a user activity event, broadcast event reporting is generally performed, that is, all clients subscribed to the AR service are notified, and as different clients operate in different hardware devices, it is required to wake up or notify different hardware devices to perform activity event processing, which results in higher system power consumption and easy waste of processing resources. For example, an athletic health application running by an application processor subscribes to the AR module for the user's walking, running, and riding conditions, and a modem (modem) running client subscribes to the AR module for the user's walking, running, and riding elevator conditions. In the related art, if the AR module detects that the user is in a walking state, a corresponding activity event needs to be reported to the application processor and the modem to wake up the application processor and notify the modem, so that the application processor and the modem perform corresponding service processing according to the walking state of the user. If the AR module detects that the user is in a riding state, the corresponding activity event needs to be reported to the application processor and the modem to wake the application processor and inform the modem, and the application processor performs corresponding service processing according to the riding state of the user. If the AR module detects that the user is in an elevator riding state, the corresponding activity event needs to be reported to the application processor and the modem to wake the application processor and inform the modem, and the modem performs corresponding service processing according to the elevator riding state of the user. Thus, no matter what type of user state is detected by the AR module, the corresponding activity event needs to be reported to the application processor and the modem, and frequent wake-up of the application processor or notification of the modem to perform service processing easily results in higher system power consumption. In addition, since only hardware devices subscribed to relevant active states actually perform corresponding service processing, if the AR module reports an active event to clients of multiple hardware devices, waste of processing resources is also caused.

Fig. 1 is a schematic diagram of an AR service architecture according to an embodiment of the present application. AR services provide an end-to-end solution from the application side to the Sensorhub (sensor hub) side. The application layer and the framework layer comprise an AP (Application Processor ) side client and transmit subscription information by calling an SDK (Software Development Kit ) interface; the AR service is arranged on an HIDL (HARDWARE INTERFACE Description Layer )/HAL (Hardware Abstraction Layer, hardware abstraction) layer and is mainly responsible for the issuing of client subscription information and the reporting of corresponding activity events; the AR virtual sensor is arranged at the sensor center side and is responsible for receiving requests issued by AR services, constructing corresponding examples according to different requests and reporting corresponding activity events to the AR services. The clients of the AR virtual sensor can also run in the sensor center and a Modem (Modem), and different clients can acquire the entering and exiting states of different AR activities by issuing different message IDs.

Referring to fig. 2, a schematic diagram of an activity event processing system in the related art is shown. The AR virtual sensor generally performs reporting of an activity event according to a single instance (instance), that is, when clients on the AP side, the Modem side, and the Sensorhub side subscribe for AR services, after a subscription request is transmitted to the AR virtual sensor, the AR virtual sensor only creates one instance, and performs data collection, algorithm result acquisition, and event reporting through the created instance, and before reporting, a corresponding activity event reporting result is respectively constructed according to different clients. However, a single instance is broadcast reporting, that is, the same activity event result may be reported to clients on different hardware sides at the same time, for example, the activity event result on the AP side may be sent to clients on the Modem side and Sensorhub side when being reported to clients on the AP side, and similarly, the activity event results sent to the Modem side and Sensorhub side may be reported to the AP side. However, for the report result of the unmatched message ID, the client may discard the report result, and may not perform corresponding service processing, which may cause waste of processing resources. In addition, when the electronic device is in a standby state, the electronic device wakes up the AP except when sending a result to the client on the AP side, and wakes up the AP when sending event results to the Modem side and Sensorhub side, which increases the number of wake-up times of the AP, resulting in an increase in system power consumption in standby. In addition, the clients on the Modem side and Sensorhub side generally need to subscribe to a higher event result reporting frequency so as to acquire algorithm results in time, avoid affecting service processing, and therefore, the number of wake-up times of the AP cannot be reduced by reducing the reporting frequency.

In order to avoid that under a single instance, when an AR virtual sensor detects a user activity event, all upper-layer clients subscribing the AR event to the AR virtual sensor are notified, so that an application processor is frequently awakened or other hardware devices are notified to process the activity event, and the system power consumption is high, and processing resources are wasted. The client is a driver of an application program or a hardware device, for example, the client of the application processor is an application program, the client of the sensor center and the client of the modem are corresponding drivers, and the client running on the hardware device means that data of the client is processed by the corresponding hardware device.

Fig. 3 is a software architecture diagram of an electronic device according to an embodiment of the present application. The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. For example, the android system is divided into four layers, namely, an application layer 101, a framework layer 102, an android runtime (Android runtime) and a system library 103, a hardware abstraction layer 104, a kernel layer 105 and a hardware layer 106 from top to bottom.

The application layer 101 may comprise a series of application packages. For example, the application package may include applications for cameras, gallery, calendar, phone calls, maps, navigation, WLAN, bluetooth, music, video, short messages, device control services, etc.

Framework layer 102 provides an application programming interface (Application Programming Interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

Wherein the window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like. The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc. The view system includes visual controls, such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text message notification icon may include a view displaying text and a view displaying a picture. The telephony manager is for providing communication functions of the electronic device. Such as the management of call status (including on, hung-up, etc.). The resource manager provides various resources for the application program, such as localization strings, icons, pictures, layout files, video files, and the like. The notification manager allows the application to display notification information in a status bar, can be used to communicate notification type messages, can automatically disappear after a short dwell, and does not require user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in a status bar, a prompt tone is emitted, the electronic device vibrates, and an indicator light blinks, etc.

Android run time includes a core library and virtual machines. Android runtime is responsible for scheduling and management of the android system. The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer 101 and the framework layer 102 run in virtual machines. The virtual machine executes java files of the application program layer and the framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.

The system library 103 may include a plurality of functional modules. Such as surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (e.g., openGL ES), 2D graphics engine (e.g., SGL), etc.

The surface manager is used for managing the display subsystem and providing fusion of 2D and 3D layers for a plurality of application programs. Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc. The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like. The 2D graphics engine is a drawing engine for 2D drawing.

The hardware abstraction layer 104 runs in user space, encapsulates kernel layer drivers, and provides a call interface to upper layers.

The kernel layer 105 is a layer between hardware and software. The kernel layer 105 contains at least a display driver, a camera driver, an audio driver, and a sensor driver.

The kernel layer 105 is the core of the operating system of the electronic device, is a first layer of software expansion based on hardware, provides the most basic functions of the operating system, is the basis for the operating system to work, is responsible for managing the processes, the memory, the device drivers, the files and the network system of the system, and determines the performance and the stability of the system. For example, the kernel may determine the time an application is operating on a certain portion of hardware.

The kernel layer 105 includes hardware-closely related programs, such as interrupt handlers, device drivers, etc., and basic, common, higher frequency of operation modules, such as clock management modules, process scheduling modules, etc., and critical data structures. The kernel layer may be provided in the processor or cured in an internal memory.

The hardware layer 106 includes hardware of the electronic device, such as a display screen, keys, cameras, etc.

As shown in fig. 1, the electronic device includes an application processor, a coprocessor (companion processor, CP), and a plurality of physical sensors. The co-processor includes an activity recognition virtual sensor including, but not limited to: the system comprises an activity recognition module and an activity recognition algorithm module, wherein the activity recognition module can acquire sensing data from a physical sensor and execute an activity recognition function, the activity recognition algorithm module comprises an activity recognition algorithm, and an activity recognition result can be determined according to the sensing data.

In one embodiment of the application, the application layer includes a plurality of applications of the electronic device, and the application layer deploys application packages (Android application package, APK), such as battery management applications, weather clock applications, sports health applications, and communication applications. The application subscribes to the AR service by issuing subscription parameters, e.g., the weather application registers callback listeners for activity recognition with the activity recognition management service (activity recognition MANAGER SERVICE, ARMS) of the framework layer through REGISTERLISTENER (registration listener), configures subscription parameters through the SDK (software development kit, SDK) interface provided by ARMS.

In one embodiment of the application, the framework layer includes an activity recognition management service, and the application communicates with the ARMS via an SDK interface provided by the ARMS. When an application is communicatively connected to an ARMS, the ARMS can uniformly manage subscription parameters for the application. Subscription parameters may include the type of activity the application subscribes to, reporting period, application information, etc. Wherein the activity type subscribed by the application program comprises an activity name and an activity state of the application program, and the activity state comprises an entering activity and an exiting activity. For example, when subscribing to a ride activity of an electronic device, the communication application includes a ride activity name, whether to enter the ride activity, whether to exit the ride activity, and so on. The application information may be an application name or an application ID.

In one embodiment of the application, the hardware abstraction layer includes a HIDL interface and a HAL interface. The HIDL interface is used for providing an interface for activity recognition, and specifically comprises subscription, unsubscribe, event callback and the like for activity recognition. And for subscription parameters issued by the application program layer, issuing the subscription parameters of the HIDL interface to the HAL interface. The HAL interface can hide the hardware interface details of a specific platform, provide a virtual hardware platform for an operating system, enable the operating system to have hardware independence and can be transplanted on various platforms. The HAL interface issues subscription parameters to an activity recognition module (i.e., an activity recognition virtual sensor) in the sensor center of the co-processor.

In an embodiment of the present application, when a client issues a subscription parameter, the subscription parameter is issued layer by layer through QMI (Qualcomm MESSAGING INTERFACE, high-pass communication interface), and when the subscription parameter is issued to a sensor center of a coprocessor, an activity recognition module in the sensor center obtains the subscription parameter issued by an application layer, creates an activity recognition instance corresponding to the client, obtains sensing data from a plurality of physical sensors according to the subscription parameter, and then sends the sensing data to an activity recognition algorithm module for activity recognition.

In an embodiment of the present application, the activity recognition module sends an activity recognition result returned by the activity recognition algorithm module to the activity recognition instance, where the activity recognition instance reports the activity recognition result according to the subscription parameter, and the activity recognition result includes an activity type, an activity recognition state, an activity recognition time, and the like. The subscription parameters can also comprise reporting modes corresponding to the application programs, and the reporting modes can be batch reporting modes and variable reporting modes. Batch reporting refers to a reporting mode of reporting the activity recognition result according to a certain period. The reporting of the change refers to a reporting mode that when the subscribed active state changes, for example, from running state to stationary state, the activity recognition instance reports the changed result.

Taking an application program running by a client as an application processor as an example, an activity recognition instance in a coprocessor reports an activity recognition result to a HAL interface through QMI, the HAL interface reports the activity recognition result reported by the activity recognition instance to an HIDL interface, the HIDL interface further reports the activity recognition result to ARMS (AR service), and the ARMS reports the activity recognition result to the client subscribing the activity type according to a monitoring mechanism. When the activity recognition result corresponds to a plurality of clients, the ARMS reports the recognition result to the corresponding plurality of clients.

Referring to fig. 4, a flowchart of an activity event processing method according to an embodiment of the application is shown. The method is applied to the electronic equipment, and the activity event processing method comprises the following steps:

S101, initializing an activity recognition module and a plurality of physical sensors of the electronic device.

In an embodiment of the present application, after the electronic device is turned on, a plurality of physical sensors are initialized to obtain a sensor manager SensorManager, a physical sensor list is obtained through sensor management, the physical sensor list is traversed to obtain an object of each physical sensor, a corresponding physical sensor usable identifier is issued through the object of each physical sensor, and the object of the physical sensor is a software program for analyzing and processing the sensing data. The physical sensor issues the status of the physical sensor through the available identification: available or unavailable, the available indicia includes a first indicia, e.g., a first indicia of 1, indicating that the physical sensor is available and a second indicia of 0, indicating that the physical sensor is not available. In this manner, the object of the physical sensor may set the identifiable flag to the first flag, thereby releasing that the physical sensor is available.

In one embodiment of the application, after the electronic device is powered on, the coprocessor creates an activity recognition sensor, wherein the activity recognition sensor comprises an activity recognition module and an activity recognition algorithm module, the activity recognition sensor is a virtual sensor, and is a sensor simulated by software and used for detecting the activity and motion information of a user, and the virtual sensor analyzes the action and behavior of the user by using a physical sensor (such as an accelerometer and a magnetometer) of the electronic device and a machine learning algorithm, and for example, the virtual sensor can be a pedometer, a walking/running detector, a sleep monitor, a motion recognizer and the like. The coprocessor creates an activity recognition sensor through an INTERNAL_APP_INIT () function, which can be used to pass on the activity type, initialization function, activity event handling function, etc. recognized by the activity recognition module. For example, the initialization function is ArInit (), and the activity event handling function is ARHANDLEEVENT ().

In one embodiment of the application, after the activity recognition module is created, the activity recognition module is initialized, and a chip platform sensor registration interface (e.g., sensorRegister interface) is called to issue the parameters of the activity recognition module. The parameters of the activity recognition module include the name, type, interrupt type, etc. of the activity recognition module. The client can search and issue configuration subscription parameters according to the types of the parameters of the activity recognition module. For example, the name of the activity recognition module may be "AR", representing a device for activity recognition; the TYPE may be expressed as sens_type_ar, sens_type_accel indicates an acceleration ACC TYPE, sens_type_modem indicates a communication protocol MODEM TYPE, sens_type_ar indicates an activity recognition AR TYPE, and the TYPE of registered activity recognition module parameter is a sens_type_ar TYPE, which is available for recognition and use by an application processor, and the interrupt TYPE is a wake-up TYPE.

In one embodiment of the application, after the activity recognition module and the plurality of physical sensors are initialized, the activity recognition module registers a listening availability status with the plurality of physical sensors. The activity recognition module obtains an instance of the sensor manager, registers a sensor listener (SensorEventListener) for each physical sensor, and registers the sensor listener to the sensor manager, thereby listening for the availability status of the plurality of physical sensors through the sensor listener. The sensor listener acquires the availability status by listening for the availability identity of the physical sensor. For example, if the sensor monitor monitors that the available identifier of a physical sensor is 1, the obtained available state of the physical sensor is available; if the sensor monitor monitors that the available mark of a physical sensor is 0, the obtained available state of the physical sensor is unavailable.

S102, communication connection between the activity recognition module and the client in the plurality of hardware devices is established.

In one embodiment of the application, the plurality of hardware devices includes, but is not limited to: an application processor, a sensor hub (Sensorhub) and a modem. After the activity recognition module completes the monitoring of registering a plurality of physical sensors, the parameters of the activity recognition module are broadcasted, namely, the parameters of the activity recognition module are reported to a plurality of hardware devices so as to inform the system that the activity recognition module is in place. Parameters of the activity recognition module include, but are not limited to: a data type and a unique identifier (SUID), wherein the data type is a type of data that can interact with the activity recognition module. The client of the hardware device may send the data type and/or the unique identifier to the activity recognition module to request to establish communication connection with the activity recognition module, the activity recognition module verifies the data type and/or the unique identifier sent by the client, and if the data type sent by the client is the same as the data type sent by the activity recognition module and the unique identifier sent by the client is the same as the unique identifier sent by the activity recognition module, it is determined that the data type and the unique identifier sent by the client pass the verification, and communication connection between the activity recognition module and the client is established.

In an embodiment of the present application, an activity recognition module is deployed in a sensor center, a client of an application processor sends a communication connection request containing a data type and/or a unique identifier to be verified to the activity recognition module through an activity management service in a hardware abstraction layer, the activity recognition module establishes communication connection with a corresponding client through the activity recognition management service after the data type and/or the unique identifier to be verified are successfully matched, a callback function is registered after the client establishes communication connection with the activity recognition module, and the activity recognition module reports an activity event (for example, an activity recognition result) to the client according to the callback function.

In one embodiment of the present application, since the activity recognition module is deployed at the sensor hub, the client of the sensor hub may send a communication connection request containing the data type and/or unique identifier to be verified directly to the activity recognition module. And after the data type to be verified and/or the unique identifier are successfully matched, the activity identification module establishes communication connection with the corresponding client. After the communication connection between the client and the activity recognition module is established, a callback function is also registered, and the activity recognition module reports an activity event to the client according to the callback function.

In an embodiment of the present application, a client of a modem is deployed in a subsystem (Modem peripheral subsystem, MPSS) of the modem, the client of the modem may send a communication connection request including a data type and/or a unique identifier to be verified to an active identification module through QMI, the active identification module establishes a communication connection with a corresponding client after the data type and/or the unique identifier to be verified are successfully matched, and after the client establishes a communication connection with the active identification module, a callback function is registered, and the active identification module reports an active event to the client according to the callback function.

In another embodiment of the present application, the client of the different hardware device may also send a SUID requesting to obtain the activity recognition module, and the activity recognition module may establish a communication connection between the client and the activity recognition module by feeding back the SUID to the client.

S103, the clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, and the activity recognition module responds to the subscription requests to create an activity recognition instance corresponding to each hardware device.

In one embodiment of the application, the subscription request includes subscription parameters and subscription instructions, the subscription parameters including, but not limited to: client information, activity name, and activity status. The method comprises the steps that a client sends a subscription request to an activity recognition module, the activity recognition module responds to a subscription instruction, an activity recognition instance is created according to subscription parameters, the activity recognition instance is used for recording association relations among client information, activity names and activity states, and when an activity recognition result is received, the activity recognition result is reported to a corresponding client according to the association relations among the client information, the activity names and the activity states. The client information may include hardware device information of the running client and basic information of the client, the basic information of the client may be a client ID or a client name, the activity name may be walking, running, riding, taking an elevator, etc., and the activity state is an activity corresponding to the entering or exiting activity name, for example, entering or exiting a walking state, etc. In one embodiment of the application, the activity recognition instance is a software function module for reporting the activity recognition result to a client in a hardware device.

In an embodiment of the application, after receiving the subscription parameters, the activity recognition module judges whether an activity recognition instance corresponding to a client sending the subscription parameters is created according to the client information, and if the activity recognition instance corresponding to the client is created, the subscription parameters are written into the corresponding created activity recognition instance; if the activity recognition instance corresponding to the client is not created, creating the activity recognition instance corresponding to the client, and writing the subscription parameters into the created activity recognition instance.

In an embodiment of the application, after receiving the subscription parameters, the activity recognition module determines a hardware device running a client sending the subscription parameters according to client information in the subscription parameters, judges whether an activity recognition instance corresponding to the hardware device is created, and if the activity recognition instance corresponding to the hardware device is created, determines that the activity recognition instance corresponding to the client is created; if the activity recognition instance corresponding to the hardware device is not created, determining that the activity recognition instance corresponding to the client is not created.

In one embodiment of the application, the activity recognition instance is created by an activity recognition module, data is shared between the activity recognition module and the activity recognition instance, and the activity recognition instance can also communicate with a corresponding client when the activity recognition module establishes a communication connection with the client.

S104, the activity recognition module acquires sensing data of a plurality of physical sensors and determines an activity recognition result according to the sensing data.

In an embodiment of the application, the plurality of physical sensors may include acceleration sensors, barometric pressure sensors, ambient light sensors, proximity light sensors, temperature sensors, magnetometers, and the like. The method comprises the steps that an activity recognition module determines sensing data needed by each client according to subscription parameters of the client, registers a data acquisition request to a physical sensor for detecting the sensing data, after the data acquisition request is registered, the physical sensor can send the detected sensing data to the activity recognition module, the activity recognition module receives the sensing data and sends the sensing data to an activity recognition algorithm module, the activity recognition algorithm module analyzes and processes the sensing data to obtain an activity recognition result, and the activity recognition result is returned to the activity recognition module. Among them, the activity recognition results include, but are not limited to: activity type and activity status.

In one embodiment of the present application, the activity recognition algorithm module may include a plurality of deep learning models, each of which is generated by training with the sensing data as a feature, and recognizes the current user activity type and activity state through the deep learning model by inputting the sensing data into each of the deep learning models, respectively. For example, the deep learning model may be a convolutional neural network model.

S105, judging whether the activity recognition result is matched with at least one activity recognition instance. If the activity recognition result matches with at least one activity recognition instance, executing S106; if the activity recognition result does not match all the activity recognition instances, the process returns to S104.

In an embodiment of the present application, after receiving the activity recognition result, the activity recognition module determines a client that has subscribed to an activity type and an activity state in the activity recognition result, further determines a hardware device running the client, and sends the activity recognition result to an activity recognition instance corresponding to the hardware device running the client.

In an embodiment of the present application, it is determined whether an activity recognition result matches at least one subscription parameter, if the activity recognition result matches the at least one subscription parameter, it is determined that the activity recognition result matches at least one activity recognition instance corresponding to client information in the at least one subscription parameter, and then the activity recognition result is sent to the matched at least one activity recognition instance. If the activity recognition result is not matched with all the subscription parameters, determining that the activity recognition result is not matched with all the activity recognition instances. If the activity type in the activity recognition result is the same as the activity type in a subscription parameter, and the activity state in the activity recognition result is the same as the activity state in the subscription parameter, the activity recognition result is determined to be matched with the subscription parameter, and then the activity recognition result is determined to be matched with an activity recognition instance corresponding to a hardware device running a client side sending the subscription parameter.

S106, reporting the activity recognition result to the client corresponding to at least one activity recognition instance matched with the activity recognition result.

In an embodiment of the present application, if the activity recognition result matches at least one activity recognition instance, the at least one activity recognition instance generates an activity event according to the activity recognition result, and reports the activity event to a client corresponding to the at least one activity recognition instance.

In an embodiment of the present application, if the activity recognition result matches with the activity recognition instance corresponding to the application processor, the activity recognition instance corresponding to the application processor reports the activity event to the activity recognition management service of the hardware abstraction layer, and the activity recognition management service reports the activity event to the corresponding client (i.e. the application program) according to the client information in the subscription parameters, so as to wake up the application processor to process the activity event. If the activity recognition result is matched with the activity recognition instance corresponding to the sensor center, the activity recognition instance corresponding to the sensor center reports the activity event to the client side of the sensor center, so that the sensor center is informed to process the activity event. If the activity recognition result is matched with the activity recognition instance corresponding to the modem, the activity recognition instance corresponding to the modem reports the activity event to the client of the modem, so that the modem is informed to process the activity event.

According to the embodiment of the application, the activity recognition module can respectively create the corresponding activity recognition examples according to the subscription requests issued by the clients of different hardware, when the activity recognition results matched with the activity recognition examples are detected, the activity recognition results are reported to the clients of the hardware devices corresponding to the activity recognition examples, the activity events are processed through the corresponding hardware devices, the activity recognition results are not required to be reported to the clients of the different hardware devices, the system power consumption is effectively reduced, and the waste of processing resources is reduced.

Referring to fig. 5, a flowchart of an activity recognition example corresponding to a subscription request of an application processor is provided in an embodiment of the present application.

S201, a client of an application processor registers an event callback monitor in an activity recognition management service.

In one embodiment of the application, a client of an application processor registers an event callback listener in an activity recognition management service, and obtains an activity recognition result in a callback mode.

S202, a client of the application processor issues subscription parameters to the activity recognition management service.

In an embodiment of the present application, the subscription parameters include client information, an activity type, and an activity state, wherein the client information may include hardware device information for running the client and basic information of the client, the basic information of the client may be a client ID or a client name, for example, the activity type may be walking, running, riding, taking a high-speed rail, etc., and the activity state may include entering and exiting, for example, entering a walking state, exiting a walking state, etc.

S203, the activity recognition management service generates a subscription request according to the subscription parameters and issues the subscription request to the activity recognition module.

In an embodiment of the application, the activity recognition management service encapsulates the subscription parameters and the subscription instructions to generate a subscription request, the subscription request is issued to the HIDL, the HIDL issues the subscription request to the HAL interface, the HAL interface issues the subscription request to the coprocessor through QMI, and the coprocessor sends the subscription request to the activity recognition module in the sensor center.

S204, the activity recognition module creates an activity recognition instance corresponding to the application processor according to the subscription request.

In an embodiment of the present application, when receiving a subscription parameter, an activity recognition module determines, according to client information in the subscription parameter, that a hardware device running a client sending the subscription parameter is an application processor, determines whether an activity recognition instance corresponding to the application processor is created, if the activity recognition instance corresponding to the application processor is created, determines that an activity recognition instance corresponding to the client is created, and then directly writes the subscription parameter into the corresponding created activity recognition instance; if the activity recognition instance corresponding to the application processor is not created, determining that the activity recognition instance corresponding to the client is not created, and then creating the activity recognition instance corresponding to the application processor.

In an embodiment of the present application, the activity recognition module (i.e., the AR virtual sensor) searches for an instance matching the client sending the subscription request through the find_instance () method, and if the acquired instance is empty, it indicates that a corresponding instance has not been created for the client, and the activity recognition module invokes the create_instance interface to complete the creation of the activity recognition instance.

Referring to fig. 6, a flowchart of an activity recognition example corresponding to a subscription request for creating a sensor center according to an embodiment of the present application is shown.

S301, a client side of a sensor center registers an event callback monitor in an activity recognition module.

In an embodiment of the application, a client side of the sensor center registers an event callback monitor in the activity recognition module, and obtains an activity recognition result in a callback mode.

S302, a client side of the sensor center issues a subscription request to the activity recognition module.

S303, the activity recognition module creates an activity recognition instance corresponding to the sensor center according to the subscription request.

In an embodiment of the application, when receiving a subscription parameter, an activity recognition module determines that a hardware device running a client sending the subscription parameter is a sensor center according to client information in the subscription parameter, judges whether an activity recognition instance corresponding to the sensor center is created, determines that the activity recognition instance corresponding to the client is created if the activity recognition instance corresponding to the sensor center is created, and then directly writes the subscription parameter into the corresponding created activity recognition instance; if the activity recognition instance corresponding to the sensor center is not created, determining that the activity recognition instance corresponding to the client is not created, and then creating the activity recognition instance corresponding to the sensor center.

Referring to fig. 7, a flowchart of an activity recognition example corresponding to a subscription request for creating a modem according to an embodiment of the present application is shown.

S401, the client of the modem registers an event callback monitor in the activity recognition module.

In one embodiment of the application, a client of the modem registers an event callback listener in an activity recognition module, and obtains an activity recognition result in a callback manner.

S402, the client of the modem issues a subscription request to the activity recognition module.

S403, the activity recognition module creates an activity recognition instance corresponding to the modem according to the subscription request.

In an embodiment of the present application, when receiving a subscription parameter, an activity recognition module determines, according to client information in the subscription parameter, that a hardware device running a client sending the subscription parameter is a modem, determines whether an activity recognition instance corresponding to the modem is created, if the activity recognition instance corresponding to the modem is created, determines that the activity recognition instance corresponding to the client is created, and then directly writes the subscription parameter into the corresponding created activity recognition instance; if the activity recognition instance corresponding to the modem is not created, determining that the activity recognition instance corresponding to the client is not created, and then creating the activity recognition instance corresponding to the modem.

Referring to fig. 8, a schematic diagram of an activity event processing system according to an embodiment of the application is shown. The method comprises the steps that an application processor, a modem and a client of a sensor center respectively send subscription requests to an AR virtual sensor, the AR virtual sensor respectively responds to subscription requests of different hardware to create an activity identification instance, a first activity identification instance created in response to the subscription requests of the client of the application processor is used for detecting an activity event of the application processor, a second activity identification instance created in response to the subscription requests of the client of the modem is used for detecting the activity event of the modem, a third activity identification instance created in response to the subscription requests of the client of the sensor center is used for detecting the activity event of the sensor center, the first activity identification instance reports the activity event to the client of the application processor when the activity event is detected, the application processor is awakened to process the activity event, the second activity identification instance reports the activity event to the client of the modem when the activity event is detected, the modem is notified to process the activity event, and the third activity identification instance reports the activity event to the client of the sensor center when the activity event is detected, and the sensor center is notified to process the activity event.

Taking an exercise health application of an application processor as an example, the exercise health application can issue a subscription request to an AR virtual sensor through an activity recognition management service of a hardware abstraction layer, the AR virtual sensor responds to the subscription request issued by the exercise health application to create a first activity recognition instance, subscription parameters in the subscription request comprise client information, an activity type and an activity state, the client information comprises information indicating that a hardware device running the exercise health application is the application processor, the activity type comprises walking, running, riding and riding, the activity state comprises entering and exiting the walking state, entering and exiting the running state, entering and exiting the riding state and entering and exiting the riding state, the AR virtual sensor registers and monitors an acceleration sensor and a magnetometer according to the subscription parameters, after the acceleration sensor and the magnetometer detect sensing data, sending the sensing data to an AR virtual sensor, sending the sensing data to an AR algorithm library (namely an activity recognition algorithm module) by the AR virtual sensor, analyzing and processing the sensing data through an algorithm in the AR algorithm library, determining an activity type and an activity state, returning the determined activity type and activity state to the AR virtual sensor, judging whether the activity type determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the motion health application or not by the AR virtual sensor, judging whether the activity state determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the motion health application or not, if the activity type determined by the AR virtual sensor is the same as the activity type in the subscription parameters issued by the motion health application by the AR virtual sensor, and determining that the activity state determined by the AR algorithm is the same as the activity state in the subscription parameters issued by the motion health application, and sending the activity recognition result to a first activity recognition instance, wherein the first activity recognition instance generates an activity recognition event, reports the activity recognition event to the sports health application through an activity recognition management service, and wakes an application processor to perform subsequent business processing, such as updating the step number, according to the activity recognition event.

Taking the wireless communication application of the modem as an example, a client of the modem (such as a driver of the modem) can issue a subscription request to the AR virtual sensor, the AR virtual sensor creates a second activity recognition instance in response to the subscription request issued by the client of the modem, subscription parameters in the subscription request include client information, an activity type and an activity state, the client information includes information indicating that a hardware device running the client is the modem, the activity type includes walking, running, riding an elevator, the activity state includes entering and exiting a walking state, entering and exiting a running state, entering and exiting a riding an elevator state, the AR virtual sensor registers monitoring with the acceleration sensor and the magnetometer according to the subscription parameters, the acceleration sensor and the magnetometer send the sensing data to the AR virtual sensor after detecting the sensing data, the AR virtual sensor sends the sensing data to an AR algorithm library, analyzes the sensing data through an algorithm in the AR algorithm library, determines an activity type and an activity state, returns the determined activity type and the determined activity state to the AR virtual sensor, judges whether the activity type determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the client of the modem, judges whether the activity state determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the client of the modem, if the AR virtual sensor determines that the activity type determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the client of the modem, and determines that the activity state determined by the AR algorithm is the same as the activity state in the subscription parameters issued by the client of the modem, and sending the activity recognition result to a second activity recognition example, generating an activity recognition event by the second activity recognition example, reporting the activity recognition event to the modem, and informing the modem to perform subsequent service processing according to the activity recognition event, for example, the modem interacts with a wireless communication application, and the wireless communication application switches the mobile phone network between Wi-Fi and a cellular network according to network information and activity states provided by the modem.

Taking a compass application of a sensor center as an example, the compass application uses a magnetic force calibration sensor, sends a subscription request to an AR virtual sensor through the magnetic force calibration sensor, the AR virtual sensor responds to the subscription request sent by the magnetic force calibration sensor, creates a third activity recognition example, subscription parameters in the subscription request comprise client information, an activity type and an activity state, the client information comprises information indicating that a hardware device running the magnetic force calibration sensor is the sensor center, the activity type comprises walking and running, the activity state comprises entering and exiting the walking state and entering and exiting the running state, the AR virtual sensor registers and monitors with an acceleration sensor and a magnetometer according to the subscription parameters, the acceleration sensor and the magnetometer send the sensing data to the AR virtual sensor after detecting the sensing data, the AR virtual sensor sends the sensing data to an AR algorithm library, analyzing and processing the sensing data through an algorithm in an AR algorithm library, determining an activity type and an activity state, returning the determined activity type and the determined activity state to an AR virtual sensor, judging whether the activity type determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the magnetic calibration sensor by the AR virtual sensor, judging whether the activity state determined by the AR algorithm is the same as the activity type in the subscription parameters issued by the magnetic calibration sensor by the AR virtual sensor, if the activity type determined by the AR virtual sensor is the same as the activity type in the subscription parameters issued by the magnetic calibration sensor by the AR virtual sensor, determining that the activity state determined by the AR algorithm is the same as the activity state in the subscription parameters issued by the magnetic calibration sensor, transmitting an activity recognition result to a third activity recognition example, generating an activity recognition event by the third activity recognition example, and reporting the activity recognition event to the magnetic force calibration sensor, informing the magnetic force calibration sensor to perform subsequent business processing according to the activity recognition event, such as calibration of a magnetometer, and sending a calibration result to a compass application.

Referring to fig. 9-10, a process timing diagram of an activity event processing method according to an embodiment of the application is shown.

S501, initializing a plurality of physical sensors.

S502, initializing an activity recognition module.

S503, the activity recognition module registers the listening availability status with the plurality of physical sensors.

S504, after the initialization of the plurality of physical sensors is completed, the plurality of physical sensors are released to be available.

S505, the plurality of physical sensors notify the activity recognition module that the seating has been completed.

S506, the activity recognition management service of the hardware abstraction layer obtains the unique recognition code of the activity recognition module.

S507, the activity recognition module establishes communication connection with the activity recognition management service according to the unique recognition code.

S508, the client of the application processor issues subscription information to the activity recognition management service.

S509, the activity recognition management service issues a subscription request to the activity recognition module.

S510, the activity recognition module creates an activity recognition instance corresponding to the client of the application processor according to the subscription request.

S511, the activity recognition module registers the sensing data acquisition request with the plurality of physical sensors.

S512, the plurality of physical sensors send the detected sensing data to the activity recognition module.

S513, the activity recognition module sends the received sensing data to the activity recognition algorithm module.

S514, the activity recognition algorithm module analyzes and processes the sensing data to obtain an activity recognition result, and returns the activity recognition result to the activity recognition module.

S515, the activity recognition module sends the activity recognition result to the activity recognition instance corresponding to the application processor.

S516, the activity recognition instance corresponding to the application processor generates an activity event according to the activity recognition result, and reports the activity event to the activity recognition management service of the hardware abstraction layer.

S517, the activity recognition management service of the hardware abstraction layer reports the activity recognition result to the client of the application processor.

S518, the client side of the sensor center acquires the unique identification code of the activity identification module.

S519, the activity recognition module establishes a communication connection with the client of the sensor center according to the unique identification code.

S520, the client of the sensor center issues a subscription request to the activity recognition module.

S521, the activity recognition module creates an activity recognition instance corresponding to the client of the sensor center according to the subscription request.

S522, the activity recognition module registers the sensed data acquisition request with the plurality of physical sensors.

S523, the plurality of physical sensors send the detected sensing data to the activity recognition module.

And S524, the activity recognition module sends the received sensing data to the activity recognition algorithm module.

S525, the activity recognition algorithm module analyzes and processes the sensing data to obtain an activity recognition result, and returns the activity recognition result to the activity recognition module.

S526, the activity recognition module sends the activity recognition result to the activity recognition instance corresponding to the sensor center.

S527, the activity recognition instance corresponding to the sensor center generates an activity event according to the activity recognition result, and reports the activity event to the client of the sensor center.

S528, the client of the modem obtains the unique identification code of the active identification module.

S529, the activity recognition module establishes a communication connection with the modem' S client according to the unique identification code.

S530, the client of the modem issues a subscription request to the activity recognition module.

S531, the activity recognition module creates an activity recognition instance corresponding to the client of the modem according to the subscription request.

S532, the activity recognition module registers the sensor data acquisition requests with the plurality of physical sensors.

S533, the plurality of physical sensors send the detected sensing data to the activity recognition module.

S534, the activity recognition module sends the received sensing data to the activity recognition algorithm module.

S535, the activity recognition algorithm module analyzes and processes the sensing data to obtain an activity recognition result, and returns the activity recognition result to the activity recognition module.

S536, the activity recognition module sends the activity recognition result to the activity recognition instance corresponding to the modem.

S537, the activity recognition example corresponding to the modem generates an activity event according to the activity recognition result, and reports the activity event to the client of the modem.

An embodiment of the present application further provides an electronic device 100, as shown in fig. 11, where the electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile Personal Computer, a UMPC, a netbook, a cellular phone, a personal digital assistant (personal DIGITAL ASSISTANT, PDA), an augmented reality (Augmented Reality, AR) device, a Virtual Reality (VR) device, an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) device, a wearable device, a vehicle-mounted device, a smart home device, and/or a smart city device, and the specific type of the electronic device 100 is not particularly limited.

The electronic device 100 may include A processor 110, an external memory interface 120, an internal memory 121, A universal serial bus (Universal Serial Bus, USB) interface 130, A charge management module 140, A power management module 141, A battery 142, an antenna 1, an antenna 2, A mobile communication module 150, A wireless communication module 160, an audio module 170, A speaker 170A, A receiver 170B, A microphone 170C, an earphone interface 170D, A sensor module 180, keys 190, A motor 191, an indicator 192, A camera 193, A display 194, and A subscriber identity module (Subscriber Identification Module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It should be understood that the illustrated structure of the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), a controller, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, and/or a neural network processor (Neural-network Processing Unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In one embodiment of the application, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to re-use the instructions or data, it may be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In one embodiment of the application, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-INTEGRATED CIRCUIT, I2C) interface, an integrated circuit built-in audio (inter-INTEGRATED CIRCUIT SOUND, I2S) interface, a pulse code modulation (Pulse Code Modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (Mobile Industry Processor Interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (Subscriber Identity Module, SIM) interface, and/or a universal serial bus (Universal Serial Bus, USB) interface, among others.

The I2C interface is a bi-directional synchronous serial bus comprising a serial data line (SERIAL DATA LINE, SDA) and a serial clock line (Derail Clock Line, SCL). In one embodiment of the application, the processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., respectively, through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through an I2C interface, such that the processor 110 communicates with the touch sensor 180K through an I2C bus interface to implement a touch function of the electronic device 100.

The I2S interface may be used for audio communication. In one embodiment of the application, the processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In an embodiment of the present application, the audio module 170 may transmit an audio signal to the wireless communication module 160 through the I2S interface, so as to implement a function of answering a call through the bluetooth headset.

PCM interfaces may also be used for audio communication to sample, quantize and encode analog signals. In one embodiment of the application, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In an embodiment of the present application, the audio module 170 may also transmit an audio signal to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through the bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus for asynchronous communications. The bus may be a bi-directional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment of the present application, a UART interface is typically used to connect the processor 110 with the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In an embodiment of the present application, the audio module 170 may transmit an audio signal to the wireless communication module 160 through a UART interface, so as to realize a function of playing music through a bluetooth headset.

The MIPI interface may be used to connect the processor 110 to peripheral devices such as a display 194, a camera 193, and the like. The MIPI interface includes a camera serial interface (CAMERA SERIAL INTERFACE, CSI), a display serial interface (DISPLAY SERIAL INTERFACE, DSI), and the like. In one embodiment of the present application, processor 110 and camera 193 communicate through a CSI interface to implement the photographing function of electronic device 100. The processor 110 and the display 194 communicate via a DSI interface to implement the display functionality of the electronic device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or as a data signal. In one embodiment of the present application, a GPIO interface may be used to connect processor 110 with camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, an MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transfer data between the electronic device 100 and a peripheral device. And can also be used for connecting with a headset, and playing audio through the headset. The interface may also be used to connect other electronic devices 100, such as AR devices, etc.

It should be understood that the interfacing relationship between the modules illustrated in the embodiments of the present application is only illustrative, and is not meant to limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also employ different interfacing manners in the above embodiments, or a combination of multiple interfacing manners.

The charge management module 140 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charge management module 140 may receive a charging input of a wired charger through the USB interface 130. In some wireless charging embodiments, the charge management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 may also supply power to the electronic device 100 through the power management module 141 while charging the battery 142.

The power management module 141 is used for connecting the battery 142, and the charge management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be configured to monitor battery capacity, battery cycle number, battery health (leakage, impedance) and other parameters. In other embodiments, the power management module 141 may also be provided in the processor 110. In other embodiments, the power management module 141 and the charge management module 140 may be disposed in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (Low Noise Amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can amplify the signal modulated by the modem processor, and convert the signal into electromagnetic waves through the antenna 1 to radiate. In an embodiment of the present application, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110. In an embodiment of the present application, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or video through the display screen 194. In one embodiment of the application, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional module, independent of the processor 110.

The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (Wireless Local Area Networks, WLAN) (e.g., wireless fidelity (WIRELESS FIDELITY, wi-Fi) network), bluetooth (BT), global navigation satellite system (Global Navigation SATELLITE SYSTEM, GNSS), frequency modulation (Frequency Modulation, FM), near field communication (NEAR FIELD communication), infrared (IR), etc., as applied to the electronic device 100. The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.

In one embodiment of the application, antenna 1 and mobile communication module 150 of electronic device 100 are coupled, and antenna 2 and wireless communication module 160 are coupled, such that electronic device 100 may communicate with a network and other devices via wireless communication technology. The wireless communication techniques may include the Global System for Mobile communications (Global System For Mobile Communications, GSM), general Packet Radio Service (GPRS), code division multiple Access (Code Division Multiple Access, CDMA), wideband code division multiple Access (Wideband Code Division Multiple Access, WCDMA), time division multiple Access (time-Division Code Division Multiple Access, TD-SCDMA), long term evolution (Long Term Evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (Global Positioning System, GPS), a global navigation satellite system (Global Navigation SATELLITE SYSTEM, GLONASS), a Beidou satellite navigation system (Beidou Navigation SATELLITE SYSTEM, BDS), a quasi-zenith satellite system (Quasi-zenith SATELLITE SYSTEM, QZSS) and/or a satellite based augmentation system (SATELLITE BASED AUGMENTATION SYSTEMS, SBAS).

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor that processes active events, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. The display panel may employ a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic LIGHT EMITTING Diode (AMOLED), a flexible Light-Emitting Diode (Flex), miniled, microled, micro-OLED, quantum Dot LIGHT EMITTING Diodes (QLED), or the like. In one embodiment of the present application, the electronic device 100 may include 1 or N display screens 194, N being a positive integer greater than 1.

The electronic device 100 may implement photographing functions through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process data fed back by the camera 193. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In an embodiment of the present application, the ISP may be provided in the camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (Charge Coupled Device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In one embodiment of the application, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to fourier transform the frequency bin energy, or the like.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: dynamic picture experts group (Moving Picture Experts Group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural Network (Neural-Network, NN) computing processor, and can rapidly process input information by referencing a biological neural Network structure, such as referencing a transmission mode among human brain neurons, and can continuously learn. Applications such as intelligent awareness of the electronic device 100 may be implemented through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, etc.

The internal memory 121 may include one or more random access memories (Random Access Memory, RAM) and one or more non-volatile memories (NVM).

The random access memory may include static random-access memory (SRAM), dynamic random-access memory (Dynamic Random Access Memory, DRAM), synchronous dynamic random-access memory (Synchronous Dynamic Random Access Memory, SDRAM), double data rate synchronous dynamic random-access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM, e.g., fifth generation DDR SDRAM is commonly referred to as DDR5 SDRAM), etc.;

The nonvolatile memory may include a disk storage device, a flash memory (flash memory).

The FLASH memory may include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. divided according to the operation principle, may include single-level memory cells (single-LEVEL CELL, SLC), multi-level memory cells (multi-LEVEL CELL, MLC), triple-level memory cells (triple-LEVEL CELL, TLC), quad-LEVEL CELL, QLC), etc. divided according to the memory specification, may include universal FLASH memory (Universal Flash Storage, UFS), embedded multimedia memory card (embedded multi MEDIA CARD, EMMC), etc.

The random access memory may be read directly from and written to by the processor 110, may be used to store executable programs (e.g., machine instructions) for an operating system or other on-the-fly programs, may also be used to store data for users and applications, and the like.

The nonvolatile memory may store executable programs, store data of users and applications, and the like, and may be loaded into the random access memory in advance for the processor 110 to directly read and write.

The external memory interface 120 may be used to connect external non-volatile memory to enable expansion of the memory capabilities of the electronic device 100. The external nonvolatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and video are stored in an external nonvolatile memory.

The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. One or more computer programs are configured to be executed by processor 110. The one or more computer programs include a plurality of instructions that when executed by the processor 110, implement the screen display detection method performed on the electronic device 100 in the above-described embodiments to implement the screen display detection function of the electronic device 100.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In an embodiment of the present application, the audio module 170 may be disposed in the processor 110, or a part of functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also referred to as a "horn," is used to convert audio electrical signals into sound signals. The electronic device 100 may listen to music, or to hands-free conversations, through the speaker 170A.

A receiver 170B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal. When electronic device 100 is answering a telephone call or voice message, voice may be received by placing receiver 170B in close proximity to the human ear.

Microphone 170C, also referred to as a "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can sound near the microphone 170C through the mouth, inputting a sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, and may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may also be provided with three, four, or more microphones 170C to enable collection of sound signals, noise reduction, identification of sound sources, directional recording functions, etc.

The earphone interface 170D is used to connect a wired earphone. The headset interface 170D may be a USB interface 130 or a 3.5mm open mobile electronic device 100 platform (Open Mobile Terminal Platform, OMTP) standard interface, a american cellular telecommunications industry association (Cellular Telecommunications Industry Association of the USA, CTIA) standard interface.

The keys 190 include a power-on key, a volume key, etc. The keys 190 may be mechanical keys. Or may be a touch key. The electronic device 100 may receive key inputs, generating key signal inputs related to user settings and function controls of the electronic device 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration alerting as well as for touch vibration feedback. For example, touch operations acting on different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects by touching different areas of the display screen 194. Different application scenarios (such as time reminding, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

The indicator 192 may be an indicator light, may be used to indicate a state of charge, a change in charge, a message indicating a missed call, a notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card may be inserted into the SIM card interface 195, or removed from the SIM card interface 195 to enable contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 195 may support Nano SIM cards, micro SIM cards, and the like. The same SIM card interface 195 may be used to insert multiple cards simultaneously. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as communication and data communication. In one embodiment of the application, the electronic device 100 employs esims, namely: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100. The embodiment of the present application further provides a computer storage medium, in which computer instructions are stored, which when executed on the electronic device 100, cause the electronic device 100 to execute the above-mentioned related method steps to implement the activity event processing method in the above-mentioned embodiment.

The embodiment of the application also provides a computer program product, which when run on a computer, causes the computer to execute the related steps to realize the activity event processing method in the embodiment.

In addition, the embodiment of the application also provides a device, which can be a chip, a component or a module, and the device can comprise a processor and a memory which are connected; the memory is configured to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the activity event processing method in the above method embodiments.

The electronic device, the computer storage medium, the computer program product, or the chip provided in this embodiment are used to execute the corresponding methods provided above, so that the beneficial effects thereof can be referred to the beneficial effects in the corresponding methods provided above, and will not be described herein.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts shown as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or, that is, a part or all of the technical solution contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present application without departing from the spirit and scope of the technical solution of the present application.

Claims

1. An activity event processing method applied to an electronic device, wherein the method comprises the following steps:

initializing an activity recognition module and a plurality of physical sensors of the electronic device;

Establishing a communication connection between the activity recognition module and a client in a plurality of hardware devices of the electronic device, wherein the plurality of hardware devices comprise an application processor;

the method comprises the steps that clients in the plurality of hardware devices respectively send subscription requests to an activity recognition module, the activity recognition module responds to the subscription requests to create an activity recognition instance corresponding to each hardware device, wherein the clients of the application processor send subscription parameters to an activity recognition management service in a hardware abstraction layer of the electronic device, the activity recognition management service generates subscription requests according to the subscription parameters and sends the subscription requests to the activity recognition module, and the activity recognition module creates the activity recognition instance corresponding to the application processor according to the subscription requests;

the activity recognition module acquires sensing data of the plurality of physical sensors and determines an activity recognition result according to the sensing data;

And if the activity recognition result is matched with at least one activity recognition instance, reporting the activity recognition result to a client corresponding to the at least one activity recognition instance, wherein if the activity recognition result is matched with the activity recognition instance corresponding to the application processor, the activity recognition instance corresponding to the application processor reports the activity recognition result to the client of the application processor.

2. The activity event processing method of claim 1, wherein initializing an activity recognition module of the electronic device comprises:

and creating the activity recognition module through a coprocessor of the electronic equipment, and configuring parameters of the activity recognition module, wherein the parameters of the activity recognition module comprise the name of the activity recognition module, the type of the activity to be recognized, the data type and the unique identifier.

3. The activity event processing method of claim 1, wherein initializing a plurality of physical sensors of the electronic device comprises:

acquiring a sensor manager, and acquiring a physical sensor list through the sensor manager;

and acquiring the object of each physical sensor in the physical sensor list, and issuing the available identification of the corresponding physical sensor through the object of each physical sensor.

4. The activity event processing method of claim 1, wherein the method further comprises:

the activity recognition module registers a listening availability status with the plurality of physical sensors.

5. The activity event processing method of claim 4, wherein the activity recognition module registering a listening availability status with the plurality of physical sensors comprises:

The activity recognition module obtains an instance of a sensor manager, registers a sensor listener for each physical sensor, and registers the sensor listener to the sensor manager, thereby listening for available states of the plurality of physical sensors through the sensor listener.

6. The activity event processing method of claim 1, wherein the establishing a communication connection between the activity recognition module and a client in a plurality of hardware devices of the electronic device comprises:

the clients of the plurality of hardware devices send the data types and the unique identifiers of the activity recognition modules to the activity recognition modules, and request to establish communication connection with the activity recognition modules;

The activity recognition module verifies the data type and the unique identifier sent by the client;

If the data type sent by the client is the same as the data type of the active identification module, and the unique identifier sent by the client is the same as the unique identifier of the active identification module, determining that the data type and the unique identifier sent by the client pass verification, and establishing communication connection between the active identification module and the client.

7. The activity event processing method of claim 1, wherein the activity recognition management service generating a subscription request according to the subscription parameters and issuing the subscription request to the activity recognition module comprises:

The activity recognition management service encapsulates the subscription parameters and the subscription instructions to generate the subscription request, the subscription request is issued to a sensor center of a coprocessor of the electronic device, and the sensor center sends the subscription request to the activity recognition module.

8. The activity event processing method of claim 1, wherein the plurality of hardware devices includes a sensor hub, clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, and the activity recognition module responding to the subscription requests to create the activity recognition instance corresponding to each hardware device includes:

The client of the sensor center issues a subscription request to the activity recognition module, wherein the subscription request comprises subscription parameters and subscription instructions;

and the activity recognition module responds to the subscription instruction, and creates an activity recognition instance corresponding to the sensor center according to the subscription parameter.

9. The activity event processing method of claim 1, wherein the plurality of hardware devices includes a modem, clients in the plurality of hardware devices respectively send subscription requests to the activity recognition module, and the activity recognition module responding to the subscription requests, creating the activity recognition instance corresponding to each hardware device includes:

the client of the modem issues a subscription request to the activity recognition module, wherein the subscription request comprises subscription parameters and subscription instructions;

and the activity recognition module responds to the subscription instruction and creates an activity recognition instance corresponding to the modem according to the subscription parameter.

10. The activity event processing method according to any of claims 7 to 9, wherein the subscription parameters include client information, activity type and activity status.

11. The activity event processing method of claim 10, wherein the method further comprises:

If the activity type in the activity recognition result is the same as the activity type in at least one subscription parameter, and the activity state in the activity recognition result is the same as the activity state in the at least one subscription parameter, determining that the activity recognition result is matched with the at least one subscription parameter, and further determining that the activity recognition result is matched with at least one activity recognition instance corresponding to the client information in the at least one subscription parameter.

12. The activity event processing method of claim 1, wherein the activity recognition module obtaining sensing data of the plurality of physical sensors and determining an activity recognition result from the sensing data comprises:

The activity recognition module determines sensing data required by each client according to subscription parameters of each client, and registers a data acquisition request to a physical sensor for detecting the sensing data;

After registering the data acquisition request, the physical sensor sends the detected sensing data to the activity recognition module;

The activity recognition module receives the sensing data and sends the sensing data to an activity recognition algorithm module;

The activity recognition algorithm module obtains an activity recognition result based on the sensing data and returns the activity recognition result to the activity recognition module.

13. An electronic device, the electronic device comprising a memory and a processor:

Wherein the memory is used for storing program instructions;

the processor configured to read and execute the program instructions stored in the memory, which when executed by the processor, cause the electronic device to perform the activity event processing method according to any of claims 1 to 12.

14. A chip coupled to a memory in an electronic device, wherein the chip is configured to control the electronic device to perform the activity event processing method of any of claims 1-12.

15. A computer storage medium storing program instructions which, when run on an electronic device, cause a processor of the electronic device to perform the activity event processing method of any of claims 1 to 12.