CN111562972A

CN111562972A - Ubiquitous operating system for crowd sensing

Info

Publication number: CN111562972A
Application number: CN202010330505.XA
Authority: CN
Inventors: 於志文; 刘一萌; 郭斌; 廖佳豪; 苏江宾
Original assignee: Northwestern Polytechnical University
Current assignee: Northwestern Polytechnical University
Priority date: 2020-04-24
Filing date: 2020-04-24
Publication date: 2020-08-21
Also published as: WO2021213293A1

Abstract

The invention provides a ubiquitous operating system facing crowd-sourcing perception, in particular to a CrowdOS, which further designs a set of comprehensive processing mechanism and core functional components by deeply analyzing the complex environment and diversified characteristics of crowd-sourcing tasks; the method comprises three core mechanisms, namely task semantic analysis, user scheduling, system resource management and task result deep feedback interaction. According to the method, a CrowdOS is utilized to solve the problems that an existing mobile crowd-sourcing sensing or crowdsourcing platform lacks a unified system structure and algorithms or modules in related researches are incompatible, a task analysis and scheduling mechanism establishes a bridge between a task and an OS kernel through a task resource map, and a reasonable allocation strategy is selected in a self-adaptive mode aiming at heterogeneous tasks; the resource management mechanism abstracts heterogeneous physical and virtual resources in the system and provides uniform software definition and management; the result quality optimization mechanism quantifies and optimizes the quality of the results.

Description

Ubiquitous operating system for crowd sensing

Technical Field

The invention relates to the fields of mobile computing and ubiquitous operating systems and the field of crowd sensing, in particular to an operating system based on a system bottom layer architecture.

Background

With the rise of crowd-sourced and crowd-sourced application technologies, a great deal of platforms and application software based on crowd-sourced ideas are emerging, such as Amazon Mechanical turn [ documents "m.buhrmester, t.kwang, and s.d.gosling," Amazon's Mechanical-effect turn: a new source of inexpensiveness, yet high-quality, data? "Perspectives on clinical science, vol.6, No.1, pp.3-5,2011.", CrowdFlower, food and travel platform, and information analysis platform. In addition, a plurality of crowdsourced application programs such as water quality detection, air quality detection and traffic jam condition investigation are adopted to solve the problems in different fields through a crowd-sourcing idea. The applications and technologies developed by the crowdsourcing idea are summarized into the first generation crowd-sourcing technology, and the characteristics of the technology comprise: tasks are issued through an internet platform, and a large-scale problem is solved by using a problem segmentation idea. These platforms themselves act as a vehicle for task distribution and result collection, do not contain analysis and evaluation for the tasks themselves, and do not optimize the quality of the results collected by the platform.

Along with the development of the crowd sourcing application and the related technologies of mobile terminals, portable sensing devices and the like, the technology is summarized as a second generation crowd sourcing technology. These techniques are applied in The fields of environmental monitoring, utility monitoring, etc., such as The documents "R.K. Ganti, F.Ye, and H.Lei", "Mobile crown sensing: current state and future exchange", "IEEEcommunications Magazine, vol.49, No.11, pp.32-39,2011", and The documents "B.Guo, Z.Wang, Z.Yu, Y.Wang, N.Y.Yen, R.Huang, and X.ZHou", "Mobile crown sensing and calculating: The viewing of an empirical human-powered sensing apparatus," ACMComputing surys (CSUR), vol.48, No.1, p.7. More well-known systems include CommonSense, Ear-Phone, Chimera, Creekwatch, and PhotoCity, among others. In addition, related crowd sensing technologies are also widely studied. For example, Wang et al studied the problem of multitask collaborative allocation in crowd-sourcing applications. Guo et al present challenges to optimizing the sensed data. Restuccia et al studied quality improvement methods of crowd-sensing data. In addition, the incentive mechanism and privacy protection of crowdsourcing workers have been a problem of intensive research. Most of these applications and techniques are used in scientific research to target the acquisition of certain sensor data. The developed software can only complete a single task, cannot be reused and cannot be migrated to other tasks. These techniques are also mostly performed under ideal circumstances based on a number of assumed conditions, and only simple simulation experiments are completed. The study did not consider the combination with the actual platform. By integrating theoretical analysis and actual experience, the invention deeply considers the difficulties faced by the first generation and the second generation crowd-sourcing technology in development and popularization, and expects to research a uniform architecture to intensively solve the difficulties.

Difficulties and challenges faced by current platforms include:

1) a framework capable of handling multiple types of tasks is lacking and needs to enable a unified and deep understanding of the tasks. The crowdsourcing platform is usually just used as an internet bulletin board for centrally publishing and collecting tasks, and lacks a function of treating different tasks differently. Most current crowd sensing applications use custom software and specific sensing devices to perform tasks, the applications and tasks being in a one-to-one binding relationship. Therefore, the application software generally lacks universality and expansibility and is difficult to migrate to other types of tasks.

2) There is a lack of abstraction and unified management of various resources in crowd-sourcing systems. The system cannot jointly analyze and schedule people, tasks, equipment resources, software resources, and generated knowledge data and other resources. Most of the current technical researches are carried out to solve specific problems under ideal environment, and are based on a plurality of conditional assumptions. Since the assumed conditions and scene settings of the research are different, the technologies are isolated from each other. Due to the lack of communication bridges among these scattered studies, the local methods proposed by various scholars are difficult to be popularized to practical applications.

3) Results quality assessment and optimization methods are lacking. The crowdsourcing platform generally only summarizes the task result data and does not further evaluate and analyze. Some simple result screening methods may be preset in the crowd-sourcing application, but since these methods are usually specific to a specific type of task or data, they are difficult to generalize to other tasks or data types and cannot support complex data processing tasks (such as data semantic understanding). Some optimization methods are performed after the task publisher takes the results, such as cleaning and screening of data by the user, and the operation is independent of the publishing platform or application, and the data aggregation cost of crowd sensing cannot be reduced.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides a ubiquitous operating system facing crowd sensing, in particular to a ubiquitous operating system CrowdOS facing crowd sensing, and a set of comprehensive processing mechanism and core functional components are designed based on an operating system framework through deep analysis of the complex environment and diversified characteristics of crowd sensing tasks. As an incubator and accelerator for crowd sourcing applications, CrowdOS runs as middleware between the native operating system and the application layer of a heterogeneous device. The invention focuses on the crowdOS core architecture and its three core mechanisms, namely: and performing task semantic analysis, user scheduling, system resource management and task result deep feedback interaction.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a ubiquitous operating system facing crowd sensing is characterized in that a task publisher inputs original task data through an intelligent terminal and submits the original task data to a platform, and the platform captures a task, analyzes the task and assigns a unique task ID to the task; after a task enters a platform, the task is analyzed and corresponding task feature vectors are generated, feature splicing is carried out on the task feature vectors and other known discrete features, and characteristics are extracted through the task vectors, wherein the characteristics include but are not limited to task types, the number of required participants, task execution places, sensors required to be used and types of collected data; the ubiquitous operating system facing the crowd sensing completes the processes of user scheduling and task allocation by executing task reasoning, association and matching operations; after receiving the tasks, the participants select interested tasks to execute and upload the acquired perception data or design scheme to a ubiquitous operating system facing crowd sensing; after data enter the ubiquitous operating system facing the crowd sensing, abstracting and defining software for tasks, users and process resources, selecting task middleware required to be used by the ubiquitous operating system facing the crowd sensing according to the description of task characteristic information, and summarizing the collected data; and finally, returning the result to the task publisher, evaluating and feeding back the task result by the publisher, and finishing the life cycle of the crowd sourcing task after the publisher receives the result.

The ubiquitous operating system facing the crowd sensing is realized by adopting an operating system CrowdOS, the CrowdOS runs between a native operating system and an upper application and comprises a sensing end and a service end, a software carrier of the sensing end comprises two types of equipment, the first type is portable intelligent sensing equipment with a human-computer interaction function, and the second type is a fixed sensor deployed in the physical world; the CrowdOS adopts a cloud-edge-end deployment mode; the sensing end is deployed on various terminal sensing devices and collects data information, and the server end is deployed on a cloud server or an edge server and performs comprehensive management on system resources and data resources and responds to system operation in real time;

in the functional layer part of the sensing end, firstly, a publisher uploads a task to a system based on a CrowdOS through an interactive function of an intelligent terminal, and a participant browses and executes the task published to the system through the sensing end; when the participants receive the tasks, the sensing end starts to acquire data; the system support layer acquires the current equipment state, then uniformly performs interface packaging on the acquired sensor data and unifies the data transmission format, and then stores the data into a corresponding data structure body in a network or Bluetooth transmission mode; for sensing equipment without human-computer interaction, once the equipment is activated by a task in a system and passes task verification, the equipment starts to automatically collect and upload sensing data according to a preset rule;

the Server-end provides comprehensive management service, is deployed on a Server cluster, a cloud Server or an edge Server, and comprises eight modules, namely a Task Pool module (Task Pool module), a resource management module (resource management), a data management center module (DM center), an internal and external interface module (I & E interface), a Knowledge base module (Knowledge base), a System plug-in module (System plug), Task result quality optimization (TRO) and a combined Storage and retrieval module (Storage and query); firstly, after a task publisher transmits task data to a server from a sensing end, the server understands and represents the tasks through a task pool module, analyzes, schedules, distributes and finely tunes the received tasks through the task pool module, and distributes crowd-sourcing tasks to users in a platform; in the resource management module, software definition is carried out on various resources and comprehensive management of equipment, users, environment and task process scheduling is completed; then entering a data management center module, wherein the module provides classified storage and quick retrieval functions of massive heterogeneous data, a combined storage and retrieval module stores and processes data acquired through a sensing end, extracts useful information and transmits the useful information to a knowledge base module; for the collected data, the data quality is optimized through a task result quality optimization module, and after the optimization is completed, the server feeds the final result back to a task publisher of the sensing end through a network, so that the whole task message is circulated between the sensing end and the server; the system plug-in module provides privacy protection, credit evaluation and user incentive for a user; in the internal and external interface module, the internal interface is used for maintaining and updating the operating system, and the external interface is provided for the third-party application and used for calling the software interface.

In the CrowdOS, five dynamic Agents are constructed to generate a system resource map after a Task enters to manage the Task and the resource in the system, wherein the Agents comprise a Task Agent Task-Agent (TA), a User Agent User-Agent (UA), an equipment Agent Device (DA), an Environment Agent Environment-Agent (EA) and a Process Agent Process-Agent (PA); wherein, the TA contains detailed information of each task, including but not limited to task category, execution time, location, collected data format, and the UA is abstract description of the user in the system, and records the information of the user, including but not limited to issued and executed tasks, credit level; DA is description of terminal equipment resources, and types and current state information of the equipment are recorded; EA abstracting the software and hardware environment resources of the current system, including but not limited to the current CPU, memory, storage and use conditions, the number of users in the system and the total amount of available equipment; the PA manages all task processes existing in the current system, including but not limited to process states, priorities and scheduling strategies; agents complete real-time interaction and updating among each other through semaphores defined in the internal structure after acquiring task information.

In the task pool, a task information extraction part firstly carries out semantic analysis and feature extraction on a task when a crowd-sourcing task enters a system, carries out natural language analysis on the received task, carries out word segmentation processing on the task described by the language, then carries out operation without distinguishing languages by the system, finally extracts task key information including but not limited to the execution mode, the place, the time and the number of participants of the task, splices the extracted task information and discrete features obtained by regular click selection, inputs the spliced features into a deep neural network for uniform coding, and outputs a high-dimensional task intermediate vector; and finally, mapping the vector to the Task-agent of the Task through decoding to finish the conversion process from the Task vector to the agents.

In the task pool, an Agents generation part comprises 5 Agents: the Task _ agent, the Environment _ agent, the Process _ agent, the Device _ agent and the User _ agent are obtained by Task analysis and representation processes; wherein, the taskID is a unique identifier of the task in the system; the process _ state represents the current state of the task process, whether the task process is in a generating state, an executing state or a feedback state and the like; the state assists the Process _ agent to manage the task Process, the Prio represents the priority of the task, the value is 0-15, and the system schedules the task Process according to the priority sequence; taskInfo is a structure containing detailed information of tasks, such as task time, location, vector representation, etc.; the Classication represents the category of the task, such as a data annotation class, a sensing information acquisition class, a questionnaire answer class and the like; topic represents the subject of the task and can be extracted from keyword information, such as audio collection, photo collection and the like; deviceNum, deviceInfo, deviceID represent the number of devices participating in the execution of the task, device detailed information, and device ID list, respectively; sensing _ Data is a collected task Data set pointer to the cube address where the Data is stored.

In the task pool, a scheduling and distributing part, and a task scheduling subframe comprises a strategy base, a mapping model and a strategy management module; firstly, analyzing and reasoning the content of a task resource map, mapping a task ID into a strategy library, and storing a task allocation function and a serial number thereof in the strategy library so as to complete the task allocation strategy selection process; and then, scheduling operation is carried out on the equipment and the users according to the selected distribution strategy, and the tasks are pushed to the proper users.

The resource management module comprises but is not limited to a user, a perception terminal, a system environment, a task process, system software, task data and a knowledge base, abstracts a management object of the crowdOS into 4 types, and executes the management object according to different periods of the task entering the system, and the resource management module specifically comprises the following steps:

1) device, user and environment management

When the sensing end is accessed through a network, a signal is triggered, the equipment automatically sends current state information to the system, wherein the current state information comprises but is not limited to equipment type, residual electric quantity, position information and storage occupancy rate, and the state information is captured and stored by the system developed based on a CrowdOS framework through a Device-agent;

the system draws a User portrait through the User-agent, the User interacts with the system through the sensing end, the User-agent stores the name, age and participated tasks of the User, and meanwhile, the User credit level, User preference, interest fields and other personalized information can be generated according to the situation that the User participates in the tasks;

the Environment resource records the server architecture and the processing capacity, the resources are stored in an Environment-agent and are updated at regular time, the agent has an alarm function, pre-judgment is carried out according to the current system state and the increase and decrease conditions of the task quantity, and if the CPU utilization rate or the storage occupancy rate of the system reaches a rated threshold value, the system gives an alarm.

2) Task process scheduling management

The Process-agent is a set of state information of the current stage of the task, and a Process identification number is distributed to each task to serve as a unique mark of the task Process in the system; the ID is simultaneously stored in the Task-agent and the Process-agent, and accompanies the whole life cycle of the Task; the PA class includes task process information, including but not limited to TPID: a process unique identifier; process _ state: the current task is in seven switchable states in the system; process _ stream: a process scheduling policy including First Come First Served (FCFS), round Robin (RB), task priority, highest response ratio priority (HRRN), and feedback priority; process _ prio represents the process priority, and from 0 to 15, 0 represents the highest priority and decreases in sequence;

the task process state comprises seven states of a creation state, a generation state, a distribution state, an execution state, a processing state, a feedback state and a termination state; the flow relationship between states is: firstly, when a user publishes a task in a system, a task creating state is firstly adopted; entering a task generating state after system authentication and analysis; entering a distribution state after task scheduling and distribution; after the release is finished, the participants can execute the tasks and then enter an execution state; the participants upload the collected data to the system for processing and enter a processing state; summarizing the result to a task publisher through data visualization presentation, and entering a feedback state; when the task result is not verified by the publisher, the system stays in the feedback state temporarily, and then the task process returns to the generation state, the distribution state or the processing state again after reasoning and correction, and then is executed in sequence; after the publisher verifies the task submission result, the task enters a termination state, and the life cycle of the whole task is finished;

the task process scheduling algorithm in the system is selected from one of the following methods:

1) first Come First Served (FCFS): the task which enters the system first is processed preferentially, and resources and services are provided for the task;

2) a circulation method: generating task interruption at a periodic interval, placing a currently running process in a task ready queue, and selecting a next ready process to run based on the FCFS;

3) task priority: processing high-level tasks in priority, wherein the tasks with the same priority are in accordance with the FCFS principle;

4) highest response ratio first (HRRN), R ═ w + s)/s, where R denotes the response ratio, w denotes the time that has been waiting, and s denotes the time that is expected to be served;

5) feedback priority: for the task entering the feedback state, two levels are adjusted up on the basis of the original priority, and system resources are used preferentially;

3) heterogeneous multimodal data resource management

For the management of multimodal data, the steps are as follows:

1) collecting and storing data;

2) constructing a retrieval method of unstructured data based on crowd sourcing, and once data preparation is completed, starting to construct a data stack;

3) managing and storing task data by adopting a data cube technology, constructing a multi-feature cube structure (MC), and retrieving unstructured data according to the constructed data cube;

4) with the increase of the task amount in the system, the completed task data and the intermediate data are regularly cleaned, and the data subjected to deep analysis is transferred to a knowledge base for management;

4) knowledge base management

The knowledge of the OS falls into two categories: prior knowledge (EK) and New Knowledge (NK); the NK is extracted from the task or data, which helps to improve the system mechanism or update the model. The process of knowledge management is as follows:

firstly, the system distinguishes the knowledge which can improve the system performance, update the model or improve the task result quality from the prior knowledge, and the information or the knowledge which is useful for the user or a third party is not included in the knowledge range; secondly, for knowledge to be discovered, the system provides a corresponding mechanism or algorithm to mine the knowledge, wherein the mechanism or algorithm comprises but is not limited to a deep learning algorithm, an online updating algorithm and a transfer learning algorithm; thirdly, the knowledge base generalizes the knowledge and stores the knowledge in the database according to the type, form and abstraction level of the knowledge. Knowledge is not centrally stored in some management list or module in the system, it exists in distributed fashion among various modules or lists in the system. The knowledge base mainly records knowledge addresses and the intrinsic relations among knowledge, and establishes a knowledge network according to the knowledge addresses and the intrinsic relations.

The result quality optimization module comprises an interaction layer, a reasoning layer and an execution layer; firstly, in an interaction layer, a publisher evaluates a task result by inputting evaluation information or clicking a button on a human-computer interface, and analyzes diversified evaluation contents; if the evaluation shows acceptance or satisfaction, executing an ending instruction, and ending the task; if the quality problem exists in the result through analysis, the next layer is entered; secondly, entering an inference layer, performing key information extraction and deep analysis operation on feedback contents of a publisher, inferring possible reasons of quality problems, mapping the reasons into a problem coding library according to an inference model established by the system, and finding corresponding error codes; thirdly, entering an execution layer, and mapping the problem codes and corresponding internal operations; after the correction is finished, the task enters a new process state, the task result corrected through various ways enters the interaction layer again, the interaction layer feeds back the result to the publisher, the interaction layer waits for a new evaluation result to be given, the whole optimization process is executed in sequence according to steps and forms a closed loop, and the process is terminated until the publisher is satisfied with the task result.

The invention has the beneficial effects that a ubiquitous operating system facing crowd-sourcing perception is provided, and the problem that the existing mobile crowd-sourcing perception or crowdsourcing platform lacks a uniform system structure and algorithms or modules in related researches are incompatible is solved by using a CrowdOS. The composition of the architecture and the contents contained in each module and the relationship between the modules are described in detail. In addition, the implementation idea of three core mechanisms in the CrowdOS kernel architecture is explained in detail: the task analysis and scheduling mechanism establishes a bridge between a task and an OS kernel through a task resource map, and then adaptively selects a reasonable allocation strategy aiming at the heterogeneous task; the resource management mechanism abstracts heterogeneous physical and virtual resources in the system and provides uniform software definition and management for the physical and virtual resources; the result quality optimization mechanism aims to quantify and optimize the quality of the results through quality assessment and shallow depth inference mechanisms and strategies that integrate specific quality issues.

By being based onAnd the method for developing the application example by the architecture evaluates the correctness of the CrowdOS, the effectiveness of the kernel module and the overall development efficiency, and compares the optimization speed and the energy consumption of the results before and after use. CrowdOS and the application instance WeSense developed based on this architecture were evaluated mainly from four aspects: correctness and efficiency (Ev)₁) Availability and availability (Ev)₂) Optimizing the result quality assessment (E v)₃) Performance, load and stress test (Ev)₄)。

Drawings

FIG. 1 is a block diagram of a crowd sensing ecosystem.

FIG. 2 is a block diagram of a CrowdOS core architecture.

Fig. 3 is a system resource map.

FIG. 4 is a task parsing and scheduling framework diagram.

Fig. 5 is a task process state switching diagram.

Fig. 6 is a diagram of a result quality optimization framework.

FIG. 7 is a WeSense availability assessment. FIG. 7(a) is a display and search of a home page, crowd sourcing task; FIG. 7(b) is a task detail page; FIG. 7(c) is a task submission page.

Fig. 8 is a development efficiency evaluation graph in which fig. 8(a) is time required to complete the f1-3 test with GA and GB, and fig. 8(b) is a time consumption comparison diagram of all tests based on M1 and M2.

FIG. 9 is a core framework validity assessment, the task in FIG. 9(a) being a graph of the effect of random assignment; FIG. 9(b) is an effect of using a location-based task assignment algorithm, and FIG. 9(c) is a comparison of the effects of the two methods; fig. 9(d) is a diagram illustrating selection of the super privacy preserving mode.

FIG. 10 is an optimized time-versus-time diagram, wherein FIG. 10(a) is a data format correction request interface and FIG. 10(b) is a correction prompt message received by a participant; FIG. 10(c) is a graph comparing the time consumption of the two optimization methods.

Figure 11 comparative plot of performance and pressure testing.

Detailed Description

The invention is further illustrated with reference to the following figures and examples.

The invention mainly comprises a core architecture of CrowdOS and three important mechanisms. The main contributions and innovation points of CrowdOS are as follows:

first, a crowd-sourcing aware operating system framework was designed and the core architecture of crowdOS was introduced in detail.

Secondly, the system analyzes the crowd-sourcing task based on the natural language processing correlation technique and performs fine-grained semantic analysis and modeling on the task by combining discrete features. Therefore, a bridge for communication between tasks and the system, namely a task analysis and scheduling mechanism is established. The method is based on natural language interactive understanding, and solves the problem that the platform mentioned in the first challenge only has a summary function on task results or can only process a single task through a template.

Third, the operating system abstracts and software defines the various resources (user resources, task resources, system resources) required for crowd sourcing task execution. Thereby establishing a system resource map. And a foundation is provided for the uniform and efficient management of various resources in the system. The resource management mechanism solves the second challenge.

Fourthly, aiming at the problem of task result quality, the system provides an evaluation method and an optimization mechanism, namely a result quality optimization mechanism. The mechanism is based on the idea of deep human-computer fusion, and mainly solves the two quality problems of sparse results and high result error rate by using methods such as natural language feedback interaction, deep layer reasoning and the like. The human-computer cooperative task result optimization mechanism solves a third challenge.

A ubiquitous operating system oriented to crowd sensing, as shown in fig. 1, is a building diagram of a crowd sensing ecosystem, and shows a task execution process and a life cycle. The ubiquitous operating system architecture facing the crowd sensing comprises a) a server cluster, b) an intelligent terminal, c) sensing equipment, d) a sensor, e) a communication network, f) a basic software layer, g) a platform and application software, and h) participants and crowd sensing tasks, wherein a-f serve as carriers of software or hardware infrastructure supports. The lifetime and flow of the ubiquitous operating system facing crowd sensing are shown in fig. 1, a task publisher inputs original task data through an intelligent terminal and submits the original task data to a platform, and the platform captures a task, analyzes the task and assigns a unique task ID to the task; after a task enters a platform, the task is analyzed and corresponding task feature vectors are generated, feature splicing is carried out on the task feature vectors and other known discrete features, and characteristics are extracted through the task vectors, wherein the characteristics include but are not limited to task types, the number of required participants, task execution places, sensors required to be used and types of collected data; the ubiquitous operating system facing the crowd sensing completes the processes of user scheduling and task allocation by executing task reasoning, association and matching operations; after receiving the tasks, the participants select interested tasks to execute and upload the acquired perception data or design scheme to a ubiquitous operating system facing crowd sensing; after data enter the ubiquitous operating system facing the crowd sensing, abstracting and defining software for tasks, users and process resources, selecting task middleware required to be used by the ubiquitous operating system facing the crowd sensing according to the description of task characteristic information, and summarizing the collected data; and finally, returning the result to the task publisher, evaluating and feeding back the task result by the publisher, and finishing the life cycle of the crowd sourcing task after the publisher receives the result.

In order to solve the circulation and execution problems of various crowd-sourcing tasks in a crowd-sourcing ecosystem, the invention designs an operating system CrowdOS to realize a ubiquitous operating system facing crowd-sourcing perception, and the CrowdOS core architecture and three subsystem frameworks contained in the core architecture are as follows:

fig. 2 is a block diagram of a core architecture of a CrowdOS, where the ubiquitous operating system for crowd Sensing is implemented by using an operating system CrowdOS, and it can be seen from fig. 2 that the CrowdOS operates between a native operating system and an upper application and includes a Sensing end and a service end, where a software carrier of the Sensing end (Sensing-end) includes two types of devices, where the first type is a portable intelligent Sensing device with a human-computer interaction function, such as a smart phone, a smart watch, and the like; the second category is fixed sensors deployed in the physical world, which do not need to interact directly with people, such as automotive sensors, water quality detection sensors, air quality sensors, etc.; the CrowdOS adopts a cloud-edge-end deployment mode; the sensing end is deployed on various terminal sensing devices and used for collecting data information of environment, business, society, crowd and the like, and the server end is deployed on a cloud server or an edge server and used for carrying out comprehensive management on system resources and data resources and responding to system operation in real time; the system software deployed on the edge server is usually cut and lightweight, for example, redundant modules are removed, and only the most core data processing part, visual comprehensive processing and other functions are reserved and still deployed on the cloud server.

a Server-end provides comprehensive management service, is deployed on a Server cluster, a cloud Server or an edge Server, comprises eight modules in total and belongs to a core processing framework of an operating system; firstly, after a Task publisher transmits Task data to a server from a sensing end, the server understands and represents the tasks through a Task Pool module (Task Pool module), analyzes, schedules, distributes and finely tunes the received tasks through the Task Pool module, and distributes crowd-sourcing tasks to users in a platform; in a Resource management module (Resource management), software definition is carried out on various resources, and comprehensive management of Devices, Users, environments and task Process scheduling (processes) is completed; then entering a data management center module (DM center), wherein the module provides classified Storage and quick retrieval functions of massive heterogeneous data, a combined Storage and retrieval module (Storage and query) stores and processes data collected by a sensing end, extracts useful information and transmits the information to a Knowledge base module (Knowledge base); and for the collected data, optimizing the data quality through a task result quality optimization (TRO) module, and feeding the final result back to a task publisher of the sensing end through a network by the service end after the optimization is completed, so that the whole task message is circulated between the sensing end and the service end. In addition, a System plug-in module (System plug) provides rich System characteristic functions for users, including privacy protection, credit evaluation and user motivation; in the internal and external interface modules (I & E interfaces), the internal interface is used for facilitating maintenance and updating functions of a CrowdOS developer on an operating system, and the external interface is provided for a developer of third-party application, so that the developer can conveniently call a software interface to develop a personalized crowd sensing application system of the CrowdOS based on a CrowdOS framework.

In the implementation process of a CrowdOS architecture, various entities and virtual resources in a ubiquitous operating system, which are sensed in a crowd sensing mode, are abstracted by the CrowdOS, so that software definition and expression are completed, and the entities and the virtual resources are expressed by using symbols and relations among the symbols; after a Task enters a system, five dynamic Agents are constructed to generate a system resource map so as to manage the Task and the resource in the system, the construction of the Agents and the generation and management of the resource map are completed in a Task pool and a resource management module in fig. 2, and the Agents comprise a Task Agent Task-Agent (TA), a User Agent (UA), a Device Agent (DA), an Environment Agent (EA) and a Process Agent Process-Agent (PA).

As shown in FIG. 3, all resources in the system are defined by five agents. Wherein, the TA contains detailed information of each task, including but not limited to task category, execution time, location, collected data format, and the UA is abstract description of the user in the system, and records the information of the user, including but not limited to issued and executed tasks, credit level; DA is description of terminal equipment resources, and records the type and current state information of the equipment; EA abstracts the software and hardware environment resources of the current system, including but not limited to the current CPU, memory, storage and use conditions, the number of users in the system and the total amount of available equipment; the PA manages all task processes existing in the current system, including but not limited to process states, priorities and scheduling strategies; agents complete real-time interaction and updating among each other through semaphores defined in the internal structure after acquiring task information.

FIG. 4 is a Task parsing and scheduling framework diagram illustrating the detailed design of the Task Pool portion of the Task Pool of FIG. 2, which solves the first challenge of the present invention. The method comprises the following two steps: firstly, deeply understanding a crowd-sourcing task, and extracting commonalities and differences of the tasks at a fine granularity; secondly, the system needs to reasonably distribute tasks to participants to ensure that the information collection process is completed in the shortest time and under the lowest energy consumption condition.

As shown in fig. 4, the task information extraction part a marked above is shown, when a crowd sourcing task enters the system, the system first performs semantic parsing and feature extraction on the task, the system performs natural language analysis on the received task, performs word segmentation processing on the task described by languages such as chinese and english, and then performs operations of part-of-speech tagging, named entity recognition and keyword extraction on the indistinguishable languages, and finally extracts task key information including but not limited to the execution mode, location, time and number of participants of the task; the system further splices the extracted task information and discrete features obtained by regular click selection, inputs the spliced features into a deep neural network for uniform coding, and outputs a high-dimensional task intermediate vector; finally, the conversion process of the Task vector to the agents is completed by decoding the Task-agent mapping the vector to the Task, and the P1 marked below the fig. 4.

The B Agents Generation section, as labeled above in FIG. 4, contains 5 Agents: a description of specific definition is expanded by taking the structure of the Task _ agent as an example; the TA comprises the common and individual information of all tasks and is obtained through the task analysis and representation process in the last step; wherein, the taskID is a unique identifier of the task in the system; the process _ state represents the current state of the task process, whether the task process is in a generating state, an executing state or a feedback state and the like; the state assists the Process _ agent to manage the task Process, the Prio represents the priority of the task, the value is 0-15, and the system schedules the task Process according to the priority sequence; taskInfo is a structure containing detailed information of tasks, such as task time, location, vector representation, etc.; the Classication represents the category of the task, such as a data annotation class, a sensing information acquisition class, a questionnaire answer class and the like; topic represents the subject of the task and can be extracted from keyword information, such as audio collection, photo collection and the like; deviceNum, deviceInfo, deviceID represent the number of devices participating in the execution of the task, device detailed information, and device ID list, respectively; sensing _ Data is a collected task Data set pointer to the cube address where the Data is stored.

As shown in the scheduling and allocating part C marked above fig. 4, the task scheduling sub-frame includes a policy base, a mapping model, and a policy management module; the system firstly analyzes and infers the content of the task resource map, and maps the task ID to a specific strategy in a strategy library, wherein the strategy library stores the commonly used task allocation function and the serial number thereof, thereby completing the task allocation strategy selection process; and then the system executes scheduling operation on the equipment and the users according to the selected distribution strategy and pushes the tasks to the proper users.

The Resource Management framework is a concrete implementation of Resource Management module Resource Management in fig. 2, and the content of Resource Management includes, but is not limited to, user, sensing terminal, system environment, task process, system software, task data, and knowledge base. The management object of the crowdOS is abstracted into 4 types, and how the system manages four types of objects is described below, the following four types of objects all belong to one part of resource management, and the four managed parts are sequentially executed according to different periods when tasks enter the system.

1) Device, user and environment management

When the sensing end accesses the system through the network, a signal is triggered, the equipment automatically sends current state information to the system, wherein the current state information comprises but is not limited to equipment type, residual electric quantity, position information and storage occupancy rate, and the state information is captured and stored by a Device-agent based on a system developed by a CrowdOS framework;

the system draws User (task participants and publishers) images through the User-agent, and the User interacts with the system through the sensing end, for example, the tasks are published through a smart phone application interactive interface. The User-agent stores the name, age and task of the User, and generates the credit rating, User preference, interest field and other personalized information of the User according to the condition that the User participates in the task.

The environmental resource records the server architecture and processing capability, such as centralized, distributed, or edge deployment architecture, the number of CPUs in the system, system CPU utilization, memory usage, available disk storage space, and system access volume. These resources are stored in an Environment-agent and updated regularly to ensure that the system gets up-to-date data; the agent has an alarm function, prejudges according to the current system state and the increase and decrease conditions of the task amount, and gives an alarm if the CPU utilization rate or the storage occupancy rate of the system reaches a rated threshold value.

2) Task process scheduling management

This section is a detailed description of Process (Process) management in the resource management module of FIG. 2; a Process-agent (PA) is a set of state information of the current stage of the task, and the system allocates a Process identification number to each task to be used as a unique mark of the task Process in the system; this ID is stored in both the Task-agent and the Process-agent, along with the entire life cycle of the Task. The PA class contains rich task process information including but not limited to TPID: a process unique identifier; process _ state: the current task is in seven switchable states in the system; process _ stream: a process scheduling policy including First Come First Served (FCFS), round Robin (RB), task priority, highest response ratio priority (HRRN), and feedback priority; process _ prio represents the process priority, and from 0-15, 0 represents the highest priority, decreasing in order.

The task process state comprises seven states of a creation state, a generation state, a distribution state, an execution state, a processing state, a feedback state and a termination state; the flow relationship between the states is as shown in fig. 5, firstly, when a user issues a task in the system, a task creation state is firstly performed; entering a task generating state after system authentication and analysis; entering a distribution state after task scheduling and distribution; after the release is finished, the participants can execute the tasks and then enter an execution state; the participants upload the collected data to the system for processing and enter a processing state; summarizing the result to a task publisher through data visualization presentation, and entering a feedback state; when the task result is not verified by the publisher, the system stays in the feedback state temporarily, and then the task process returns to the generation state, the distribution state or the processing state again after reasoning and correction, and then is executed in sequence; and after the publisher verifies the task submission result, the task enters a termination state, and the life cycle of the whole task is finished. The text above the state in FIG. 5 represents the subject executing the state, either the publisher, the system, or the participant. The text above the arrows represents the operations required to move from one state to another.

The task process scheduling algorithm in the system selects one of the following methods:

5) feedback priority: and for the task entering the feedback state, two levels are adjusted up on the basis of the original priority, and the system resources are used preferentially. The selection of the scheduling algorithm is determined according to a scheduling algorithm flag bit in the agent.

3) Heterogeneous multimodal data resource management

The system manages two types of data resources, namely data carried by a task, namely original data (RD). The second is New Data (ND) that the participant uploads to the system during the performance of the task. The RD typically includes text, images, and data sets that need to be labeled to describe the task. The ND participant uploads various types of sensory data including, but not limited to, textual descriptions, sensor data, statistical charts, design documents, and tagged data.

For the management of multimodal data, the steps are as follows:

1) data needs to be collected and stored;

2) constructing a retrieval method of unstructured data based on crowd sourcing, wherein once data preparation is completed, a system can start to construct a data stack;

3) the invention adopts a data cube technology to manage and store task data, constructs a multi-feature cube structure (MC), and retrieves unstructured data according to the constructed data cube;

4) as the amount of tasks in the system increases, some of the completed task data and intermediate data are periodically cleaned up, and some of the deeply analyzed data are transferred to the knowledge base for management.

4) Knowledge base management

Knowledge in a system is divided into two categories, one being existing knowledge and the other being newly mined from tasks or data. The prior knowledge includes expert policies, decision rules or network models that are defined in the system in advance. Including but not limited to task allocation policies already existing in the policy repository, inference trees in the feedback mechanism. New knowledge mining identifies effective, novel, potentially useful, and interpretable content, methods, models from a data set or from existing knowledge. The new knowledge in CrowdOS is an extension on the basis of the original knowledge. The method comprises the improvement of a decision method, the addition of rules or the update of a network model, and does not comprise the knowledge of irregular explosive expansion.

The knowledge base summarizes and stores the knowledge according to the type, form and hierarchy of the knowledge. Knowledge is not centrally stored in some management list or module in the system, it exists in distributed fashion among various modules or lists in the system. The knowledge base records knowledge addresses, the intrinsic associations between knowledge, and knowledge network topology.

The result quality optimization framework is a TRO frame module in the server side of fig. 2. In order to solve the problem that the quality of task results does not meet the release requirement, a deep feedback framework (DFHMI) based on man-machine cooperative interaction is included in the TRO to simulate the process of thinking of human beings, and the problem is solved by means of analysis and reasoning. Task quality issues fall into two categories. The first is sparse result quantity, and the second is overproof result error rate. The optimization method used by the invention is based on 5Agents in the system, so that the user can conveniently interact with the system at a deeper level, and the specific implementation is described in detail in FIG. 6.

FIG. 6 is a diagram of a result quality optimization framework including an interaction layer, an inference layer, and an execution layer. Firstly, in an interaction layer, a publisher evaluates a task result by inputting evaluation information or clicking a button on a human-computer interface, and a system is responsible for analyzing diversified evaluation contents; if the evaluation shows that the evaluation is accepted or satisfied, the system executes an end instruction, and the task is ended; if the quality problem exists in the result through analysis, the next layer is entered; secondly, entering an inference layer, the system can perform key information extraction and deep analysis operation on feedback contents of the publisher to infer possible reasons of quality problems, and then according to an inference model established by the system, the reasons are mapped into a problem coding library to find corresponding error codes; third, the execution layer is entered where the system maps the problem code with the corresponding internal operations, most of the mapping mechanisms of the system have been defined initially. These operations are implemented by modifying the values in the agents, and other types of operations are also included. After the correction is finished, the task enters a new process state. And the task results corrected through various ways enter the interaction layer again, are fed back to the publisher, and wait for the interaction layer to give a new evaluation result. The whole optimization process is sequentially executed according to steps and forms a closed loop, and the optimization process is terminated until a publisher is satisfied with the task result.

The inference layer and the execution layer are connected through a mapping table (RSMT) between an inference decision tree and a system update operation base. The workflow of RSMT is as follows:

first, the reason for sparseness is found, which is called the shallow reason for being easily understood or directly obtained from feedback suggestions. While the deep causes are not generally directly accessible to the system, they need to be obtained by joint analysis in combination with task related information.

Second, a tree is built for problem reasons. In the initial stage, a model based on a decision tree is explored, a data sparse problem is used as a root node of the tree, a first layer node represents a shallow reason, a newly-appeared shallow reason can be added as a child node of the root node, and all the shallow reasons are brother nodes. The deep cause is usually based on the shallow cause and is the extension of the node in the longitudinal direction, which is called deep node. Thirdly, as the scale of the problem increases, in order to reduce the search time, the system periodically performs updating operations such as pruning and increasing branches on the tree, and ensures that the decision tree is maintained within a certain scale range.

Example (b):

1. experimental setup

The present invention develops a WeSense use case on which users can publish and perform various sensing tasks, such as road congestion information collection, air quality condition monitoring, product price research, and the like. During the implementation of WeSense, E v1 and Ev2 will be evaluated based on two development methods: m1 (independent development) and M2 (development based on CrowdOS interface). Ev3 is an evaluation of the result quality optimization mechanism (TRO), while Ev4 is a WeSense's overall performance and stress test supported by CrowdOS. Table 1 illustrates the development and testing environment. In Ev1 and Ev2, the time required for the relevant function module to complete under the conditions of M1 and M2 is compared and analyzed, and then the function to be tested and the module F { fi } are divided into five parts (Fnum ═ 5).

f 1: and (5) a task real-time release function.

f 2: and a task allocation algorithm module.

f 3: and a privacy protection module.

f 4: crowdsourcing data collection and upload functions.

f 5: and a result quality optimization function.

The experimental environment is preset and the relevant software is installed. 9 volunteers familiar with the Java programming language (Λ num ═ 9) were hired and given two weeks to develop applications. It takes 25 minutes first (timec1) to introduce crowdOS functionality and all APIs. 9 volunteers were first assigned to group a and then switched to group B, i.e. each volunteer served both GA and GB members at different times, GA _ num-GB _ num-9. Members of the GA use M1, while members of the GB use M2. Each volunteer participated in all tests, totaling (GA _ num + GB _ num) × Fnum ═ 90.

TABLE 1 application runtime and development environment

The four evaluation indexes are compared as follows.

E_v1：M₁And M₂Developing mode comparison, and completing F { F_iThe time consumption of, and the completeness and correctness of the task being completed.

E_v2: comparative G_AAnd G_BTwo groups of testers completed F { F_iThe effect and time consumption of.

E_v3: comparing the differences of optimization effect and time consumption by using a TRO frame and other optimization methods.

E_v4: and testing system stability and pressure.

2 evaluation of effectiveness and efficiency

Through detection, all 90 tests pass correctness screening, and a simple introduction is made to the developed WeSense user interface. FIG. 7(a) is a home page, display and search of crowd sourcing tasks, as shown in FIG. 7; FIG. 7(b) is a task detail page that can be clicked on to view a task of interest; FIG. 7(c) is a task submission page: and submitting the completed result data.

Fig. 8 is an analysis and comparison of development cycles, i.e., efficiency assessment. Wherein FIG. 8(a) shows the use of G_AAnd G_BCompletion f_1-3The time required for the test. FIG. 8(b) shows a graph based on M₁And M₂Time consumption comparison of all tests. Such asFIG. 8(b) shows F { F } according to the formula (1)_iMean development time from original DT_GA(tc_fi) Reduced to DT {12.1, 13.5, 7.3, 10.1, 14.1}_GB(tc_fi) 3.1, 4.7, 0.8, 4.5, 5.4.

According to the formula (2), f₁→f₅The overall Development Efficiency (DE) of (a) is improved by 310%.

By analysis and verification of test f₂And f₃To prove its effectiveness and usability. Even M if used after calling CrowdAPI₂The mode improves performance, meaning that CrowdOS is indeed effective. As shown in fig. 9, is core framework validity assessment, where the tasks in fig. 9(a) are randomly assigned and the orange circle is the geographic scope of task release; the blue circle in FIG. 9(b) is the effect of using the location-based task assignment algorithm; FIG. 9(c) compares the effect of the two methods; fig. 9(d) shows the super privacy protection mode being selected.

The reduction in development time after invocation of CrowdAPI reflects availability. As can be seen from FIG. 8(a), M is used₁In contrast, with M₂F of (a)₂And f₃The average time consumption was reduced by 65.4% and 88.7%. Furthermore, as the algorithm library expands, the advantages of CrowdOS are highlighted. By using the method, the development time of each functional module can be greatly reduced, and the whole visualization effect and program readability can be improved.

3 evaluation of result quality optimization mechanism

Through simulation experiments, the correctness and the optimization effect of the application program interface in the TRO framework are evaluated.

Take the problem of data format error as an example. For the collected data D (n x V), where n represents the number of participants and V represents the amount of data contributed by each participant. The time taken to correct the data format by the TRO framework is T_iB＝T_iζB+T_iηBWherein T is_iζBIs the time (T) of the interface operation_iζB<6min), the interface effect is shown in FIG. 10 (a). T is_iηBIs the time each participant spends correcting the format and resubmitting the data, T_iηBIs equal to V. If there is no TRO framework, the time required for the publisher to correct all data formats is T_iA。T_iA＝T_iζA+T_iηAWherein T is_iζAIs the preparation time (T) before correcting the data format_iζA<30min)，T_iηAIs the time taken to process the data format, T_iηA∝n*V。

Equation (3) gives the time taken to deal with the data format problem by two optimization methods. Fig. 10(c) shows the variation of the distance between the a and B curves with the data volume D (n × V). I.e. the comparison of the optimization method and the time consumption. Wherein fig. 10(a) is a data format correction request interface; and FIG. 10(b) is a correction prompt message received by a participant; FIG. 10(c) compares the time consumption of the two optimization methods as the number of participants increases.

The correlation between task results and task requirements can be reflected in a number of ways, such as when the requirements are provided in the form of video, but the participants submit images; or the actual location of the uploaded data does not match the location in the task requirements. The optimization mechanism re-screens the users with high confidence level, updates task characteristics according to specific information (such as geographical location), and then re-pushes the task and feedback information to the original participants, as shown in fig. 10 (b).

It can be seen that the use of the TRO framework not only avoids energy consumption due to large amounts of processing, but is also suitable for various types of tasks. The framework uses the concept of segmentation and integration to correctly hand over the work of different phases to people or machines. As shown in fig. 10(c), the time consumption of TRO is relatively stable compared to other optimization methods, and does not increase significantly as the number of participants increases. Meanwhile, many optimization problems are more suitable to be solved by TRO, and compared with pure machine optimization, resource consumption can be greatly reduced.

4. Performance, load and pressure testing

The sensing end and the server end of WeSense are comprehensively tested from the following two aspects.

And (5) testing performance and load. Tasks of different scales are loaded in sequence, and the response time of the system, the utilization rate of a CPU and a memory and the energy consumption of a sensing end are measured. Each test was run ten times and the data was monitored in real time using the android studio's profile performance analyzer while the application was running. The results are shown in FIG. 11. Although the number of tasks increases, the system response time is substantially within 0.22s, and the CPU and memory usage is also kept in the range of 3% -6% and 0.87% -1.14%. The energy consumption was kept substantially below a minimum level (L1: light), indicating that the system is functioning well.

TABLE 2 stability and pressure test

REC number	1000	5000	10000	50000	100000
						CRASH times	0	0	0	0	0
ANR times	0	0	0	0	0
						TOTAL	0	0	0	0	0

Stability and stress testing. Two test methods were combined. First, the server runs continuously for 7 × 24 hours. During this time, the number and content of tasks are updated by the mobile device. And the output logs of the sensor end and the server end are continuously observed and recorded, and abnormal conditions such as dead halt, software errors and the like do not occur. Secondly, after setting up the test environment, stability and stress tests were performed on WeSense using android sdk Monkey software. For example, Monkey-pcom. hills. WeSense-v-v 1000 is sent to request the execution of 1000 Random Command Events (RCE), e.g., Map key, Home key. The number of occurrences of crach and ANR (application unresponsive) is recorded. CRASH refers to the situation where a program is abnormally stopped or exited upon the occurrence of an application error. ANR means that when the Android system detects that an application does not respond to an input event within 5 seconds or does not perform a broadcast within 10 seconds, it will cause a prompt to be unresponsive. The test results are shown in table 2. Combining the above two test results, it can be seen that the system can operate stably and efficiently under different pressure conditions.

Claims

1. A ubiquitous operating system for crowd-sourcing awareness, comprising:

according to the ubiquitous operating system facing the crowd sensing, a task publisher inputs original task data through an intelligent terminal and submits the original task data to a platform, and the platform captures a task, analyzes the task and assigns a unique task ID to the task; after a task enters a platform, the task is analyzed and corresponding task feature vectors are generated, feature splicing is carried out on the task feature vectors and other known discrete features, and characteristics are extracted through the task vectors, wherein the characteristics include but are not limited to task types, the number of required participants, task execution places, sensors required to be used and types of collected data; the ubiquitous operating system facing the crowd sensing completes the processes of user scheduling and task allocation by executing task reasoning, association and matching operations; after receiving the tasks, the participants select interested tasks to execute and upload the acquired perception data or design scheme to a ubiquitous operating system facing crowd sensing; after data enter the ubiquitous operating system facing the crowd sensing, abstracting and defining software for tasks, users and process resources, selecting task middleware required to be used by the ubiquitous operating system facing the crowd sensing according to the description of task characteristic information, and summarizing the collected data; and finally, returning the result to the task publisher, evaluating and feeding back the task result by the publisher, and finishing the life cycle of the crowd sourcing task after the publisher receives the result.

2. The ubiquitous operating system for crowd-sourcing aware oriented as claimed in claim 1, wherein:

the server provides comprehensive management service, is deployed on a server cluster, a cloud server or an edge server, and comprises eight modules, namely a task pool module, a resource management module, a data management center module, an internal and external interface module, a knowledge base module, a system plug-in module and a task result quality optimization and joint storage and retrieval module; firstly, after a task publisher transmits task data to a server from a sensing end, the server understands and represents the tasks through a task pool module, analyzes, schedules, distributes and finely tunes the received tasks through the task pool module, and distributes crowd-sourcing tasks to users in a platform; in the resource management module, software definition is carried out on various resources and comprehensive management of equipment, users, environment and task process scheduling is completed; then entering a data management center module, wherein the module provides classified storage and quick retrieval functions of massive heterogeneous data, a combined storage and retrieval module stores and processes data acquired through a sensing end, extracts useful information and transmits the useful information to a knowledge base module; for the collected data, the data quality is optimized through a task result quality optimization module, and after the optimization is completed, the server feeds the final result back to a task publisher of the sensing end through a network, so that the whole task message is circulated between the sensing end and the server; the system plug-in module provides privacy protection, credit evaluation and user incentive for a user; in the internal and external interface module, the internal interface is used for maintaining and updating the operating system, and the external interface is provided for the third-party application and used for calling the software interface.

3. The ubiquitous operating system for crowd-sourcing awareness according to claim 2, wherein:

in the CrowdOS, five dynamic Agents are constructed to generate a system resource map after a Task enters to manage the Task and the resource in the system, wherein the Agents comprise a Task Agent Task-Agent, a User Agent User-Agent, an equipment Agent Device-Agent, an Environment Agent Environment-Agent and a Process Agent Process-Agent; wherein, the TA contains detailed information of each task, including but not limited to task category, execution time, location, collected data format, and the UA is abstract description of the user in the system, and records the information of the user, including but not limited to issued and executed tasks, credit level; DA is description of terminal equipment resources, and types and current state information of the equipment are recorded; EA abstracting the software and hardware environment resources of the current system, including but not limited to the current CPU, memory, storage and use conditions, the number of users in the system and the total amount of available equipment; the PA manages all task processes existing in the current system, including but not limited to process states, priorities and scheduling strategies; agents complete real-time interaction and updating among each other through semaphores defined in the internal structure after acquiring task information.

4. The ubiquitous operating system for crowd-sourcing awareness according to claim 2, wherein:

in the task pool, a task information extraction part firstly carries out semantic analysis and feature extraction on a task when a crowd-sourcing task enters a system, carries out natural language analysis on the received task, carries out word segmentation processing on the task described by the language, then carries out operation without distinguishing languages by the system, finally extracts task key information including but not limited to the execution mode, the place, the time and the number of participants of the task, splices the extracted task information and discrete features obtained by regular click selection, inputs the spliced features into a deep neural network for uniform coding, and outputs a high-dimensional task intermediate vector; finally, mapping the vector to the Task-agent of the Task through decoding to complete the conversion process from the Task vector to the agents;

in the task pool, an Agents generation part comprises 5 Agents: the Task _ agent, the Environment _ agent, the Process _ agent, the Device _ agent and the User _ agent are obtained by Task analysis and representation processes; wherein, the taskID is a unique identifier of the task in the system; the process _ state represents the current state of the task process, whether the task process is in a generating state, an executing state or a feedback state and the like; the state assists the Process _ agent to manage the task Process, the Prio represents the priority of the task, the value is 0-15, and the system schedules the task Process according to the priority sequence; taskInfo is a structure containing detailed information of tasks, such as task time, location, vector representation, etc.; the Classication represents the category of the task, such as a data annotation class, a sensing information acquisition class, a questionnaire answer class and the like; topic represents the subject of the task, and is extracted from keyword information, such as audio collection, photo collection and the like; deviceNum, deviceInfo, deviceID represent the number of devices participating in the execution of the task, device detailed information, and device ID list, respectively; sensing _ Data is a collected task Data set pointer, pointing to the cube address where Data is stored;

5. The ubiquitous operating system for crowd-sourcing awareness according to claim 2, wherein:

1) device, user and environment management

the Environment resource records the server architecture and the processing capacity, resources are stored in an Environment-agent and are updated at regular time, the agent has an alarm function, pre-judgment is carried out according to the current system state and the increase and decrease conditions of the task quantity, and if the CPU utilization rate or the storage occupancy rate of the system reaches a rated threshold value, the system gives an alarm;

2) task process scheduling management

3) heterogeneous multimodal data resource management

For the management of multimodal data, the steps are as follows:

1) collecting and storing data;

4) knowledge base management

The knowledge of the OS falls into two categories: prior knowledge (EK) and New Knowledge (NK); NK is extracted from tasks or data, which helps to improve system mechanics or update models; the process of knowledge management is as follows:

firstly, the system distinguishes the knowledge which can improve the system performance, update the model or improve the task result quality from the prior knowledge, and the information or the knowledge which is useful for the user or a third party is not included in the knowledge range; secondly, for knowledge to be discovered, the system provides a corresponding mechanism or algorithm to mine the knowledge, wherein the mechanism or algorithm comprises but is not limited to a deep learning algorithm, an online updating algorithm and a transfer learning algorithm; thirdly, the knowledge base generalizes the knowledge and stores the knowledge in the database according to the type, form and abstraction level of the knowledge.

6. The ubiquitous operating system for crowd-sourcing awareness according to claim 2, wherein: