WO2021213293A1 - Ubiquitous operating system oriented toward group intelligence perception - Google Patents

Abstract

Provided by the present invention is a ubiquitous operating system, specifically CrowdOS, that is oriented towards group intelligence perception. By means of the in-depth analysis of the complex environment and diversified features of group intelligence tasks, a set of comprehensive processing mechanisms and core functional components are designed, comprising three core mechanisms, i.e. task semantic analysis and user scheduling, system resource management, and in-depth feedback interaction of task results. The present invention uses CrowdOS to solve the problems of the lack of a unified system structure of existing mobile group intelligence perception or a crowdsourcing platform and the incompatibility of algorithms or modules in related research. The task analysis and scheduling mechanism establishes a bridge between tasks and OS kernels by means of a task resource map, and adaptively selects a reasonable allocation strategy for heterogeneous tasks. The resource management mechanism abstracts heterogeneous physical and virtual resources in the system to provide unified software definition and management. A result quality optimization mechanism quantifies and optimizes the quality of the results.

Description

A ubiquitous operating system oriented to group intelligence perception Technical field

The present invention relates to the field of mobile computing and ubiquitous operating systems and the field of group intelligence perception, in particular to system-based

The operating system of the underlying architecture.

Background technique

With the rise of crowdsourcing and group intelligence application technologies, a large number of platforms and application software based on group intelligence ideas have emerged, such as Amazon Mechanical Turk [document "M.Buhrmester, T.Kwang, and SDGosling,"Amazon's mechanical-ical turk: A new source of expensive, yet high-quality, data? "Perspectives on psychological science, vol. 6, no. 1, pp. 3–5, 2011."], CrowdFlower, a food and tourism platform, and an intelligence analysis platform. In addition, there are many crowdsourced applications, such as water quality testing, air quality testing, and traffic congestion surveys, all of which solve problems in different fields through the wisdom of the group. The applications and technologies developed based on crowdsourcing ideas are summarized as the first-generation group intelligence technology. Its characteristics include: publishing tasks through the Internet platform and using the idea of problem segmentation to solve large-scale problems. These platforms themselves serve as the media for task release and result collection, do not include analysis and evaluation of the task itself, and do not optimize the quality of the results collected by the platform.

With the development of mobile terminals and portable sensing devices, the group intelligence application and related technologies are summarized as the second-generation group intelligence technology. These technologies are used in environmental monitoring, public facility monitoring and other fields, such as the literature "RKGanti, F. Ye, and H. Lei, "Mo bile crowdsensing: current stag and future challenges," IEEE Communications Magazine, vol. 49, no.11,pp.32–39,2011.", and the literature "B.Guo,Z.Wang,Z.Yu,Y.Wang,NYYen,R.Huang,and X.Zhou,"Mobile crowd sensing and computing: The review of an emerging human-powered sensing paradigm, "ACM Computing Surveys (CSUR), vol. 48, no. 1, p. 7, 2015.". The more well-known systems include Common Sense, Ear-Phone, Chimera, Creekwatch, and PhotoCity. In addition, related group intelligence perception technology has also been extensively studied. For example, Wang et al. studied the problem of multi-task cooperative allocation in the application of group intelligence. Guo et al. put forward the challenge of optimizing the sensing data. F. Restuccia et al. studied the quality improvement methods of crowd perception data. In addition, the incentive mechanism and privacy protection of crowdsourcing workers have become a subject of in-depth research. These applications and technologies are mostly used in scientific research, with the goal of collecting some kind of sensor data. The developed software can only complete a single task and cannot be reused, and cannot be migrated to other tasks. Most of these technologies are based on many assumptions in an ideal environment, and only simple simulation experiments have been completed. The study did not consider the combination with the actual platform. Comprehensive theoretical analysis and practical experience, the present invention deeply considers the difficulties faced by the development and promotion of the first and second generation of smart technologies, and hopes to study a unified architecture to focus on solving these difficulties.

The difficulties and challenges facing the current platform include:

1) There is a lack of a framework that can handle multiple types of tasks, and the framework requires a unified and in-depth understanding of tasks. The crowdsourcing platform is usually just an Internet bulletin board for centralized publishing and collecting tasks, and lacks the function of treating different tasks differently. The current group intelligence applications mostly use customized software and specific sensing devices to perform tasks, and applications and tasks have a one-to-one binding relationship. Therefore, application software generally lacks universality and scalability, and it is difficult to migrate to other types of tasks.

2) Lack of abstraction and unified management of various resources in the group intelligence system. The system cannot perform joint analysis and scheduling of resources such as people, tasks, equipment resources, software resources, and the knowledge data generated. Most of the current technical researches are carried out in an ideal environment to solve specific problems, based on many conditional assumptions. Because the research assumptions and scenario settings are different, the technologies are isolated from each other. Due to the lack of communication bridges between these scattered studies, the local methods proposed by various scholars are also difficult to generalize to practical applications.

3) Lack of results quality evaluation and optimization methods. The crowdsourcing platform usually only summarizes the task result data without further evaluation and analysis. Some simple results filtering methods may be preset in the Qunzhi application, but because these methods are usually for specific types of tasks or data, it is difficult to generalize to other tasks or data types and cannot support complex data processing tasks (such as data semantic understanding). Wait). There are also some optimization methods that are performed after the task publisher gets the results. For example, the user cleans and filters the data. This operation has nothing to do with the publishing platform or application, and cannot reduce the data aggregation cost of the group intelligence perception.

Summary of the invention

In order to overcome the shortcomings of the prior art, the present invention provides a ubiquitous operating system oriented to group intelligence perception, specifically a ubiquitous operating system oriented to group intelligence, CrowdOS, through the in-depth understanding of the complex environment and diversified characteristics of group intelligence tasks Analysis, and then designed a set of comprehensive processing mechanism and core functional components based on the operating system framework. As an incubator and accelerator for Qunzhi applications, CrowdOS runs as a middleware between the native operating system and the application layer of heterogeneous devices. The present invention focuses on the core architecture of CrowdOS and its three core mechanisms, namely: task semantic analysis and user scheduling, system resource management and deep feedback interaction of task results.

The technical solutions adopted by the present invention to solve its technical problems are:

A ubiquitous operating system for group intelligence perception. The task publisher inputs the original task data through the smart terminal and submits it to the platform. After the task is captured by the platform, the task is analyzed and a unique task ID is assigned to the task; After entering the platform, perform task analysis and generate the corresponding task feature vector, and perform feature splicing with other known discrete features, and extract the characteristics through the task vector. The characteristics include but are not limited to the task type, the number of participants required, and the task The execution location, the type of sensors and data collected; the ubiquitous operating system for group intelligence perception completes the process of user scheduling and task allocation by performing task inference, association and matching operations; after the participants receive the task, they have a sense of choice The task of interest is executed and the collected perception data or design plan is uploaded to the ubiquitous operating system for group intelligence perception; when the data enters the ubiquitous operating system for group intelligence perception, the task, user, and process resources are performed Abstract and software definition, according to the description of task feature information, the ubiquitous operating system for group intelligence selects the task middleware to be used and summarizes the collected data; finally the results are returned to the task publisher, and the publisher performs the task results Evaluation and feedback, when the publisher receives the results, the life cycle of the group intelligence task ends.

The ubiquitous operating system oriented to group intelligence perception is implemented by the operating system CrowdOS. CrowdOS runs between the native operating system and upper-layer applications, and includes the sensing end and the server. The software carrier of the sensing end includes two types of equipment, the first type is Portable intelligent sensing devices with human-computer interaction functions. The second category is fixed sensors deployed in the physical world; CrowdOS adopts a cloud-side-to-end deployment method; the sensing end is deployed on various terminal sensing devices to collect data, information and services The end is deployed on a cloud server or edge server to comprehensively manage system resources and data resources and respond to system operations in real time;

In the functional layer part of the perception terminal, first the publisher uploads the task to the CrowdOS-based system through the interactive function of the smart terminal, and the participant browses through the perception terminal and executes the tasks that have been published in the system; when the participant receives the task, the perception terminal Data collection begins; the system support layer obtains the current device status, then uniformly interface encapsulates the acquired sensor data and unifies the data transmission format, and then stores the data in the corresponding data structure through the network or Bluetooth transmission; A sensing device that does not require human-computer interaction. Once the device is activated by a task in the system and the device passes the task verification, the device will automatically collect and upload perception data according to predetermined rules;

The server-end (Server-end) provides comprehensive management services, which are deployed on server clusters, cloud servers or edge servers, including task pool modules (Taskpool modules), resource management modules (Resource management), and data management center modules (DM). center), internal and external interface module (I&E interface), knowledge base module (Knowledge base), system plug-in module (System Plugin), task result quality optimization (TRO) and joint storage and retrieval module (Storage and query) eight modules; First, after the task publisher transmits the task data from the sensing end to the server, the server understands and expresses the task through the task pool module, and parses, schedules, allocates, and fine-tunes the received tasks through the task pool module. Qunzhi tasks are allocated to users in the platform; in the resource management module, various resources are defined by software and the comprehensive management of equipment, users, environment, and task process scheduling is completed; then enter the data management center module, which provides a large number of Classification and storage of heterogeneous data and fast retrieval functions. The combined storage and retrieval module stores and processes the data collected through the sensing terminal, extracts useful information and transmits it to the knowledge base module; for the collected data, the quality of the task results is optimized The module optimizes data quality. After the optimization is completed, the server will feed back the final result to the task publisher on the sensing side through the network, thereby completing the circulation of the entire task message between the sensing side and the server; the system plug-in module provides users with privacy protection, Credit evaluation and user incentives; in the internal and external interface modules, the internal interface is used to maintain and update the operating system, and the external interface is provided to third-party applications for invoking the software interface.

In CrowdOS, five dynamic Agents are constructed to generate system resource maps after the task enters to manage the tasks and resources in the system. Agents include task agent Task-agent (TA), user agent User-agent (UA), Device-agent (DA), Environment-agent (EA) and Process-agent (PA); where TA contains detailed information about each task, including but not limited to task type, execution time, Location, collected data format, UA is an abstract description of the user in the system, and records the user’s information, including but not limited to the tasks posted and performed, and credit rating; DA is the description of the terminal equipment resources, recording the device’s information Type, current status information; EA abstracts the hardware and software environment resources of the current system itself, including but not limited to the current CPU, memory, storage usage, the number of users in the system and the total amount of available equipment; PA manages all existing systems in the current system Task process, including but not limited to process status, priority, scheduling strategy; Agents complete mutual real-time interaction and update through the semaphore defined in the internal structure after obtaining task information.

In the task pool, in the task information extraction part, when the group intelligence task enters the system, the task is firstly analyzed by semantic analysis and feature extraction, and natural language analysis is performed on the received task. For tasks described by language, the system performs word segmentation processing. Then the system performs operations that do not distinguish between languages, and finally extracts mission-critical information, including but not limited to the task execution method, location, time, number of participants, the extracted task information and the discrete features obtained through rule click selection For splicing, input the spliced features into a deep neural network for unified encoding, and output a high-dimensional task intermediate vector; finally, map the vector to the task-agent of the task through decoding, and complete the conversion process from task vector to agents .

In the task pool, the Agents generation part contains 5 Agents: Task_agent, Environment_agent, Process_agent, Device_agent, User_agent, which are obtained through the process of task analysis and representation; among them, taskID is the unique identifier of the task in the system; process_state represents the The current state of the task process, whether it is in the generation state, the execution state or the feedback state, etc.; this state assists Process_agent in task process management, Prio represents the priority of the task, the value is 0-15, and the system performs the task according to the priority order Process scheduling; taskInfo is a structure that contains detailed task information, such as task time, location, vector representation, etc.; Classification represents the category to which the task belongs, such as data labeling, sensor information collection, questionnaire answering, etc.; Topic Represents the subject of the task, which can be extracted from keyword information, such as audio collection, photo collection, etc.; deviceNum, deviceInfo, and deviceID respectively represent the number of devices participating in the task, device detailed information, and device ID list; Sensing_Data is the collection The pointer to the task data set that points to the address of the cube where the data is stored.

In the task pool, the scheduling and allocation part, the task scheduling sub-framework includes a strategy library, a mapping model, and a strategy management module; first, the content of the task resource map is analyzed and reasoned, and the task ID is mapped to the strategy library. The task allocation function and its number are stored in the library, so as to complete the task allocation strategy selection process; then according to the selected allocation strategy, perform scheduling operations on the devices and users, and push tasks to appropriate users.

The resource management module includes, but is not limited to, users, perception terminals, system environments, task processes, system software, task data, and knowledge bases. The management objects of CrowdOS are abstracted into 4 categories, and they are executed sequentially according to the different periods when tasks enter the system. as follows:

1) Device, user and environment management [0020] When the sensing terminal is connected through the network, the signal is triggered, and the device automatically sends the current status information to the system, including but not limited to device type, remaining power, location information, storage occupancy rate, based on The system developed by the CrowdOS framework captures and stores state information through the Device-agent agent;

The system portrays the user portrait through the User-agent. The user interacts with the system by relying on the perception end. The Useragent saves the user's name, age, and tasks that have participated in it. At the same time, it generates user credit ratings, user preferences, areas of interest, and others based on the user's participation in tasks. Personalized information;

Environment resource records server architecture and processing capabilities. Resources are stored in the Environment-agent and updated regularly. The agent has an alarm function and makes predictions based on the current system status and the increase or decrease of the task volume. If the system CPU utilization rate or storage occupancy rate is When the rated threshold is reached, the system will give an alarm.

2) Task process scheduling management

Process-agent is a collection of the status information of the current stage of the task. A process identification number is assigned to each task as the only sign of the existence of the task process in the system; the ID is stored in both the Task-agent and the Process-agent and accompanies the entire life cycle of the task; The PA class contains task process information, including but not limited to TPID: process unique identifier; process_state: the current state of the task in the system, there are seven switchable states; process_strategy: process scheduling strategy, including first come, first served ( FCFS), round-robin method (RB), task priority, highest response ratio priority (HRRN), feedback priority; process_prio represents the process priority, from 0-15, 0 represents the highest priority, in descending order;

The task process state includes seven states: creation state, generation state, allocation state, execution state, processing state, feedback state and termination state; the flow relationship between states is: First, when a user publishes a task in the system, the first is task creation State; enter the task generation state after passing system authentication and analysis; enter the distribution state after task scheduling and assignment; after the release is completed, participants can execute the task, and then enter the execution state; participants upload the collected data to the system After processing, enter the processing state; after data visualization, the results are summarized to the task publisher and enter the feedback state; when the task result fails the publisher’s acceptance verification, the system will temporarily stay in the feedback state, and then after reasoning and correction, the task progresses Return to the generation state, the allocation state or the processing state again, and then execute sequentially; after the publisher verifies the task submission result, the task enters the termination state, and the entire task life cycle ends;

The task process scheduling algorithm in the system can choose one of the following methods:

1) First-come, first-served (FCFS): Prioritize tasks that enter the system first, and provide resources and services for them;

2) Circular method: Generate task interrupts at a periodic interval, place the currently running process in the task ready queue, and select the next ready process to run based on FCFS;

3) Task priority: Prioritize high-level tasks, and tasks with the same priority shall follow the FCFS principle;

4) The highest response ratio priority (HRRN), R=(w+s)/s, where R represents the response ratio, w represents the waiting time, and s represents the time expected to be served;

5) Feedback priority: For tasks that enter the feedback state, two levels will be raised on the basis of the original priority, and system resources will be used first;

3) Heterogeneous multi-modal data resource management

For the management of multi-modal data, the steps are as follows:

1) Collect and store data;

2) Construct an unstructured data retrieval method based on Qunzhi. Once the data preparation is completed, start to build the data stack;

3) Using data cube technology to manage and store task data, construct a multi-character cube structure (MC), and retrieve unstructured data based on the constructed data cube;

4) As the amount of tasks in the system grows, the completed task data and intermediate data are regularly cleaned up, and the data that has undergone in-depth analysis will be transferred to the knowledge base for management;

4) Knowledge base management

OS knowledge is divided into two categories: existing knowledge (EK) and new knowledge (NK); NK is extracted from tasks or data, which helps to improve system mechanisms or update models. The process of knowledge management is as follows:

First, the system distinguishes from existing knowledge that can improve system performance, update models, or improve the quality of task results. Information or knowledge useful to users or third parties is not included in the scope of knowledge here; The system provides corresponding mechanisms or algorithms to mine the discovered knowledge, including but not limited to deep learning algorithms, online update algorithms, and migration learning algorithms. Third, the knowledge base can extract knowledge based on the type, form, and abstraction level of the knowledge. Perform induction and database storage. Knowledge is not centrally stored in a certain management list or module in the system, it is distributed in various modules or lists of the system. The knowledge base mainly records knowledge addresses and internal relationships between knowledge, and establishes a knowledge network based on these.

The result quality optimization module includes an interaction layer, a reasoning layer, and an execution layer; firstly, in the interaction layer, the publisher evaluates the task results by inputting evaluation information or clicking buttons on the man-machine interface, and analyzes the diversified evaluation content ; If the evaluation shows acceptance or satisfaction, the end instruction is executed, and the task is terminated; if the result is found to have quality problems through analysis, then enter the next layer; secondly, enter the reasoning layer, and perform key information extraction and depth on the publisher’s feedback content Analyze the operation, infer the possible causes of the quality problem, and then map the reason to the problem code library according to the inference model established by the system, and find the corresponding error code; third, enter the execution layer, and the problem code will be coded with the corresponding internal operation After the correction is completed, the task enters a new process state. The task results corrected through various channels will enter the interactive layer again and be fed back to the publisher, waiting for the interactive layer to give new evaluation results. The entire optimization process is step by step Execute and form a closed loop, and terminate until the publisher is satisfied with the task result.

The beneficial effect of the present invention is to propose a ubiquitous operating system oriented to group intelligence perception, and use CrowdOS to solve the problems of the existing mobile group intelligence perception or the lack of a unified architecture of the crowdsourcing platform and the incompatibility of algorithms or modules in related research. The composition of the architecture and the contents of each module and the relationship between them are described in detail. In addition, the implementation ideas of the three core mechanisms in the CrowdOS kernel architecture are explained in detail: Among them, the task analysis and scheduling mechanism establishes a bridge between the task and the OS kernel through the task resource graph, and then adapts to heterogeneous tasks. Choose a reasonable allocation strategy; the resource management mechanism abstracts heterogeneous physical and virtual resources in the system, and provides them with unified software definition and management; the result quality optimization mechanism aims at quality evaluation and shallow in-depth inference mechanisms and integration Strategies for specific quality issues to quantify and optimize the quality of the results.

By developing application examples based on this architecture, the correctness of CrowdOS, the effectiveness of the kernel module and the overall development efficiency are evaluated, and the optimization speed and energy consumption of the results before and after use are compared. CrowdOS and WeSense, an application example developed based on the architecture, are mainly evaluated from four aspects: correctness and efficiency (Ev ₁ ), effectiveness and availability (Ev ₂ ), optimization result quality evaluation (E v ₃ ), performance, load and pressure Test (Ev ₄ ).

Description of the drawings

Figure 1 is a block diagram of the group intelligence perception ecosystem.

Figure 2 is a block diagram of the core architecture of CrowdOS.

Figure 3 is a system resource map.

Figure 4 is a framework diagram of task analysis and scheduling.

Figure 5 is a state switching diagram of the task process.

Figure 6 is a framework diagram of result quality optimization.

Figure 7 is the WeSense usability evaluation. Figure 7(a) is the homepage, showing and searching of the group intelligence tasks; Figure 7(b) is the task detail page; Figure 7(c) is the task submission page.

Figure 8 is a development efficiency evaluation diagram, where Figure 8 (a) is the time required to complete the f1-3 test with GA and GB, and Figure 8 (b) is a comparison diagram of the time consumption of all tests based on M1 and M2.

Figure 9 is the effectiveness evaluation of the core framework. The tasks in Figure 9(a) are the effect diagrams of random assignment; Figure 9(b) is the effect of using the location-based task allocation algorithm, and Figure 9(c) compares the two methods. Figure 9(d) is a schematic diagram of selecting the super privacy protection mode.

Figure 10 is a comparison diagram of optimization time, in which Figure 10 (a) is the data format correction request interface, Figure 10 (b) is the correction prompt message received by the participants; Figure 10 (c) is the time consumption comparison of the two optimization methods picture.

Figure 11 Performance and stress test comparison chart.

Detailed ways

The present invention will be further described below in conjunction with the drawings and embodiments.

The present invention mainly includes the core architecture of CrowdOS and three important mechanisms. The main contributions and innovations of CrowdOS are as follows:

First, the framework of the group intelligence perception operating system is designed, and the core architecture of CrowdOS is introduced in detail.

Second, the system analyzes group intelligence tasks based on natural language processing related technologies, and performs fine-grained semantic analysis and modeling of tasks combined with discrete features. Accordingly, a bridge between the task and the system is established, that is, the task analysis and scheduling mechanism. Based on natural language interaction understanding, it solves the problem that the platform mentioned in the first challenge only has a summary function for task results or can only handle a single task through a template.

Third, the operating system abstracts all kinds of resources (user resources, task resources, system resources) required for the execution of group intelligence tasks and defines them in software. And then established a system resource map. It provides the cornerstone for the unified and efficient management of various resources in the system. The resource management mechanism solves the second challenge.

Fourth, in view of the task result quality problem, the system proposes an evaluation method and optimization mechanism, that is, the result quality optimization mechanism. This mechanism is based on the idea of deep human-computer integration, using natural language feedback interaction and deep and shallow reasoning methods, which mainly solves the two types of quality problems of sparse results and high error rate of results. This man-machine coordination task result optimization mechanism solves the third challenge.

A ubiquitous operating system oriented to group intelligence perception, as shown in Figure 1, is an architecture diagram of the group intelligence perception ecosystem, showing the task execution process and life cycle. The ubiquitous operating system architecture for group intelligence perception includes a) server clusters, b) smart terminals, c) sensing devices, d) sensors, e) communication networks, f) basic software layer, g) platform and application software, h) Participants and group intelligence tasks, where af is used as a software carrier or hardware infrastructure support. The life cycle and flow of the ubiquitous operating system for group intelligence perception is shown in Figure 1. The task publisher inputs the original task data through the smart terminal and submits it to the platform. After the platform captures the task, it analyzes the task and gives it to the Tasks are assigned a unique task ID; when the task enters the platform, the task is analyzed and the corresponding task feature vector is generated, and the feature is spliced with other known discrete features, and the feature is extracted through the task vector. The feature includes but is not limited to the task Types, number of participants required, task execution location, required sensors and types of collected data; the ubiquitous operating system for group intelligence perception completes the user scheduling and task allocation process by performing task reasoning, association and matching operations; After the participant receives the task, he selects the task of interest to execute and uploads the collected perception data or design plan to the ubiquitous operating system for group intelligence perception; when the data enters the ubiquitous operating system for group intelligence perception , Abstract and software definition of tasks, users, and process resources. According to the description of task characteristic information, the ubiquitous operating system for group intelligence perception selects the task middleware that needs to be used and summarizes the collected data; finally returns the results to the task The publisher, the publisher evaluates and feedbacks the task results, when the publisher receives the results, the life cycle of the group intelligence task ends.

In order to solve the problem of circulation and execution of various group intelligence tasks in the group intelligence ecosystem, the present invention designs the operating system CrowdOS to implement a ubiquitous operating system oriented to group intelligence perception. The core architecture of CrowdOS and the three subsystem frameworks included in the core architecture are as follows :

Figure 2 is a block diagram of the core architecture of CrowdOS. The ubiquitous operating system for group intelligence perception is implemented by the operating system CrowdOS. It can be seen from Figure 2 that CrowdOS runs between the native operating system and upper-layer applications, including the perception end and the server end. Among them, the software carrier of the sensing end (Sensing-end) includes two types of devices. The first type is a portable intelligent sensing device with human-computer interaction functions, such as smart phones, smart watches, etc.; the second type is a fixed type deployed in the physical world. Sensors, they do not need to directly interact with people, such as car sensors, water quality detection sensors, air quality sensors, etc.; CrowdOS adopts a cloud-side-end deployment method; the sensing end is deployed on various terminal sensing devices to collect environment, business, society, For data information such as crowds, the server is deployed on cloud servers or edge servers to comprehensively manage system resources and data resources and respond to system operations in real time; the system software deployed on edge servers is usually tailored and lightweight. For example, the redundant modules are removed, and only the core data processing part, visual integrated processing and other functions are still deployed on the cloud server.

In the functional layer part of the perception terminal, first the publisher uploads the task to the CrowdOS-based system through the interactive function of the smart terminal, and the participant browses through the perception terminal and executes the tasks that have been published in the system; when the participant receives the task, the perception terminal Data collection begins; the system support layer obtains the current device status, then uniformly interface encapsulates the acquired sensor data and unifies the data transmission format, and then stores the data in the corresponding data structure through the network or Bluetooth transmission; A sensing device that does not require human-computer interaction. Once the device is activated by a task in the system and the device passes the task verification, the device will automatically collect and upload perception data according to predetermined rules;

The server-end provides comprehensive management services, which are deployed on server clusters, cloud servers or edge servers. It includes a total of eight modules and belongs to the core processing framework of the operating system. After the data is transmitted to the server, the server understands and expresses the task through the task pool module (Task Pool module), analyzes, schedules, allocates, and fine-tunes the received tasks through the task pool module, and assigns the group intelligence task to Users in the platform; in the resource management module (Resource management), software defines various resources and completes the comprehensive management of devices, users, environment, and task process scheduling (Process); then Enter the data management center module (DM center), which provides classified storage and fast retrieval functions for massive amounts of heterogeneous data. The joint storage and retrieval module (Storage and query) stores and processes the data collected through the sensing end, and extracts Useful information is transmitted to the knowledge base module; for the collected data, the task result quality optimization (TRO) module is used to optimize the data quality. After the optimization is completed, the server will send the final result back to the sensory side through the network for task release In order to complete the flow of the entire task message between the perception end and the server end. In addition, the system plug-in module (System Plugin) provides users with a wealth of system features, including privacy protection, credit evaluation, and user incentives; in the internal and external interface module (I&E interface), the internal interface is to facilitate CrowdOS developers’ access to the operating system For maintenance and update functions, the external interface is provided to third-party application developers, so that they can call the software interface and develop their personalized swarm perception application system based on the CrowdOS architecture.

In the process of implementing the CrowdOS architecture, CrowdOS abstracted various entities and virtual resources in the ubiquitous operating system for group intelligence, completed the software definition and expression, and expressed the entity and virtual resources with the relationship between symbols and symbols. ; After the task enters the system, five dynamic Agents are constructed to generate system resource maps to manage the tasks and resources in the system. The construction of Agents and the generation and management of resource maps are completed in the task pool and resource management module in Figure 2. Agents Including task agent Task-agent (TA), user agent User-agent (UA), device agent (DA), environment agent Environment-agent (EA) and process agent Process-agent (PA).

As shown in Figure 3, all resources in the system are defined by five agents. TA contains detailed information of each task, including but not limited to task type, execution time, location, and data format collected. UA is an abstract description of the user in the system and records the user's information, including but not limited to release And the tasks performed, the credit rating; DA is the description of the terminal device resources, recording the type of the device, the current status information; EA abstracts the software and hardware environment resources of the current system itself, including but not limited to the current CPU, Memory, storage usage, and the number of users in the system and the total amount of available equipment; PA manages all task processes in the current system, including but not limited to process status, priority, and scheduling strategy; Agents pass internally after obtaining task information The semaphore defined in the structure completes the real-time interaction and update between each other.

Fig. 4 task analysis and scheduling framework diagram is a detailed design of the task pool part of Task Pool in Fig. 2, which solves the first challenge proposed by the present invention. It is divided into two steps: first, deep understanding of group intelligence tasks, and fine-grained extraction of commonalities and differences of tasks; second, the system needs to allocate tasks to participants reasonably to ensure the shortest time and lowest energy consumption conditions Complete the information gathering process.

As shown in Figure 4, as shown in the A task information extraction part marked above, when the group intelligence task enters the system, the system first performs semantic analysis and feature extraction on the task. The system will perform natural language analysis on the received task. For tasks described in languages such as English, the system performs word segmentation, and then the system performs language-insensitive operations such as part-of-speech tagging, named entity recognition, and keyword extraction, and finally extracts key information of the task, including but not limited to the task execution method, location, Time, the number of participants; the system further splices the extracted task information and the discrete features obtained through regular click selection, and inputs the spliced features into a deep neural network for unified coding, and outputs a high-dimensional task The intermediate vector; finally, the vector is mapped to the task-agent of the task through decoding, and the conversion process from the task vector to the agent P1 marked below in Figure 4 is completed.

The B Agents generation part marked at the top of Figure 4 contains 5 Agents: Task_agent, Environment_agent, Process_agent, Device_agent, and User_agent. Take the structure of Task-agent as an example to expand the specific definition description; TA contains all the common and individual information of the task, It is obtained through the process of task analysis and presentation in the previous step; among them, taskID is the unique identifier of the task in the system; process_state indicates the current state of the task process, whether it is in the generation state, the execution state or the feedback state; this state assists Process_agent manages the task process. Prio represents the priority of the task, with a value of 0-15. The system schedules the task process according to the priority order; taskInfo is a structure that contains task detailed information, such as task time, location, and vector Representation, etc.; Classification represents the category to which the task belongs, such as data labeling, sensor information collection, questionnaire answering, etc.; Topic represents the topic of the task, which can be extracted from keyword information, such as audio collection, photo collection Etc.; deviceNum, deviceInfo, and deviceID respectively represent the number of devices participating in the task, detailed device information, and a list of device IDs; Sensing_Data is a pointer to the collected task data set, pointing to the address of the cube where the data is stored.

As shown in the C scheduling and allocation part marked at the top of Figure 4, the task scheduling subframe includes a strategy library, a mapping model, and a strategy management module; the system first analyzes and infers the content of the task resource map, and maps the task ID to the strategy The specific strategies in the library, where commonly used task allocation functions and their numbers are stored in the strategy library, so as to complete the task allocation strategy selection process; then the system performs scheduling operations on devices and users according to the selected allocation strategy, and pushes tasks to the appropriate User.

The resource management framework is the specific implementation of the Resource Management module Resource Management in Figure 2. The content of resource management includes but is not limited to users, sensing terminals, system environments, task processes, system software, task data, and knowledge bases. The management objects of CrowdOS are abstracted into four categories. The following describes how the system manages the four types of objects. The following four parts are part of resource management, and the four parts of management are executed in order according to the different periods when tasks enter the system.

1) Equipment, user and environmental management

When the sensing terminal accesses the system through the network, it will trigger a signal, and the device will automatically send current status information to the system, including but not limited to device type, remaining power, location information, and storage occupancy. The system developed based on the CrowdOS framework passes Device-agent The agent captures and stores state information;

The system portrays the user (task participant and publisher) portrait through User-agent, and the user interacts with the system by relying on the sensing terminal, for example, to publish tasks through the smart phone application interactive interface. User-agent saves the user's name, age, and tasks that have participated in it. At the same time, it generates user credit ratings, user preferences, areas of interest, and other personalized information based on the user's participation in tasks.

Environmental resource records server architecture and processing capabilities, for example, centralized, distributed, or edge deployment architecture, the number of CPUs in the system, system CPU occupancy, memory usage, available disk storage space, and system access. These resources are stored in the Environment-agent and updated regularly to ensure that the system obtains the latest data; the agent has an alarm function, and makes predictions based on the current system status and the increase or decrease of the task volume. If the system CPU utilization or storage is occupied If the rate reaches the rated threshold, the system will give an alarm.

2) Task process scheduling management

This part is a detailed introduction to the process (Process) management in the resource management module in Figure 2; Process-agent (PA) is a collection of task current stage status information, and the system assigns a process identification number to each task as the task process in the system The only sign of existence; this ID is stored in both Task-agent and Process-agent, and accompanies the entire life cycle of the task. The PA class contains a wealth of task process information, including but not limited to TPID: process unique identifier; Process-state: the state of the current task in the system, there are seven switchable states; Process-strategy: process scheduling strategy , Including first-come, first-served (FCFS), round-robin (RB), task priority, highest response ratio priority (HRRN), feedback priority; process_prio represents the process priority, from 0-15, 0 represents the highest priority, in descending order .

The task process state includes seven states: creation state, generation state, allocation state, execution state, processing state, feedback state and termination state; the flow relationship between the states is shown in Figure 5. First, when a user publishes a task in the system The first is the task creation state; after system authentication and analysis, it enters the task generation state; after task scheduling and assignment, it enters the distribution state; after the release is completed, participants can execute the task and then enter the execution state; the data collected by the participants Upload to the system for processing and enter the processing state; after the data is visualized, the results are summarized to the task publisher and enter the feedback state; when the task result fails to pass the publisher’s acceptance verification, the system will temporarily stay in the feedback state, and then go through reasoning and Correction, the task process returns to the generation state, the allocation state or the processing state again, and then executes sequentially; after the task submission result is verified by the publisher, the task enters the termination state, and the entire task life cycle ends. The text above the state in Figure 5 represents the subject that executes the state, which is the publisher, system, or participant. The text above the arrow indicates the operations required to go from one state to another.

The task process scheduling algorithm in the system selects one of the following methods:

1) First-come, first-served (FCFS): Prioritize tasks that enter the system first, and provide resources and services for them;

2) Circular method: Generate task interrupts at a periodic interval, place the currently running process in the task ready queue, and select the next ready process to run based on FCFS;

3) Task priority: Prioritize high-level tasks, and tasks with the same priority shall follow the FCFS principle;

4) The highest response ratio priority (HRRN), R=(w+s)/s, where R represents the response ratio, w represents the waiting time, and s represents the time expected to be served;

5) Feedback priority: For tasks that enter the feedback state, the original priority is adjusted up to two levels, and system resources are used first. The selection of the scheduling algorithm is determined according to the scheduling algorithm flag in the agent.

3) Heterogeneous multi-modal data resource management

The system manages two types of data resources. One is the data carried by the task itself, which is called raw data (RD). The second is the new data (ND) that participants upload to the system during the execution of the task. RD usually includes text, images and data sets that need to be labeled to describe the task. Various perception data uploaded by ND participants, including but not limited to text descriptions, sensor data, statistical charts, design documents, and tagged data.

For the management of multi-modal data, the steps are as follows:

1) Data needs to be collected and stored;

2) Construct an unstructured data retrieval method based on Qunzhi. Once the data preparation is completed, the system can start to build the data stack;

3) The present invention uses data cube technology to manage and store task data, construct a multi-feature cube structure (MC), and retrieve unstructured data based on the constructed data cube;

4) As the amount of tasks in the system grows, part of the completed task data and intermediate data will be regularly cleaned up, and some data that has undergone in-depth analysis will be transferred to the knowledge base for management.

4) Knowledge base management

First, the system distinguishes from existing knowledge that can improve system performance, update models, or improve the quality of task results. Information or knowledge useful to users or third parties is not included in the scope of knowledge here; The system provides corresponding mechanisms or algorithms to mine the discovered knowledge, including but not limited to deep learning algorithms, online update algorithms, and migration learning algorithms. Third, the knowledge base can extract knowledge based on the type, form, and abstraction level of the knowledge. Perform induction and database storage. Knowledge is not centrally stored in a certain management list or module in the system, it is distributed in various modules or lists of the system. The knowledge base mainly records knowledge addresses and internal relationships between knowledge, and establishes a knowledge network based on these.

The knowledge in the system is divided into two categories, one is the existing knowledge, and the other is the knowledge newly mined from tasks or data. Existing knowledge includes expert strategies, decision rules or network models defined in the system in advance. Including but not limited to the existing task allocation strategies in the strategy library, and the reasoning tree in the feedback mechanism. New knowledge mining is to identify effective, novel, potentially useful and interpretable content, methods, and models from data sets or existing knowledge. The new knowledge in CrowdOS is an extension of the original knowledge. It includes the improvement of decision-making methods, the addition of rules or the update of the network model, and does not include the knowledge of irregular and explosive expansion.

The knowledge base summarizes and stores this knowledge according to the type, form and level of knowledge. Knowledge is not centrally stored in a certain management list or module in the system, it is distributed in various modules or lists of the system. The knowledge base records the knowledge address, the internal relationship between knowledge, and the knowledge network topology.

The result quality optimization module includes an interaction layer, a reasoning layer, and an execution layer; firstly, in the interaction layer, the publisher evaluates the task results by inputting evaluation information or clicking buttons on the man-machine interface, and analyzes the diversified evaluation content ; If the evaluation shows acceptance or satisfaction, the end instruction is executed, and the task is terminated; if the result is found to have quality problems through analysis, then enter the next layer; secondly, enter the reasoning layer, and perform key information extraction and depth on the publisher’s feedback content Analyze the operation, infer the possible causes of the quality problem, and then map the reason to the problem code library according to the inference model established by the system to find the corresponding error code; third, enter the execution layer, and the problem code will be coded with the corresponding internal operation After the correction is completed, the task enters a new process state. The task results corrected through various channels will enter the interactive layer again and be fed back to the publisher, waiting for the interactive layer to give new evaluation results. The entire optimization process is step by step Execute and form a closed loop, and terminate until the publisher is satisfied with the task result.

The result quality optimization framework is the TRO framework module in the server side of Figure 2. In order to solve the problem that the quality of task results does not meet the release requirements, TRO includes a deep feedback framework based on human-computer interaction (DFHMI) to imitate the process of human thinking about problems, and rely on analysis and reasoning to solve problems. Task quality issues are divided into two categories. The first is that the number of results is sparse, and the second is that the error rate of results exceeds the standard. The optimization method used in the present invention is based on the 5Agents in the system, which is convenient for the user to interact with the system at a deeper level. The specific implementation is described in detail in FIG. 6.

Figure 6 is the result quality optimization framework diagram, including the interaction layer, the reasoning layer and the execution layer. First, in the interactive layer, the publisher evaluates the task result by inputting evaluation information or clicking the button on the human-machine interface. The system is responsible for analyzing the diversified evaluation content; if the evaluation shows acceptance or satisfaction, the system will execute the end instruction. The task is terminated; if the result of the analysis is found to have quality problems, it will enter the next layer; secondly, enter the reasoning layer, the system will perform key information extraction and in-depth analysis operations on the publisher’s feedback content to reason about the possible causes of the quality problems. Then, according to the reasoning model established by the system, the reason is mapped to the problem code library, and the corresponding error code is found; third, enter the execution layer, where the system will map the problem code and the corresponding internal operation. The system has already been Most of the mapping mechanisms are defined. These operations are implemented by modifying the values in the Agent, and other types of operations are also included. After the correction is completed, the task will enter a new process state. The task results corrected through various channels will enter the interactive layer again and be fed back to the publisher, waiting for the interactive layer to give new evaluation results. The entire optimization process is executed step by step and forms a closed loop until the publisher is satisfied with the task result.

The reasoning layer and the execution layer are connected through the mapping table (RSMT) between the reasoning decision tree and the system update operation library. The working process of RSMT is as follows:

First, find the reason for the sparse results. For those that are easily understood by people or can be directly obtained from feedback suggestions, they are called shallow reasons. The deep reasons are usually not directly obtained by the system, and they need to be obtained through joint analysis combined with task-related information.

Second, build a tree for the cause of the problem. The initial exploration is based on the decision tree model. The data sparseness problem is regarded as the root node of the tree. The first layer node represents the shallow cause, and the new shallow cause can be added as a child node of the root node. All the shallow causes are brothers to each other. node. The deep cause is usually established on the basis of the shallow cause, which is the extension of the node in the vertical direction, which is called the deep node. Third, as the scale of the problem increases, in order to reduce the retrieval time, the system regularly updates the tree such as branching and pruning to ensure that the decision tree is maintained within a certain scale.

Examples:

1. Experimental settings

The present invention develops WeSense use cases on which users can publish and perform various sensing tasks, such as road congestion information collection, air quality status monitoring, product price research, etc. During the implementation of WeSense, Ev1 and Ev2 will be evaluated based on two development methods: M1 (independent development) and M2 (development based on the CrowdOS interface). Ev3 is an evaluation of the result quality optimization mechanism (TRO), while Ev4 is the overall performance and stress test of WeSense supported by CrowdOS. Table 1 describes the development and testing environment. In Ev1 and Ev2, the time required to complete the relevant function modules under the conditions of M1 and M2 is compared and analyzed, and then the function and module F{fi} to be tested are divided into five parts (Fnum=5).

f1: Task real-time release function.

f2: Task allocation algorithm module.

f3: Privacy protection module.

f4: Crowdsourcing data collection and upload function.

f5: The result quality optimization function.

The experimental environment is set up in advance and related software is installed. We hired 9 volunteers who are familiar with the Java programming language (Λnum = 9), and gave them two weeks to develop applications. First, it took 25 minutes (timec1) to introduce the functions and all APIs of CrowdOS. First assign 9 volunteers to group A, and then switch them to group B, that is, each volunteer serves as GA and GB members at different times, GA_num=GB_num=9. GA members use M1, and GB members use M2. Each volunteer participated in all the tests, and the total number of tests was (GA_num+GB_num)*Fnum=90.

Table 1 Application operation and development environment

The comparison of the four evaluation indicators is as follows.

E _v1 : _{Comparison of M 1} and M ₂ development models, comparing _{the time consumed to complete F{f i} }, as well as the completeness and correctness of the completed tasks.

E _v2 : Compare the effect and time consumption of F{f _{i } between the two groups of G A} and G _{B testers.}

E _v3 : Compare the difference in optimization effect and time consumption between using the TRO framework and other optimization methods.

E _v4 : System stability and stress test.

2 Effectiveness and efficiency evaluation

After testing, all 90 tests passed the correctness screening, and a brief introduction to the developed WeSense user interface. As shown in Figure 7, Figure 7(a) is the homepage, showing and searching for group intelligence tasks; Figure 7(b) is the task detail page, you can click to view the task of interest; Figure 7(c) is the task submission Page: Submit the completed result data.

Figure 8 is the analysis and comparison of the development cycle, that is, the efficiency evaluation. Figure 8(a) shows the time required to complete the f _{1-3 test with G A} and G _B. Figure 8(b) shows the time consumption comparison of all tests _{based on M 1} and M _2. As shown in Figure 8(b), according to formula (1), the average development time of _{F{f i } is reduced from the original DT GA} (tc _fi )={12.1, 13.5, 7.3, 10.1, 14.1} to DT _GB (tc _fi ) = {3.1, 4.7, 0.8, 4.5, 5.4} hours.

According to formula (2), the overall development efficiency (DE) of _{f 1} → f _{5 is increased by 310%.}

Through analysis and verification tests f ₂ and f ₃ to prove its effectiveness and usability. If the M ₂ mode is used to improve performance after calling CrowdAPI, it means that CrowdOS is indeed effective. As shown in Figure 9, it is the evaluation of the effectiveness of the core framework. The tasks in Figure 9(a) are randomly assigned, and the orange circle is the geographic scope of the task release; the blue circle in Figure 9(b) uses location-based The effect of the task allocation algorithm; Figure 9(c) compares the effects of the two methods; Figure 9(d) is the selection of the super privacy protection mode.

After calling CrowdAPI, the shortened development time reflects the availability. It can be seen from Fig. 8(a) that _{compared with using M 1} , the average time consumption of f ₂ and f ₃ using M ₂ is reduced by 65.4% and 88.7%. In addition, with the expansion of the algorithm library, the advantages of CrowdOS have been highlighted. Using it can not only greatly reduce the development time of each functional module, but also improve the overall visualization effect and program readability.

3 Evaluation of result quality optimization mechanism

Through simulation experiments, the correctness and optimization effect of the application program interface in the TRO framework are evaluated.

Take the problem of data format errors as an example. For the collected data D(n*V), where n represents the number of participants, and V represents the amount of data contributed by each participant. The time taken to correct the data format through the TRO framework is T _iB =T _iζB +T _iηB , where T _iζB is the time of interface operation (T _iζB <6min), and the interface effect is shown in Figure 10(a). T _iηB is the time each participant spends on correcting the format and resubmitting the data, that is, T _{iηB ∝V} . If there is no TRO framework, the time required for the publisher to correct all data formats is T _iA . T _iA = T _iζA + T _iηA , where T _iζA is the preparation time before correcting the data format (T _iζA <30 min), T _iηA is the time spent processing the data format, T _iηA ∝n*V.

The formula (3) gives the time spent to deal with the data format problem through two optimization methods. Figure 10(c) shows how the distance between the A and B curves varies with the amount of data D(n*V). That is, the comparison of optimization methods and time consumption. Figure 10(a) is the data format correction request interface; Figure 10(b) is the correction prompt message received by the participants; Figure 10(c) compares the two optimization methods as the number of participants increases. the time consumption.

The correlation between task results and task requirements can be reflected in many ways, for example, when the requirements are provided in the form of video, but participants submit images; or the actual location of the uploaded data does not match the location in the task requirements . The optimization mechanism will re-screen users with high creditworthiness, update task features based on specific information (such as geographic location), and then re-push the task and feedback information to the original participants, as shown in Figure 10(b).

It can be seen that the use of the TRO framework not only avoids the energy consumption caused by a large amount of processing, but is also suitable for various types of tasks. The framework uses the idea of segmentation and integration to correctly transfer the work of different stages to humans or machines. As shown in Figure 10(c), compared with other optimization methods, the time consumption of TRO is relatively stable and does not increase significantly as the number of participants increases. At the same time, many optimization problems are more suitable to be solved by TRO, which can greatly reduce resource consumption compared with pure machine optimization.

4. Performance, load and stress testing

The sensory and server side of WeSense have been fully tested from the following two aspects.

Performance and load testing. Tasks of different scales were loaded sequentially, and the system response time, CPU and memory usage rate, and energy consumption of the sensor were measured. Run each test ten times, and use AndroidStudio's profile performance analyzer to monitor the data in real time while the application is running. The result is shown in Figure 11. Although the number of tasks has increased, the system response time is basically within 0.22s, and the CPU and memory usage rates also remain within the range of 3%-6% and 0.87%-1.14%. The energy consumption is basically kept below the minimum level (L1: light), which indicates that the system is functioning well.

Table 2 Stability and stress test

REC		number	1000	5000	10000	50000
CRASH		times	0	0	0	0
ANR		times	0	0	0	0
TOTAL	0	0	0	0	0

Stability and stress testing. Combines two test methods. First, the server runs continuously for 7x24 hours. During this period, the number and content of tasks are updated via mobile devices. Continuously observe and record the output logs of the sensor side and the server side, and there is no abnormal situation such as crashes or software errors. Secondly, after setting up the test environment, use Android SDKMonkey software to conduct stability and stress tests on WeSense. For example, send Monkey-pcom.hills.WeSense-v-v1000 to request execution of 1000 random command events (RCE), such as Map key, Home key. Record the number of occurrences of CRASH and ANR (application not responding). CRASH refers to the situation where the program stops or exits abnormally when an application error occurs. ANR means that when the Android system detects that the application does not respond to input events within 5 seconds or does not perform broadcast within 10 seconds, it will trigger an unresponsive prompt. The test results are shown in Table 2. Combining the above two test results, we can see that the system can operate stably and efficiently under different pressure conditions.

Claims (6)

1. A ubiquitous operating system oriented to group intelligence perception, which is characterized by:

   In the ubiquitous operating system oriented to group intelligence perception, the task publisher inputs the original task data through the smart terminal and submits it to the platform. After the task is captured by the platform, the task is analyzed and a unique task ID is assigned to the task; After entering the platform, perform task analysis and generate the corresponding task feature vector, and perform feature splicing with other known discrete features, and extract the characteristics through the task vector. The characteristics include but are not limited to the task type, the number of participants required, and the task The execution location, the type of sensors and data collected; the ubiquitous operating system for group intelligence perception completes the process of user scheduling and task allocation by performing task inference, association and matching operations; after the participants receive the task, they have a sense of choice The task of interest is executed and the collected perception data or design plan is uploaded to the ubiquitous operating system for group intelligence perception; when the data enters the ubiquitous operating system for group intelligence perception, the task, user, and process resources are performed Abstract and software definition, according to the description of task feature information, the ubiquitous operating system for group intelligence selects the task middleware to be used and summarizes the collected data; finally the results are returned to the task publisher, and the publisher performs the task results Evaluation and feedback, when the publisher receives the results, the life cycle of the group intelligence task ends.
2. The ubiquitous operating system oriented to group intelligence perception according to claim 1, characterized in that:

   The ubiquitous operating system oriented to group intelligence perception is implemented by the operating system CrowdOS. CrowdOS runs between the native operating system and upper-layer applications, and includes the sensing end and the server. The software carrier of the sensing end includes two types of equipment, the first type is Portable intelligent sensing devices with human-computer interaction functions. The second category is fixed sensors deployed in the physical world; CrowdOS adopts a cloud-side-to-end deployment method; the sensing end is deployed on various terminal sensing devices to collect data, information and services The end is deployed on a cloud server or edge server to comprehensively manage system resources and data resources and respond to system operations in real time;

   In the functional layer part of the perception terminal, first the publisher uploads the task to the CrowdOS-based system through the interactive function of the smart terminal, and the participant browses through the perception terminal and executes the tasks that have been published in the system; when the participant receives the task, the perception terminal Data collection begins; the system support layer obtains the current device status, then uniformly interface encapsulates the acquired sensor data and unifies the data transmission format, and then stores the data in the corresponding data structure through the network or Bluetooth transmission; A sensing device that does not require human-computer interaction. Once the device is activated by a task in the system and the device passes the task verification, the device will automatically collect and upload perception data according to predetermined rules;

   The server provides comprehensive management services, deployed on server clusters, cloud servers or edge servers, including task pool modules, resource management modules, data management center modules, internal and external interface modules, knowledge base modules, system plug-in modules, and tasks Result quality optimization and eight modules of joint storage and retrieval modules; first, after the task publisher transmits the task data from the sensing end to the server, the server understands and expresses the task through the task pool module, and receives the data through the task pool module. Analyze, schedule, allocate and fine-tune the tasks of the company, and assign the group intelligence tasks to the users in the platform; in the resource management module, software defines various resources and completes the comprehensive management of equipment, users, environment and task process scheduling ; Then enter the data management center module, which provides classified storage and fast retrieval of massive heterogeneous data. The combined storage and retrieval module stores and processes the data collected through the sensing terminal, extracts useful information and transmits it to the knowledge base Module; for the collected data, the task result quality optimization module is used to optimize the data quality. After the optimization is completed, the server sends the final result back to the task publisher of the sensor through the network, so as to complete the entire task message on the sensor and the server. The system plug-in module provides users with privacy protection, credit evaluation and user incentives; in the internal and external interface modules, the internal interface is used to maintain and update the operating system, and the external interface is provided to third-party applications to call the software interface .
3. The ubiquitous operating system oriented to group intelligence perception according to claim 2, characterized in that:

   In CrowdOS, five dynamic Agents are constructed to generate system resource maps after the task is entered to manage the tasks and resources in the system. Agents include task agent Task-agent, user agent User-agent, and device agent Device-agent. , Environment-agent and Process-agent; TA contains detailed information about each task, including but not limited to task type, execution time, location, collected data format, UA is an abstraction of users in the system Description, record the user’s information, including but not limited to the tasks issued and performed, and credit rating; DA is the description of the terminal equipment resources, recording the type of the equipment, current status information; EA abstracts the current software system itself Hardware environment resources, including but not limited to the current CPU, memory, storage usage and the number of users in the system and the total amount of available equipment; PA manages all task processes in the current system, including but not limited to process status, priority, and scheduling strategy ; Agents complete mutual real-time interaction and update through the semaphore defined in the internal structure after obtaining the task information.
4. The ubiquitous operating system oriented to group intelligence perception according to claim 2, characterized in that:

   In the task pool, in the task information extraction part, when the group intelligence task enters the system, the task is firstly analyzed by semantic analysis and feature extraction, and natural language analysis is performed on the received task. For tasks described by language, the system performs word segmentation processing. Then the system performs operations that do not distinguish between languages, and finally extracts mission-critical information, including but not limited to the task execution method, location, time, number of participants, the extracted task information and the discrete features obtained through rule click selection For splicing, input the spliced features into a deep neural network for unified encoding, and output a high-dimensional task intermediate vector; finally, map the vector to the task-agent of the task through decoding, and complete the conversion process from task vector to agents ；

   In the task pool, the Agent generation part contains 5 Agents: Task_agent, Environment_agent, Process_agent, Device_agent, User_agent, which are obtained through the process of task analysis and representation; among them, taskID is the unique identifier of the task in the system; process_state represents the The current state of the task process, whether it is in the generation state, the execution state or the feedback state, etc.; this state assists Process_agent in task process management, Prio represents the priority of the task, the value is 0-15, and the system performs the task according to the priority order Process scheduling; taskInfo is a structure that contains detailed task information, such as task time, location, vector representation, etc.; Classification represents the category to which the task belongs, such as data labeling, sensor information collection, questionnaire answering, etc.; Topic Represents the subject of the task, extracted from keyword information, such as audio collection, photo collection, etc.; deviceNum, deviceInfo, and deviceID represent the number of devices participating in the task, device detailed information, and device ID list; Sensing_Data is collected Pointer to the task data set, pointing to the address of the cube where the data is stored;

   In the task pool, the scheduling and allocation part, the task scheduling sub-framework includes a strategy library, a mapping model, and a strategy management module; first, the content of the task resource map is analyzed and reasoned, and the task ID is mapped to the strategy library. The task allocation function and its number are stored in the library, so as to complete the task allocation strategy selection process; then according to the selected allocation strategy, perform scheduling operations on the devices and users, and push tasks to appropriate users.
5. The ubiquitous operating system oriented to group intelligence perception according to claim 2, characterized in that:

   The resource management module includes, but is not limited to, users, perception terminals, system environments, task processes, system software, task data, and knowledge bases. The management objects of CrowdOS are abstracted into 4 categories, and they are executed sequentially according to the different periods when tasks enter the system. as follows:

   1) Equipment, user and environmental management

   When the sensing terminal is connected through the network, the signal is triggered, and the device automatically sends the current status information to the system, including but not limited to device type, remaining power, location information, and storage occupancy. The system developed based on the CrowdOS framework uses the Device-agent agent Capture and store status information;

   The system portrays the user portrait through the User-agent, and the user interacts with the system on the perceptual end. The User-agent saves the user's name, age, and tasks participated in, and at the same time generates user credit ratings, user preferences, and areas of interest based on the user's participation in tasks And other personalized information;

   Environment resource records server architecture and processing capabilities. Resources are stored in the Environment-agent and updated regularly. The agent has an alarm function and makes predictions based on the current system status and the increase or decrease of the task volume. If the system CPU utilization rate or storage occupancy rate is When the rated threshold is reached, the system will give an alarm;

   2) Task process scheduling management

   Process-agent is a collection of the status information of the current stage of the task. A process identification number is assigned to each task as the only sign of the existence of the task process in the system; the ID is stored in both the Task-agent and the Process-agent and accompanies the entire life cycle of the task; The PA class contains task process information, including but not limited to TPID: process unique identifier; process_state: the current state of the task in the system, there are seven switchable states; process_strategy: process scheduling strategy, including first come, first served ( FCFS), round-robin method (RB), task priority, highest response ratio priority (HRRN), feedback priority; process_prio represents the process priority, from 0-15, 0 represents the highest priority, in descending order;

   The task process state includes seven states: creation state, generation state, allocation state, execution state, processing state, feedback state and termination state; the flow relationship between states is: First, when a user publishes a task in the system, the first is task creation State; enter the task generation state after passing system authentication and analysis; enter the distribution state after task scheduling and assignment; after the release is completed, participants can execute the task, and then enter the execution state; participants upload the collected data to the system After processing, enter the processing state; after data visualization, the results are summarized to the task publisher and enter the feedback state; when the task result fails the publisher’s acceptance verification, the system will temporarily stay in the feedback state, and then after reasoning and correction, the task progresses Return to the generation state, the allocation state or the processing state again, and then execute sequentially; after the publisher verifies the task submission result, the task enters the termination state, and the entire task life cycle ends;

   The task process scheduling algorithm in the system can choose one of the following methods:

   1) First-come, first-served (FCFS): Prioritize tasks that enter the system first, and provide resources and services for them;

   2) Circular method: Generate task interrupts at a periodic interval, place the currently running process in the task ready queue, and select the next ready process to run based on FCFS;

   3) Task priority: Prioritize high-level tasks, and tasks with the same priority shall follow the FCFS principle;

   4) The highest response ratio priority (HRRN), R=(w+s)/s, where R represents the response ratio, w represents the waiting time, and s represents the time expected to be served;

   5) Feedback priority: For tasks that enter the feedback state, two levels will be raised on the basis of the original priority, and system resources will be used first;

   3) Heterogeneous multi-modal data resource management For the management of multi-modal data, the steps are as follows:

   1) Collect and store data;

   2) Construct an unstructured data retrieval method based on Qunzhi. Once the data preparation is completed, start to build the data stack;

   3) Using data cube technology to manage and store task data, construct a multi-character cube structure (MC), and retrieve unstructured data based on the constructed data cube;

   4) As the amount of tasks in the system grows, the completed task data and intermediate data are regularly cleaned up, and the data that has undergone in-depth analysis will be transferred to the knowledge base for management;

   4) Knowledge base management

   OS knowledge is divided into two categories: existing knowledge (EK) and new knowledge (NK); NK is extracted from tasks or data, which helps to improve system mechanisms or update models; the process of knowledge management is as follows:

   First, the system distinguishes from existing knowledge that can improve system performance, update models, or improve the quality of task results. Information or knowledge useful to users or third parties is not included in the scope of knowledge here; The system provides corresponding mechanisms or algorithms to mine the discovered knowledge, including but not limited to deep learning algorithms, online update algorithms, and migration learning algorithms. Third, the knowledge base can extract knowledge based on the type, form, and abstraction level of the knowledge. Perform induction and database storage.
6. The ubiquitous operating system oriented to group intelligence perception according to claim 2, characterized in that:

   The result quality optimization module includes an interaction layer, a reasoning layer, and an execution layer; firstly, in the interaction layer, the publisher evaluates the task results by inputting evaluation information or clicking buttons on the man-machine interface, and analyzes the diversified evaluation content ; If the evaluation shows acceptance or satisfaction, the end instruction is executed, and the task is terminated; if the result is found to have quality problems through analysis, then enter the next layer; secondly, enter the reasoning layer, and perform key information extraction and depth on the publisher’s feedback content Analyze the operation, infer the possible causes of the quality problem, and then map the reason to the problem code library according to the inference model established by the system to find the corresponding error code; third, enter the execution layer, and the problem code will be coded with the corresponding internal operation After the correction is completed, the task enters a new process state. The task results corrected through various channels will enter the interactive layer again and be fed back to the publisher, waiting for the interactive layer to give new evaluation results. The entire optimization process is step by step Execute and form a closed loop, and terminate until the publisher is satisfied with the task result.

Images