CN113760558A

CN113760558A - Data processing method, device and computer storage medium

Info

Publication number: CN113760558A
Application number: CN202010486508.2A
Authority: CN
Inventors: 林世勤; 孙卓金; 熊健
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-06-01
Filing date: 2020-06-01
Publication date: 2021-12-07

Abstract

The embodiment of the invention provides a data processing method, data processing equipment and a computer storage medium, wherein the data processing method comprises the following steps: receiving data to be processed sent by an application program and parameter information corresponding to the data to be processed through an external interface of system service in an operating system; determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data; and returning feedback data to the application program through the external interface. The user can package various data processing models into system service and perform data interaction externally through the external interface, so that the application program can realize data processing by using various data processing models, the data processing model can be called in the application to perform data processing without deeply learning the data processing model, the development difficulty of the related application of the data processing model is reduced, and the efficiency of developing the application is improved.

Description

Data processing method, device and computer storage medium

Technical Field

The present invention relates to the field of artificial intelligence technology, and in particular, to a data processing method, device and computer storage medium.

Background

Artificial Intelligence (AI) is a technical science for studying and developing theories, methods, techniques and application systems for simulating, extending and expanding human Intelligence, and in the field of Artificial Intelligence, different functions can be realized by establishing different data processing models. However, in the process of establishing the data processing model, the inference engine, and some preprocessing and post-processing technologies need to be known to a certain extent, and then the technologies are combined to make an AI scene on the application program side, so that the development difficulty of the artificial intelligence related application is increased.

Disclosure of Invention

Embodiments of the present invention provide a data processing method, an apparatus, and a computer storage medium to solve some or all of the above problems.

According to a first aspect of the embodiments of the present invention, there is provided a data processing method, including: receiving data to be processed sent by an application program and parameter information corresponding to the data to be processed through an external interface of system service in an operating system; determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data; and returning feedback data to the application program through the external interface.

According to a second aspect of embodiments of the present invention, there is provided a data processing apparatus including: a transmission module and a system service module; the transmission module is used for receiving data to be processed sent by an application program and parameter information corresponding to the data to be processed through an external interface of system service in an operating system; the system service module is used for determining a data processing model matched with the parameter information in the system service and calling the data processing model to process the data to be processed to obtain feedback data; and the transmission module is also used for returning feedback data to the application program through the external interface.

According to a third aspect of embodiments of the present invention, there is provided an electronic apparatus, including: the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the corresponding operation of the data processing method of the first aspect.

According to a fourth aspect of embodiments of the present invention, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the data processing method of the first aspect.

According to the data processing method, the data processing device and the computer storage medium, the data to be processed sent by the application program and the parameter information corresponding to the data to be processed are received through the external interface of the system service in the operating system; determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data; and returning feedback data to the application program through the external interface. The user can package various data processing models into system service and perform data interaction externally through the external interface, so that the application program can realize data processing by using various data processing models, the data processing model can be called in the application to perform data processing without deeply learning the data processing model, the development difficulty of the related application of the data processing model is reduced, and the efficiency of developing the application is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present invention, and it is also possible for a person skilled in the art to obtain other drawings based on the drawings.

FIG. 1a is a diagram illustrating a system architecture provided in the related art;

FIG. 1b is a schematic diagram of a system architecture according to an embodiment of the present application;

fig. 2 is a flowchart of a data processing method according to an embodiment of the present application;

fig. 3 is a flowchart of a data processing method according to a second embodiment of the present application;

fig. 3a is a schematic diagram of a system architecture according to a second embodiment of the present application;

fig. 3b is a schematic diagram of another system architecture according to the second embodiment of the present application;

fig. 4 is a flowchart of a data processing method according to a third embodiment of the present application;

fig. 4A is a schematic view of a speech recognition scene according to a third embodiment of the present application;

fig. 5 is a flowchart of a data processing method according to a fourth embodiment of the present application;

fig. 6 is a block diagram of a data processing device according to a fifth embodiment of the present application;

fig. 7 is a block diagram of another data processing apparatus according to a fifth embodiment of the present application;

fig. 8 is a block diagram of a further data processing device according to a fifth embodiment of the present application;

fig. 9 is a block diagram of a further data processing apparatus according to a fifth embodiment of the present application;

fig. 10 is a block diagram of an electronic device according to a sixth embodiment of the present application.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.

The following further describes specific implementation of the embodiments of the present invention with reference to the drawings.

Artificial intelligence is widely applied to various fields, artificial intelligence generally utilizes an artificial intelligence model to realize different functions, and many application programs contain the artificial intelligence model to realize corresponding functions, as shown in fig. 1a, the application programs contain the artificial intelligence model, if data is processed through the artificial intelligence model in the running process of the application program, a reasoning engine in an operating system is called to realize data operation to obtain an operation result, in the development stage of the application program, a developer needs to embed a complete program of the artificial intelligence model in the application program, which comprises the preprocessing, the model reasoning, the post-processing and the like of the artificial intelligence model, the developer needs to deeply know about the artificial intelligence model, which has great difficulty for the corresponding developer and also takes a great deal of time.

The first embodiment,

Referring to fig. 1b, in a system architecture of an operating system to which a system service according to an embodiment of the present disclosure is applied, the system service may include a technology stack, where the technology stack includes an external interface and at least one data processing model, and the data processing model may include an AI model, and the technology stack may include an AI scenario technology stack, where the external interface and the at least one data processing model are packaged together in the system service for a user to use. For example, in the development process of the application program, the required data processing model can be used for data processing only by calling the system service without deeply knowing the data processing model, so that the difficulty of opening the application program related to the data processing model is reduced, and the development efficiency is improved.

Compared with the system architecture of the operating system shown in fig. 1a, in the system architecture of the operating system shown in fig. 1b, the entire system service can be directly called by the application program, the function of the data processing model can be realized, the system service integrates all functions of the entire data processing model, an application developer does not need to know the data processing model too much, and only needs to exchange data with the system service through an external interface provided by the system service in the application program, call the system service to execute corresponding operation, and obtain feedback data through the external interface. The difficulty of an application developer in developing the data processing model related application program is greatly reduced.

The technical solution of the present application will be described in detail with reference to the system architecture of the operating system shown in fig. 1 b. An embodiment of the present application provides a data processing method, which may be applied to the system architecture shown in fig. 1b, and it should be noted that an execution main body of the data processing method may be an electronic device, where the electronic device is installed with an operating system, and may execute an application program, and in particular, may perform an operation of a data processing model. The Operating System may include a mobile phone Operating System, a computer Operating System, a Real Time Operating System (RTOS), and the like, which is not limited in the present application. The electronic device can be a smart phone, a smart sound box, a tablet computer, a smart watch, a vehicle-mounted terminal and the like. Referring to fig. 2, fig. 2 is a flowchart of a data processing method according to an embodiment of the present application, where the method includes the following steps:

step 201, receiving data to be processed sent by an application program and parameter information corresponding to the data to be processed through an external interface of a system service in an operating system.

The data to be processed is data to be processed by a data processing model, the parameter information is used for determining the data processing model for processing the data to be processed, and the parameter information may include an identifier of the data processing model.

The system service may comprise a technology stack, for example, the system service comprises an AI scenario technology stack, the system service comprises at least one data processing model (e.g., AI model) for processing images and/or speech, and the functions of the system service may refer to the functions of the system service in the system architecture shown in fig. 1 b.

The data to be processed (or the collected data to be detected) may be collected by the electronic device, or may be collected by other devices, and then transmitted to the electronic device. The data to be processed may include at least one of image data and voice data, etc., and the type of the data to be processed is different for different tasks. For example, if the target task is image detection or image recognition, the data to be processed corresponds to the acquired image data; and if the target task is voice recognition or voice detection, the data to be processed corresponds to the collected voice data.

For example, in order to facilitate data input to the system service, the external interface includes a first data interface, where the first data interface is an interface for inputting data to the system service, and receives data to be processed sent by an application program and parameter information corresponding to the data to be processed through the external interface of the system service in the operating system, and the method includes: and receiving to-be-processed data and parameter information corresponding to the to-be-processed data sent by the application program through a first data interface.

Step 202, determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data.

It should be noted that the parameter information may be generated by the data processing model that the application program needs to call in the running process, the parameter information is matched with the data processing model that the application program needs to call, the parameter information may include an identifier of the data processing model that the application program needs to call, of course, the parameter information may also correspond to the application program and the data processing model, the parameter information may include an identifier of the application program, and the corresponding data processing model may be determined according to the identifier information of the application program.

Here, a specific example that one system service further includes an inference engine is used to describe a process of processing data to be processed by using the system service, determine a data processing model matched with parameter information in the system service, and call the data processing model to process the data to be processed to obtain feedback data, where the process includes:

determining a data processing model matched with the parameter information in the system service; preprocessing data to be processed by using a data processing model to obtain data to be operated; invoking an inference engine to perform data processing model inference on data to be operated to obtain an operation result; and carrying out post-processing on the operation result to obtain feedback data.

In a specific implementation, invoking an inference engine to perform data processing model inference on data to be operated to obtain an operation result, including: invoking an inference engine to analyze the data processing model, and sending a calculation instruction to at least one processor according to an analysis result to calculate the data to be operated; and obtaining an operation result according to the data operated by the at least one processor.

With reference to the system architecture shown in fig. 1b, the process of performing data processing by using the data processing model may include three processes, namely, pre-processing, data processing model inference and post-processing, specifically, in an alternative implementation, the pre-processing and the post-processing include data processing by using at least one algorithm, and further, the pre-processing may include data processing by using at least one image algorithm/voice algorithm, for example, the pre-processing may include image size normalization processing, image rotation correction processing, image binarization processing, voice data filtering processing, and the like. The post-processing may include data processing using at least one mapping algorithm, and mapping the operation result of the data processing model to obtain feedback data.

It should be noted that, in a specific example, the data processing model may correspond to a scene, and the corresponding data processing model is selected to process the data to be processed by determining the scene. For example, optionally, the parameter information includes a scene parameter and a model parameter, a data processing model matched with the parameter information in the system service is determined, and the data processing model is invoked to process the data to be processed to obtain feedback data, including:

determining a scene corresponding to a scene parameter in a system service in a scene mode; determining a data processing model corresponding to the model parameters in at least one data processing model corresponding to the scene; and calling a data processing model to process the data to be processed to obtain feedback data. It should be noted that the scene may be an application scene of the application program, for example, the scene may include an object recognition scene, a target tracking scene, a face recognition scene, a gesture recognition scene, and the like, which is only exemplary and not meant to limit the present application, and different scenes correspond to different data processing models, so that only the scene of the application program needs to be determined, and the corresponding data processing model can be determined, which is more convenient.

Based on the above example, in an implementation manner, a user may actively control whether to enter a scene mode, and in the scene mode, the user may automatically determine a scene and invoke a data processing model corresponding to the scene to process data to be processed, for example, the method further includes:

and receiving scene mode switching operation, and switching the system service into a scene mode according to the scene mode switching operation. The method comprises the steps of entering a scene mode through scene mode switching operation, receiving common mode switching operation, switching system service into the common mode according to the common mode switching operation, determining a data processing model to be called according to user selection without scene determination in the common mode, and enabling user selection to be more flexible and meeting different user requirements. Of course, this is merely an example and does not represent a limitation of the present application.

Based on the above example, in an implementation manner, a user may add a new scene or delete a scene by himself, for example, the method further includes:

receiving operation of establishing a scene, and generating new scene information according to the operation of establishing the scene, wherein the new scene information comprises a data processing model corresponding to the new scene; and establishing a new scene according to the new scene information, and updating the system service. The user can manage the scene such as adding and deleting, and the like, so that the method is more flexible and further meets more user requirements.

Based on the above example, the associated scenes may be combined to form a preset mode, for example, in an implementation manner, the scene parameters correspond to at least two associated scenes, and the invoking of the data processing model processes the data to be processed to obtain feedback data includes:

and calling the combined models corresponding to at least two associated scenes to process the data to be processed to obtain feedback data, wherein the combined models are obtained by combining the data processing models respectively corresponding to the at least two associated scenes. And the associated scenes form a preset mode, and the data processing models corresponding to the associated scenes are combined, so that the use is more convenient for a user, and the whole data processing process can be completed by calling once.

Based on the above example, different charges may be applied to different scenarios, and of course, different charges may also be applied to a preset mode formed by the associated scenario, for example, in one implementation, the method further includes: determining consumption information corresponding to the scene parameters; and returning consumption information to the application program through the external interface.

Based on the above example, for the scenes in the system service of the application, the scenes can be sorted according to the calling frequency of the scenes, and the higher the calling frequency is, the earlier the sorting is; or, sorting according to the charging standard of the scene, wherein the higher the charging scene is, the higher the sorting is; of course, the sequence of the scenes is only exemplary, and the sequence can be displayed to the user, so that the user can more intuitively see the calling situation of each scene, and the user can use the sequence conveniently.

And step 203, returning feedback data to the application program through the external interface.

In conjunction with the description of the external interface in step 201, optionally, the feedback data may be input from the system service to the application program through the external interface between the system service and the application program. The external interface can be an interface between the system service and external communication, and data exchange between the application program and the system service can be realized through the external interface. In the application program, after the feedback data is returned to the application program, the final detection/identification result can be displayed to the user.

Illustratively, the external interface includes a second data interface, the second data interface is an interface for outputting data from the system service to the outside, and the feedback data is returned to the application program through the external interface, including: and returning the feedback data to the application program through the second data interface.

According to the data processing method provided by the embodiment of the application, the data to be processed and the parameter information corresponding to the data to be processed, which are sent by an application program, are received through an external interface of a system service in an operating system; determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data; and returning feedback data to the application program through the external interface. The user can package various data processing models into system service and perform data interaction externally through the external interface, so that the application program can realize data processing by using various data processing models, the data processing model can be called in the application to perform data processing without deeply learning the data processing model, the development difficulty of the related application of the data processing model is reduced, and the efficiency of developing the application is improved.

Example II,

Based on the data processing method described in the first embodiment, a second embodiment of the present application provides a data processing method, which is a further description of the data processing method described in the first embodiment, as shown in fig. 3, fig. 3 is a flowchart of the data processing method provided in the second embodiment of the present application, and the method includes the following steps:

step 301, inputting data to be processed and parameter information corresponding to the data to be processed into a system service through a first data interface.

Optionally, when the application executes a target task, the data to be processed is acquired, where the target task may be any one task in the execution process of the application, and the target task is a task that needs to be implemented by using a data processing model, and for example, the target task may include a face recognition task, an object recognition task, a gesture recognition task, an object classification task, a voice recognition task, and the like.

The data to be processed (or the collected data to be detected) may be collected by the electronic device, or may be collected by other devices, and then transmitted to the electronic device. The data to be processed may include at least one of image data and voice data, and the type of the data to be processed is different for different tasks. For example, if the target task is image detection or image recognition, the data to be processed is the acquired image data; and if the target task is voice recognition or voice detection, the data to be processed is the collected voice data.

Step 302, determining a data processing model matched with the parameter information in the system service.

The application programs are different, and the called data processing models may be different, for example, if the application program is a face recognition program, the called data processing model may be a face recognition model; as another example, if the application is a voice detection application, the invoked data processing model may be a voice detection model.

Further optionally, in the running process of the application program, the executed tasks are different, and the invoked data processing models may also be different, and when the application program executes the target task, the target task may be a task completed by the corresponding data processing model, and the target task may also be a function that can be realized by the corresponding data processing model, that is, the data processing model corresponds to the task that can be completed by the data processing model. For example, if the target task is face recognition, the data to be processed is acquired image data, and the acquired image data is input into a face recognition model in the system service, so that feedback data can be obtained; and if the target task is voice detection, the data to be processed is collected voice data, and the collected voice book is input into a voice detection model in the system service, so that feedback data can be obtained.

It should be noted that the correspondence between the tasks and the data processing models may be maintained through a preset map, that is, the preset map may include a correspondence between at least one task and at least one data processing model, and the preset map may be represented by a table, a function, a formula, and the like, and of course, may also be represented by other manners, which is only exemplary described here, and the present embodiment does not limit this.

And 303, preprocessing the data to be processed by utilizing the determined data processing model to obtain the data to be operated.

And 304, calling a reasoning engine to carry out data processing model reasoning on the data to be calculated to obtain a calculation result.

In a specific implementation manner, invoking an inference engine to perform data processing model inference on data to be operated to obtain an operation result, including: invoking an inference engine to analyze the data processing model, and sending a calculation instruction to at least one processor according to an analysis result to calculate the data to be operated; and obtaining an operation result according to the data operated by the at least one processor. In particular, the function of the inference engine is illustrated. The method comprises the steps of analyzing and splitting the operation of each layer of a data processing model by using an inference engine, generating at least one calculation instruction, namely splitting all the operations of each layer of data to be operated by the data processing model into a plurality of simple operation processes, allocating corresponding processors to the simple operation processes for processing, generating the calculation instruction according to the allocated results, wherein the calculation instruction can instruct the processors to execute the corresponding operation processes, respectively sending the at least one calculation instruction to the corresponding processors, sending the calculation instruction and the data to be operated of the corresponding processors to the processors together, obtaining the operated data after the processors perform the operation, analyzing and splitting each layer of the data processing model, performing the operation by using the processors, and obtaining the operation result according to the data operated by the at least one processor.

And 305, performing post-processing on the operation result to obtain feedback data.

The process of data processing by using the data processing model may include three processes of preprocessing, data processing model inference and post-processing, specifically, in an alternative implementation, the preprocessing and the post-processing include data processing by using at least one algorithm, and further, the preprocessing may include data processing by using at least one image algorithm/voice algorithm, for example, the preprocessing may include image size normalization processing, image rotation correction processing, image binarization processing, voice data filtering processing, and the like. The post-processing may include data processing using at least one mapping algorithm, and mapping the operation result of the data processing model to obtain feedback data.

And step 306, returning the feedback data to the application program through the second data interface.

The first data interface and the second data interface can be the same interface or two independent interfaces, namely, data interaction between the system service and the application program can be realized through one unified external interface, data transmission from the application program to the system service can also be realized through the two data interfaces, and data transmission from the system service to the application program can also be realized. Of course, the above are exemplary illustrations and do not represent a limitation of the present application.

With reference to the description of steps 301-306, a system architecture diagram is provided in the second embodiment of the present application, as shown in fig. 3 a. In fig. 3a, the operating system includes an encapsulated AI scene technology stack, the AI scene technology stack belongs to a system service, that is, the system service includes the AI scene technology stack, fig. 3a illustrates the AI scene technology stack as an example, and does not represent that the present application is limited thereto, and the AI scene technology stack includes an AI scene engine (i.e., an external interface), an AI scene service processing module, and an inference engine. The upper layer applications are also shown in fig. 3 a.

The AI scenario service processing module includes at least one AI model (i.e. data processing model), and fig. 3a illustrates some functions of the AI scenario service processing module from the perspective of a data processing flow, including preprocessing, AI model inference (i.e. data processing model inference) and post-processing, which are integrated to complete tasks of various AI models.

Specifically, the preprocessing refers to preprocessing the acquired data to be processed, and the preprocessing includes processing the acquired data to be processed in order to improve detection and recognition accuracy. The acquired data to be processed (or acquired data to be detected) may include image data, voice data, and the like, and the preprocessing includes performing size normalization on the image data, that is, scaling the acquired image so that the size of the image meets the requirements of the AI model; the preprocessing can also comprise the step of carrying out rotation correction on the image data, namely, the acquired image is rotated so as to be convenient for identification and detection; the pre-processing can also comprise the binarization processing of the image, and in some scenes, the binarization processing of the image can be convenient for identification and detection; the preprocessing may also include adjusting parameters such as brightness and saturation of the image data; the preprocessing also can include filtering the voice data, and filtering noise to improve detection accuracy. Of course, this is only an exemplary description, and in each execution process, only the processing useful for the detection and the identification is executed, and not all the processing modes included in the preprocessing are executed once.

The AI model inference refers to a process of inputting data into an AI model and then using the AI model to perform operations on the input data, as shown in fig. 3a, the AI model inference requires to call an inference engine, the inference engine can analyze the computation of each layer in the AI model, split the complex logic computation into many small and simple computation tasks, and allocate the computation tasks to each processor for computation, or allocate hardware resources to the computation tasks for computation to obtain the computation result of the AI model.

The post-processing refers to processing the operation result of the AI model to obtain feedback data transmitted to the application layer.

The pre-processing and post-processing processes need to call corresponding algorithms in an image/voice algorithm library for processing, and the image/voice algorithm library comprises at least one image/voice algorithm, such as an image size normalization algorithm, an image rotation correction algorithm, an image binarization algorithm, a voice data filtering algorithm, a mapping algorithm, and the like.

The external interface is an interface for data interaction between the AI scene technology stack and an upper application program, when the application program needs to call the AI scene technology stack, the data is transmitted into the AI scene technology stack through the external interface, and the AI scene technology stack returns feedback data (namely detection and identification results) to the application program through the external interface.

With reference to the system architecture shown in fig. 3a, fig. 3b is a schematic diagram of another system architecture provided in the second embodiment of the present application, where the system architecture shown in fig. 3b is a further schematic diagram of the system architecture shown in fig. 1b from an AI model perspective, and fig. 3b mainly shows which AI models and AI scenes that an AI scene technology stack can include, where the scenario described in the first embodiment is an AI scene for example, which does not represent that the present application is limited thereto. As shown in fig. 3b, the AI scene technology stack may include an image classification model, an image detection model, a voice recognition model, and the like, which may be applied to different scenes, for example, the image detection model may be used for a target tracking scene, and the image classification model may be used for object recognition, face recognition, gesture recognition, and the like; as another example, the speech detection model and the speech recognition model may be used in scenarios such as speech control. The preprocessing and post-processing of these AI models are based on various image algorithm processes and speech algorithm processes. In fig. 3b, two functional modules of the AI scenario engine (i.e. external interface) are also shown, one is an AI scenario service communication module (i.e. first data interface) and the other is an AI scenario service interface (i.e. second data interface), the AI scenario service communication module is used for transmitting data of the application program to the AI scenario technology stack, and the AI scenario service interface is used for transmitting data of the AI scenario technology stack to the application program. Fig. 3b also shows hardware resources provided by an operating system kernel (kernel), and the AI scenario technology stack implements various functions by using the hardware resources provided by the operating system kernel, including a Central Processing Unit (CPU), a Memory (Memory), Display hardware (Display), an image processor (GPU), and the Display hardware may include a Display card, and of course, fig. 3a and fig. 3b are only two system architectures provided from different angles in the embodiment of the present application, and do not represent that the present application is limited thereto.

In connection with the data processing method described in the second embodiment and the AI scenario corresponding to fig. 3b, three specific examples are listed here to illustrate the application of the data processing model corresponding to various scenarios.

In a first example, taking a Video Blog (english: Video Blog, Vlog) as an example, in a process of making a Video by the Video Blog, a Video material is usually shot first, in the process of shooting the Video material, a camera can follow a target object, the camera collects an image as data to be processed, the data to be processed is input to a system service through a first data interface, a target tracking scene corresponding to parameter information in the system service is determined, an image detection model corresponding to the target tracking scene is called to perform image detection on the data to be processed, feedback data is obtained according to a detection result, and the feedback data is returned through a second data interface. The feedback data may indicate an offset of the target object from the center of the image in the image, and the camera angle is adjusted according to the offset such that the target object is located at the center of the image. Of course, the exemplary description is indicated here, and it may also be ensured that the target object is located in a preset area of the image in the target tracking process, which is not limited in this application.

In a second example, taking live video as an example, in the live video process, filtering processing such as face beautification can be performed on a face, and a face area needs to be determined by recognizing the face in an image. The camera collects images as data to be processed, the data to be processed is input into the system service through the first data interface, a face recognition scene corresponding to parameter information in the system service is determined, a face recognition model corresponding to the face recognition scene is called to perform face recognition on the data to be processed, feedback data are obtained according to a recognition result, and the feedback data are returned through the second data interface. The feedback data may indicate a face region in the image, and the face region in the image is filtered according to the feedback data.

In a third example, taking security monitoring as an example, a preset area is monitored, and if a dangerous event is determined to occur, a warning is given. The camera acquires an image as data to be processed, the data to be processed is input into the system service through the first data interface, an image detection scene corresponding to parameter information in the system service is determined, a face recognition model corresponding to the image detection scene is called to perform image detection on the data to be processed, feedback data are obtained according to a detection result, and the feedback data are returned through the second data interface. The feedback data may indicate whether a hazardous event is contained in the image and a warning may be issued if the feedback data indicates that a hazardous event is occurring. Of course, the three examples are merely illustrative and do not represent that the present application is limited thereto.

Example III,

With reference to the system architecture shown in fig. 1b in the first embodiment, based on the data processing method shown in fig. 2 in the first embodiment, a third embodiment of the present application provides a data processing method based on the data processing method shown in fig. 2, where the data processing method provided in the third embodiment of the present application specifically uses processing of voice data as an example to further describe the data processing method shown in fig. 2, and of course, the present embodiment is only an exemplary description, and does not represent that the present application is limited thereto. Referring to fig. 4, fig. 4 is a flowchart of a data processing method according to a third embodiment of the present application, where the method includes the following steps:

step 401, the smart sound box collects user voice to obtain initial voice data.

The smart speaker may include a speaker and a microphone, and may acquire the user voice through the microphone to obtain initial voice data, where the initial voice data is to-be-processed data of the application program described in the first embodiment.

Step 402, inputting initial voice data into a system service through a first data interface in the system service.

The first data interface may be an interface for inputting data from an application into a system service. It should be noted that the operating system of the smart speaker may be a real-time operating system, and the system service is based on a function module of the operating system, which is, of course, only described here by way of example, and the operating system of the smart speaker may also be another operating system, which is not limited in this application.

And 403, processing the initial voice data through a voice recognition model in the system service to obtain feedback data.

The speech recognition model belongs to a data processing model, and the processing of the initial speech data by the speech recognition model can comprise the processes of preprocessing, speech recognition model reasoning and postprocessing. Specifically, the preprocessing comprises denoising and boosting offsetting initial voice data, preprocessing the initial voice data to obtain voice data to be operated, performing voice recognition model reasoning on the voice data to be operated, calling a reasoning engine to analyze a voice recognition model, determining at least one operation task for the voice data to be operated, distributing the operation tasks to each processor, operating the voice data to be operated by the processors according to the operation tasks, obtaining final operation results according to the data operated by each processor, and performing postprocessing on the operation results to obtain feedback data, wherein the feedback data comprises the acquired result of the voice recognition of the user.

Step 404, returning the feedback data to the application program through a second data interface in the system service.

The application program can respond to the user voice according to the feedback data, for example, the recognition result contained in the feedback data is a wake-up voice, that is, the user wakes up the smart sound box, and then reply voice such as "i am on" can be played; for another example, if the identification result included in the feedback data is to play a certain resource, which may include an audio resource, a video resource, an image resource, and the like, the corresponding resource is obtained from the network for playing.

With reference to the description of step 401 and step 404, as shown in fig. 4A, fig. 4A is a scene schematic diagram of voice recognition provided in the third embodiment of the present application, as shown in fig. 4A, a user sends voice to an intelligent sound box, the intelligent sound box acquires initial voice data, the initial voice data is input to a system service through a first data interface in the system service, the initial voice data is recognized by using a voice recognition model in the system service to obtain feedback data, the feedback data is returned to an application program through a second data interface, the voice of the user can be responded according to the feedback data, as shown in fig. 4A, if the voice of the user is "play", a resource to be played can be obtained by interacting with a cloud according to the feedback data, and the resource is played for the user.

According to the data processing method provided by the embodiment of the application, data interaction is realized through the external interface between the application program and the system service, voice recognition is realized through the system service, an application developer does not need to deeply know a voice recognition model, the difficulty of application development is reduced, and the efficiency of application development is improved.

Example four,

With reference to the system architecture shown in fig. 1b in the first embodiment, based on the data processing method shown in fig. 2 in the first embodiment, a fourth embodiment of the present application provides a data processing method based on the data processing method shown in fig. 2, where the smart speaker includes a display screen in the present embodiment, and the data processing method provided in the fourth embodiment of the present application specifically uses processing of image data as an example to further explain the data processing method shown in fig. 2, which, of course, is only an exemplary description, and does not represent that the present application is limited thereto. Referring to fig. 5, fig. 5 is a flowchart of a data processing method according to a fourth embodiment of the present application, where the method includes the following steps:

step 501, the intelligent sound box acquires the body movement of the user to obtain initial image data.

The smart sound box may include a camera and a display screen, and may acquire the user limb movement through the camera to obtain initial image data, where the initial image data is to-be-processed data of the application program described in the first embodiment.

Step 502, inputting initial image data into a system service through a first data interface in the system service.

The first data interface may be an interface for inputting data from an application into a system service.

Step 503, processing the initial image data through an image classification model in the system service to obtain feedback data.

The image classification model belongs to a data processing model, and the processing of the initial image data by the image classification model can comprise the processes of preprocessing, image classification model reasoning and postprocessing. Specifically, the preprocessing comprises image size normalization processing, image rotation correction processing, image binarization processing and the like on initial image data, the image data to be processed is obtained after the preprocessing is carried out on the initial image data, image classification model reasoning is carried out on the image data to be processed, a reasoning engine is called to analyze an image classification model, at least one operation task for the voice data to be processed is determined and is distributed to each processor, the processor carries out operation according to the operation tasks, then a final operation result is obtained according to the data operated by each processor, then the operation result is subjected to postprocessing to obtain feedback data, and the feedback data comprises the result obtained after the collected user voice is identified. For example, in this embodiment, the image classification model may classify the image into "return, forward, confirm, cancel" 4 classes, and of course, the image classification model may also classify the image into other number of classes, such as classifying the image into 2 classes, 3 classes, 5 classes, 6 classes, and the like, which is not limited in this application. The operation results can include four results of 00, 01, 10 and 11, which respectively correspond to the four types of images, and the operation results are mapped to obtain feedback data, so that the classification of the limb actions of the user is realized.

Step 504, feedback data is returned to the application program through a second data interface in the system service.

The application program may respond to the user limb movement according to the feedback data, for example, the recognition result included in the feedback data is returned, that is, the last display interface viewed by the user may be returned; for another example, if the recognition result included in the feedback data is a confirmation, the option currently selected by the user in the display interface is confirmed, for example, if the option selected by the user is a movie, the point should be played, and if the option selected by the user is a setting option, the setting interface is displayed. Of course, this is merely an example and does not represent a limitation of the present application.

According to the data processing method, data interaction is achieved through the external interface between the application program and the system service, image classification is achieved through the system service, an application developer does not need to deeply know an image classification model, the difficulty of application development is reduced, and the efficiency of application development is improved.

Example V,

With reference to the system architecture shown in fig. 1b in the first embodiment, based on the data processing methods described in the first to fourth embodiments, a fifth embodiment of the present application provides a data processing apparatus for executing the data processing methods described in the first to fourth embodiments, as shown in fig. 6, where fig. 6 is a block diagram of the data processing apparatus provided in the fifth embodiment of the present application, and the data processing apparatus 60 includes: a transmission module 601 and a system service module 602;

the transmission module 601 is configured to obtain data to be processed and parameter information corresponding to the data to be processed through an external interface of a system service, where an artificial intelligence processing instruction carries the data to be processed and the parameter information, and the system service includes at least one data processing model;

the system service module 602 is configured to determine a data processing model matched with the parameter information in the system service, and call the data processing model to process data to be processed to obtain feedback data;

the transmission module 601 is further configured to return feedback data to the application program through the external interface.

Optionally, in an implementation, the system service further includes an inference engine, as shown in fig. 7, and the system service module 602 includes: a matching unit 6021, a pre-processing unit 6022, a model reasoning unit 6023, and a post-processing unit 6024;

a matching unit 6021 for determining a data processing model matched with the parameter information in the system service;

the preprocessing unit 6022 is configured to preprocess the data to be processed by using the determined data processing model to obtain data to be operated;

the model reasoning unit 6023 is used for calling a reasoning engine to perform data processing model reasoning on data to be calculated to obtain a calculation result;

and the post-processing unit 6024 is configured to perform post-processing on the operation result to obtain feedback data.

Optionally, in an implementation manner, the model inference unit 6032 is specifically configured to invoke an inference engine to analyze the data processing model, and send a calculation instruction to the at least one processor according to an analysis result to calculate the data to be calculated; and obtaining an operation result according to the data operated by the at least one processor.

Optionally, in an implementation manner, the external interface includes a first data interface, and the transmission module 601 is specifically configured to receive, through the first data interface, to-be-processed data sent by the application program and parameter information corresponding to the to-be-processed data.

Optionally, in an implementation manner, the external interface includes a second data interface, and the transmission module 601 is specifically configured to return feedback data to the application program through the second data interface.

Optionally, in an implementation manner, the system service module 602 is configured to determine, in the scene mode, a scene corresponding to the scene parameter in the system service; determining a data processing model corresponding to the model parameters in at least one data processing model corresponding to the scene; and calling a data processing model to process the data to be processed to obtain feedback data.

Optionally, in an implementation manner, as shown in fig. 8, the data processing apparatus 60 further includes a switching module 603;

the switching module 603 is configured to receive a scene mode switching operation, and switch the system service to the scene mode according to the scene mode switching operation.

Optionally, in an implementation manner, the system service module 602 is further configured to receive a scene establishment operation, and generate new scene information according to the scene establishment operation, where the new scene information includes a data processing model corresponding to a new scene; and establishing a new scene according to the new scene information, and updating the system service.

Optionally, in an implementation, the scene parameters correspond to at least two associated scenes;

the system service module 602 is configured to invoke a combination model corresponding to at least two associated scenes to process data to be processed, so as to obtain feedback data, where the combination model is obtained by combining data processing models corresponding to the at least two associated scenes, respectively.

Optionally, in one implementation, as shown in fig. 9, the data processing device 60 further includes a consumption module 604;

a consumption module 604, configured to determine consumption information corresponding to the scene parameter; and returning consumption information to the application program through the external interface.

Optionally, in one implementation, the system service includes an artificial intelligence scenario technology stack and the data processing model includes an artificial intelligence model.

According to the data processing equipment provided by the embodiment of the application, the data to be processed and the parameter information corresponding to the data to be processed, which are sent by an application program, are received through an external interface of a system service in an operating system; determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data; and returning feedback data to the application program through the external interface. The user can package various data processing models into system service and perform data interaction externally through the external interface, so that the application program can realize data processing by using various data processing models, the data processing model can be called in the application to perform data processing without deeply learning the data processing model, the development difficulty of the related application of the data processing model is reduced, and the efficiency of developing the application is improved.

Example six,

With reference to the system architecture shown in fig. 1b in the first embodiment, based on the data processing methods described in the first to fourth embodiments, a sixth embodiment of the present application provides an electronic device for executing the data processing methods described in the first to fourth embodiments, as shown in fig. 10, where fig. 10 is a block diagram of the electronic device provided in the sixth embodiment of the present application, and the electronic device 100 includes:

as shown in fig. 10, the electronic device may include: a processor (processor)1002, a Communications Interface 1004, a memory 1006, and a Communications bus 1008.

Wherein:

the processor 1002, communication interface 1004, and memory 1006 communicate with each other via a communication bus 1008.

A communication interface 1004 for communicating with other electronic devices such as a terminal device or a server.

The processor 1002 is configured to execute the program 1010, and may specifically perform relevant steps in the data processing method embodiment described above.

In particular, the program 1010 may include program code that includes computer operating instructions.

The processor 1002 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement an embodiment of the present invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 1006 is used for storing the program 1010. The memory 1006 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 1010 may be specifically configured to enable the processor 1002 to execute any one of the data processing methods of the first to fourth embodiments.

For specific implementation of each step in the program 1010, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing data processing method embodiments, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.

Example seven,

With reference to the system architecture shown in fig. 1b in the first embodiment, based on the data processing methods described in the first to fourth embodiments, a seventh embodiment of the present application provides a computer storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the data processing methods described in the first to fourth embodiments.

In the embodiment of the application, a user can utilize various data processing models to realize data processing through system service, and the data processing models can be called in the application to perform data processing without deeply learning the data processing models, so that the development difficulty of the related applications of the data processing models is reduced, and the efficiency of developing the application is improved.

It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present invention may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present invention.

The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code initially stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the data processing methods described herein. Further, when a general-purpose computer accesses code for implementing the data processing method shown herein, execution of the code converts the general-purpose computer into a special-purpose computer for executing the data processing method shown herein.

Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

The above embodiments are only for illustrating the embodiments of the present invention and not for limiting the embodiments of the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention, so that all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.

Claims

1. A method of data processing, comprising:

receiving data to be processed sent by an application program and parameter information corresponding to the data to be processed through an external interface of system service in an operating system;

determining a data processing model matched with the parameter information in the system service, and calling the data processing model to process the data to be processed to obtain feedback data;

and returning the feedback data to the application program through the external interface.

2. The method of claim 1, wherein the system service includes an inference engine, and the determining a data processing model in the system service that matches the parameter information and invoking the data processing model to process the data to be processed to obtain feedback data includes:

determining a data processing model matched with the parameter information in the system service;

preprocessing the data to be processed by utilizing the data processing model to obtain data to be operated;

calling the inference engine to perform data processing model inference on the data to be operated to obtain an operation result;

and carrying out post-processing on the operation result to obtain the feedback data.

3. The method of claim 2, wherein the invoking the inference engine to perform data processing model inference on the data to be operated to obtain an operation result comprises:

calling the inference engine to analyze the data processing model, and sending a calculation instruction to at least one processor according to an analysis result so as to calculate the data to be operated;

and obtaining the operation result according to the data operated by the at least one processor.

4. The method of claim 1, wherein the external interface comprises a first data interface, and the receiving, through the external interface of the system service in the operating system, the to-be-processed data sent by the application program and the parameter information corresponding to the to-be-processed data comprises:

and receiving the data to be processed sent by the application program and the parameter information corresponding to the data to be processed through the first data interface.

5. The method of claim 1, wherein the external interface comprises a second data interface, the returning the feedback data to the application program through the external interface comprising:

and returning the feedback data to the application program through the second data interface.

6. The method of claim 1, wherein the parameter information includes scene parameters and model parameters, and determining a data processing model in the system service that matches the parameter information and invoking the data processing model to process the data to be processed to obtain feedback data includes:

under a scene mode, determining a scene corresponding to the scene parameter in the system service; determining a data processing model corresponding to the model parameter from at least one data processing model corresponding to the scene;

and calling the data processing model to process the data to be processed to obtain the feedback data.

7. The method of claim 6, wherein the method further comprises:

receiving a scene mode switching operation, and switching the system service to the scene mode according to the scene mode switching operation.

8. The method of claim 6, wherein the method further comprises:

receiving operation of establishing a scene, and generating new scene information according to the operation of establishing the scene, wherein the new scene information comprises a data processing model corresponding to the new scene;

and establishing the new scene according to the new scene information, and updating the system service.

9. The method of claim 6, wherein the scenario parameters correspond to at least two associated scenarios, and the invoking the data processing model to process the data to be processed to obtain the feedback data comprises:

and calling the combined models corresponding to the at least two associated scenes to process the data to be processed to obtain the feedback data, wherein the combined models are obtained by combining the data processing models respectively corresponding to the at least two associated scenes.

10. The method of claim 6, wherein the method further comprises:

determining consumption information corresponding to the scene parameters; and returning the consumption information to the application program through the external interface.

11. The method of any of claims 1-10, wherein the system service comprises an artificial intelligence scenario technology stack and the data processing model comprises an artificial intelligence model.

12. A data processing apparatus comprising: a transmission module and a system service module;

the transmission module is used for receiving data to be processed sent by an application program and parameter information corresponding to the data to be processed through an external interface of a system service in an operating system;

the system service module is used for determining a data processing model matched with the parameter information in the system service and calling the data processing model to process the data to be processed to obtain feedback data;

the transmission module is further configured to return the feedback data to the application program through the external interface.

13. The device of claim 12, wherein the system service comprises an inference engine, the system service module comprising: the system comprises a matching unit, a pre-processing unit, a model reasoning unit and a post-processing unit;

the matching unit is used for determining a data processing model matched with the parameter information in the system service;

the preprocessing unit is used for preprocessing the data to be processed by utilizing the data processing model to obtain data to be operated;

the model reasoning unit is used for calling the reasoning engine to carry out data processing model reasoning on the data to be calculated to obtain a calculation result;

and the post-processing unit is used for post-processing the operation result to obtain the feedback data.

14. The device according to claim 13, wherein the model inference unit is specifically configured to invoke the inference engine to analyze the data processing model, and send a calculation instruction to at least one processor according to an analysis result to calculate the data to be calculated; and obtaining the operation result according to the data operated by the at least one processor.

15. The device according to claim 12, wherein the external interface includes a first data interface, and the transmission module is specifically configured to receive the to-be-processed data and the parameter information corresponding to the to-be-processed data, which are sent by the application program, through the first data interface.

16. The device according to claim 12, wherein the external interface comprises a second data interface, and the transmission module is specifically configured to return the feedback data to the application program through the second data interface.

17. The apparatus of claim 12, wherein,

the system service module is used for determining a scene corresponding to the scene parameter in the system service in a scene mode; determining a data processing model corresponding to the model parameter from at least one data processing model corresponding to the scene; and calling the data processing model to process the data to be processed to obtain the feedback data.

18. The apparatus of claim 17, wherein the data processing apparatus further comprises a switching module;

the switching module is used for receiving scene mode switching operation and switching the system service into the scene mode according to the scene mode switching operation.

19. The apparatus of claim 17, wherein,

the system service module is also used for receiving the operation of establishing a scene and generating new scene information according to the operation of establishing the scene, wherein the new scene information comprises a data processing model corresponding to the new scene; and establishing the new scene according to the new scene information, and updating the system service.

20. The device of claim 17, wherein the scene parameters correspond to at least two associated scenes;

the system service module is configured to invoke a combination model corresponding to the at least two associated scenes to process the data to be processed, so as to obtain the feedback data, where the combination model is obtained by combining data processing models corresponding to the at least two associated scenes, respectively.

21. The device of claim 17, wherein the data processing device further comprises a consumption module;

the consumption module is used for determining consumption information corresponding to the scene parameters; and returning the consumption information to the application program through the external interface.

22. The apparatus of any of claims 12-21, wherein the system service comprises an artificial intelligence scenario technology stack and the data processing model comprises an artificial intelligence model.

23. An electronic device, the electronic device comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the data processing method according to any one of claims 1-11.

24. A computer storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 11.