WO2017193769A1 - 机器学习模型之间进行转换的方法与设备 - Google Patents

机器学习模型之间进行转换的方法与设备 Download PDF

Info

Publication number
WO2017193769A1
WO2017193769A1 PCT/CN2017/080844 CN2017080844W WO2017193769A1 WO 2017193769 A1 WO2017193769 A1 WO 2017193769A1 CN 2017080844 W CN2017080844 W CN 2017080844W WO 2017193769 A1 WO2017193769 A1 WO 2017193769A1
Authority
WO
WIPO (PCT)
Prior art keywords
machine learning
learning model
parameter information
language
distributed
Prior art date
Application number
PCT/CN2017/080844
Other languages
English (en)
French (fr)
Inventor
梅鹍
安春霖
程大伟
刘汪根
Original Assignee
星环信息科技（上海）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 星环信息科技（上海）有限公司 filed Critical 星环信息科技（上海）有限公司
Publication of WO2017193769A1 publication Critical patent/WO2017193769A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present application relates to the field of computers, and in particular, to a technology for converting between a stand-alone machine learning model and a distributed machine learning model.
  • the R language is mainly used for statistical analysis, drawing and data mining, and provides many third-party packages for data analysis and data mining.
  • the Python language was developed by scientists in the field of computing and can be applied to the fields of statistics and data mining. It also provides a number of libraries for machine learning.
  • Spark which provides an abstract distributed dataset RDD that makes the programming model more flexible and efficient, can make full use of memory to improve performance
  • SparkR adds support for the R language API and runtime to Spark. It mainly consists of two parts: the SparkR package and the JVM backend.
  • the SparkR package is an extension package for R. After installation, the RDD and DataFrame APIs are provided in the runtime environment of R.
  • the JVM backend is a component of Spark Core that provides a bridge between the R interpreter and the JVM virtual machine, allowing R code to create instances of Java classes, call instance methods of Java objects, or static methods of Java classes.
  • Pyspark is a support for adding python language APIs to Spark. It is mainly used for communication between the local python language and java SparkContext objects via Py4J.
  • SparkR has two main drawbacks.
  • the implementation of the SparkR RDD API is degraded compared to the native Scala RDD API, mainly because when the SparkR RDD API is called, the R worker process is started, and the partition data is passed to the R worker. Return the result.
  • serialization and deserialization of partitioned data also increases time overhead, resulting in performance degradation.
  • SparkR's support for R data types and R native machine learning models is not perfect. This results in the incompatibility between the R native stand-alone machine learning model and the Spark native machine learning model, which greatly affects the usability of SparkR.
  • Pyspark also has two flaws.
  • a method for converting a distributed machine learning model into a stand-alone machine learning model comprises:
  • model training parameter information includes distributed training data
  • model training parameter information is in the format of the description file, parsing the model training parameter information into a description file; otherwise, performing data type conversion on the model training parameter information;
  • the distributed machine learning model is converted to a stand-alone machine learning model.
  • parameter information of the single machine learning model is a format of the description file, parsing the parameter information of the single machine learning model parameter information, otherwise, the single machine learning model Parameter information for data type conversion;
  • the distributed training data is predicted according to the distributed machine learning model.
  • a first device for converting a distributed machine learning model into a stand-alone machine learning model wherein the device comprises:
  • a first device configured to acquire model training parameter information, where the model training parameter information Including distributed training data;
  • model training parameter information is a format of a description file, performing profile file analysis on the model training parameter information; otherwise, performing data type conversion on the model training parameter information;
  • a third device configured to train a distributed machine learning model according to the model training parameter information
  • a fourth device for converting the distributed machine learning model into a stand-alone machine learning model.
  • a second device for converting a stand-alone machine learning model into a distributed machine learning model wherein the device comprises:
  • a sixth device for acquiring a stand-alone machine learning model
  • a seventh device configured to extract the parameter information of the single machine learning model, if the parameter information of the single machine learning model is a format of the description file, parsing the parameter information of the single machine learning model parameter, otherwise,
  • the single machine learning model parameter information performs data type conversion
  • An eighth device configured to construct a distributed machine learning model according to the stand-alone machine learning model
  • a ninth device configured to predict the distributed training data according to the distributed machine learning model.
  • a third device for converting between a stand-alone machine learning model and a distributed machine learning model, wherein the device comprises the first device as described above, and as described above Second device.
  • the present application acquires model training parameter information, performs data type conversion or description file parsing, and then trains a distributed machine learning model according to the model training parameter information, and then the distributed machine.
  • the learning model is converted into a stand-alone machine learning model; the application also acquires a stand-alone machine learning model, extracts the parameter information of the single machine learning model, performs data type conversion or description file analysis on the parameter information of the single machine learning model, and then according to the
  • the single machine learning model constructs a distributed machine learning model; this application can realize the mutual conversion between the stand-alone machine learning model and the distributed machine learning model, greatly increasing the compatibility between the two.
  • the present application can implement mutual conversion between a stand-alone machine learning model of a stand-alone language such as R or Python and a distributed machine learning model based on languages such as Java and Scala.
  • the application does not need to start a local process and a distributed process for type conversion during conversion, which reduces time overhead and improves performance.
  • FIG. 1 shows a flow chart of a method for converting a distributed machine learning model to a stand-alone machine learning model in accordance with an aspect of the present application
  • FIG. 2 illustrates a flow chart of a method for converting a stand-alone machine learning model to a distributed machine learning model in accordance with another aspect of the present application
  • FIG. 3 illustrates a first apparatus diagram for converting a distributed machine learning model to a stand-alone machine learning model in accordance with still another aspect of the present application
  • FIG. 4 illustrates a second device schematic for converting a stand-alone machine learning model to a distributed machine learning model in accordance with still another aspect of the present application.
  • the terminal, the device of the service network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory.
  • RAM random access memory
  • ROM read only memory
  • Memory is an example of a computer readable medium.
  • Computer readable media includes both permanent and non-persistent, removable and non-removable media.
  • Information storage can be implemented by any method or technology.
  • the information can be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, read only A compact disk read only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic tape cartridge, magnetic tape storage or other magnetic storage device or any other non-transportable medium that can be used for storage to be accessed by a computing device Information.
  • computer readable media does not include non-transitory computer readable media, such as modulated data signals and carrier waves.
  • FIG. 1 shows a flow chart of a method for converting a distributed machine learning model to a stand-alone machine learning model in accordance with an aspect of the present application.
  • the method includes step S11, step S12, step S13, and step S14.
  • step S11 the first device 1 acquires model training parameter information, wherein the model training parameter information includes distributed training data; in step S12, the first device 1 if the model training parameter information is a description Formatting the file, performing profile file analysis on the model training parameter information; otherwise, performing data type conversion on the model training parameter information; in step S13, the first device 1 training according to the model training parameter information
  • the distributed machine learning model; in step S14, the first device 1 converts the distributed machine learning model into a stand-alone machine learning model.
  • the first device 1 includes, but is not limited to, a user equipment, a network device, or a device formed by integrating a user equipment and a network device through a network.
  • the user equipment includes, but is not limited to, any mobile electronic product that can interact with a user through a touchpad, such as a smart phone, a tablet computer, a notebook computer, etc., and the mobile electronic product can adopt any operating system, such as Android operating system, iOS operating system, etc.
  • the network device includes an electronic device capable of automatically performing numerical calculation and information processing according to an instruction set or stored in advance, and the hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), and a programmable gate.
  • ASIC application specific integrated circuit
  • the network device includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a plurality of servers; wherein the cloud is composed of a large number of computers or network servers based on Cloud Computing Among them, cloud computing is a kind of distributed computing, a virtual supercomputer composed of a group of loosely coupled computers.
  • the network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network), and the like.
  • the first device 1 may also be a script program running on the user equipment, the network device, or the user equipment and the network device, the network device, the touch terminal, or the network device and the touch terminal integrated through the network. . of course, Those skilled in the art should understand that the foregoing first device 1 is only an example, and other existing devices that may be present or may appear in the future, as applicable to the present application, are also included in the scope of protection of the present application, and The reference is included here.
  • step S11 the first device 1 acquires model training parameter information, wherein the model training parameter information includes distributed training data.
  • the distributed training data is used to train a distributed machine learning model.
  • stand-alone training data is used to train a stand-alone machine learning model.
  • the single machine learning model is used for prediction for a small amount of locally predicted data
  • the distributed machine learning model is used for prediction for a large amount of predicted data for distributed storage.
  • the first device 1 acquires model training parameter information based on the first language.
  • the first language includes but is not limited to the R language, the Python language, and the like.
  • R is the language and operating environment for statistical analysis and drawing.
  • Python is an object-oriented, interpreted computer programming language.
  • the model training parameter information may be a format of a description file, such as a data description file in a JSON format, a model description file in an XML format or a PMML format; and the model training parameter information may also be in the form of a function parameter.
  • JSON is a lightweight data exchange language that defines a lightweight data exchange format.
  • XML is an extensible markup language similar to HTML that is used to mark electronic files to have a structured markup language that can be used to mark data and define data types.
  • PMML is a de facto standard language for presenting data mining models.
  • the model training parameter information is in the form of a function parameter
  • the model training parameter information based on the R language or the Python language is acquired, the distributed training data is introduced as a parameter of the distributed machine learning model training function.
  • the first device 1 acquires model training parameter information sent by the user equipment.
  • a user can write an R language script in an R language development environment on a user device (eg, a laptop, etc.) or write a python language script in a Python language development environment.
  • the R language script and the Python language script include the model training parameter information. Then, the user may send the R language script or the described by the user equipment to the first device 1 Python language script.
  • step S12 if the model training parameter information is the format of the description file, the first device 1 performs profile description on the model training parameter information, otherwise, performs data type conversion on the model training parameter information.
  • the model training parameter information is a format of a description file
  • the model training parameter information based on the first language is parsed into the model training parameter information based on the second language; if the model training parameter information is a function
  • the form of the parameter converts the model training parameter information based on the first language into the model training parameter information based on the second language.
  • this is done by data type conversion or profile analysis.
  • the first device 1 performs data type conversion on the model training parameter information by using a language conversion tool of the first language and the second language to obtain the model based on the second language. Training parameter information; or performing profile file analysis on the model training parameter information by using a profile analysis tool to obtain the model training parameter information based on the second language.
  • the second language includes but is not limited to a Java language, a Scala language, etc.
  • the language conversion tool includes, but is not limited to, rJava
  • the description file parsing tools include, but are not limited to, xml file parsing, json data format parsing, and PMML. File analysis, etc.
  • Java is an object-oriented programming language that can write cross-platform applications
  • Scala is a multi-paradigm programming language, a Java-like programming language
  • rJava is a communication interface between R language and Java language. Calls are implemented through the underlying JNI (Java Native Interface), allowing Java objects and methods to be called directly in R.
  • rJava also provides the function of Java call R, which is implemented by JRI (Java/R Interface). Because rJava is the underlying interface and uses JNI as an interface call, it is very efficient.
  • model training parameter information is in the form of a function parameter
  • rJava can be used to perform data type conversion on the model training parameter information, and the model training parameter information is converted into a parameter recognizable by Java and Scala language.
  • the training parameter information of the model is the format of the description file
  • the model data description file may be parsed by using an xml file parsing tool, a json file parsing tool or a PMML file parsing tool, and parsed into parameter information identifiable by Java and Scala language. .
  • first language second language
  • language conversion tool and file parsing tool
  • the tool and file parsing tool is only an example, and other existing or future first language, second language, language conversion tool and file parsing tool may be applicable to the present application, and should also be included in the scope of the present application. It is hereby incorporated by reference.
  • step S13 the first device 1 trains the distributed machine learning model based on the model training parameter information.
  • the Java- and Scala-recognizable parameters of the data type conversion can be passed to the distributed function interface defined by the underlying machine learning library through rJava, and the distribution is trained.
  • Machine learning model Mainly using J functions of rJava, the usage is J ("class name”, “function method”, parameter one, parameter two, 7), where a "function method" corresponds to a training distributed machine Learning the training function of the model, the J function plays the role of calling the Java function method.
  • a plurality of distributed machine learning models can be trained correspondingly through training functions of a plurality of distributed machine learning models.
  • the model training parameter information is a format of a description file
  • the data type can be parsed into a parameter information that can be recognized by Java and Scala language through an xml file parsing, a JSON file parsing tool, or a PMML file parsing tool, and transmitted to the underlying machine.
  • the learning library defines a distributed function interface to train a distributed machine learning model.
  • step S14 the first device 1 converts the distributed machine learning model into a stand-alone machine learning model.
  • the distributed machine learning model is mainly used for predicting a large amount of predicted data stored in the cloud
  • the single machine learning model is mainly used for predicting a small amount of local predicted data on the user equipment side.
  • the first device 1 converts the distributed machine learning model into a stand-alone machine learning model based on the first language.
  • the distributed machine learning model described in the second language based on Java or Scala, etc. is converted into the stand-alone machine learning model described in the first language based on R or Python.
  • the first device 1 converts the distributed machine learning model into a stand-alone machine learning model based on the first language using the language conversion tool or the profile analysis tool.
  • the model training parameter information is in the form of a function parameter, it can be based on rJava
  • the distributed machine learning model described by a language such as Java or Scala is converted into the stand-alone machine learning model based on the R language description.
  • the model training parameter information is a format of the description file
  • the distributed machine learning model may be converted into R or Python based on the description file parsing tool such as an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • a stand-alone machine learning model of the first language is converted into rJava
  • the distributed machine learning model described by a language such as Java or Scala is converted into the stand-alone machine learning model based on the R language description.
  • the distributed machine learning model may be converted into R or Python based on the description file parsing tool such as an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • the first device 1 extracts the distributed machine learning model parameter information; if the distributed machine learning model parameter information is a format of a description file, using the description file parsing tool pair The distributed machine learning model parameter information is used to perform profile analysis, otherwise, the distributed machine learning model parameter information is converted by the language conversion tool to obtain a machine learning model parameter based on the first language. Information; constructing a stand-alone machine learning model based on the first language based on the machine learning model parameter information.
  • the distributed machine learning model parameter information may be extracted, and then data type conversion is performed by rJava, and the distribution is described based on a language such as Java or Scala.
  • the machine learning model parameter information is transformed into model parameter information based on language descriptions such as R or Python, and then a single machine learning model based on R or Python language description is constructed according to model parameter information based on language descriptions such as R or Python.
  • the model description file of the distributed machine learning model may be extracted, and then the model description is performed by using an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • the file is parsed, and the model described by xml or json is converted into model parameter information based on language descriptions such as R or python, and then the single machine learning based on the first language description such as R or python is constructed according to the model parameter information. model.
  • the method further comprises: the first device 1 returns the stand-alone machine learning model to the user equipment.
  • the stand-alone machine learning model is returned to the user equipment for predicting a small amount of prediction data locally at the user equipment end.
  • a local small amount of prediction data of the user equipment end may be predicted by calling a predict function that is included in the single machine learning model.
  • step S26 the second device 2 acquires a stand-alone machine learning model; in step S27, the second device 2 extracts the single machine learning model parameter information, if the single machine learning model parameter information is a description file a format, the description file is parsed by the single machine learning model parameter information; otherwise, the single machine learning model parameter information is data type converted; in step S28, the second device 2 is based on the single machine learning model
  • the distributed machine learning model is constructed; in step S29, the second device 2 predicts the distributed training data according to the distributed machine learning model.
  • the second device 2 includes, but is not limited to, a user equipment, a network device, or a device formed by integrating a user equipment and a network device through a network.
  • the user equipment includes, but is not limited to, any mobile electronic product that can interact with a user through a touchpad, such as a smart phone, a tablet computer, a notebook computer, etc., and the mobile electronic product can adopt any operating system, such as Android operating system, iOS operating system, etc.
  • the network device includes an electronic device capable of automatically performing numerical calculation and information processing according to an instruction set or stored in advance, and the hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), and a programmable gate.
  • ASIC application specific integrated circuit
  • the network device includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a plurality of servers; wherein the cloud is composed of a large number of computers or network servers based on Cloud Computing Among them, cloud computing is a kind of distributed computing, a virtual supercomputer composed of a group of loosely coupled computers.
  • the network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network), and the like.
  • the second device 2 may also be a script program running on the user equipment, the network device, or the user equipment and the network device, the network device, the touch terminal, or the network device and the touch terminal integrated through the network. .
  • the foregoing second device 2 is only an example, and other existing or future possible second devices 2, as applicable to the present application, are also included in the scope of protection of the present application, and This is hereby incorporated by reference.
  • step S26 the second device 2 acquires a stand-alone machine learning model.
  • the stand-alone machine learning model is trained according to local stand-alone training data of the user equipment end, where the single machine learning model sent by the user equipment is obtained.
  • distributed The machine learning model is trained based on distributed training data.
  • the single machine learning model is used for prediction for a small amount of locally predicted data
  • the distributed machine learning model is used for prediction for a large amount of predicted data for distributed storage.
  • the second device 2 acquires a stand-alone machine learning model based on the first language.
  • the first language includes but is not limited to the R language, the Python language, and the like.
  • R is the language and operating environment for statistical analysis and drawing.
  • Python is an object-oriented, interpreted computer programming language.
  • the first language based machine learning model may be a stand-alone machine learning model based on a language such as R or Python.
  • step S27 the second device 2 extracts the single machine learning model parameter information, and if the single machine learning model parameter information is a format of the description file, parsing the description file of the single machine learning model parameter information, Otherwise, data conversion is performed on the single machine learning model parameter information.
  • the stand-alone machine learning model parameter information is first extracted.
  • the parameter information of the single machine learning model may be a format of a description file, such as a data description file in a JSON format, a model description file in an XML format or a PMML format, and the parameter information of the single machine learning model may also be a form of a function parameter. . If the single machine learning model parameter information is in the format of the description file, the description file is parsed; if the single machine learning model parameter information is in the form of a function parameter, the data type conversion is performed.
  • JSON is a lightweight data exchange language that defines a lightweight data exchange format.
  • XML is an extensible markup language similar to HTML that is used to mark electronic files to have a structured markup language that can be used to mark data and define data types.
  • PMML is a de facto standard language for presenting data mining models.
  • the second device 2 performs data type conversion on the single machine learning model parameter information by using a language conversion tool of the first language and the second language to obtain a machine based on the second language. Learning the model parameter information; or using the description file parsing tool to perform parsing of the single machine learning model parameter information to obtain the second language based on the second language Machine learning model parameter information.
  • the second language includes, but is not limited to, a Java language, a Scala language, etc.
  • the language conversion tools include, but are not limited to, rJava.
  • Java is an object-oriented programming language that can write cross-platform applications
  • Scala is a multi-paradigm programming language, a Java-like programming language
  • rJava is a communication interface between R language and Java language. Calls are implemented through the underlying JNI (Java Native Interface), allowing Java objects and methods to be called directly in R.
  • rJava also provides the function of Java call R, which is implemented by JRI (Java/R Interface). Because rJava is the underlying interface and uses JNI as an interface call, it is very efficient.
  • the parameter information of the single machine learning model is in the form of a function parameter
  • the parameter information of the single machine learning model may be extracted, and then the data type conversion of the model parameter information by rJava, and the model parameter information based on the R language description Convert to model parameter information described in languages such as Java or Scala.
  • the single machine learning model parameter information is a format of the description file
  • the description file of the single machine learning model may be extracted, and then the model is described by an xml file parsing tool, a JSON file parsing tool, or a PMML file parsing tool.
  • the file is converted into model parameter information based on a language such as Java or Scala.
  • first language, second language, language conversion tool and file analysis tool are only examples, other existing or future first language, second language, language conversion tools and The document parsing tool, as applicable to the present application, is also intended to be included within the scope of the present application and is hereby incorporated by reference.
  • step S28 the second device 2 constructs a distributed machine learning model based on the stand-alone machine learning model.
  • a distributed machine learning model can be obtained by calling the constructor of the distributed machine model through rJava.
  • the main use of rJava's .jnew function, the usage is .jnew ("model class name", parameter one, parameter two, ...), where a "model class name" corresponds to a distributed machine learning model
  • the Java class name, the .jnew function acts as a Java object, which is a distributed machine learning model.
  • a variety of distributed machine learning models can be constructed correspondingly through the constructors of various distributed machine learning models.
  • the second device 2 constructs the parameter information according to the machine learning model.
  • a distributed machine learning model based on the second language is created.
  • a distributed machine learning model based on a language description such as Java or Scala may be constructed according to the machine learning model parameter information described in a language such as Java or Scala.
  • machine learning model xml description file, the JSON description file, the PMML description file, and the like described in a language such as Java or Scala may be used to construct a distribution based on a language description such as Java or Scala by using a corresponding file parsing tool.
  • Machine learning model may be used to construct a distribution based on a language description such as Java or Scala by using a corresponding file parsing tool.
  • step S29 the second device 2 predicts the distributed training data according to the distributed machine learning model.
  • the distributed machine learning model is used to predict large amounts of predictive data for cloud distributed storage.
  • the distributed training data can be predicted by invoking a prediction function of the distributed machine learning model.
  • the usage is model$predict(data,%), where model refers to the distributed machine learning model, predict refers to the prediction function of the distributed machine learning model, data is Refers to the distributed training data.
  • the method further comprises: the second device 2 acquiring the distributed training data sent by the user equipment.
  • the user equipment may obtain the distributed training data stored in a distributed storage medium or a distributed database through a data extraction function, and then the user equipment transmits the distributed training data to the second device 2.
  • FIG 3 shows a first device 1 for converting a distributed machine learning model into a stand-alone machine learning model according to still another aspect of the present application, wherein the first device 1 comprises a first device 11, a second device 12, The third device 13 and the fourth device 14.
  • the first device 11 acquires model training parameter information, where the model training parameter information includes distributed training data; and if the model training parameter information is the format of the description file, the second device 12 Decoding the model training parameter information, otherwise performing data type conversion on the model training parameter information; the third device 13 training the distributed machine learning model according to the model training parameter information; The four device 14 converts the distributed machine learning model into a stand-alone machine learning model.
  • the first device 1 includes, but is not limited to, a user equipment, a network device, or a device formed by integrating a user equipment and a network device through a network.
  • the user equipment includes but is not limited to A mobile electronic product, such as a smart phone, a tablet computer, a notebook computer, or the like, which can be used for human interaction with a user through a touch panel, and the mobile electronic product can adopt any operating system, such as an Android operating system, an iOS operating system, and the like.
  • the network device includes an electronic device capable of automatically performing numerical calculation and information processing according to an instruction set or stored in advance, and the hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), and a programmable gate.
  • ASIC application specific integrated circuit
  • the network device includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a plurality of servers; wherein the cloud is composed of a large number of computers or network servers based on Cloud Computing Among them, cloud computing is a kind of distributed computing, a virtual supercomputer composed of a group of loosely coupled computers.
  • the network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network), and the like.
  • the first device 1 may also be a script program running on the user equipment, the network device, or the user equipment and the network device, the network device, the touch terminal, or the network device and the touch terminal integrated through the network. .
  • the foregoing first device 1 is only an example, and other existing devices that may be present or may appear in the future, as applicable to the present application, are also included in the scope of protection of the present application, and This is hereby incorporated by reference.
  • the first device 11 acquires model training parameter information, wherein the model training parameter information includes distributed training data.
  • the distributed training data is used to train a distributed machine learning model.
  • stand-alone training data is used to train a stand-alone machine learning model.
  • the single machine learning model is used for prediction for a small amount of locally predicted data
  • the distributed machine learning model is used for prediction for a large amount of predicted data for distributed storage.
  • the first device 11 acquires model training parameter information based on the first language.
  • the first language includes but is not limited to the R language, the Python language, and the like.
  • R is the language and operating environment for statistical analysis and drawing.
  • Python is an object-oriented, interpreted computer programming language.
  • the model training parameter information may be a format of a description file, such as a data description file in a JSON format, a model description file in an XML format or a PMML format; and the model training parameter information may also be in the form of a function parameter.
  • JSON is a lightweight data exchange language that defines a lightweight data exchange. Change the format.
  • XML is an extensible markup language similar to HTML that is used to mark electronic files to have a structured markup language that can be used to mark data and define data types.
  • PMML is a de facto standard language for presenting data mining models.
  • the model training parameter information is in the form of a function parameter
  • the model training parameter information based on the R language or the Python language is acquired, the distributed training data is introduced as a parameter of the distributed machine learning model training function.
  • the first device 11 acquires model training parameter information sent by the user equipment.
  • a user can write an R language script in an R language development environment on a user device (eg, a laptop, etc.) or write a python language script in a Python language development environment.
  • the R language script and the Python language script include the model training parameter information. Then, the user can send the R language script or the Python language script to the first device 1 through the user equipment.
  • the second device 12 performs profile description on the model training parameter information; otherwise, performs data type conversion on the model training parameter information.
  • the model training parameter information is a format of a description file
  • the model training parameter information based on the first language is parsed into the model training parameter information based on the second language; if the model training parameter information is a function
  • the form of the parameter converts the model training parameter information based on the first language into the model training parameter information based on the second language.
  • this is done by data type conversion or profile analysis.
  • the second device 12 performs data type conversion on the model training parameter information by using a language conversion tool of the first language and the second language to obtain the model training parameter information based on the second language. Or; performing profile file analysis on the model training parameter information by using a profile analysis tool to obtain the model training parameter information based on the second language.
  • the second language includes but is not limited to a Java language, a Scala language, etc.
  • the language conversion tool includes, but is not limited to, rJava
  • the description file parsing tools include, but are not limited to, xml file parsing, json data format parsing, and PMML. File analysis, etc.
  • Java is an object-oriented programming language that can write cross-platform applications
  • Scala is a multi-paradigm programming language, a Java-like programming language
  • rJava is a communication interface between R language and Java language. Pass The underlying JNI (Java Native Interface) implementation call allows Java objects and methods to be called directly in R.
  • rJava also provides the function of Java call R, which is implemented by JRI (Java/R Interface). Because rJava is the underlying interface and uses JNI as an interface call, it is very efficient.
  • JNI Java Native Interface
  • model training parameter information is in the form of a function parameter
  • rJava can be used to perform data type conversion on the model training parameter information, and the model training parameter information is converted into a parameter recognizable by Java and Scala language.
  • the training parameter information of the model is the format of the description file
  • the model data description file may be parsed by using an xml file parsing tool, a json file parsing tool or a PMML file parsing tool, and parsed into parameter information identifiable by Java and Scala language. .
  • first language, second language, language conversion tool and file analysis tool are only examples, other existing or future first language, second language, language conversion tools and The document parsing tool, as applicable to the present application, is also intended to be included within the scope of the present application and is hereby incorporated by reference.
  • the third device 13 trains a distributed machine learning model based on the model training parameter information.
  • the Java- and Scala-recognizable parameters of the data type conversion can be passed to the distributed function interface defined by the underlying machine learning library through rJava, and the distribution is trained.
  • Machine learning model Mainly using J functions of rJava, the usage is J ("class name”, “function method”, parameter one, parameter two, 7), where a "function method" corresponds to a training distributed machine Learning the training function of the model, the J function plays the role of calling the Java function method.
  • a plurality of distributed machine learning models can be trained correspondingly through training functions of a plurality of distributed machine learning models.
  • the model training parameter information is a format of a description file
  • the data type can be parsed into a parameter information that can be recognized by Java and Scala language through an xml file parsing, a JSON file parsing tool, or a PMML file parsing tool, and transmitted to the underlying machine.
  • the learning library defines a distributed function interface to train a distributed machine learning model.
  • the fourth device 14 converts the distributed machine learning model into a stand-alone machine learning model.
  • the distributed machine learning model is mainly used for predicting a large amount of predicted data of cloud distributed storage, and the single machine learning model is mainly used for locally pre-localizing a small amount of user equipment.
  • the measured data is used for prediction.
  • the fourth device 14 converts the distributed machine learning model into a stand-alone machine learning model based on the first language.
  • the distributed machine learning model described in the second language based on Java or Scala, etc. is converted into the stand-alone machine learning model described in the first language based on R or Python.
  • the fourth device 14 converts the distributed machine learning model into a stand-alone machine learning model based on the first language using the language conversion tool or the profile analysis tool.
  • the distributed machine learning model described in language such as Java or Scala may be converted into the stand-alone machine learning model based on the R language description by rJava.
  • the model training parameter information is a format of the description file
  • the distributed machine learning model may be converted into R or Python based on the description file parsing tool such as an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • a stand-alone machine learning model of the first language such as an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • the fourth device 14 extracts the distributed machine learning model parameter information; if the distributed machine learning model parameter information is a format of a description file, the distribution is performed by using the description file parsing tool The machine learning model parameter information is used to perform profile analysis, otherwise, the distributed machine learning model parameter information is converted by the language conversion tool to obtain machine learning model parameter information based on the first language; The machine learning model parameter information construct is based on a stand-alone machine learning model of the first language.
  • the distributed machine learning model parameter information may be extracted, and then data type conversion is performed by rJava, and the distribution is described based on a language such as Java or Scala.
  • the machine learning model parameter information is transformed into model parameter information based on language descriptions such as R or Python, and then a single machine learning model based on R or Python language description is constructed according to model parameter information based on language descriptions such as R or Python.
  • the model description file of the distributed machine learning model may be extracted, and then the model description is performed by using an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • the file is parsed, and the model described by xml or json is converted into model parameter information based on language descriptions such as R or python, and then the first language description based on R or python is constructed according to the model parameter information.
  • Machine learning model is a format of the description file
  • the model description file of the distributed machine learning model may be extracted, and then the model description is performed by using an xml file parsing tool, a json file parsing tool, or a PMML file parsing tool.
  • the file is parsed, and the model described by xml or json is converted into model parameter information based on language descriptions such as R or python, and then the first language description based on R or python is constructed
  • the first device 1 further comprises a fifth device (not shown); the fifth device returns the stand-alone machine learning model to the user equipment.
  • the stand-alone machine learning model is returned to the user equipment for predicting a small amount of prediction data locally at the user equipment end.
  • a local small amount of prediction data of the user equipment end may be predicted by calling a predict function that is included in the single machine learning model.
  • FIG 4 shows a second device 2 for converting a stand-alone machine learning model into a distributed machine learning model according to still another aspect of the present application, wherein the second device 2 includes a sixth device 26, a seventh device 27, The eighth device 28 and the ninth device 29.
  • the sixth device 26 acquires a stand-alone machine learning model; the seventh device 27 extracts the single machine learning model parameter information, and if the single machine learning model parameter information is a format of a description file, Descriptive file parsing of the single machine learning model parameter information, otherwise, data type conversion is performed on the single machine learning model parameter information; the eighth device 28 constructs a distributed machine learning model according to the single machine learning model; The ninth device 29 predicts the distributed training data according to the distributed machine learning model.
  • the second device 2 includes, but is not limited to, a user equipment, a network device, or a device formed by integrating a user equipment and a network device through a network.
  • the user equipment includes, but is not limited to, any mobile electronic product that can interact with a user through a touchpad, such as a smart phone, a tablet computer, a notebook computer, etc., and the mobile electronic product can adopt any operating system, such as Android operating system, iOS operating system, etc.
  • the network device includes an electronic device capable of automatically performing numerical calculation and information processing according to an instruction set or stored in advance, and the hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), and a programmable gate.
  • ASIC application specific integrated circuit
  • the network device includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a plurality of servers; wherein the cloud is composed of a large number of computers or network servers based on Cloud Computing Among them, cloud computing is a kind of distributed computing, a virtual supercomputer composed of a group of loosely coupled computers.
  • the network includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless ad hoc network (Ad Hoc network), and the like.
  • the second device 2 can also be shipped A script program on the device formed by the user equipment, the network device, or the user equipment and the network device, the network device, the touch terminal, or the network device and the touch terminal integrated through the network.
  • the foregoing second device 2 is only an example, and other existing or future possible second devices 2, as applicable to the present application, are also included in the scope of protection of the present application, and This is hereby incorporated by reference.
  • the sixth device 26 acquires a stand-alone machine learning model.
  • the stand-alone machine learning model is trained according to local stand-alone training data of the user equipment end, where the single machine learning model sent by the user equipment is obtained. Accordingly, the distributed machine learning model is trained based on distributed training data.
  • the single machine learning model is used for prediction for a small amount of locally predicted data
  • the distributed machine learning model is used for prediction for a large amount of predicted data for distributed storage.
  • the sixth device 26 acquires a stand-alone machine learning model based on the first language.
  • the first language includes but is not limited to the R language, the Python language, and the like.
  • R is the language and operating environment for statistical analysis and drawing.
  • Python is an object-oriented, interpreted computer programming language.
  • the first language based machine learning model may be a stand-alone machine learning model based on a language such as R or Python.
  • the seventh device 27 extracts the parameter information of the single machine learning model. If the parameter information of the single machine learning model is the format of the description file, the description file of the single machine learning model parameter information is parsed; otherwise, The single machine learning model parameter information performs data type conversion.
  • the stand-alone machine learning model parameter information is first extracted.
  • the parameter information of the single machine learning model may be a format of a description file, such as a data description file in a JSON format, a model description file in an XML format or a PMML format, and the parameter information of the single machine learning model may also be a form of a function parameter. . If the single machine learning model parameter information is in the format of the description file, the description file is parsed; if the single machine learning model parameter information is in the form of a function parameter, the data type conversion is performed.
  • JSON is a lightweight data exchange language that defines a lightweight data exchange. Change the format.
  • XML is an extensible markup language similar to HTML that is used to mark electronic files to have a structured markup language that can be used to mark data and define data types.
  • PMML is a de facto standard language for presenting data mining models.
  • the seventh device 27 performs data type conversion on the single machine learning model parameter information by using a language conversion tool of the first language and the second language to obtain a machine learning model parameter based on the second language.
  • Information or, the description file parsing of the single machine learning model parameter information is performed by using a description file parsing tool to obtain machine learning model parameter information based on the second language.
  • the second language includes, but is not limited to, a Java language, a Scala language, etc.
  • the language conversion tools include, but are not limited to, rJava.
  • Java is an object-oriented programming language that can write cross-platform applications
  • Scala is a multi-paradigm programming language, a Java-like programming language
  • rJava is a communication interface between R language and Java language. Calls are implemented through the underlying JNI (Java Native Interface), allowing Java objects and methods to be called directly in R.
  • rJava also provides the function of Java call R, which is implemented by JRI (Java/R Interface). Because rJava is the underlying interface and uses JNI as an interface call, it is very efficient.
  • the parameter information of the single machine learning model is in the form of a function parameter
  • the parameter information of the single machine learning model may be extracted, and then the data type conversion of the model parameter information by rJava, and the model parameter information based on the R language description Convert to model parameter information described in languages such as Java or Scala.
  • the single machine learning model parameter information is a format of the description file
  • the description file of the single machine learning model may be extracted, and then the model is described by an xml file parsing tool, a JSON file parsing tool, or a PMML file parsing tool.
  • the file is converted into model parameter information based on a language such as Java or Scala.
  • first language, second language, language conversion tool and file analysis tool are only examples, other existing or future first language, second language, language conversion tools and The document parsing tool, as applicable to the present application, is also intended to be included within the scope of the present application and is hereby incorporated by reference.
  • the eighth device 28 constructs a distributed machine learning model based on the stand-alone machine learning model.
  • the distributed machine model can be called by rJava to get the distributed machine. Learning model.
  • the Java class name, the .jnew function acts as a Java object, which is a distributed machine learning model.
  • a variety of distributed machine learning models can be constructed correspondingly through the constructors of various distributed machine learning models.
  • the eighth device 28 constructs a distributed machine learning model based on the second language based on the machine learning model parameter information.
  • a distributed machine learning model based on a language description such as Java or Scala may be constructed according to the machine learning model parameter information described in a language such as Java or Scala.
  • machine learning model xml description file, the JSON description file, the PMML description file, and the like described in a language such as Java or Scala may be used to construct a distribution based on a language description such as Java or Scala by using a corresponding file parsing tool.
  • Machine learning model may be used to construct a distribution based on a language description such as Java or Scala by using a corresponding file parsing tool.
  • the ninth device 29 predicts the distributed training data according to the distributed machine learning model.
  • the distributed machine learning model is used to predict large amounts of predictive data for cloud distributed storage.
  • the distributed training data can be predicted by invoking a prediction function of the distributed machine learning model.
  • the usage is model$predict(data,%), where model refers to the distributed machine learning model, predict refers to the prediction function of the distributed machine learning model, data is Refers to the distributed training data.
  • the second device 2 further includes a tenth device (not shown); the tenth device acquires distributed training data sent by the user equipment.
  • the user equipment may obtain the distributed training data stored in a distributed storage medium or a distributed database through a data extraction function, and then the user equipment transmits the distributed training data to the second device 2.
  • the present application also provides a third device for converting between a stand-alone machine learning model and a distributed machine learning model, wherein the third device comprises the first device 1 as described above, and the second device 2 as described above .
  • the third device can convert the distributed machine learning model into a stand-alone machine learning model by the method of one aspect of the present application, or by another party of the present application.
  • the method described converts a stand-alone machine learning model into a distributed machine learning model; that is, the third device can implement bidirectional conversion between a stand-alone machine learning model and a distributed machine learning model.
  • the present application acquires model training parameter information, performs data type conversion or description file parsing, and then trains a distributed machine learning model according to the model training parameter information, and then the distributed machine.
  • the learning model is converted into a stand-alone machine learning model; the application also acquires a stand-alone machine learning model, extracts the parameter information of the single machine learning model, performs data type conversion or description file analysis on the parameter information of the single machine learning model, and then according to the
  • the single machine learning model constructs a distributed machine learning model; this application can realize the mutual conversion between the stand-alone machine learning model and the distributed machine learning model, greatly increasing the compatibility between the two.
  • the present application can implement a mutual conversion between a stand-alone machine learning model of a stand-alone language such as R or Python and a distributed machine learning model based on a language such as Java or Scala, and the present application does not need to be started locally when performing the conversion.
  • Type-converted worker processes and distributed processes reduce time overhead and improve performance.
  • the present application can be implemented in software and/or a combination of software and hardware, for example, using an application specific integrated circuit (ASIC), a general purpose computer, or any other similar hardware device.
  • the software program of the present application can be executed by a processor to implement the steps or functions described above.
  • the software programs (including related data structures) of the present application can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like.
  • some of the steps or functions of the present application may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.
  • a portion of the present application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide a method and/or technical solution in accordance with the present application.
  • the program instructions for invoking the method of the present application may be stored in a fixed or removable recording medium, and/or transmitted by a data stream in a broadcast or other signal bearing medium, and/or stored in a The working memory of the computer device in which the program instructions are run.
  • an embodiment in accordance with the present application includes a device including a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, triggering
  • the apparatus operates based on the aforementioned methods and/or technical solutions in accordance with various embodiments of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Devices For Executing Special Programs (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Feedback Control In General (AREA)

Abstract

一种单机机器学习模型与分布式机器学习模型之间进行转换的方法与设备。与现有技术相比，该方法获取模型训练参数信息，并对其进行数据类型转换或描述文件解析，然后根据所述模型训练参数信息训练出分布式机器学习模型（S13），再将所述分布式机器学习模型转换为单机机器学习模型（S14）；该方法还获取单机机器学习模型（S26），提取出所述单机机器学习模型参数信息，对所述单机机器学习模型参数信息进行数据类型转换或描述文件解析，然后根据所述单机机器学习模型构造分布式机器学习模型（S28）。该方法可实现单机机器学习模型与分布式机器学习模型之间的相互转换，大大增加了两者之间的兼容性。

Description

机器学习模型之间进行转换的方法与设备 技术领域
本申请涉及计算机领域，尤其涉及一种单机机器学习模型与分布式机器学习模型之间进行转换的技术。
背景技术
R语言主要用于统计分析、绘图和数据挖掘，提供了很多第三方包用于数据分析和数据挖掘。Python语言是计算领域科学家开发的，可以运用到统计学和数据挖掘领域，也提供了很多用于机器学习的函数库。然而在大数据时代，面对海量数据处理，由于R语言和python语言的核心运行环境是单线程的，只能局限在处理单机能负载的数据量，无法对大数据量进行处理。而Spark(其提供了一个抽象的分布式数据集RDD，使编程模型更加灵活和高效，能够充分利用内存来提升性能)作为高效的分布式计算平台，可以很好地弥补R和python的可伸缩性问题。在此背景下，SparkR和pyspark应运而生。SparkR是往Spark中增加R语言API和运行时的支持，其主要包括两个部分：SparkR包和JVM后端。SparkR包是一个R的扩展包，安装之后，在R的运行环境里提供了RDD和DataFrame API。JVM后端是Spark Core中的一个组件，提供了R解释器和JVM虚拟机之间的桥接功能，能够让R代码创建Java类的实例、调用Java对象的实例方法或者Java类的静态方法。Pyspark是往Spark中增加python语言API的支持。其主要是通过Py4J，用于本地python语言和java SparkContext objects的通信。
然而，目前SparkR和Pyspark技术都存在缺陷。SparkR主要有两方面的缺陷。一方面是SparkR RDD API的实现相比原生的Scala RDD API来说性能有所下降，主要是因为在调用SparkR RDD API的时候，要启动R worker进程，将分区数据传给R worker，R worker再将结果返回。另外分区数据的序列化和反序列化也会增加时间开销，带来性能下降。另一方面是SparkR对R数据类型以及R原生机器学习模型的支持还不是很完善。 这导致R原生单机机器学习模型和Spark原生机器学习模型之间不能相互通用，这一点大大影响了SparkR的可用性。Pyspark也有两方面的缺陷。一方面是数据和代码传输的时候会启动Py4J和spark work进程，数据和函数的序列化和反序列化会增加时间开销，带来性能下降。另一方面是pyspark不能实现python原生单机机器学习模型和spark原生机器学习模型之间的通用。
发明内容
本申请的一个目的是提供一种单机机器学习模型与分布式机器学习模型之间进行转换的方法与设备。
根据本申请的一个方面，提供了一种用于将分布式机器学习模型转换为单机机器学习模型的方法，其中，该方法包括：
获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据；
若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换；
根据所述模型训练参数信息，训练出分布式机器学习模型；
将所述分布式机器学习模型转换为单机机器学习模型。
根据本申请的另一个方面，还提供了一种用于将单机机器学习模型转换为分布式机器学习模型的方法，其中，该方法包括：
获取单机机器学习模型；
提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换；
根据所述单机机器学习模型构造分布式机器学习模型；
根据所述分布式机器学习模型对分布式训练数据进行预测。
根据本申请的又一个方面，还提供了一种用于将分布式机器学习模型转换为单机机器学习模型的第一设备，其中，该设备包括：
第一装置，用于获取模型训练参数信息，其中，所述模型训练参数信息 包括分布式训练数据；
第二装置，用于若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换；
第三装置，用于根据所述模型训练参数信息，训练出分布式机器学习模型；
第四装置，用于将所述分布式机器学习模型转换为单机机器学习模型。
根据本申请的再一个方面，还提供了一种用于将单机机器学习模型转换为分布式机器学习模型的第二设备，其中，该设备包括：
第六装置，用于获取单机机器学习模型；
第七装置，用于提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换；
第八装置，用于根据所述单机机器学习模型构造分布式机器学习模型；
第九装置，用于根据所述分布式机器学习模型对分布式训练数据进行预测。
根据本申请的另一个方面，还提供了一种单机机器学习模型与分布式机器学习模型之间进行转换的第三设备，其中，该设备包括如上所述的第一设备，以及如上所述的第二设备。
与现有技术相比，本申请获取模型训练参数信息，并对其进行数据类型转换或描述文件解析，然后根据所述模型训练参数信息训练出分布式机器学习模型，再将所述分布式机器学习模型转换为单机机器学习模型；本申请还获取单机机器学习模型，提取出所述单机机器学习模型参数信息，对所述单机机器学习模型参数信息进行数据类型转换或描述文件解析，然后根据所述单机机器学习模型构造分布式机器学习模型；本申请可实现单机机器学习模型与分布式机器学习模型之间的相互转换，大大增加了两者之间的兼容性。进一步地，本申请可实现R或者python等单机语言原生的单机机器学习模型与基于Java、Scala等语言的分布式机器学习模型之间的相互转换，同时 本申请在进行转换时不需要启动本地进行类型转换的work进程和分布式进程，减少了时间的开销，提高了性能。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1示出根据本申请一个方面的一种用于将分布式机器学习模型转换为单机机器学习模型的方法流程图；
图2示出根据本申请另一个方面的一种用于将单机机器学习模型转换为分布式机器学习模型的方法流程图；
图3示出根据本申请又一个方面的一种用于将分布式机器学习模型转换为单机机器学习模型的第一设备示意图；
图4示出根据本申请再一个方面的一种用于将单机机器学习模型转换为分布式机器学习模型的第二设备示意图。
附图中相同或相似的附图标记代表相同或相似的部件。
具体实施方式
下面结合附图对本申请作进一步详细描述。
在本申请一个典型的配置中，终端、服务网络的设备和可信方均包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读 光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括非暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
图1示出根据本申请一个方面的一种用于将分布式机器学习模型转换为单机机器学习模型的方法流程图。
该方法包括步骤S11、步骤S12、步骤S13和步骤S14。具体地，在步骤S11中，第一设备1获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据；在步骤S12中，第一设备1若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换；在步骤S13中，第一设备1根据所述模型训练参数信息，训练出分布式机器学习模型；在步骤S14中，第一设备1将所述分布式机器学习模型转换为单机机器学习模型。
在此，所述第一设备1包括但不限于用户设备、网络设备、或用户设备与网络设备通过网络相集成所构成的设备。所述用户设备其包括但不限于任何一种可与用户通过触摸板进行人机交互的移动电子产品，例如智能手机、平板电脑、笔记本电脑等，所述移动电子产品可以采用任意操作系统，如android操作系统、iOS操作系统等。其中，所述网络设备包括一种能够按照事先设定或存储的指令，自动进行数值计算和信息处理的电子设备，其硬件包括但不限于微处理器、专用集成电路(ASIC)、可编程门阵列(FPGA)、数字处理器(DSP)、嵌入式设备等。所述网络设备其包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云；在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个虚拟超级计算机。所述网络包括但不限于互联网、广域网、城域网、局域网、VPN网络、无线自组织网络(Ad Hoc网络)等。优选地，第一设备1还可以是运行于所述用户设备、网络设备、或用户设备与网络设备、网络设备、触摸终端或网络设备与触摸终端通过网络相集成所构成的设备上的脚本程序。当然， 本领域技术人员应能理解上述第一设备1仅为举例，其他现有的或今后可能出现的第一设备1如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
在步骤S11中，第一设备1获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据。
在此，所述分布式训练数据用于训练出分布式机器学习模型。相应地，单机训练数据用于训练出单机机器学习模型。通常情况下，针对本地少量的预测数据利用所述单机机器学习模型进行预测，针对分布式存储的大量预测数据利用所述分布式机器学习模型进行预测。
优选地，在步骤S11中，第一设备1获取基于第一语言的模型训练参数信息。
在此，所述第一语言包括但不限于R语言、Python语言等。其中，R是用于统计分析、绘图的语言和操作环境，Python是一种面向对象、解释型计算机程序设计语言。所述模型训练参数信息可以是描述文件的格式，如JSON格式的数据描述文件，XML格式或PMML格式的模型描述文件；所述模型训练参数信息也可以是函数参数的形式。
其中，JSON是一种轻量级的数据交换语言，定义了一种轻量级的数据交换格式。XML是一种类似于HTML的扩展性标识语言，用于标记电子文件使其具有结构性的标记语言，可以用来标记数据和定义数据类型等。PMML是一种事实标准语言，用于呈现数据挖掘模型。
例如，若所述模型训练参数信息为函数参数的形式，当获取基于R语言或者python语言的模型训练参数信息，所述分布式训练数据作为分布式机器学习模型训练函数的一个参数传入。
优选地，在步骤S11中，第一设备1获取用户设备发送的模型训练参数信息。
例如，用户可在用户设备(例如笔记本电脑等)上的R语言开发环境中编写R语言脚本，或者在python语言开发环境中编写python语言脚本。其中，所述R语言脚本和所述python语言脚本包括所述模型训练参数信息。然后，用户可通过所述用户设备向所述第一设备1发送所述R语言脚本或所述 python语言脚本。
在步骤S12中，第一设备1若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换。
例如，若所述模型训练参数信息为描述文件的格式，将基于第一语言的所述模型训练参数信息解析为基于第二语言的所述模型训练参数信息；若所述模型训练参数信息为函数参数的形式，将基于第一语言的所述模型训练参数信息转换为基于第二语言的所述模型训练参数信息。在此，通过数据类型转换或者描述文件解析来达到此目的。
优选地，在步骤S12中，第一设备1利用所述第一语言与第二语言的语言转换工具对所述模型训练参数信息进行数据类型转换，以获得基于所述第二语言的所述模型训练参数信息；或者，利用描述文件解析工具对所述模型训练参数信息进行描述文件解析，以获得基于所述第二语言的所述模型训练参数信息。
在此，所述第二语言包括但不限于Java语言、Scala语言等，所述语言转换工具包括但不限于rJava，所述描述文件解析工具包括但不限于xml文件解析、json数据格式解析和PMML文件解析等。其中，Java是一种可以撰写跨平台应用程序的面向对象的程序设计语言；Scala是一门多范式的编程语言，一种类似Java的编程语言；rJava是一个R语言和Java语言的通信接口，通过底层JNI(Java Native Interface，Java本地接口)实现调用，允许在R中直接调用Java的对象和方法。rJava还提供了Java调用R的功能，是通过JRI(Java/R Interface)实现的。正是由于rJava是底层接口，并使用JNI作为接口调用，所以效率非常高。
例如，若所述模型训练参数信息为函数参数的形式，可利用rJava对所述模型训练参数信息进行数据类型转换，将所述模型训练参数信息转换为Java、Scala语言可识别的参数。若所述模型训练参数信息为描述文件的格式，可以利用xml文件解析工具、json文件解析工具或PMML文件解析工具对模型数据描述文件进行解析，将其解析为Java、Scala语言可识别的参数信息。
当然，本领域技术人员应能理解上述第一语言、第二语言、语言转换工 具和文件解析工具仅为举例，其他现有的或今后可能出现的第一语言、第二语言、语言转换工具和文件解析工具如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
在步骤S13中，第一设备1根据所述模型训练参数信息，训练出分布式机器学习模型。
例如，若所述模型训练参数信息为函数参数的形式，可通过rJava将数据类型转换后的Java、Scala语言可识别的参数，传给底层机器学习库定义好的分布式函数接口，训练出分布式机器学习模型。主要是利用rJava的J函数，用法为J("类名"，"函数方法"，参数一，参数二,...)，其中，一个“函数方法”对应一种训练出分布式机器学习模型的训练函数，J函数起调用Java函数方法的作用。在此，可通过多种分布式机器学习模型的训练函数相应地训练出多种分布式机器学习模型。
再例如，若所述模型训练参数信息为描述文件的格式，可通过xml文件解析、JSON文件解析工具或者PMML文件解析工具将数据类型解析为Java、Scala语言可以识别的参数信息，传给底层机器学习库定义好的分布式函数接口，训练出分布式机器学习模型。
在步骤S14中，第一设备1将所述分布式机器学习模型转换为单机机器学习模型。
例如，所述分布式机器学习模型主要用于对云端分布式存储的大量预测数据进行预测，所述单机机器学习模型主要用于对用户设备端本地少量的预测数据进行预测。
优选地，在步骤S14中，第一设备1将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
例如，将基于Java或Scala等所述第二语言描述的所述分布式机器学习模型转换为基于R或Python等所述第一语言描述的所述单机机器学习模型。
优选地，在步骤S14中，第一设备1利用所述语言转换工具或所述描述文件解析工具将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
例如，若所述模型训练参数信息为函数参数的形式，可通过rJava将基于 Java或Scala等语言描述的所述分布式机器学习模型转换为基于R语言描述的所述单机机器学习模型。若所述模型训练参数信息为描述文件的格式，可以利用xml文件解析工具、json文件解析工具或者PMML文件解析工具等所述描述文件解析工具将所述分布式机器学习模型转换为基于R或者Python等所述第一语言的单机机器学习模型。
更优选地，在步骤S14中，第一设备1提取出所述分布式机器学习模型参数信息；若所述分布式机器学习模型参数信息为描述文件的格式，则利用所述描述文件解析工具对所述分布式机器学习模型参数信息进行描述文件解析，否则，利用所述语言转换工具对所述分布式机器学习模型参数信息进行数据类型转换，以获得基于所述第一语言的机器学习模型参数信息；根据所述机器学习模型参数信息构造基于所述第一语言的单机机器学习模型。
例如，若所述模型训练参数信息为函数参数的形式，可提取出所述分布式机器学习模型参数信息，然后通过rJava对其进行数据类型转换，将基于Java或Scala等语言描述的所述分布式机器学习模型参数信息转换为基于R或Python等语言描述的模型参数信息，再根据基于R或Python等语言描述的模型参数信息构造出基于R或Python等语言描述的单机机器学习模型。
再例如，若所述模型训练参数信息为描述文件的格式，可提取出所述分布式机器学习模型的模型描述文件，然后利用xml文件解析工具、json文件解析工具或者PMML文件解析工具对模型描述文件进行解析，将利用xml或者json描述的模型转换为基于R或者python等语言描述的模型参数信息，再根据所述模型参数信息构造出基于R或者python等所述第一语言描述的单机机器学习模型。
优选地，所述方法还包括：第一设备1将所述单机机器学习模型返回至所述用户设备。
例如，将所述单机机器学习模型返回至所述用户设备，用于对所述用户设备端本地少量的预测数据进行预测。具体地，可通过调用所述单机机器学习模型自带的predict(预测)函数对所述用户设备端本地少量的预测数据进行预测。
图2示出根据本申请另一个方面的一种用于将单机机器学习模型转换 为分布式机器学习模型的方法流程图。该方法包括步骤S26、步骤S27、步骤S28和步骤S29。具体地，在步骤S26中，第二设备2获取单机机器学习模型；在步骤S27中，第二设备2提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换；在步骤S28中，第二设备2根据所述单机机器学习模型构造分布式机器学习模型；在步骤S29中，第二设备2根据所述分布式机器学习模型对分布式训练数据进行预测。
在此，所述第二设备2包括但不限于用户设备、网络设备、或用户设备与网络设备通过网络相集成所构成的设备。所述用户设备其包括但不限于任何一种可与用户通过触摸板进行人机交互的移动电子产品，例如智能手机、平板电脑、笔记本电脑等，所述移动电子产品可以采用任意操作系统，如android操作系统、iOS操作系统等。其中，所述网络设备包括一种能够按照事先设定或存储的指令，自动进行数值计算和信息处理的电子设备，其硬件包括但不限于微处理器、专用集成电路(ASIC)、可编程门阵列(FPGA)、数字处理器(DSP)、嵌入式设备等。所述网络设备其包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云；在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个虚拟超级计算机。所述网络包括但不限于互联网、广域网、城域网、局域网、VPN网络、无线自组织网络(Ad Hoc网络)等。优选地，第二设备2还可以是运行于所述用户设备、网络设备、或用户设备与网络设备、网络设备、触摸终端或网络设备与触摸终端通过网络相集成所构成的设备上的脚本程序。当然，本领域技术人员应能理解上述第二设备2仅为举例，其他现有的或今后可能出现的第二设备2如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
在步骤S26中，第二设备2获取单机机器学习模型。
例如，所述单机机器学习模型根据用户设备端本地的单机训练数据训练得到，在此，获取用户设备发送的所述单机机器学习模型。相应地，分布式 机器学习模型根据分布式训练数据训练得到。通常情况下，针对本地少量的预测数据利用所述单机机器学习模型进行预测，针对分布式存储的大量预测数据利用所述分布式机器学习模型进行预测。
优选地，在步骤S26中，第二设备2获取基于第一语言的单机机器学习模型。
在此，所述第一语言包括但不限于R语言、Python语言等。其中，R是用于统计分析、绘图的语言和操作环境，Python是一种面向对象、解释型计算机程序设计语言。
例如，所述基于第一语言的单机机器学习模型可以是基于R或Python等语言描述的单机机器学习模型。
在步骤S27中，第二设备2提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换。
例如，在根据所述单机机器学习模型构造分布式机器学习模型之前，首先提取出所述单机机器学习模型参数信息。其中，所述单机机器学习模型参数信息可以是描述文件的格式，如JSON格式的数据描述文件，XML格式或PMML格式的模型描述文件；所述单机机器学习模型参数信息也可以是函数参数的形式。若所述单机机器学习模型参数信息为描述文件的格式，则对其进行描述文件解析；若所述单机机器学习模型参数信息为函数参数的形式，则对其进行数据类型转换。
在此，JSON是一种轻量级的数据交换语言，定义了一种轻量级的数据交换格式。XML是一种类似于HTML的扩展性标识语言，用于标记电子文件使其具有结构性的标记语言，可以用来标记数据和定义数据类型等。PMML是一种事实标准语言，用于呈现数据挖掘模型。
优选地，在步骤S27中，第二设备2利用所述第一语言与第二语言的语言转换工具对所述单机机器学习模型参数信息进行数据类型转换，以获得基于所述第二语言的机器学习模型参数信息；或者，利用描述文件解析工具对所述单机机器学习模型参数信息进行描述文件解析，以获得基于所述第二语 言的机器学习模型参数信息。
在此，所述第二语言包括但不限于Java语言、Scala语言等，所述语言转换工具包括但不限于rJava。其中，Java是一种可以撰写跨平台应用程序的面向对象的程序设计语言；Scala是一门多范式的编程语言，一种类似Java的编程语言；rJava是一个R语言和Java语言的通信接口，通过底层JNI(Java Native Interface，Java本地接口)实现调用，允许在R中直接调用Java的对象和方法。rJava还提供了Java调用R的功能，是通过JRI(Java/R Interface)实现的。正是由于rJava是底层接口，并使用JNI作为接口调用，所以效率非常高。
例如，若所述单机机器学习模型参数信息为函数参数的形式，可提取出所述单机机器学习模型参数信息，然后通过rJava对模型参数信息进行数据类型转换，将基于R语言描述的模型参数信息转换为基于Java或Scala等语言描述的模型参数信息。
再例如，若所述单机机器学习模型参数信息为描述文件的格式，可提取出所述单机机器学习模型的描述文件，然后通过xml文件解析工具、JSON文件解析工具或者PMML文件解析工具将模型描述文件转换成基于Java或者Scala等语言描述的模型参数信息。
当然，本领域技术人员应能理解上述第一语言、第二语言、语言转换工具和文件解析工具仅为举例，其他现有的或今后可能出现的第一语言、第二语言、语言转换工具和文件解析工具如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
在步骤S28中，第二设备2根据所述单机机器学习模型构造分布式机器学习模型。
例如，可通过rJava调用分布式机器模型的构造函数，得到分布式机器学习模型。主要是利用rJava的.jnew函数，用法为.jnew("模型类名"，参数一，参数二，...)，其中，一个“模型类名”对应一种分布式机器学习模型的Java类名，.jnew函数起创建一个Java对象的作用，该对象即是分布式机器学习模型。在此，可通过多种分布式机器学习模型的构造函数相应地构造出多种分布式机器学习模型。
优选地，在步骤S28中，第二设备2根据所述机器学习模型参数信息构 造基于所述第二语言的分布式机器学习模型。
例如，可以根据基于Java或Scala等语言描述的所述机器学习模型参数信息构造出基于Java或Scala等语言描述的分布式机器学习模型。
再例如，还可以根据基于Java或者Scala等语言描述的所述机器学习模型xml描述文件，JSON描述文件和PMML描述文件等，利用相对应的文件解析工具构造出基于Java或者Scala等语言描述的分布式机器学习模型。
在步骤S29中，第二设备2根据所述分布式机器学习模型对分布式训练数据进行预测。
例如，将所述分布式机器学习模型用于对云端分布式存储的大量预测数据进行预测。具体地，可通过调用所述分布式机器学习模型的预测函数，对所述分布式训练数据进行预测。主要是利用model$predict函数，用法为model$predict(data,...)，其中，model是指所述分布式机器学习模型，predict是指所述分布式机器学习模型的预测函数，data是指所述分布式训练数据。
优选地，所述方法还包括：第二设备2获取用户设备发送的分布式训练数据。
例如，用户设备可通过数据提取函数得到存储在分布式存储介质或者分布式数据库中的所述分布式训练数据，然后用户设备将所述分布式训练数据发送至第二设备2。
图3示出根据本申请又一个方面的一种用于将分布式机器学习模型转换为单机机器学习模型的第一设备1，其中，第一设备1包括第一装置11、第二装置12、第三装置13和第四装置14。
具体地，所述第一装置11获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据；所述第二装置12若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换；所述第三装置13根据所述模型训练参数信息，训练出分布式机器学习模型；所述第四装置14将所述分布式机器学习模型转换为单机机器学习模型。
在此，所述第一设备1包括但不限于用户设备、网络设备、或用户设备与网络设备通过网络相集成所构成的设备。所述用户设备其包括但不限于任 何一种可与用户通过触摸板进行人机交互的移动电子产品，例如智能手机、平板电脑、笔记本电脑等，所述移动电子产品可以采用任意操作系统，如android操作系统、iOS操作系统等。其中，所述网络设备包括一种能够按照事先设定或存储的指令，自动进行数值计算和信息处理的电子设备，其硬件包括但不限于微处理器、专用集成电路(ASIC)、可编程门阵列(FPGA)、数字处理器(DSP)、嵌入式设备等。所述网络设备其包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云；在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个虚拟超级计算机。所述网络包括但不限于互联网、广域网、城域网、局域网、VPN网络、无线自组织网络(Ad Hoc网络)等。优选地，第一设备1还可以是运行于所述用户设备、网络设备、或用户设备与网络设备、网络设备、触摸终端或网络设备与触摸终端通过网络相集成所构成的设备上的脚本程序。当然，本领域技术人员应能理解上述第一设备1仅为举例，其他现有的或今后可能出现的第一设备1如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
所述第一装置11获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据。
在此，所述分布式训练数据用于训练出分布式机器学习模型。相应地，单机训练数据用于训练出单机机器学习模型。通常情况下，针对本地少量的预测数据利用所述单机机器学习模型进行预测，针对分布式存储的大量预测数据利用所述分布式机器学习模型进行预测。
优选地，所述第一装置11获取基于第一语言的模型训练参数信息。
在此，所述第一语言包括但不限于R语言、Python语言等。其中，R是用于统计分析、绘图的语言和操作环境，Python是一种面向对象、解释型计算机程序设计语言。所述模型训练参数信息可以是描述文件的格式，如JSON格式的数据描述文件，XML格式或PMML格式的模型描述文件；所述模型训练参数信息也可以是函数参数的形式。
其中，JSON是一种轻量级的数据交换语言，定义了一种轻量级的数据交 换格式。XML是一种类似于HTML的扩展性标识语言，用于标记电子文件使其具有结构性的标记语言，可以用来标记数据和定义数据类型等。PMML是一种事实标准语言，用于呈现数据挖掘模型。
例如，若所述模型训练参数信息为函数参数的形式，当获取基于R语言或者python语言的模型训练参数信息，所述分布式训练数据作为分布式机器学习模型训练函数的一个参数传入。
优选地，所述第一装置11获取用户设备发送的模型训练参数信息。
例如，用户可在用户设备(例如笔记本电脑等)上的R语言开发环境中编写R语言脚本，或者在python语言开发环境中编写python语言脚本。其中，所述R语言脚本和所述python语言脚本包括所述模型训练参数信息。然后，用户可通过所述用户设备向所述第一设备1发送所述R语言脚本或所述python语言脚本。
所述第二装置12若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换。
例如，若所述模型训练参数信息为描述文件的格式，将基于第一语言的所述模型训练参数信息解析为基于第二语言的所述模型训练参数信息；若所述模型训练参数信息为函数参数的形式，将基于第一语言的所述模型训练参数信息转换为基于第二语言的所述模型训练参数信息。在此，通过数据类型转换或者描述文件解析来达到此目的。
优选地，所述第二装置12利用所述第一语言与第二语言的语言转换工具对所述模型训练参数信息进行数据类型转换，以获得基于所述第二语言的所述模型训练参数信息；或者，利用描述文件解析工具对所述模型训练参数信息进行描述文件解析，以获得基于所述第二语言的所述模型训练参数信息。
在此，所述第二语言包括但不限于Java语言、Scala语言等，所述语言转换工具包括但不限于rJava，所述描述文件解析工具包括但不限于xml文件解析、json数据格式解析和PMML文件解析等。其中，Java是一种可以撰写跨平台应用程序的面向对象的程序设计语言；Scala是一门多范式的编程语言，一种类似Java的编程语言；rJava是一个R语言和Java语言的通信接口，通 过底层JNI(Java Native Interface，Java本地接口)实现调用，允许在R中直接调用Java的对象和方法。rJava还提供了Java调用R的功能，是通过JRI(Java/R Interface)实现的。正是由于rJava是底层接口，并使用JNI作为接口调用，所以效率非常高。
例如，若所述模型训练参数信息为函数参数的形式，可利用rJava对所述模型训练参数信息进行数据类型转换，将所述模型训练参数信息转换为Java、Scala语言可识别的参数。若所述模型训练参数信息为描述文件的格式，可以利用xml文件解析工具、json文件解析工具或PMML文件解析工具对模型数据描述文件进行解析，将其解析为Java、Scala语言可识别的参数信息。
当然，本领域技术人员应能理解上述第一语言、第二语言、语言转换工具和文件解析工具仅为举例，其他现有的或今后可能出现的第一语言、第二语言、语言转换工具和文件解析工具如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
所述第三装置13根据所述模型训练参数信息，训练出分布式机器学习模型。
例如，若所述模型训练参数信息为函数参数的形式，可通过rJava将数据类型转换后的Java、Scala语言可识别的参数，传给底层机器学习库定义好的分布式函数接口，训练出分布式机器学习模型。主要是利用rJava的J函数，用法为J("类名"，"函数方法"，参数一，参数二,...)，其中，一个“函数方法”对应一种训练出分布式机器学习模型的训练函数，J函数起调用Java函数方法的作用。在此，可通过多种分布式机器学习模型的训练函数相应地训练出多种分布式机器学习模型。
再例如，若所述模型训练参数信息为描述文件的格式，可通过xml文件解析、JSON文件解析工具或者PMML文件解析工具将数据类型解析为Java、Scala语言可以识别的参数信息，传给底层机器学习库定义好的分布式函数接口，训练出分布式机器学习模型。
所述第四装置14将所述分布式机器学习模型转换为单机机器学习模型。
例如，所述分布式机器学习模型主要用于对云端分布式存储的大量预测数据进行预测，所述单机机器学习模型主要用于对用户设备端本地少量的预 测数据进行预测。
优选地，所述第四装置14将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
例如，将基于Java或Scala等所述第二语言描述的所述分布式机器学习模型转换为基于R或Python等所述第一语言描述的所述单机机器学习模型。
优选地，所述第四装置14利用所述语言转换工具或所述描述文件解析工具将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
例如，若所述模型训练参数信息为函数参数的形式，可通过rJava将基于Java或Scala等语言描述的所述分布式机器学习模型转换为基于R语言描述的所述单机机器学习模型。若所述模型训练参数信息为描述文件的格式，可以利用xml文件解析工具、json文件解析工具或者PMML文件解析工具等所述描述文件解析工具将所述分布式机器学习模型转换为基于R或者Python等所述第一语言的单机机器学习模型。
更优选地，所述第四装置14提取出所述分布式机器学习模型参数信息；若所述分布式机器学习模型参数信息为描述文件的格式，则利用所述描述文件解析工具对所述分布式机器学习模型参数信息进行描述文件解析，否则，利用所述语言转换工具对所述分布式机器学习模型参数信息进行数据类型转换，以获得基于所述第一语言的机器学习模型参数信息；根据所述机器学习模型参数信息构造基于所述第一语言的单机机器学习模型。
例如，若所述模型训练参数信息为函数参数的形式，可提取出所述分布式机器学习模型参数信息，然后通过rJava对其进行数据类型转换，将基于Java或Scala等语言描述的所述分布式机器学习模型参数信息转换为基于R或Python等语言描述的模型参数信息，再根据基于R或Python等语言描述的模型参数信息构造出基于R或Python等语言描述的单机机器学习模型。
再例如，若所述模型训练参数信息为描述文件的格式，可提取出所述分布式机器学习模型的模型描述文件，然后利用xml文件解析工具、json文件解析工具或者PMML文件解析工具对模型描述文件进行解析，将利用xml或者json描述的模型转换为基于R或者python等语言描述的模型参数信息，再根据所述模型参数信息构造出基于R或者python等所述第一语言描述的单 机机器学习模型。
优选地，所述第一设备1还包括第五装置(图中未示出)；所述第五装置将所述单机机器学习模型返回至所述用户设备。
例如，将所述单机机器学习模型返回至所述用户设备，用于对所述用户设备端本地少量的预测数据进行预测。具体地，可通过调用所述单机机器学习模型自带的predict(预测)函数对所述用户设备端本地少量的预测数据进行预测。
图4示出根据本申请再一个方面的一种用于将单机机器学习模型转换为分布式机器学习模型的第二设备2，其中，第二设备2包括第六装置26、第七装置27、第八装置28和第九装置29。
具体地，所述第六装置26获取单机机器学习模型；所述第七装置27提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换；所述第八装置28根据所述单机机器学习模型构造分布式机器学习模型；所述第九装置29根据所述分布式机器学习模型对分布式训练数据进行预测。
在此，所述第二设备2包括但不限于用户设备、网络设备、或用户设备与网络设备通过网络相集成所构成的设备。所述用户设备其包括但不限于任何一种可与用户通过触摸板进行人机交互的移动电子产品，例如智能手机、平板电脑、笔记本电脑等，所述移动电子产品可以采用任意操作系统，如android操作系统、iOS操作系统等。其中，所述网络设备包括一种能够按照事先设定或存储的指令，自动进行数值计算和信息处理的电子设备，其硬件包括但不限于微处理器、专用集成电路(ASIC)、可编程门阵列(FPGA)、数字处理器(DSP)、嵌入式设备等。所述网络设备其包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云；在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个虚拟超级计算机。所述网络包括但不限于互联网、广域网、城域网、局域网、VPN网络、无线自组织网络(Ad Hoc网络)等。优选地，第二设备2还可以是运 行于所述用户设备、网络设备、或用户设备与网络设备、网络设备、触摸终端或网络设备与触摸终端通过网络相集成所构成的设备上的脚本程序。当然，本领域技术人员应能理解上述第二设备2仅为举例，其他现有的或今后可能出现的第二设备2如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
所述第六装置26获取单机机器学习模型。
例如，所述单机机器学习模型根据用户设备端本地的单机训练数据训练得到，在此，获取用户设备发送的所述单机机器学习模型。相应地，分布式机器学习模型根据分布式训练数据训练得到。通常情况下，针对本地少量的预测数据利用所述单机机器学习模型进行预测，针对分布式存储的大量预测数据利用所述分布式机器学习模型进行预测。
优选地，所述第六装置26获取基于第一语言的单机机器学习模型。
在此，所述第一语言包括但不限于R语言、Python语言等。其中，R是用于统计分析、绘图的语言和操作环境，Python是一种面向对象、解释型计算机程序设计语言。
例如，所述基于第一语言的单机机器学习模型可以是基于R或Python等语言描述的单机机器学习模型。
所述第七装置27提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换。
例如，在根据所述单机机器学习模型构造分布式机器学习模型之前，首先提取出所述单机机器学习模型参数信息。其中，所述单机机器学习模型参数信息可以是描述文件的格式，如JSON格式的数据描述文件，XML格式或PMML格式的模型描述文件；所述单机机器学习模型参数信息也可以是函数参数的形式。若所述单机机器学习模型参数信息为描述文件的格式，则对其进行描述文件解析；若所述单机机器学习模型参数信息为函数参数的形式，则对其进行数据类型转换。
在此，JSON是一种轻量级的数据交换语言，定义了一种轻量级的数据交 换格式。XML是一种类似于HTML的扩展性标识语言，用于标记电子文件使其具有结构性的标记语言，可以用来标记数据和定义数据类型等。PMML是一种事实标准语言，用于呈现数据挖掘模型。
优选地，所述第七装置27利用所述第一语言与第二语言的语言转换工具对所述单机机器学习模型参数信息进行数据类型转换，以获得基于所述第二语言的机器学习模型参数信息；或者，利用描述文件解析工具对所述单机机器学习模型参数信息进行描述文件解析，以获得基于所述第二语言的机器学习模型参数信息。
在此，所述第二语言包括但不限于Java语言、Scala语言等，所述语言转换工具包括但不限于rJava。其中，Java是一种可以撰写跨平台应用程序的面向对象的程序设计语言；Scala是一门多范式的编程语言，一种类似Java的编程语言；rJava是一个R语言和Java语言的通信接口，通过底层JNI(Java Native Interface，Java本地接口)实现调用，允许在R中直接调用Java的对象和方法。rJava还提供了Java调用R的功能，是通过JRI(Java/R Interface)实现的。正是由于rJava是底层接口，并使用JNI作为接口调用，所以效率非常高。
例如，若所述单机机器学习模型参数信息为函数参数的形式，可提取出所述单机机器学习模型参数信息，然后通过rJava对模型参数信息进行数据类型转换，将基于R语言描述的模型参数信息转换为基于Java或Scala等语言描述的模型参数信息。
再例如，若所述单机机器学习模型参数信息为描述文件的格式，可提取出所述单机机器学习模型的描述文件，然后通过xml文件解析工具、JSON文件解析工具或者PMML文件解析工具将模型描述文件转换成基于Java或者Scala等语言描述的模型参数信息。
当然，本领域技术人员应能理解上述第一语言、第二语言、语言转换工具和文件解析工具仅为举例，其他现有的或今后可能出现的第一语言、第二语言、语言转换工具和文件解析工具如可适用于本申请，也应包含在本申请保护范围以内，并在此以引用方式包含于此。
所述第八装置28根据所述单机机器学习模型构造分布式机器学习模型。
例如，可通过rJava调用分布式机器模型的构造函数，得到分布式机 器学习模型。主要是利用rJava的.jnew函数，用法为.jnew("模型类名"，参数一，参数二，...)，其中，一个“模型类名”对应一种分布式机器学习模型的Java类名，.jnew函数起创建一个Java对象的作用，该对象即是分布式机器学习模型。在此，可通过多种分布式机器学习模型的构造函数相应地构造出多种分布式机器学习模型。
优选地，所述第八装置28根据所述机器学习模型参数信息构造基于所述第二语言的分布式机器学习模型。
例如，可以根据基于Java或Scala等语言描述的所述机器学习模型参数信息构造出基于Java或Scala等语言描述的分布式机器学习模型。
再例如，还可以根据基于Java或者Scala等语言描述的所述机器学习模型xml描述文件，JSON描述文件和PMML描述文件等，利用相对应的文件解析工具构造出基于Java或者Scala等语言描述的分布式机器学习模型。
所述第九装置29根据所述分布式机器学习模型对分布式训练数据进行预测。
例如，将所述分布式机器学习模型用于对云端分布式存储的大量预测数据进行预测。具体地，可通过调用所述分布式机器学习模型的预测函数，对所述分布式训练数据进行预测。主要是利用model$predict函数，用法为model$predict(data,...)，其中，model是指所述分布式机器学习模型，predict是指所述分布式机器学习模型的预测函数，data是指所述分布式训练数据。
优选地，所述第二设备2还包括第十装置(图中未示出)；所述第十装置获取用户设备发送的分布式训练数据。
例如，用户设备可通过数据提取函数得到存储在分布式存储介质或者分布式数据库中的所述分布式训练数据，然后用户设备将所述分布式训练数据发送至第二设备2。
本申请还提供了一种单机机器学习模型与分布式机器学习模型之间进行转换的第三设备，其中，第三设备包括如上所述的第一设备1，以及如上所述的第二设备2。
在优选的实施例中，所述第三设备既可通过本申请一个方面所述方法将分布式机器学习模型转换为单机机器学习模型，也可通过本申请另一个方 面所述方法将单机机器学习模型转换为分布式机器学习模型；即，所述第三设备可实现单机机器学习模型与分布式机器学习模型之间的双向转换。
与现有技术相比，本申请获取模型训练参数信息，并对其进行数据类型转换或描述文件解析，然后根据所述模型训练参数信息训练出分布式机器学习模型，再将所述分布式机器学习模型转换为单机机器学习模型；本申请还获取单机机器学习模型，提取出所述单机机器学习模型参数信息，对所述单机机器学习模型参数信息进行数据类型转换或描述文件解析，然后根据所述单机机器学习模型构造分布式机器学习模型；本申请可实现单机机器学习模型与分布式机器学习模型之间的相互转换，大大增加了两者之间的兼容性。进一步地，本申请可实现R或者python等单机语言原生的单机机器学习模型与基于Java、Scala等语言的分布式机器学习模型之间的相互转换，同时本申请在进行转换时不需要启动本地进行类型转换的work进程和分布式进程，减少了时间的开销，提高了性能。
需要注意的是，本申请可在软件和/或软件与硬件的组合体中被实施，例如，可采用专用集成电路(ASIC)、通用目的计算机或任何其他类似硬件设备来实现。在一个实施例中，本申请的软件程序可以通过处理器执行以实现上文所述步骤或功能。同样地，本申请的软件程序(包括相关的数据结构)可以被存储到计算机可读记录介质中，例如，RAM存储器，磁或光驱动器或软磁盘及类似设备。另外，本申请的一些步骤或功能可采用硬件来实现，例如，作为与处理器配合从而执行各个步骤或功能的电路。
另外，本申请的一部分可被应用为计算机程序产品，例如计算机程序指令，当其被计算机执行时，通过该计算机的操作，可以调用或提供根据本申请的方法和/或技术方案。而调用本申请的方法的程序指令，可能被存储在固定的或可移动的记录介质中，和/或通过广播或其他信号承载媒体中的数据流而被传输，和/或被存储在根据所述程序指令运行的计算机设备的工作存储器中。在此，根据本申请的一个实施例包括一个装置，该装置包括用于存储计算机程序指令的存储器和用于执行程序指令的处理器，其中，当该计算机程序指令被该处理器执行时，触发该装置运行基于前述根据本申请的多个实施例的方法和/或技术方案。
对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。

Claims (21)

  1. 一种用于将分布式机器学习模型转换为单机机器学习模型的方法，其中，该方法包括：
    获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据；
    若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换；
    根据所述模型训练参数信息，训练出分布式机器学习模型；
    将所述分布式机器学习模型转换为单机机器学习模型。
  2. 根据权利要求1所述的方法，其中，所述获取模型训练参数信息包括：
    获取基于第一语言的模型训练参数信息；
    其中，所述将所述分布式机器学习模型转换为单机机器学习模型包括：
    将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
  3. 根据权利要求2所述的方法，其中，所述对所述模型训练参数信息进行数据类型转换包括：
    利用所述第一语言与第二语言的语言转换工具对所述模型训练参数信息进行数据类型转换，以获得基于所述第二语言的所述模型训练参数信息；
    其中，所述对所述模型训练参数信息进行描述文件解析包括：
    利用描述文件解析工具对所述模型训练参数信息进行描述文件解析，以获得基于所述第二语言的所述模型训练参数信息。
  4. 根据权利要求3所述的方法，其中，所述将所述分布式机器学习模型转换为单机机器学习模型包括：
    利用所述语言转换工具或所述描述文件解析工具将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
  5. 根据权利要求4所述的方法，其中，所述将所述分布式机器学习模型转换为单机机器学习模型包括：
    提取出所述分布式机器学习模型参数信息；
    若所述分布式机器学习模型参数信息为描述文件的格式，则利用所述描述文件解析工具对所述分布式机器学习模型参数信息进行描述文件解析，否 则，利用所述语言转换工具对所述分布式机器学习模型参数信息进行数据类型转换，以获得基于所述第一语言的机器学习模型参数信息；
    根据所述机器学习模型参数信息构造基于所述第一语言的单机机器学习模型。
  6. 根据权利要求1所述的方法，其中，所述获取模型训练参数信息包括：
    获取用户设备发送的模型训练参数信息；
    其中，所述方法还包括：
    将所述单机机器学习模型返回至所述用户设备。
  7. 一种用于将单机机器学习模型转换为分布式机器学习模型的方法，其中，该方法包括：
    获取单机机器学习模型；
    提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换；
    根据所述单机机器学习模型构造分布式机器学习模型；
    根据所述分布式机器学习模型对分布式训练数据进行预测。
  8. 根据权利要求7所述的方法，其中，所述获取单机机器学习模型包括：
    获取基于第一语言的单机机器学习模型。
  9. 根据权利要求8所述的方法，其中，所述对所述单机机器学习模型参数信息进行数据类型转换包括：
    利用所述第一语言与第二语言的语言转换工具对所述单机机器学习模型参数信息进行数据类型转换，以获得基于所述第二语言的机器学习模型参数信息；
    其中，所述对所述单机机器学习模型参数信息进行描述文件解析包括：
    利用描述文件解析工具对所述单机机器学习模型参数信息进行描述文件解析，以获得基于所述第二语言的机器学习模型参数信息；
    其中，所述根据所述单机机器学习模型构造分布式机器学习模型包括：
    根据所述机器学习模型参数信息构造基于所述第二语言的分布式机器学习模型。
  10. 根据权利要求7至9中任一项所述的方法，其中，所述方法还包括：
    获取用户设备发送的分布式训练数据。
  11. 一种用于将分布式机器学习模型转换为单机机器学习模型的第一设备，其中，该设备包括：
    第一装置，用于获取模型训练参数信息，其中，所述模型训练参数信息包括分布式训练数据；
    第二装置，用于若所述模型训练参数信息为描述文件的格式，则对所述模型训练参数信息进行描述文件解析，否则，对所述模型训练参数信息进行数据类型转换；
    第三装置，用于根据所述模型训练参数信息，训练出分布式机器学习模型；
    第四装置，用于将所述分布式机器学习模型转换为单机机器学习模型。
  12. 根据权利要求11所述的设备，其中，所述第一装置用于：
    获取基于第一语言的模型训练参数信息；
    其中，所述第二装置用于：
    将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
  13. 根据权利要求12所述的设备，其中，所述对所述模型训练参数信息进行数据类型转换包括：
    利用所述第一语言与第二语言的语言转换工具对所述模型训练参数信息进行数据类型转换，以获得基于所述第二语言的所述模型训练参数信息；
    其中，所述对所述模型训练参数信息进行描述文件解析包括：
    利用描述文件解析工具对所述模型训练参数信息进行描述文件解析，以获得基于所述第二语言的所述模型训练参数信息。
  14. 根据权利要求13所述的设备，其中，所述第四装置用于：
    利用所述语言转换工具或所述描述文件解析工具将所述分布式机器学习模型转换为基于所述第一语言的单机机器学习模型。
  15. 根据权利要求14所述的设备，其中，所述第四装置用于：
    提取出所述分布式机器学习模型参数信息；
    若所述分布式机器学习模型参数信息为描述文件的格式，则利用所述描述文件解析工具对所述分布式机器学习模型参数信息进行描述文件解析，否则，利用所述语言转换工具对所述分布式机器学习模型参数信息进行数据类型转换，以获得基于所述第一语言的机器学习模型参数信息；
    根据所述机器学习模型参数信息构造基于所述第一语言的单机机器学习模型。
  16. 根据权利要求11所述的设备，其中，所述第一装置用于：
    获取用户设备发送的模型训练参数信息；
    其中，所述设备还包括：
    第五装置，用于将所述单机机器学习模型返回至所述用户设备。
  17. 一种用于将单机机器学习模型转换为分布式机器学习模型的第二设备，其中，该设备包括：
    第六装置，用于获取单机机器学习模型；
    第七装置，用于提取出所述单机机器学习模型参数信息，若所述单机机器学习模型参数信息为描述文件的格式，则对所述单机机器学习模型参数信息进行描述文件解析，否则，对所述单机机器学习模型参数信息进行数据类型转换；
    第八装置，用于根据所述单机机器学习模型构造分布式机器学习模型；
    第九装置，用于根据所述分布式机器学习模型对分布式训练数据进行预测。
  18. 根据权利要求17所述的设备，其中，所述第六装置用于：
    获取基于第一语言的单机机器学习模型。
  19. 根据权利要求18所述的设备，其中，所述对所述单机机器学习模型参数信息进行数据类型转换包括：
    利用所述第一语言与第二语言的语言转换工具对所述单机机器学习模型参数信息进行数据类型转换，以获得基于所述第二语言的机器学习模型参数信息；
    其中，所述对所述单机机器学习模型参数信息进行描述文件解析包括：
    利用描述文件解析工具对所述单机机器学习模型参数信息进行描述文件 解析，以获得基于所述第二语言的机器学习模型参数信息；
    其中，所述第八装置包括：
    根据所述机器学习模型参数信息构造基于所述第二语言的分布式机器学习模型。
  20. 根据权利要求17至19中任一项所述的设备，其中，所述设备还包括：
    第十装置，用于获取用户设备发送的分布式训练数据。
  21. 一种单机机器学习模型与分布式机器学习模型之间进行转换的第三设备，其中，该设备包括如权利要求11至16中任一项所述的第一设备，以及如权利要求17至20中任一项所述的第二设备。
PCT/CN2017/080844 2016-05-11 2017-04-18 机器学习模型之间进行转换的方法与设备 WO2017193769A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610308127.9 2016-05-11
CN201610308127.9A CN106022483B (zh) 2016-05-11 2016-05-11 机器学习模型之间进行转换的方法与设备

Publications (1)

Publication Number Publication Date
WO2017193769A1 true WO2017193769A1 (zh) 2017-11-16

Family

ID=57099955

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/080844 WO2017193769A1 (zh) 2016-05-11 2017-04-18 机器学习模型之间进行转换的方法与设备

Country Status (2)

Country Link
CN (1) CN106022483B (zh)
WO (1) WO2017193769A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107992299A (zh) * 2017-11-27 2018-05-04 郑州云海信息技术有限公司 神经网络超参数提取转换方法、系统、装置及存储介质
CN108829415A (zh) * 2018-05-29 2018-11-16 努比亚技术有限公司 模型加载方法、服务器及计算机可读存储介质
CN109460396A (zh) * 2018-10-12 2019-03-12 中国平安人寿保险股份有限公司 模型处理方法及装置、存储介质和电子设备
CN111045688A (zh) * 2019-12-06 2020-04-21 支付宝(杭州)信息技术有限公司 一种模型安全部署和预测的方法和系统
CN111275199A (zh) * 2020-01-17 2020-06-12 深圳壹账通智能科技有限公司 一种深度学习模型文件的转换方法、系统、计算机设备及计算机可读存储介质

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022483B (zh) * 2016-05-11 2019-06-14 星环信息科技（上海）有限公司 机器学习模型之间进行转换的方法与设备
CN108537340B (zh) * 2017-03-02 2021-04-27 北京君正集成电路股份有限公司 模型数据的读取方法和装置
CN109426651A (zh) * 2017-06-20 2019-03-05 北京小米移动软件有限公司 文件转换的方法及装置
US11823067B2 (en) 2017-06-27 2023-11-21 Hcl Technologies Limited System and method for tuning and deploying an analytical model over a target eco-system
CN107679625B (zh) * 2017-08-30 2019-09-17 第四范式（北京）技术有限公司 针对数据记录执行机器学习的分布式系统及其方法
CN108154238B (zh) * 2017-12-25 2020-11-27 东软集团股份有限公司 机器学习流程的迁移方法、装置、存储介质和电子设备
CN108334895B (zh) * 2017-12-29 2022-04-26 腾讯科技（深圳）有限公司 目标数据的分类方法、装置、存储介质及电子装置
CN110083334B (zh) * 2018-01-25 2023-06-20 百融至信(北京)科技有限公司 模型上线的方法及装置
CN109343857B (zh) * 2018-08-22 2023-07-21 中国平安人寿保险股份有限公司 线上部署机器学习模型的方法、设备和存储介质
CN109784494A (zh) * 2018-11-28 2019-05-21 同盾控股有限公司 一种基于pyspark的机器学习方法和装置
CN109669701A (zh) * 2018-12-17 2019-04-23 深圳魔数智擎科技有限公司 对机器学习模型进行编译处理的方法、编译器及服务器
WO2020247499A1 (en) 2019-06-03 2020-12-10 Cerebri AI Inc. Machine learning pipeline optimization
TWI773907B (zh) * 2019-07-11 2022-08-11 緯創資通股份有限公司 資料擷取裝置與資料運算系統及方法
CN110837896B (zh) * 2019-11-22 2022-07-08 中国联合网络通信集团有限公司 机器学习模型的存储和调用方法、装置
CN110852449B (zh) * 2019-11-25 2023-11-14 北京百度网讯科技有限公司 模型迁移方法和电子设备
CN111522545B (zh) * 2020-07-06 2020-10-20 嘉兴太美医疗科技有限公司 基于Java的动态构建模型的方法、装置、系统和可读介质
CN111858348A (zh) * 2020-07-23 2020-10-30 深圳慕智科技有限公司 一种基于神经网络输出向量的测试数据度量方法
CN112861934A (zh) * 2021-01-25 2021-05-28 深圳市优必选科技股份有限公司 一种嵌入式终端的图像分类方法、装置及嵌入式终端

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101710361A (zh) * 2009-11-13 2010-05-19 北京航空航天大学 基于面向服务架构的分布式交通仿真平台及其仿真方法
US20110099131A1 (en) * 2009-10-22 2011-04-28 Yahoo! Inc. Pairwise ranking-based classifier
CN103701664A (zh) * 2013-12-25 2014-04-02 北京航天测控技术有限公司 一种设备的测试运行时方法及测试服务器
CN104572072A (zh) * 2014-12-01 2015-04-29 北京百度网讯科技有限公司 一种对基于mvc模式的程序的语言转换方法与设备
CN104778254A (zh) * 2015-04-20 2015-07-15 北京蓝色光标品牌管理顾问股份有限公司 一种非参数的话题自动标注的分布式系统和标注方法
CN105426344A (zh) * 2015-11-09 2016-03-23 南京大学 基于Spark的分布式大规模矩阵乘法的矩阵计算方法
CN106022483A (zh) * 2016-05-11 2016-10-12 星环信息科技（上海）有限公司 机器学习模型之间进行转换的方法与设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1734457A (zh) * 2004-10-12 2006-02-15 上海燃料电池汽车动力系统有限公司 车载分布式控制系统的计算机辅助开发方法
CN103246516B (zh) * 2013-05-16 2017-02-08 中国科学院计算机网络信息中心 一种基于互联网的遥感数据分析工具的封装服务方法
US10564937B2 (en) * 2014-07-18 2020-02-18 Sap Se Relational logic integration
CN104834730B (zh) * 2015-05-15 2018-06-01 北京京东尚科信息技术有限公司 数据分析系统和方法
CN105389220B (zh) * 2015-11-09 2019-02-15 南京大学 在交互式r语言平台中进行并行线性代数计算的方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110099131A1 (en) * 2009-10-22 2011-04-28 Yahoo! Inc. Pairwise ranking-based classifier
CN101710361A (zh) * 2009-11-13 2010-05-19 北京航空航天大学 基于面向服务架构的分布式交通仿真平台及其仿真方法
CN103701664A (zh) * 2013-12-25 2014-04-02 北京航天测控技术有限公司 一种设备的测试运行时方法及测试服务器
CN104572072A (zh) * 2014-12-01 2015-04-29 北京百度网讯科技有限公司 一种对基于mvc模式的程序的语言转换方法与设备
CN104778254A (zh) * 2015-04-20 2015-07-15 北京蓝色光标品牌管理顾问股份有限公司 一种非参数的话题自动标注的分布式系统和标注方法
CN105426344A (zh) * 2015-11-09 2016-03-23 南京大学 基于Spark的分布式大规模矩阵乘法的矩阵计算方法
CN106022483A (zh) * 2016-05-11 2016-10-12 星环信息科技（上海）有限公司 机器学习模型之间进行转换的方法与设备

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107992299A (zh) * 2017-11-27 2018-05-04 郑州云海信息技术有限公司 神经网络超参数提取转换方法、系统、装置及存储介质
CN108829415A (zh) * 2018-05-29 2018-11-16 努比亚技术有限公司 模型加载方法、服务器及计算机可读存储介质
CN109460396A (zh) * 2018-10-12 2019-03-12 中国平安人寿保险股份有限公司 模型处理方法及装置、存储介质和电子设备
CN109460396B (zh) * 2018-10-12 2024-06-04 中国平安人寿保险股份有限公司 模型处理方法及装置、存储介质和电子设备
CN111045688A (zh) * 2019-12-06 2020-04-21 支付宝(杭州)信息技术有限公司 一种模型安全部署和预测的方法和系统
CN111275199A (zh) * 2020-01-17 2020-06-12 深圳壹账通智能科技有限公司 一种深度学习模型文件的转换方法、系统、计算机设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN106022483A (zh) 2016-10-12
CN106022483B (zh) 2019-06-14

Similar Documents

Publication Publication Date Title
WO2017193769A1 (zh) 机器学习模型之间进行转换的方法与设备
US10169031B2 (en) Program code library searching and selection in a networked computing environment
US8156473B2 (en) Model oriented debugging
US10176270B2 (en) Performance of template based javascript widgets
US8239839B2 (en) Asynchrony debugging using web services interface
JP2019512973A (ja) リアルタイムデータフロープログラミングのための効率的な状態機械
US11550698B2 (en) Providing additional stack trace information for time-based sampling in asynchronous execution environments
CN109873735B (zh) H5页面的性能测试方法、装置和计算机设备
JP2018525714A (ja) アプリケーションのシミュレーション
CN110457144A (zh) 一种实现前端应用的方法、装置、介质和设备
CN113515271B (zh) 服务代码生成方法、装置、电子设备及可读存储介质
CN108984202B (zh) 一种电子资源分享方法、装置和存储介质
CN112383533A (zh) 报文格式转换方法及装置
US9934029B2 (en) Annotation driven representational state transfer (REST) web services
CN112328219A (zh) 业务访问处理方法、装置、系统和计算机设备
US11689630B2 (en) Request processing method and apparatus, electronic device, and computer storage medium
WO2024124833A1 (zh) 计算函数构建方法、计算引擎、电子设备及可读存储介质
US20140298455A1 (en) Cryptographic mechanisms to provide information privacy and integrity
CN113138767B (zh) 代码语言转换方法、装置、电子设备及存储介质
CN113687880B (zh) 一种调用组件的方法、装置、设备和介质
EP3785777B1 (en) Information processing system, information processing method, and program for development apparatus
CN111782196A (zh) 基于mvp架构的开发方法及装置
US20160034326A1 (en) Monitoring a business transaction utilizing php engines
CN118689484A (zh) 一种任务执行方法、装置、设备及存储介质
CN116820566A (zh) 数据处理方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17795387

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17795387

Country of ref document: EP

Kind code of ref document: A1