WO2017124974A1 - 逻辑回归梯度的计算方法和装置 - Google Patents

逻辑回归梯度的计算方法和装置 Download PDF

Info

Publication number
WO2017124974A1
WO2017124974A1 PCT/CN2017/071130 CN2017071130W WO2017124974A1 WO 2017124974 A1 WO2017124974 A1 WO 2017124974A1 CN 2017071130 W CN2017071130 W CN 2017071130W WO 2017124974 A1 WO2017124974 A1 WO 2017124974A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
logistic regression
weight vector
gradient
user data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/071130
Other languages
English (en)
French (fr)
Inventor
周俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Publication of WO2017124974A1 publication Critical patent/WO2017124974A1/zh
Anticipated expiration legal-status Critical
Priority to US16/043,043 priority Critical patent/US10970596B2/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0242—Determining effectiveness of advertisements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0277—Online advertisement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding

Definitions

  • the present application relates to the field of Internet technologies, and in particular, to a method and an apparatus for calculating a logical regression gradient.
  • Logistic regression is the most commonly used model in the online advertisement CTR (Click-Through-Rate) and is used on a large scale in the industry. Who can train the logistic regression model in a limited time, can better provide users with reasonable advertising and enhance the user experience.
  • the most central part is the calculation of the gradient.
  • the existing gradient calculation method is accelerated by starting multi-threading on the computing machine.
  • each thread needs a full gradient vector to be stored in the memory. That is to say, if 10 threads are started to calculate the gradient on a computing machine, assuming 1 gradient vector is 1 billion, which is double data, then 1 thread needs to consume about 7.5G of memory, and a total of need to consume the computing machine. 75G of memory. This is usually very difficult for existing computing machines.
  • the present application aims to solve at least one of the technical problems in the related art to some extent.
  • the first object of the present application is to propose a computational method of logistic regression gradient, which enables the computing machine to support the training of the ultra-large-scale logistic regression model, improve the calculation speed, reduce the training time, and greatly reduce the calculation.
  • the amount of memory used by the machine is to propose a computational method of logistic regression gradient, which enables the computing machine to support the training of the ultra-large-scale logistic regression model, improve the calculation speed, reduce the training time, and greatly reduce the calculation.
  • a second object of the present application is to propose a computing device for a logical regression gradient.
  • the first aspect of the present application provides a method for calculating a logistic regression gradient, including the following steps: acquiring training data, where the training data includes X-line user data and Y-line click data, where The Y row click data corresponds to the X row user data; the X row user data is converted into X column data; the X column data and the weight vector are segmented to form N X column data segmentation blocks And weight vector segmentation; respectively, starting N threads to generate N sub-logical regression gradients according to the N X column data segmentation block and weight vector segmentation block and corresponding Y row click data; and The sub-logical regression gradients are spliced to form a full-scale logistic regression gradient.
  • the training data and the weight vector are divided into a plurality of small areas that can be stored in the memory, and the corresponding number of threads are started to perform parallel calculation on a small area, thereby enabling the computing machine to support the large size.
  • the training of the scale logistic regression model improves the calculation speed, reduces the training time, and greatly reduces the memory usage of the computing machine.
  • the second aspect of the present application provides a computing device for a logical regression gradient, comprising: an obtaining module, configured to acquire training data, wherein the training data includes X rows of user data and Y rows of click data.
  • the Y row click data corresponds to the X row user data;
  • the conversion module is configured to convert the X row user data into X column data;
  • the segmentation module is configured to use the X column data and the weight
  • the vector is segmented to form N X-column data slice blocks and weight vector slice blocks;
  • a generation module is configured to respectively start N threads to segment and block according to the N X column data segmentation blocks and weight vectors.
  • the Y rows click data to generate N sub-logical regression gradients;
  • a splicing module for splicing the N sub-logical regression gradients to form a full-scale logistic regression gradient.
  • the computing device of the embodiment of the present application divides the training data and the weight vector into a plurality of small areas that can be stored in the memory, and starts a corresponding number of threads to perform parallel calculation on a small area, thereby enabling the computing machine to support the large size.
  • the training of the scale logistic regression model improves the calculation speed, reduces the training time, and greatly reduces the memory usage of the computing machine.
  • FIG. 1 is a flow chart of a method for calculating a logistic regression gradient according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of dividing a weight vector W by a column segmentation device according to an embodiment of the present application
  • FIG. 3 is a flow chart of a method for calculating a logistic regression gradient according to another embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a computing device of a logical regression gradient according to an embodiment of the present application
  • FIG. 5 is a schematic structural diagram of a computing device of a logical regression gradient according to another embodiment of the present application.
  • first and second are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated.
  • features defining “first” and “second” may include one or more of the features either explicitly or implicitly.
  • the meaning of "a plurality” is two or more unless specifically and specifically defined otherwise.
  • logistic regression formula is:
  • each thread has a gradient vector g as large as the weight vector w, wherein the gradient of the i-th coordinate is:
  • s1 is the first training data and sn is the nth training data.
  • Each thread traverses all the training data belonging to the thread, resulting in a complete gradient vector g i . Then, the calculation results of all the threads are combined to obtain the final gradient vector g.
  • the current shortcoming is that when the size of the feature is large, the memory required by each computing device is the number of threads * weight vector w, resulting in huge memory consumption of the computing machine, a single computer can not be loaded, and the logic cannot be trained. Regression model. In addition, more CPUs cannot be used for calculations, because the more CPUs are used, the more memory the computing machine consumes, resulting in slower training and higher training costs.
  • the calculation method of the logistic regression gradient in the present application divides the weight space and the training data into a plurality of smaller regions that can be stored in the memory, and calculates the plurality of regions in parallel, thereby improving the calculation speed and reducing the calculation speed. Training time and greatly reduced the memory usage of the computing machine.
  • the weight vector and the training data are segmented by columns, and each thread calculates a part of the segmented weight column and the corresponding data column.
  • the original can only start a few threads at most, and now how many CPUs are used, how many threads can be started, which is not limited by the memory limit of the computing machine, thereby improving the CPU utilization of the computing machine. Therefore, in this way, the training of the ultra-large-scale logistic regression model can be completed, and the training of the logistic regression model is not limited to the memory and CPU limits of the computing machine, and more resources can be used to accelerate the training, thereby reducing the training. Training time and cost.
  • the calculation method of the logistic regression gradient in the present application not only can reduce the training cost of the logistic regression model, but also can realize the training of a very large-scale logistic regression model, thereby being able to more effectively describe the long tail requirement and satisfy the user's personality. Demand.
  • 1 is a flow chart of a method for calculating a logistic regression gradient of an embodiment of the present application.
  • the calculation method includes:
  • enter user data first in the following format:
  • the first behavior is the first user data X, wherein 1.3.5.8.9.10.23.22 represents different user characteristics in the first user data X;
  • the second behavior is second user data X, wherein 2.6.8.8.9.12. 34.56 represents different user characteristics in the second user data X;
  • Nth acts as the Nth user data, wherein 1.3.6.8.10.11.12 represents different user characteristics in the Nth user data.
  • the first behavior is the first click data Y; the second behavior is the second click data Y; and the Nth behavior is the Nth click data.
  • first user data X of the first row and the first click data Y of the first row are merged together to be the first complete training data;
  • X and the second click data Y of the second row are merged together to be the second complete training data;
  • the Nth user data X of the Nth row and the Nth click data Y of the Nth row are merged at Together, is the Nth complete training data.
  • X-line user data is converted into X-column data by a column data conversion device.
  • the X column data is segmented by a column sharding device to obtain corresponding N X column data dicing blocks.
  • N 3
  • the X column data is divided into three X column data segmentation blocks, and the three X column data segmentation blocks are:
  • the weight vector W is segmented by a column segmentation device to obtain a corresponding weight vector segmentation block.
  • W1 corresponds to an element with a subscript of 1-7 in W
  • W2 corresponds to an element with a subscript of 8-9 in W
  • W3 corresponds to an element of 10-56 in the following table.
  • the number of columns in the X-line user data is the same as the vector W width of the weight vector. That is to say, the number of columns of the X-line user data is as wide as the weight vector W. For example, if the length of the weight vector W is 100, then each row of user data in the X-line user data is The largest occurrence of the element is also 100, so each row of user data in the X-line user data is sparse, that is, each user data includes only a part of the features.
  • the former has the characteristics of Hunan and male, the latter has the characteristics of Beijing and women, corresponding to the two rows of user data in the X-line user data, the first line may be 1.7, while the second line may be 9.10.
  • the first line may be a click, and the second column Y may be a non-click.
  • N is the number of CPUs that can be called in the computing machine, wherein each CPU can call one thread separately.
  • each of the N threads in the started computing machine respectively obtains a weight vector segmentation block corresponding to the thread and a corresponding X column data segmentation block, for example, thread 1 corresponds to X1/W1, and thread 2 corresponds to X2/. W2, and thread 3 corresponds to X3/W3. Further, each thread can perform and calculate, and the corresponding logistic regression gradients g 1 , g 2 , and g 3 are respectively calculated according to the calculation formula of the gradient described above.
  • the calculated logistic regression gradients g 1 , g 2 and g 3 are combined, that is, the logistic regression gradients g 1 , g 2 and g 3 are summed, so that the full logical regression gradient g of the computing machine can be obtained. .
  • the training data and the weight vector are divided into a plurality of small areas that can be stored in the memory, and the corresponding number of threads are started to perform parallel calculation on a small area, thereby enabling the computing machine to support the large size.
  • the training of the scale logistic regression model improves the calculation speed, reduces the training time, and greatly reduces the memory usage of the computing machine.
  • FIG. 3 is a flow chart of a method for calculating a logistic regression gradient of another embodiment of the present application.
  • the calculation method includes:
  • enter user data first in the following format:
  • the first behavior is the first user data X, wherein 1.3.5.8.9.10.23.22 represents different user characteristics in the first user data X;
  • the second behavior is second user data X, wherein 2.6.8.8.9.12. 34.56 represents different user characteristics in the second user data X;
  • Nth acts as the Nth user data, wherein 1.3.6.8.10.11.12 represents different user characteristics in the Nth user data.
  • the first behavior is the first click data Y; the second behavior is the second click data Y; and the Nth behavior is the Nth click data.
  • first user data X of the first row and the first click data Y of the first row are merged together to be the first complete training data;
  • X and the second click data Y of the second row are merged together to be the second complete training data;
  • the Nth user data X of the Nth row and the Nth click data Y of the Nth row are merged at Together, is the Nth complete training data.
  • X-line user data is converted into X-column data by a column data conversion device.
  • the X column data is segmented by a column sharding device to obtain corresponding N X column data dicing blocks.
  • N 3
  • the X column data is divided into three X column data segmentation blocks, and the three X column data segmentation blocks are:
  • the weight vector W is segmented by a column segmentation device to obtain a corresponding weight vector segmentation block.
  • W1 corresponds to an element with a subscript of 1-7 in W
  • W2 corresponds to an element with a subscript of 8-9 in W
  • W3 corresponds to an element of 10-56 in the following table.
  • the number of columns in the X-line user data is the same as the vector W width of the weight vector. That is to say, the number of columns of the X-line user data is as wide as the weight vector W. For example, if the length of the weight vector W is 100, the largest appearing element in each row of user data in the X-line user data is also 100, so X Each row of user data in the row user data is sparse, that is, each user data includes only a portion of the features.
  • the former has the characteristics of Hunan and male, the latter has the characteristics of Beijing and women, corresponding to the two rows of user data in the X-line user data, the first line may be 1.7, while the second line may be 9.10.
  • the first line may be a click, and the second column Y may be a non-click.
  • N is the number of CPUs that can be called in the computing machine, wherein each CPU can call one thread separately.
  • each of the N threads in the started computing machine respectively obtains a weight vector segmentation block corresponding to the thread and a corresponding X column data segmentation block, for example, thread 1 corresponds to X1/W1, and thread 2 corresponds to X2/ W2, and thread 3 corresponds to X3/W3. Further, each thread can perform and calculate, and the corresponding logistic regression gradients g 1 , g 2 , and g 3 are respectively calculated according to the calculation formula of the gradient described above.
  • the calculated logistic regression gradients g 1 , g 2 and g 3 are combined, that is, the logistic regression gradients g 1 , g 2 and g 3 are summed, so that the full logical regression gradient g of the computing machine can be obtained. .
  • the weight vector is calculated according to the following formula:
  • Weight vector of the T+1th round weight vector of the T round W + step size * full logical regression gradient g,
  • the step size is a floating point number, for example, 0.01.
  • the training data and the weight vector are divided into a plurality of small areas that can be stored in the memory, and the corresponding number of threads are started to perform parallel calculation on a small area, and then each computer is calculated.
  • the full logical regression gradient is aggregated, which can further increase the calculation speed and reduce the training time.
  • the present application also proposes a computing device for a logical regression gradient.
  • the computing device includes: an obtaining module 100, a converting module 200, a sharding module 300, a generating module 400, and a splicing module 500.
  • the obtaining module 100 is configured to acquire training data, where the training data includes X rows.
  • the conversion module 200 is configured to convert X-line user data into X-column data.
  • the sharding module 300 is configured to slice the X column data and the weight vector to form N X column data dicing blocks and weight vector dicing blocks.
  • N is the number of CPUs that can be called in the computing machine, and the number of columns in the X-line user data is the same as the vector width of the weight vector.
  • the generating module 400 is configured to respectively start N threads to generate N sub-logical regression gradients according to the N X column data segmentation block and the weight vector segmentation block and the corresponding Y row click data.
  • the splicing module 500 is configured to splicing the N sub-logical regression gradients to form a full-scale logistic regression gradient.
  • the computing device of the embodiment of the present application divides the training data and the weight vector into a plurality of small areas that can be stored in the memory, and starts a corresponding number of threads to perform parallel calculation on a small area, thereby enabling the computing machine to support the large size.
  • the training of the scale logistic regression model improves the calculation speed, reduces the training time, and greatly reduces the memory usage of the computing machine.
  • the computing device includes: an obtaining module 100, a converting module 200, a sharding module 300, a generating module 400, and a splicing module 500.
  • the summary module 600 is configured to summarize the full-quantity logistic regression gradient calculated by each machine to generate a final full-quantity logistic regression gradient.
  • the update module 700 is configured to update the weight vector based on the final full-quantity logistic regression gradient.
  • the computing device of the embodiment of the present application divides the training data and the weight vector into a plurality of small areas that can be stored in the memory, and starts a corresponding number of threads to perform parallel calculation on a small area, and then calculates each computer.
  • the full logical regression gradient is collected and can be entered Improve calculation speed and reduce training time in one step.
  • portions of the application can be implemented in hardware, software, firmware, or a combination thereof.
  • multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Mathematical Physics (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Strategic Management (AREA)
  • Development Economics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Probability & Statistics with Applications (AREA)
  • Game Theory and Decision Science (AREA)
  • General Business, Economics & Management (AREA)
  • Operations Research (AREA)
  • Databases & Information Systems (AREA)
  • Algebra (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Complex Calculations (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种逻辑回归梯度的计算方法和装置。该计算方法包括：获取训练数据，其中，训练数据包括X行用户数据和Y行点击数据，其中，Y行点击数据与X行用户数据对应（S101）；将X行用户数据转换为X列数据（S102）；对X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块（S103）；分别启动N个线程根据N个X列数据切分块和权重向量切分块以及对应的Y行点击数据生成N个子逻辑回归梯度（S104）；以及将N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度（S105）。该计算方法使得计算机器可以支持超大规模逻辑回归模型的训练，提高计算速度，减少训练时间，并且极大地降低计算机器的内存使用量。

Description

逻辑回归梯度的计算方法和装置 技术领域
本申请涉及互联网技术领域，尤其涉及一种逻辑回归梯度的计算方法和装置。
背景技术
逻辑回归作为网络广告CTR(Click-Through-Rate，点击到达率)中最为常用的模型，在业界中被大规模使用。谁能在有限的时间里面训练出逻辑回归模型，就能更好地为用户提供合理的广告，提升用户的使用体验。
在训练逻辑回归模型的过程中，最为核心的部分是梯度的计算。现有的梯度的计算方法，是通过在计算机器上启动多线程的方式来进行加速，然而，每个线程均需要一份全量的梯度向量存储在内存中。也就是说，在一台计算机器上如果启动10个线程来计算梯度，假设1份梯度向量为10亿，均为double数据，那么1个线程需要消耗约7.5G的内存，一共需要消耗计算机器75G的内存。这对于现有的计算机器来说通常是很难承受的。
进一步而言，在互联网领域中，要想提高网络广告的CTR，就必须要使用更多的训练数据，更大的特征来刻画长尾，从而满足用户需求。例如，在训练逻辑回归模型时,考虑到用户的性别、年龄、目前所在地等细粒度的特征，能够更加有效刻画用户的需求，从而更加精准的满足用户个性化需求。因此，如此大的逻辑回归模型使用现有技术很难进行训练。
申请内容
本申请旨在至少在一定程度上解决相关技术中的技术问题之一。
为此，本申请的第一个目的在于提出一种逻辑回归梯度的计算方法，该计算方法使得计算机器可以支持超大规模逻辑回归模型的训练，提高计算速度，减少训练时间，并且极大地降低计算机器的内存使用量。
本申请的第二个目的在于提出一种逻辑回归梯度的计算装置。
为达上述目的，本申请第一方面实施例提出了一种逻辑回归梯度的计算方法，包括以下步骤：获取训练数据，其中，所述训练数据包括X行用户数据和Y行点击数据，其中，所述Y行点击数据与所述X行用户数据对应；将所述X行用户数据转换为X列数据；对所述X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块；分别启动N个线程根据所述N个X列数据切分块和权重向量切分块以及对应的所述Y行点击数据生成N个子逻辑回归梯度；以及将所述N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
本申请实施例的计算方法，将训练数据和权重向量切分成内存中可以存放下的多个小区域，并启动对应个数的线程对个小区域进行并行计算，由此使得计算机器可以支持超大规模逻辑回归模型的训练，提高计算速度，减少训练时间，并且极大地降低计算机器的内存使用量。
为达上述目的，本申请第二方面实施例提出了一种逻辑回归梯度的计算装置，包括：获取模块，用于获取训练数据，其中，所述训练数据包括X行用户数据和Y行点击数据，其中，所述Y行点击数据与所述X行用户数据对应；转换模块，用于将所述X行用户数据转换为X列数据；切分模块，用于对所述X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块；生成模块，用于分别启动N个线程根据所述N个X列数据切分块和权重向量切分块以及对应的所述Y行点击数据生成N个子逻辑回归梯度；拼接模块，用于将所述N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
本申请实施例的计算装置，将训练数据和权重向量切分成内存中可以存放下的多个小区域，并启动对应个数的线程对个小区域进行并行计算，由此使得计算机器可以支持超大规模逻辑回归模型的训练，提高计算速度，减少训练时间，并且极大地降低计算机器的内存使用量。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1是本申请一个实施例的逻辑回归梯度的计算方法的流程图；
图2是本申请一个实施例的通过列切分装置将权重向量W进行切分的示意图；
图3是本申请另一个实施例的逻辑回归梯度的计算方法的流程图；
图4是本申请一个实施例的逻辑回归梯度的计算装置的结构示意图；以及
图5是本申请另一个实施例的逻辑回归梯度的计算装置的结构示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本申请的描述中，“多个”的含义是两个或两个以上，除非另有明确具体的限定。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所 属技术领域的技术人员所理解。
具体而言，逻辑回归计算公式为：
P(y|x,w)＝1/(1+exp(-wx))，
其中，w为权重向量，x为训练数据。通过上述的公式来预测用户的点击概率。因此，在通过梯度下降算法来训练逻辑回归模型时，梯度的计算公式为：
1/(1+exp(ywx))*y*x，
其中，y为用户的点击结果，由于我们拿到的日志是用户已经操作后的，如果用户进行了点击，则y＝1；如果用户没有点击，则y＝-1。
目前，相关技术中是启动多个线程，每个线程均有一个跟权重向量w一样大的梯度向量g，其中，第i个坐标的梯度为：
Figure PCTCN2017071130-appb-000001
其中，s1为第一个训练数据，sn为第n个训练数据。每个线程均遍历属于该线程的所有的训练数据，从而得到一个完整的梯度向量gi。然后，合并所有线程的计算结果，得到最终的梯度向量g。
然而，目前存在的缺点是，当特征的规模很大时，每个计算机器需要的内存为线程个数*权重向量w，导致计算机器的内存消耗巨大，单个计算机器通常加载不了，无法训练逻辑回归模型。此外，无法利用更多的CPU进行计算，因为CPU用的越多，计算机器的内存消耗的就越大，导致训练速度慢，训练成本大。
然而，本申请中逻辑回归梯度的计算方法将权重空间和训练数据切分成内存中可以存放下的多个更小的区域，并将该多个区域并行计算，由此可以提高计算速度，减少了训练时间，并且极大地降低了计算机器的内存使用量。
进一步而言，在计算梯度时，将权重向量和训练数据按列进行切分，每个线程计算一部分切分后的权重列和对应的数据列。由此，每个线程只需要持有梯度的不同区域，因此计算梯度所消耗的计算机器的内存由原来 的N份(其中，计算机器使用N个CPU，就消耗N份计算机器的内存，例如，计算机器使用10个CPU来参与梯度计算，那么N＝10)变成1份，由此极大的降低了计算机器的内存消耗。而在计算速度上，原来最多只能启动几个线程，现在使用了多少个CPU，就可以启动多少个线程，不受限于计算机器的内存限制，由此提高了计算机器的CPU利用率。因此，通过这样的方式，可以完成超大规模逻辑回归模型的训练，同时训练逻辑回归模型时不受限于计算机器的内存和CPU的限制，可以使用更多的资源来加速训练，由此降低了训练的时间跟成本。也就是说，本申请中的逻辑回归梯度的计算方法，不仅可以降低逻辑回归模型的训练成本，还可以实现训练超大规模的逻辑回归模型，从而能够更加有效的刻画长尾需求，满足用户的个性化需求。
图1是本申请一个实施例的逻辑回归梯度的计算方法的流程图。
如图1所示，计算方法包括：
S101，获取训练数据，其中，训练数据包括X行用户数据和Y行点击数据，其中，Y行点击数据与X行用户数据对应。
例如，以以下格式先输入用户数据：
X:[1.3.5.8.9.10.23.22
2.4.6.8.9.12.34.56
…………………
1.3.6.8.10.11.12]，
其中，第一行为第一个用户数据X，其中1.3.5.8.9.10.23.22代表第一个用户数据X中的不同用户特征；第二行为第二个用户数据X，其中2.4.6.8.9.12.34.56代表第二个用户数据X中的不同用户特征；而第N行为第N个用户数据，其中1.3.6.8.10.11.12代表第N个用户数据中的不同用户特征。
进而，再以以下格式输入点击数据：
Y:[1.0.0.0
2.0.0.0
………
0.0.1.0],
其中，第一行为第一个点击数据Y；第二行为第二个点击数据Y；而第N行为第N个点击数据。
应当理解的是，上述第一行的第一个用户数据X和第一行的第一个点击数据Y合并在一起，才是第一个完整的训练数据；第二行的第二个用户数据X和第二行的第二个点击数据Y合并在一起，才是第二个完整的训练数据；而第N行的第N个用户数据X和第N行的第N个点击数据Y合并在一起，才是第N个完整的训练数据。
S102，将X行用户数据转换为X列数据。
具体地，通过列数据转换装置将X行用户数据转换为X列数据。
S103，对X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块。
具体地，通过列切分装置将X列数据进行切分以得到对应的N个X列数据切分块。例如，N＝3，即将X列数据切分为3个X列数据切分块，这三个X列数据切分块分别是：
Figure PCTCN2017071130-appb-000002
进而，如图2所示，通过列切分装置将权重向量W进行切分以得到对应的权重向量切分块。例如，W1对应W中下标为1-7的元素，W2对应W中下标为8-9的元素，而W3对应W中下表为10-56的元素。
在本发明的一个实施例中，X行用户数据中的列数与权重向量的向量W宽度相同。也就是说，X行用户数据的列数和权重向量W一样宽，例如，如果权重向量W的长度是100，那么X行用户数据中每行用户数据中 最大出现的元素也为100，因此X行用户数据中的每行用户数据都是稀疏的，也就是说每个用户数据只包括一部分的特征。例如，湖南的男性用户跟北京的女性用户，前者出现了湖南和男性的特征，后者出现了北京和女性的特征，对应到X行用户数据中的两行用户数据中，第一行可能是1.7，而第二行可能是9.10。X行用户数据中Y也有两列，第一列Y可能是点击，第二列Y可能是未点击。
S104，分别启动N个线程根据N个X列数据切分块和权重向量切分块以及对应的Y行点击数据生成N个子逻辑回归梯度。
在本发明的一个实施例中，N为计算机器中可调用的CPU的个数，其中，每个CPU可以分别调用一个线程。
具体地，启动的计算机器中N个线程中的每个线程分别得到该线程对应的权重向量切分块与对应的X列数据切分块，例如线程1对应X1/W1，线程2对应X2/W2，而线程3对应X3/W3。进而，每个线程可以进行并经计算，根据上述中记载的梯度的计算公式分别计算出对应的逻辑回归梯度g1,g2和g3。
S105，将N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
具体地，将计算得到的逻辑回归梯度g1,g2和g3进行合并，即对逻辑回归梯度g1,g2和g3进行求和，从而可以得到该计算机器的全量逻辑回归梯度g。
本申请实施例的计算方法，将训练数据和权重向量切分成内存中可以存放下的多个小区域，并启动对应个数的线程对个小区域进行并行计算，由此使得计算机器可以支持超大规模逻辑回归模型的训练，提高计算速度，减少训练时间，并且极大地降低计算机器的内存使用量。
图3是本申请另一个实施例的逻辑回归梯度的计算方法的流程图。
如图3所示，计算方法包括：
S301，获取训练数据，其中，训练数据包括X行用户数据和Y行点击数据，其中，Y行点击数据与X行用户数据对应。
例如，以以下格式先输入用户数据：
X:[1.3.5.8.9.10.23.22
2.4.6.8.9.12.34.56
…………………
1.3.6.8.10.11.12]，
其中，第一行为第一个用户数据X，其中1.3.5.8.9.10.23.22代表第一个用户数据X中的不同用户特征；第二行为第二个用户数据X，其中2.4.6.8.9.12.34.56代表第二个用户数据X中的不同用户特征；而第N行为第N个用户数据，其中1.3.6.8.10.11.12代表第N个用户数据中的不同用户特征。
进而，再以以下格式输入点击数据：
Y:[1.0.0.0
2.0.0.0
………
0.0.1.0],
其中，第一行为第一个点击数据Y；第二行为第二个点击数据Y；而第N行为第N个点击数据。
应当理解的是，上述第一行的第一个用户数据X和第一行的第一个点击数据Y合并在一起，才是第一个完整的训练数据；第二行的第二个用户数据X和第二行的第二个点击数据Y合并在一起，才是第二个完整的训练数据；而第N行的第N个用户数据X和第N行的第N个点击数据Y合并在一起，才是第N个完整的训练数据。
S302，将X行用户数据转换为X列数据。
具体地，通过列数据转换装置将X行用户数据转换为X列数据。
S303，对X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块。
具体地，通过列切分装置将X列数据进行切分以得到对应的N个X列数据切分块。例如，N＝3，即将X列数据切分为3个X列数据切分块，这三个X列数据切分块分别是：
Figure PCTCN2017071130-appb-000003
进而，如图2所示，通过列切分装置将权重向量W进行切分以得到对应的权重向量切分块。例如，W1对应W中下标为1-7的元素，W2对应W中下标为8-9的元素，而W3对应W中下表为10-56的元素。
在本发明的一个实施例中，X行用户数据中的列数与权重向量的向量W宽度相同。也就是说，X行用户数据的列数和权重向量W一样宽，例如，如果权重向量W的长度是100，那么X行用户数据中每行用户数据中最大出现的元素也为100，因此X行用户数据中的每行用户数据都是稀疏的，也就是说每个用户数据只包括一部分的特征。例如，湖南的男性用户跟北京的女性用户，前者出现了湖南和男性的特征，后者出现了北京和女性的特征，对应到X行用户数据中的两行用户数据中，第一行可能是1.7，而第二行可能是9.10。X行用户数据中Y也有两列，第一列Y可能是点击，第二列Y可能是未点击。
S304，分别启动N个线程根据N个X列数据切分块和权重向量切分块以及对应的Y行点击数据生成N个子逻辑回归梯度。
在本发明的一个实施例中，N为计算机器中可调用的CPU的个数，其中，每个CPU可以分别调用一个线程。
具体地，启动的计算机器中N个线程中的每个线程分别得到该线程对应的权重向量切分块与对应的X列数据切分块，例如线程1对应X1/W1， 线程2对应X2/W2，而线程3对应X3/W3。进而，每个线程可以进行并经计算，根据上述中记载的梯度的计算公式分别计算出对应的逻辑回归梯度g1,g2和g3。
S305，将N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
具体地，将计算得到的逻辑回归梯度g1,g2和g3进行合并，即对逻辑回归梯度g1,g2和g3进行求和，从而可以得到该计算机器的全量逻辑回归梯度g。
S306，将每个机器计算的全量逻辑回归梯度进行汇总以生成最终的全量逻辑回归梯度，其中，计算机器为多个。
具体地，当计算全量逻辑回归梯度的计算机器的个数为个时，可以汇集所有的计算机器计算得到的全量逻辑回归梯度，并将所有的计算机器计算得到的全量逻辑回归梯度进行合并，即将所有的计算机器计算得到的全量逻辑回归梯度加在一起。
S307，根据最终的全量逻辑回归梯度对权重向量进行更新。
具体地，根据以下公式计算权重向量：
第T+1轮的权重向量＝第T轮的权重向量W+步长*全量逻辑回归梯度g，
其中，步长为一个浮点数，例如0.01。
本申请实施例的计算方法，将训练数据和权重向量切分成内存中可以存放下的多个小区域，并启动对应个数的线程对个小区域进行并行计算，进而将每个计算机器计算得到的全量逻辑回归梯度进行汇集，由此可以进一步提高计算速度，减少训练时间。
为了实现上述实施例，本申请还提出一种逻辑回归梯度的计算装置。
图4是本申请一个实施例的逻辑回归梯度的计算装置的结构示意图，如图4所示，计算装置包括：获取模块100、转换模块200、切分模块300、生成模块400和拼接模块500。
其中，获取模块100用于获取训练数据，其中，训练数据包括X行用 户数据和Y行点击数据，其中，Y行点击数据与X行用户数据对应。
转换模块200用于将X行用户数据转换为X列数据。
切分模块300用于对X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块。其中，N为计算机器中可调用的CPU的个数，X行用户数据中的列数与权重向量的向量宽度相同。
生成模块400用于分别启动N个线程根据N个X列数据切分块和权重向量切分块以及对应的Y行点击数据生成N个子逻辑回归梯度。
拼接模块500用于将N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
需要说明的是，前述对逻辑回归梯度的计算方法的解释说明也适用于该实施例的逻辑回归梯度的计算装置，其实现原理类似，此处不再赘述。
本申请实施例的计算装置，将训练数据和权重向量切分成内存中可以存放下的多个小区域，并启动对应个数的线程对个小区域进行并行计算，由此使得计算机器可以支持超大规模逻辑回归模型的训练，提高计算速度，减少训练时间，并且极大地降低计算机器的内存使用量。
图5是本申请另一个实施例的逻辑回归梯度的计算装置的结构示意图，如图5所示，计算装置包括：获取模块100、转换模块200、切分模块300、生成模块400、拼接模块500、汇总模块600和更新模块700。
其中，汇总模块600用于将每个机器计算的全量逻辑回归梯度进行汇总以生成最终的全量逻辑回归梯度。
更新模块700用于根据最终的全量逻辑回归梯度对权重向量进行更新。
需要说明的是，前述对逻辑回归梯度的计算方法的解释说明也适用于该实施例的逻辑回归梯度的计算装置，其实现原理类似，此处不再赘述。
本申请实施例的计算装置，将训练数据和权重向量切分成内存中可以存放下的多个小区域，并启动对应个数的线程对个小区域进行并行计算，进而将每个计算机器计算得到的全量逻辑回归梯度进行汇集，由此可以进 一步提高计算速度，减少训练时间。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (10)

  1. 一种逻辑回归梯度的计算方法，其特征在于，包括以下步骤：
    获取训练数据，其中，所述训练数据包括X行用户数据和Y行点击数据，其中，所述Y行点击数据与所述X行用户数据对应；
    将所述X行用户数据转换为X列数据；
    对所述X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块；
    分别启动N个线程根据所述N个X列数据切分块和权重向量切分块以及对应的所述Y行点击数据生成N个子逻辑回归梯度；以及
    将所述N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
  2. 如权利要求1所述的逻辑回归梯度的计算方法，其特征在于，所述N为计算机器中可调用的CPU的个数。
  3. 如权利要求2所述的逻辑回归梯度的计算方法，其特征在于，所述计算机器为多个，所述方法还包括：
    将每个机器计算的全量逻辑回归梯度进行汇总以生成最终的全量逻辑回归梯度。
  4. 如权利要求3所述的逻辑回归梯度的计算方法，其特征在于，还包括：
    根据最终的全量逻辑回归梯度对所述权重向量进行更新。
  5. 如权利要求1所述的逻辑回归梯度的计算方法，其特征在于，所述X行用户数据中的列数与所述权重向量的向量宽度相同。
  6. 一种逻辑回归梯度的计算装置，其特征在于，包括：
    获取模块，用于获取训练数据，其中，所述训练数据包括X行用户数据和Y行点击数据，其中，所述Y行点击数据与所述X行用户数据对应；
    转换模块，用于将所述X行用户数据转换为X列数据；
    切分模块，用于对所述X列数据和权重向量进行切分以形成N个X列数据切分块和权重向量切分块；
    生成模块，用于分别启动N个线程根据所述N个X列数据切分块和权重向量切分块以及对应的所述Y行点击数据生成N个子逻辑回归梯度；以及
    拼接模块，用于将所述N个子逻辑回归梯度进行拼接以形成全量逻辑回归梯度。
  7. 如权利要求6所述的逻辑回归梯度的计算装置，其特征在于，所述N为计算机器中可调用的CPU的个数。
  8. 如权利要求7所述的逻辑回归梯度的计算装置，其特征在于，所述计算机器为多个，还包括：
    汇总模块，用于将每个机器计算的全量逻辑回归梯度进行汇总以生成最终的全量逻辑回归梯度。
  9. 如权利要求8所述的逻辑回归梯度的计算装置，其特征在于，还包括：
    更新模块，用于根据最终的全量逻辑回归梯度对所述权重向量进行更新。
  10. 如权利要求6所述的逻辑回归梯度的计算装置，其特征在于，所述X行用户数据中的列数与所述权重向量的向量宽度相同。
PCT/CN2017/071130 2016-01-22 2017-01-13 逻辑回归梯度的计算方法和装置 Ceased WO2017124974A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/043,043 US10970596B2 (en) 2016-01-22 2018-07-23 Logistic regression gradient calculation method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610045655.XA CN106997333B (zh) 2016-01-22 2016-01-22 逻辑回归梯度的计算方法和装置
CN201610045655.X 2016-01-22

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/043,043 Continuation US10970596B2 (en) 2016-01-22 2018-07-23 Logistic regression gradient calculation method and apparatus

Publications (1)

Publication Number Publication Date
WO2017124974A1 true WO2017124974A1 (zh) 2017-07-27

Family

ID=59361469

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/071130 Ceased WO2017124974A1 (zh) 2016-01-22 2017-01-13 逻辑回归梯度的计算方法和装置

Country Status (4)

Country Link
US (1) US10970596B2 (zh)
CN (1) CN106997333B (zh)
TW (1) TW201732644A (zh)
WO (1) WO2017124974A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090248595A1 (en) * 2008-03-31 2009-10-01 Yumao Lu Name verification using machine learning
CN103745225A (zh) * 2013-12-27 2014-04-23 北京集奥聚合网络技术有限公司 分布式ctr预测模型训练的方法和系统
CN104536983A (zh) * 2014-12-08 2015-04-22 北京掌阔技术有限公司 一种预测广告点击率的方法和装置
CN105045819A (zh) * 2015-06-26 2015-11-11 深圳市腾讯计算机系统有限公司 一种训练数据的模型训练方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2956800B2 (ja) * 1991-09-19 1999-10-04 株式会社日立製作所 連立一次方程式に関する計算装置
US20130054566A1 (en) * 2011-08-31 2013-02-28 Microsoft Corporation Acceleration of ranking algorithms using a graphics processing unit
CN103996088A (zh) * 2014-06-10 2014-08-20 苏州工业职业技术学院 基于多维特征组合逻辑回归的广告点击率预测方法
WO2017004866A1 (zh) * 2015-07-09 2017-01-12 华为技术有限公司 一种广告价值确定方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090248595A1 (en) * 2008-03-31 2009-10-01 Yumao Lu Name verification using machine learning
CN103745225A (zh) * 2013-12-27 2014-04-23 北京集奥聚合网络技术有限公司 分布式ctr预测模型训练的方法和系统
CN104536983A (zh) * 2014-12-08 2015-04-22 北京掌阔技术有限公司 一种预测广告点击率的方法和装置
CN105045819A (zh) * 2015-06-26 2015-11-11 深圳市腾讯计算机系统有限公司 一种训练数据的模型训练方法及装置

Also Published As

Publication number Publication date
CN106997333A (zh) 2017-08-01
CN106997333B (zh) 2020-07-28
US20180330191A1 (en) 2018-11-15
TW201732644A (zh) 2017-09-16
US10970596B2 (en) 2021-04-06

Similar Documents

Publication Publication Date Title
Qi et al. Not all noises are created equally: Diffusion noise selection and optimization
Peng et al. GLU3. 0: Fast GPU-based parallel sparse LU factorization for circuit simulation
US20200184366A1 (en) Scheduling task graph operations
WO2020163315A1 (en) Systems and methods for artificial intelligence with a flexible hardware processing framework
CN107657599A (zh) 基于混合粒度划分和动态负载分配的遥感图像融合系统并行实现方法
US11231961B2 (en) Scheduling operations
CN103632336B (zh) 基于负载分配的遥感影像cpu/gpu协同处理方法
WO2024104232A1 (zh) 用于训练神经网络的方法、装置、设备和存储介质
WO2021147567A1 (zh) 卷积运算方法及芯片
CN106844024B (zh) 一种自学习运行时间预测模型的gpu/cpu调度方法及系统
WO2017124974A1 (zh) 逻辑回归梯度的计算方法和装置
CN113538687B (zh) 基于vtk的有限元可视化方法、系统、装置及存储介质
CN106599898A (zh) 一种图像特征提取方法及系统
CN120706547A (zh) 智能计算中心云平台基于算力资源动态调整的推理加速方法及装置
CN113971428B (zh) 数据处理方法、系统、设备、程序产品及存储介质
CN119225814A (zh) 一种多线程并行计算效率优化系统
CN112733415B (zh) 一种薄壁弹性体边界的无网格处理方法、装置、终端设备及计算介质
CN112099737A (zh) 存储数据的方法、装置、设备和存储介质
WO2022227169A1 (zh) 图像分类方法、装置、电子设备及存储介质
CN121188009B (zh) 向分布式模型提供数据的方法、装置、设备和存储介质
WO2017028728A1 (zh) 点击到达率ctr的确定方法及装置
Vora et al. Hybrid Approach to Parallel Stochastic Gradient Descent
CN118397298B (zh) 基于混合池化的自注意力空间金字塔池化方法及相关组件
US12619548B2 (en) Apparatus and method for molecular dynamics simulation
US20250156331A1 (en) Apparatus and method for molecular dynamics simulation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17741005

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17741005

Country of ref document: EP

Kind code of ref document: A1