WO2019140577A1 - 大数据的计算方法及系统 - Google Patents

大数据的计算方法及系统 Download PDF

Info

Publication number
WO2019140577A1
WO2019140577A1 PCT/CN2018/073072 CN2018073072W WO2019140577A1 WO 2019140577 A1 WO2019140577 A1 WO 2019140577A1 CN 2018073072 W CN2018073072 W CN 2018073072W WO 2019140577 A1 WO2019140577 A1 WO 2019140577A1
Authority
WO
WIPO (PCT)
Prior art keywords
instruction
control terminal
calculation
partial
data
Prior art date
Application number
PCT/CN2018/073072
Other languages
English (en)
French (fr)
Inventor
衣佳鑫
张北江
Original Assignee
新联智慧信息技术（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 新联智慧信息技术（深圳）有限公司 filed Critical 新联智慧信息技术（深圳）有限公司
Priority to PCT/CN2018/073072 priority Critical patent/WO2019140577A1/zh
Publication of WO2019140577A1 publication Critical patent/WO2019140577A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a method and system for calculating big data.
  • McKinsey The earliest mention of the "big data” era is the world-renowned consulting firm McKinsey, McKinsey said: "Data has penetrated into every industry and business function field today and has become an important production factor. People are exploring and using massive data. A new wave of productivity growth and the wave of consumer surplus.” “Big data” has existed in the fields of physics, biology, environmental ecology, and military, finance, communications, etc., but because of the Internet in recent years. And the development of the information industry has attracted people's attention.
  • Big Data is often used to describe a large amount of unstructured and semi-structured data created by a company that spends too much time and money when downloaded to a relational database for calculation. Big data computing is often associated with cloud computing because real-time large dataset calculations require a framework like MapReduce to distribute work to dozens, hundreds, or even thousands of computers.
  • the existing cloud platform is slow to process big data, which cannot meet the requirements of users and affect the user experience.
  • the embodiment of the present invention provides a method and a system for calculating big data, and performs processing of big data by configuring terminals of the cloud platform to improve the processing speed of big data and improve user experience.
  • an embodiment of the present application provides a method for calculating big data, where the method includes the following steps:
  • the control terminal receives the data amount of the big data to be calculated and the calculation instruction
  • the control terminal splits the calculation instruction into the first calculation sub-instruction and the second calculation sub-instruction, and splits the storage address of the big data to be calculated into a plurality of partial storage addresses;
  • the control terminal acquires the load value of the computing terminal and the number of transmission hops
  • the control terminal distributes the plurality of partial storage addresses and the second calculation sub-instruction to the computing terminal according to the transmission hop count and the load value;
  • the computing terminal extracts part of the data corresponding to the part of the storage address, and executes the second calculation sub-instruction to obtain a partial result, and returns the partial result to the control terminal;
  • the control terminal obtains the final result of the big data to be calculated according to the first calculation sub-instruction of the partial result.
  • a computing system for big data comprising: a control terminal and a computing terminal; wherein
  • control terminal configured to receive a data amount of the big data to be calculated and a calculation instruction; split the calculation instruction into the first calculation sub-instruction and the second calculation sub-instruction, and split the storage address of the big data to be calculated into multiple a partial storage address; obtaining a load value of the computing terminal and a transmission hop count; distributing the plurality of partial storage addresses and the second calculation sub-instruction to the computing terminal according to the transmission hop count and the load value;
  • a computing terminal configured to extract part of data corresponding to the partial storage address, execute the second calculation sub-instruction to obtain a partial result, and return the partial result to the control terminal;
  • the control terminal is further configured to extract part of the data corresponding to the partial storage address, execute the second calculation sub-instruction on the partial data to obtain a partial result, and return the partial result to the control terminal.
  • an embodiment of the present application provides a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform the method as described in the first aspect.
  • an embodiment of the present application provides a computer program product, comprising: a non-transitory computer readable storage medium storing a computer program, the computer being operative to cause a computer to perform the method as described in the first aspect Methods.
  • the technical solution provided by the present application splits the big data to be calculated into multiple computing processing terminals to execute the second computing sub-instruction in parallel to obtain partial processing results, and the control terminal combines the partial processing results to obtain the final result.
  • the control terminal does not need to release the big data to be calculated, and only splits the storage address of the big data to be calculated to realize the splitting of the big data, so that the data transmission between the cloud platforms only needs The transmission of the corresponding storage address does not require transmission of large data and reduces the amount of transmission, so it has the advantages of short calculation time and improved user experience.
  • 1 is a schematic flow chart of a method for calculating big data
  • FIG. 2 is a schematic diagram of a method for calculating big data provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a hardware provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a network topology according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a computing system for big data provided by an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of another terminal according to an embodiment of the present application.
  • references to "an embodiment” herein mean that a particular feature, result, or characteristic described in connection with the embodiments can be included in at least one embodiment of the invention.
  • the appearances of the phrases in various places in the specification are not necessarily referring to the same embodiments, and are not exclusive or alternative embodiments that are mutually exclusive. Those skilled in the art will understand and implicitly understand that the embodiments described herein can be combined with other embodiments.
  • a mobile terminal also called a User Equipment (UE) is a device that provides voice and/or data connectivity to users, for example, a handheld device with an infinite connection function, an in-vehicle device, and the like.
  • Common terminals include, for example, mobile phones, tablets, laptops, PDAs, mobile internet devices (mobile Internet device, MID), wearable devices such as smart watches, smart bracelets, pedometers, etc.
  • control terminal is specifically configured to perform a cumulative calculation according to the partial result to obtain a final calculation result.
  • control terminal is specifically configured to perform a matrix multiplication matrix operation as the calculation instruction, and divide the matrix multiplication matrix operation into a multiplication operation and an accumulation operation, wherein the accumulation operation is a first operation sub instruction, and the multiplication operation is The second operator instruction.
  • control terminal is further configured to identify a data type of the final result, and determine whether to convert the data type according to the data type.
  • FIG. 1 is a method for calculating big data.
  • the method for calculating big data includes the following steps.
  • the method may be performed by a terminal, and the terminal may specifically be: a mobile phone, a tablet computer, or a notebook.
  • MID mobile internet device
  • Step S101 The terminal receives big data and calculates an instruction.
  • Step S102 The terminal extracts big data, and calculates the big data to obtain a calculation result
  • Step S103 The terminal returns the calculation result to the control terminal corresponding to the cloud platform.
  • FIG. 2 is a method for calculating big data. As shown in FIG. 2, the method is executed by any terminal in the cloud platform. The method is shown in the hardware architecture shown in FIG. The network topology is implemented. The method is as shown in FIG. 2, and includes the following steps:
  • Step S201 the control terminal receives the data amount of the big data to be calculated and the calculation instruction
  • Step S202 the control terminal splits the calculation instruction into the first calculation sub-instruction and the second calculation sub-instruction, and splits the storage address of the big data to be calculated into a plurality of partial storage addresses;
  • Step S203 the control terminal acquires the load value of the computing terminal and the number of transmission hops
  • Step S204 The control terminal distributes the plurality of partial storage addresses and the second calculation sub-instruction to the computing terminal according to the transmission hop count and the load value.
  • Step S205 The computing terminal extracts part of the data corresponding to the partial storage address, performs a second calculation sub-instruction on the partial data to obtain a partial result, and returns the partial result to the control terminal.
  • Step S206 The control terminal obtains a final result of the big data to be calculated according to the first calculation sub-instruction executed on the partial result.
  • the technical solution provided by the present application splits the big data to be calculated into a plurality of computing processing terminals to execute the second computing sub-instruction in parallel to obtain a partial processing result, and the control terminal combines the partial processing results to obtain a final result, and processes the same.
  • the control terminal does not need to deliver the big data to be calculated, and only splits the storage address of the big data to be calculated to realize the splitting of the big data, so that the data transmission between the cloud platforms only needs to transmit the corresponding storage.
  • the address does not need to transmit big data and reduces the amount of transmission, so it has the advantages of short calculation time and improved user experience.
  • the implementation method of the foregoing step S206 may be specifically:
  • the control terminal calculates the final calculation result according to the cumulative calculation of the partial result.
  • the implementation method of the foregoing step S202 may specifically be:
  • the matrix multiplication matrix operation is divided into a multiplication operation and an accumulation operation, wherein the accumulation operation is a first operation sub-instruction, and the multiplication operation is a second operation sub-instruction.
  • the foregoing method may further include:
  • FIG. 5 provides a computing system for big data, where the system includes: a control terminal 501 and a computing terminal 502;
  • control terminal configured to receive a data amount of the big data to be calculated and a calculation instruction; split the calculation instruction into the first calculation sub-instruction and the second calculation sub-instruction, and split the storage address of the big data to be calculated into multiple a partial storage address; obtaining a load value of the computing terminal and a transmission hop count; distributing the plurality of partial storage addresses and the second calculation sub-instruction to the computing terminal according to the transmission hop count and the load value;
  • a computing terminal configured to extract part of data corresponding to the partial storage address, execute the second calculation sub-instruction to obtain a partial result, and return the partial result to the control terminal;
  • the control terminal is further configured to extract part of the data corresponding to the partial storage address, execute the second calculation sub-instruction on the partial data to obtain a partial result, and return the partial result to the control terminal.
  • the technical solution provided by the present application splits the big data to be calculated into a parallel processing of a plurality of computing processing terminals to obtain a partial processing result, and the control terminal combines the partial processing results to obtain a final result, and the processor does not need to control the terminal.
  • the big data to be calculated is sent out, and only the storage address of the big data to be calculated can be split to realize the splitting of the big data, so that the data transmission between the cloud platforms only needs to transmit the corresponding storage address, and no large transmission is needed. Data, reducing the amount of transmission, so it has the advantage of short calculation time and improved user experience.
  • FIG. 6 is a block diagram showing a partial structure of a terminal provided by an embodiment of the present application.
  • the server includes: a radio frequency (RF) circuit 910, a memory 920, an input unit 930, a sensor 950, an audio circuit 960, and wireless fidelity (Wireless).
  • RF radio frequency
  • Fidelity, WiFi Fidelity, WiFi
  • application processor AP980 application processor AP980 and power supply 990 and other components.
  • FIG. 6 does not constitute a limitation to the smart device, and may include more or less components than those illustrated, or some components may be combined, or different component arrangements.
  • the input unit 930 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the smart device.
  • the input unit 930 may include a touch display screen 933, a stylus 931, and other input devices 932.
  • the input unit 930 can also include other input devices 932.
  • other input devices 932 may include, but are not limited to, one or more of physical buttons, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.
  • the AP 980 is a control center for smart devices that connects various portions of the entire smart device using various interfaces and lines, by executing or executing software programs and/or modules stored in the memory 920, and invoking data stored in the memory 920, executing The intelligent device's various functions and processing data, so that the smart device is monitored as a whole.
  • the AP 980 may include one or more processing units; optionally, the AP 980 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like, and the modulation solution The processor mainly handles wireless communication. It can be understood that the above modem processor may not be integrated into the AP 980.
  • the AP980 can be integrated with the face recognition module. Of course, in the actual application, the face recognition module can also be separately set or integrated in the camera 770. The face recognition module shown in FIG. 6 is integrated in the AP980. example.
  • memory 920 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • the RF circuit 910 can be used for receiving and transmitting information.
  • RF circuit 910 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low) Noise Amplifier, LNA), duplexer, etc.
  • RF circuitry 910 can also communicate with the network and other devices via wireless communication.
  • the above wireless communication may use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (General Packet) Radio Service, GPRS), Code Division Multiple Access (Code) Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), E-mail, Short Message Service (Short Messaging) Service, SMS), etc.
  • GSM Global System of Mobile communication
  • General Packet General Packet Radio Service
  • GPRS General Packet Radio Service
  • CDMA Code Division Multiple Access
  • WCDMA Wideband Code Division Multiple Access
  • LTE Long Term Evolution
  • SMS Short Message Service
  • the smart device may also include at least one type of sensor 950, such as a light sensor, a motion sensor, a proximity sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the touch display screen according to the brightness of the ambient light, and the proximity sensor can turn off the touch display when the mobile phone moves to the ear. And / or backlight.
  • the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity.
  • Proximity sensor can be used to detect the distance between the phone and the user.
  • Other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like that can be configured in the mobile phone are not described herein.
  • An audio circuit 960, a speaker 961, and a microphone 962 can provide an audio interface between the user and the smart device.
  • the audio circuit 960 can transmit the converted electrical data of the received audio data to the speaker 961 for conversion to the sound signal by the speaker 961; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal by the audio circuit 960. After receiving, it is converted into audio data, and then the audio data is played by the AP 980, sent to the other mobile phone via the RF circuit 910, or the audio data is played to the memory 920 for further processing.
  • WiFi is a short-range wireless transmission technology
  • the mobile phone can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 970, which provides users with wireless broadband Internet access.
  • FIG. 6 shows the WiFi module 970, it can be understood that it does not belong to the essential configuration of the smart device, and can be omitted as needed within the scope of not changing the essence of the application.
  • the smart device also includes a power supply 990 (such as a battery or a power module) that supplies power to various components.
  • a power supply 990 such as a battery or a power module
  • the power supply can be logically connected to the AP980 through a power management system to manage charging, discharging, and power management through the power management system.
  • the embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, the computer program causing the computer to perform some or all of the steps of any of the methods described in the foregoing method embodiments.
  • the computer includes a terminal device.
  • the embodiment of the present application further provides a computer program product, comprising: a non-transitory computer readable storage medium storing a computer program, the computer program being operative to cause a computer to perform the operations as recited in the foregoing method embodiments Part or all of the steps of either method.
  • the computer program product can be a software installation package, the computer including a terminal device.
  • the steps of the method or algorithm described in the embodiments of the present application may be implemented in a hardware manner, or may be implemented by a processor executing software instructions.
  • Software instructions can be composed of corresponding software modules, which can be stored in random access memory (Random) Access Memory, RAM), Flash, Read Only Memory (Read Only Memory, ROM), Erasable Programmable Read Only Memory (Erasable) Programmable ROM (EPROM), electrically erasable programmable read only memory (EEPROM), registers, hard disk, removable hard disk, compact disk read only (CD-ROM) or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor to enable the processor to read information from, and write information to, the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and the storage medium can be located in an ASIC.
  • the ASIC can be located in an access network device, a target network device, or a core network device.
  • the processor and the storage medium may also exist as discrete components in the access network device, the target network device, or the core network device.
  • the functions described in the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the processes or functions described in accordance with embodiments of the present application are generated in whole or in part.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center By wire (eg coaxial cable, fiber optic, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (eg infrared, wireless, microwave, etc.) to another website, computer, server or data center.
  • the computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media.
  • the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, digital video disc (Digital) Video Disc, DVD)), or semiconductor media (for example, solid state drive (Solid State Disk, SSD)).

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种大数据的计算方法及系统，所述方法包括如下步骤：控制终端接收待计算的大数据的数据量以及计算指令（S201）；控制终端将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址（S202）；控制终端获取计算终端的负载值以及传输跳数（S203）；控制终端依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端（S204）；计算终端提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端（S205）；控制终端依据对该部分结果执行第一计算子指令得到该待计算的大数据的最终结果（S206）。该方法具有用户体验度高的优点。

Description

大数据的计算方法及系统 技术领域
本发明涉及通信技术领域，尤其涉及一种大数据的计算方法及系统。
 背景技术
最早提出“大数据”时代到来的是全球知名咨询公司麦肯锡，麦肯锡称：“数据，已经渗透到当今每一个行业和业务职能领域，成为重要的生产因素。人们对于海量数据的挖掘和运用，预示着新一波生产率增长和消费者盈余浪潮的到来。” “大数据”在物理学、生物学、环境生态学等领域以及军事、金融、通讯等行业存在已有时日，却因为近年来互联网和信息行业的发展而引起人们关注。
随着云时代的来临，大数据（Big data）也吸引了越来越多的关注。大数据（Big data）通常用来形容一个公司创造的大量非结构化和半结构化数据，这些数据在下载到关系型数据库用于计算时会花费过多时间和金钱。大数据计算常和云计算联系到一起，因为实时的大型数据集计算需要像MapReduce一样的框架来向数十、数百或甚至数千的电脑分配工作。
在现今的社会，大数据的应用越来越彰显他的优势，它占领的领域也越来越大，电子商务、O2O、物流配送等，各种利用大数据进行发展的领域正在协助企业不断地发展新业务，创新运营模式。有了大数据这个概念，对于消费者行为的判断，产品销售量的预测，精确的营销范围以及存货的补给已经得到全面的改善与优化。
现有云平台处理大数据的速度慢，无法满足用户的要求，影响用户体验度。
 技术问题
本申请实施例提供一种大数据的计算方法及系统，通过对云平台的终端进行调配执行大数据的处理，提高大数据的处理速度，提高用户体验度。
 技术解决方案
第一方面，本申请实施例提供一种大数据的计算方法，所述方法包括如下步骤：
控制终端接收待计算的大数据的数据量以及计算指令；
控制终端将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址；
控制终端获取计算终端的负载值以及传输跳数；
控制终端依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端；
计算终端提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端；
控制终端依据对该部分结果执行第一计算子指令得到该待计算的大数据的最终结果。
第二方面，提供一种大数据的计算系统，所述系统包括：控制终端和计算终端；其中，
控制终端，用于接收待计算的大数据的数据量以及计算指令；将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址；获取计算终端的负载值以及传输跳数；依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端；
计算终端，用于提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端；
控制终端，还用于提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端。
第三方面，本申请实施例提供一种计算机可读存储介质，其存储用于电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如第一方面所述的方法。
第四方面，本申请实施例提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机可操作来使计算机执行如第一方面所述的方法。
采用本申请实施例，具有以下有益效果：
 有益效果
可以看出，本申请提供的技术方案将待计算的大数据进行拆分给多个计算处理终端并行的执行第二计算子指令得到部分处理结果，控制终端将该部分处理结果组合得到最终的结果，并且其处理器，控制终端不用下发待计算的大数据，仅仅拆分待计算的大数据的存储地址即可实现对大数据的拆分，这样云平台之间的数据的传输仅仅只需传输对应的存储地址，无需传输大数据，减少传输量，所以其具有计算时间短，提高用户体验度的优点。
 附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是一种大数据的计算方法的流程示意图；
图2是本申请实施例提供的大数据的计算方法的示意图；
图3是本申请实施例提供的一种硬件构架示意图；
图4是本申请实施例提供的一种网络拓扑的结构示意图；
图5是本申请实施例提供的一种大数据的计算系统示意图；
图6是本申请实施例提供的另一种终端的结构示意图。
 本发明的实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的说明书和权利要求书及所述附图中的术语“第一”、“第二”、“第三”和“第四”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结果或特性可以包含在本发明的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
以下，对本申请中的部分用语进行解释说明，以便于本领域技术人员理解。
移动终端，又称之为用户设备（User Equipment，UE），是一种向用户提供语音和/或数据连通性的设备，例如，具有无限连接功能的手持式设备、车载设备等。常见的终端例如包括：手机、平板电脑、笔记本电脑、掌上电脑、移动互联网设备（mobile internet device，MID）、可穿戴设备，例如智能手表、智能手环、计步器等。
可选的，所述控制终端，具体用于依据将该部分结果执行累加计算得到最终的计算结果。
可选的，所述控制终端，具体用于如该计算指令为矩阵乘矩阵运算，将该矩阵乘矩阵运算拆分成乘法运算和累加运算，其中累加运算为第一运算子指令，乘法运算为第二运算子指令。
可选的，所述控制终端，还用于识别该最终结果的数据类型，依据该数据类型确定是否转换该数据类型。
参阅图1，图1为一种大数据的计算方法，如图1所示，该大数据的计算方法包括如下步骤，该方法可以由终端执行，该终端具体可以为：手机、平板电脑、笔记本电脑、掌上电脑、移动互联网设备（mobile internet device，MID）、可穿戴设备中的一种或多种。
步骤S101、终端接收大数据以及计算指令；
步骤S102、终端提取大数据，对该大数据进行计算得到计算结果；
步骤S103、终端将计算结果返回给云平台对应的控制终端。
参阅图2，图2为一种大数据的计算方法，如图2所示，该方法由云平台内的任意一个终端执行，该方法在如图3所示的硬件构架和如图4所示的网络拓扑结构内实现，该方法如图2所示，包括如下步骤：
步骤S201、控制终端接收待计算的大数据的数据量以及计算指令；
步骤S202、控制终端将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址；
步骤S203、控制终端获取计算终端的负载值以及传输跳数；
步骤S204、控制终端依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端。
步骤S205、计算终端提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端。
步骤S206、控制终端依据对该部分结果执行第一计算子指令得到该待计算的大数据的最终结果。
本申请提供的技术方案将待计算的大数据进行拆分给多个计算处理终端并行的执行第二计算子指令得到部分处理结果，控制终端将该部分处理结果组合得到最终的结果，并且其处理器，控制终端不用下发待计算的大数据，仅仅拆分待计算的大数据的存储地址即可实现对大数据的拆分，这样云平台之间的数据的传输仅仅只需传输对应的存储地址，无需传输大数据，减少传输量，所以其具有计算时间短，提高用户体验度的优点。
可选的，上述步骤S206的实现方法具体可以为：
控制终端依据将该部分结果执行累加计算得到最终的计算结果。
可选的，上述步骤S202的实现方法具体可以为：
如该计算指令为矩阵乘矩阵运算，将该矩阵乘矩阵运算拆分成乘法运算和累加运算，其中累加运算为第一运算子指令，乘法运算为第二运算子指令。
可选的，上述方法还可以包括：
识别该最终结果的数据类型，依据该数据类型确定是否转换该数据类型。
参阅图5，图5提供一种大数据的计算系统，所述系统包括：控制终端501和计算终端502；其中，
控制终端，用于接收待计算的大数据的数据量以及计算指令；将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址；获取计算终端的负载值以及传输跳数；依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端；
计算终端，用于提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端；
控制终端，还用于提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端。
本申请提供的技术方案将待计算的大数据进行拆分给多个计算处理终端并行的处理得到部分处理结果，控制终端将该部分处理结果组合得到最终的结果，并且其处理器，控制终端不用下发待计算的大数据，仅仅拆分待计算的大数据的存储地址即可实现对大数据的拆分，这样云平台之间的数据的传输仅仅只需传输对应的存储地址，无需传输大数据，减少传输量，所以其具有计算时间短，提高用户体验度的优点。
图6示出的是与本申请实施例提供的终端的部分结构的框图。参考图6，服务器包括：射频（Radio Frequency，RF）电路910、存储器920、输入单元930、传感器950、音频电路960、无线保真（Wireless Fidelity，WiFi）模块970、应用处理器AP980以及电源990等部件。本领域技术人员可以理解，图6中示出的智能设备结构并不构成对智能设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图6对智能设备的各个构成部件进行具体的介绍：
输入单元930可用于接收输入的数字或字符信息，以及产生与智能设备的用户设置以及功能控制有关的键信号输入。具体地，输入单元930可包括触控显示屏933、手写笔931以及其他输入设备932。输入单元930还可以包括其他输入设备932。具体地，其他输入设备932可以包括但不限于物理按键、功能键（比如音量控制按键、开关按键等）、轨迹球、鼠标、操作杆等中的一种或多种。
AP980是智能设备的控制中心，利用各种接口和线路连接整个智能设备的各个部分，通过运行或执行存储在存储器920内的软件程序和/或模块，以及调用存储在存储器920内的数据，执行智能设备的各种功能和处理数据，从而对智能设备进行整体监控。可选的，AP980可包括一个或多个处理单元；可选的，AP980可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到AP980中。上述AP980可以集成人脸识别模组，当然在实际应用中，上述人脸识别模组也可以单独设置或集成在摄像头770内，如图6所示的人脸识别模组以集成在AP980内为例。
此外，存储器920可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
RF电路910可用于信息的接收和发送。通常，RF电路910包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器（Low Noise Amplifier，LNA）、双工器等。此外，RF电路910还可以通过无线通信与网络和其他设备通信。上述无线通信可以使用任一通信标准或协议，包括但不限于全球移动通讯系统 （Global System of Mobile communication，GSM）、通用分组无线服务（General Packet Radio Service，GPRS）、码分多址（Code Division Multiple Access，CDMA）、宽带码分多址（Wideband Code Division Multiple Access, WCDMA）、长期演进 （Long Term Evolution，LTE）、电子邮件、短消息服务（Short Messaging Service，SMS）等。
智能设备还可包括至少一种传感器950，比如光传感器、运动传感器、近距离传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节触控显示屏的亮度，接近传感器可在手机移动到耳边时，关闭触控显示屏和/或背光。作为运动传感器的一种，加速计传感器可检测各个方向上（一般为三轴）加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用（比如横竖屏切换、相关游戏、磁力计姿态校准）、振动识别相关功能（比如计步器、敲击）等; 近距离传感器可以用于检测手机与用户之间距离。至于手机还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不再赘述。
音频电路960、扬声器961，传声器962可提供用户与智能设备之间的音频接口。音频电路960可将接收到的音频数据转换后的电信号，传输到扬声器961，由扬声器961转换为声音信号播放；另一方面，传声器962将收集的声音信号转换为电信号，由音频电路960接收后转换为音频数据，再将音频数据播放AP980处理后，经RF电路910以发送给比如另一手机，或者将音频数据播放至存储器920以便进一步处理。
WiFi属于短距离无线传输技术，手机通过WiFi模块970可以帮助用户收发电子邮件、浏览网页和访问流式媒体等，它为用户提供了无线的宽带互联网访问。虽然图6示出了WiFi模块970，但是可以理解的是，其并不属于智能设备的必须构成，完全可以根据需要在不改变申请的本质的范围内而省略。
智能设备还包括给各个部件供电的电源990（比如电池或电源模块），可选 的，电源可以通过电源管理系统与AP980逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。
本申请实施例还提供一种计算机存储介质，其中，该计算机存储介质存储用于电子数据交换的计算机程序，该计算机程序使得计算机执行如上述方法实施例中记载的任一方法的部分或全部步骤，所述计算机包括终端设备。
本申请实施例还提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机执行如上述方法实施例中记载的任一方法的部分或全部步骤。该计算机程序产品可以为一个软件安装包，所述计算机包括终端设备。
本申请实施例所描述的方法或者算法的步骤可以以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器（Random Access Memory，RAM）、闪存、只读存储器（Read Only Memory，ROM）、可擦除可编程只读存储器（Erasable Programmable ROM，EPROM）、电可擦可编程只读存储器（Electrically EPROM，EEPROM）、寄存器、硬盘、移动硬盘、只读光盘（CD-ROM）或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于接入网设备、目标网络设备或核心网设备中。当然，处理器和存储介质也可以作为分立组件存在于接入网设备、目标网络设备或核心网设备中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请实施例所描述的功能可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线（例如同轴电缆、光纤、数字用户线（Digital Subscriber Line，DSL））或无线（例如红外、无线、微波等）方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质（例如，软盘、硬盘、磁带）、光介质（例如，数字视频光盘（Digital Video Disc，DVD））、或者半导体介质（例如，固态硬盘（Solid State Disk，SSD））等。
以上所述的具体实施方式，对本申请实施例的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本申请实施例的具体实施方式而已，并不用于限定本申请实施例的保护范围，凡在本申请实施例的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本申请实施例的保护范围之内。

Claims (10)

  1. 一种大数据的计算方法，其特征在于，所述方法包括如下步骤：
    控制终端接收待计算的大数据的数据量以及计算指令；
    控制终端将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址；
    控制终端获取计算终端的负载值以及传输跳数；
    控制终端依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端；
    计算终端提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端；
    控制终端依据对该部分结果执行第一计算子指令得到该待计算的大数据的最终结果。
  2. 根据权利要求1所述的方法，其特征在于，所述控制终端依据对该部分结果执行第一计算子指令得到该待计算的大数据的最终结果，包括：
    控制终端依据将该部分结果执行累加计算得到最终的计算结果。
  3. 根据权利要求1所述的方法，其特征在于，所述控制终端将计算指令拆分成第一计算子指令和第二计算子指令，包括：
    如该计算指令为矩阵乘矩阵运算，将该矩阵乘矩阵运算拆分成乘法运算和累加运算，其中累加运算为第一运算子指令，乘法运算为第二运算子指令。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    识别该最终结果的数据类型，依据该数据类型确定是否转换该数据类型。
  5. 一种大数据的计算系统，其特征在于，所述系统包括：控制终端和计算终端；其中，
    控制终端，用于接收待计算的大数据的数据量以及计算指令；将计算指令拆分成第一计算子指令和第二计算子指令，将该待计算的大数据的存储地址拆分成多个部分存储地址；获取计算终端的负载值以及传输跳数；依据该传输跳数以及负载值将该多个部分存储地址以及第二计算子指令分发给计算终端；
    计算终端，用于提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端；
    控制终端，还用于提取该部分存储地址对应的部分数据，将该部分数据执行第二计算子指令得到部分结果，将该部分结果返回给控制终端。
  6. 根据权利要求5所述的系统，其特征在于，
    所述控制终端，具体用于依据将该部分结果执行累加计算得到最终的计算结果。
  7. 根据权利要求5所述的系统，其特征在于，
    所述控制终端，具体用于如该计算指令为矩阵乘矩阵运算，将该矩阵乘矩阵运算拆分成乘法运算和累加运算，其中累加运算为第一运算子指令，乘法运算为第二运算子指令。
  8. 根据权利要求5所述的系统，其特征在于，
    所述控制终端，还用于识别该最终结果的数据类型，依据该数据类型确定是否转换该数据类型。
  9. 一种计算机可读存储介质，其特征在于，其存储用于电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如权利要求1-4任一项所述的方法。
  10. 一种计算机程序产品，其特征在于，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序可操作来使计算机执行如权利要求1-4任一项所述的方法。
PCT/CN2018/073072 2018-01-17 2018-01-17 大数据的计算方法及系统 WO2019140577A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/073072 WO2019140577A1 (zh) 2018-01-17 2018-01-17 大数据的计算方法及系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/073072 WO2019140577A1 (zh) 2018-01-17 2018-01-17 大数据的计算方法及系统

Publications (1)

Publication Number Publication Date
WO2019140577A1 true WO2019140577A1 (zh) 2019-07-25

Family

ID=67300918

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/073072 WO2019140577A1 (zh) 2018-01-17 2018-01-17 大数据的计算方法及系统

Country Status (1)

Country Link
WO (1) WO2019140577A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104915247A (zh) * 2015-04-29 2015-09-16 上海瀚银信息技术有限公司 一种实时数据计算方法及系统
US20170046420A1 (en) * 2015-08-10 2017-02-16 Salesforce.Com, Inc. Systems and methods of improving parallel functional processing
CN106686117A (zh) * 2017-01-20 2017-05-17 郑州云海信息技术有限公司 一种分布式计算集群的数据存储处理系统及方法
CN106790693A (zh) * 2017-02-20 2017-05-31 北京孚耐尔科技有限公司 一种网络数据包的分析方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104915247A (zh) * 2015-04-29 2015-09-16 上海瀚银信息技术有限公司 一种实时数据计算方法及系统
US20170046420A1 (en) * 2015-08-10 2017-02-16 Salesforce.Com, Inc. Systems and methods of improving parallel functional processing
CN106686117A (zh) * 2017-01-20 2017-05-17 郑州云海信息技术有限公司 一种分布式计算集群的数据存储处理系统及方法
CN106790693A (zh) * 2017-02-20 2017-05-31 北京孚耐尔科技有限公司 一种网络数据包的分析方法及系统

Similar Documents

Publication Publication Date Title
CN110582017B (zh) 一种视频播放方法、装置、终端及存储介质
US10959124B2 (en) Uplink data transmission method, terminal, network side device and system
CN110069715B (zh) 一种信息推荐模型训练的方法、信息推荐的方法及装置
CN107204989B (zh) 广告拦截方法、终端、服务器和存储介质
WO2018219119A1 (zh) 应用功能的控制方法及相关产品
CN106776036A (zh) 一种资源分配方法及装置
CN111050370A (zh) 网络切换方法、装置、存储介质及电子设备
CN106095489B (zh) 一种应用控制方法及移动终端
WO2019041280A1 (zh) 应用资源推荐方法及相关设备
CN107992498B (zh) 一种将数据导入数据仓库的方法及系统
CN113034189A (zh) 归因方法、装置、计算机设备和存储介质
JP2022506052A (ja) モデルファイル管理方法および端末デバイス
CN106980554B (zh) 一种进程处理方法及装置
WO2019140567A1 (zh) 大数据的分析方法及系统
WO2019072179A1 (zh) 应用程序运行控制方法及装置
CN111359210B (zh) 一种数据处理方法、装置、电子设备以及存储介质
CN108279978A (zh) 大数据的计算方法及系统
WO2017206851A1 (zh) 安装任务的分配方法及移动终端
CN116596202A (zh) 工单处理方法、相关装置及存储介质
WO2020052354A1 (zh) 游戏加载方法及相关产品
CN110019648B (zh) 一种训练数据的方法、装置及存储介质
WO2019140565A1 (zh) 云存储命令管理方法及系统
WO2019140577A1 (zh) 大数据的计算方法及系统
CN115589432A (zh) 消息推送管理方法、装置、介质及设备
CN111262737B (zh) 一种端口配置管理方法、装置、存储介质及终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18901670

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.11.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18901670

Country of ref document: EP

Kind code of ref document: A1