CN113747186B - Data processing method, device, terminal and storage medium - Google Patents

Data processing method, device, terminal and storage medium Download PDF

Info

Publication number
CN113747186B
CN113747186B CN202110963270.2A CN202110963270A CN113747186B CN 113747186 B CN113747186 B CN 113747186B CN 202110963270 A CN202110963270 A CN 202110963270A CN 113747186 B CN113747186 B CN 113747186B
Authority
CN
China
Prior art keywords
server
client
target
live broadcast
edge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110963270.2A
Other languages
Chinese (zh)
Other versions
CN113747186A (en
Inventor
陆瀛海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN202110963270.2A priority Critical patent/CN113747186B/en
Publication of CN113747186A publication Critical patent/CN113747186A/en
Application granted granted Critical
Publication of CN113747186B publication Critical patent/CN113747186B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004—Server selection for load balancing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/52—Network services specially adapted for the location of the user terminal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23103—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion using load balancing strategies, e.g. by placing or distributing content on different disks, different memories or different servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2405—Monitoring of the internal components or processes of the server, e.g. server load
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention provides a data processing method, a device, a terminal and a storage medium, wherein the method comprises the following steps: the method comprises the steps that a live broadcast server obtains first state information of a client and second state information of each edge server, determines a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and uploads a log file to the corresponding target server through the client; the target server receives the log file sent by the client, determines a target log file according to the log extraction instruction after receiving the log extraction instruction sent by the live broadcast server, and sends the target log file to the live broadcast server; the method has the advantages that the uplink bandwidth of the live broadcast server can be effectively saved, the load of the live broadcast server is reduced, meanwhile, the live broadcast server obtains the needed log file by sending the log extraction instruction, and the calculation force can be saved; therefore, log file collection under live broadcast equal-altitude concurrency and large-broadband scenes can be realized.

Description

Data processing method, device, terminal and storage medium
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a data processing method, device, terminal, and storage medium.
Background
With the development of internet technology, network live broadcast gradually becomes one of internet propagation modes. In live broadcast scenes, full link tracking needs to be built. When a user enters a live broadcasting room to watch live broadcasting, or sends chat messages in the live broadcasting room, or shares a live broadcasting room link, a corresponding client of the user sends a corresponding ping back log to a server. Because the number of people watching the live broadcast room is large, the number of times of entering and exiting the live broadcast room at the same time and the interaction of chat messages are large, a large number of Pingback logs are sent back to the server instantaneously, and a large amount of bandwidth is occupied.
In the prior art, in order to ensure that a live stream has enough bandwidth, the cloud control client typically stops sending the Pingback log to save the bandwidth. This can render the associated functions of the Pingback system unusable.
Disclosure of Invention
The invention provides a data processing method, a data processing device, a terminal and a computer readable storage medium, so as to solve the problem of insufficient bandwidth caused by instantaneous high concurrency of a Pingback log in the prior art to a certain extent.
According to a first aspect of the present invention, there is provided a data processing method for use in a data processing system comprising a live server and a plurality of edge servers, the live server being communicatively connected to each of the edge servers, the method comprising:
The live broadcast server acquires first state information of a client and second state information of each edge server;
the live broadcast server determines a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and informs the client to upload a log file to the corresponding target server;
the target server receives the log file sent by the client;
and after receiving the log extraction instruction sent by the live broadcast server, the target server determines a target log file according to the log extraction instruction and sends the target log file to the live broadcast server.
Optionally, the first status information includes first service network information of the client and first location information of the client; the second state information comprises second service network information of the edge server and second position information of the edge server; the live broadcast server determines a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and comprises the following steps:
The live broadcast server determines the edge server with the same second service network information as the first service network information as a candidate server;
the live broadcast server determines the geographic distance between the client and each candidate server according to the first position information and the second position information of each candidate server;
and the live broadcast server determines a target server corresponding to the client from the candidate servers of which the geographic distance meets the preset distance condition.
Optionally, the live broadcast server determines a target server corresponding to the client from candidate servers with the distance meeting a preset distance condition, and further includes:
the live broadcast server obtains the load value of each candidate server;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the load value meets a preset load condition and the geographic distance meets a preset distance condition.
Optionally, the live broadcast server determines a target server corresponding to the client from candidate servers whose geographic distance meets a preset distance condition, and further includes:
The live broadcast server acquires the historical online time and the historical offline time of each candidate server;
the live broadcast server determines the stable value of each candidate server according to the historical online time and the historical offline time;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the stable value meets a preset stable condition and the geographic distance meets a preset distance condition.
Optionally, after receiving the log extraction instruction sent by the live broadcast server, the target server determines a target log file according to the log extraction instruction, and sends the target log file to the live broadcast server, including:
the target server receives a log extraction instruction sent by the live broadcast server, wherein the log extraction instruction comprises a target client identifier;
and the target server determines a target log file from the log files according to the target client identifier, and sends the target log file to the live broadcast server.
Optionally, after the target server receives the log file sent by the client, the method further includes:
And the target server compresses the log file.
According to a second aspect of the present invention, there is provided a data processing method, the method comprising:
acquiring first state information of a client and second state information of each edge server;
determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and informing the client to upload a log file to the corresponding target server;
determining an edge server for storing the log file to be extracted, and sending a log extraction instruction to the determined edge server;
and receiving a response message of the log extraction instruction.
Optionally, the first status information includes first service network information of the client and first location information of the client; the second state information comprises second service network information of the edge server and second position information of the edge server; the live broadcast server determines a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and comprises the following steps:
The live broadcast server determines the edge server with the same second service network information as the first service network information as a candidate server;
the live broadcast server determines the distance between the client and each candidate server according to the first position information and the second position information of each candidate server;
and the live broadcast server determines a target server corresponding to the client from the candidate servers with the distances meeting the preset distance condition.
Optionally, the live broadcast server determines a target server corresponding to the client from candidate servers with the distance meeting a preset distance condition, and further includes:
the live broadcast server obtains the load value of each candidate server;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the load value meets a preset load condition and the distance meets a preset distance condition.
Optionally, the live broadcast server determines a target server corresponding to the client from candidate servers with the distance meeting a preset distance condition, and further includes:
the live broadcast server acquires the historical online time and the historical offline time of each candidate server;
The live broadcast server determines the stable value of each candidate server according to the historical online time and the historical offline time;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the stable value meets a preset stable condition and the distance meets a preset distance condition.
Optionally, the method further comprises:
and setting corresponding reporting rules for each edge server, so that the edge servers send log files according to specific rules.
Optionally, the method further comprises:
and receiving the log file sent by the edge server according to the reporting rule.
According to a third aspect of the present invention, there is provided a data processing apparatus, the apparatus comprising:
the state information acquisition module is used for acquiring first state information of the client and second state information of each edge server;
the target server determining module is used for determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and notifying the client to upload the log file to the corresponding target server;
The extraction instruction sending module is used for determining an edge server for storing the log file to be extracted and sending a log extraction instruction to the determined edge server;
and the response message receiving module is used for receiving the response message of the log extraction instruction.
Optionally, the first status information includes first service network information of the client and first location information of the client; the second state information comprises second service network information of the edge server and second position information of the edge server; the target server determining module includes:
a first determining module, configured to determine, as a candidate server, an edge server in which the second service network information is the same as the first service network information;
a second determining module, configured to determine a distance between the client and each candidate server according to the first location information and second location information of each candidate server;
and the target determining module is used for determining a target server corresponding to the client from the candidate servers of which the distances meet the preset distance condition.
Optionally, the target determining module further includes:
The load value determining module is used for obtaining the load value of each candidate server;
and the third determining module is used for determining a target server corresponding to the client from candidate servers of which the load value meets a preset load condition and the distance meets a preset distance condition.
Optionally, the target determining module further includes:
the online and offline time acquisition module is used for acquiring the historical online time and the historical offline time of each candidate server;
the stable value determining module is used for determining the stable value of each candidate server according to the historical online time and the historical offline time;
and the fourth determining module is used for determining a target server corresponding to the client from candidate servers of which the stable value meets a preset stable condition and the distance meets a preset distance condition.
Optionally, the method further comprises:
and the reporting rule setting module is used for setting corresponding reporting rules for each edge server so that the edge servers can send log files according to specific rules.
Optionally, the method further comprises:
and the log file receiving module is used for receiving the log file sent by the edge server according to the reporting rule.
According to a fourth aspect of the present invention, there is provided a terminal comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the data processing method as described above.
According to a fifth aspect of the present invention there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the data processing method as described above.
Compared with the background art, the embodiment of the invention has the following advantages:
the data processing system comprises a live broadcast server and a plurality of edge servers, wherein the live broadcast server is in communication connection with each edge server, the first state information of a client and the second state information of each edge server are acquired through the live broadcast server, a target server corresponding to the client is determined from the plurality of edge servers according to the first state information and the second state information, and a log file is uploaded to the corresponding target server through the client; the target server receives the log file sent by the client, determines a target log file according to the log extraction instruction after receiving the log extraction instruction sent by the live broadcast server, and sends the target log file to the live broadcast server; the method has the advantages that the log files of the corresponding clients are directly received through the edge server, the uplink bandwidth of the live broadcast server can be effectively saved, the load of the live broadcast server is reduced, meanwhile, the live broadcast server obtains the needed log files by sending the log extraction instruction, and the calculation force can be saved; therefore, log file collection under live broadcast equal-altitude concurrency and large-broadband scenes can be realized.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flowchart illustrating a data processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data processing system architecture in an embodiment of the invention;
fig. 3 is a schematic flow chart of a data processing method according to a second embodiment of the present invention;
FIG. 4 is a schematic block diagram of a data processing apparatus according to a third embodiment of the present invention;
fig. 5 is a schematic block diagram of a data processing apparatus according to a fourth embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention. The method may be applied to a data processing system, which may be in particular a Pingback system. The PingBack system is a system for collecting log files and feeding back the log files to the client based on a load balancing server.
Taking a live broadcast application program as an example, under the condition of starting a service function of the Pingback system, when a user starts the live broadcast application program to enter a live broadcast room, sends chat messages in the live broadcast room, analyzes interactive operations such as link of the live broadcast room and the like, the Pingback system receives corresponding ping back log files, namely the log files in the embodiment of the invention.
Referring to FIG. 2, FIG. 2 is a diagram illustrating an architecture of a data processing system according to an embodiment of the present invention. The data processing system architecture of the embodiment of the invention comprises a first layer architecture and a second layer architecture; the first layer of architecture comprises a live broadcast server, and the second layer of architecture comprises a plurality of edge servers; the live broadcast server is in communication connection with each edge server. The edge computing server may be a CDN edge server or a router, gateway, computer, etc. with a computing storage function.
In the embodiment of the invention, the method specifically comprises the following steps:
step 101, the live broadcast server acquires first state information of a client and second state information of each edge server.
In this embodiment, the live broadcast server and the edge server may be independent physical servers, or cloud servers that provide cloud servers, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, security services, and basic cloud computing servers such as big data and artificial intelligence platforms.
The client may comprise software, such as a computer program, running in a physical device of the type smart phone, desktop computer, tablet computer, notebook computer, etc. Each client side can send a report message to the live broadcast server when starting operation, or the client side can send the report message to the live broadcast server at regular time in the operation process, or the client side can send the report message to the live broadcast server when detecting that the network information change (for example, an operator is switched) or the geographic position information change exceeds a preset threshold value. And sending a report message to the live broadcast server through the client to request the live broadcast server to redetermine the target edge server corresponding to the client and reacquire the edge server list.
It should be noted that, the client can detect the network state and the geographic position information at any time, so that the live broadcast server can adjust the edge server receiving the log file of the client in time. For example, when the change of the operator or the geographic position is switched to exceed the preset threshold value in the process of uploading the log file by the client, the client can timely determine a new edge server for receiving the log file of the client by sending a report operator message or a geographic position message to the live server, and upload the log file of the non-uploaded part to the new edge server. Because the live broadcast server is in communication connection with each edge server, when the live broadcast server needs to acquire the log file of the client, the log file can be acquired from the original edge server and the new edge server for receiving the log file of the client, so that the complete log file is ensured to be acquired.
The report message includes first status information of the client, where the first status information may include first service network information and first location information of the client, where the first service network information may refer to an operator of a network service on which the client depends when sending the report message. The first location information may refer to geographical location information where the client sends the report message. Thus, the live server may obtain the first state information of each client, i.e. obtain the first service network information and the first location information of each client.
Each edge server in communication with the live server may periodically send a status message to the live server, which may include service network information and location information of the edge server. Thus, the live server may acquire the second state information of each edge server, i.e. acquire the second service network information and the second location information of each edge server.
And 102, determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information by the live broadcast server, and informing the client to upload the log file to the corresponding target server.
In this embodiment, the live broadcast server may schedule the edge server as the log file receiving server of the corresponding client. The log file sent by the client can be stored in the corresponding edge server. Therefore, the phenomenon that the uplink bandwidth of the live broadcast server is influenced by the fact that a plurality of clients send log files at the same time is effectively prevented.
In order to improve network efficiency, when determining log file receiving servers of all clients, determining target servers corresponding to all clients according to first state information of all clients and second state information of all edge servers, and notifying all clients to upload log files to the corresponding target servers.
For ease of understanding and description, the description below is made from the standpoint of determining a target server to which one client corresponds.
In an optional embodiment of the present invention, the determining, by the live broadcast server according to the first status information and the second status information, a target server corresponding to the client from a plurality of edge servers includes:
the live broadcast server determines the edge server with the same second service network information as the first service network information as a candidate server;
the live broadcast server determines the geographic distance between the client and each candidate server according to the first position information and the second position information of each candidate server;
and the live broadcast server determines a target server corresponding to the client from the candidate servers of which the geographic distance meets the preset distance condition.
In this embodiment, the live broadcast server may determine, from the first state information of the client, the first service network information and the first location information of the client, and may determine, from the second state information of each edge server, the second service network information and the second location information of each edge server; then, an edge server whose second service network information is the same as the first service network information of the client is determined as a candidate server. The number of candidate servers is typically greater than 1.
After the candidate servers are determined, the live broadcast server further determines the geographic distance between the client and each candidate server according to the first position information of the client and the second position information of each candidate server; and then, determining a target server corresponding to the client from candidate servers of which the geographic distance meets the preset distance condition.
The preset distance conditions may be set in combination with an actual scene, for example, the preset distance conditions may be the first n of the values of the plurality of geographic distances after being sequenced from small to large, where n is greater than or equal to 1; and randomly determining a target server corresponding to the client from the candidate servers corresponding to the n geographic distances. For another example, the preset distance condition may be a specific geographic distance threshold, and then the target server corresponding to the client is randomly determined from the candidate servers corresponding to the geographic distance threshold, where the geographic distance value is smaller than the geographic distance threshold.
The live server may also determine a candidate server closest to the client's geographic distance as the target server corresponding to the client.
It will be appreciated that in this embodiment, the target server corresponding to a certain client may also correspond to other clients, that is, in this embodiment, there may be a plurality of clients corresponding to the same target server. And the number of target servers corresponding to one client at the same time is 1.
In this embodiment, according to a first network service of a client and a geographic distance between the client and an edge server, determining a target server corresponding to the client from a plurality of edge servers; the target server corresponding to the client is determined from the geographic distance between the network service and the client, so that the transmission efficiency of the subsequent client for transmitting the log file to the target server can be improved, and network resources are saved.
In an alternative embodiment, after determining the plurality of candidate servers, or when the number of candidate servers satisfying the preset distance condition is a plurality, the live server may determine the target server by according to the network distance between the client and each candidate server (or the candidate servers satisfying the preset distance condition). The network distance is different from the above-mentioned geographical distance, it being understood that the geographical distance is determined by the geographical locations of both; and the network distance is determined by the network communication speed of the two.
In an example, the live server may instruct the client to send the test data packet to each candidate server (or candidate servers meeting the preset distance condition), where each candidate server (or candidate servers meeting the preset distance condition) returns corresponding response information to the client after receiving the test data packet; the client returns the time corresponding to the response information of each candidate server (or the candidate servers meeting the preset distance condition) to the live broadcast server, and the longer the time is, the longer the corresponding network distance is; and the live broadcast server determines the target server according to the time when the client receives the response information. Specifically, the candidate server (or the candidate server satisfying the preset distance condition) having the shortest time to receive the response information may be determined as the target server.
In another example, the live server may instruct the client to send the test data packet to each candidate server (or candidate servers meeting the preset distance condition), and each candidate server (or candidate server meeting the preset distance condition) may determine the hop count (hop) of the test packet after receiving the test data packet, and feed back the hop count to the live server, where the more the hop count, the longer the corresponding network distance; and after receiving the hop counts fed back by each candidate server (or candidate servers meeting the preset distance condition), the live broadcast server determines the target server according to the hop counts. Specifically, the candidate server with the smallest hop count (or the candidate server satisfying the preset distance condition) may be determined as the target server.
Further, in an optional embodiment of the present invention, the determining, by the live broadcast server, a target server corresponding to the client from candidate servers whose geographic distance satisfies a preset distance condition may further include:
the live broadcast server obtains the load value of each candidate server;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the load value meets a preset load condition and the geographic distance meets a preset distance condition.
In this embodiment, the first status information sent by each edge server to the live server may further include a load value. Accordingly, the live server may determine a load value of each edge server from the received first state information.
After the candidate servers are determined, the live broadcast server further determines the geographic distance between the client and each candidate server according to the load value of each candidate server; and determining a target server corresponding to the client from candidate servers of which the load values meet preset load conditions and the geographic distances meet the preset conditions.
The process of determining the candidate server whose geographic distance satisfies the preset condition is similar to that described above, and will not be repeated here.
The preset load conditions may be set in combination with an actual scenario, for example, the preset load conditions may be the first m of the load values of the plurality of candidate servers after being sequenced in order from small to large, where m is greater than or equal to 1; as another example, the preset load condition may be a specific load threshold, and when the load value is less than the load threshold, it indicates that the preset load condition is satisfied.
In this embodiment, a plurality of candidate servers whose load values satisfy a preset load condition may be determined first, and then a candidate server whose geographic distance satisfies a preset distance condition may be determined from the plurality of candidate servers whose load values satisfy the preset load condition; or determining a plurality of candidate servers with geographic distances meeting the preset distance condition, and determining candidate servers with load values meeting the preset load condition from the plurality of candidate servers with geographic distances meeting the preset distance condition; and finally, determining the target server from candidate servers of which the load values meet the preset load conditions and the geographic distances meet the preset conditions.
According to the method, a target server corresponding to a client is determined from a plurality of edge servers according to a first network service of the client, a geographic distance between the client and the edge servers and a load value of the edge servers; the target server corresponding to the client is determined from the angles of network service, relative geographic distance and server load, so that the transmission efficiency of the subsequent client for transmitting the log file to the target server can be improved, and network resources are saved.
Further, in another optional embodiment of the present invention, the determining, by the live broadcast server, a target server corresponding to the client from candidate servers whose geographic distance satisfies a preset distance condition may further include:
acquiring the historical online time and the historical offline time of the candidate server;
determining a stable value of the candidate server according to the historical online time and the historical offline time;
and determining the candidate servers of which the stable values meet preset stable conditions and the geographic distances meet preset distance conditions as target servers corresponding to the clients.
In this embodiment, the live server may further determine an online time and a offline time of each edge server according to a communication connection state with each edge server. Therefore, after the candidate servers are determined, the live server can acquire the historical online time and the historical offline time of each candidate server in the preset historical time period. The historical online time of the candidate server may refer to the time when the candidate server and the live broadcast server are in communication connection in a preset historical time period; the historical offline time of the candidate server may refer to the time that the candidate server is not connected to the live server within a preset historical period of time.
As an example, the preset historical time period is the previous week from the current time, and when a certain candidate server is in 7 days of the previous week, the time of keeping communication connection with the live broadcast server is 5 days, namely, the historical online time is 5 days; the time for disconnecting the communication connection is 2 days, namely the historical offline time is 2 days; a stable value for the candidate server may be calculated from the historical online time and the historical offline time, which may be the ratio of the historical offline time to the historical online time, the stable value in the above example being 2/5=0.4. The larger the stable value is, the more unstable the communication connection state between the corresponding candidate server and the live server is.
The process of determining the candidate server whose geographic distance satisfies the preset condition is similar to that described above, and will not be repeated here.
The preset stability condition may be set in combination with an actual scene, for example, the preset stability condition may be the first p after the stability values of the plurality of candidate servers are ordered in order from small to large, where p is greater than or equal to 1. For another example, the preset stability condition may be a specific stability threshold, and when the stability value is smaller than the preset stability threshold, it indicates that the preset stability condition is satisfied.
In this embodiment, a plurality of candidate servers whose stable values satisfy a preset stable condition may be determined first, and then a candidate server whose geographic distance satisfies a preset distance condition is determined from the plurality of candidate servers whose stable values satisfy the preset stable condition; or determining a plurality of candidate servers with geographic distances meeting the preset distance condition, and determining candidate servers with stable values meeting the preset stable condition from the plurality of candidate servers with geographic distances meeting the preset distance condition; and finally, determining the target server from candidate servers of which the stable values meet preset stable conditions and the geographic distances meet the preset conditions.
According to the method, a target server corresponding to a client is determined from a plurality of edge servers according to a first network service of the client, a geographic distance between the client and the edge servers and a stable value of the edge servers; that is, the target server corresponding to the client is determined from the viewpoints of network service, relative distance and server stability, so that the transmission efficiency and stability of the log file transmitted to the target server by the subsequent client can be improved.
It should be noted that, in the embodiment of the present invention, the target server corresponding to the client may be determined from a plurality of edge servers according to one or more of the first network service of the client, the geographic distance between the client and the edge server, the network distance between the client and the edge server, the stability value of the edge server, and the load value of the edge server. The order of the judgment of the above conditions may not be limited.
Further, in addition to any of the above optional embodiments, the target server may be determined according to a storable space of the edge server. Specifically, the live broadcast server may further acquire storable spaces of the edge servers, and use the storable spaces satisfying a preset storage condition as one of conditions for determining the target server. The preset storage conditions may be set in combination with an actual scene, for example, the first q of the storable spaces after being sequenced in the order from large to small may be that q is greater than or equal to 1; for another example, the preset storage condition may be a specific storage threshold, and when the storable space value is greater than the preset storage threshold, it indicates that the preset storage condition is satisfied.
After determining the target server, the live server may send relevant information of the target server to the client, and notify the client to upload the log file to its corresponding target server. The related information may be identification information of the target server, or an IP address of the target server, etc.
And step 103, the target server receives the log file sent by the client.
In the embodiment of the invention, the target server can be used as the target servers of a plurality of clients; thus, the target server may receive log files sent by its corresponding individual clients.
In an alternative embodiment of the present invention, the target server may process the log file after receiving the log file of the client. The specific processing of the log file may include, but is not limited to, the following examples:
for example, the target server may establish a correspondence between the client and the log file, so that when the log file of a certain client needs to be acquired, the log file of the client may be quickly and accurately found.
For another example, the target server may compress the received log file, so that when the log file needs to be sent to the live broadcast server, by sending the compressed log file, the consumed bandwidth resource may be reduced, and the transmission efficiency may be improved; optionally, in order to improve the utilization rate of the storage space, the target server may further delete the corresponding original log file after performing compression processing on the log file.
For another example, the target server may delete the log file that is out of date according to the storage time of the log file. In practical application, the effective value of the log file is generally considered to be reduced along with time, and when the target server determines that the storage time of a certain log file reaches a preset time threshold, the log file with the storage time reaching the preset time threshold is deleted, so that storage resources are saved; in addition, the log file actively deleted by the target server does not need to be uploaded to the live broadcast server, so that the uplink bandwidth can be saved.
Step 104, after receiving the log extraction instruction sent by the live broadcast server, the target server determines a target log file according to the log extraction instruction, and sends the target log file to the live broadcast server.
In this embodiment, when the log file has a requirement of viewing, the live broadcast server may send a log extraction instruction to the edge server to instruct the edge server to send the corresponding log file to the live broadcast server. That is, after receiving the log extraction instruction sent by the live broadcast server, the target server can determine the target log file according to the log extraction instruction, and send the target log file to the live broadcast server.
Optionally, after receiving the log extraction instruction sent by the live broadcast server, the target server determines a target log file according to the log extraction instruction, and sends the target log file to the live broadcast server, where the method includes:
the target server receives a log extraction instruction sent by the live broadcast server, wherein the log extraction instruction comprises a target client identifier;
and the target server determines a target log file from the log files according to the target client identifier, and sends the target log file to the live broadcast server.
In this embodiment, when the live broadcast server needs to extract the log file of the target client, the target server corresponding to the target client may be determined first, and a log extraction instruction including the identifier of the target client may be generated; and then sending the log extraction instruction to the determined target server. Because the target server can correspond to a plurality of clients and receive the log files of the clients corresponding to the target server, the target server determines the target log file from the stored log files according to the target client identification in the log extraction instruction after receiving the log extraction instruction, and sends the target log file to the live broadcast server.
Optionally, after receiving the log extraction instruction sent by the live broadcast server, the target server determines a target log file according to the log extraction instruction, and sends the target log file to the live broadcast server, where the method includes:
the target server receives a log extraction instruction sent by the live broadcast server, wherein the log extraction instruction comprises time information;
and the target server determines a target log file from the log files according to the time information and sends the target log file to the live broadcast server.
In this embodiment, when the live broadcast server needs to extract the log file in a certain period of time, a log extraction instruction containing time information may be generated, and the log extraction instruction may be sent to each edge server (target server) to instruct the edge server to send the log file in the period of time corresponding to the time information to the live broadcast server.
It should be noted that, the log extraction instruction may further include a target client identifier and time information, so as to instruct the target server to send the log file of the target client in the time period corresponding to the time information to the live broadcast server.
Alternatively, after the target server transmits the target log file to the live server, the target log file may be deleted from the storage space of the target server.
Further, in an optional embodiment of the present invention, the target server may further send a log file to the live broadcast server according to a preset reporting rule.
Specifically, the live broadcast server may set a corresponding reporting rule for each edge server, or each edge server may have a corresponding reporting rule; the edge server sends the log file to the live broadcast server according to a specific rule; the edge server can be specified to actively upload the received log file to the live broadcast server according to a specific time point; or the edge server actively uploads the received log file to the live broadcast server according to a specific time interval; or the edge server uploads the log file of the specified client to the live broadcast server when receiving the log file of the specified client.
It should be noted that the reporting rules corresponding to different edge servers may be the same or different. Illustratively, the reporting rule for edge server a may be 1 per day: 00am uploads the log file stored by the 00am to a live broadcast server; while the reporting rule of edge server B may be 2:00am uploads the log file it stores to the live server. As another example, since the target servers corresponding to the same client at different times may be different, i.e., different edge servers may store log files of the same client at different times; when the client A is a designated client, the corresponding target server of the client A in the first time period is an edge server C, and the corresponding target server in the second time period is an edge server D; the reporting rule of the edge server C may be that the log file of the client A is uploaded to the live broadcast server in real time; the reporting rule of the edge server D may also be to upload the log file of the client a to the live broadcast server in real time.
In the embodiment of the invention, a data processing system comprises a live broadcast server and a plurality of edge servers, wherein the live broadcast server is in communication connection with each edge server, the first state information of a client and the second state information of each edge server are acquired through the live broadcast server, a target server corresponding to the client is determined from the plurality of edge servers according to the first state information and the second state information, and a log file is uploaded to the corresponding target server through the client; the target server receives the log file sent by the client, determines a target log file according to the log extraction instruction after receiving the log extraction instruction sent by the live broadcast server, and sends the target log file to the live broadcast server; the method has the advantages that the log files of the corresponding clients are directly received through the edge server, the uplink bandwidth of the live broadcast server can be effectively saved, the load of the live broadcast server is reduced, meanwhile, the live broadcast server obtains the needed log files by sending the log extraction instruction, and the calculation force can be saved; therefore, log file collection under live broadcast equal-altitude concurrency and large-broadband scenes can be realized.
Referring to fig. 3, fig. 3 is a flowchart illustrating a data processing method according to a second embodiment of the present invention. The method mainly provides a data processing method from the viewpoint of a live broadcast server.
In the embodiment of the invention, the method specifically comprises the following steps:
step 301, obtaining first state information of a client and second state information of each edge server.
This step may be described with reference to step 101 in the first embodiment of the present invention, which is not described herein.
And step 302, determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and informing the client to upload the log file to the corresponding target server.
This step may be described with reference to step 102 in the first embodiment of the present invention, which is not described herein.
Step 303, determining an edge server storing the log file to be extracted, and sending a log extraction instruction to the determined edge server.
In this embodiment, when there is a need to view a log file, an edge server storing the log file having the need to view is determined, that is, an edge server storing a log file to be extracted is determined, and a log extraction instruction is sent to the determined edge server. The log extraction instruction is used for instructing the edge server to send the corresponding log file to the live broadcast server.
In an alternative embodiment, when the live broadcast server needs to extract the log file of the target client, the target server corresponding to the target client may be determined first, and a log extracting instruction including the identifier of the target client may be generated; and then sending the log extraction instruction to the determined target server.
In another alternative embodiment, when the live server needs to extract the log file within a certain period of time, a log extraction instruction containing time information may be generated, and the log extraction instruction may be sent to each edge server.
In yet another optional embodiment, when the live broadcast server needs to extract the log file in a certain period of time of the target client, the target server corresponding to the target client may be determined first, and a log extraction instruction including the target client identifier and the time information may be generated; and then sending the log extraction instruction to the determined target server.
Step 304, receiving a response message of the log extraction instruction.
The edge server receiving the log extraction instruction returns a corresponding response message to the live broadcast server. Specifically, after receiving the log extraction instruction, the edge server may send the target log file stored therein and corresponding to the log extraction instruction to the live broadcast server.
Optionally, before sending the target log file to the live broadcast server, the edge server may process the target log file, including compressing the target log file, and sending the compressed target log file to the live broadcast server.
Further, in an optional embodiment of the present application, the method may further include:
and setting corresponding reporting rules for each edge server, so that the edge servers send log files according to specific rules.
The edge server can be specified to actively upload the received log file to the live broadcast server according to a specific time point; or the edge server actively uploads the received log file to the live broadcast server according to a specific time interval; or the specified edge server uploads the log file of the specified client to the live broadcast server when receiving the log file of the specified client; etc.
It should be noted that the reporting rules corresponding to different edge servers may be the same or different. Illustratively, the reporting rule for edge server a may be 1 per day: 00am uploads the log file stored by the 00am to a live broadcast server; while the reporting rule of edge server B may be 2:00am uploads the log file it stores to the live server. As another example, since the target servers corresponding to the same client at different times may be different, i.e., different edge servers may store log files of the same client at different times; when the client A is a designated client, the corresponding target server of the client A in the first time period is an edge server C, and the corresponding target server in the second time period is an edge server D; the reporting rule of the edge server C may be that the log file of the client A is uploaded to the live broadcast server in real time; the reporting rule of the edge server D may also be to upload the log file of the client a to the live broadcast server in real time.
In another optional embodiment of the present application, the method may further include:
and receiving the log file sent by the edge server according to the reporting rule.
In this embodiment, the reporting rule may be set by the live broadcast server or may be set by each edge server. The description of the edge server sending the log file to the live broadcast server according to the reporting rule can be referred to above, and will not be repeated here.
In the embodiment of the invention, the live broadcast server acquires the first state information of the client and the second state information of each edge server; determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and informing the client to upload a log file to the corresponding target server; determining an edge server for storing the log file to be extracted, and sending a log extraction instruction to the determined edge server; receiving a response message of the log extraction instruction; the method has the advantages that the log files of the corresponding clients are directly received through the edge server, the uplink bandwidth of the live broadcast server can be effectively saved, the load of the live broadcast server is reduced, meanwhile, the live broadcast server obtains the needed log files by sending the log extraction instruction, and the calculation force can be saved; therefore, log file collection under live broadcast equal-altitude concurrency and large-broadband scenes can be realized.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 4, there is shown a schematic block diagram of a data processing apparatus according to a third embodiment of the present invention, where the apparatus corresponds to the above-mentioned method embodiment, and the apparatus may specifically include the following modules:
a state information obtaining module 401, configured to obtain first state information of a client and second state information of each edge server;
a target server determining module 402, configured to determine a target server corresponding to the client from a plurality of edge servers according to the first status information and the second status information, and notify the client to upload a log file to the corresponding target server;
An extraction instruction sending module 403, configured to determine an edge server storing a log file to be extracted, and send a log extraction instruction to the determined edge server;
and a response message receiving module 404, configured to receive a response message of the log extraction instruction.
Optionally, the first status information includes first service network information of the client and first location information of the client; the second state information comprises second service network information of the edge server and second position information of the edge server; the target server determination module 402 includes:
a first determining module, configured to determine, as a candidate server, an edge server in which the second service network information is the same as the first service network information;
a second determining module, configured to determine a distance between the client and each candidate server according to the first location information and second location information of each candidate server;
and the target determining module is used for determining a target server corresponding to the client from the candidate servers of which the distances meet the preset distance condition.
Optionally, the target determining module further includes:
The load value determining module is used for obtaining the load value of each candidate server;
and the third determining module is used for determining a target server corresponding to the client from candidate servers of which the load value meets a preset load condition and the distance meets a preset distance condition.
Optionally, the target determining module further includes:
the online and offline time acquisition module is used for acquiring the historical online time and the historical offline time of each candidate server;
the stable value determining module is used for determining the stable value of each candidate server according to the historical online time and the historical offline time;
and the fourth determining module is used for determining a target server corresponding to the client from candidate servers of which the stable value meets a preset stable condition and the distance meets a preset distance condition.
Optionally, the method further comprises:
and the reporting rule setting module is used for setting corresponding reporting rules for each edge server so that the edge servers can send log files according to specific rules.
Optionally, the method further comprises:
and the log file receiving module is used for receiving the log file sent by the edge server according to the reporting rule.
With reference to FIG. 5, there is shown a schematic block diagram of a data processing apparatus according to a fourth embodiment of the present invention, the apparatus being applicable to edge servers in a data processing system, the data processing system including a live server and a plurality of edge servers, the live server being communicatively connected to each of the edge servers; the device specifically comprises the following modules:
a log receiving module 501, configured to receive a log file sent by a client corresponding to the log file determined by the live broadcast server;
the instruction receiving module 502 is configured to receive a log extraction instruction sent by the live broadcast server; the log extraction instruction comprises a target client identifier;
the target log sending module 503 is configured to determine a target log file according to the target client identifier, and send the target log file to the live broadcast server.
Optionally, the apparatus further comprises:
and the log compression module is used for compressing the log file.
Optionally, the apparatus further comprises:
the time judging module is used for judging whether the storage time of the log file reaches a preset time threshold value or not;
the first log deleting module is used for deleting the log files with the storage time reaching a preset time threshold.
Optionally, the apparatus further comprises:
the rule receiving module is used for receiving the report rule message of the log file sent by the live broadcast server; the report rule information comprises report rules of the log file;
and the log sending module is used for sending the corresponding log file to the live broadcast server according to the reporting rule.
Optionally, the apparatus further comprises:
and the second log deleting module is used for deleting the log files successfully sent to the live broadcast server.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
The embodiment of the invention also provides a terminal which can comprise a processor, a memory and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the processes of the data processing method embodiment described above when being executed by the processor, and can achieve the same technical effects, and the repetition is avoided, so that the description is omitted.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the respective processes of the data processing method embodiment described above, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
As will be readily appreciated by those skilled in the art: any combination of the above embodiments is possible, and thus is an embodiment of the present invention, but the present specification is not limited by the text.
The data processing methods provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a system constructed with aspects of the present invention will be apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in a data processing method according to an embodiment of the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.

Claims (9)

1. A data processing method, the method being applied to a data processing system, the data processing system comprising a live server and a plurality of edge servers, the live server being communicatively coupled to each of the edge servers, the method comprising:
The live broadcast server acquires first state information of a client and second state information of each edge server;
the live broadcast server determines a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and informs the client to upload a log file to the corresponding target server;
the target server receives the log file sent by the client;
after receiving a log extraction instruction sent by the live broadcast server, the target server determines a target log file according to the log extraction instruction and sends the target log file to the live broadcast server;
the live broadcast server determines a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and the method comprises the following steps:
the live broadcast server acquires the historical online time and the historical offline time of each edge server;
the live broadcast server determines the stable value of each edge server according to the historical online time and the historical offline time;
And the live broadcast server determines a target server corresponding to the client from the edge servers of which the stable values meet preset stable conditions.
2. The method of claim 1, wherein the first status information includes first service network information of the client and first location information of the client; the second state information comprises second service network information of the edge server and second position information of the edge server; the live broadcast server determines a target server corresponding to the client from edge servers of which the stable values meet preset stable conditions, and the target server comprises:
the live broadcast server determines the edge server with the same second service network information as the first service network information as a candidate server;
the live broadcast server determines the geographic distance between the client and each candidate server according to the first position information and the second position information of each candidate server;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the geographic distance meets a preset distance condition and the stable value meets a preset stable condition.
3. The method of claim 2, wherein the live server determines a target server corresponding to the client from among candidate servers whose geographic distance satisfies a preset distance condition and whose stable value satisfies a preset stability condition, further comprising:
the live broadcast server obtains the load value of each candidate server;
and the live broadcast server determines a target server corresponding to the client from candidate servers of which the load value meets a preset load condition, the geographic distance meets a preset distance condition and the stable value meets a preset stable condition.
4. The method according to claim 1, wherein the target server, after receiving the log extraction instruction sent by the live broadcast server, determines a target log file according to the log extraction instruction, and sends the target log file to the live broadcast server, including:
the target server receives a log extraction instruction sent by the live broadcast server, wherein the log extraction instruction comprises a target client identifier;
and the target server determines a target log file from the log files according to the target client identifier, and sends the target log file to the live broadcast server.
5. The method of claim 1, further comprising, after the target server receives the log file sent by the client:
and the target server compresses the log file.
6. A method of data processing, the method comprising:
acquiring first state information of a client and second state information of each edge server;
determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and informing the client to upload a log file to the corresponding target server;
determining an edge server for storing the log file to be extracted, and sending a log extraction instruction to the determined edge server;
receiving a response message of the log extraction instruction;
wherein determining, from the plurality of edge servers, a target server corresponding to the client according to the first status information and the second status information includes:
acquiring historical online time and historical offline time of each edge server;
determining a stable value of each edge server according to the historical online time and the historical offline time;
And determining a target server corresponding to the client from the edge servers of which the stable values meet preset stable conditions.
7. A data processing apparatus, the apparatus comprising:
the state information acquisition module is used for acquiring the first state information of the client and the second state information of each edge server;
the target server determining module is used for determining a target server corresponding to the client from a plurality of edge servers according to the first state information and the second state information, and notifying the client to upload the log file to the corresponding target server;
the extraction instruction sending module is used for determining an edge server for storing the log file to be extracted and sending a log extraction instruction to the determined edge server;
the response message receiving module is used for receiving the response message of the log extraction instruction;
wherein, the target server determining module includes:
the online and offline time acquisition module is used for acquiring the historical online time and the historical offline time of each edge server;
the stable value determining module is used for determining the stable value of each edge server according to the historical online time and the historical offline time;
And the stability-based determining module is used for determining a target server corresponding to the client from the edge servers of which the stability values meet preset stability conditions.
8. A terminal, comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, implements the data processing method according to claim 6.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the data processing method according to claim 6.
CN202110963270.2A 2021-08-20 2021-08-20 Data processing method, device, terminal and storage medium Active CN113747186B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110963270.2A CN113747186B (en) 2021-08-20 2021-08-20 Data processing method, device, terminal and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110963270.2A CN113747186B (en) 2021-08-20 2021-08-20 Data processing method, device, terminal and storage medium

Publications (2)

Publication Number Publication Date
CN113747186A CN113747186A (en) 2021-12-03
CN113747186B true CN113747186B (en) 2023-04-28

Family

ID=78732165

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110963270.2A Active CN113747186B (en) 2021-08-20 2021-08-20 Data processing method, device, terminal and storage medium

Country Status (1)

Country Link
CN (1) CN113747186B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115440070A (en) * 2022-07-22 2022-12-06 中智行(苏州)科技有限公司 Automatic driving traffic signal lamp information acquisition system and method based on vehicle and road coordination
CN115941794A (en) * 2022-11-16 2023-04-07 北京奇艺世纪科技有限公司 File storage method, system and device and electronic equipment
CN116017145B (en) * 2022-12-27 2023-08-01 深圳市快美妆科技有限公司 Remote intelligent control system and method for live camera

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102780777B (en) * 2012-07-19 2015-04-08 北京蓝汛通信技术有限责任公司 Log collecting method and log collecting system
CN106302577B (en) * 2015-05-19 2019-07-09 广州四三九九信息科技有限公司 Log processing method
US10904307B2 (en) * 2016-12-14 2021-01-26 Verizon Digital Media Services Inc. Distributed management of live stream storage

Also Published As

Publication number Publication date
CN113747186A (en) 2021-12-03

Similar Documents

Publication Publication Date Title
CN113747186B (en) Data processing method, device, terminal and storage medium
EP3595268A1 (en) Streaming media resource distribution method, system, edge node and central dispatching system
US11284126B2 (en) Method and system for streaming media live broadcast
CN109474684B (en) Method, device, terminal equipment and storage medium for acquiring live video stream
US9173006B2 (en) Method for live broadcasting in a distributed network and apparatus for the same
CN105681387A (en) Method, device and system for uploading live video
CN105656876A (en) Live video play method, device and system
CN107231350B (en) Message processing method and device
CN107734350B (en) Live broadcasting system and live broadcasting method
CN109348257B (en) Pull stream control method and device and live broadcast system
CN109067910A (en) A kind of method and device that message pulls
CN102111426A (en) Method, system and device for implementing point-to-point technology
CN103945198A (en) System and method for controlling streaming media route of video monitoring system
CN113811013B (en) Cluster capacity dynamic adjustment method, device, storage medium and communication system
CN113055692A (en) Data processing method and device
CN109348243B (en) Subscription processing method and device, live broadcast system, storage medium and computer equipment
CN109561137B (en) Method, device, terminal equipment and medium for establishing P2P network
CN111541555A (en) Group chat optimization method and related product
CN103547339A (en) File uploading method, client side and server side
CN111541684B (en) Signaling sending method and device of live broadcast room, server and storage medium
CN111625743A (en) Resource loading method and device and electronic equipment
CN103327046A (en) P2SP system scheduling method, equipment and system based on node service capacity
CN113014672B (en) Message pushing method and device, electronic equipment and storage medium
CN114257828A (en) Live broadcast audit content processing method and system
CN115208955B (en) Method, device, computer equipment and medium for processing resource request

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant