CN116980388A

CN116980388A - Data processing method, apparatus, computer, storage medium, and program product

Info

Publication number: CN116980388A
Application number: CN202310579699.0A
Authority: CN
Inventors: 庄志鹏; 张亮
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-05-22
Filing date: 2023-05-22
Publication date: 2023-10-31

Abstract

The embodiment of the application discloses a data processing method, a device, a computer, a storage medium and a program product, wherein the method comprises the following steps: receiving a data packet to be analyzed sent by media forwarding equipment, and acquiring a media service instance aiming at a first service object based on object position information of the first service object in the data packet to be analyzed; receiving a media subscription message for a first service object sent by a first combined device, and adding the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device; decoding the initial media data of the data packet to be analyzed to obtain decoded media data, acquiring a subscription object from a media service instance, and transmitting the decoded media data to the subscription object; the subscription object includes a first combined device. By adopting the application, the efficiency of media data processing can be improved.

Description

Data processing method, apparatus, computer, storage medium, and program product

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, computer, storage medium, and program product.

Background

In order to reduce the downstream bandwidth of the server, the occupancy rate of the central processing unit (Central Processing Unit, CPU) of the client is saved, and the mixing process is generally performed. With the increase of the importance of the spatial audio, in order to better restore the real world sound, the uplink voice data with higher code rate and sampling rate is generally needed, and the spatial audio algorithm consumes CPU resources very much. In order to better realize the audio mixing process, the voices of all users in the same room are generally integrated on the same audio mixing server, and the voices are output after the audio mixing is completed.

Disclosure of Invention

The embodiment of the application provides a data processing method, a data processing device, a computer, a storage medium and a program product, which can improve the accuracy and the detection efficiency of data processing.

In one aspect, an embodiment of the present application provides a data processing method, where the method includes:

receiving a data packet to be analyzed sent by media forwarding equipment, and acquiring a media service instance aiming at a first service object based on object position information of the first service object in the data packet to be analyzed;

receiving a media subscription message for a first service object sent by a first combined device, and adding the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device;

decoding the initial media data in the data packet to be analyzed to obtain decoded media data, acquiring a subscription object from the media service instance, and transmitting the decoded media data to the subscription object; the subscription object includes a first combined device.

receiving initial media data generated by a first service object, and determining first combination equipment corresponding to the initial media data;

acquiring a first decoding device, and transmitting a data packet to be analyzed generated based on initial media data to the first decoding device so that the first decoding device constructs a media service instance aiming at a first service object;

Transmitting the service address of the first decoding device to the first combining device, so that the first combining device transmits a media subscription message to the first decoding device based on the service address; the media subscription message is used for the first decoding device to send the decoded media data obtained by decoding the initial media data to the first combining device.

In one aspect, an embodiment of the present application provides a data processing apparatus, including:

the data receiving module is used for receiving the data packet to be analyzed sent by the media forwarding equipment;

the instance acquisition module is used for acquiring a media service instance aiming at the first service object based on the object position information of the first service object in the data packet to be analyzed;

the message receiving module is used for receiving a media subscription message aiming at a first service object and sent by the first combined equipment;

a device adding module for adding the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device;

the data decoding module is used for decoding the initial media data in the data packet to be analyzed to obtain decoded media data;

The object determining module is used for acquiring a subscription object from the media service instance;

the decoding and transmitting module is used for transmitting the decoded media data to the subscription object; the subscription object includes a first combined device.

Wherein, this example obtains the module, includes:

the instance searching unit is used for searching a media service instance aiming at the first service object based on the object position information of the first service object in the data packet to be analyzed;

the data construction unit is used for constructing a subscription management set and a data decoding instance for the first service object if the media service instance for the first service object is not found; the subscription management set is used for managing the combined equipment for sending the media subscription message;

and the instance combination unit is used for combining the subscription management set and the data decoding instance to form a media service instance aiming at the first service object.

The device adding module is specifically configured to:

adding the first combined device to a subscription management set based on the media subscription message;

the data decoding module comprises:

the decoding processing unit is used for decoding the initial media data in the data packet to be analyzed to obtain first decoded data;

The decoding buffer unit is used for constructing a decoding buffer area in the data decoding example, analyzing the decoding processing process of the initial media data and generating decoding auxiliary buffer data aiming at the initial media data;

and the decoding generation unit is used for forming the first decoding data and the decoding auxiliary cache data into decoding media data.

Wherein the apparatus further comprises:

the first deleting module is used for acquiring the object subscription duration of the subscription object, deleting the first subscription object of which the object subscription duration is greater than or equal to the subscription maintenance time threshold, and sending a subscription failure message aiming at the first business object to the first subscription object; or,

the second deleting module is used for acquiring the subscription period associated with the subscription object, deleting the second subscription object and sending a subscription failure message aiming at the first service object to the second subscription object; the second subscription object refers to a subscription object having a subscription period less than or equal to the system network time.

The media service instance also comprises a packet mapping instance, and the data packet to be analyzed also comprises service information;

the apparatus further comprises:

the service adding module is used for adding the service information in the data packet to be analyzed into the packet mapping instance;

The data sending module is used for obtaining media association data associated with the initial media data based on the service information when obtaining the decoded media data and sending the media association data to the subscription object; the media-associated data includes service information.

the media acquisition module is used for receiving initial media data generated by the first business object;

the combination determining module is used for determining first combination equipment corresponding to the initial media data;

the decoding acquisition module is used for acquiring the first decoding equipment;

the to-be-parsed sending module is used for sending the to-be-parsed data packet generated based on the initial media data to the first decoding equipment so that the first decoding equipment constructs a media service instance aiming at the first service object;

the address sending module is used for sending the service address of the first decoding device to the first combination device so that the first combination device sends a media subscription message to the first decoding device based on the service address; the media subscription message is used for the first decoding device to send the decoded media data obtained by decoding the initial media data to the first combining device.

Wherein the combination determination module comprises:

the combined searching unit is used for acquiring first media data associated with the initial media data and searching combined equipment corresponding to the first media data; the first media data refers to media data combined with the initial media data;

the combination acquisition unit is used for determining the combination equipment corresponding to the first media data as first combination equipment if the combination equipment corresponding to the first media data is found;

the combination obtaining unit is further configured to obtain a first combination device if the combination device corresponding to the first media data is not found.

Wherein, this decoding acquires the module, includes:

the first acquisition unit is used for acquiring the equipment loads corresponding to the N decoding equipment respectively, and determining the decoding equipment with the minimum equipment load as the first decoding equipment; n is a positive integer; or,

a second obtaining unit, configured to determine, as a first decoding device, a decoding device associated with object location information of the first service object; or,

and the third acquisition unit is used for randomly selecting the first decoding device from the N decoding devices.

The sending module to be analyzed comprises:

The information acquisition unit is used for acquiring object position information of the first service object and acquiring media association data corresponding to the initial media data; the media related data is used for representing related information of a service generating the initial media data;

and the data packaging unit is used for packaging the object position information, the initial media data and the media related data of the first service object into a data packet to be analyzed, and sending the data packet to be analyzed to the first decoding equipment.

Wherein the apparatus further comprises:

the coordinate conversion module is used for acquiring the object coordinates and the region division sizes of the service objects, and carrying out coordinate conversion on the object coordinates of the service objects based on the region division sizes to obtain the object position information of the service objects; the business objects include a first business object;

the hash processing module is used for carrying out hash processing on the object position information of the service object and determining position management equipment corresponding to the service object;

the object request module is used for sending a receiving determination request to the first position management equipment corresponding to the first service object when receiving the initial media data generated by the first service object, so that the first position management equipment determines a second service object for receiving the initial media data based on the receiving determination request;

And the object sending module is used for acquiring the second service object from the first position management device and sending the second service object to the first combination device so that the first combination device can send a message to the second service object based on the decoded media data when receiving the decoded media data.

Wherein the apparatus further comprises:

the heartbeat detection module is used for respectively carrying out periodic heartbeat detection on the N decoding devices and the M combination devices;

the equipment deleting module is used for deleting abnormal equipment if detecting that the abnormal equipment exists in the N decoding equipment and the M combined equipment based on the periodic heartbeat detection; the abnormal device refers to a device which continuously does not receive the number of times of heartbeat replies aiming at periodic heartbeat detection and is larger than or equal to a heartbeat abnormal threshold value;

the subscription deleting module is used for acquiring second decoding equipment associated with the abnormal equipment if the abnormal equipment is the combined equipment, and sending a first abnormal reminding message of the abnormal equipment to the second decoding equipment so that the second decoding equipment deletes subscription information of the abnormal equipment based on the first abnormal reminding message;

the address deleting module is used for acquiring a second combined device associated with the abnormal device if the abnormal device is a decoding device, and sending a second abnormal reminding message of the abnormal device to the second combined device so that the second combined device deletes the service address of the abnormal device based on the second abnormal reminding message.

In one aspect, the embodiment of the application provides a computer device, which comprises a processor, a memory and an input/output interface;

the processor is respectively connected with the memory and the input/output interface, wherein the input/output interface is used for receiving data and outputting data, the memory is used for storing a computer program, and the processor is used for calling the computer program so as to enable the computer equipment containing the processor to execute the data processing method in one aspect of the embodiment of the application.

An aspect of an embodiment of the present application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the data processing method in the aspect of an embodiment of the present application.

In one aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternatives in an aspect of the embodiments of the application. In other words, the computer instructions, when executed by a processor, implement the methods provided in the various alternatives in one aspect of the embodiments of the present application.

The implementation of the embodiment of the application has the following beneficial effects:

in the embodiment of the application, the data packet to be analyzed sent by the media forwarding device can be received, and the media service instance aiming at the first service object is acquired based on the object position information of the first service object in the data packet to be analyzed; receiving a media subscription message for a first service object sent by a first combined device, and adding the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device; decoding the initial media data in the data packet to be analyzed to obtain decoded media data, acquiring a subscription object from the media service instance, and transmitting the decoded media data to the subscription object; the subscription object includes a first combined device. Through the above process, various devices can be adopted to perform data interaction, and based on subscription conditions, the first decoding server can send the received data packet to be analyzed to the combined device which needs to receive the decoding result only by once coding, so that the data decoding efficiency is improved. In addition, the data processing process is distributed to a plurality of devices, and the processes such as data decoding and data mixing can be used for large-scale spatial audio mixing processing, so that the universality and the efficiency of the data processing are improved.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a diagram of a network interaction architecture for data processing according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a data processing scenario provided in an embodiment of the present application;

FIG. 3 is a flow chart of a method for data processing according to an embodiment of the present application;

fig. 4 is a schematic diagram of a data forwarding process according to an embodiment of the present application;

FIG. 5 is a diagram of a device interaction architecture provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a data processing interaction flow provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of a location acquisition scenario provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of an alternative location maintenance scenario provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

FIG. 10 is a schematic diagram of another data processing apparatus according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

If the data of the object (such as the user) needs to be collected in the application, before and during the collection, a prompt interface or a popup window is displayed, wherein the prompt interface or the popup window is used for prompting the user to collect certain data currently, and the relevant step of data acquisition is started only after the confirmation operation of the user to the prompt interface or the popup window is obtained, otherwise, the process is ended. The acquired user data is used in a reasonable and legal scene, application, or the like. Optionally, in some scenarios where user data is required but not authorized by the user, authorization may be requested from the user, and the user data may be reused when authorization passes. And the use of the user data complies with the regulations of the legal regulations.

In the embodiment of the present application, please refer to fig. 1, fig. 1 is a network interaction architecture diagram for data processing provided in the embodiment of the present application, and as shown in fig. 1, a data processing system in the present application may include a service client 101, a media forwarding device 102, a decoding device 103, a combining device 104, and so on. Wherein the service client 101 may send the media data to the media forwarding device 102 when generating the media data, the media forwarding device 102 may send the media data to the decoding device 103, and send the service address of the decoding device 103 to the combining device 104. Wherein the combining device 104 may send a media subscription message to the decoding device 103 based on the service address of the decoding device 103; the decoding device 103 may add the combining device 104 to the media service instance corresponding to the service client 101 based on the media subscription message, perform decoding processing on the media data, and send the result of the decoding processing to the combining device 104. Wherein, the media forwarding device 102 is configured to forward media data; the combining device 104 is configured to perform a combining process on a plurality of media data, that is, a device that combines multiple media data into one media data, for example, may be a mixing server, and is configured to perform spatial audio mixing; the decoding device 103 is used for unpacking and decoding the media data, and the like. Optionally, the data processing system may further comprise a location management device (Location Based Services, LBS) for determining and storing object location information of the service objects, and a listener (listner) of the media data, i.e. the second service object, etc., may be determined based on the object location information of the respective service objects. The media data (such as initial media data) in the present application may be audio data, video data, text data, or mixed data, and the mixed data refers to data composed of multiple data formats, such as data composed of text data and mixed data. Mixing refers to the process of mixing two or more audio streams together to form a single audio stream.

Specifically, referring to fig. 2, fig. 2 is a schematic diagram of a data processing scenario provided in an embodiment of the present application. As shown in fig. 2, when the first service object generates initial media data, the first service client 201 where the first service object is located may send the initial media data to the media forwarding device 202. The media forwarding device 202 may determine a first combining device 203 corresponding to the initial media data, and obtain a first decoding device 204. Further, the media forwarding device 202 may process the initial media data, generate a data packet to be parsed, and send the data packet to be parsed to the first decoding device 204; the service address of the first decoding device 204 may be sent to the first combining device 203. The execution sequence of the process of sending the data packet to be parsed and the service address of the first decoding device 204 by the media forwarding device 202 is not limited. The first decoding device 204 may decode the data packet to be parsed to obtain decoded media data; the first combining device 203 may send a media subscription message to the first decoding device 204 based on the service address of the first decoding device 204, and the first decoding device 204 may add the first combining device 203 to the media service instance of the first service object based on the media subscription message. When the first decoding device 204 obtains the decoded media data, the decoded media data may be sent to the first combining device 203. By cooperation of multiple devices, each of the different devices can perform their own tasks, and perform different processing procedures at the same time, for example, when the first decoding device 204 decodes the data packet to be parsed, the first combining device 203 can generate and send a media service instance for the first service object to the first decoding device 204, and so on. That is, the execution sequence of the execution process between the different devices is not limited, and the limitation on the execution sequence only occurs when some data sent by other devices is needed, for example, when the first decoding device 204 sends the decoded media data to the first combining device 203, the media service instance sent by the first combining device 203 needs to be acquired first, so that cooperation between multiple devices is realized, and the efficiency of data processing is effectively improved. In addition, a mode of publishing and subscribing (namely media subscription information and the like) is adopted, so that decoding of the same media data is only needed once, decoded data can be sent to needed combined equipment according to a publishing and subscribing idea, directional transmission of the data is realized, and data processing efficiency is improved.

Wherein, publish-subscribe mode: in a software architecture, a publish-subscribe can be considered a paradigm of messages that a sender of a message (called a publisher) does not send messages directly to a particular recipient (called a subscriber), but rather the published messages are classified into different categories without knowing which subscribers (if any) may be present. Likewise, a subscriber may express interest in one or more categories, receive only messages of interest, and need not know which publishers (if any) may exist. In the application, the subscriber (i.e. the combined device) can directly send the media subscription information to the needed decoding device (the publisher), and the publisher (i.e. the decoding device) can send the data to the combined device subscribed based on the subscription information maintained by the publisher (i.e. the decoding device), thereby improving the data processing efficiency.

It should be understood that each device (such as a media forwarding device, a decoding device, and a combination device) mentioned in the embodiments of the present application may be considered as a computer device, and the computer device in the embodiments of the present application includes, but is not limited to, a terminal device or a server. In other words, the computer device may be a server or a terminal device, or may be a system formed by the server and the terminal device. The above-mentioned terminal device may be an electronic device, including but not limited to a mobile phone, a tablet computer, a desktop computer, a notebook computer, a palm computer, a vehicle-mounted device, an augmented Reality/Virtual Reality (AR/VR) device, a head-mounted display, a smart television, a wearable device, a smart speaker, a digital camera, a camera, and other mobile internet devices (mobile internet device, MID) with network access capability, or a terminal device in a scene such as a train, a ship, or a flight. The servers mentioned above may be independent physical servers, may be server clusters or distributed systems formed by a plurality of physical servers, and may also be cloud servers for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-road collaboration, content distribution networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms, which are not limited herein.

Optionally, the data related to the embodiment of the present application may be stored in a computer device, or may be stored based on a cloud storage technology or a blockchain network, which is not limited herein.

Further, referring to fig. 3, fig. 3 is a flowchart of a method for data processing according to an embodiment of the present application. As shown in fig. 3, the embodiment of the method uses a decoding device as an execution subject. Wherein the number of decoding devices (meta_decoders) can be considered as N, N being a positive integer; the number of the combination devices (meta_mixer) is M, M is a positive integer, and this embodiment describes using any one decoding device as an execution body, and the decoding device may be referred to as a first decoding device, and the data processing process includes the following steps:

step S301, receiving a data packet to be parsed sent by a media forwarding device, and acquiring a media service instance for a first service object based on object location information of the first service object in the data packet to be parsed.

In the embodiment of the present application, the first decoding device may receive the data packet to be parsed sent by the media forwarding device, where the data packet to be parsed may include initial media data generated by the first service object, object location information of the first service object, and so on. Optionally, the data packet to be parsed may further include, but is not limited to, service information, where the service information is used to represent related information of a service generating the initial media data, and the service may be considered as a scenario generating the initial media data, for example, the service information may include, but is not limited to, a first service scenario generating the initial media data, and scenario position information of a first service object in the service scenario, where the first service scenario may be represented by a first service identifier. For example, when the application program is used by the first service object, initial media data is generated, the service refers to a service scene generated by the initial media data in the application program, for example, the application program is game application a, the initial media data is generated by the first service object on a game map, the service refers to the game map of the first service object in the game application a, and the service information can include, but is not limited to, the service scene of the first service object in the game application a, scene position information of the first service object on the game map of the game application a, and the like.

Further, a media service instance (speaker_channel) of the first service object may be obtained based on object location information of the first service object in the data packet to be parsed, where the media service instance may be considered to include a subscription management set and a data decoding instance, and optionally may also include a packet mapping instance and the like. Optionally, in a decoding device, a service object may correspond to a media service instance, where the first decoding device may search for the media service instance of the first service object based on object location information of the first service object in the data packet to be parsed, and if so, directly obtain the media service instance corresponding to the first service object; if the media service instance of the first service object is not found, the media service instance of the first service object is constructed. Alternatively, a service scenario of a service object may correspond to a media service instance, that is, in a service scenario, typically listeners of a service object are obtained in the same way, or, alternatively, listeners of a service object are unchanged. For example, in a game application, if a service scenario corresponding to a first service object is a game team copy, a listener of the first service object is determined according to an object voice mode of the first service object, if the object voice mode of the first service object is a team voice mode, the listener of the first service object is a service object that is team with the first service object; if the object voice mode of the first service object is a full team voice mode, the listener of the first service object is all service objects in the service scene where the first service object is located, and so on. For example, the business scenario to which the first business object corresponds is a game map, the listener of the first business object is determined based on the location of the first business object in the game map, and so on.

Specifically, the first decoding device may search for a media service instance for the first service object based on the object location information of the first service object in the data packet to be parsed. If a media service instance aiming at the first service object is found, the media service instance is directly obtained, wherein the media service instance comprises a subscription management set and a data decoding instance, and optionally, the media service instance also comprises a packet mapping instance and the like. If the media service instance for the first service object is not found, then the media service instance for the first service object is created. Specifically, when a media service instance for a first service object is created, a subscription management set and a data decoding instance can be built for the first service object, and optionally, a package mapping instance can also be built. The subscription management set may be a linked list, a queue, or a list, such as a least recently used buffer (Least Recently Used Cache, LRUCache) linked list, which is not limited herein, and is used for managing subscriber information subscribed to the first service object, that is, information subscribed to the combination device of the first service object, that is, when the first decoding device decodes to obtain the decoded media data, the first decoding device sends the decoded media data to the corresponding combination device based on the subscription management set. The data decoding example is used for storing decoding related data of the media data by the first decoding device; the package mapping instance is used for managing related information of the media data. Further, the subscription management set and the data decoding instance may be combined into a media service instance for the first service object. Optionally, when there is a package mapping instance, the subscription management set, the data decoding instance, and the package mapping instance may be combined into a media service instance for the first service object. The subscription management set, the data decoding instance, the packet mapping instance and the like included in the media service instance are all changed, that is, the stored content is changed along with the operation of the first decoding device. Optionally, if one service scenario of one service object corresponds to one media service instance, the first decoding device may further generate a first service scenario of initial media data for the media service instance associated with the first service object.

Optionally, when the media service instance further includes a packet mapping instance and the data packet to be parsed further includes service information, the service information in the data packet to be parsed may also be added to the packet mapping instance, and optionally, the object location information of the first service object may also be added to the packet mapping instance. Optionally, the service information may further include, but is not limited to, a packet parsing sequence number (raw_client_data) of the data packet to be parsed, service transparent information (passthrough 2 so) and the like, where the service transparent information is used to indicate information related to transmission of the initial media data, such as a coding manner and a data transmission manner of the initial media data, and the data transmission manner is used to indicate a manner of transmitting the initial media data to the first decoding device.

Step S302, a media subscription message for a first service object sent by a first combined device is received, and the first combined device is added to a media service instance of the first service object based on the media subscription message.

In the embodiment of the present application, the first decoding device receives a media subscription message sent by the first combining device for the first service object, adds the first combining device to the media service instance of the first service object based on the media subscription message, specifically, may be based on the media subscription message, adds the first combining device to a subscription management set in the media service instance of the first service object, for example, the subscription management set is "combining device a— combining device B", and adds the first combining device to the media service instance, so as to obtain the subscription management set as "combining device a— combining device b— first combining device. Wherein the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device. The media subscription message is used for indicating the first decoding device, and the first combination device needs to acquire the related media data of the first service object, that is, subscribe to the first decoding device in advance, so that when the first decoding device generates the decoded media data, the first decoding device can send the decoded media data to the first combination device based on the subscription task generated by the media subscription message. That is, the first decoding device creates a subscription task for a combination device (referred to as a subscription object) included in the subscription management set, the subscription task being for triggering a process of transmitting the decoded media data to the subscription object when the first decoding device generates the decoded media data.

Step S303, decoding the initial media data in the data packet to be analyzed to obtain decoded media data, obtaining a subscription object from the media service instance, and sending the decoded media data to the subscription object.

In an embodiment of the application, the subscription object comprises a first combined device. Specifically, the first decoding device may decode the initial media data in the data packet to be parsed to obtain first decoded data, where the obtained media data is generally encoded, that is, the initial media data forwarded by the media forwarding device. Specifically, a common decoding manner may be adopted to decode the initial media data in the data packet to be parsed to obtain first decoded data; or, the encoding parameters can be obtained from the media forwarding device, and the encoding parameters are adopted to decode the initial media data to obtain first decoded data; or, a decoding mode corresponding to the negotiation coding method may be obtained, and the decoding mode is used to decode the initial media data to obtain the first decoded data, where the negotiation coding method and the decoding mode corresponding to the negotiation coding method refer to a coding and decoding method that the first decoding device and the media forwarding device negotiate in advance. Optionally, an encoding mode of the initial media data may be obtained from the packet mapping instance, and decoding processing may be performed on the initial media data based on the encoding mode to obtain first decoded data; or, the coding mode of the initial media data can be directly obtained from the data packet to be analyzed, and the initial media data is decoded to obtain first decoded data; or the first decoding device can carry out coding and decoding negotiation with the media forwarding device, and the initial media data is decoded through a coding and decoding negotiation result to obtain first decoded data; alternatively, the initial media data may be decoded by using an existing decoding method to obtain the first decoded data. Further, a decoding Buffer area (jitter_buffer) may be constructed in the data decoding example, the decoding processing procedure of the initial media data may be parsed, decoding auxiliary Buffer data for the initial media data may be generated, the decoding auxiliary Buffer data may be added to the decoding Buffer area, the decoding auxiliary Buffer data may be used for performing out-of-order processing, blank frame or packet filling processing, and the like on the first decoded data generated in the decoding processing procedure of the initial media data, and the decoding fault tolerance processing on the media data may be implemented through the decoding auxiliary Buffer data, so as to improve the decoding accuracy and fault tolerance of the media data, that is, the decoding auxiliary Buffer data may include, but is not limited to, the decoding sequence and abnormal decoding frame of the first decoded data, and the like, and may be used for performing supplementary explanation on the first decoded data. Optionally, one media data may correspond to one decoding buffer, and the first decoding device may construct a decoding buffer corresponding to the initial media data in the data decoding instance; and if the decoding buffer area is found, the decoding auxiliary buffer data can be directly determined as the decoding buffer area corresponding to the initial media data. Further, the first decoded data and the decoding auxiliary buffer data are combined into decoded media data. Alternatively, the first decoded data may be added to the decoding buffer.

Further, the subscription object corresponding to the first service object may be obtained from the subscription management set included in the media service instance, and the decoded media data may be sent to the subscription object, where the subscription object included in the subscription management set maintained by the first decoding device may be considered as a subscription task for the first service object, that is, a combination device that needs to obtain media data generated by the first service object, by this way, only one decoding needs to be performed, and the decoded media data may be sent to the combination device that needs to obtain the media data, so that data processing efficiency may be improved. Optionally, when the decoded media data is obtained, media related data associated with the initial media data may be obtained from the packet mapping instance based on the service information, specifically, the media related data may be obtained based on the packet parsing sequence number in the service information, and the media related data is sent to the subscription object. Wherein the media association data may include, but is not limited to, service information of the first service object. Specifically, the first decoding device may obtain a packet parsing sequence number corresponding to the initial media data, obtain, based on the packet parsing sequence number, object location information of the media related data and the first service object from the packet mapping instance, encapsulate the decoded media data, the media related data and the object location information of the first service object into a decoded data packet, and send the decoded data packet to the subscription object obtained from the media service instance. By sending the object position information of the first service object, the media associated data and the like to the subscription object, the subscription object can acquire relevant information corresponding to the decoded media data, further the decoded media data can be further processed based on the relevant information, and the efficiency and the accuracy of data processing are improved.

Optionally, an object subscription duration of the subscription object may be obtained, and the first subscription object whose object subscription duration is greater than or equal to the subscription maintenance time threshold may be deleted, and optionally, a subscription failure message for the first service object may be sent to the first subscription object. Or, the subscription period associated with the subscription object may be acquired, the second subscription object may be deleted, and optionally, a subscription expiration message for the first service object may be sent to the second subscription object; the second subscription object refers to a subscription object with a subscription period less than or equal to the system network time, for example, the subscription period of the subscription object a is No. 3 months 13, and when the system network time reaches No. 3 months 13, the subscription object a is deleted. Or when the abnormal reminding message for the third subscription object is received, deleting the third subscription object, optionally, sending a subscription invalidation message for the first service object to the third subscription object, and the like. Optionally, any one or more modes can be adopted to manage the subscription objects, when deleting the subscription objects, an equipment exception message can be sent to the deleted subscription objects (such as the first subscription object or the second subscription object, etc.), through the above processes, management of the subscription objects can be realized, and the expired subscription objects can be deleted, so that not only can the storage space be saved, but also data can be prevented from being sent to irrelevant combined equipment, the network transmission quantity is reduced, the data transmission efficiency is improved, and further the data processing efficiency is improved.

Further, referring to fig. 4, fig. 4 is a schematic diagram of a data forwarding process according to an embodiment of the present application. As shown in fig. 4, the method embodiment takes the media forwarding device as an execution body, and specifically, the process may include the following steps:

step S401, receiving initial media data generated by the first service object, and determining a first combination device corresponding to the initial media data.

In the embodiment of the present application, the media forwarding device may receive initial media data generated by the first service object, and specifically may acquire initial media data generated and sent by the first service object through the first service client. Further, first media data associated with the initial media data can be acquired, and a combination device corresponding to the first media data is searched; the first media data refers to media data that is combined with the initial media data, that is, media data that needs to be fused with the initial media data, or jointly processed, that is, the first media data is media data that needs to be fused with the initial media data into one piece of media data. Optionally, the first service identifier associated with the initial media data may be obtained, and the first media data corresponding to the first service identifier is obtained, that is, the media data generated in the same service scene may be processed by the same combination device, so that when the combination device subsequently combines multiple paths of media data, the required media data may be obtained at one time, and the efficiency of data processing is improved. Specifically, if the combined device corresponding to the first media data is found, determining the combined device corresponding to the first media data as the first combined device; if the combination device corresponding to the first media data is not found, the initial media data can be processed by any combination device, and the first combination device can be obtained. Or, the first service identifier associated with the initial media data may be directly obtained, the combination device corresponding to the first service identifier is searched, and if the combination device corresponding to the first service identifier is found, the combination device corresponding to the first service identifier is determined to be the first combination device; and if the combined equipment corresponding to the first service identifier is not found, acquiring the first combined equipment. That is, if there is already a processing path of the media data in the service scenario, the existing processing path of the media data can be directly multiplexed, so as to improve the data processing efficiency.

When the first combination device is acquired, one combination device can be randomly selected from M combination devices to serve as the first combination device; or, the first task number respectively associated with the M combining devices may be acquired, and the first combining device is determined from the M combining devices based on the first task number respectively associated with the M combining devices, for example, a combining device with the minimum first task number may be determined as the first combining device, and the first task number may be used to represent the number of service identifiers associated with the combining device, and the like; alternatively, first load information corresponding to each of the M combination devices may be acquired, the first combination device may be determined from the M combination devices based on the first load information corresponding to each of the M combination devices, and the like. Wherein M is a positive integer. The first load information may include, but is not limited to, a maximum combination path number max_availability of the corresponding combination device, a current processing path number cur_process, a unit increase path number each_list_in_create corresponding to a single listener, a bandwidth load, and the like, where the maximum combination path number is used to represent a maximum number of media data that the corresponding combination device can process simultaneously, such as a maximum processed mix path number; the current processing route number is used for representing the quantity of the media data currently processed by the corresponding combined equipment; the unit increasing the number of ways refers to increasing the number of ways of media data which a listener may increase in a service scene corresponding to the current service identifier, and the like. Specifically, the media forwarding device may determine a combination device with a minimum bandwidth load as a first combination device; or, the processing occupancy rate corresponding to each of the M combination devices may be determined based on the current processing route number and the maximum processing route number corresponding to each of the M combination devices, and the combination device with the smallest processing occupancy rate is determined as the first combination device, where for the processing occupancy rate corresponding to any one combination device=the current processing route number/the maximum processing route number; or, based on the current processing path number and the maximum processing path number corresponding to the M combination devices respectively, determining an idle processing space corresponding to the M combination devices respectively, and determining the combination device with the maximum idle processing space as the first combination device, wherein for the idle processing space = maximum combination path number-current processing path number corresponding to any combination device; alternatively, the spatial expansion rates corresponding to the M combining devices may be determined based on the current processing path number, the maximum combining path number, and the unit increase path number corresponding to the M combining devices, respectively, and the combining device having the largest spatial expansion rate may be determined as the first combining device, or the like, where the spatial expansion rate= (maximum combining path number-current processing path number)/unit increase path number of any one combining device, or the like.

The media forwarding device may obtain, when needed, load information corresponding to the M combination devices from the M combination devices, respectively. Alternatively, the load information that is newly transmitted by the M combined devices may be directly acquired based on periodic heartbeat detection of the M combined devices. Alternatively, an association relationship between the first service identifier and the first combining device, such as "first service identifier-first combining device", may be stored.

Step S402, the first decoding device is obtained, and the data packet to be parsed generated based on the initial media data is sent to the first decoding device, so that the first decoding device constructs a media service instance for the first service object.

In the embodiment of the application, the media forwarding device can acquire the first decoding device, specifically, can acquire the device loads corresponding to the N decoding devices respectively, and determines the decoding device with the smallest device load as the first decoding device, so that the more available processing resources are provided for the decoding device with smaller load, the more data decoding efficiency can be improved; n is a positive integer. Or determining the decoding device associated with the object position information of the first service object as the first decoding device, so that the same decoding device is adopted for decoding the same service object, and at the moment, only one decoding device is needed for the same service object, one media service instance is maintained, so that the data quantity required to be stored can be reduced, and the data storage space is saved. Or, the first decoding device is selected randomly from the N decoding devices, so that the efficiency of obtaining by the decoding devices can be improved. The first decoding device may be obtained in any of the ways described above.

Optionally, the decoding device associated with the first service object may be searched, and if the decoding device associated with the first service object is found, the decoding device associated with the first service object is determined to be the first decoding device; and if the decoding equipment associated with the first business object is not found, acquiring the first decoding equipment. Or, the decoding device associated with the service scene of the first service object can be searched, and if the decoding device associated with the service scene of the first service object is searched, the decoding device associated with the service scene of the first service object is determined to be the first decoding device; and if the decoding equipment associated with the business scene of the first business object is not found, acquiring the first decoding equipment. By acquiring the first decoding device in the above manner, the same service object or the same service scene of the same service object can be multiplexed by the same decoding device, namely, the processing path is multiplexed, so that the media service instance of the same service object or the same service scene of the same service object can be maintained by the same decoding device, thereby saving the storage space and improving the data processing efficiency.

When the required decoding device is not found and the first decoding device is required to be acquired, one decoding device can be randomly selected from the N decoding devices to serve as the first decoding device. Alternatively, the second task number respectively associated with the N decoding devices may be acquired, and the first decoding device may be determined from the N decoding devices, where the second task number is used to represent the number of service objects associated with the corresponding decoding device, or, taking the service object as an example, assuming that "decoding device 1-service object a and service object B" exist, the second task number representing the decoding device 1 is 2. Or, second load information corresponding to each of the N decoding devices may be obtained, and the first decoding device is determined based on the second load information corresponding to each of the N decoding devices, where the second load information may include, but is not limited to, a maximum decoding path number of the corresponding decoding device, a current decoding path number, a bandwidth load, and the like, where the maximum decoding path number is used to represent a maximum number of media data that the corresponding decoding device may decode simultaneously, that is, at most, how many media data may be decoded simultaneously; the current decoding pass number is used to represent the number of media data currently being decoded by the corresponding decoding apparatus, and the like. Specifically, a decoding device with the smallest bandwidth load may be determined as the first decoding device; or, the decoding occupancy rate corresponding to the N decoding devices respectively may be determined based on the current decoding path number and the maximum decoding path number corresponding to the N decoding devices respectively, and the decoding device with the smallest decoding occupancy rate is determined as the first decoding device, which refers to the processing occupancy rate of the combining device; alternatively, the spatial decoding space corresponding to each of the N decoding apparatuses may be determined based on the current decoding path number and the maximum decoding path number corresponding to each of the N decoding apparatuses, the decoding apparatus having the largest free decoding space may be determined as the first decoding apparatus, see the free processing space of the combining apparatus, and the like. Through the above process, parameters (such as the second load information) which affect the decoding efficiency of the decoding device and the like can be adopted to determine the currently required decoding device, so that the decoding processing efficiency can be improved to a certain extent. Optionally, after determining the first decoding device, an association relationship between the first service object and the first decoding device may be stored, for example, "the first service object is the first decoding device", or an association relationship between a service scene of the first service object and the first decoding device may be stored, for example, "the service scene of the first service object is the first decoding device", etc., where the specifically stored association relationship may be determined based on the condition of searching for the first decoding device, for example, the association relationship between the first service object and the first decoding device may be stored if the first decoding device is searched for based on the first service object.

Further, object position information of the first service object can be obtained, the object position information of the first service object and the initial media data are packaged, a data packet to be analyzed is obtained, and the data packet to be analyzed is sent to the first decoding device. Optionally, media associated data corresponding to the initial media data may also be obtained; the media related data is used for representing related information of a service generating initial media data, such as service information corresponding to the initial media data; and encapsulating the object position information, the initial media data and the media related data of the first service object into a data packet to be analyzed, and sending the data packet to be analyzed to the first decoding equipment so that the first decoding equipment decodes the data packet to be analyzed to obtain the needed media data.

Step S403, the service address of the first decoding device is sent to the first combining device, so that the first combining device sends the media subscription message to the first decoding device based on the service address.

In the embodiment of the present application, the service address of the first decoding device and other information of the first service object may be sent to the first combining device, where the other information of the first service object may be regarded as information other than the information sent to the first decoding device in the information of the first service object acquired by the media forwarding device. The media subscription message is used for the first decoding device to send the decoded media data obtained by decoding the initial media data to the first combining device, and specifically refer to the related description shown in step S302 in fig. 3.

Optionally, the media forwarding device may perform periodic heartbeat detection on the N decoding devices and the M combining devices, where N is a positive integer, and M is a positive integer, and if heartbeat requests are sent to the N decoding devices and the M combining devices respectively every 10 s. The heartbeat detection refers to sending a simple communication packet according to a certain waiting time, so as to determine whether an object receiving the communication packet operates normally. Specifically, if abnormal equipment exists in the N decoding equipment and the M combined equipment based on the periodic heartbeat detection, deleting the abnormal equipment; an abnormal device refers to a device that continuously does not receive the number of heartbeat replies for periodic heartbeat detection, and is greater than or equal to a heartbeat abnormality threshold, for example, assuming that the heartbeat abnormality threshold is 3 times, the device is considered to be an abnormal device if the heartbeat replies of one device are not received 3 times continuously. Optionally, the heartbeat reply may include load information of the devices, e.g., the heartbeat reply of any one combination device may include first load information of the combination device, the heartbeat reply of any one decoding device may include second load information of the decoding device, etc. Further, if the abnormal device is a combined device, it indicates that subscription tasks corresponding to the respective decoding devices are correspondingly invalid, and a second decoding device associated with the abnormal device may be obtained, and a first abnormal reminding message of the abnormal device is sent to the second decoding device, so that the second decoding device deletes subscription information of the abnormal device based on the first abnormal reminding message, where the second decoding device refers to that a media service instance including the abnormal device is maintained, and specifically, when deleting subscription information of the abnormal device, may be deleting the abnormal device in a subscription management set including the media service instance of the abnormal device; if the abnormal equipment is decoding equipment, the abnormal equipment cannot normally perform the decoding process, so that the combined equipment which receives the service address of the abnormal equipment does not need to continuously maintain the service address, specifically, a second combined equipment associated with the abnormal equipment is obtained, a second abnormal reminding message of the abnormal equipment is sent to the second combined equipment, the second combined equipment deletes the service address of the abnormal equipment based on the second abnormal reminding message, and the second combined equipment is the combined equipment which maintains the service address of the abnormal equipment. For example, after detecting that the first combined device is an abnormal device after one heartbeat, the first decoding device may send a first abnormality alert message of the first combined device to the first decoding device, and based on the first abnormality alert message, the first decoding device may delete the first combined device in the subscription management set including the first combined device, and so on.

Specifically, referring to fig. 5, fig. 5 is a schematic diagram of an apparatus interaction structure provided by an embodiment of the present application, and as shown in fig. 5, apparatuses related to the embodiment of the present application include, but are not limited to, a service client (such as a service client a and a service client B), a media forwarding apparatus, a location management apparatus, a combination apparatus, and a decoding apparatus. The service client can perform data uplink aiming at the media forwarding device, namely, upload the generated initial media data to the media forwarding device; the media forwarding device may report the location of each service client, and when media data needs to be sent, the location management device requests a listener, and the location management device issues an object based on the request. Further, the media forwarding device may determine a combination device and a decoding device for processing media data, send the media data to be decoded to the decoding device, and send the determined service address of the decoding device to the combination device; the decoding device may perform decoding processing on the received media data to obtain decoded data; the combination device may perform a subscription request to a corresponding decoding device based on the acquired service address of the decoding device, and when the decoding device obtains the decoded data, the decoding device may send the decoded data to the corresponding combination device based on the subscription request.

Specifically, referring to fig. 6, fig. 6 is a schematic diagram of a data processing interaction flow provided in an embodiment of the present application. As shown in fig. 6, the process may include the steps of:

in step S601, object location information of the service object is reported.

In the embodiment of the application, the object coordinates and the region division sizes of the service objects are obtained, and the object coordinates of the service objects are subjected to coordinate conversion based on the region division sizes to obtain the object position information of the service objects; the business object comprises a first business object. The object coordinates refer to the collected position coordinates of the service object, and the area division size refers to the size of the divided management area when the object coordinates of the service object are managed. Specifically, the map may be divided into regions with the same interval r, the object coordinates of the service object are mapped into the regions, for example, mapped to the lower left corner of the region where the service object is located, and the coordinates of the lower left corner of the region where the service object is located are determined as the object position information of the service object. Where r is the area division size. For example, referring to fig. 7, fig. 7 is a schematic view of a position acquisition scene provided by an embodiment of the present application, as shown in fig. 7, the media forwarding device may acquire the object coordinates of the service object 7011, and perform coordinate conversion on the object coordinates of the service object based on the region division size r to obtain the object position information 7021 of the service object; the object coordinates of the business object 7012 are acquired, and the object coordinates of the business object are subjected to coordinate conversion based on the region division size r, so that object position information 7022 of the business object and the like are obtained. In this way, object position information of any one business object can be acquired, and can be denoted as (x, y, z, r). Further, hash processing may be performed on object position information of the service object, a position management device corresponding to the service object may be determined, for example, summing residual processing is performed on object position information of the service object, a position index may be determined, a position management device corresponding to the position index is determined to correspond to the service object, for example, summing residual processing may be performed on object position information of the service object based on the number h of devices of the position management device, and a position index, for example, a position index= (x+y+z)% h may be determined; alternatively, the object position information of the business object may be summed up and the remainder processed based on the map size, the position index may be determined, etc.; alternatively, the object position information of the business object may be subjected to a summation modulo process to determine a position index, e.g., position index= |x+y+z|, etc. Of course, other hash algorithms may be used to hash the object position information of the service object, to determine the position management device corresponding to the service object.

Alternatively, the media forwarding device may determine, based on the map dividing dimension and the position uplink interval, a position management device associated with the object position information of the service object as the position management device corresponding to the service object. The map dividing dimension refers to a dimension for dividing the map into areas, namely, a dimension of the areas, and if the map is divided into the areas by a two-dimensional space, the map dividing dimension is 2; if the map is divided into areas in a three-dimensional space, the map division dimension is 3 or the like. The position uplink interval refers to the maximum interval area number between the position management device and the object position information to be sent when the object position information of the service object is reported, and optionally, the position uplink interval may be determined according to a media transmission distance, for example, if the position uplink interval is 1, the position uplink interval indicates that the position management device corresponding to the object position information of the service object and the position management device adjacent to the object position information are determined as the position management device corresponding to the service object. For example, assuming that the map division dimension is 3 and the position uplink interval is 1, the position management device corresponding to the business object includes position management devices respectively corresponding to 27 areas centered on the position management device of the object position information of the business object. For example, assuming that the map division dimension is 2 and the position uplink interval is 1, referring to fig. 8, fig. 8 is a schematic diagram of an optional position maintenance scenario provided by an embodiment of the present application. As shown in fig. 8, it is assumed that there are a business object 8011 and a business object 8012, and the position management device corresponding to the business object 8011 is nine position management devices centered on the position management device corresponding to the object position information 8021 of the business object 8011, as indicated by a circle "o" in fig. 8; the position management device corresponding to the service object 8012 is nine position management devices centering on the position management device corresponding to the object position information 8022 of the service object 8012, as indicated by a circle "Δ" in fig. 8.

Through the above process, the position management device corresponding to any one business object can be determined. Further, the media forwarding device may send the object location information of the service object to the location management device corresponding to the service object, for example, send the object location information of the service object to the location management device corresponding to the service object through a positioning signal (lbssignal). In the method shown in fig. 8, each of the position management devices stores an object set, taking the position management devices 8a, 8b, and 8c as an example, the object set 8031 in the position management device 8a includes "business object 8011— business object 8012", the object set 8032 in the position management device 8b includes "business object 8012", and the object set 8033 in the position management device 8c includes "business object 8011". Alternatively, the coordinates of any one location management device may be used as a key (key), and the service object included in the object set may be regarded as a value (value) of the key.

Further, upon receiving the initial media data generated by the first service object, step S602 is performed to send a reception determination request to the first location management device corresponding to the first service object, so that the first location management device determines, based on the reception determination request, the second service object that receives the initial media data. The first position management device is used for representing a position management device for managing a first service object, and the second service object is a service object capable of receiving initial media data sent by the first service object.

Step S602, a transmission-reception determination request.

In the embodiment of the present application, the media forwarding device may acquire a first location management device corresponding to the first service object, and send a reception determination request (lbs data) to the first location management device. The receiving determining request may include a first service identifier corresponding to the first service object.

Step S603, return to the second business object.

In an embodiment of the present application, the first location management device may determine the second service object receiving the initial media data based on the reception determination request. Specifically, when the hash processing is performed on the object position information of the service object and the position management device corresponding to the service object is determined, the first position management device may obtain, based on the object position information of the first service object, a second service object that may receive media data of the first service object. For example, a second business object in the same business scenario as the first business object may be acquired based on the first business identification. Alternatively, in the game scenario, the first location management device may acquire the game location of the first business object in the game scenario, acquire the second business object located within a range centered on the game location of the first business object in the game scenario and having a receivable distance as a size, that is, in the game scenario, if the business object transmits a message in a nearby channel or a current channel or the like, only other business objects located in the vicinity of the business object can acquire the message transmitted by the business object, and thus, the second business object may be acquired based on the principle.

Or, when determining the position management devices of the service objects based on the map dividing dimension and the position uplink interval, the number of the first position management devices is a plurality of, and the first position management device may determine the service objects except the first service object in the managed object set as the second service object.

Further, the first location management device may send the second service object to the media forwarding device, that is, the media forwarding device may obtain the second service object from the first location management device, send the second service object to the first combining device, so that the first combining device sends a message to the second service object based on the decoded media data when receiving the decoded media data, where the process of determining the first combining device may refer to the related description in step S401 of fig. 4, and further, step S604 is performed.

Step S604, send the data packet to be resolved.

In the embodiment of the present application, the media forwarding device may acquire the first decoding device, and send the data packet to be parsed to the first decoding device, which may specifically refer to the description related to step S402 in fig. 4, and is not limited herein.

Step S605, media decoding information is transmitted.

In an embodiment of the present application, the media forwarding device may send media decoding information to the first combining device, where the media decoding information may include, but is not limited to, a service address of the first decoding device and other information of the first service object, and specifically refer to the related description in step S403 in fig. 4. Wherein the media decoding information may be a member structure, and the other information of the first service object may include, but is not limited to, a first service identifier (app id), a room name (roommeme), a room identifier (roommed), a speaker (speakers), a listener (listeners), and the like, wherein the room name and the room identifier are used for distinguishing services for transmitting media data; a speaker is used to represent a business object that produces media data, referred to herein as a first business object; a listener is used to represent a business object, referred to herein as a second business object, that can receive the media data.

Step S606, a media subscription message is sent.

In the embodiment of the application, the first combined device can send the media subscription message to the first decoding device based on the service address of the first decoding device.

In step S607, the decoding process is performed on the initial media data.

In the embodiment of the application, the first decoding device can analyze the data packet to be analyzed to obtain the decoded media data. Specifically, decoding is performed on initial media data in a data packet to be analyzed to obtain decoded media data, the data packet to be analyzed is analyzed to obtain media related data of a first service object, and the decoded media data, the media related data, and the like are packaged into a decoded data packet. See for details the relevant description in step S303 of fig. 3.

Step S608, the decoded media data is transmitted.

In the embodiment of the present application, the first decoding device may send the decoded data packet to the first combining device. The first combining device may determine to-be-combined media data to be combined based on media associated data in the decoding data packet, and the like, perform combining processing on the to-be-combined media data to obtain combined media data, and send the combined media data to the second service object, where the to-be-combined media data includes the decoding media data. For example, the media data to be combined is audio data, and the first combining device may perform a mixing process on the media data to be combined to obtain combined media data, where the combined media data is one-way data, and the mixing process refers to a process of integrating multiple sources of sound (i.e., the media data to be combined) into one stereo audio track or one single audio track, so that multiple ways of media data are integrated into one-way media data.

Optionally, when the media data is combined, the media data in the same service scene that is received simultaneously may be combined, that is, the number of the first service objects may be multiple, and the initial media data of the multiple first service objects may be sent to the same decoding device and the same combining device to be processed, so that the first combining device may combine the decoded media data corresponding to the initial media data of the multiple first service objects to obtain combined media data, that is, at this time, the media data to be combined includes decoded media data corresponding to the multiple first service objects respectively. The application can also be applied to game applications, wherein the map refers to a game map in the game application, and the object position refers to the position coordinates of the business object in the game map. For example, the number of the first service objects is k, where k is a positive integer, and in the game application, the k first service objects send the initial media data at the same time, and the media forwarding device may determine the first decoding devices corresponding to the k initial media data respectively, see, specifically, the description related to step S402 of fig. 4, where the first decoding devices corresponding to the k initial media data respectively may have the same decoding device or may be different from each other. Further, since the k initial media data need to be combined, the first combining device corresponding to the k initial media data may be determined to be the same. The media forwarding device may generate k data packets to be parsed corresponding to the initial media data respectively, send the k data packets to be parsed to the corresponding first decoding devices respectively, and send service addresses of the first decoding devices corresponding to the k data packets to be parsed to the first combining device respectively. The first combination device can send media subscription information to first decoding devices corresponding to k data packets to be analyzed respectively; each first decoding device may generate a subscription task of the first combining device based on the media subscription message, specifically referring to step S302 of fig. 3, and when decoding to obtain decoded media data, send the obtained decoded media data to the first combining device. The first combining device may combine the k decoded media data to obtain combined media data, and may output the combined media data in the gaming application.

Through the above process, the cooperation between different devices is realized, and the different devices can independently operate, so that the multi-process synchronous processing can be realized, and the data processing efficiency is improved. And through the subscription-release mode, each device can maintain the historical processing condition of the device, such as a data transmission path and the like, and maintain the current data transmission path of the device, namely, which device is used for acquiring data, and the processed data is sent to which device and the like, so that the media data only needs to be decoded once and can be directly sent to the device needing to acquire the media data, and the efficiency and the accuracy of data processing are improved. Even when the large-scale spatial audio mixing process is performed, that is, the number of the first service objects is a plurality, the initial media data of each first service object which needs to be subjected to the mixing process can be based on the media forwarding device, and k first service objects are assumed, k is a positive integer, through the steps, the initial media data of each first service object is sent to the corresponding first decoding device, and the service address of the first decoding device is sent to the corresponding first combination device, for example, the media forwarding device determines that the initial media data of the first service object 1 corresponds to the first decoding device 1 and the first combination device 1, the initial media data of the first service object 2 corresponds to the first decoding device 2 and the second combination device 2, …, and the data packet to be analyzed generated according to the initial media data of the first service object 1 can be sent to the first decoding device 1 and the service address of the first combination device 1; the data packet to be analyzed generated according to the initial media data of the first service object 2 is sent to the first decoding device 2, the service address of the first decoding device 2 is sent to the first combining device 2 and the like, so that each decoding device and the combining device can work separately and cooperate, and based on a publish-subscribe mode, each decoding device can send the decoded media data to the corresponding combining device directly based on a maintained media service instance when decoding to obtain the decoded media data, thereby realizing the directional transmission of the data and improving the regularity and efficiency of the data transmission.

Further, referring to fig. 9, fig. 9 is a schematic diagram of a data processing apparatus according to an embodiment of the application. The data processing means may be a computer program (comprising program code etc.) running in a computer device, for example the data processing means may be an application software; the device can be used for executing corresponding steps in the method provided by the embodiment of the application. As shown in fig. 9, the data processing apparatus 900 may be used in the computer device in the embodiment corresponding to fig. 3, and specifically, the apparatus 900 may include: a data receiving module 11, an instance acquiring module 12, a message receiving module 13, a device adding module 14, a data decoding module 15, an object determining module 16 and a decoding transmitting module 17.

The data receiving module 11 is configured to receive a data packet to be parsed sent by the media forwarding device;

an instance obtaining module 12, configured to obtain a media service instance for a first service object based on object location information of the first service object in the data packet to be parsed;

a message receiving module 13, configured to receive a media subscription message for a first service object sent by a first combination device;

a device addition module 14 for adding the first combined device to the media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device;

The data decoding module 15 is configured to decode the initial media data in the data packet to be parsed to obtain decoded media data;

an object determining module 16, configured to obtain a subscription object from a media service instance;

a decode-and-send module 17 for sending the decoded media data to a subscription object; the subscription object includes a first combined device.

Wherein the instance acquisition module 12 comprises:

an instance searching unit 121, configured to search a media service instance for a first service object based on object location information of the first service object in a data packet to be parsed;

a data construction unit 122, configured to construct a subscription management set and a data decoding instance for the first service object if the media service instance for the first service object is not found; the subscription management set is used for managing the combined equipment for sending the media subscription message;

an instance combining unit 123, configured to combine the subscription management set with the data decoding instance to form a media service instance for the first service object.

The device adding module 14 is specifically configured to:

the data decoding module 15 includes:

A decoding processing unit 151, configured to decode the initial media data in the data packet to be parsed to obtain first decoded data;

the decoding buffer unit 152 is configured to construct a decoding buffer in the data decoding instance, analyze a decoding process of the initial media data, and generate decoding auxiliary buffer data for the initial media data;

the decoding generation unit 153 is configured to combine the first decoded data and the decoding auxiliary buffer data to form decoded media data.

Wherein, this device 900 still includes:

a first deleting module 18, configured to obtain an object subscription duration of the subscription object, delete a first subscription object whose object subscription duration is greater than or equal to a subscription maintenance time threshold, and send a subscription failure message for the first service object to the first subscription object; or,

a second deleting module 19, configured to acquire a subscription period associated with the subscription object, delete the second subscription object, and send a subscription invalidation message for the first service object to the second subscription object; the second subscription object refers to a subscription object having a subscription period less than or equal to the system network time.

The apparatus 900 further comprises:

the service adding module 20 is configured to add service information in the data packet to be parsed to the packet mapping instance;

a data transmitting module 21, configured to acquire media related data associated with the initial media data based on the service information and transmit the media related data to the subscription object when obtaining the decoded media data; the media-associated data includes service information.

The embodiment of the application provides a data processing device which can receive a data packet to be analyzed sent by media forwarding equipment and acquire a media service instance aiming at a first service object based on object position information of the first service object in the data packet to be analyzed; receiving a media subscription message for a first service object sent by a first combined device, and adding the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device; decoding the initial media data in the data packet to be analyzed to obtain decoded media data, acquiring a subscription object from the media service instance, and transmitting the decoded media data to the subscription object; the subscription object includes a first combined device. Through the above process, various devices can be adopted to perform data interaction, and based on subscription conditions, the first decoding server can send the received data packet to be analyzed to the combined device which needs to receive the decoding result only by once coding, so that the data decoding efficiency is improved. In addition, the data processing process is distributed to a plurality of devices, and the processes such as data decoding and data mixing can be used for large-scale spatial audio mixing processing, so that the universality and the efficiency of the data processing are improved.

Further, referring to fig. 10, fig. 10 is a schematic diagram of another data processing apparatus according to an embodiment of the present application. The data processing means may be a computer program (comprising program code etc.) running in a computer device, for example the data processing means may be an application software; the device can be used for executing corresponding steps in the method provided by the embodiment of the application. As shown in fig. 10, the data processing apparatus 1000 may be used in the computer device in the embodiment corresponding to fig. 4, and specifically, the apparatus may include: the device comprises a media acquisition module 31, a combination determination module 32, a decoding acquisition module 33, a sending module 34 to be analyzed and an address sending module 35.

A media acquisition module 31, configured to receive initial media data generated by the first service object;

a combination determining module 32, configured to determine a first combination device corresponding to the initial media data;

a decoding acquisition module 33, configured to acquire a first decoding device;

a to-be-parsed sending module 34, configured to send a to-be-parsed data packet generated based on the initial media data to the first decoding device, so that the first decoding device constructs a media service instance for the first service object;

An address sending module 35, configured to send a service address of the first decoding device to the first combining device, so that the first combining device sends a media subscription message to the first decoding device based on the service address; the media subscription message is used for the first decoding device to send the decoded media data obtained by decoding the initial media data to the first combining device.

Wherein the combination determination module 32 comprises:

a combination searching unit 321, configured to obtain first media data associated with the initial media data, and search a combination device corresponding to the first media data; the first media data refers to media data combined with the initial media data;

a combination obtaining unit 322, configured to determine, if a combination device corresponding to the first media data is found, the combination device corresponding to the first media data as a first combination device;

the combination obtaining unit 322 is further configured to obtain a first combination device if the combination device corresponding to the first media data is not found.

Wherein the decoding acquisition module 33 includes:

a first obtaining unit 331, configured to obtain device loads corresponding to the N decoding devices, and determine a decoding device with a minimum device load as a first decoding device; n is a positive integer; or,

A second obtaining unit 332, configured to determine, as a first decoding device, a decoding device associated with object location information of the first service object; or,

the third obtaining unit 333 is configured to randomly select the first decoding device from the N decoding devices.

The sending module to be parsed 34 includes:

an information obtaining unit 341, configured to obtain object location information of a first service object, and obtain media association data corresponding to initial media data; the media related data is used for representing related information of a service generating the initial media data;

the data encapsulation unit 342 is configured to encapsulate the object location information, the initial media data, and the media related data of the first service object into a data packet to be parsed, and send the data packet to be parsed to the first decoding device.

Wherein the apparatus 1000 further comprises:

the coordinate conversion module 36 is configured to obtain an object coordinate and a region division size of the service object, and perform coordinate conversion on the object coordinate of the service object based on the region division size to obtain object position information of the service object; the business objects include a first business object;

a hash processing module 37, configured to perform hash processing on object location information of a service object, and determine a location management device corresponding to the service object;

An object request module 38, configured to, when receiving initial media data generated by the first service object, send a reception determination request to a first location management device corresponding to the first service object, so that the first location management device determines, based on the reception determination request, a second service object that receives the initial media data;

the object sending module 39 is configured to obtain the second service object from the first location management device, and send the second service object to the first combining device, so that when the first combining device receives the decoded media data, the first combining device sends a message to the second service object based on the decoded media data.

Wherein the apparatus 1000 further comprises:

the heartbeat detection module 40 is configured to perform periodic heartbeat detection on the N decoding devices and the M combination devices respectively;

the device deleting module 41 is configured to delete an abnormal device if it is detected that an abnormal device exists in the N decoding devices and the M combined devices based on the periodic heartbeat detection; the abnormal device refers to a device which continuously does not receive the number of times of heartbeat replies aiming at periodic heartbeat detection and is larger than or equal to a heartbeat abnormal threshold value;

the subscription deleting module 42 is configured to acquire a second decoding device associated with the abnormal device if the abnormal device is a combined device, and send a first abnormal reminding message of the abnormal device to the second decoding device, so that the second decoding device deletes subscription information of the abnormal device based on the first abnormal reminding message;

The address deleting module 43 is configured to obtain a second combination device associated with the abnormal device if the abnormal device is a decoding device, and send a second abnormality alert message of the abnormal device to the second combination device, so that the second combination device deletes the service address of the abnormal device based on the second abnormality alert message.

Through the above process, the process realized by the media forwarding device is described, and the media forwarding device is used for forwarding the media data, so that the arrival and departure of the media data can be determined, and the operation of the decoding device and the combined device is coordinated, so that the existing data, such as the media service instance maintained in the decoding device, and the media data forwarding path maintained by the media forwarding device, can be multiplexed as much as possible, the collaboration among different devices is improved, and the efficiency and accuracy of data processing are improved.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in fig. 11, the computer device in the embodiment of the present application may include: one or more processors 1101, memory 1102, and an input-output interface 1103. The processor 1101, memory 1102, and input-output interface 1103 are connected by a bus 1104. The memory 1102 is used for storing a computer program, where the computer program includes program instructions, and the input/output interface 1103 is used for receiving data and outputting data, for example, for data interaction between a decoding device and a combination device, or for data interaction between a service client and a media forwarding device, or for data interaction between the media forwarding device and a location management device, or for data interaction between the media forwarding device and the combination device or the decoding device; the processor 1101 is configured to execute program instructions stored in the memory 1102.

Wherein the processor 1101, when integrated in a decoding device, may perform the following operations:

Wherein the processor 1101, when integrated in a media forwarding device, may perform the following operations:

In some possible embodiments, the processor 1101 may be a central processing unit (central processing unit, CPU), which may also be other general purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), off-the-shelf programmable gate arrays (field-programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 1102 may include read only memory and random access memory, and provides instructions and data to the processor 1101 and input output interface 1103. A portion of memory 1102 may also include non-volatile random access memory. For example, memory 1102 may also store information of device type.

In a specific implementation, the computer device may execute, through each functional module built in the computer device, an implementation manner provided by each step in fig. 3 or fig. 4, and specifically, the implementation manner provided by each step in fig. 3 or fig. 4 may be referred to, which is not described herein again.

An embodiment of the present application provides a computer device, including: the processor, the input/output interface and the memory acquire the computer program in the memory through the processor, execute the steps of the method shown in fig. 3, and perform the data processing operation. The embodiment of the application realizes that the data packet to be analyzed sent by the media forwarding equipment is received, and the media service instance aiming at the first service object is obtained based on the object position information of the first service object in the data packet to be analyzed; receiving a media subscription message for a first service object sent by a first combined device, and adding the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device; decoding the initial media data in the data packet to be analyzed to obtain decoded media data, acquiring a subscription object from the media service instance, and transmitting the decoded media data to the subscription object; the subscription object includes a first combined device. Through the above process, various devices can be adopted to perform data interaction, and based on subscription conditions, the first decoding server can send the received data packet to be analyzed to the combined device which needs to receive the decoding result only by once coding, so that the data decoding efficiency is improved. In addition, the data processing process is distributed to a plurality of devices, and the processes such as data decoding and data mixing can be used for large-scale spatial audio mixing processing, so that the universality and the efficiency of the data processing are improved.

The embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, where the computer program is adapted to be loaded by the processor and execute the data processing method provided by each step in fig. 3 or fig. 4, and specifically refer to an implementation manner provided by each step in fig. 3 or fig. 4, which is not described herein again. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium according to the present application, please refer to the description of the method embodiments of the present application. As an example, a computer program may be deployed to be executed on one computer device or on multiple computer devices at one site or distributed across multiple sites and interconnected by a communication network.

The computer readable storage medium may be a data processing apparatus provided in any of the foregoing embodiments or an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card (flash card) or the like, which are provided on the computer device. Further, the computer-readable storage medium may also include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and executes the computer instructions, so that the computer device executes the methods provided in the various alternative modes in fig. 3 or fig. 4, data interaction by adopting various devices is realized, and based on subscription conditions, the first decoding server only needs to encode the received data packet to be analyzed once and can send the data packet to the combined device which needs to receive the decoding result, thereby improving the efficiency of data decoding. In addition, the data processing process is distributed to a plurality of devices, and the processes such as data decoding and data mixing can be used for large-scale spatial audio mixing processing, so that the universality and the efficiency of the data processing are improved.

The terms first, second and the like in the description and in the claims and drawings of embodiments of the application are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or modules but may, in the alternative, include other steps or modules not listed or inherent to such process, method, apparatus, article, or device.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in this description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The method and related apparatus provided in the embodiments of the present application are described with reference to the flowchart and/or schematic structural diagrams of the method provided in the embodiments of the present application, and each flow and/or block of the flowchart and/or schematic structural diagrams of the method may be implemented by computer program instructions, and combinations of flows and/or blocks in the flowchart and/or block diagrams. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or structural diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or structures.

The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs.

The modules in the device of the embodiment of the application can be combined, divided and deleted according to actual needs.

The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.

Claims

1. A method of data processing, the method comprising:

receiving a media subscription message sent by a first combination device and aiming at the first service object, and adding the first combination device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device;

decoding the initial media data in the data packet to be analyzed to obtain decoded media data, acquiring a subscription object from the media service instance, and transmitting the decoded media data to the subscription object; the subscription object includes the first combined device.

2. The method of claim 1, wherein the obtaining a media service instance for a first service object based on object location information of the first service object in the data packet to be parsed comprises:

searching a media service instance aiming at a first service object based on the object position information of the first service object in the data packet to be analyzed;

if the media service instance aiming at the first service object is not found, a subscription management set and a data decoding instance are built aiming at the first service object; the subscription management set is used for managing the combined equipment for sending the media subscription message;

and combining the subscription management set and the data decoding instance to form a media service instance aiming at the first service object.

3. The method of claim 2, wherein decoding the initial media data in the data packet to be parsed to obtain decoded media data comprises:

decoding the initial media data in the data packet to be analyzed to obtain first decoded data;

constructing a decoding buffer area in the data decoding example, analyzing the decoding processing process of the initial media data, and generating decoding auxiliary buffer data aiming at the initial media data;

And combining the first decoding data and the decoding auxiliary cache data to form decoding media data.

4. The method of claim 1, wherein the method further comprises:

acquiring an object subscription duration of the subscription object, deleting a first subscription object of which the object subscription duration is greater than or equal to a subscription maintenance time threshold, and sending a subscription failure message for the first business object to the first subscription object; or,

acquiring a subscription period associated with the subscription object, deleting a second subscription object, and sending a subscription failure message aiming at the first service object to the second subscription object; the second subscription object refers to a subscription object with a subscription period less than or equal to the system network time.

5. The method of claim 1, wherein the media service instance further comprises a packet map instance, and the data packet to be parsed further comprises service information;

the method further comprises the steps of:

adding the service information in the data packet to be analyzed into the packet mapping instance;

when the decoded media data is obtained, based on the service information, media associated data associated with the initial media data is obtained from the package mapping instance, and the media associated data is sent to the subscription object; the media association data includes the service information.

6. A method of data processing, the method comprising:

acquiring a first decoding device, and sending a data packet to be analyzed generated based on the initial media data to the first decoding device so that the first decoding device builds a media service instance aiming at the first service object;

transmitting a service address of the first decoding device to the first combining device, so that the first combining device transmits a media subscription message to the first decoding device based on the service address; the media subscription message is used for the first decoding device to send the decoded media data obtained by decoding the initial media data to the first combination device.

7. The method of claim 6, wherein the determining the first combined device to which the initial media data corresponds comprises:

acquiring first media data associated with the initial media data, and searching for combined equipment corresponding to the first media data; the first media data refers to media data combined with the initial media data;

If the combined equipment corresponding to the first media data is found, determining the combined equipment corresponding to the first media data as first combined equipment;

and if the combined equipment corresponding to the first media data is not found, acquiring the first combined equipment.

8. The method of claim 6, wherein the obtaining the first decoding device comprises:

acquiring the equipment loads corresponding to the N decoding equipment respectively, and determining the decoding equipment with the minimum equipment load as first decoding equipment; n is a positive integer; or,

determining a decoding device associated with the object position information of the first service object as a first decoding device; or,

and randomly selecting a first decoding device from the N decoding devices.

9. The method of claim 6, wherein the sending the data packet to be parsed generated based on the initial media data to the first decoding device comprises:

acquiring object position information of the first service object, and acquiring media association data corresponding to the initial media data; the media association data is used for representing related information of a service generating the initial media data;

And packaging the object position information of the first service object, the initial media data and the media related data into a data packet to be analyzed, and sending the data packet to be analyzed to the first decoding equipment.

10. The method of claim 9, wherein the method further comprises:

acquiring object coordinates and region division sizes of a service object, and performing coordinate conversion on the object coordinates of the service object based on the region division sizes to obtain object position information of the service object; the business object comprises the first business object;

performing hash processing on the object position information of the service object, and determining position management equipment corresponding to the service object;

when initial media data generated by the first service object is received, sending a receiving determining request to first position management equipment corresponding to the first service object, so that the first position management equipment determines a second service object for receiving the initial media data based on the receiving determining request;

the second service object is obtained from the first position management device, and the second service object is sent to the first combination device, so that when the first combination device receives the decoded media data, a message is sent to the second service object based on the decoded media data.

11. The method of claim 6, wherein the method further comprises:

respectively performing periodic heartbeat detection on N decoding devices and M combination devices, and deleting abnormal devices if detecting that the abnormal devices exist in the N decoding devices and the M combination devices based on the periodic heartbeat detection; the abnormal device is a device which continuously does not receive the number of times of heartbeat replies aiming at the periodic heartbeat detection and is larger than or equal to a heartbeat abnormal threshold value;

if the abnormal equipment is combined equipment, acquiring second decoding equipment associated with the abnormal equipment, and sending a first abnormal reminding message of the abnormal equipment to the second decoding equipment so that the second decoding equipment deletes subscription information of the abnormal equipment based on the first abnormal reminding message;

if the abnormal equipment is decoding equipment, acquiring second combined equipment associated with the abnormal equipment, and sending a second abnormal reminding message of the abnormal equipment to the second combined equipment so that the second combined equipment deletes the service address of the abnormal equipment based on the second abnormal reminding message.

12. A data processing apparatus, the apparatus comprising:

an instance obtaining module, configured to obtain a media service instance for a first service object based on object location information of the first service object in the data packet to be parsed;

a message receiving module, configured to receive a media subscription message for the first service object sent by a first combination device;

a device adding module, configured to add the first combined device to a media service instance of the first service object based on the media subscription message; the media subscription message is generated by the first combining device based on the service address of the first decoding server sent by the media forwarding device;

a decoding and transmitting module, configured to transmit the decoded media data to the subscription object; the subscription object includes the first combined device.

13. A data processing apparatus, the apparatus comprising:

the to-be-parsed sending module is used for sending a to-be-parsed data packet generated based on the initial media data to the first decoding equipment so that the first decoding equipment builds a media service instance aiming at the first service object;

an address sending module, configured to send a service address of the first decoding device to the first combining device, so that the first combining device sends a media subscription message to the first decoding device based on the service address; the media subscription message is used for the first decoding device to send the decoded media data obtained by decoding the initial media data to the first combination device.

14. A computer device, comprising a processor, a memory, and an input-output interface;

the processor is connected to the memory and the input-output interface, respectively, wherein the input-output interface is used for receiving data and outputting data, the memory is used for storing a computer program, and the processor is used for calling the computer program to enable the computer device to execute the method of any one of claims 1-5 or execute the method of any one of claims 6-11.

15. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 1-5 or to perform the method of any of claims 6-11.

16. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the method of any one of claims 1-5 or perform the method of any one of claims 6-11.