CN115150368A

CN115150368A - Media file association processing method, device, medium and electronic equipment

Info

Publication number: CN115150368A
Application number: CN202110351646.4A
Authority: CN
Inventors: 胡颖
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-03-31
Filing date: 2021-03-31
Publication date: 2022-10-04
Anticipated expiration: 2041-03-31
Also published as: CN115150368B

Abstract

The embodiment of the application provides a method for processing association of media files an apparatus, a medium, and an electronic device. The media file association processing method comprises the following steps: acquiring a media file, wherein the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources; the media file is parsed and the media file is parsed, obtaining the incidence relation between the media resource data contained in the media file and the plurality of media resources; and presenting the media resource data contained in the media file according to the incidence relation among the plurality of media resources. The technical scheme of the embodiment of the application can realize the indication of the incidence relation between the media resources, thereby meeting the business requirements of different application scenes.

Description

Media file association processing method, device, medium and electronic equipment

Technical Field

The present application relates to the field of computer and communication technologies, and in particular, to a method, an apparatus, a medium, and an electronic device for associating a media file.

Background

In a point cloud consumption scene, a plurality of point cloud files may have relevance, for example, in a digital cultural heritage, due to the cultural significance and the educational significance of a cultural relic, the presentation of the cultural relic needs to be performed by combining a plurality of forms, and one cultural relic corresponds to a plurality of different point cloud media resources. However, it is difficult to show the association relationship between these point cloud media resources in the related art.

Disclosure of Invention

Embodiments of the present application provide a method, an apparatus, a medium, and an electronic device for processing association of media files, so that indication of association between media resources can be implemented at least to a certain extent, and service requirements of different application scenarios can be satisfied.

Other features and advantages of the present application will be apparent from the following detailed description, or may be learned by practice of the application.

According to an aspect of the embodiments of the present application, there is provided a method for processing association of media files, including: acquiring a media file, wherein the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources; analyzing the media file to obtain the incidence relation between the media resource data contained in the media file and the plurality of media resources; and presenting the media resource data contained in the media file according to the incidence relation among the plurality of media resources.

According to an aspect of the embodiments of the present application, there is provided a method for processing association of media files, including: generating a media file, wherein the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources; and sending the media file to a data receiver so that the data receiver analyzes the media file and presents media resource data contained in the media file according to the association relationship among the plurality of media resources obtained by analysis.

According to an aspect of the embodiments of the present application, there is provided an association processing apparatus for media files, including: the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is configured to acquire a media file, the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources; the analysis unit is configured to analyze the media file to obtain an association relation between the media resource data contained in the media file and the plurality of media resources; and the processing unit is configured to present the media resource data contained in the media file according to the association relation among the plurality of media resources.

In some embodiments of the present application, based on the foregoing scheme, the media file includes a plurality of media resource data; the processing unit is configured to: and sequentially presenting the plurality of media resource data according to the incidence relation among the plurality of media resources.

In some embodiments of the present application, based on the foregoing solution, the object association data box further includes ingress media identification information, where the ingress media identification information is used to indicate the first consumed media resource data; the processing unit is configured to: preferentially presenting the first consumed media resource data according to the entrance media identification information; after presenting the first consumed media asset data, presenting other media asset data of the plurality of media asset data than the first consumed media asset data.

In some embodiments of the present application, based on the foregoing scheme, the information of the plurality of media assets in the object association data box includes: the media assets include identification information of each of the plurality of media assets, portal media asset identification information indicating whether each of the media assets is a first consumed media asset, and description information of each of the media assets.

In some embodiments of the present application, based on the foregoing solution, the object association data box further includes: a number of the plurality of media resources; the association relationship in the object association data box comprises: a group identification for identifying a group of entities to which the plurality of media assets belong.

In some embodiments of the present application, based on the foregoing solution, the association processing apparatus further includes: a receiving unit, configured to receive, before acquiring a media file, DASH (Dynamic Adaptive Streaming over HTTP) signaling sent by a data source side, where the DASH signaling includes entry media resource identification information and a type field of an entity group, the type field of the entity group is a set value, and is used to indicate that media resources in the entity group have an association relationship, and the entry media resource identification information is used to indicate a first consumed media resource; the acquisition unit is configured to: and acquiring the media file according to the DASH signaling.

In some embodiments of the present application, based on the foregoing solution, the obtaining unit is configured to: sending a data acquisition request to the data source side according to the DASH signaling, wherein the data acquisition request is used for requesting to acquire media resource data corresponding to all or part of media resources in the entity group; and receiving the media file returned by the data source side according to the data acquisition request.

In some embodiments of the present application, based on the foregoing scheme, the obtaining unit is configured to: if the current network state is better than the set value, sending a data acquisition request to the data source side to request to acquire media resource data corresponding to all media resources in the entity group; and if the current network state is lower than a set value, sending a data acquisition request to the data source side to request to acquire media resource data corresponding to part of media resources in the entity group, wherein the part of media resources comprises the first consumed media resource.

In some embodiments of the present application, based on the foregoing scheme, the DASH signaling further includes identification information of an entity group, where the identification information of the entity group is used to indicate a group identifier of the entity group.

In some embodiments of the present application, based on the foregoing scheme, the media file comprises a point cloud media file, and the media resource comprises a point cloud media resource.

According to an aspect of the embodiments of the present application, there is provided an association processing apparatus for media files, including: the generating unit is configured to generate a media file, wherein the media file comprises media resource data and an object associated data box, and the object associated data box comprises information of a plurality of media resources and association relations among the plurality of media resources; and the sending unit is configured to send the media file to a data receiving party so that the data receiving party can analyze the media file and present media resource data contained in the media file according to the association relation among the media resources obtained through analysis.

In some embodiments of the present application, based on the foregoing solution, the generating unit is further configured to: before generating a media file, generating a DASH signaling, where the DASH signaling includes entry media resource identification information and a type field of an entity group, where the type field of the entity group is a set value for indicating that media resources in the entity group have an association relationship, and the entry media resource identification information is used for indicating a first consumed media resource; the transmitting unit is further configured to: sending the DASH signaling to the data receiving party, and receiving a data acquisition request sent by the data receiving party; the device for processing the association of the media files further comprises: and the determining unit is configured to determine the media resource data required to be transmitted to the data receiving party according to the data acquisition request, so as to generate the media file according to the media resource data required to be transmitted to the data receiving party.

According to an aspect of embodiments of the present application, there is provided a computer-readable medium on which a computer program is stored, the computer program, when executed by a processor, implementing a method of association processing of media files as described in the above embodiments.

According to an aspect of an embodiment of the present application, there is provided an electronic device including: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the associated processing method of media files as described in the above embodiments.

According to an aspect of embodiments herein, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the method for processing the association of the media file provided in the various alternative embodiments described above.

In the technical solutions provided in some embodiments of the present application, the media file includes an object association data box, and the object association data box includes information of a plurality of media resources and an association relationship between the plurality of media resources, and then the media file is analyzed, and the media resource data included in the media file is presented according to the association relationship between the plurality of media resources obtained by the analysis, so that the association relationship between the media resources can be indicated through the object association data box, thereby implementing an indication of the association relationship between the media resources, and further meeting service requirements of different application scenarios.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:

FIG. 1 shows a schematic diagram of an exemplary system architecture to which aspects of embodiments of the present application may be applied;

fig. 2 is a schematic diagram showing the placement of a video encoding apparatus and a video decoding apparatus in a streaming system;

FIG. 3 shows a flow diagram of a method of association processing of media files according to one embodiment of the present application;

FIG. 4 shows a flow diagram of a method of association processing of media files according to one embodiment of the present application;

FIG. 5 shows a flow diagram of a method of association processing of media files according to one embodiment of the present application;

FIG. 6 shows a flow diagram of a method of association processing of media files according to one embodiment of the application;

FIG. 7 shows a flow diagram of a method of association processing of media files according to one embodiment of the application;

FIG. 8 shows a block diagram of an association processing device for media files according to an embodiment of the application;

FIG. 9 shows a block diagram of an association processing device for media files according to an embodiment of the application;

FIG. 10 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flowcharts shown in the figures are illustrative only and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

It should be noted that: reference herein to "a plurality" means two or more. "and/or" describe the association relationship of the associated objects, meaning that there may be three relationships, e.g., A and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

Fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied.

As shown in fig. 1, the system architecture 100 includes a plurality of end devices that may communicate with each other over, for example, a network 150. For example, the system architecture 100 may include a first end device 110 and a second end device 120 interconnected by a network 150. In the embodiment of fig. 1, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

For example, the first terminal device 110 may encode point cloud video data (e.g., a point cloud video stream collected by the terminal device 110) for transmission over the network 150 to the second terminal device 120, the encoded point cloud video data being transmitted in one or more encoded video streams, the second terminal device 120 may receive the encoded point cloud video data from the network 150, decode the encoded point cloud video data to recover the point cloud video data, and display a video picture according to the recovered point cloud video data.

In one embodiment of the present application, the system architecture 100 may include a third end device 130 and a fourth end device 140 that perform bi-directional transmission of encoded point cloud video data, such as may occur during video communications. For bi-directional data transmission, each of the third and

fourth end devices

130, 140 may encode point cloud video data (e.g., a stream of point cloud video pictures captured by the end device) for transmission over the network 150 to the other of the third and

fourth end devices

130, 140. Each of the third and

fourth end devices

130, 140 may also receive encoded point cloud video data transmitted by the other of the third and

fourth end devices

130, 140, and may decode the encoded point cloud video data to recover the point cloud video data, and may display a point cloud video picture on an accessible display device according to the recovered point cloud video data.

In the embodiment of fig. 1, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 may be a server, a personal computer, and a smart phone, but the principles disclosed herein may not be limited thereto. Embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and/or dedicated video conferencing equipment. Network 150 represents any number of networks including, for example, wired and/or wireless communication networks that communicate encoded point cloud video data between first terminal device 110, second terminal device 120, third terminal device 130, and fourth terminal device 140. The communication network 150 may exchange data in circuit-switched and/or packet-switched channels. The network may include a telecommunications network, a local area network, a wide area network, and/or the internet. For purposes of this application, the architecture and topology of the network 150 may be immaterial to the operation of the present disclosure, unless explained below.

In one embodiment of the present application, fig. 2 illustrates the placement of video encoding devices and video decoding devices in a streaming environment. The subject matter disclosed herein is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV (television), storing compressed video on digital media including CDs, DVDs, memory sticks, and the like.

The streaming system may include an acquisition subsystem 213, and the acquisition subsystem 213 may include a video source 201, such as a digital camera, a point cloud media generation device, etc., that creates an uncompressed video picture stream 202. In an embodiment, the video picture stream 202 includes samples taken by a digital camera, or samples taken or generated by a point cloud media. The video picture stream 202 is depicted as a thick line to emphasize a high data amount video picture stream compared to the encoded video data 204 (or the encoded video codestream 204), the video picture stream 202 can be processed by an electronic device 220, the electronic device 220 comprising a video encoding device 203 coupled to a video source 201. The video encoding device 203 may comprise hardware, software, or a combination of hardware and software to implement or embody aspects of the disclosed subject matter as described in greater detail below. The encoded video data 204 (or encoded video codestream 204) is depicted as a thin line to emphasize the lower data amount of the encoded video data 204 (or encoded video codestream 204) as compared to the video picture stream 202, which may be stored on the streaming server 205 for future use. One or more streaming client subsystems, such as client subsystem 206 and client subsystem 208 in fig. 2, may access streaming server 205 to retrieve

copies

207 and 209 of encoded video data 204. Client subsystem 206 may include, for example, video decoding device 210 in electronic device 230. Video decoding device 210 decodes incoming copies 207 of the encoded video data and generates an output video picture stream 211 that may be presented on a display 212 (e.g., a display screen) or another presentation device. In some streaming systems, encoded video data 204, video data 207, and video data 209 (e.g., video streams) may be encoded according to certain video encoding/compression standards.

It should be noted that

electronic devices

220 and 230 may include other components not shown in the figures. For example, the electronic device 220 may comprise a video file decoding device, and the electronic device 230 may also comprise a video file encoding device.

In one embodiment of the present application, the video data in the above embodiments may be point cloud media data. The point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface attributes of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may have color, material or other information according to different application scenes. Typically, each point in the point cloud has the same number of additional attributes. The Point Cloud media can be further classified into a Point Cloud media compressed in a VPCC (Video-based Point Cloud Compression) manner and a Point Cloud media compressed in a GPCC (Geometry-based Point Cloud Compression) manner from the viewpoint of an encoding manner. In the file encapsulation of the point cloud media, three-dimensional position information is generally called a geometric Component (Geometry Component) of the point cloud file, attribute information is called an Attribute Component (Attribute Component) of the point cloud file, and one point cloud file has only one geometric Component, but one or more Attribute components may exist.

The point cloud can flexibly and conveniently express the space structure and the surface attribute of a three-dimensional object or scene, so the application is wide, and the main application scenes can be classified into two categories: machine-perceived point clouds and human-eye perceived point clouds. Machine perception point clouds such as an autonomous navigation system, a real-time inspection system, a geographic information system, a visual sorting robot, an emergency rescue and relief robot and the like; human eyes perceive point cloud application scenes such as digital cultural heritage, free viewpoint broadcasting, three-dimensional immersion communication, three-dimensional immersion interaction and the like.

The point cloud is obtained mainly by the following ways: computer generation, 3D laser scanning, 3D photogrammetry, etc. The computer can generate a point cloud of virtual three-dimensional objects and scenes; the 3D scanning can obtain the point cloud of a static real world three-dimensional object or scene, and million-level point clouds can be obtained every second; the 3D camera can obtain the point cloud of the dynamic real world three-dimensional object or scene, and ten million-level point cloud can be obtained every second. In addition, in the medical field, a point cloud of a biological tissue or organ can be obtained from MRI (Magnetic Resonance Imaging), CT (Computed Tomography), and electromagnetic localization information. The technologies reduce the cost and the time period for acquiring the point cloud data, improve the accuracy of the data, and change the manner of acquiring the point cloud data, so that the acquisition of a large amount of point cloud data becomes possible. Along with the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, release, sharing and standardization of the point cloud data become the key of point cloud application.

After the point cloud media is encoded, the encoded data stream needs to be encapsulated and transmitted to the client. Correspondingly, at the point cloud media player end, the acquired point cloud file needs to be unpacked, then decoded, and finally the decoded data stream is presented.

In a point cloud consumption scene, a plurality of point cloud files may have relevance, for example, in a digital cultural heritage, due to cultural significance and educational significance of cultural relics, presentation of the cultural relics is often performed by combining a plurality of forms, and one cultural relic corresponds to a plurality of different point cloud media resources. However, it is difficult to express the association relationship between the point cloud media resources in the related art. Based on this, the embodiments of the present application provide the following solutions:

fig. 3 shows a flowchart of an association processing method of a media file according to an embodiment of the present application, which may be executed by a media playing device, which may be a smartphone, a tablet computer, or the like. Referring to fig. 3, the method for processing association of media files at least includes steps S310 to S330, which are described in detail as follows:

in step S310, a media file is obtained, where the media file includes media resource data and an object association data box, and the object association data box includes information of multiple media resources and association relationships between the multiple media resources.

It should be noted that: the media resource data is specific content of the media resource, for example, if the media resource is a video of an introduction article a, the media resource data is specific content (video picture, introduction audio, etc.) of the introduction video of the article a. In one embodiment of the present application, the media file may be a point cloud media file, and then the media resource may be a point cloud media resource, and the media resource data is point cloud media resource data.

Optionally, the media file may include one media resource data, or may also include a plurality of media resource data, which is determined according to a request of the data receiving side, that is, the data receiving side may request to acquire one or more media resource data according to a requirement. However, regardless of whether the media file includes one media resource data or a plurality of media resource data, the object association data box includes information of a plurality of media resources and an association relationship between the plurality of media resources.

In one embodiment of the present application, the information of the plurality of media assets in the object association data box may include: the media asset management system includes identification information of each of a plurality of media assets, portal media asset identification information indicating whether each media asset is a first consumed media asset, and description information of each media asset. Optionally, the identification information of each media resource may be, for example, an entry _ id, the entry media resource identification information for indicating whether each media resource is the first consumed media resource may be, for example, an entry _ media _ flag, and the description information of each media resource may be, for example, a media _ description.

In one specific example, assume that there are 3 media assets, the 1 st media asset being used to show the appearance of the golden crown, the 2 nd media asset being used to show the queen wearing the golden crown, and the 3 rd media asset being used to show the manufacturing process of the golden crown. Then the information of the plurality of media assets in the object association data box can be expressed as follows:

{entity_id＝1；entry_media_flag＝1；media_description＝‘gold crown’}；

{entity_id＝2；entry_media_flag＝0；media_description＝‘queen wearing gold crown’}；

{entity_id＝3；entry_media_flag＝0；media_description＝‘gold crown manufacturing process’}。

wherein, entry _ media _ flag =1 indicates that the 1 st media resource is the first consumed media resource, i.e. an entry media resource.

In an embodiment of the present application, the association relationship contained in the object association data box may be: a group identification for identifying a group of entities to which the plurality of media assets belong. For example, group _ id may be used to indicate that, in this embodiment, the association relationship between the media resources is used to indicate that a plurality of media resources belong to one entity group. Optionally, the object association data box may further include a number of multiple media resources, for example, the number may be denoted by num _ entites _ in _ group, and for the above example related to the golden crown, num _ entites _ in _ group =3.

Of course, in other embodiments of the present application, the association relationship included in the object association data box may also be a presentation order of the plurality of media resources therein when presenting, and the like.

In step S320, the media file is parsed to obtain the association relationship between the media resource data and the plurality of media resources included in the media file.

As described in the foregoing embodiments, after parsing a media file, one media resource data may be obtained, or multiple media resource data may be obtained.

In step S330, the media resource data contained in the media file is presented according to the association relationship among the plurality of media resources.

In an embodiment of the present application, if a plurality of media resource data are contained in a media file, the plurality of media resource data may be presented in sequence according to an association relationship between the plurality of media resources when presenting.

Optionally, if the object association data box further includes portal media identification information, where the portal media identification information is used to indicate first consumed media resource data, when the multiple media resource data are sequentially presented, the first consumed media resource data may be preferentially presented according to the portal media identification information, and after the first consumed media resource data is presented, other media resource data, except the first consumed media resource data, in the multiple media resource data may be presented.

In one embodiment of the present application, if one media resource data is included in the media file, the one media resource data may be directly presented when presenting.

Based on the technical solution of the embodiment shown in fig. 3, as shown in fig. 4, the method for processing a media file according to the present application at least includes steps S410 to S440, wherein details of implementation of steps S420 to S440 are consistent with those of steps S310 to S330 in the embodiment shown in fig. 3, and are described in detail as follows:

in step S410, a DASH signaling sent by the data source side is received, where the DASH signaling includes entry media resource identification information and a type field of an entity group, where the type field of the entity group is a set value for indicating that media resources in the entity group have an association relationship, and the entry media resource identification information is used for indicating a first consumed media resource.

In an embodiment of the present application, the type field of the entity group included in the DASH signaling may be, for example, entitygroupcurjype, and specifically, if entitygroupcurjype = 'oapc', it may be stated that the media resources in the entity group have an association relationship. The entry media resource identification information included in the DASH signaling may be, for example, an entitygroup @ entry _ media _ id, and specifically, if the entitygroup @ entry _ media _ id =1, it may be described that the identification of the first consumed media resource is 1.

Optionally, the DASH signaling may further include identification information of the entity group, where the identification information of the entity group is used to indicate a group identification of the entity group. For example, the identification information of the entity group may be entitygroup @ group _ id.

In step S420, a media file is obtained according to the DASH signaling, where the media file includes media resource data and an object association data box, and the object association data box includes information of multiple media resources and an association relationship between the multiple media resources.

In an embodiment of the present application, the process of acquiring the media file according to DASH signaling may be: and sending a data acquisition request to the data source side according to the DASH signaling, wherein the data acquisition request is used for requesting to acquire media resource data corresponding to all or part of media resources in the entity group, and then receiving a media file returned by the data source side according to the data acquisition request.

Optionally, all media resources or part of the media resources in the request to acquire entity group may be determined according to the network status, or may also be determined according to the actual service requirement.

If the network state is determined according to the network state, if the current network state is superior to a set value (if the network state is superior to the set value, the current network state is superior), sending a data acquisition request to the data source side to request to acquire media resource data corresponding to all media resources in the entity group; if the current network state is lower than the set value (if the network state is lower than the set value, the current network state is poor), a data acquisition request is sent to the data source side to request to acquire media resource data corresponding to part of media resources in the entity group, wherein the part of media resources comprises the first consumed media resource.

It should be noted that since the first consumed media asset is an ingress media asset, it is necessary to include at least the ingress media asset if a part of the media asset is requested. Even if the requested media resource is determined according to the actual traffic demand, it is still necessary to include at least the ingress media resource if only part of the media resource needs to be requested.

In addition, other details in step S420 are similar to those in step S310 and are not repeated.

In step S430, the media file is parsed to obtain the association relationship between the media resource data and the plurality of media resources included in the media file.

In step S440, the media resource data contained in the media file is presented according to the association relationship among the plurality of media resources.

It should be noted that: other details in step S440 are similar to those in step S330, and are not repeated.

Fig. 3 and fig. 4 are diagrams illustrating a technical solution of an embodiment of the present application from a receiving side of a media resource, and implementation details of the embodiment of the present application are further described from a data source side in conjunction with fig. 5 as follows:

fig. 5 shows a flowchart of an association processing method of a media file according to an embodiment of the present application, which may be performed by a media generation device, which may be a smartphone, a tablet computer, or the like. Referring to fig. 5, the method for processing association of media files at least includes steps S510 to S520, which are described in detail as follows:

in step S510, a media file is generated, where the media file includes media resource data and an object association data box, and the object association data box includes information of a plurality of media resources and association relationships between the plurality of media resources.

Optionally, the media file may include one media resource data, or may also include multiple media resource data, which is determined according to a request of the data receiving side, that is, the data receiving side may request to acquire one or more media resource data according to a requirement. However, regardless of whether the media file includes one media resource data or a plurality of media resource data, the object association data box includes information of a plurality of media resources and an association relationship between the plurality of media resources.

In one specific example, assume that there are 3 media assets, the 1 st media asset being used to show the appearance of a gold crown, the 2 nd media asset being used to show a queen wearing a gold crown, and the 3 rd media asset being used to show the manufacturing process of a gold crown. Then the information of the plurality of media assets in the object association data box can be expressed as follows:

wherein entry _ media _ flag =1 indicates that the 1 st media asset is the first consumed media asset, i.e. is an entry media asset.

In an embodiment of the present application, the association relationship contained in the object association data box may be: a group identification for identifying a group of entities to which the plurality of media assets belong. For example, group _ id may be used to indicate that, in this embodiment, the association relationship between the media resources is used to indicate that a plurality of media resources belong to one entity group. Optionally, the object association data box may further include a number of multiple media resources, such as num _ entries _ in _ group, where, for the above example regarding golden crown, num _ entries _ in _ group =3.

In step S520, the media file is sent to the data receiving party, so that the data receiving party analyzes the media file, and presents the media resource data included in the media file according to the association relationship between the media resources obtained through analysis.

In an embodiment of the present application, the process of presenting the media resource data included in the media file by the data receiving side may refer to the technical solution of the foregoing embodiment, and is not described in detail again.

Based on the technical solution of the embodiment shown in fig. 5, as shown in fig. 6, the method for associating and processing a media file according to the present application at least includes steps S610 to S650, wherein details of implementation of steps S640 to S650 are consistent with those of steps S510 to S520 in the embodiment shown in fig. 5, and the following is described in detail:

in step S610, a DASH signaling is generated, where the DASH signaling includes entry media resource identification information and a type field of an entity group, the type field of the entity group is a set value for indicating that media resources in the entity group have an association relationship, and the entry media resource identification information is used for indicating a first consumed media resource.

In an embodiment of the present application, the type field of the entity group included in the DASH signaling may be, for example, an entitygroup @ _ group _ type, and specifically, if the entitygroup @ _ group _ type = 'oapc', it may be stated that the media resources in the entity group have an association relationship. The entry media resource identification information included in the DASH signaling may be, for example, an entitygroup @ entry _ media _ id, and specifically, if the entitygroup @ entry _ media _ id =1, it may be described that the identification of the first consumed media resource is 1.

In step S620, DASH signaling is sent to the data receiving side, and a data acquisition request sent by the data receiving side is received.

In step S630, the media resource data that needs to be transmitted to the data receiver is determined according to the data acquisition request.

In an embodiment of the present application, the data obtaining request sent by the data receiving side may be for requesting to obtain media resource data corresponding to all or part of the media resources in the entity group. For example, the data receiver may request to acquire all or part of the media resources in the entity group according to the network status or according to the actual service requirement. Specifically, reference may be made to the technical solutions of the foregoing embodiments, which are not described in detail.

In step S640, a media file is generated according to the media resource data that is transmitted to the data receiving side as needed, where the media file includes media resource data and an object association data box, and the object association data box includes information of a plurality of media resources and association relations between the plurality of media resources.

In particular, the object association data box and the media asset data that needs to be transmitted to the data recipient may be compressed to generate a media file. It should be noted that: other details in step S640 are similar to those in step S510, and are not repeated.

In step S650, the media file is sent to the data receiving party, so that the data receiving party can analyze the media file, and present the media resource data included in the media file according to the association relationship between the media resources obtained through analysis.

It should be noted that: other details in step S650 are similar to those in step S520, and are not repeated.

The technical solutions of the embodiments of the present application are described above from the receiving side and the data source side of the media resource, and the implementation details of the embodiments of the present application are further described below with reference to fig. 7 as a whole:

as shown in fig. 7, taking the example that the server generates the point cloud media file and the client consumes the point cloud media file as an example, the method may specifically include the following steps S701 to S706:

in step S701, the server generates a DASH command.

In one embodiment of the present application, ingress media resource identification information may be included in DASH signaling to indicate the media resource as initially consumed (i.e., first consumed). Based on this, the technical solution in the embodiment of the present application extends the DASH signaling, which is specifically shown in table 1:

TABLE 1

In table 1, 1 and N in the column of "usage method (Use)" indicate the number (specifically, integers), M indicates mandirectory (i.e., mandatory), and O indicates Optional. Omaf2 and dash in the "Data type (Data type)" column are used to indicate that the type is a Data type defined corresponding to the standard; xs denotes a short int type. Meanwhile, the Adaptation Sets represent a set of video streams in DASH, and one Adaptation set may include a plurality of repetitions. RepresentationRepresentationsMedia components in DASH, for example, a video file of a certain resolution may be considered a Representation.

As can be seen from table 1, the embodiment of the present application defines a new element entitygroup @ entry _ media _ ID for DASH signaling, the element belongs to an optional field, the data type of the element belongs to the type that the character string has no space, and the element is used to indicate the representation form ID as the initial consuming entity in the point cloud media association group.

In a specific example, the server is assumed to store media resources T1 to T3 related to an exhibit a in a museum, where T1 is a point cloud model (e.g., a point cloud model of a golden crown) of the exhibit a; t2 is a common video picture (such as a queen wearing a golden crown) when the exhibit A is used; t3 is a point cloud sequence medium (such as a manufacturing process of a golden crown) in the manufacturing process of the exhibit A. Then the generated DASH signaling may be as follows:

EntityGroup@group_type＝‘oapc’；EntityGroup@group_id＝1；

{EntityGroup.EntityIdList@asid＝1；EntityGroup.EntityIdList@rsid＝1}

{EntityGroup.EntityIdList@asid＝1；EntityGroup.EntityIdList@rsid＝2}

{EntityGroup.EntityIdList@asid＝1；EntityGroup.EntityIdList@rsid＝3}

EntityGroup@entry_media_id＝1；

wherein, entityGroup @ group _ type = 'oapc' indicates that all entities (namely media resources) in the entity group have an association relation on content; entitygroup @ group _ id =1 indicates that the entity group identification is 1; the entitygroup @ entry _ media _ ID =1 indicates that the media asset ID as the initial consuming entity is 1.

Meanwhile, the embodiment of the application also defines a new data box: the object association data box (i.e., objectAssociationBox) is specifically defined as follows:

Box Type：‘oapc’

Container：GroupsListBox

Mandatory：No

Quantity：Zero or More

that is, if the group type of the entityto group box is 'oapc', it indicates that there is a content association relationship between all entities (i.e., media assets) in the entity group, and one or more media assets in the entity group can be presented according to requirements when presented. Meanwhile, the object association data box belongs to an optional item, and 0 or more may be set.

In one embodiment of the present application, the syntax of the object association data box is described as follows:

in the above-mentioned syntax description, object association data boxes ObjectAssociationBox, entitygroupbox ('oapc'), unsigned integer variable group _ id (i.e. entity group identification) and num _ entries _ in _ group (i.e. number of entities within a group) are defined; an unsigned integer variable entry _ id (i.e., entity identification) and entry _ media _ flag are defined together, and a reserved field (reserved) and a string-type media description field (media _ description) are defined.

Where group _ ID is a non-negative integer assigned to a particular group that cannot equal the group _ ID value of any other entityto group box, such as any item _ ID value or any track _ ID value of a group listbox hierarchy level (i.e. file, movie or track) when the group listbox is contained at the file level.

num _ entries _ in _ group represents the number of entry _ id values within an entity group. If there is an item with item _ ID equal to entry _ ID in the hierarchy level containing the GroupsListBox (i.e. file, movie or track), the object _ ID is parsed into one item, or if there is a track with track _ ID equal to entry _ ID and the GroupsListBox is contained in the file level, it is parsed into a track.

For the above example of a museum exhibit, the object association data box ObjectAssociationBox is represented as follows:

ObjectAssociationBox：group_id＝1；num_entities_in_group＝3；

In step S702, the server sends DASH signaling to the client.

In step S703, the client requests the point cloud media file from the server.

Optionally, the client may request the point cloud media file according to a network condition, for example, if the network state of the client 1 is poor, only the media resource data corresponding to the initial media resource may be requested; if the network status of the client 2 is good, the client may request media resource data corresponding to all media resources in the entity group.

In step S704, the server encapsulates and generates a point cloud media file.

Specifically, the server generates a point cloud media file according to a request of the client, for example, the server generates a point cloud media file F1 according to a request of the client 1, where the point cloud media file F1 includes data of a media resource T1 and an object association data box; the server generates a point cloud media file F2 according to the request of the client 2, wherein the point cloud media file F2 comprises data of media resources T1 to T3 and an object association data box.

In step S705, the server transmits the point cloud media file to the client.

In step S706, the client decapsulates the point cloud media file and presents the corresponding media resource.

Specifically, after receiving the point cloud media file F1, the client 1 presents T1 media resources according to the content in the entity group. After receiving the point cloud media file F2, the client 2 presents the T1 media resource according to the content in the entity group, and then presents the T2 and T3 media resources correspondingly according to the subsequent requirements of the user.

It can be seen that in the technical solution of the above embodiment of the present application, according to the content of the point cloud object, multiple media resources having an association relationship on the content may be associated using the object association box, and each field in the data box is correspondingly filled according to the presentation form of different media resources, so that the indication of the association relationship between the media resources is realized, and further, the service requirements of different application scenarios may be met, for example, a client may purposefully request and present specific content, thereby optimizing bandwidth consumption in the transmission process and improving viewing and consuming experience of a user.

The following describes an embodiment of an apparatus of the present application, which may be used to execute an association processing method for a media file in the foregoing embodiment of the present application. For details that are not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the association processing method of media files described above in the present application.

Fig. 8 shows a block diagram of an association processing apparatus for a media file according to an embodiment of the present application, where the association processing apparatus for the media file may be disposed in a media playing device, and the media playing device may be a smartphone, a tablet computer, or the like.

Referring to fig. 8, an apparatus 800 for associating and processing a media file according to an embodiment of the present application includes: an acquisition unit 802, a parsing unit 804 and a processing unit 806.

The obtaining unit 802 is configured to obtain a media file, where the media file includes media resource data and an object association data box, and the object association data box includes information of a plurality of media resources and association relationships among the plurality of media resources; the analyzing unit 804 is configured to analyze the media file to obtain an association relationship between the media resource data included in the media file and the plurality of media resources; the processing unit 806 is configured to present the media asset data contained in the media file according to the association relationship between the plurality of media assets.

In some embodiments of the present application, based on the foregoing scheme, the media file includes a plurality of media resource data; the processing unit 806 is configured to: and sequentially presenting the plurality of media resource data according to the incidence relation among the plurality of media resources.

In some embodiments of the present application, based on the foregoing solution, the object association data box further includes ingress media identification information, where the ingress media identification information is used to indicate the first consumed media resource data; the processing unit 806 is configured to: preferentially presenting the first consumed media resource data according to the entrance media identification information; after presenting the first consumed media asset data, presenting other media asset data of the plurality of media asset data than the first consumed media asset data.

In some embodiments of the present application, based on the foregoing solution, the association processing apparatus 800 further includes: a receiving unit, configured to receive, before acquiring a media file, a DASH signaling sent by a data source side, where the DASH signaling includes entry media resource identification information and a type field of an entity group, the type field of the entity group is a set value, so as to indicate that media resources in the entity group have an association relationship, and the entry media resource identification information is used to indicate a first consumed media resource; the acquisition unit is configured to: and acquiring the media file according to the DASH signaling.

In some embodiments of the present application, based on the foregoing solution, the obtaining unit 802 is configured to: sending a data acquisition request to the data source side according to the DASH signaling, wherein the data acquisition request is used for requesting to acquire media resource data corresponding to all or part of media resources in the entity group; and receiving the media file returned by the data source side according to the data acquisition request.

In some embodiments of the present application, based on the foregoing solution, the obtaining unit 802 is configured to: if the current network state is better than the set value, sending a data acquisition request to the data source side to request to acquire media resource data corresponding to all media resources in the entity group; and if the current network state is lower than a set value, sending a data acquisition request to the data source side to request to acquire media resource data corresponding to part of media resources in the entity group, wherein the part of media resources comprises the first consumed media resource.

In some embodiments of the present application, based on the foregoing scheme, the DASH signaling further includes identification information of an entity group, where the identification information of the entity group is used to indicate a group identification of the entity group.

Fig. 9 shows a block diagram of a media file association processing apparatus according to an embodiment of the present application, which may be disposed in a media generation device, which may be a smartphone, a tablet computer, or the like.

Referring to fig. 9, an apparatus 900 for associating media files according to an embodiment of the present application includes: a generating unit 902 and a transmitting unit 904.

The generating unit 902 is configured to generate a media file, where the media file includes media resource data and an object association data box, and the object association data box includes information of a plurality of media resources and association relations among the plurality of media resources; the sending unit 904 is configured to send the media file to a data receiving party, so that the data receiving party parses the media file, and presents media resource data included in the media file according to an association relationship between multiple media resources obtained by parsing.

In some embodiments of the present application, based on the foregoing scheme, the generating unit 902 is further configured to: before generating a media file, generating a DASH signaling, where the DASH signaling includes entry media resource identification information and a type field of an entity group, where the type field of the entity group is a set value for indicating that media resources in the entity group have an association relationship, and the entry media resource identification information is used for indicating a first consumed media resource; the sending unit 904 is further configured to: sending the DASH signaling to the data receiving party, and receiving a data acquisition request sent by the data receiving party; the device 900 for processing association of media files further comprises: and the determining unit is configured to determine the media resource data required to be transmitted to the data receiving party according to the data acquisition request, so as to generate the media file according to the media resource data required to be transmitted to the data receiving party.

It should be noted that the computer system 1000 of the electronic device shown in fig. 10 is only an example, and should not bring any limitation to the functions and the application scope of the embodiments of the present application.

As shown in fig. 10, the computer system 1000 includes a Central Processing Unit (CPU) 1001 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1002 or a program loaded from a storage portion 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for system operation are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An Input/Output (I/O) interface 1005 is also connected to the bus 1004.

The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a Network interface card such as a Local Area Network (LAN) card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.

In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 1009 and/or installed from the removable medium 1011. When the computer program is executed by a Central Processing Unit (CPU) 1001, various functions defined in the system of the present application are executed.

It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with a computer program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method described in the above embodiments.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, and may also be implemented by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains.

It will be understood that the present application is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An association processing method for a media file is characterized by comprising the following steps:

acquiring a media file, wherein the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources;

analyzing the media file to obtain the incidence relation between the media resource data contained in the media file and the plurality of media resources;

and presenting the media resource data contained in the media file according to the incidence relation among the plurality of media resources.

2. The method for processing association of media files according to claim 1, wherein the media files comprise a plurality of media resource data;

presenting media resource data contained in the media file according to the association relationship among the plurality of media resources, including: and sequentially presenting the plurality of media resource data according to the incidence relation among the plurality of media resources.

3. The method for associating and processing the media file as claimed in claim 2, wherein the object association data box further comprises an entry media identification information, and the entry media identification information is used for indicating the first consumed media resource data;

sequentially presenting the plurality of media resource data according to the incidence relation among the plurality of media resources, comprising: preferentially presenting the first consumed media resource data according to the entrance media identification information;

after presenting the first consumed media asset data, presenting other media asset data of the plurality of media asset data than the first consumed media asset data.

4. The method of claim 1, wherein the information of the plurality of media assets in the object association data box comprises:

the media assets include identification information of each of the plurality of media assets, portal media asset identification information indicating whether each of the media assets is a first consumed media asset, and description information of each of the media assets.

5. The method for associating and processing the media file according to claim 4, wherein the object association data box further comprises: a number of the plurality of media resources;

the association relationship in the object association data box comprises: a group identification for identifying a group of entities to which the plurality of media assets belong.

6. The method for processing association of media files according to claim 1, wherein before acquiring the media files, the method further comprises:

receiving dynamic adaptive streaming DASH signaling sent by a data source side and based on HTTP, wherein the DASH signaling comprises entry media resource identification information and a type field of an entity group, the type field of the entity group is a set value and is used for indicating that media resources in the entity group have an association relation, and the entry media resource identification information is used for indicating a first consumed media resource;

and acquiring the media file according to the DASH signaling.

7. The method according to claim 6, wherein acquiring the media file according to the DASH signaling includes:

sending a data acquisition request to the data source side according to the DASH signaling, wherein the data acquisition request is used for requesting to acquire media resource data corresponding to all or part of media resources in the entity group;

and receiving the media file returned by the data source side according to the data acquisition request.

8. The method according to claim 7, wherein sending a data acquisition request to the data source side according to the DASH signaling includes:

if the current network state is better than the set value, sending a data acquisition request to the data source side to request to acquire media resource data corresponding to all media resources in the entity group;

and if the current network state is lower than a set value, sending a data acquisition request to the data source side to request to acquire media resource data corresponding to part of media resources in the entity group, wherein the part of media resources comprises the first consumed media resource.

9. The method according to claim 6, wherein the DASH signaling further includes identification information of an entity group, and the identification information of the entity group is used to indicate a group identification of the entity group.

10. The method of processing association of media files according to any of claims 1 to 9, wherein the media files comprise point cloud media files and the media assets comprise point cloud media assets.

11. An association processing method for media files is characterized by comprising the following steps:

generating a media file, wherein the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources;

and sending the media file to a data receiving party so that the data receiving party analyzes the media file and presents the media resource data contained in the media file according to the association relation among the plurality of media resources obtained by analysis.

12. The method of associating a media file as claimed in claim 11, wherein before generating the media file, the method further comprises:

generating a DASH signaling, where the DASH signaling includes entry media resource identification information and a type field of an entity group, where the type field of the entity group is a set value for indicating that media resources in the entity group have an association relationship, and the entry media resource identification information is used for indicating a first consumed media resource;

sending the DASH signaling to the data receiving party, and receiving a data acquisition request sent by the data receiving party;

and determining the media resource data required to be transmitted to the data receiving party according to the data acquisition request, so as to generate the media file according to the media resource data required to be transmitted to the data receiving party.

13. An apparatus for associating and processing media files, comprising:

the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is configured to acquire a media file, the media file comprises media resource data and an object association data box, and the object association data box comprises information of a plurality of media resources and association relations among the plurality of media resources;

the analysis unit is configured to analyze the media file to obtain an association relation between the media resource data contained in the media file and the plurality of media resources;

and the processing unit is configured to present the media resource data contained in the media file according to the incidence relation among the plurality of media resources.

14. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of associated processing of a media file according to any one of claims 1 to 12.

15. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out a method of associated processing of a media file as claimed in any one of claims 1 to 12.