CN110929056B - Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device - Google Patents
Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device Download PDFInfo
- Publication number
- CN110929056B CN110929056B CN201811091556.0A CN201811091556A CN110929056B CN 110929056 B CN110929056 B CN 110929056B CN 201811091556 A CN201811091556 A CN 201811091556A CN 110929056 B CN110929056 B CN 110929056B
- Authority
- CN
- China
- Prior art keywords
- multimedia file
- playing
- file
- data track
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Television Signal Processing For Recording (AREA)
Abstract
The invention provides a multimedia file generating method, a multimedia file playing method, a multimedia file generating device and a multimedia file playing device. The playing method of the multimedia file comprises the following steps: receiving a multimedia file including a panoramic film associated with a time axis; extracting a first data track of a multimedia file to obtain a plurality of first object positions of a first image object in the panoramic film relative to a time axis; when the panoramic film is played, displaying a picture corresponding to the first image object and shown on a screen; in response to detecting the selection operation applied to the icon, a playing angle for playing the panoramic film is determined according to the position of the first object recorded in the first data track, and a frame including the first image object is played based on the playing angle.
Description
Technical Field
The present invention relates to a video playing technology, and more particularly, to a method and an apparatus for generating a multimedia file, and a method and an apparatus for playing a multimedia file.
Background
Nowadays, 360 degree movies and panoramic cameras are becoming more popular, and users can watch 360 degree movies (also called panoramic movies) through a computer screen. Images of different angles shot by the plurality of lenses are sewn through post-processing images, so that a 360-degree film without a view dead angle can be generated, and the effect that a user can see his own situation can be provided.
When a user views a 360-degree movie on a computer screen, the user can only see one corner of the entire 360-degree scene. The user needs to adjust the viewing angle of the 360-degree movie to view different corners of the 360-degree scene. Therefore, when the user watches a 360-degree movie, the user needs to manually adjust the playing viewing angle to search for the object of interest, which greatly reduces the convenience of the user in watching the 360-degree movie. Furthermore, due to the performance of the general consumer electronic products, it is difficult for the general consumer electronic devices to perform real-time image object recognition and tracking on 360-degree movies. Therefore, when a user wants to lock an interested object for watching, the user needs to manually control the playing angle of view at any time and any place along with the movement of the interested object.
Disclosure of Invention
In view of the foregoing, the present invention provides a multimedia file generating method and a multimedia file generating apparatus, which can create a specific track of a multimedia file based on location information of an image object to generate the multimedia file including a panoramic film and having location information recorded therein.
The invention also provides a multimedia file playing method and a multimedia file playing device, which can acquire the position information of the image object according to the specific data track in the multimedia file so as to dynamically adjust the playing angle of view according to the position information of the image object which is interested by a user.
The embodiment of the invention provides a method for generating a multimedia file, which is suitable for a multimedia file generating device and comprises the following steps: obtaining a panorama associated with a time axis, wherein the panorama comprises at least one image object; obtaining a plurality of object positions of the image object relative to a time axis; making the object positions into an object position file; at least one data track of the multimedia file is generated according to the object position file so as to generate the multimedia file which comprises the panoramic film and is recorded with the object position.
The embodiment of the invention provides a multimedia file generating device, which comprises a storage device and a processor. The storage device stores a plurality of modules. The processor is coupled to the storage device and loads and executes the modules in the storage device. The module comprises a film obtaining module, a position obtaining module, a file making module and a file embedding module. The film obtaining module obtains a panorama film associated with a time axis, wherein the panorama film comprises at least one image object. The position obtaining module obtains a plurality of object positions of the image object relative to the time axis. The document making module makes the object positions into an object position document. The file embedding module generates at least one data track of the multimedia file according to the object position file so as to generate the multimedia file which comprises the panoramic film and is recorded with the object position.
Correspondingly, an embodiment of the present invention provides a method for playing a multimedia file, which is suitable for a multimedia file playing apparatus, and the method includes the following steps: receiving a multimedia file including a panorama film associated with a timeline; extracting a first data track of a multimedia file to obtain a plurality of first object positions of a first image object in the panoramic film relative to a time axis; when the panoramic film is played, displaying a picture corresponding to the first image object and shown on a screen; in response to detecting the selection operation applied to the icon, a playing angle for playing the panoramic film is determined according to the position of the first object recorded in the first data track, and a frame including the first image object is played based on the playing angle.
Correspondingly, an embodiment of the present invention provides a multimedia file playing device, which includes a screen, a storage device storing a plurality of modules, and a processor. The processor is coupled with the storage device and the screen, and loads and executes the module in the storage device. The module comprises a film receiving module, a data track extracting module, an interface providing module and a film playing module. The movie reception module receives a multimedia file including a panorama movie associated with a time axis. The data track extraction module extracts a first data track of the multimedia file to obtain a plurality of first object positions of a first image object in the panoramic film relative to a time axis. When the panoramic film is played, the interface providing module displays a picture corresponding to the first image object and displayed on the screen. The film playing module determines a playing angle for playing the panoramic film according to the position of the first object recorded by the first data track in response to detecting the selection operation applied to the graphic, and plays the picture including the first image object based on the playing angle.
Based on the above, the multimedia file generating apparatus can establish the specific data track of the multimedia file according to the object position of the image object appearing in the panorama. Based on this, when playing the panoramic film, the multimedia file playing device can acquire the object position of the image object in the panoramic film from the specific data track of the multimedia file, and further dynamically determine the playing view angle according to the object position of the image object. Therefore, a user can lock a specific image object in the panoramic film for watching without manually adjusting the playing angle of the panoramic film at any time.
In order to make the aforementioned and other features and advantages of the invention more comprehensible, embodiments accompanied with figures are described in detail below.
Drawings
FIG. 1 is a block diagram of a multimedia file generating apparatus according to an embodiment of the present invention.
Fig. 2 is a flowchart illustrating a method of generating a multimedia file according to an embodiment of the present invention.
Fig. 3A and 3B are schematic diagrams illustrating object positions corresponding to time intervals according to an embodiment of the invention.
FIG. 4 is an exemplary object location file according to an embodiment of the present invention
FIG. 5 is a diagram illustrating a multimedia file architecture according to an embodiment of the present invention.
FIG. 6 is a block diagram of a multimedia file playback apparatus according to an embodiment of the present invention.
Fig. 7 is a flowchart illustrating a method for playing a multimedia file according to an embodiment of the present invention.
Fig. 8A and 8B are schematic diagrams illustrating an exemplary playing of a multimedia file according to an embodiment of the invention.
[ notation ] to show
10: multimedia file generating device
60: multimedia file playing device
110. 610: processor with a memory having a plurality of memory cells
120. 620: storage device
630: screen
121: film acquisition module
122: position acquisition module
123: file making module
124: file embedding module
621: film receiving module
622: data track extraction module
623: interface providing module
624: film playing module
P1 to P3: time interval
40: article position file
50: multimedia file
51: header head
52: multimedia data
521: video data track
522: audio data track
523: caption data track
524: object position data rail
80. 86: picture frame
82: virtual control button
I1 to I3: illustration of the drawings
83. 84: image object
S201 to S204, S701 to S704: step (ii) of
Detailed Description
Some embodiments of the invention will now be described in detail with reference to the drawings, wherein like reference numerals are used to refer to like or similar elements throughout the several views. These examples are only a part of the present invention and do not disclose all possible embodiments of the present invention. Rather, these embodiments are merely exemplary of the method and multimedia file generating apparatus of the present invention as set forth in the claims.
FIG. 1 is a block diagram of a multimedia file generating apparatus according to an embodiment of the present invention, which is for convenience of illustration only and is not intended to limit the present invention. First, fig. 1 first describes all the components and configuration relationships of the multimedia file generating apparatus, and the detailed functions will be disclosed together with fig. 2.
Referring to fig. 1, the multimedia file generating apparatus 10 may be any electronic apparatus with computing capability, such as a desktop computer, a notebook computer, a server, etc., but the present invention is not limited thereto. The multimedia file generating apparatus 10 includes a processor 110 and a storage device 120, and the functions thereof are as follows:
the storage device 120 is, for example, any type of fixed or removable Random Access Memory (RAM), read-only memory (ROM), flash memory (flash memory), or the like or combination thereof. In the present embodiment, the storage device 120 is used for recording a movie acquisition module 121, a location acquisition module 122, a file creation module 123, and a file embedding module 124.
The Processor 110 is, for example, a Central Processing Unit (CPU), or other Programmable general purpose or special purpose Microprocessor (Microprocessor), digital Signal Processor (DSP), programmable controller, application Specific Integrated Circuit (ASIC), programmable Logic Device (PLD), or other similar Device or combination thereof, and is connected to the storage Device 120.
In the present embodiment, the module stored in the storage device 120 is, for example, a computer program, and can be loaded by the processor 110 to execute the method for generating a multimedia file according to the present embodiment.
Fig. 2 is a flowchart illustrating a multimedia file generating method according to an embodiment of the present invention, and the method flow of fig. 2 can be implemented by the elements of the multimedia file generating apparatus 10 of fig. 1. Referring to fig. 1 and fig. 2, the following describes detailed steps of the method for generating a multimedia file according to the present embodiment in combination with various components and devices of the multimedia file generating device 10 in fig. 1.
In step S201, the movie acquisition module 121 acquires a panoramic movie associated with a time axis, wherein the panoramic movie includes at least one image object. Here, the movie retrieving module 121 may retrieve the panoramic movie from an image acquiring module (not shown) of the multimedia file generating apparatus 10 itself or from other electronic devices. A panoramic film, which may also be referred to as a 360-degree film, is composed of video frames corresponding to different time stamps (timestamps) on a time axis, and the video frames are 360-degree images stored in a specific format. The specific format is, for example, an Equiangular format or the like. It should be noted that, in the embodiment of the present invention, the panorama film includes at least one image object generated by shooting at least one object, that is, the image object is presented in a video frame of the panorama film. The image object in the panoramic film is, for example, a human face, but the present invention is not limited thereto, and may be other kinds of image objects.
In step S202, the position obtaining module 122 obtains a plurality of object positions of the image object relative to the time axis. In one embodiment, the object positions of the image object can be visually observed by a film editor in advance and manually edited to generate the image object. In other words, the position obtaining module 122 can obtain a plurality of object positions of the image object in a three-dimensional coordinate system by allowing a film editor to watch the panoramic film with naked eyes and labeling the object positions of the image object. Alternatively, in one embodiment, the object positions of the graphical object relative to the timeline may be automatically generated by an object detection and recognition algorithm of the image processing technique. In other words, by using the object detection and identification algorithm to track a specific image object in the panoramic film, the position obtaining module 122 can obtain a plurality of object positions of the image object relative to different time intervals in a three-dimensional coordinate system. The object position of the image object may be represented by the spherical coordinates of a spherical coordinate system, for example.
In an embodiment, the object positions of the image object respectively correspond to a plurality of time intervals on a time axis. That is, the object positions of the image objects can be sampled according to a fixed or unfixed time interval. Referring to fig. 3A, fig. 3A is a schematic diagram illustrating positions of a plurality of objects corresponding to a plurality of time intervals according to an embodiment of the invention. For a video object, the position obtaining module 122 can obtain the object position (r 1, θ 1, ψ 1) corresponding to the time interval P1, the object position (r 2, θ 2, ψ 2) corresponding to the time interval P2, and the object position (r 3, θ 3, ψ 3) corresponding to the time interval P3. It should be noted that the time lengths of the time intervals P1 to P3 may be the same or different, and the present invention is not limited thereto.
In addition, in an embodiment, the number of image objects in the panoramic film may be more than two. Thus, the at least one image object in the panoramic film may include a first image object and a second image object. Correspondingly, the object positions relative to the time axis will include a plurality of first object positions of the first-image object and a plurality of second object positions of the second-image object. Referring to fig. 3B, fig. 3B is a schematic diagram illustrating positions of a plurality of objects corresponding to a plurality of time intervals according to an embodiment of the invention. For the first image object, the position obtaining module 122 can obtain the object position (r 4, θ 4, ψ 4) corresponding to the time interval P1 and the object position (r 5, θ 5, ψ 5) corresponding to the time interval P2. For the second image object, the position obtaining module 122 can obtain the object position (r 6, θ 6, ψ 6) corresponding to the time interval P1 and the object position (r 7, θ 7, ψ 7) corresponding to the time interval P2.
Then, returning to the flow of fig. 2, in step S203, the document making module 123 makes the object positions into an object position document. Specifically, the file creating module 123 may compile the object positions corresponding to the time intervals on the time axis into an object position file in a preset file format. In one embodiment, the object location file may be generated in a manner similar to the generation of the movie subtitle file. Referring to fig. 4, fig. 4 is a diagram illustrating an example of an object location file according to an embodiment of the invention. The object location file 40 records object locations of two image objects in the panorama, named object name a and object name B, respectively, and the object locations are recorded at regular time intervals. The example shown in fig. 4 is a time interval of 1 second, but the invention is not limited thereto. For example, at time point 00. At time point 00.
In addition, in an embodiment, the file creating module 123 may map the object positions recorded as the three-dimensional position coordinates into two-dimensional position coordinates, and record the two-dimensional position coordinates in the object position file. In general, each video frame in a panoramic film is stored by mapping a panoramic image into a two-dimensional image, such as in the Equiangular format. The object positions recorded as a plurality of three-dimensional position coordinates (e.g., spherical coordinates) can also be mapped to two-dimensional position coordinates in a two-dimensional coordinate system and stored, so as to reduce the data size of the object position file.
Then, in step S204, the file embedding module 124 generates at least one data track of the multimedia file according to the object location file to generate a multimedia file including the panoramic film and having the object location recorded therein. Specifically, fig. 5 is a schematic diagram of a multimedia file architecture according to an embodiment of the present invention. The multimedia file 50 includes a header 51 and multimedia data 52, and the multimedia data 52 includes multimedia data that can be classified into a plurality of data tracks. In other words, the multimedia file 50 may include a plurality of data tracks. The header 51 records therein a description of the characteristics of these tracks and the number of these tracks, which may include a video track 521, an audio track 522, a subtitle track 523, and an object position track 524. Wherein, the video data track is used for classifying the video data; the audio data tracks are used for classifying the audio data, and different audio data tracks can represent different languages; the subtitle data track is used to classify subtitle data, and different subtitle data tracks may represent subtitles in different languages.
In one embodiment, when the object location file includes a plurality of first object locations of the first graphic object and a plurality of second object locations of the second graphic object (as shown in the example of fig. 4), the file embedding module 124 may generate a first data track corresponding to the first graphic object and embed the first object locations (e.g., (r 4, θ 4, ψ 4), (r 6, θ 6, ψ 6)) in the object location file into the first data track. On the other hand, the file embedding module 124 may generate a second data track corresponding to the second image object, and embed the second object position in the object position file (e.g., (r 5, θ 5, ψ 5), (r 7, θ 7, ψ 7), (r 9, θ 9, ψ 9) of fig. 4) into the second data track. That is, the number of object position data tracks is determined by the number of labeled view image objects, and the object position of each image object is recorded by the corresponding object position data track. That is, different object position data tracks may represent position information of different image objects.
It is noted that, compared to the conventional multimedia file, the multimedia file 50 of the present embodiment further includes an object position data track 524 for recording the object position. The file embedding module 124 can establish at least one data track (i.e., the object location data track 524) of the multimedia file 50 according to the object location file, such as embedding the data in the object location file 40 shown in fig. 4 into the object location data track 524 of the multimedia file 50. Herein, embedding specific data into at least one data track of the multimedia file 50 represents embedding specific data into data blocks of the data track in the multimedia file 50. Furthermore, the header 51 records the description of the characteristics of the object location data tracks and the number of the object location data tracks. In this way, the player for playing the multimedia file 50 can obtain the position information of one or more image objects in the panoramic image from the object position track 524, in addition to playing the panoramic image in the multimedia file 50.
After describing how to generate the multimedia file recorded with the object positions of the image objects in the panoramic film, the following embodiments will describe how to play the panoramic film according to the multimedia file of the present disclosure.
Fig. 6 is a block diagram of a multimedia file playing apparatus according to an embodiment of the present invention, which is for convenience of illustration only and is not intended to limit the present invention. First, fig. 6 first describes all the components and configuration relationships of the multimedia file playing apparatus, and the detailed functions will be disclosed together with fig. 7.
Referring to fig. 6, the multimedia file playing device 60 may be any electronic device with computing capability and image display capability, such as a desktop computer, a notebook computer, a smart phone, a tablet, and the like, which is not limited in the present invention. The multimedia file playing device 60 includes a processor 610, a storage device 620 and a screen 630.
The storage device 620 can be any type of fixed or removable random access memory, read only memory, flash memory, or the like, or any combination thereof. In the present embodiment, the storage device 620 is used for recording a movie receiving module 621, a data track retrieving module 622, an interface providing module 623, and a movie playing module 624. In one embodiment, the module may be implemented as a software player.
The processor 610 is, for example, a central processing unit or other programmable general or special purpose microprocessor, digital signal processor, programmable controller, application specific integrated circuit, programmable logic device, or the like, or a combination thereof, coupled to the storage device 620.
The screen 630 is used for displaying the image outputted from the multimedia file playing device 60 for the user to view. In the present embodiment, the multimedia file playing device 60 is, for example, a Liquid Crystal Display (LCD), a Light-Emitting Diode (LED) Display, a Field Emission Display (FED), or other types of displays.
In the present embodiment, the module stored in the storage device 620 is, for example, a computer program, and can be loaded by the processor 610 to execute the method for playing a multimedia file according to the present embodiment.
Fig. 7 is a flowchart illustrating a method for playing a multimedia file according to an embodiment of the present invention, and the method flowchart of fig. 7 can be implemented by the elements of the multimedia file playing apparatus 60 of fig. 6. Referring to fig. 6 and fig. 7, the following describes detailed steps of the method for playing a multimedia file according to the present embodiment in conjunction with various elements and devices of the multimedia file generating device 60 in fig. 6.
In step S701, the movie receiving module 621 receives a multimedia file including a panorama movie associated with a time axis. The movie receiving module 621 may receive a multimedia file including a panoramic movie via a wired or wireless network, and may also read the multimedia file stored in the storage device 620 or other external storage devices. In step S702, the track extraction module 622 extracts a first track of the multimedia file to obtain a plurality of first object positions of a first image object in the panorama film relative to a time axis. Specifically, the track extraction module 622 can demultiplex (demux) the multimedia file to obtain the multimedia data corresponding to each track. In one embodiment, the tracks of the multimedia file may include a video track, an audio track, a subtitle track, and an object position track. The track extraction module 622 can extract multimedia data classified into an object position track from the multimedia file, where the multimedia data classified into the object position track is a plurality of first object positions of a first image object in the panorama relative to a time axis. The object positions in the object position data track are described in detail in the foregoing embodiments, and are not described again. Similarly, the track extraction module 622 can also extract the video data classified into the video tracks from the multimedia file and decode the video data to obtain a plurality of video frames of the panorama film.
Thereafter, in step S703, when the panoramic film is played, the interface providing module 623 displays a frame corresponding to the first image object and shown on the screen 630. Specifically, the interface providing module 623 can provide a user interface of the player, which can include a frame playing area and a playing control column. It should be noted that by parsing the number of object location tracks in the header (e.g., the header 51 shown in fig. 5) of the multimedia file, the interface providing module 623 can know how many image objects are marked in advance in the movie content of the panorama film. Thus, while playing the panoramic film, the interface providing module 623 may display the image of the image object labeled in advance on the screen 630. The graphic representations can be any shape of interactive objects, and the names or representative patterns of the corresponding image objects are presented in each graphic representation to quickly guide the user to the emphasis on the panoramic image. In addition, each icon can be displayed on the edge of the playing frame or in the playing control column of the player, so as to avoid affecting the user's watching of the panoramic film.
The processor 610 then continuously detects whether the user selects any icon, and responds to the detection of the selection operation applied to a certain icon by the user. Therefore, in step S704, in response to detecting the selection operation applied to the icon, the movie playing module 624 determines a playing angle for playing the panoramic movie according to the position of the first object recorded in the first track, and plays the frame including the first image object based on the playing angle. That is, when the user selects the icon corresponding to the first image object, the film playing module 624 can obtain the current object position of the first image object in the panorama film from the object position data track. Then, the film playing module 624 can determine the playing angle of view according to the current object position of the first image object, and the playing frame will be shifted from the preset area of the panoramic film to the first area where the first image object is located, so that the user can quickly view the selected key object.
It is noted that the first object position of the selected first image object may change. Taking fig. 3B as an example, the first object position of the first image object may be changed from (r 4, θ 4, ψ 4) to (r 6, θ 6, ψ 6). If the playing angle is not adjusted, the first image object may disappear from the playing frame. In an embodiment, the movie playing module 624 may switch the playing angle according to the changed position of the first object again in response to recognizing the change of the position of the first object. Taking fig. 3B as an example, in response to the first object position of the first image object changing from (r 4, θ 4, ψ 4) to (r 6, θ 6, ψ 6), the movie playback module 624 switches the playback view angle from the first view angle to the second view angle. Correspondingly, the playing frame is adjusted from the original first area to the second area where the first image object is located. That is, the film playing module 624 plays the first area of the panoramic film at the first viewing angle in the time interval P1, and then plays the second area of the panoramic film at the second viewing angle in the time interval P2. Thus, the user can continuously watch the selected key object without manually adjusting the playing angle of view.
It is appreciated that the number of image objects may be more than two. In one embodiment, in addition to extracting the first track of the multimedia file, the track extraction module 622 can also extract a second track of the multimedia file to obtain a plurality of second object positions of the second image object in the panoramic film relative to the time axis. Thus, when the panorama film is played, the interface providing module 623 will also display another picture corresponding to the second image object on the screen 630. Then, in response to detecting the selection operation applied to another illustration, the movie playing module 624 switches the playing angle according to the position of the second image object recorded in the second data track, and plays the frame including the second image object based on the switched playing angle.
For example, fig. 8A and 8B are schematic diagrams illustrating an exemplary playing of a multimedia file according to an embodiment of the invention. Referring to fig. 8A, when the multimedia file playing apparatus 60 plays the multimedia file generated by the present disclosure, the panoramic film is played along with the time axis. The user can adjust the playing angle of the panoramic film by operating the virtual control button 82. The multimedia file playing device 60 can obtain the description characteristics of the object position data track and the number of the object position data tracks according to the header of the multimedia file, so as to obtain the number of the image objects labeled in advance, the object names, and the like. In the present example, assuming that the number of the image objects labeled in advance is 3, the multimedia playing device 60 will display three icons I1-I3 on the frame 80, and the three icons I1-I3 respectively show the representative names 'A', 'B' and 'C' of the three image objects.
Assuming that the user wants to watch the image object 83 corresponding to the icon I1 (i.e. the key character a), in response to detecting that the user performs a selection operation on the icon I1, the multimedia file playing apparatus 60 determines a playing perspective for playing the panoramic film according to the object position of the image object 83 recorded in the object position data track, so as to play the frame 80 including the image object 83 according to the just determined playing perspective. In this example, the selected image object 83 will be located in the middle of the frame 80. Then, assuming that the user wants to view the image object 84 corresponding to the icon I2 (i.e. the key character B), in response to detecting that the user performs a selection operation on the icon I2, the multimedia file playing apparatus 60 switches the playing perspective according to the object position of the image object 84 recorded in the object position data track, and plays the frame 86 including the image object 84 based on the switched playing perspective. In the present example, after switching the playback perspective, the selected image object 84 is located in the middle of the frame 86.
In summary, in the embodiment of the present invention, the multimedia file including the panoramic film further includes an object position track recorded with position information of the image object. The multimedia file generating device embeds the object position of the image object into the multimedia file, so that the multimedia file playing device can instantly know the object position of the specific image object according to the object position data track during playing the panoramic film. Therefore, the multimedia playing device of the user does not need to have strong computing capability to identify and track the image object. In addition, after the user selects the image object of interest, the multimedia file playing device can dynamically adjust the playing angle of the panoramic film according to the object position of the image object, so as to achieve the playing function of tracking the specific image object. Therefore, the user does not need to manually adjust the playing angle of view to ensure that the user can watch the interested image object, thereby greatly improving the convenience of watching the 360-degree film. The invention also can enable the user to quickly browse the key points in the panoramic film, so that the user can have direct and quick operation and watching experience when watching the panoramic film.
Although the present invention has been described with reference to the above embodiments, it should be understood that various changes and modifications can be made therein by those skilled in the art without departing from the spirit and scope of the invention.
Claims (16)
1. A method for generating a multimedia file, the method being adapted for a multimedia file generating apparatus, the method comprising:
obtaining a panorama associated with a time axis, wherein the panorama comprises at least one image object;
obtaining a plurality of object positions of the at least one image object relative to the time axis;
making the plurality of object positions into an object position file; and
generating at least one data track of a multimedia file according to the object position file to generate the multimedia file which comprises the panoramic film and is recorded with the plurality of object positions,
wherein the multimedia file comprises a header and a plurality of data tracks, the plurality of data tracks comprises a video data track and at least one object position data track for recording the plurality of object positions, and the header records a description of characteristics of the at least one object position data track and the number of the at least one object position data track,
wherein the at least one graphic object comprises a first graphic object, the plurality of object locations relative to the timeline comprise a plurality of first object locations of the first graphic object, and generating the at least one data track of the multimedia file according to the object location file comprises:
embedding the plurality of first object positions in the object position file into a first data track.
2. The method of claim 1, wherein the at least one image object comprises a second image object, the plurality of object locations relative to the timeline comprises a plurality of second object locations of the second image object, and generating the at least one track of the multimedia file from the object location file comprises:
embedding the plurality of second object positions in the object position file into a second data track.
3. The method of claim 1, wherein the object positions of the at least one image object correspond to time intervals on the time axis, respectively.
4. The method of claim 1, wherein the step of producing the plurality of item locations into the item location file comprises:
mapping the object positions recorded as the three-dimensional position coordinates into two-dimensional position coordinates, and recording the two-dimensional position coordinates into the object position file.
5. A multimedia file generating apparatus comprising:
a storage device storing a plurality of modules;
a processor, coupled to the storage device, for loading and executing the module in the storage device, the module comprising:
a movie obtaining module, configured to obtain a panoramic movie associated with a time axis, where the panoramic movie includes at least one image object;
a position obtaining module for obtaining a plurality of object positions of the at least one image object relative to the time axis;
the file making module is used for making the plurality of object positions into an object position file; and
a file embedding module for generating at least one data track of a multimedia file according to the object position file to generate the multimedia file which comprises the panoramic film and is recorded with the plurality of object positions,
wherein the multimedia file comprises a header and a plurality of data tracks, the plurality of data tracks comprises a video data track and at least one object position data track for recording the plurality of object positions, and the header records a description of characteristics of the at least one object position data track and the number of the at least one object position data track,
wherein the at least one image object comprises a first image object, the plurality of object locations relative to the timeline comprise a plurality of first object locations of the first image object, and the file embedding module embeds the plurality of first object locations in an object location file into a first data track.
6. The multimedia file generating device as claimed in claim 5, wherein said at least one image object comprises a second image object, said plurality of object positions with respect to said timeline comprises a plurality of second object positions of said second image object,
wherein the file embedding module embeds the plurality of second object locations in the object location file into a second data track.
7. The apparatus for generating multimedia files according to claim 5, wherein the object positions of the at least one image object respectively correspond to time intervals on the time axis.
8. The multimedia file generation device of claim 5, wherein the file creation module further maps the plurality of object positions recorded as a plurality of stereoscopic position coordinates into a plurality of two-dimensional position coordinates, and records the plurality of two-dimensional position coordinates in the object position file.
9. A playing method of a multimedia file is suitable for a multimedia file playing device, and comprises the following steps:
receiving a multimedia file including a panorama film associated with a timeline;
extracting a first data track of the multimedia file to obtain a plurality of first object positions of a first image object in the panoramic film relative to the time axis;
when the panoramic film is played, displaying a picture corresponding to the first image object and shown on a screen; and
in response to detecting the selection operation applied to the icon, determining a playing angle for playing the panoramic image according to the positions of the first objects recorded in the first track, and playing a frame including the first image object based on the playing angle,
the multimedia file comprises a header and a plurality of data tracks, wherein the plurality of data tracks comprise a video data track and at least one object position data track for recording a plurality of object positions, and the header records the description of the characteristics of the at least one object position data track and the number of the at least one object position data track.
10. The method of claim 9, wherein the step of determining the playing perspective for playing the panoramic film according to the plurality of first object positions recorded by the track in response to detecting the selection operation applied to the icon comprises:
in response to identifying a change in the plurality of first object positions, switching the playback perspective.
11. The method of claim 9, further comprising:
extracting a second data track of the multimedia file to obtain a plurality of second object positions of a second image object in the panoramic film relative to the time axis; and
when the panoramic film is played, displaying another picture corresponding to the second image object and shown on the screen.
12. The method of claim 11, wherein after the steps of determining the playback perspective for playing back the panoramic film according to the plurality of first object positions recorded in the first track and playing back the picture including the first image object based on the playback perspective in response to detecting the selection operation applied to the illustration, the method further comprises:
in response to detecting the selection operation applied to the other icon, switching the playback angle of view according to the plurality of second object positions recorded by the second data track, and playing a frame including the second image object based on the switched playback angle of view.
13. A multimedia file playback apparatus comprising:
a screen;
a storage device storing a plurality of modules;
a processor, coupled to the storage device and the screen, for loading and executing the module in the storage device, the module comprising:
a movie receiving module receiving a multimedia file including a panorama movie associated with a time axis;
a data track extraction module, configured to extract a first data track of the multimedia file to obtain a plurality of first object positions of a first image object in the panorama relative to the time axis;
the interface providing module is used for displaying a picture corresponding to the first image object and shown on the screen when the panoramic film is played; and
a video playback module, responsive to detecting a selection operation applied to the icon, for determining a playback perspective for playing back the panoramic video according to the positions of the first objects recorded in the first track, and playing back a frame including the first image object based on the playback perspective,
the multimedia file comprises a header and a plurality of data tracks, wherein the plurality of data tracks comprise a video data track and at least one object position data track for recording a plurality of object positions, and the header records the description of the characteristics of the at least one object position data track and the number of the at least one object position data track.
14. The multimedia file playback device of claim 13, wherein the movie playback module switches the playback perspective in response to identifying a change in the position of the first plurality of objects.
15. The multimedia file playback device of claim 13, wherein the track extraction module extracts a second track of the multimedia file to obtain a plurality of second object positions of a second image object in the panoramic film relative to the time axis; and when the panoramic film is played, the interface providing module displays another picture corresponding to the second image object and shown on the screen.
16. The multimedia file playing apparatus as claimed in claim 15, wherein the movie playing module switches the playing perspective according to the plurality of second object positions recorded by the second data track in response to detecting the selection operation applied to the another graphic, and plays the frame including the second image object based on the switched playing perspective.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811091556.0A CN110929056B (en) | 2018-09-19 | 2018-09-19 | Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811091556.0A CN110929056B (en) | 2018-09-19 | 2018-09-19 | Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110929056A CN110929056A (en) | 2020-03-27 |
| CN110929056B true CN110929056B (en) | 2023-04-07 |
Family
ID=69855054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811091556.0A Active CN110929056B (en) | 2018-09-19 | 2018-09-19 | Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110929056B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006004544A (en) * | 2004-06-18 | 2006-01-05 | Inventec Multimedia & Telecom Corp | Multifunctional multimedia recording and reproducing apparatus |
| CN106954095A (en) * | 2017-04-17 | 2017-07-14 | 腾讯科技（深圳）有限公司 | The player method and device of a kind of multimedia file |
| CN107147824A (en) * | 2016-06-22 | 2017-09-08 | 深圳市量子视觉科技有限公司 | The output intent and device of multi-angle video |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8094182B2 (en) * | 2006-11-16 | 2012-01-10 | Imove, Inc. | Distributed video sensor panoramic imaging system |
| US20080294691A1 (en) * | 2007-05-22 | 2008-11-27 | Sunplus Technology Co., Ltd. | Methods for generating and playing multimedia file and recording medium storing multimedia file |
| WO2011114610A1 (en) * | 2010-03-18 | 2011-09-22 | パナソニック株式会社 | Omnidirectional image processing device and omnidirectional image processing method |
| US9781349B2 (en) * | 2016-01-05 | 2017-10-03 | 360fly, Inc. | Dynamic field of view adjustment for panoramic video content |
| CN106445437A (en) * | 2016-09-08 | 2017-02-22 | 深圳市金立通信设备有限公司 | Terminal and view angle switching method thereof |
| CN106331732B (en) * | 2016-09-26 | 2019-11-12 | 北京疯景科技有限公司 | Generate, show the method and device of panorama content |
| CN107888987B (en) * | 2016-09-29 | 2019-12-06 | 华为技术有限公司 | Panoramic video playing method and device |
| CN107633241B (en) * | 2017-10-23 | 2020-11-27 | 三星电子（中国）研发中心 | Method and device for automatically marking and tracking object in panoramic video |
-
2018
- 2018-09-19 CN CN201811091556.0A patent/CN110929056B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006004544A (en) * | 2004-06-18 | 2006-01-05 | Inventec Multimedia & Telecom Corp | Multifunctional multimedia recording and reproducing apparatus |
| CN107147824A (en) * | 2016-06-22 | 2017-09-08 | 深圳市量子视觉科技有限公司 | The output intent and device of multi-angle video |
| CN106954095A (en) * | 2017-04-17 | 2017-07-14 | 腾讯科技（深圳）有限公司 | The player method and device of a kind of multimedia file |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110929056A (en) | 2020-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107633241B (en) | Method and device for automatically marking and tracking object in panoramic video | |
| KR101946019B1 (en) | Video processing apparatus for generating paranomic video and method thereof | |
| US8875212B2 (en) | Systems and methods for remote control of interactive video | |
| US11211097B2 (en) | Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus | |
| US8656282B2 (en) | Authoring tool for providing tags associated with items in a video playback | |
| CN110286773A (en) | Information providing method, device, equipment and storage medium based on augmented reality | |
| US20160198097A1 (en) | System and method for inserting objects into an image or sequence of images | |
| US8457407B2 (en) | Electronic apparatus and image display method | |
| US11734931B2 (en) | Information processing apparatus, information processing method, and storage medium | |
| ES2914124T3 (en) | Media targeting | |
| CN113806036A (en) | Output of virtual content | |
| EP3236336B1 (en) | Virtual reality causal summary content | |
| TW201520827A (en) | System and method of providing augmented reality effect for multi-media data | |
| CN112232260A (en) | Subtitle region identification method, device, equipment and storage medium | |
| KR102505973B1 (en) | Image processing apparatus, control method thereof and computer readable medium having computer program recorded therefor | |
| US8244005B2 (en) | Electronic apparatus and image display method | |
| JP6203188B2 (en) | Similar image search device | |
| US20050251741A1 (en) | Methods and apparatus for capturing images | |
| CN106936830B (en) | Multimedia data playing method and device | |
| US20110305430A1 (en) | Electronic apparatus and movie playback method | |
| CN110929056B (en) | Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device | |
| US20230043683A1 (en) | Determining a change in position of displayed digital content in subsequent frames via graphics processing circuitry | |
| KR101518696B1 (en) | System for augmented reality contents and method of the same | |
| KR101843024B1 (en) | System and Computer Implemented Method for Playing Compoiste Video through Selection of Environment Object in Real Time Manner | |
| TWI762830B (en) | System for displaying hint in augmented reality to play continuing film and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |