KR102314354B1 - Method for receiving web contents for 360-degree multimedia device and apparatus for the same - Google Patents

Abstract

The present invention relates to a method and an apparatus for receiving web content for a 360-degree multimedia device. More specifically, the present invention relates to a method and an apparatus for providing HTML5-based web content which can be executed and played back in a virtual reality (VR) or an augmented reality (AR) environment. The method and apparatus for receiving web content for a 360-degree multimedia device in accordance with an embodiment of the present invention can use HTML5 to generate web content which interacts with a user on a universally-used user terminal such as a mobile phone, a personal computer, and the like. The method and apparatus for receiving web content for a 360-degree multimedia device in accordance with an embodiment of the present invention can provide web content which interacts with a user by joining the HTML5 to image data. The method and apparatus for receiving web content for a 360-degree multimedia device in accordance with an embodiment of the present invention can allow a user terminal to use only a web browser without installation of a separate program and the like to consume the web content even with a small amount of a memory by joining the HTML5 to the image data. In accordance with the present invention, the apparatus for generating web content which interacts with a user comprises: a reception unit for receiving image data, one or more hypertext markup language (HTML) file, and time information corresponding to each HTML file; a web content generation unit for generating web content; a transmission unit for transmitting the generated web content to a user terminal; and a display unit.

Description

Web content receiving method and apparatus for 360-degree multimedia device

The present invention relates to a method and apparatus for receiving web content for a 360-degree multimedia device. More particularly, it relates to a method and an apparatus for providing HTML5-based web content that can be executed and played in a VR/AR environment.

In the past, there are many implementation examples of playing a moving picture using a flash on a personal computer (PC). Various functions are implemented, such as playing a video on a web browser, playing forward, playing backward, and playing at a desired time.

In addition, it is also possible to display the data related to the video in synchronization with the web browser. However, since such flash content is not applied to mobile terminals, functions such as displaying related learning materials or solving problems or taking notes while playing educational video content on the mobile terminal cannot be implemented.

That is, on a mobile terminal, educational video content can only be reproduced on one screen, and related learning materials cannot be displayed while viewing the educational video content, and the user cannot take notes or solve problems.

HTML5 (hypertext mark-up language 5), the latest protocol, has the advantage of being able to implement flash-like videos and music, as well as colorful graphic effects, on a web browser without installing active X. .

However, it is impossible to implement such simultaneous output and synchronization even by HTML5 that implements a web page on a web browser of a mobile terminal. Therefore, it is not possible with only the existing HTML5 convention, and it is possible only with a change to the HTML5 convention or the addition of a new configuration.

No. 10-2013-0082732 (2013.07.22)

An object of the present invention for solving the above problems is to provide a method and apparatus for receiving web content for a 360-degree multimedia device.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be able

In order to solve the above problems, a method for receiving web content for a 360-degree multimedia device according to embodiments includes receiving data for web content, the data for the web content includes 360-degree image data, one or more data including HTML files, eXtensible Markup Language (XML) data representing the temporal information corresponding to each HTML file, and a JavaScript source for parsing the XML data; parsing data for the web content; rendering the web content based on a position of a user's viewport; may include, and the web content may be configured to reproduce the 360-degree image data based on the JavaScript source.

Further, the web content is configured to drive an HTML file corresponding to each time information by extracting current time information based on the JavaScript source, and the web content is a DTD (Document Type Definition) used to parse the XML data It may contain more data.

Further, in the method for receiving web content for a 360-degree multimedia device according to embodiments, the data for the web content includes one or more widgets, and the one or more widgets include the one or more HTML Files may be mapped to the 360-degree image data.

Furthermore, in the method for receiving web content for a 360-degree multimedia device according to embodiments, each HTML file may further include location information indicating a location in the 360-degree image data of at least one widget mapped by the HTML file. can

Furthermore, in the method for receiving web content for a 360-degree multimedia device according to embodiments, the location information includes first azimus information indicating the leftmost position of the at least one widget on a sphere, and the topmost position of the at least one widget. It may include second azimuth information indicating a right position, first elevation information indicating an uppermost position of the at least one widget, and second elevation information indicating a lowermost position of the at least one widget. .

Furthermore, each of the HTML files may further include rotation information indicating a degree of rotation of the at least one widget.

The method and the apparatus for receiving web content for a 360-degree multimedia device according to embodiments may generate web content that can interact with a user in a general-purpose user terminal device such as a mobile device or a PC by using HTML5.

A method and apparatus for receiving web content for a 360-degree multimedia device according to embodiments may provide web content that can interact with a user by combining HTML5 with image data.

A method and apparatus for receiving web content for a 360-degree multimedia device according to embodiments, by combining HTML5 with image data, allows the user's terminal device to use only a web browser without installing a separate program, using only a web browser with a small amount of memory. You can consume content.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

Other aspects, features and benefits as described above of certain preferred embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings.
1 illustrates a system for generating interactive web content according to embodiments.
FIG. 2 is an exemplary implementation diagram of a learning apparatus enabling interaction between a learning area and a video area using HTML5 in a mobile terminal according to an embodiment of the present invention.
3 illustrates web content and a method of reproducing web content according to embodiments.
4 shows an example of web content according to embodiments.
5 shows an example of the structure of a 360-degree multimedia device 50 according to embodiments.
6 shows 360-degree contents according to embodiments.
7 illustrates an example of web content for a 360-degree multimedia device according to embodiments.
8 illustrates a method of indicating a location of a widget in web content for a 360-degree multimedia device according to embodiments.
9 illustrates a structure of web content according to embodiments.
10 is a flowchart illustrating a method for receiving web content for a 360-degree multimedia device according to embodiments.
It should be noted that throughout the drawings, like reference numerals are used to denote the same or similar elements, features, and structures.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a system for generating interactive web content according to embodiments.

The system for generating interactive web content according to embodiments may be performed by the generating device (or its server) 100 for generating interactive web content according to embodiments. The generating device 100 performs a method of generating interactive web content according to embodiments.

The generating device 100 may provide web content according to embodiments to users. The generating device 100 may provide, for example, a user's desired web content to the user's terminal device 101 according to a user's request.

The generating device 100 may, for example, transmit ( 10b ) a finished product of the web content that the user intends to produce according to the user's request ( 10a ). The web content 10d may mean, for example, web-based content that can interact with a user. The web content 10d may be composed of, for example, a program or a set of commands run in a web browser.

The web content 10d may be web-based content driven by HyperText Markup Language 5 (HTML5). The web content 10d may be, for example, learning content driven by HTML5 and provided to a user by including one or more of a video, music, and photo. In the past, since flash was not implemented on a mobile terminal, in the case of educational video content, only the educational video content could be played, but a separate learning area could not be implemented at the same time. However, the web content 10d according to the embodiments may be configured to display the video content of the video area and the learning material of the learning area in synchronization with time according to the HTML5 convention.

That is, in other words, the user may create web content (eg, learning content, etc.) that the user wants to produce by using the user's terminal device 101 . The user's terminal device 101 transmits (10a) information on the web content that the user wants to produce to the generating device 101 according to the embodiments. The generating device 101 receives information about the web content, and generates the web content 10d that the user wants to produce based on the information. The generating device 100 transmits the generated web content 10d to the user's terminal device 101 .

The user's terminal device 101 according to the embodiments may be a 360-degree multimedia device that receives 360-degree image data (image data), decodes and parses them, and renders the user's viewport. have. For example, the user's terminal device 101 according to embodiments may mean a head-mounted display. The user's terminal device 101 according to the embodiments may be a device that receives 360-degree image/video data and processes them to enable the user to consume web content in a VR environment.

Due to this configuration, the system for generating interactive web content according to the embodiments uses HTML5 in an environment other than a browser environment running on a PC to provide learning content that enables interaction between the learning area and the video area. can create

The method and the apparatus for generating interactive web content according to the embodiments may generate web content that can interact with the user in a general-purpose user terminal device such as a mobile device or a PC by using HTML5.

A method and an apparatus for generating interactive web content according to embodiments may provide web content that can interact with a user by combining HTML5 with image data.

The method and the apparatus for generating interactive web content according to the embodiments allow the user's terminal device to use only a web browser without installing a separate program by combining HTML5 with image data even with a small amount of memory. You can consume content.

2 shows an example of web content according to embodiments.

The web content 20 shown in FIG. 2 may mean part or all of the web content 10d described in FIG. 1 .

Referring to FIG. 2 , web content 20 according to embodiments may include an HTML5 area 21A and/or an image area 21B. The web content 20 may include both the HTML5 area 21A and/or the image area 21B, or only one of them. Also, the region may occupy a partial region on the web content 20 , but may not occupy a specific region.

The web content 20 may be an image whose content changes over time. For example, the content of the web content 20 appearing at the first time may be different from the content of the web content 20 appearing at the second time. The web content 20 may be an image whose content changes according to a user's manipulation or input. For example, the content of the web content 20 may change according to a user's manipulation.

A configuration that changes according to a user's manipulation within the web content 20 may be specified by HTML5 or a file such as CSS or JavaScript referring to HTML5.

The HTML5 region 21A may be driven by HTML5. The HTML5 region may be driven by HTML5, but may also be driven by a CSS file, a JavaScript file, and the like that refer to HTML5.

The image area 21B may be an area driven by an image file. The image file may be, for example, a file including a moving picture, a file including music (audio), an image file, or a file including text.

The web content 20 according to the embodiments may include a configuration for connecting 22 an image content of an image region and a region driven by HTML5 or the like in the HTML5 region. The connection 22 may be made based on time or may be made by a signal received from a user. That is, the web content 20 is configured such that the video content of the video area and the area driven by HTML5 or the like in the HTML5 area are displayed in synchronization with time (22).

For example, the web content 20 according to the embodiments may be configured such that a specific operation (or change) of the image area 21B and a specific operation (or change) of the HTML5 area 21A occurs simultaneously at the first time. For example, the web content 20 according to the embodiments may be configured such that, when an instructor shown in the image area 21B points to a specific point with her right hand, a change occurs in a portion indicating a specific point in the HTML area. The HTML area may mean an area configured based on an iframe.

For example, the web content 20 according to the embodiments corresponds to each page of the learning material corresponding to each image frame of the video content, so that no matter which scene of the video content is viewed, the learning material linked thereto. The pages may be configured to be output simultaneously.

Also, for example, on the learning material page, the user can draw lines, write notes, solve problems, or automatically score while learning, so that the user can review and repeat his or her notes, notation, incorrect answers, etc. It may be configured to

The web content 20 according to embodiments may include a component 21C for receiving a signal from a user. The component 21C for receiving a signal from the user may be, for example, a component including an icon for requesting that the user play/pause/stop web content. The component 21C for receiving a signal from the user is, for example, a component representing a progress bar for displaying current playing time information of web content and/or total playing time information of web content can be The component 21C for receiving a signal from the user may include a component indicating the total volume, double speed, and page (slide) of web content.

Web content according to embodiments may receive an input signal from a user based on a component 21C for receiving a signal from the user. The web content may be configured to drive an HTML file corresponding to each time information based on a JavaScript source based on a signal received from the user. For example, when the user reproduces the web content of the first time using the component 21C, the web content may drive HTML files corresponding to the first time.

3 illustrates web content and a method of reproducing web content according to embodiments.

The web content according to the embodiments shown in FIG. 3 means web content according to the embodiments shown in FIGS. 1 to 2 . The generating device shown in FIG. 3 may be generated by the generating device 100 described with reference to FIGS. 1 to 2 .

The web content according to the embodiments shown in FIG. 3 may be reproduced on the user's terminal device. For example, the user's terminal device downloads files (eg, image files, HTML files, and/or other JavaScript files, CSS files, etc.) constituting web content through a web browser. The user's terminal device plays (drives) the web content by using the files constituting the web content.

Web content according to embodiments may include multimedia content 30 and/or at least one HTML source 31 . The multimedia content 30 may mean image content in the image area of FIG. 2 . The multimedia content 30 may be called, for example, image content. The HTML source 31 may mean a source describing a region driven by HTML5 or the like in the HTML5 region. The HTML source 31 may be composed of HTML5-based source code. For example, the HTML source 31 may mean a source code composed of an iframe tag. The HTML source 31 may be a source code composed of HTML5, and may include CSS and/or JavaScript.

The multimedia content 30 may be expressed as images and/or sounds according to the passage of time. The multimedia content 30 may include a plurality of frames in units of time.

The multimedia content 30 may be combined with at least one HTML source 31 via time. For example, referring to FIG. 3 , the fourth time unit of the multimedia content 30 may be combined with the HTML source 31 of HTML5 01 . For example, referring to FIG. 3 , the 13th time unit of the multimedia content 30 may be combined with the HTML source 31 of HTML5 03 . For example, the multimedia content 30 may be configured to perform an action as indicated in the HTML source 31 at a particular time.

An HTML5 integrated player 32 according to embodiments will be described.

The HTML5 integrated player 32 plays web content according to embodiments. The HTML5 integrated player 32 may be included in the user's terminal device 101 of FIG. 1 . The HTML5 integrated player 32 may be a program installed in the user's terminal device or a web browser supported by the user's terminal device.

The HTML5 integrated player 32 plays the above-described web content based on the time value. Web content according to embodiments generates time value data from a playback start time of the web content. Time value data is periodically input to the integrated player. The input information is transferred back to the HTML5 container. When the time value information input in the HTML5 container in advance and the time value information generated in the multimedia are the same, the HTML5 unit page document corresponding thereto is called as the HTML source 31 .

The HTML5 integrated player 32 is one of the components that generate time value information like multimedia. For example, when any change in time value data occurs due to the user's manipulation of the player, the player transmits the changed time value information to the multimedia content and the HTML5 container. The time value transmitted to the multimedia is synchronized so that the video is played from the corresponding time point. The time value passed to the HTML5 container calls the HTML5 unit page in the same way as in sequential playback.

Here, the multimedia content 30 according to the embodiments may be a 360-degree image and/or video data performed in a VR/AR environment. For example, the multimedia content 30 according to the embodiments may be a panoramic image/video, an omnidirectional image/video, or point cloud data. The multimedia content 30 according to the embodiments may be data in the form of an ISOBMFF file.

4 shows an example of web content according to embodiments.

The web content according to the embodiments shown in FIG. 4 may be divided into, for example, a full video type, a mixed type, and/or an audio type. The web content according to the embodiments shown in FIG. 3 may mean the web content shown in FIGS. 1 to 3 .

The full image type may be a type of web content in which an HTML region composed of HTML and an image region all overlap. For example, in the full video type, the image area may be configured as the entire screen (or the entire area part of the web content), and the area driven by HTML may be configured as part or all of the screen area.

For example, the overall image type may be a form in which an image area is arranged on the entire screen of a designated content and an iframe having the same size as the moving image area is arranged above the image layout. In the full video format, the HTML5 unit page may set a transparent background so that the video screen being played in the lower part can be exposed together. The web content of the entire image type may provide additional information presented on the screen along with various graphic effects according to the content of the image.

The hybrid type may mean a case in which the HTML region and/or the image region composed of HTML is included in only a partial region of the entire screen (or the entire region of the web content). Here, the HTML region and/or the image region may overlap each other or may be arranged non-overlapping.

For example, the mixed type may be a format in which an iframe and a video are respectively arranged by dividing the screen of the content. For example, referring to the drawing shown in the center of FIG. 4 , the web content in the form of arranging an iframe on the left side of the web content and an explanation video on the right side may be a mixed type. In this type, the image area and/or the HTML area can be freely adjusted in its arrangement and size. For example content of the partial image type, an image was captured using the chroma key technique and edited with the same background as the background of the HTML5 unit page of the iframe to visually appear as one content. If it is difficult to shoot chroma key, it is okay to configure the iframe and the image area in a separate form. It is a format in which the additional information of the iframe is always exposed according to the time value of the image.

The audio type may be a type of web content in which an image file is an audio file, consists of only an HTML region, and constitutes an entire screen using the time value of the audio file. The voice-type web content provides a screen composed of images and/or text. In selecting the type of HTML5 multimedia convergence content, the composition method completely followed the external standard of screen design and the type of multimedia resource used, and did not consider the effectiveness of the content. However, all content types developed based on Flash were considered so that they could be expressed.

5 shows an example of the structure of a 360-degree multimedia device 50 according to embodiments.

The 360-degree multimedia device 50 according to embodiments may refer to the user's terminal device 101 shown in FIG. 1 . The 360-degree multimedia device 50 according to the embodiments may mean, for example, a Head Mount Display (HMD). The 360-degree multimedia device 50 according to the embodiments may include, for example, a display for implementing a VR/AR function, and receives and processes 360-degree image data for implementing VR/AR to provide a user It may mean a device capable of rendering.

The 360-degree image data may mean, for example, 360-degree image data and 360-degree video data. Specifically, 360 degree image data may mean a panoramic image/video or an omnidirectional image/video.

The 360-degree multimedia device 50 according to embodiments may include, for example, a reception processing unit 51 , a parsing unit 52 , a rendering unit 53 , and/or a display unit 54 .

The reception processing unit 51 may receive web content from a server or a web content provider. The reception processing unit 51 may receive, for example, one or more files constituting the web content, image data constituting the web content, and the like. Here, one or more files constituting the web content and image data constituting the web content may be compressed to form a single file. For example, the compressed file may be an encapsulated file based on the ISOBMFF format. The reception processing unit 51 may decapsulate the encapsulated file to extract one or more files constituting the web content, image data constituting the web content, and the like.

The parsing unit 52 parses one or more files constituting the web content and image data constituting the web content. One or more files constituting the web content include, for example, 360-degree image data, one or more HTML files, XML (eXtensible Markup Language) data representing the time information corresponding to each HTML file, and the XML It may include data including a JavaScript source for parsing the data. The parsing unit 52 includes 360-degree image data, one or more HTML files, XML (eXtensible Markup Language) data indicating the time information corresponding to each HTML file, and Java for parsing the XML data. Data including script (JavaScript) source is extracted from received files, classified and decoded. The parsing unit 52 extracts (parses) XML (eXtensible Markup Language) data representing time information by using the above-described JavaScript source.

The rendering unit 53 includes 360-degree image data parsed and/or decoded by the parsing unit 52, one or more HTML files, and XML (eXtensible Markup Language) representing the time information corresponding to each HTML file. Data and a JavaScript source for parsing the XML data is used to create (reconstruct) an area (image, etc.) on the screen that is displayed to the user.

The rendering unit 53 reconstructs the area shown to the user. Accordingly, the rendering unit 53 generates an image/image shown to the user based on the position where the user looks, that is, viewport information. Viewport information may be generated within a 360 degree multimedia device in accordance with embodiments.

The display unit 54 displays an area (image) on the screen generated and/or reconstructed by the rendering unit 53 .

6 shows 360-degree contents according to embodiments.

FIG. 6A illustrates projection of 360-degree content on a plane according to embodiments. That is, FIG. 6(A) shows 360-degree content as a rectangular image 60 . 6(B) shows the projection of 360-degree content on a sphere surface 62 according to embodiments.

The multimedia device according to the embodiments receives the 360-degree content according to the embodiments as a rectangular image 60 as shown in FIG. 6(A) . That is, the received 360-degree image data 60 according to embodiments may be an image or an image 60 in the form of a rectangular image as shown in FIG. 6(A) .

A multimedia device according to embodiments receives image data in the form of a rectangular image 60 as shown in FIG. 6(A), and among them, based on a user's gaze, that is, a viewport, the user The viewing area 61 is parsed and decoded to render (reconstruct) it.

The area 61 that the user looks at may be referred to as, for example, a viewport. The information indicating the viewport 61 may be generated by the multimedia device according to the embodiments.

The 360-degree content 60 according to embodiments may express the location of a specific point with azimuth and elevation. For example, the point pointing to 61a may be expressed as being located at an azimuth of +60 degrees and an elevation of +60 degrees. That is, the point 61a may mean a point that the user looks at when the user rotates 60 degrees clockwise from the central point in the horizontal direction, and rotates the head 60 degrees upward.

A multimedia device according to embodiments generates information indicating a viewport 61 . Information representing the viewport 61 may be expressed by azimuth and elevation.

7 illustrates an example of web content for a 360-degree multimedia device according to embodiments.

The 360-degree multimedia device according to the embodiments reconstructs (renders) the web content 70 shown in FIG. 7 . That is, the 360-degree multimedia device according to the embodiments receives data for web content. Data for web content according to embodiments includes 360-degree image data, one or more HTML files, XML (eXtensible Markup Language) data indicating the time information corresponding to each HTML file, and parsing the XML data ( It may include data including a JavaScript source for parsing).

The 360-degree multimedia device according to the embodiments may reconstruct the web content 70 by receiving data for the web content and decoding them. Regions can be reconstructed (rendered).

The web content 70 according to embodiments may include one or more widgets 71a to 71d. One or more widgets 71a to 71d may be included in one or more HTML files in data for web content. That is, data for web content according to embodiments may include one or more HTML files including data representing one or more widgets.

The widgets 71a to 71c refer to, for example, components rendered by being overlaid on 360 degree image/video data.

For example, the widget may be text data 71a. Also, for example, the widget may be image data 71b. Also, for example, the widget may be an arrow-shaped icon 71d indicating a specific location.

For example, the widget may be a button 71c and thus a component that changes to another text, image, or widget based on a user's click or user's interaction signal. For example, referring to FIG. 7 , a button 71c that is a widget may be configured to show new text based on a user's click.

The XML data may include, for example, information indicating a time at which one or more of the above-described HTML files are mapped to the above-described web content 70 . In addition, the XML data may further include information indicating a location mapped to the web content 70 based on information indicating time.

For example, the XML data may signal to map the human-shaped widget 71b included in the first HTML file to the first location at the first time. Here, the XML data may further include information indicating the first time and information indicating the first location. The information indicating the first location may be a location appearing in a VR environment. Accordingly, information indicating the first position may be expressed by an azimuth and/or an elevation value on a sphere surface.

That is, data for web content may include one or more widgets, and the one or more widgets may be mapped to the 360-degree image data by the XML data. In addition, the XML data may further include location information indicating a location in the 360 degree image data of at least one widget mapped by the XML data. Here, the location information includes first azimus information indicating a leftmost position of the at least one widget on a sphere, second azimus information indicating a rightmost position of the at least one widget, and an uppermost side of the at least one widget. It may include first elevation information indicating the position of , and second elevation information indicating the lowest position of the at least one widget.

8 illustrates a method of indicating a location of a widget in web content for a 360-degree multimedia device according to embodiments.

8 illustrates a method for widgets to signal a location to appear in a user's device apparatus (VR environment) according to embodiments; Information indicating the positions of widgets may be displayed in two ways as shown in FIG. 8 . The information indicating the location of the widgets may refer to the location information described with reference to FIG. 7 , and may be included in XML data to signal location information for each widget.

XML data according to embodiments provides a method of representing one or more widgets 80A and 80B provided by an HTML file on a sphere. The XML data includes mode information indicating how the widgets 80A, 80B are represented on a sphere, and left_azimus information indicating the leftmost position of the widgets 80A, 80B ( Azimuth_left information), right_azimuth information indicating the right-most position of the widgets (80A, 80B), and uppermost_elevation information indicating the uppermost position of the widgets (80A, 80B) (Elevation_top information), the bottom_elevation information indicating the lowest-bottom position of the widgets (80A, 80B), the widget (widget) (80A, 80B) orientation information indicating whether the orientation information (orientation information) , When the widgets (80A, 80B) are arrows, information indicating the direction the arrow points, and information indicating the degree to which the widgets (80A, 80B) are rotated on a spherical surface are further added may include

FIG. 8(A) shows a widget 80A expressed on a spherical surface according to the first mode. FIG. 8(A) shows a case in which mode information of XML data is a first mode. The first mode is configured such that the widget is positioned in an area 80A formed by two horizontally parallel circles and two vertically parallel circles. Two horizontally parallel circles have different elevation values and may mean circles on a parallel sphere, and two vertically parallel circles have different azimuth values and represent a circle on a parallel sphere. can mean

FIG. 8(B) shows a widget 80B expressed on a spherical surface according to the second mode. FIG. 8B shows a case in which mode information of XML data is the second mode. The second mode is configured such that the widget is positioned in an area 80B created by four large circles. The four large circles are composed of large circles created by the intersection of a sphere surface and a plane passing through the center point. The four large circles may include two vertically parallel azimuth large circles and/or two horizontally parallel elevation large circles.

9 illustrates a structure of web content according to embodiments.

Specifically, FIG. 9 shows a data structure of interactive web content according to embodiments. The interactive web content according to the embodiments shown in FIG. 9 may be generated by the generating device 100 of FIG. 1 . The interactive web content according to the embodiments means interactive web content according to the embodiments shown in FIGS. 2 and 4 .

The interactive web content according to the embodiments may be driven in a web browser. Accordingly, the interactive web content according to the embodiments may include front-end files and/or back-end files for driving the web content. Front-end files refer to files including data constituting the appearance of web content when web content is provided to a user. Back-end files are data for representing a configuration in which a plurality of HTML files of web content are combined with image data when web content is provided to a user.

The backend files may include, for example, an XML (eXtensible Markup Language) file for linking image data in web content and a plurality of HTML sources (HTML files).

Furthermore, the backend files may further include a Document Type Definition (DTD) file for web content to interpret and/or parse an XML (eXtensible Markup Language) file according to embodiments.

The front-end files may include, for example, an HTML5 file, a CSS file, and a JavaScript file constituting web content.

The XML (eXtensible Markup Language) file may be composed of, for example, the following syntax.

<?xml version="1.0"encoding="UTF-8"standalone="no"?>
<?xml-stylesheettype="text/xsl"href="./slide_info.xsl"?>
<!DOCTYPE data SYSTEM "./slide_info.dtd">
<data>
<!―It may be a part of the view configuration information -->
<options name="Player Option">
<type>mix</type><!-- video(full video type)| mix| sound (voice type)-->
<setting>
<contents><!--Area where video data is played-->
<width>1008</width>
<height>515</height>
</contents>
<player><!-- player position and size (px) -->
<azimuth1>0</azimuth1>
<elevation1>480</elevation1>
<azimuth2>1008</azimuth2>
<elevation2>35</elevation2>
</player>
<video><!-- video -->
<pos><!-- Position and size (px) -->
<mode>1</mode>
<azimuth1>648</azimuth1>
<elevation1>0</elevation1>
<azimuth2>360</azimuth2>
<elevation2>480</elevation2>
</pos>
<fileinfo><!-- File information -->
<path>../user/media/mp4/</path>
<filename>03_1</filename>
<extension>mp4</extension>
</fileinfo>
</video>
<slide><!-- Slide Sync -->
<pos><!-- Position and size (px) -->
<mode>2</mode>
<azimuth1>0</azimuth1>
<elevation1>0</elevation1>
<azimuth2>70</azimuth2>
<elevation2>70</elevation2>
</pos>
<fileinfo><!-- File information -->
<timeunit>sec</timeunit>
<!-- sec or ms -->
<path>../user/sink/03_s/</path>
<extension>html</extension>
</fileinfo>
</slide>
</setting>
<functions>
<sink>true</sink><!-- sink slide show ↔ -->
<volume>true</volume>
<!-- volume element show ↔ hide -->
<seek>true</seek>
<!-- progressbar seek possible show ↔ hide -->
<playbackrate>true</playbackrate>
<!―Adjust double speed show ↔ hide -->
</functions>
</options><!-- Individual Sync slides information -->
<slides name="Sync Slide List"><!-- Common path for Sync slides -->
<slide time="0"file="03_tab01"/>
<!-- Information on individual sync slides|@time: sync time, @file: file name excluding common path and extension-->
<slide time="4"file="03_tab02">
<mode>1</mode>
<azimuth1>30</azimuth1>
<elevation1>20</elevation1>
<azimuth2>50</azimuth2>
<elevation2>40</elevation2>
</slide>
<slidetime="492"file="03_tab03">
<mode>1</mode>
<azimuth1>-10</azimuth1>
<elevation1>-30</elevation1>
<azimuth2>10</azimuth2>
<elevation2>30</elevation2>
</slide>
<slidetime="623"file="03_tab04">
<mode>2</mode>
<rotation>30</rotation>
<azimuth1>-50</azimuth1>
<elevation1>-40</elevation1>
<azimuth2>10</azimuth2>
<elevation2>0</elevation2>
</slide>
<slidetime="805"file="03_tab05"/>
<arrowpos>3040</arrowpos>
<arrowvector>1-1</arrowvector>
<mode>2</mode>
<azimuth1>0</azimuth1>
<elevation1>0</elevation1>
<azimuth2>20</azimuth2>
<elevation2>20</elevation2>
</slide>
<slidetime="990"file="03_tab06"/>
... ...
<slide time="2190"file="03_tab18"/>
</slides>
</data>

As described above, the XML file may include a <type> tag for indicating whether the web content according to the embodiments is a mixed type, a full image type, or an audio type. For example, if the content in the <type> tag is video, it may indicate that it is a full video type, if it is mix, it is a mixed type, and if it is a sound, it may indicate that it is a voice type.

The <contents> tag indicates an area in which image data and a plurality of HTML sources according to embodiments are reproduced in web content. <width> in the <contents> tag indicates width information of the corresponding image data, and <height> tag indicates height information of the corresponding image data. The <player> tag indicates the location and/or size of the area of the manipulation unit of the web content according to the embodiments.

The <video> tag indicates an area in which image data according to embodiments is reproduced in web content. <pos> in the <video> tag indicates a position in the web content where the image data is reproduced. <fileinfo> in the <video> tag indicates information about an image file representing image data. The <path> tag indicates the path of the image file, <filename> indicates the name of the image file, and <extension> indicates the extension information of the image data.

The <slide> tag indicates, for example, information about HTML sources (HTML files) combined with image data according to embodiments. <pos> indicates a position in web content to which an HTML file combined with HTML image data is applied. <fileinfo> in the <video> tag indicates information about the HTML file. The <path> tag indicates the path of the HTML file, <filename> indicates the name of the HTML file, and <extension> indicates the extension information of the HTML file.

The <mode> tag is, for example, mode information indicating a method of expressing a widget represented by HTML sources (HTML files) combined with image data according to embodiments on a sphere of a VR environment. The mode information may mean the mode information as described with reference to FIG. 8 .

The <azimuth1> tag may mean Azimuth_left information indicating the leftmost position of the widgets 80A and 80B described with reference to FIG. 8 .

The <azimuth2> tag may mean Azimuth_right information indicating the right-most positions of the widgets 80A and 80B described with reference to FIG. 8 .

The <elevation1> tag may mean Elevation_top information indicating the uppermost position of the widgets 80A and 80B described with reference to FIG. 8 .

The <elevation2> tag may mean the lowest_elevation_bottom information indicating the lowest-bottom positions of the widgets 80A and 80B described with reference to FIG. 8 .

The <arrowpos> tag is information indicating the position of the arrow when the widget is an arrow.

The <arrowvector> tag may be information indicating a direction indicated by an arrow when the widgets 80A and 80B described in FIG. 8 are arrows.

The <rotation> tag is information indicating the degree to which the widgets 80A and 80B described with reference to FIG. 8 are rotated.

The XML file according to the embodiments may further include a <sink> tag for indicating whether to be displayed in the web content or to be hidden in the web content by applying a specific HTML file.

The XML file according to embodiments may further include a <volume> tag for controlling whether to adjust, turn on, or turn off the volume of specific image data.

The XML file according to embodiments may further include a <seek> tag for determining whether to show or hide a progress bar for controlling to play a specific time of web content.

The <slides> element is information for indicating a time when HTML files according to embodiments are combined with image data.

An apparatus for generating interactive web content according to embodiments may receive image data, one or more HyperText Markup Language (HTML) files, and time information corresponding to each HTML file from a user's terminal device.

An apparatus for generating interactive web content according to embodiments may generate web content. For example, the generating device may map one or more HTML files with time information corresponding to each received HTML file, and add them to sub-tags of a <slides> element of the XML file.

Due to this configuration, the web content may be configured to reproduce the image data based on a JavaScript source, and based on the JavaScript source, to drive an HTML file corresponding to each time information by extracting the current playback time information can be configured.

For example, according to the XML file described above, the web content is configured to load and execute the file 03_tab08.html when the time is 1348.

The web content according to the embodiments may further include DTD (Document Type Definition) data used to parse the above-described XML file. In the above-described XML file, for example, the <!DOCTYPE data SYSTEM "./slide_info.dtd"> tag is used and the DTD file is referred to as slide_info.dtd.

The HTML5 file (the main HTML file) and the CSS file 81a refer to files composed of an HTML source for displaying the appearance of web content. The HTML5 file may refer to a JavaScript file 81b.

The JavaScript file 81b may include a source code for extracting a current playback time of web content. That is, the web content may be configured to reproduce image data according to embodiments based on a JavaScript source, and an HTML file corresponding to each time information by extracting current playback time information of the web content based on the JavaScript source may be configured to drive The JavaScript file 81b is not configured as a separate file, and may be embedded by a <script> tag in HTML.

The JavaScript file 81b may be configured as follows.

1 $(document).ready(function(){
2 var video = $('video');
3 var thisSecond = -1;
4 video.on('timeupdate', function(){
5 video.fadeIn("slow");
6 var currentPos = parseInt(video[0].currentTime);
7 if(thisSecond!=currentPos){
8 setTab(currentPos);
9 setList(currentPos);
10 }
11 thisSecond = currentPos;
12 });
13 var tabNums = [0, 1, 489, 620, 802, 987, 1183, 1345, 1471, 1574, 1768, 1876, 1930, 1994, 2048, 2108, 2155, 2187];
14 var seq = [0, 0];
15 var seq1 = [tabNums[1]];
16 var seq2 = [tabNums[2], tabNums[2]+1, tabNums[2]+2,tabNums[2]+3,tabNums[2]+4,tabNums[2]+5];
17 var seq3 = [tabNums[3], tabNums[3]+1, tabNums[3]+2,tabNums[3]+3];
18 var seq4 = [tabNums[4], tabNums[4]+1, tabNums[4]+2,tabNums[4]+3,tabNums[4]+4,tabNums[4]+5,tabNums[4]+ 6];
...
})

Referring to the above-described JavaScript source, web content may extract current time information (Lines 4 to 12) based on the above-described JavaScript source, and HTML corresponding to each time information based on the extracted time information You can run the file. The web content may set the driving time of the HTML files (Lines 13 to 18, etc.) to drive the HTML file corresponding to each time information.

Method for receiving web content for a 360-degree multimedia device according to embodiments and using the device HTML5 may generate web content that can interact with a user in a general-purpose user terminal device such as a mobile device or a PC.

A method for receiving web content for a 360-degree multimedia device according to embodiments and combining HTML5 with image data of the device may provide web content that can interact with a user.

A method and apparatus for receiving web content for a 360-degree multimedia device according to embodiments, by combining HTML5 with image data, allows the user's terminal device to use only a web browser without installing a separate program, using only a web browser with a small amount of memory. You can consume content.

10 is a flowchart illustrating a method for receiving web content for a 360-degree multimedia device according to embodiments.

A method for receiving web content for a 360-degree multimedia device according to embodiments includes receiving data for web content (S1000), parsing data for web content (S1001), and viewing the web content in a user's viewport ) based on the location of the rendering step (S1002), may further include the step of displaying the web content (S1003).

Step S1000 is a step in which the 360-degree multimedia device according to the embodiments receives data for web content. Here, the data for the web content includes 360 degree image data, one or more HTML files, XML (eXtensible Markup Language) data indicating the time information corresponding to each HTML file, and parsing the XML data. It may include data including a JavaScript source for

Web content according to embodiments is configured to reproduce the 360-degree image data based on the JavaScript source. The web content may be configured to drive an HTML file corresponding to each time information by extracting current time information based on the JavaScript source, and the web content is a DTD (Document Type Definition) used to parse the XML data. It may contain more data.

One or more HTML files according to embodiments may include one or more widgets, and one or more widgets may be mapped to the 360-degree image data by the XML data. . XML data according to embodiments may further include location information indicating a location in the 360-degree image data of at least one widget mapped by the XML data.

The location information according to the embodiments includes first azimus information indicating the leftmost position of the at least one widget on a sphere, second azimus information indicating the rightmost position of the at least one widget, and the at least one The first elevation information indicating the uppermost position of the widget, the second elevation information indicating the lowermost position of the at least one widget, and each of the HTML files, the rotation indicating the degree of rotation of the at least one widget may contain information.

The methods according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software.

Examples of computer-readable media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as at least one software module to perform the operations of the present invention, and vice versa.

In addition, the above-described method or apparatus may be implemented by combining all or part of its configuration or function, or may be implemented separately.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

Claims (5)

a receiver for receiving image data, one or more HTML (HyperText Markup Language) files, and time information corresponding to each HTML file;
a web content generator for generating web content;
a transmission unit for transmitting the generated web content to a user's terminal device; and
a display unit; including,
The web content is:
1) XML (eXtensible Markup Language) data representing the one or more HTML files and the time information corresponding to each HTML file, and data including a JavaScript source for parsing the XML data including,
2) at least one of an HTML region driven by HTML or an image region driven by the image data,
3) based on the JavaScript source, the video content of the video area and the HTML area are configured to be displayed in synchronization with time;
4) including a component for receiving a signal from the user,
5) is configured to drive an HTML file corresponding to the time information based on the signal and the JavaScript source input from the user, and when the signal input from the user corresponds to the web content of the first time The web content drives the HTML files corresponding to the first time,
The web content includes front-end files and back-end files for driving the web content,
The backend files further include DTD (Document Type Definition) data used to parse the XML data,
The XML data is:
Whether the web content is i) a hybrid type including both the HTML region and the image region, ii) the image region is a full image type corresponding to the entire region of the web content, or iii) includes only the HTML region <type> tag indicating whether it is negative or not; and
a <contents> tag indicating a region in which the image data and a plurality of HTML sources are reproduced in the web content; containing,
A device that creates interactive web content. delete delete delete delete

Images