KR102280539B1 - System for providing panorama virtual reality based portal service - Google Patents

Abstract

Provided is a system for providing a panorama virtual reality-based portal service, which includes: at least one production terminal for uploading at least one type of immersive content including 360-degree panoramic content in real time; a user terminal that searches for and plays at least one type of sensory content and receives real-time streaming when the live is performed on at least one production terminal; and a portal service providing server including a storage unit for registering at least one production terminal so as to be mapped to a channel name for identifying each channel when the at least one production terminal opens a channel, an upload unit configured to register and display in a channel when at least one type of sensory content is uploaded from at least one production terminal, a transmitter for listing at least one sensory content to which a tag corresponding to the search word is attached when inputting a search term from the user terminal and transmitting the list to the user terminal, and a playback control unit to play in the user terminal when any one sensory content is selected in the user terminal.

Description

SYSTEM FOR PROVIDING PANORAMA VIRTUAL REALITY BASED PORTAL SERVICE

The present invention relates to a panoramic VR-based portal service providing system, and provides a system that can search, play, and share immersive content captured in 360 panoramas for each country, city, or attraction on an integrated platform.

The technical characteristics of 5G such as ultra-high speed, ultra-low latency, and hyper-connection are expected to bring about major changes in content format, production environment, and distribution method. In the 5G era, since a large amount of data can be transmitted without delay, various contents can become common and real-time interaction with contents can be increased. Among them, immersive content is by far the most anticipated. Realistic content refers to next-generation content that maximizes human five senses and provides a realistic experience. Extended reality (XR) such as virtual reality (VR), augmented reality (AR), and mixed reality (MR) are representative examples of immersive content. Among extended reality contents, VR contents are attracting attention as a new contents service specialized for 5G, especially as HMD is popularized. If you use the 5G network, you can use the ultra-high-resolution, high-definition screen as a large display while on the go, and feel the scene as realistically as 3D content.

At this time, the configuration of a portal site that can match travel guides and search VR contents using VR contents was researched and developed. In this regard, Korean Patent Application Laid-Open No. 2020-0060035 (May 29, 2020), which is a prior art, has been researched and developed. public) and Korean Patent No. 10-2080614 (announced on April 23, 2020), receive an image captured by a guide from a guide terminal, edit and register it as VR image content, receive input from a user terminal, and travel destination Searches for VR image contents registered in the image editing register based on the location of , and provides VR image contents of a suitable travel destination to the user terminal, and provides a guide for registering VR image contents selected from among the VR image contents provided to the user terminal with the user. Creating, editing, and activating VR content including a matching configuration and at least one VR video is shared on the HTML site, and the VR content searched for by the recommended keyword is prioritized according to the number of clicks searched by netizens. Each configuration is disclosed in which anyone can easily create-edit-activate VR contents through a GUI (Graphic User Interface) menu by sequentially arranging and displaying them on an HTML site.

However, since VR contents are produced and distributed only in a small number of specific contents for each institution or company, they are not contents that can be searched on the integrated portal site. Even if you purchase an HMD that can be viewed in VR, the number of contents that can be viewed through the HMD is insufficient because the contents are interspersed with each other, and there are also negative prospects that 3D contents will follow the path. For this reason, VR video, which is relatively easy to produce and disseminate, is attracting attention as a way to make up for the lack of content in the initial VR market stage, and the market is rapidly forming centering on mobile. However, VR video is an interactive element that is important in VR content. does not exist, so it is merely observing the virtual world. As a result, there is a limit to the activation of this VR content ecosystem at the level of simply viewing VR images. Therefore, research and development of a platform that can interact with a portal site that can distribute VR content in an integrated way is required.

An embodiment of the present invention is designed to build a portal site that can distribute and search only VR contents by integrating the distribution network of VR contents, and enables anyone to register, search, and interact with each other in real time, focusing on panoramic VR. It allows you to virtually visit, experience, and acquire information about travel places you want to visit at any time. By analyzing and evaluating registered Panorama VR, reliability can be increased by selecting excellent content, and Panoramic VR already produced and distributed is shared on the integrated platform using related URLs, and various evaluation methods are used as incentives for providers. By allowing revenue to return, it is possible to provide a method of providing a panoramic VR-based portal service that can induce the production of more content. However, the technical task to be achieved by the present embodiment is not limited to the technical task as described above, and other technical tasks may exist.

As a technical means for achieving the above technical problem, an embodiment of the present invention provides at least one production terminal for uploading at least one type of immersive content including 360-degree panoramic content in real time, at least Search and play one type of sensory content, and when a live is performed on at least one production terminal, a user terminal receiving real-time streaming and a channel name that identifies each channel when opening a channel on at least one production terminal A storage unit for registering at least one production terminal, an uploading unit for registering and displaying on a channel when at least one type of sensory content is uploaded from the at least one production terminal, and a corresponding search word when inputting a search word from the user terminal A portal service providing server comprising: a transmission unit that lists up at least one sensory content with tags and then transmits it to the user terminal; and a reproduction control unit that plays the sensory content in the user terminal when any one sensory content is selected from the user terminal. include

According to any one of the above-described problem solving means of the present invention, by integrating the distribution network of VR contents, a portal site that can distribute and search only VR contents is built, and anyone can register, search, and interact with contents in real time, centering on panoramic VR. It is designed to enable dialogue between people, and allows users to virtually visit, experience, and acquire information about travel places they want to visit at any time. The reliability can be increased by selecting excellent content by analyzing and evaluating the registered panoramic VR by a dedicated expert, and the previously produced and distributed panoramic VR can be shared on the integrated platform using related URLs, and various evaluation methods It can induce the production of more content by allowing the revenue to return to the provider as an incentive by the company, providing a travel alternative to those who cannot travel in the era of Corona (COVID19).

1 is a diagram for explaining a system for providing a panoramic VR-based portal service according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a portal service providing server included in the system of FIG. 1 .
3 and 4 are diagrams for explaining an embodiment in which a panoramic VR-based portal service according to an embodiment of the present invention is implemented.
5 is an operation flowchart illustrating a method for providing a panoramic VR-based portal service according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

The terms "about", "substantially", etc. to the extent used throughout the specification are used in a sense at or close to the numerical value when the manufacturing and material tolerances inherent in the stated meaning are presented, and serve to enhance the understanding of the present invention. To help, precise or absolute figures are used to prevent unfair use by unscrupulous infringers of the stated disclosure. As used throughout the specification of the present invention, the term “step for (to)” or “step for” does not mean “step for”.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, '~' refers to components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

In this specification, some of the operations or functions described as being performed by the terminal, apparatus, or device may be performed instead of in a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the terminal's unique number or personal identification information, which is the identification data of the terminal. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a system for providing a panoramic VR-based portal service according to an embodiment of the present invention. Referring to FIG. 1 , the panoramic VR-based portal service providing system 1 includes at least one user terminal 100 , a portal service providing server 300 , at least one production terminal 400 , and at least one expert terminal ( 500 ), a travel agency terminal 600 , and an agency server 700 . However, since the panoramic VR-based portal service providing system 1 of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

In this case, each component of FIG. 1 is generally connected through a network 200 . For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to the portal service providing server 300 through the network 200 . And, the portal service providing server 300, through the network 200, at least one user terminal 100, at least one production terminal 400, at least one expert terminal 500, and a travel agency terminal 600 ) can be associated with Also, at least one production terminal 400 may be connected to the portal service providing server 300 through the network 200 . In addition, the at least one expert terminal 500 may be connected to the at least one user terminal 100 , the portal service providing server 300 , and the at least one production terminal 400 through the network 200 . Finally, the travel agency terminal 600 may be connected to the user terminal 100 , the production terminal 400 , the portal service providing server 300 , and at least one expert terminal 500 through the network 200 .

Here, the network refers to a connection structure in which information exchange is possible between each node, such as a plurality of terminals and servers, and an example of such a network includes a local area network (LAN), a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi (Wi-Fi) , Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( Near-Field Communication) networks, satellite broadcast networks, analog broadcast networks, Digital Multimedia Broadcasting (DMB) networks, and the like are included, but are not limited thereto.

In the following, the term at least one is defined as a term including the singular and the plural, and even if at least one term does not exist, each element may exist in the singular or plural, and may mean the singular or plural. it will be self-evident In addition, that each component is provided in singular or plural may be changed according to embodiments.

The at least one user terminal 100 may be a terminal of a user who searches for and plays sensory content using a web page, an app page, a program, or an application related to the panorama VR-based portal service.

Here, the at least one user terminal 100 may be implemented as a computer that can access a remote server or terminal through a network. Here, the computer may include, for example, a navigation device, a laptop computer equipped with a web browser, a desktop computer, and a laptop computer. In this case, the at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smart phone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

The portal service providing server 300 may be a server that provides a panoramic VR-based portal service web page, an app page, a program, or an application. In addition, the portal service providing server 300 provides an integrated platform capable of searching, distributing, and distributing sensory content, and includes a provider such as at least one production terminal 400 and at least one user terminal 100 . It may be a server that serves to connect consumers. In addition, the portal service providing server 300 hires at least one expert terminal 500 to rank or select each sensory content, and at least one travel agency terminal 600 protects copyrights such as distribution rights and rental rights. In the case of asking for permission, it may be a server that mediates with the production terminal 400 that is the copyright holder. In addition, the portal service providing server 300 requests a URL, etc. from at least one institution server 700 to collect the sensory content scattered on each site on the platform according to an embodiment of the present invention, and the platform of the present invention. It may be a server that converts and uploads it according to the format according to the standard, or pastes the URL into HTML to play. Also, the portal service providing server 300 may be a server that provides comments or real-time chatting to enable real-time interaction.

Here, the portal service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a navigation device, a laptop computer equipped with a web browser, a desktop computer, and a laptop computer.

The at least one production terminal 400 may be a terminal of a manufacturer using a panorama VR-based portal service related web page, an app page, a program, or an application. In this case, the producer may be a travel expert (not an expert of the expert terminal 500) who goes on a travel-related field trip, but is not limited to travel. The production terminal 400 may be a terminal that uploads the produced sensory content to the portal service providing server 300 and receives a preset reward or incentive accordingly. To this end, the production terminal 400 may be a terminal that establishes a channel and uploads at least one sensory content in the established channel.

Here, the at least one production terminal 400 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, a navigation device, a laptop computer equipped with a web browser, a desktop computer, and a laptop computer. In this case, the at least one manufacturing terminal 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one manufacturing terminal 400, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smart phone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

The at least one expert terminal 500 may be an expert's terminal that evaluates the quality of the sensory content of the production terminal 400 using a web page, an app page, a program, or an application related to the panoramic VR-based portal service. In this case, the quality may be the quality of the content itself, or it may be the quality of the resolution, shooting technique, or editorial technique, and preferably, it may mean seeing how faithfully the content is written to the subject. The expert terminal 500 may upload quality evaluation data for each sensory content to the portal service providing server 300 , and the portal service providing server 300 may not publish or share content of less than a preset quality. Alternatively, a method of posting all content by classifying only a rating may be used, which may vary according to embodiments. The expert terminal 500 may be a terminal of an expert who is an employee hired by the company of the portal service providing server 300 , or a terminal of a person registered as an expert although it is an external person.

Here, the at least one expert terminal 500 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a navigation device, a laptop computer equipped with a web browser, a desktop computer, and a laptop computer. In this case, the at least one expert terminal 500 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one expert terminal 500, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smart phone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

The travel agency terminal 600 is a travel agency, travel agency, or travel agency that wants to distribute or publish travel content from the portal service providing server 300 using a panorama VR-based portal service related web page, app page, program or application. It may be a terminal of a related institution. Since it is more efficient to obtain permission for use of already produced content, that is, a licensing contract, than to directly produce travel content, the travel agency terminal 600 can use it. A contrasting concept is the institution server 700 , which serves as a provider of providing sensory content produced by itself to the portal service providing server 300 .

Here, the travel agency terminal 600 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a navigation device, a laptop computer equipped with a web browser, a desktop computer, and a laptop computer. In this case, the travel agency terminal 600 may be implemented as a terminal capable of accessing a remote server or terminal through a network. The travel agency terminal 600 is, for example, a wireless communication device that guarantees portability and mobility, such as navigation, Personal Communication System (PCS), Global System for Mobile communications (GSM), Personal Digital Cellular (PDC), and PHS (Personal Digital Cellular) (PHS). Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal , and may include all types of handheld-based wireless communication devices such as smartphones, smartpads, and tablet PCs.

2 is a block diagram for explaining a portal service providing server included in the system of FIG. 1, and FIGS. 3 and 4 illustrate an embodiment in which a panoramic VR-based portal service according to an embodiment of the present invention is implemented. It is a drawing for

Referring to FIG. 2 , the portal service providing server 300 includes a storage unit 310 , an upload unit 3200 , a transmission unit 330 , a reproduction control unit 340 , a classification unit 350 , a compensation unit 360 , It may include a collecting unit 370 , an awarding unit 380 , and an intermediate unit 390 .

At least one user terminal 100, at least one production terminal 400, and at least one expert When transmitting a panoramic VR-based portal service application, program, app page, web page, etc. to the terminal 500, at least one user terminal 100, at least one production terminal 400, and at least one expert terminal ( 500) may install or open a panorama VR-based portal service application, program, app page, web page, and the like. In addition, the service program may be driven in at least one user terminal 100 , at least one production terminal 400 , and at least one expert terminal 500 using a script executed in a web browser. Here, the web browser is a program that enables the use of a web (WWW: World Wide Web) service, and refers to a program that receives and displays hypertext written in HTML (Hyper Text Mark-up Language), for example, Netscape. , Explorer, Chrome, and the like. In addition, the application means an application on the terminal, for example, includes an app (App) executed in a mobile terminal (smartphone).

Referring to FIG. 2 , the storage unit 310 may register at least one production terminal 400 to be mapped to a channel name identifying each channel when a channel is opened in the at least one production terminal 400 .

The upload unit 320 may register and display in a channel when at least one type of sensory content is uploaded by the at least one production terminal 400 . The at least one production terminal 400 may upload at least one type of immersive content including 360-degree panoramic content in real time. For the production of 360-degree panoramic contents among immersive contents, first, the equipment for information acquisition can be selected according to the indoor and outdoor types. A camera can also be physically coupled to a multicopter to acquire aerial images. Afterwards, it is possible to provide a tool that can finally create content by editing the acquired image according to the HMD output coordinate system.

After designating a path for the user to move in advance to create indoor content, a point is set in the middle of the moving path. It can be composed of a panoramic image taken with a 360-degree camera of an image taken between designated points, and an internal image using an HMD. While viewing the video from a 360-degree free viewpoint through the HMD device, the user feels that he or she is actually moving. At this time, in order to give the user a vivid sense of presence as if they are in the real scene, different images are taken at each point in time, and the first screen of the reproduced video matches the last screen of the previous video.

In addition, it is possible to acquire an aerial image of the exterior taken by combining a multicopter and a 360-degree camera to create outdoor content. For example, a 360-degree camera Richo Theta S and a 3DR SOLO multicopter can be used for image acquisition, but this will depend on the equipment each manufacturer has. In addition to linking the captured video with the HMD, it can also provide a tool to edit the video for use in virtual reality. The video editing tools may be, for example, but not limited to, Unity 3D and Premier CC 2015.

In Unity3D, an image is inserted into a sphere to express a 360-degree image without distortion, and when the user walks into the sphere, it can be set to play the video entering the entrance of the tourist destination. For example, arrow buttons corresponding to front, back, left and right may be inserted in each direction of a road that the user can select at a fork in the video playback based on the main viewpoint of the user wearing the HMD. When a user gazes at an arrow button in a desired direction for more than 2 seconds or clicks with a mouse, an image as if it is actually moving in that direction may be displayed. After moving in the selected direction, when arriving at a designated point, a tool that can be implemented so that an arrow button to select the moving direction appears again may be provided. However, in order to secure the disadvantage that it is difficult to accurately recognize what the user is gazing at even when looking at the arrow when looking from a distance, an aiming point may be inserted to indicate what the user is gazing at. The aiming point moves along with the movement of the user's gaze while wearing the HMD, and a script in which the size of the aiming point is adjusted according to the distance from the gazing object may be added.

Through aerial images taken outdoors at high altitude, users can not only appreciate parts that are difficult to experience in real travel, but also view more diverse and holistic landscapes, enabling realistic virtual travel. In addition, by adding a script that adjusts the video playback speed through input from the keyboard, it is possible to set the non-interested places to pass quickly. Conversely, by adding a stop function, an interface can be built so that the image can be stopped at a scene of interest to further appreciate the surrounding scenery.

In this case, the at least one expert terminal 500 may analyze and evaluate at least one type of sensory content uploaded from the at least one production terminal 400 to select excellent content having a preset quality or higher. All content may be disclosed in a manner that assigns a rating or level to each sensory content, or content that does not satisfy a preset rating or level may be selected in a manner not to be published.

Here, each sensory content is tagged. The tag may be attached in the production terminal 400, but may be attached in a manner of selecting a tag categorized in the order of country name-region name-attraction name for a search system. With the tag attached in this way, when the user terminal 100 selects a tag when searching, it is possible to search for a desired travel destination or attraction simply by selecting the tag without entering a search word.

Currently, the Korean Cultural Information Service operates and manages about 70 million pieces of traditional culture contents, but there is a lack of content linkage according to the unique number system integrated with other contents and an accurate unique number system according to the classification system of traditional culture contents. As the amount of web documents increases, the importance of retrieval, management, and filtering to find information suitable for users is being emphasized. Search engines from websites such as Naver, Google, and Daum that include functions of search, management, and filtering to provide the information each user is looking for. However, due to massive data and indiscriminate collection of documents, search engines are having difficulties in providing useful information suitable for users.

Accordingly, in an embodiment of the present invention, it can be solved by applying document classification to provide an integrated platform. Document classification may consist of text, music, images, video, etc., but unless otherwise specified, it is called text classification. In the past, document classification was done manually, but now, large-scale documents are classified efficiently through machine learning techniques such as Naive Bayes, Support Vector Machine, K-NN, and Neural Net. Accordingly, in an embodiment of the present invention, a model for automatically classifying content may be used.

First, the TF-IDF model and the Bag of Words model are used to convert subtitle data or voice data in travel content into Speech To Text (STT) and extract the frequency of words within the text set to be learned through the preprocessed data. Apply to extract words. It is possible to use a model that automatically classifies texts according to each category by using the Support Vector Machine algorithm on the extracted words and categories. In detail, the TF-IDF model and the Bag of Words model are used and compared to extract the word frequency from the preprocessed data. The TF-IDF model is a numerical expression of which word is important in a specific document when there are multiple documents. TF is a value of how often a specific word appears in a document as a word frequency, and IDF is a value of how commonly it appears in the entire document.

In this way, TF and IDF are combined to extract key words from the document. Bag of Words constitutes the number of occurrences of each word in a sentence. Bag of Words shows only the number of important words without the order of the sentences. By applying these two models, word extraction is performed. The text is classified using the Support Vector Machine, one of the machine learning techniques, through the extracted words and predefined categories. The Support Vector Machine is a classification algorithm that determines which category when new data is input based on the collected data. For objective evaluation of this model, 90% of the dataset is used as training data and the remaining 10% is used as test data to verify the content classification model.

When a search word is input in the user terminal 100 , the transmitter 330 may list up at least one sensory content to which a tag corresponding to the search word is attached and then transmit it to the user terminal 100 . For example, when an Italian-Rome tag is searched, if there are 10 sensory contents to which the tag is attached, the 10 pieces may be listed in the order of inquiry, popularity, etc. and displayed on the user terminal 100 .

The reproduction control unit 340 causes the user terminal 100 to play when any one sensory content is selected in the user terminal 100 . The user terminal 100 may receive real-time streaming when searching for and playing at least one type of sensory content and performing a live on at least one production terminal 400 . In the case of augmented reality content or panoramic 360-degree content, the user terminal 100 may execute itself, but in the case of virtual reality content, interworking between the user terminal 100 and the HMD may be required.

The at least one type of sensory content may be travel-themed content. In this case, when receiving at least one type of sensory content from the at least one production terminal 400, the classification unit 350 may provide a category to sequentially select a country, a region, and an attraction as described above. . Even travel-themed content can be sub-categorized into news, entertainment, documentary, drama, concert, and sports. If the content of the content is classified through the subtitles through the above-described TF-IDF, the following classification is to classify the genre.

In this case, each detailed classification can be performed through the image classification technology, and the image classification model that received the highest score in GoogLeNet, that is, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) may be used. This is called Inception v1, and its biggest feature is that it deepens the depth of the CNN network, but does not significantly increase the amount of computation. Alternatively, by introducing Residual Connection, a technology that goes beyond some layers, such as ResNet, a technology that enables high-accuracy image classification while reducing the amount of computation can be used. Like Inception-ResNet, high image quality is achieved by fusion of Inception module and Residual Connection. Techniques that show classification accuracy may also be used.

The compensator 360 identifies the category distribution of the sensory content uploaded by the at least one production terminal 400, extracts categories in which no sensory content is registered or is registered with a preset number or less, and adds the extracted categories to the category. When registering realistic content, a higher reward than a preset reward level may be paid. In this case, in an embodiment of the present invention, since categories exist for each region or each attraction, the creator who registers the content can take a higher reward than the set reward in a category with sparse content because people do not visit it the most. For example, if there is 100 content in the Rome category, which people have visited the most, and there is 1 content in the Monaco category in Africa, then the creator who adds content to the Monaco category will be rewarded 100 times more than the person who adds content to the Rome category. may obtain

The collection unit 370 collects URL addresses of travel-related sensory content already produced and provided from at least one institution server 700 , converts the travel-related sensory content to be streamed in the format of the portal service providing server, and uploads it can do. For example, suppose that the Ministry of Culture, Sports and Tourism posted panoramic 360-degree content introducing Jeju Island on the website of the Ministry of Culture, Sports and Tourism, and that Jeju Special Self-Governing Province also posted panoramic 360-degree content introducing Jeju-do. The two contents are different contents produced by different organizations or the same theme, that is, Jeju Island. People who want to go on a virtual tour of Jeju Island do not know where the Jeju virtual travel contents are. In general, they have to search for Jeju Island on a portal site and then find the contents that are played back as virtual reality contents within the search results. In order to eliminate such inconvenience and provide integrated information, in an embodiment of the present invention, scattered contents may be collected through the collection unit 370 .

In this case, the collection unit 370 may use a web crawler, which is a computer program that searches the World Wide Web in an organized and automated manner. The work of a web crawler is called web crawling or spidering, and it is a form of bot or software agent. There are two main types of web crawlers: general web crawlers and distributed web crawlers. When explaining the basic operation of a web crawler, it starts by fetching a URL from the URL Frontier module and fetching a web page of the URL using the HTTP protocol. Then, the Fetch module stores the web page in temporary storage, the Parser module extracts the text and links, and the text is sent to the Indexer. In the case of a link, whether it should be added to the URL frontier is determined by going through the Content Seen, URL Filter, and Duplication URL Element modules. At this time, since it is virtually impossible to crawl all web documents with a general web crawler, a distributed web crawler can be further used.

Distributed web crawler is largely divided into two types, one of which is centralized and the other is P2P (or Fully-Distributed). A centralized distributed web crawler is a structure in which the URL manager acts like a server and the crawler acts as a client. After downloading the document from the crawler, extracting the OutLink URL and passing it to the URL manager, the URL manager checks whether the URL of the downloaded document is the URL and removes the URL duplication. In other words, the URL manager takes over the URL duplication and URL management in the general web crawler. On the other hand, in the P2P method, each crawler has a completely independent structure. In the P2P method, each crawler operates like a general web crawler. Each crawler downloads documents, extracts OutLink URLs, and deduplicates URLs. Each crawler operates independently. To do this, the list of downloaded URLs managed by each crawler must be mutually exclusive. Otherwise, the phenomenon of downloading the same document from different crawlers will occur. As a way to solve this, each crawler can manage the download URL domains by dividing them exclusively. In other words, it is a method of managing only what belongs to the download domain and passing the rest of the URLs to other crawlers. If this method is used, each crawler can operate independently.

Next, web content needs to be extracted. Web content extraction technology provides a function to automatically extract facial data, body data, and sound data, which are content to be used for information analysis, from a web document. A web content extraction system is a device that extracts only content by automatically producing rules for extracting content. A Rule Generator that automatically generates content extraction rules, and a Navigation Content Eliminator that removes navigation content from a given web document. , it can be composed of a content extractor (Core Context Extractor) that extracts content through comparison of keyword similarity with content extraction rules. Of course, it is possible to collect real-time data in various ways other than the above-described crawling method, and is not limited to the listed ones, and is not excluded for reasons not listed.

The reward unit 380 may provide an incentive to the at least one production terminal 400 based on at least one evaluation criterion including the number of views of at least one type of sensory content.

When receiving a request for distribution of at least one type of sensory content from the at least one travel agency terminal 600 , the intermediary unit 390 establishes a copyright contract for the requested sensory content with the production terminal 400 at least one Intermediary between the travel agency terminals 600 may be performed.

In addition, an embodiment of the present invention may provide interactive VR content. First, the interaction data is created at the same time when the VR image is produced. When a VR image is first produced as an image through 3D rendering, only 3D spatial information, object location information, and real-time interaction event information are extracted from the implemented 3D data and converted into data. When the VR image is played back on the user player, this interaction data is synchronized and reproduced in real time with information necessary for interaction with the VR image, except for 3D rendering. Second, the VR image includes 3D environment information (object location, user location) that changes with the passage of time. This information should be equal to the time the VR video is played. Third, interaction event information is synchronized with the VR video playback time in the user player and processed in real time.

A method of using such VR image data and interaction data (3D spatial information, object information, interaction script) separately can secure various advantages. In particular, it has compatibility with the existing method as a method of playing only VR images on lower platforms where it is difficult to apply interaction. In addition, by leaving the VR image as it is and generating several different interaction data, it is possible to easily create modified content, and only the interaction data can be modified and updated to reflect it in the service in real time. In addition, it avoids 3D real-time rendering, but it can be produced and utilized in high quality in advance through VR video, freeing from the limitations of platforms with various performance.

The core of interactive VR video content is the development of the player. The player downloads and plays the VR image content desired by the user from the server, and internally 3D physics calculation for event processing is performed in real time. This is synchronized with the video being played, and allows objects or events played in the video to actually interact. Content is developed in two forms in one project. The first is a project for producing VR images, and the second is a project for data extraction for interaction played in real time on the player. In this way, it is possible to realize the synchronization of the interaction data and the VR image.

Construct a planned three-dimensional space and place interactive objects. Implement things that happen over time and let the content proceed automatically, from the beginning to the end, without any interaction. Because the result is produced as a video, users can escape from the restrictions on platform performance. In particular, when rendering and displaying real-time 3D content on a mobile device, since the hardware performance of the terminal is not sufficient, problems of quality degradation including resolution and frame rate are inevitable. Content quality degradation is a major disadvantage in virtual reality environments where realistic expression and immersion are important. In contrast, when the VR image approached in the present invention is synthesized and the game engine image is produced, high-quality, high-resolution 3D image production that is difficult to reproduce on mobile is relatively easy.

When constructing data corresponding to interaction, 3D spatial information, information on the location and collision range of an object, information on a movement path of a camera moving in virtual space, and the like may be included. When working by imaging 3D model data, it may be relatively easy to secure 3D information for interaction. In order to secure 3D information from the actual image in the future, a separate step of converting into such a 3D space may be added and 3D information may be secured. Data used for video production is converted to interaction information using Unity's Asset Bundle technique. In the player, data can be loaded at runtime and synchronized with the video to be played. In addition to 3D Unity, you can use various programs.

Hereinafter, an operation process according to the configuration of the portal service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as an example. However, it will be apparent that the embodiment is only one of various embodiments of the present invention and is not limited thereto.

Referring to FIG. 3 , (a) the portal service providing server 300 opens a channel when creating an account in at least one production terminal 400, and uploads each content to the channel of each producer when sensory content is uploaded. do. Then, a level is created for each country, region, and attraction, and a tag is attached or received from the production terminal 400 and attached to classify.

(b) the portal service providing server 300, when a tag is selected in the user terminal 100 or a search word corresponding to the tag is input, lists up the searched content and transmits it to the user terminal 100, (c) When any one content is selected, the content is output using the user terminal 100 or an HMD interworking with the user terminal 100 . In addition, (d) the portal service providing server 300 receives a content rating from at least one expert terminal 500 and selectively publishes it on the site, or discloses all but displays each rating so that the user can search by rating. It can also be configured to do this.

At this time, the portal service providing server 300 provides a platform that can communicate with each other by checking this in the production terminal 400 when a comment is made or a real-time chat is made in the user terminal 100, (b) at least one If the travel agency terminal 600 desires to use the content on the platform of the present invention, it may serve as a mediator of a contract with a producer who is a copyright holder. The contract based on the license is made by agreement between the two parties, so it is not determined by either format or price. However, the format of the contract may be standardized and provided by the portal service providing server 300 , and the portal service providing server 300 may serve as a third trusted organization.

(c) The portal service providing server 300 may collect the sensory content scattered in each institution server 700 with a web crawler and convert it to be output on the platform of the present invention, or bring the URL as HTML as it is and install a separate player You can even have it stream automatically without you.

The matters not described with respect to the method for providing the panoramic VR-based portal service in FIGS. 2 to 4 are the same as those described for the method for providing the panoramic VR-based portal service through FIG. 1 or can be easily inferred from the described content. Therefore, the description below will be omitted.

5 is a diagram illustrating a process in which data is transmitted/received between components included in the panoramic VR-based portal service providing system of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process in which data is transmitted and received between each component will be described with reference to FIG. 5, but the present application is not limited to such an embodiment, and the example shown in FIG. 5 according to the various embodiments described above will be described. It is apparent to those skilled in the art that the data transmission/reception process may be changed.

Referring to FIG. 5 , the portal service providing server registers at least one production terminal to be mapped to a channel name that identifies each channel when at least one production terminal opens a channel ( S5100 ).

In addition, the portal service providing server registers and displays on a channel when at least one type of sensory content is uploaded from at least one production terminal (S5200), and when a search word is input from a user terminal, a tag corresponding to the search word is added After listing at least one attached sensory content, it is transmitted to the user terminal (S5300).

In addition, the portal service providing server allows the user terminal to play when any one sensory content is selected in the user terminal (S5400).

The order between the above-described steps (S5100 to S5400) is only an example and is not limited thereto. That is, the order between the above-described steps S5100 to S5400 may be mutually changed, and some of these steps may be simultaneously executed or deleted.

Matters not described with respect to the method for providing a panoramic VR-based portal service in FIG. 5 are the same as those described for the method for providing a panoramic VR-based portal service through FIGS. 1 to 4 or can be easily inferred from the described content Therefore, the description below will be omitted.

The panoramic VR-based portal service providing method according to the embodiment described with reference to FIG. 5 may also be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. . Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described panoramic VR-based portal service providing method according to an embodiment of the present invention may be executed by an application basically installed in a terminal (which may include a program included in a platform or operating system, etc. basically installed in the terminal). Also, it may be executed by an application (ie, a program) installed directly in the master terminal by a user through an application providing server such as an application store server, an application, or a web server related to the corresponding service. In this sense, the above-described panoramic VR-based portal service providing method according to an embodiment of the present invention is implemented as an application (ie, program) installed by default in a terminal or directly installed by a user, and can be read by a computer such as a terminal. It can be recorded on a recording medium.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (7)

at least one production terminal 400 for uploading at least one type of immersive content including 360-degree panoramic content in real time;
a user terminal 100 that searches for and plays the at least one type of sensory content and receives real-time streaming when the live is performed on the at least one production terminal 400;
When opening a channel in the at least one production terminal 400, the storage unit 310 for registering the at least one production terminal 400 to be mapped to a channel name identifying each channel, the at least one production terminal 400 ), the upload unit 320 for registering and displaying on the channel when at least one type of sensory content is uploaded, and at least one tag corresponding to the search word when inputting a search word from the user terminal 100 The transmission unit 330 lists up the sensory content of the user terminal 100 and transmits it to the user terminal 100, and when the user terminal 100 selects any one sensory content, the user terminal 100 plays the content Portal service providing server 300 including a control unit 340; and
At least one expert terminal 500 for analyzing and evaluating at least one type of sensory content uploaded from the at least one production terminal 400 to select excellent content with a preset quality or higher;
The portal service providing server 300,
a classification unit 350 that provides a category to sequentially select a country, a region, and an attraction when the at least one type of sensory content is registered from the at least one production terminal 400;
identifying the category distribution of the sensory content uploaded by the at least one production terminal 400, extracting categories in which no sensory content is registered or less than a preset number, and registering the sensory content in the extracted category a compensation unit 360 configured to provide a higher reward than the preset reward level in a category with sparse content because people do not visit it the most by paying a higher reward than the preset reward level;
A collection unit 370 that collects URL addresses of travel-related sensory content pre-produced and provided from at least one institution server, converts the travel-related sensory content to be streamed in the format of the portal service providing server 300, and uploads it );
a reward unit 380 providing an incentive to the at least one production terminal 400 based on at least one evaluation criterion including the number of views of the at least one type of sensory content; and
When receiving a request for distribution of the at least one type of sensory content from at least one travel agency terminal 600 , the production terminal 400 and the at least one travel agency terminal sign a copyright contract for the sensory content requested to be distributed. (600) Intermediary portion 390 for intermediary; further comprising
Panoramic VR-based portal service provision system.
delete delete delete delete delete delete

Images