KR102049773B1 - A system and methods for the solutions of making up personalized tourism and cultural 3D mixed reality contents with operating the digital signage - Google Patents

Abstract

The present invention relates to a system for operating a personalized tour content 3D mixed reality generating solution and a digital signage and a method thereof which efficiently improve digital signage management/operation and a customer service. According to the present invention, the system comprises: a 3D mixed reality raw information input unit for collecting 2D/3D image information; a 3D mixed reality content integration processing unit for generating 3D mixed reality information; a 3D mixed reality content output terminal unit for displaying the 3D mixed reality information; a real-time streaming processing unit for outputting a real-time streaming solution service; and a 3D mixed reality database managing unit.

Description

A system and methods for the solutions of making up personalized tourism and cultural 3D mixed reality contents with operating the digital signage}

Based on mixed reality (including virtual / augmented reality) based devices and systems and digital remote control guidance terminal technology such as functional signage, the tourism industry is digitized into 3D mixed reality contents to suit local characteristics. It is a system and method for establishing a tourism system base of a new paradigm that can access tourism contents in cyber space and the real world by unifying it.

Mixed Reality (MR) technology is a technology that overlaps a virtual object having additional information in a real world seen by the user and displays it as a single image, which is also referred to as a step forward in Augmented Reality (AR). Augmented Reality (AR) is a technology that displays 3D virtual images over reality and displays the necessary information through display technology, while VR is a technology that uses 100% virtual images rather than reality. You'll need to wear a goggle headset to experience VR.

Going one step further, the technology that combines real and virtual world information to create a space that merges the two worlds is called 'mixed reality'. It is a technology that takes advantage of AR and VR. Mixed reality (MR) is characterized by combining the real world and virtual information. Mixed reality technology, which is a mixture of real and virtual objects, allows the user to see the real environment, providing better realism and additional new information. In other words, it provides a new concept that can interact in real time between what is real and what is virtual.

Mixed reality technology means "visualization techniques" that add virtual reality to the reality seen through a camera mounted on a computer or smartphone. While virtual reality only provides a fully artificial environment, mixed reality provides a much enhanced real world with the addition of an artificial environment to the real world. In other words, mixed reality does not completely transform the real world into a virtual world but supplements the virtual world to the real world, which not only strengthens the real world but also enhances the user's cognition and interaction with the real world.

According to the mixed reality technology trend report, these technologies could only be used in special fields until the 2000s, but since the 2000s, GPS (Global Positioning System), cameras and display devices are equipped, which makes it the best smartphone for realizing mixed reality. This development and popularization made it available to the general public. As such mixed reality is easily accessible in real life, mixed reality has emerged as a promising technology of the future. Market research firm Gartner has selected mixed reality as one of the 10 promising technologies, and Juniper Research predicts that the mobile augmented reality market will grow to $ 1.5 billion after 2015. .

Currently, the most widely used areas of mixed reality technology are local search and tourism, and it provides a service that augments the area of interest around the user's location with mixed reality.

In particular, the intensive information in the tourism industry has a higher value than other industries, so mixed reality can be easily established as a decisive factor in the tourism industry. Accordingly, in the tourism industry, the importance of smart tourism, which provides tourist information and services such as tourist destinations, food, transportation, reservations, and travel guides to tourists, has emerged.

However, the existing augmented reality technology is used as a representative example of digital restoration of cultural assets and providing tourist information, and virtual reality such as 3D graphics in the real world seen through a camera mounted on a smartphone. It is the most advanced technology (Kounavis, 2012), which is currently used most widely in the area of search and tourism, but still has limitations on user's cognitive ability, interaction and real-time presence.

In addition, new improvement factors for customer service are needed for user terminal and digital signage management / operation and service, for example, various additional services must be enabled through various digital signage, and digital signage for each location must be efficiently managed. As digital signage spreads, the content information to be added to each digital signage increases, and it is a challenge to change various types of content required for each digital signage installed in each region.

The present invention is to solve the above-mentioned problems of the prior art, the tourism sector has the characteristic of selling intangible goods based on the experience of tourists, the prior expectations formed before tourism plays a more important role in decision making Therefore, it aims to provide a digital signage solution and a solution that can support tourism content (resources) based on mixed reality that can satisfy these characteristics.

The relationship between the expected expectations and satisfaction of the mixed reality of tourists and the intention of continuous use is identified. Based on this, the actual data are optimized to meet the expected efforts, expectations, and satisfaction of the tourists. Another goal is to create, disseminate and manage the environment in a short time.

It is another object of the present invention to enable tourists who are using the mixed reality in the field of tourism in the future to understand the expectations of tourists and to facilitate the production of contents and methods to meet the expectations.

The present invention proposes a new data management configuration that processes data transmitted to each digital signage in real time in order to improve efficiency by connecting the digital signage for tourism guidance through a network. Particularly, various data such as real-time mass digital signage operation and environment information collected by using big data analysis technology and cloud-based environment by mobile app using QR code are extracted and databased as a parameter. By discovering new improvement factors in digital signage management / operation and service, we will effectively improve digital signage management / operation and customer service.

In the present invention, a digital signage remote management system including a server system for managing and monitoring each digital signage has advanced IT technology, sensor technology, network, wireless communication technology, and control function in a mixed reality digital signage infrastructure that is basically built. It will converge to present remote monitoring technology combined with IoT-based technology. Therefore, the technical configuration of the present invention enables not only digital signage management / operation, but also various digital signage additional services, and furthermore, it is possible to efficiently manage digital signage distributed in each region. By improving the efficiency of managing digital signage, it is possible to dramatically improve the operational stability of digital signage managers.It is not only improving the content service efficiency of digital signage, but also optimizing the content demand and supply of customers, thus providing stable service throughout the tourism area. Will contribute to.

In addition, the present invention increases the content information to be added to each digital signage as the digital signage spreads, and efficiently handles the problem of changing various types of content required for each digital signage installed in each region to change in a timely manner. Another object of the present invention is to establish an intelligent digital signage operating environment based on big data that can be effectively processed according to an evolved configuration of a mixed reality digital signage infrastructure.

In addition, according to the present invention, as digital signage is spread, tourism information generated in each region is left as a large amount of high-quality information as time passes, and this information is efficiently processed to provide a mixed reality digital signage infrastructure operating environment with rich contents. It is another object of the present invention to construct.

Accordingly, it is an object of the present invention to efficiently process accumulated information on a 3D mixed reality service system, convert it into useful information, and predict the demand of tourism content therefrom to build an effective and optimized tourism content development and operation environment.

In order to achieve the above object, the present invention discloses a 3D mixed reality solution and digital signage management system and method for personalized tourism content.

3D mixed reality raw information input unit as an example of the 3D mixed reality solution and digital signage management system and method for personalized tourism content according to the present invention; 3D mixed reality content integration processing unit; 3D mixed reality content output terminal unit; A real time streaming processor; And 3D mixed reality database management unit.

The 3D mixed reality content integration processing unit, real-time streaming processing unit, the 3D mixed reality database management unit is preferably installed and operated by a single server through a network, or by installing each processing unit and the management unit in different servers to establish a network The system can be operated while maintaining the mutual organic interworking relationship by the integrated control procedure.

The 3D mixed reality raw information input unit includes a 3D authoring tool for generating 3D modeling and basic work source data for generating 3D mixed reality content, and a camera photographing module for generating data by recording a 3D image input from a camera. Generation unit; A sensor unit for generating detection information generated by environmental state detection or content movement; And it comprises a marker or QR code collection unit for generating the original information by the marker or QR code input.

The 3D mixed reality content integration processing unit is a data conversion unit for performing a pre-process for generating the 3D virtual reality content from the information input from the raw data generation unit;

A 3D content generation unit generating 3D virtual reality content using the converted content of the data conversion unit and storing the generated content in a content database;

An event management unit managing event data through the detection information input from the sensor unit and storing the event data in a content database;

An event synthesizing unit for synthesizing the real image and the 3D virtual reality image information by correlating the event information with the object data to the 3D virtual reality content generated by the 3D content generating unit;

A marker for analyzing and storing the information input through the marker or the QR code collecting unit, the marker for analyzing and storing the relationship between the actual operation and the actual image and the 3D virtual reality image;

Marker information is synthesized and stored in order to generate 3D mixed reality information through 3D virtual reality and event information and actual motion as marker information through marker information, and the detected QR code is stored as user information through each database. QR code synthesis unit;

WebGL-based virtual reality script and user interface script to generate a 3D mixed reality content by generating a dedicated script (for example, a script for the HMD of each user's model) for each 3D mixed reality content output terminal unit A 3D mixed reality synthesizing unit generating a script to serve through the unit to synthesize all correlation information in the 3D virtual reality image to generate a final 3D mixed reality content and storing the generated content in a content database;

In order to output the synthesized 3D mixed reality contents to the user terminal or digital signage, the user terminal or digital signage output program (viewer engine) is checked and installed, updated, the contents are transmitted, and the status of each user terminal. Check the user information stored by the information or QR code in the integrated management database to provide services or contents extracted from big data, and manage the output program (viewer engine) installation and update by collecting the status of each digital signage remotely. 3D mixed reality service output management unit that transmits and updates contents suitable for each digital signage, and transmits / manages 3D mixed reality content output terminal state (data for contents, operation / management, etc.) through bidirectional communication. It is configured to include.

The data conversion unit for preprocessing the raw data generation unit information as a step before generating the 3D content in the 3D mixed reality content integration unit, converts the basic source data input from the 3D authoring tool of the raw data generation unit into VRML data, and the camera. The video recording data, which is the raw data input from the photographing module, may be implemented. The 3D content generating unit may generate the VRML data converted by the data converting unit into object content and spatial content. May include a configuration for generating panorama VR content and photo VR content and storing the generated 3D content data in a content database.

In the 3D mixed reality content processing unit, the raw data generation unit, the data conversion unit, the 3D content generation unit, the event synthesis unit, the marker, the QR code synthesis unit, and the 3D mixed reality synthesis unit are mutually compatible and matched with each other. The data formats can include not only various open data packet formats that can be interoperable / converted at each processing stage, but also various data packet formats developed by developers.

The 3D mixed reality synthesis unit in the 3D mixed reality content integration processing unit may further include a configuration for transmitting the generated 3D mixed reality data to the real-time streaming server.

The 3D mixed reality content output terminal unit exchanges various data (data for content, operation, management, etc.) with the 3D mixed reality service output management unit through a bidirectional communication channel, a user PC connected to a wired / wireless public network, a short range wireless communication network ( Bluetooth, NFC, WiFi, Zigbee, etc.) a user terminal including a user smart terminal, a VR device and a VR guidance system and a 3D printer;

It includes a communication module connected to a wired / wireless public network, and includes an image display device such as an LED and an LCD that can output digital contents, and a digital signage unit that can selectively add front and rear cameras.

In the 3D mixed reality content output terminal, the digital signage can service output the 3D mixed reality content using the actual background using a general displayer and a rear camera using a transparent panel, and the 3D printer is suitable for tourism characteristics. The appearance design of the content (character or the like) can be output.

In addition, the digital signage in the 3D mixed reality content output terminal unit includes a polyhedral three-dimensional digital signage configuration in which various 3D mixed reality content output terminals are installed in the outer and inner spaces of a polyhedron shape including a cylindrical shape.

The real-time streaming processor is configured to include a real-time streaming solution server that provides a solution for real-time streaming service of the 3D image obtained from the raw data generation unit, the 3D content generation unit, and the 3D mixed reality synthesis unit.

The 3D mixed reality database management unit converts data generated by the data converter, 3D content data generated by the 3D content generator, content data stored by the event synthesizer, content data stored by the marker, QR code synthesizer, A content database storing content data generated by the 3D mixed reality synthesis unit;

The event data stored by the event management unit, the markers, the marker or QR code data stored in the QR code management unit, the user terminal and digital signage state / management / operation information, QR code, etc. stored in the 3D mixed reality service output management unit Integrated management database for storing the input user information;

The present invention provides the 3D mixed reality service for collecting / processing data generated in each output unit terminal for each 3D mixed reality service output terminal (such as a digital signage or a VR device and a related VR guidance system) in real time in an integrated management database. Contains a database server. In particular, 3D mixing by extracting the correlation between data from real-time mass digital signage operation and member information data collected using big data analysis technology and cloud-based environment by mobile app using QR code By storing in the integrated management database, it is possible to extract new improvement factors in digital signage management / operation and customer management and service to efficiently improve digital signage management / operation and customer service.

In the present invention, the 3D mixed reality remote management system including a 3D mixed reality service output management unit for managing and monitoring each 3D mixed reality service output unit is a network and wireless communication technology and control function in the basic 3D mixed reality solution and digital signage configuration of the present invention. This convergence suggests remote monitoring technology combined with IoT-based technology. Therefore, the technical configuration of the present invention enables not only 3D mixed reality tourism contents service but also various additional services, and can efficiently manage 3D mixed reality service output units distributed in each tourism area, and provides content services through a real-time wired / wireless two-way digital network. It is possible to drastically improve the operational stability by improving management efficiency and management efficiency, and it is possible to realize stable and efficient operation of the entire 3D mixed reality system by optimizing expenses not only for tourism content but also for maintaining the 3D mixed reality device. do.

The 3D mixed reality remote management system including the 3D mixed reality service output management unit of the present invention efficiently processes big data information on the 3D mixed reality service system, converts it into useful information, predicts the demand of tourism content therefrom, and develops an effective tourism content. Build an operating environment.

According to the configuration of the present invention, the operation error, abnormality detection, periodic inspection information, etc. of the digital signage are estimates of operating rules through correlation analysis between terminal data accumulated over many years and the determination of whether the real-time data and the estimated operating rules match. to be. In addition, demand forecasting of tourist consumers is a QR code and a mobile app, which is a process of estimating preferred contents through analyzing consumer's consumption behavior and developing new contents to solve them. Such data analysis has been a difficult task to be performed in the past, but as the configuration of the present invention, it is possible to extract data by collecting data from the integrated 3D mixed reality server unit system and analyzing the correlation of these data.

Therefore, the configuration of the present invention proposes a configuration optimized for 3D mixed reality efficiency operation by utilizing the property of the tourism industry well. To this end, the present invention is a 3D mixed reality service output management unit that collects the information installed in the various sensor and internal and external information processing configuration and wired and wireless communication configuration, 3D mixed reality service output terminal unit in each tourism area installed in the 3D mixed reality content output terminal unit It consists of a service output unit of the state management module, a database server for analyzing and processing the big data in the integrated server unit and outputs the analyzed data to the administrator terminal, and a database for storing the data.

Each 3D mixed reality content output terminal generates location-based information generated from GPS sensors, real-time status information, various sensor information, QR codes, and customer information generated by the mobile app. The collected information is stored in a database, and a large amount of accumulated information is analyzed and managed through each database analysis technique.

The result information according to the database analysis procedure may include a customer's tourism behavior prediction service, which is the most useful prediction that can extract the correlation between time and content demand by collecting data of elements by a certain collection time period By selecting a model, correlation prediction model coefficients can be extracted. For example, as a statistical estimation method using the external environment (temperature, humidity, weather, region) and the preferred content type (history, culture, art, taste food, etc.) as a variance, a correlation analysis or a regression analysis method is used. It is possible to set up a multi-faceted prediction model that uses a combination of statistical analysis techniques, analyze or use data mining (big data) associative analysis, cluster analysis, decision making, or predict intelligent estimation curves. In the case of using, the result information can be extracted by setting the multi-faceted prediction model.

The estimated values predicted by the predictive model and real-time measured values can be compared, and the estimated model parameters and environmental factor variances can be selected within the range in which the estimated values estimated by the predicted model are reliable, and the database is selected using the selected estimated model. The analysis analyzes which parameters (content type, environmental factors) make customers prefer tourism contents, and predicts to develop such tourism contents in advance.

The real-time information of the 3D mixed reality service output unit (digital signage) collected by the present invention can be analyzed in various forms, and can include a change procedure of the collection sensor as necessary, so that the sensor can be changed. Analysis can be attempted, and it is possible to secure a comprehensive operation and management monitoring system of the innovative 3D mixed reality tourism contents service that coexists with the new ubiquitous and IoT systems instead of the operation of the conventional virtual reality output device itself.

The analysis screen of the 3D mixed reality service output management unit service output unit state management module is a block for viewing the status information of each 3D mixed reality content output terminal unit, status information and management information, member information by QR code, and communication management information. There is a select button for performing such a function. The service output unit state management module of the 3D mixed reality content output terminal unit may be implemented on a web basis, and may include an application program module (API) for expressing analysis results in various output formats.

The state management procedure of the 3D mixed reality service output management unit through the user terminal of the service output unit state management module 2512 may have the following menu procedure. In the login menu, user authentication and server login are performed. Through the content management menu, each content list display generated by the 3D mixed reality content processing unit 2, each content management menu (add / modify / delete, etc.), 3D mixed Reality sensor management menu, marker, member management menu input by QR code, content output management menu (content update, device status management) for content output terminal (user terminal, digital signage, 3D printer, VR device, etc.) , Other service management), a device status management menu for a content output terminal, a real time streaming server management menu, and a 3D mixed reality database management server menu.

The tourism sector has the characteristic of selling intangible goods based on the experiences of tourists, and the mixed expectations based on the mixed reality that can satisfy these characteristics, because the pre-expectations formed before tourism play a more important role in decision making. It has the effect of providing digital signage and solutions that can support content (resources).

The relationship between the expected expectations and satisfaction of the mixed reality of tourists and the intention of continuous use is identified. Based on this, the actual data are optimized to meet the expected efforts, expectations, and satisfaction of the tourists. In the environment, it is possible to produce, disseminate, and manage in a short time, and propose a new data management configuration that processes data sent to each digital signage in real time in order to make it effective by connecting tourist information digital signage through a network. By extracting the correlation between data from the real-time mass digital signage operation and environmental information data collected by using big data analysis technology and cloud-based environment by mobile app using QR code, New improvements to your management / operation and services The discovery of phosphorus will effectively improve digital signage management / operation and customer service.

In addition, according to the present invention, as digital signage spreads, content information to be added to each digital signage increases, and it is left as a processing task to change various types of content required for each digital signage installed in each region. It will provide an optimized digital signage operating environment that can be effectively processed according to the evolving composition of mixed reality digital signage infrastructure that efficiently processes local needs and changes them in a timely manner.

Figure 1a is a block diagram of a personalized tourism content 3D mixed reality creation solution and digital signage management system system according to an embodiment of the present invention.
FIG. 1B is a detailed configuration diagram of the raw data generating unit of FIG. 1A.
FIG. 1C is a detailed configuration diagram of the data converter of FIG. 1A.
FIG. 1D is a detailed configuration diagram of the 3D content generating unit of FIG. 1A.
FIG. 1E is a diagram illustrating a relationship between a real time video input and a real time streaming processor of FIG. 1B.
2 is a diagram illustrating a relationship between the sensor unit and the event management unit of FIG. 1A.
3 is a diagram illustrating a relationship between the marker, the QR code collecting unit, the marker, and the QR code managing unit of FIG.
4 is a detailed configuration diagram of the 3D mixed reality synthesis unit of FIG. 1A.
5 is a detailed configuration diagram of the 3D mixed reality service output management unit of FIG. 1A.
6 is a detailed configuration diagram of the 3D mixed reality database manager of FIG. 1A.
7 is a diagram illustrating various user terminals according to an embodiment of the present invention.
8 is a block diagram of various digital signage according to an embodiment of the present invention.
9 is a configuration diagram of a 3D mixed reality digital signage equipped with a rear camera according to another embodiment of the present invention.
10 is a block diagram of a polyhedron solid 3D mixed reality digital signage according to another embodiment of the present invention, in which various user terminals are combined and installed.
11 is a flowchart illustrating a service output unit state management module of a 3D mixed reality service output management unit according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, a configuration according to a preferred embodiment of the present invention will be described in detail.

The present invention may have a variety of embodiments, including one embodiment, specific embodiments are shown in the drawings or detailed description, but not limited to the configuration of all equivalent to equivalent technical scope included in the technical idea of the present invention. It should be seen as including.

In the present specification, the 3D mixed reality solution and digital signage operating system and method for personalized tourism content in order to clearly explain the technical concept of the present invention, the 3D mixed reality raw information input unit; 3D mixed reality content integration processing unit; 3D mixed reality content output terminal unit; A real time streaming processor; The configuration of the 3D mixed reality database manager will be described.

1A is a schematic diagram of a 3D mixed reality solution and a digital signage management system and method for personalized tourism content according to an embodiment of the present invention, a 3D mixed reality raw information input unit 1; 3D mixed reality content processing unit (2); 3D mixed reality content output terminal unit 3; A real time streaming processor 4; In addition, the 3D mixed reality database management unit 5 is configured to include a 3D mixed reality solution for tourism content and digital signage operation.

In more detail, the 3D mixed reality solution for the personalized tourism content and the system configuration for digital signage operation are configured to collect various 2D / 3D image information from the 3D mixed reality raw information input unit 1 and the 3D mixed reality content The integrated processor 2 generates 3D mixed reality information through various procedures, displays the 3D mixed reality content output terminal 3, and outputs the real time streaming soluene service through the real time streaming processor 4, and 3D mixed. The contents generated in the real database manager 5 are stored.

The 3D mixed reality content integration processing unit 2, real time streaming processing unit 4, and 3D mixed reality database management unit 5 is preferably installed and operated by a single server through a network, or each processing unit and management unit It can be installed on different servers and the system can be operated while maintaining the mutual interworking relationship by the integrated control procedure through the network.

The source information input unit 1 of FIG. 1A includes a source data generation unit 11; A sensor unit 12 for detecting an environmental state in FIG. 1A; And it comprises a marker or QR code collecting unit 13 for generating the raw information by the marker or QR code input, the detailed configuration as an embodiment of the raw data generating unit 11 is shown in Figure 1b The detailed functions that may be included as an embodiment of the sensor unit 12 are shown in FIG. 2, and the 3D authoring tool 111, the camera photographing module 112, and the real-time photographing module 113 shown in FIG. 1B. 7 may include 3D authoring tools using a computer, a camera of a user's smartphone, a tourist 2D / 3D image raw data photographed from a diver or a sports action camcorder, and a drone photographing camcorder as shown in FIG. 7. .

The 3D mixed reality content integrating unit 2 of FIG. 1A is a data converting unit 211 which is a step of performing pre-processing for generating 3D virtual reality content from the information input from the raw data generating unit 11;

A 3D content generation unit 212 generating 3D virtual reality content using the converted content of the data conversion unit 211 and storing the generated content in a content database;

An event management unit 221 managing event data through the detection information input from the sensor unit 12 and storing the event data in a content database;

An event synthesizing unit 222 for synthesizing 3D virtual reality image information with actual information through correlation of object information with 3D virtual reality content generated by the 3D content generating unit 212;

A marker for analyzing and storing the information input through the marker or the QR code collecting unit 13 to analyze and store the relationship between the user's actual operation and the real image and the 3D virtual reality image, a QR code management unit 231;

Marker information is synthesized and stored in order to generate 3D mixed reality information through 3D virtual reality and event information and actual motion as marker information through marker information, and the detected QR code is stored as user information through each database. QR code synthesis unit 232;

WebGL-based virtual reality scripts and user interface scripts for each of the 3D mixed reality content output terminal to generate a dedicated script (for example, a script for the HMD of each user's model) by varying the 3D mixed reality content A 3D mixed reality content output terminal 3 generates a script for servicing and synthesizes all correlation information in the 3D virtual reality image to generate the final 3D mixed reality content and stores the generated content in the content database 51. 3D mixed reality synthesis unit 241;

In order to output the synthesized 3D mixed reality content to the user terminal 31 or the digital signage 32, the output program (viewer engine) of the user terminal or the digital signage is installed, updated, and the content is transmitted. And check the user information stored by the status information or QR code of each user terminal in the integrated management database 52 to provide a service or content extracted from big data, and collect and output the status of each digital signage remotely The program (viewer engine) manages the installation and update (Figs. 5 and 2511a), transmits and updates content suitable for each digital signage (Figs. 5 and 2511b), and displays the state of the 3D mixed reality content output terminal through bidirectional communication. The transmission / management (data for contents, operation / management, etc.) (FIG. 5, 2512) includes a 3D mixed reality service output management unit 251. FIG.

FIG. 1B is a detailed configuration diagram of the 3D mixed reality raw information input unit 1 of FIG. 1A and inputted from a camera and a 3D authoring tool 111 generating 3D modeling and basic work source data for generating 3D mixed reality content. And a camera photography module 112 for recording 3D video to generate data.

1B and 1C, the data conversion unit 211 which preprocesses the raw data generator 11 information as a pre-step of generating 3D content in the 3D mixed reality content integration processor 2 is a 3D authoring tool of the raw data generator. The basic source data, which is the raw data input at 111, is converted into VRML data (FIGS. 1C and 2111), and the image recording data, which is the raw data input from the camera photographing module 112, is converted to VR data, respectively (FIG. 1C). And 2112, and the 3D content generation unit (FIGS. 1D and 212) generates VRML data converted by the data conversion unit 211 into object content and spatial content (FIGS. 1D and 2121). The VR data may include a configuration for generating panorama VR content and photo VR content (FIGS. 1D and 2121) and storing these generated 3D content data in a content database (FIGS. 1D and 2122).

The 3D mixed reality content data format may include a raw data format of the raw data generator 11, a converted data format of the data converter 211, a 3D content data format of the 3D content generator 212, and an event synthesizer 222. And markers, data formats generated in each processing procedure of the QR code synthesizing unit 232 and the 3D mixed reality synthesizing unit 241 must be mutually compatible and matched. In addition to the various open data packet formats possible, various data packet formats developed by the developer may be included.

As shown in FIG. 4, the 3D mixed reality synthesis unit 241 in the 3D mixed reality content integration processor 2 may further include a configuration 2412 for transmitting the generated 3D mixed reality data to the real time streaming server 4. .

In FIG. 1A, the 3D mixed reality content output terminal unit 3 exchanges various data (data for content, operation / management, etc.) with the 3D mixed reality service output managing unit 251 through a bidirectional communication channel. User terminal connected to a user PC connected to a local area network (Bluetooth, NFC, WiFi, Zigbee, etc.), a user smart terminal including a VR device, a VR guidance system, and a 3D printer (31, FIGS. 5 and 7). Of 2512a);

Digital signage unit 32, which includes a communication module connected to a wired / wireless public network, and which can additionally add a video display device such as LED, LCD, etc., which can output digital contents, and a user interface device, and a front and rear cameras. 2512b of FIG. 8 and 2512c of FIG. 9).

In the 3D mixed reality content output terminal unit 3, the digital signage 32 may service output the 3D mixed reality content using the actual background using a general display using a transparent panel and a rear camera (Figs. 9 and 2512c). In addition, the 3D printer can output the appearance design of the tourism content (characters, etc.) according to the characteristics of the tourist area.

As shown in FIG. 10, the digital signage in the 3D mixed reality content output terminal includes a polyhedral solid digital signage structure in which various 3D mixed reality content output terminals are installed in the outer and inner spaces of a polyhedron shape including a cylindrical shape.

8 to 10, various user terminals 2512a for 3D mixed reality output may be combined and installed in parallel.

The real time streaming processor 4 provides a solution for real-time streaming of 3D images obtained from the raw data generator 113, the 3D content generator 212, and the 3D mixed reality synthesis unit 241. It is configured to include a real-time streaming solution server 41.

The 3D mixed reality database manager 5 converts the generated data from the data converter 211, 3D content data generated by the 3D content generator 212, and content data stored by the event synthesizer 222. A content database 51 for storing the content data 232 stored in the marker, the QR code synthesizing unit, and the content data generated by the 3D mixed reality synthesizing unit 241;

The event data stored by the event management unit 221, the marker, the marker or QR code data stored in the QR code management unit 231, the user terminal 31 and the digital storage stored in the 3D mixed reality service output management unit 251 Integrated management database 52 for storing user information inputted by the status of 32 / status / management / operation information, QR code, etc .;

In the present invention, the 3D mixed reality service output unit state management module 2512 including a 3D mixed reality service output management unit for managing and monitoring each 3D mixed reality service output unit includes a network and a basic 3D mixed reality solution and digital signage configuration. Wireless communication technology and control functions are converged to present remote monitoring technology combined with IoT-based technology. Therefore, the technical configuration of the present invention enables not only 3D mixed reality tourism contents service but also various additional services, and can efficiently manage 3D mixed reality service output units distributed in each tourism area, and provides content services through a real-time wired / wireless two-way digital network. In addition, it is possible to improve the operational stability by improving the efficiency of management and management, and it is possible to improve the tourism contents and minimize the expense in terms of the cost of maintaining the 3D mixed reality device, thereby realizing stable and efficient operation of the entire 3D mixed reality system. Done.

The 3D mixed reality service output unit state management module 2512 including the 3D mixed reality service output manager of the present invention efficiently processes big data information on the 3D mixed reality service system, converts it into useful information, and predicts the demand of tourism contents therefrom. To build an effective tourism contents development and operation environment.

According to the configuration of the present invention, the operation or maintenance information of the digital signage is extracted through the correlation analysis between the terminal data accumulated during the operation period, and the management operation prediction value of the terminals is extracted and the 3D mixed reality content by the comparison between the real time data and the prediction value. The maintenance of the output terminals 3 is performed in advance. In addition, the demand forecast of tourism consumers can be referred to as a process of estimating preferred content through analysis of consumer's consumption behavior with QR codes and mobile apps and developing new contents to solve this problem. Data can be extracted by collecting data from the integrated server system and analyzing the correlation of these data in the 3D mixed reality database server.

The present invention provides a 3D mixed reality service that collects information of various sensors and devices installed in the 3D mixed reality content output terminal unit 3 and wired / wireless communication configuration, and 3D mixed reality service output terminal unit distributed in each tourist area. A service output unit state management module 2512 of the output management unit; A 3D mixed reality database server 53 for analyzing and analyzing the big data and outputting the result data to the manager terminal; And an integrated management database 52 for storing management data.

Each 3D mixed reality content output terminal 3 generates real-time status information, various sensor information, QR codes, and customer information generated by a GPS sensor together with location-based information generated from a GPS sensor. The collected information is transmitted to the management module 2512 and stored in a database, and a large amount of accumulated information is analyzed and managed through each database analysis technique.

The estimated values predicted by the predictive model and real-time measured values can be compared, and the estimated model parameters and environmental factor variances can be selected within the range in which the estimated values estimated by the predicted model are reliable, and the database is selected using the selected estimated model. The analysis analyzes which parameters (content type, environmental factors) make customers prefer tourism contents, and predicts the development of such tourism contents in advance.

In addition, the database analysis procedure may include a procedure for clustering (clustering) and monitoring the parameter factors that are characteristic according to the location as well as external environmental change factors, so that the demand for tourism content that occurs frequently only in a specific region may be separated. Analysis is possible and includes the development of content for those regions, improved 3D mixed reality service output (digital signage) operations, and new content suggestion to customers.

The real-time information of the 3D mixed reality service output unit (digital signage) collected by the present invention may be analyzed in various forms, and may include a change process of the collection sensor as needed, and various data may be changed since the sensor may be changed. Analysis can be attempted, and it is possible to secure a comprehensive operation and management monitoring system of the innovative 3D mixed reality tourism contents service that coexists with the new ubiquitous and IoT systems instead of the operation of the conventional virtual reality output device itself.

The analysis screen of the 3D mixed reality service output management unit service output unit state management module 2512 is a block and state information and management information, the member information by the QR code to view the status information of each 3D mixed reality content output terminal unit, There is a selection button for performing communication management information. The service output unit state management module of the 3D mixed reality content output terminal unit may be implemented on a web basis, and may include an application program module (API) for expressing analysis results in various output formats.

The state management procedure of the 3D mixed reality service output management unit through the user terminal of the service output unit state management module 2512 may have the following menu procedure. That is, the login menu has a user authentication and server login procedure, and each content list display generated by the 3D mixed reality content processing unit 2 through the content management menu, each content management menu (add / modify / delete, etc.), 3D mixed reality sensor unit management menu, marker, member management menu input by QR code, content output management menu (content update, device for content output terminal (user terminal, digital signage, 3D printer, VR device, etc.)) Status management, other service management), a device status management menu for a content output terminal, a real time streaming server management menu, and a 3D mixed reality database management server menu.

11 is a procedure diagram of displaying a menu on the screen of the user terminal 2512a by the service output unit state management module 2512. When a user (including an administrator) logs in through an initial screen (S2), the top menu (login) is displayed. , The top menu including some or all of the content management, live streaming server management, and database management menus is displayed on the screen (S3), the user selects the top menu (S4), and the submenus of the selected menu are displayed ( S5) After the user selects a submenu (S6 to S8) to perform the selected submenu function (S9), the user maintains the login state or returns to the initial screen by the user's selection (S10).

The menu displayed on the screen of the user terminal 2512a by the service output unit state management module 2512 is

(1) From the top menu of the initial screen, the setup menu (user log management settings), content management menu, real time streaming server management menu, and 3D mixed reality database management menu are displayed.

(2) The setting menu, which is a submenu, displays the user management menu and other configuration menus for the 3D mixed reality integrated server unit login.

(3) In the content management menu, which is a submenu, all content data generated from the 3D mixed reality integrated server part (data conversion part, 3D content generating part, event synthesizing part, marker, QR code synthesizing part, 3D mixed reality synthesizing part) Content data) list display, submenu for each content management (add / modify / delete etc.), submenu for 3D mixed reality sensor part management, member management submenu input by marker and QR code, content output terminal part Displaying the content output management submenu (content update, content status management, other service management) for the user terminal, digital signage, 3D printer, VR device, etc., the device status management submenu for the content output terminal;

(4) The submenu, Live Streaming Server Management, displays and manages the server management menu for the live streaming service.

(5) The submenu 3D mixed reality database management server menu displays information stored in the integrated management database and manages statistics.

As the submenu, the statistical management menu displays and manages statistical information of information including some or all of operation information, maintenance information, management information, and customer management information (tourist-related information) of each device, and manages the database. It may contain a menu (enter parameters for statistics for each database).

In addition, the real-time status information of each 3D mixed reality content output terminal unit stored in the integrated management database in the 3D mixed reality sensor unit management submenu and the content output terminal device state management submenu of the content management menu of the submenu is And selectively changing the parameters to be collected and analyzed according to the setting of the content output terminal unit state management submenu of the sensor of the output terminal unit and the real-time state information generation unit, and the 3D mixed reality content output terminal unit. The state monitoring object is changed by changing the information of the real time state information generating unit, and the device state management menu for the content output terminal may be included to allow the 3D mixed reality content output terminal to change the real time monitoring target. .

1: 3D mixed reality raw information input unit
11: raw data generator
111: 3D Authoring Tools 112: Camera Shooting
113: real time shooting
12: sensor unit 13: marker, QR code collection unit
2: 3D mixed reality content integrated processing unit
211: data conversion unit
2111: convert VRML data 2112: convert VR data
212: 3D content generation unit
2121: Content Creation 2122: Content Storage
221: event management unit
2211: transmitting unit 2212: receiving unit
2213: management
222: event synthesis unit
231: marker, QR code management unit
2311: QR code detection unit 2312: Marker unit
232: marker, QR code synthesis unit
241: 3D mixed reality synthesis unit
2411: Script Generation 2412: Live Streaming
251: 3D mixed reality service output management unit
2511: 3D mixed reality content output management module
2511a: Viewer Management 2511b: Output Management
2512: service output module state management module
2512a: user terminal 2512b: digital signage
3: 3D mixed reality content output terminal
31: user terminal 32: digital signage unit
4: real time streaming processing unit
5: 3D mixed reality database management unit
51: content database 52: integrated management database
53: 3D mixed reality database management server

Claims (13)

In constructing personalized tourism contents 3D mixed reality creation solution and digital signage system,
3D mixed reality raw information input unit for collecting 2D / 3D image information;
3D mixed reality content integration processing unit for generating 3D mixed reality information;
A 3D mixed reality content output terminal unit displaying 3D mixed reality information;
A real time streaming processor for outputting a real time streaming solution service;
Includes; 3D mixed reality database management unit for storing the contents generated by the content integration processing unit;
The 3D mixed reality raw information input unit comprises a raw data generator; A sensor unit detecting an environmental state; And a marker or QR code collection unit generating raw information by a marker or QR code input.
The 3D mixed reality content integration processor may include a data converter configured to perform preprocessing to generate 3D virtual reality content from information input from the raw data generator;
A 3D content generation unit generating 3D virtual reality content using the converted content of the data conversion unit and storing the generated content in a content database;
An event management unit managing event data through the detection information input from the sensor unit and storing the event data in a content database;
An event synthesizing unit for synthesizing the real image and the 3D virtual reality image information by correlating event information with object data to the 3D virtual reality content generated by the 3D content generating unit;
A marker for analyzing and storing the information input through the marker or the QR code collecting unit, the marker for analyzing and storing the relationship between the actual operation and the actual image and the 3D virtual reality image;
Marker information is synthesized and stored in order to generate 3D mixed reality information through 3D virtual reality and event information and actual motion as marker information through marker information, and the detected QR code is stored as user information through each database. QR code synthesis unit;
The event information and correlation information are synthesized with the generated 3D content image, and finally, a WebGL-based virtual reality script and a user interface script are generated as a script for each 3D mixed reality content output terminal, and 3D mixed reality content is 3D mixed reality. A 3D mixed reality synthesis unit serving through a content output terminal unit and storing the generated content in a content database;
Personalized tourism comprising a 3D mixed reality service output management unit having a 3D mixed reality content output management module in charge of viewer management and output management to output synthesized 3D mixed reality content to the 3D mixed reality content output terminal. Content 3D mixed reality creation solution and system for digital signage operation.
The method of claim 1,
A 3D authoring tool for generating 3D modeling and basic work source data for generating 3D mixed reality content; A camera photographing module for recording 3D images input from a camera to generate data; A system for the operation of digital signage and 3D mixed reality creation solution for personalized tourism contents consisting of some or all of the real-time shooting module for recording the live streaming service.
The method of claim 1,
The 3D mixed reality content output terminal unit has a bidirectional communication channel configuration for exchanging content data and data for operation / management with a 3D mixed reality service output management unit, a user PC connected to a wired / wireless public network, and a user smart having a short range wireless communication network connection function. A user terminal including a terminal, a VR device, a VR guidance system, and a 3D printer;
It has a communication module connected to the wired and wireless public network and has a video display device and a user interface device such as LED, LCD, etc. that can output digital content, and a digital signage unit that can selectively add a front camera and a rear camera; Personalized tourism contents 3D mixed reality creation solution and digital signage system.
The method of claim 1,
In the 3D mixed reality content output terminal, the digital signage can output a service of 3D mixed reality content using a real background using a general displayer and a rear camera using a transparent panel, and the 3D printer is suitable for the characteristics of the tourist area. Personalized tourism contents 3D mixed reality creation solution that can output the appearance design of tourism contents and system for digital signage operation.
The method of claim 1
In the 3D mixed reality content output terminal unit, the digital signage includes a polyhedron solid digital signage configuration in which one or more 3D mixed reality content output terminals and a user interface are installed in the outer side of the polyhedron shape and the inner space of the polyhedron. Tourism content 3D mixed reality creation solution and system for digital signage operation.
The method according to any one of claims 3 to 5
The digital signage is a system for operating a digital signage and personalized tourism content 3D mixed reality creation solution that can be installed in parallel with the 3D mixed reality content output terminals.

The method of claim 1
The real-time streaming processor comprises a real-time streaming solution server that provides a solution for real-time streaming service of the 3D image input from the raw data generation unit, the 3D content generation unit, and the 3D mixed reality synthesis unit; 3D mixed reality creation solution and system for digital signage operation.
The method of claim 2
The 3D mixed reality content output management unit includes a viewer management module for checking, installing, and updating an output program of a user terminal or digital signage to output the synthesized 3D mixed reality content to a user terminal or a digital signage;
An output management module for transmitting the synthesized 3D mixed reality content to the 3D mixed reality content output terminal;
It checks the user information collected and stored by the status information or QR code of each user terminal in the integrated management database and provides the extracted result through data analysis, and installs and updates the output program by collecting the status information of each content output terminal part. Personalized tourism content comprising; service output unit state management module for managing, transmitting and updating the content to be displayed in each digital signage, and transmit / manage the 3D mixed reality content output terminal state through the two-way communication 3D mixed reality creation solution and system for digital signage operation.
delete delete In constructing personalized tourism contents 3D mixed reality creation solution and digital signage,
3D mixed reality raw information input unit for collecting 2D / 3D image information;
3D mixed reality content integration processing unit for generating 3D mixed reality information;
A 3D mixed reality content output terminal unit displaying 3D mixed reality information;
A real time streaming processor for outputting a real time streaming solution service;
Includes; 3D mixed reality database management unit for storing the contents generated by the content integration processing unit;
The 3D mixed reality raw information input unit comprises a raw data generator; A sensor unit detecting an environmental state; And a marker or QR code collection unit generating raw information by a marker or QR code input.
The 3D mixed reality content integration processor may include a data converter configured to perform preprocessing to generate 3D virtual reality content from information input from the raw data generator;
A 3D content generation unit generating 3D virtual reality content using the converted content of the data conversion unit and storing the generated content in a content database;
An event management unit managing event data through the detection information input from the sensor unit and storing the event data in a content database;
An event synthesizing unit for synthesizing the real image and the 3D virtual reality image information by correlating event information with object data to the 3D virtual reality content generated by the 3D content generating unit;
A marker for analyzing and storing the information input through the marker or the QR code collecting unit, the marker for analyzing and storing the relationship between the actual operation and the actual image and the 3D virtual reality image;
Marker information is synthesized and stored in order to generate 3D mixed reality information through 3D virtual reality and event information and actual motion as marker information through marker information, and the detected QR code is stored as user information through each database. QR code synthesis unit;
The event information and correlation information are synthesized with the generated 3D content image, and finally, a WebGL-based virtual reality script and a user interface script are generated as a script for each 3D mixed reality content output terminal, and 3D mixed reality content is 3D mixed reality. A 3D mixed reality synthesis unit serving through a content output terminal unit and storing the generated content in a content database;
3D mixed reality service output management unit having a; 3D mixed reality content output management module responsible for viewer management and output management to output the synthesized 3D mixed reality content to the 3D mixed reality content output terminal;
How to operate digital signage and personalized tourism contents 3D mixed reality creation solution including 3D mixed reality database management unit including content database, integrated management database, and 3D mixed reality database management server to control and analyze databases.
The method of claim 11,
The 3D mixed reality content output management module and the service output unit state management module of the 3D mixed reality service output management unit may be performed by a web-based output procedure.
The state management procedure of the 3D mixed reality service output management unit through the user terminal in the service output state management module is displayed on the screen when the user logs in through the initial screen, and when the user selects the top menu, the submenu of the selected menu is displayed. Is displayed and the user selects a submenu to perform the selected submenu function and then maintains the login status or returns to the initial screen by the user's selection.
A menu displayed on the user terminal screen by the service output unit state management module displays a setting menu, a content management menu, a real-time streaming server management menu, and a 3D mixed reality database management menu in the initial menu top menu;
Displaying a user management menu and other configuration menus for the 3D mixed reality integrated server unit login in the submenu;
The content management menu, which is a sub menu, displays each content data list generated by the 3D mixed reality content processing unit, a sub menu for managing each content, a sub menu for managing the 3D mixed reality sensor unit, a member input by a marker and a QR code. A management submenu, a content output management submenu for the content output terminal unit, and a device state management submenu for the content output terminal unit;
A sub menu, the real time streaming server management menu, displays and manages a server management menu for a real time streaming service;
In the sub menu 3D mixed reality database management server menu, information stored in the integrated management database is displayed and statistics are managed.
As a submenu, the statistical management menu displays and manages statistical information of information including some or all of operation information, maintenance information, management information, and customer management information of each device, and displays a database management menu. Personalized tourism content including 3D mixed reality creation solution and digital signage operating method.
The method of claim 11,
Of the content management menu
In the 3D mixed reality sensor unit management submenu and the content output terminal unit state management submenu, real-time state information of each 3D mixed reality content output terminal unit stored in the integrated management database generates a sensor and real-time state information of the output terminal unit. And selectively changing collection and analysis target parameters according to the content output terminal unit state management submenu setting change for the unit;
An apparatus for a content output terminal for changing the state monitoring object by changing the information of the real-time state information generation unit of the 3D mixed reality content output terminal unit and controlling the real-time monitoring target change procedure of the 3D mixed reality content output terminal unit centrally. Personalized tourism content 3D mixed reality creation solution including status management menu and operation of digital signage.

Images