TW201710871A - Graphical processing of data, in particular by mesh vertices comparison - Google Patents

Abstract

Systems and methods according to present principles relate to processing and presentation of data from a database, transmitted in a pushed or pulled fashion from the database and rendered to present a 3-D video or image to a viewer. The 3-D video or image may be rendered as a 3-D CG object on a display via a, e.g., GPU or video graphics card, or integrated chipset, or rendered on a specialized device such as a virtual reality headset so as to be perceived by a viewer as a 3-D visualization.

Description

Graphic processing data, especially processing data through grid comparison Cross-references for related applications

The application claims priority based on U.S. Provisional Patent Application Serial No. 62/167,665. The application was filed on May 28, 2015, entitled "Virtual Business Cloud Systems and Methods for Products and Services in Shopping, Real Estate, Education, Tourism, and Entertainment," and Application Serial No. 62/237,090, 2015 Submitted on October 5th, entitled "Three-Dimensional (3D), Virtual Reality (VR) and Augmented Reality (AR) Business/Showroom Platform/Cloud Systems and Methods, such as Shopping, Real Estate/Lease, Education, Travel and Entertainment," These provisional patents are owned by the current patent applicants, the entire contents of which are incorporated herein by reference.

The present invention relates to graphics processing streamlines and graphical representation data.

Data visualization has been widely used. In particular, in the prior art, there are various methods for displaying data in a database. These types of data visualizations operate in different ways.

The method of describing the data, for the sake of comparison, is embodied in charts and graphs, such as column charts, pie charts, and the like. In a three-dimensional environment, data has been graphically represented, such as by rendering a three-dimensional CG object. For example, graphics engines such as Unity, Unreal Engine, and CryEngine provide important 3D object rendering capabilities. The construction of 3D data objects can also take advantage of well-known 3D graphics design applications such as Maya, Blender and 3ds Max. Designing a specific architectural framework project is available for Google's SKETCHUP, as well as other applications mentioned above.

However, so far, the integration method of the three-dimensional environment designed with the above application and some types of databases is extremely lacking. In other deficiencies, the corresponding API and database access functions cannot be established.

Therefore, creating such a three-dimensional environment is beyond the reach of ordinary people. It requires a lot of custom programming and skills to achieve. In addition, the lack of such three-dimensional and database integration makes it very difficult for ordinary users to view the data, so that the user has no choice but to use the old way, such as requiring a large amount of input and click data from the user, and thus further requires additional calculation cycles. And in many cases unnecessary battery consumption.

Therefore, both UI developers and regular users need a better way to present data in the database.

The above background gives a brief introduction to the following summary and detailed description. This background description is not intended to help determine the scope of patent claims, nor is it intended to limit patent claims to protect against all of the above deficiencies and problems.

Systems and methods in accordance with the principles of the present patents meet the above needs in a variety of ways.

In a first aspect, the present invention provides a media experience for a user by providing a method for configuring a server, including: a first server providing a first user interface, and a user (creator) can self-create and construct a virtual environment, the virtual The environment includes at least one target grid; a second user interface can be provided on the first server, or on a second server in the network that communicates with the first server, in a computer session, The first user logs into the virtual environment and moves and interacts with each other in the virtual environment, and the first user can send a session inviting the second user to share a virtual environment with the first user. After the second user accepts the invitation of the first user, the second user can share the browsing session of the computer with the first user and move or interact with each other in the virtual space together with the first user.

Implementations of the invention may include one or more of the following aspects. On the first server or the second server, the second user interface can be further used to allow the first user to build a theme grid for use in the virtual environment. The theme grid may be an avatar, or it may be a virtual environment, such as a room or building. After accepting the invitation, the second user interface may be presented to the second user. The second user interface can be further used to allow a second user to build the theme grid used in the virtual environment. The second user interface can be further used to allow the first user to move the grid relative On the target grid. There may be metadata associated with this in the target grid, and the metadata may indicate the ratio of the grid to the real entity. The theme mesh and the target mesh may be configured to the same scale so that the topic mesh can be displayed in the correct size relative to the target mesh. The first user and the second user can access the virtual environment using a virtual space (VR) or an enhanced space (AR) device.

In another aspect, the present invention is directed to a non-transitory computer readable medium that includes an illustration of forming a computing environment to perform the methods described above.

In another aspect, the present invention is directed to a method of providing a media experience. A creator can modify the media experience on the server, including: accessing the database, the database including data of available objects in the inventory; exposing a user interface operable to allow the creator to position the object at a location in a virtual environment; An operational user interface may allow the creator to create a three-dimensional model of an object or import a three-dimensional model of the object from the outside into the server.

Implementations of the invention may include one or more of the following aspects. Allowing the creator to create a three-dimensional model of the object includes allowing the creator to import an image corresponding to the object into the 3D model generation engine. The method can further include disclosing a user interface to enable the first user to access the virtual environment. The public user interface may further allow the first user to invite the second user to access the virtual environment.

In certain embodiments, the advantages of the invention include one or more of the following aspects. 3D, 2-D, AR, VR can be very convenient Created by the creator and used by other users for a variety of purposes, including the user's use of the theme grid in the virtual environment relative to the target grid location. Other advantages will be reflected in the detailed description below, including the description of the graphics and patent claims to understand these advantages.

In further aspects and embodiments, the functionality of the above-described methods of the various aspects are designed in one system and embodied in various aspects. The functionality of any of the various aspects of the embodiments, including but not limited to any of the above-mentioned embodiments, applies to all other aspects and embodiments identified herein, including but not limited to any of the embodiments mentioned above. Furthermore, the functions of the embodiments of any of the various aspects, including but not limited to any of the above-mentioned embodiments, are independently combinable, in part or in whole, in any manner with other embodiments described herein, for example, one, Two, or three or more embodiments may be combined in whole or in part. Furthermore, the functionality of any of the various aspects of the embodiments, including but not limited to any of the embodiments mentioned above, may be optional for other aspects or embodiments. Aspects or implementations of any method may be performed by aspects or implementations of another method, and any aspect or implementation of a system or device may be configured to perform aspects or implementations of another method, including but not limited to any of the above-mentioned embodiments in.

The summary provided is to introduce the selected concepts in a simplified form. This concept is further described in the Detailed Description section. The principles and other steps described in this summary are possible, and any principles or steps are not necessarily required. This summary is not intended to identify key features or essential features of the claimed subject matter, and is not intended to be used as an aid in determining the claimed The scope of the theme. The claimed subject matter is not limited to implementations that solve any or all of the disadvantages noted in any part herein.

2‧‧‧App owner

10a-10d‧‧‧Client devices

11‧‧‧Steps

12‧‧‧ server

13‧‧‧Steps

14‧‧‧Security module

15‧‧‧Steps

16‧‧‧ module

17‧‧‧Steps

18‧‧‧ database

19‧‧‧Steps

21‧‧‧Steps

22‧‧‧ module

23‧‧‧Steps

24‧‧‧ module

25‧‧‧Steps

26‧‧‧UI module

27‧‧‧Steps

28‧‧‧ Presentation module

29‧‧‧Steps

31‧‧‧Steps

32‧‧‧ modules

34‧‧‧ Presentation module

36‧‧‧3-D Modeling Tool Module

38‧‧‧Create/create tools

42‧‧‧Media Tools

44‧‧‧Communication tools

46‧‧‧Social Network Module

48‧‧‧Shopping Cart/Payment Tools

50‧‧‧ system

52‧‧‧Security module

54a-54f‧‧‧Applications

56‧‧‧Security Module

58‧‧‧User Management Module

62‧‧‧ entity management module

64‧‧‧Application Management Module

66‧‧‧Data storage modules, such as databases

68‧‧‧Media Import/Export Module

72‧‧‧Communication Management Module

74‧‧‧Transaction Management Module

75‧‧‧Flowchart

76‧‧‧Open API module

78‧‧‧Various (in some cases, commercial) engines for VR/AR/3D

82a-82f‧‧‧A variety of end users

84a-84f‧‧‧Applications

85‧‧‧Application Engine Layer

88‧‧‧Communication engine

92‧‧‧Social Engine

94‧‧‧Create/create engine layer

96‧‧‧ game engine

98‧‧‧3-D model

100‧‧‧Interface Architecture

102‧‧‧Video files

104‧‧‧3-D model generation engine

106‧‧‧Model

108‧‧‧Video stream file

116‧‧‧ interface

117‧‧‧Environmental layer

118‧‧‧End User Interface

120‧‧‧Thematic Grid

122‧‧‧ interface

124a-124f‧‧‧ indicated as application

125‧‧‧Flowchart

126‧‧‧Location-based service engine

130‧‧‧Target grid

132‧‧‧Location Capture Engine

138‧‧‧Interactive engine layer

140‧‧‧Thematic Grid

142‧‧‧Media layer

144‧‧‧Environmental layer

150‧‧‧Module Description

182‧‧‧Steps

184‧‧ steps

186‧‧‧Steps

188‧‧ steps

192‧‧ steps

194‧‧ steps

196‧‧ steps

198‧‧‧Steps

200‧‧‧ Architecture

202‧‧‧Steps

204‧‧‧Steps

206‧‧‧Steps

208‧‧‧Steps

212‧‧‧Steps

216‧‧‧Steps

218‧‧ steps

222‧‧‧Steps

224‧‧ steps

228‧‧‧Steps

232‧‧‧Steps

234‧‧‧Steps

236‧‧ steps

238‧‧‧Steps

242‧‧‧Steps

244‧‧‧Steps

246‧‧‧Steps

248‧‧‧Steps

252‧‧‧Steps

254‧‧‧Steps

256‧‧‧Steps

258‧‧ steps

262‧‧‧Steps

264‧‧‧Steps

266‧‧‧Steps

268‧‧‧Steps

272‧‧‧Steps

274‧‧‧Steps

276‧‧‧Steps

278‧‧‧Steps

282‧‧‧Steps

284‧‧ steps

286‧‧‧Steps

288‧‧ steps

296‧‧‧Steps

298‧‧‧Steps

300‧‧‧ Flowchart

302‧‧‧Steps

304‧‧‧Steps

306‧‧‧Steps

308‧‧‧Steps

312‧‧ steps

314‧‧‧Steps

316‧‧‧Steps

318‧‧‧Steps

322‧‧‧Steps

324‧‧‧Steps

326‧‧‧Steps

328‧‧‧Steps

332‧‧‧Steps

334‧‧‧Steps

336‧‧ steps

338‧‧‧Steps

342‧‧‧Steps

344‧‧‧Steps

346‧‧‧Steps

348‧‧‧Steps

350‧‧‧ Flowchart

400‧‧‧ Flowchart

422‧‧‧Service entity

424‧‧‧Designer or 3-D Artist

426‧‧‧Manufacturer

428‧‧‧ Game Maker

432‧‧‧ games

434‧‧‧ Gamers

442‧‧‧3-D Artist UI

450‧‧‧Flowchart

456‧‧‧3-D Artist UI

468‧‧‧ Game Maker UI

478‧‧‧Product or service provider

500‧‧‧flow chart

550‧‧‧flow chart

600‧‧‧ Flowchart

700‧‧‧Flowchart

750‧‧‧ method

800‧‧‧ method

850‧‧‧flow chart

900‧‧‧ Figure

950‧‧‧Flowchart

1000‧‧‧flow chart

1050‧‧‧Flowchart

1100‧‧‧ Flowchart

1150‧‧‧ Flowchart

1200‧‧‧ Flowchart

1250‧‧‧Flowchart

1275‧‧‧ Flowchart

1300‧‧‧flow chart

1350‧‧‧ Flowchart

1400‧‧‧flow chart

1450‧‧‧Flowchart

1500‧‧‧flow chart

1550‧‧‧Flowchart

1600‧‧‧flow chart

1700‧‧‧Flowchart

1800‧‧‧flow chart

Figure 1 is a schematic diagram of an implementation in accordance with the present principles

FIG. 2 illustrates an exemplary AR/VR/3D application component in accordance with the principles of the present invention.

Figure 3 illustrates a typical AR/VR/3D application component in accordance with the principles of the present invention.

4 is a flow chart showing an embodiment of a method in accordance with the principles of the present invention.

Figure 5 is a flow diagram of another implementation of a method in accordance with the principles of the invention.

Figure 6 illustrates a subjective and objective comparison of a subject grid relative to a target grid, in accordance with the principles of the present invention.

Figure 7 illustrates a subjective and objective comparison of another subject grid relative to a target grid, in accordance with the principles of the present invention.

Figure 8 illustrates an interface architecture in a modular format, in accordance with the principles of the present invention.

Figure 9 illustrates a system architecture architecture in a modular format in accordance with the principles of the present invention.

Figure 10 illustrates the architecture of a modular format system in an augmented reality (AR) application in accordance with the principles of the present invention.

Figure 11 is a flow chart for creating an avatar, in accordance with the present invention The principle.

Figure 12 is a flow diagram of creating a virtual environment in accordance with the principles of the present invention.

Figure 13 is a flow diagram of a model creation in accordance with the principles of the invention.

Figure 14 is another flow chart for creating a model in accordance with the principles of the present invention.

Figure 15 is a flow diagram of a service model creation in accordance with the principles of the present invention.

Figure 16 illustrates a method that machine learning can be used in creating a three dimensional model, in accordance with the principles of the present invention.

Figure 17 is a flow diagram for creating a trademark for an enterprise in accordance with the principles of the present invention.

18 is a flow diagram of integration with an existing transaction backend system, in accordance with the principles of the present invention.

Figure 19 illustrates an augmented reality user transaction process in accordance with the principles of the present invention.

Figure 20 illustrates a flow diagram of a three dimensional, three dimensional animation or virtual reality product displayed at the front end of a trading website, in accordance with the principles of the present invention.

Figure 21 illustrates a flow diagram of a three dimensional, three dimensional animation or virtual reality product displayed at the back end of a trading website, in accordance with the principles of the present invention.

22 is a flow diagram of a shopper/user interacting with a product in 3-D, VR or AR, in accordance with the principles of the present invention.

Figure 23 is a shopper in 3D, VR or AR Or user interaction, showing a flow chart of team shopping or social shopping, in accordance with the principles of the invention.

Figure 24A is a flow diagram of a fitted fitting in 3-D or VR, in accordance with the principles of the present invention.

Figure 24B is a flow chart of a combined garment AR, in accordance with the principles of the present invention

Figure 25 illustrates a flow chart of the steps of intelligently assisting a user or customer service in real time in 3-D, VR or AR, in accordance with the principles of the present invention.

Figure 26 shows a flow chart of the steps of the subject shopping in 3-D, VR and AR, in accordance with the principles of the present invention.

Figure 27 illustrates another flow diagram of the subject shopping steps in 3-D, VR and AR, in accordance with the principles of the present invention.

Figure 28 illustrates a flow chart of steps in augmented reality shopping in a physical mall or shopping mall or individual store, in accordance with the principles of the present invention.

Figure 29 illustrates a flow chart of the steps of shopping, including AR functions in a physical store, in accordance with the principles of the present invention.

Figure 30 illustrates a flow chart for measuring furniture, home décor, and home appliances into a space or room of 3-D, VR or AR, in accordance with the principles of the present invention.

Figure 31 illustrates a flow diagram of an AR identification and positioning system in accordance with the principles of the present invention.

Figure 32 is a functional flow diagram showing the implementation of real estate transactions in VR, AR or three dimensions, in accordance with the principles of the present invention.

Figure 33 shows a travel application using 3-D or VR functionality Step flow chart in accordance with the principles of the present invention.

Figure 34 shows a flow chart of the steps of a travel application using the AR function, in accordance with the principles of the present invention.

Figure 35 illustrates a flow diagram of an educational/classroom learning step using 3-D or VR functionality in accordance with the principles of the present invention.

Figure 36 shows a flow chart of the educational/classroom learning steps using the functionality of the AR, in accordance with the principles of the present invention.

Figure 37 illustrates a flow diagram of a product/service set in a three dimensional or VR game, in accordance with the principles of the present invention.

Figure 38 is a flow diagram showing the steps displayed in the first user interface in accordance with the integrated flow diagram of Figure 37, in accordance with the principles of the present invention.

Figure 39 is a flow diagram showing the steps displayed in the second user interface in accordance with the integrated flow diagram of Figure 37, in accordance with the principles of the present invention

Figure 40 is a flow diagram showing the steps displayed in the third user interface in accordance with the integrated flow diagram of Figure 37, in accordance with the principles of the present invention.

41 is a flow chart showing an auto-configuration 3-D model in a three-dimensional, VR or AR application that may have attributes to provide additional functionality in accordance with the principles of the present invention.

Similar reference numbers refer to similar components. Components are not drawn to scale unless otherwise stated.

In accordance with the principles of the present invention, the system and method of the present invention relates to the processing of data and the presentation of data in a database, the data being available from a database It is sent to the user interface in a push or pull manner to present a 3-D video or image for viewing by the user. The 3-D video or image can be viewed via, for example, a GPU or video graphics card, or an integrated chipset, or presented on a dedicated device, such as a virtual or augmented reality headset, to allow the viewer to view it as three-dimensional (3D). Perception.

In accordance with the principles of the present invention, systems and methods can be used to allow a user to view the creator's content more efficiently. For example, a 3-D/VR/AR device can enable a viewer, also known as an end user, to view a 3-D world of computer simulations. At present, in the network transaction setting, users use a web browser to view 2-D content. These 2-D contents have poor visual effects for end users, such as online shopping, education, travel, or entertainment. User experience. A three-dimensional virtual reality (VR) system provides viewers with significant technical benefits and allows users to navigate through virtual and/or transactional settings in a very convenient way, saving computer computing cycles and battery power. Wait. Similarly, an AR system can overcome the technical hurdles of various user network transactions to enable end users to receive, process, and display relevant traversing terrain, such as information in real-world locations, and allow users to use a very significant convenience. Ways to access products and information about products. The AR display can be 3-D or 2-D. This system also allows invitations to friends' VR or AR, allowing friends to participate in experiencing VR or AR, a 3-D or VR cloud or platform that allows users to experience 3-D or VR, including transactions in 3D or VR. For example, shopping, real estate buying/selling/renting, education and learning, entertainment experiences, or travel/adventure. Systems and methods are usually available To use VR, but it can also default to 3-D if there is a virtual reality device such as a VR headset that does not exist.

In accordance with the principles of the present invention, a 3-D, VR or AR system typically includes one or more cloud servers with software and databases in conjunction with load balancing and server security. These cloud servers are connected to end users via a network (eg, private or Internet) to one or more desktop computers, laptops, tablets, mobile phones, or any other user device with appropriate processing capabilities. Users can use, or may not need, VR glasses or other devices to enable them to enter virtual or augmented reality environments that typically emulate real or virtual worlds. A glasses is a computer environment in which a device can be connected to an end user, such as a desktop, laptop, tablet or mobile phone, or can be implemented as a stand-alone virtual reality device that emulates a virtual world dedicated to the end user or for the end user. . End users will typically have a computer, tablet or mobile phone, or other device connected to the Internet or a private network. These devices typically download software and communicate with the server. Likewise, the user's VR device can be connected to the end user device or can be used independently.

For example, referring to FIG. 1, system 50 is shown in communication with a server 12 (server i) network among a number of client devices 10A-10D. Server 12 can be associated with a particular type of application, that is, from a particular source, and it should be understood that the client device can access multiple servers and different applications.

Server 12 is shown as having components, including security module 14, Database 18, a module 22 for load balancing, includes modules 16 of the application components of the AR/VR, and includes modules 24 of the AR/VR engine components.

Referring to Figure 2, which is also not an exhaustive list, module 16 is an AR/VR application component that may include user interface module 26 and renderer module 28, and renderer module 28 is in 3-D and/or VR/ The result of graphics processing is presented in the AR environment.

Again, the AR/VR engine component, module 24, is shown with reference to FIG. It includes (an incomplete list): user interface module 32; renderer module 34 for presenting a stereoscopic space, eg, VR/AR; 3D modeling tool module 36; generation/creation tool module 38; media tool module 42; communication The tool module 44; the social network module 46, including an API for communicating with the social network; a shopping cart/payment tool module 48 and a security module 52. These modules will be discussed in more detail below.

Refer to flowchart 75 in FIG. First step, configuring the server to provide a media experience VR/AR/3D, as described herein, the first server is configured or operable to provide a first user interface operable to allow the creator to create or enter a target network Grid (step 11). In this step, the creator creates a target mesh or causes a target mesh to be entered into the system. For example, the target grid can be created with a 3-D modeling program outside the system and imported into the system graphics processing system. The target mesh can then be located in the appropriate location in the environment (step 13). For example, a creator can select multiple locations to place the target grid. Creators can also create various textures or shades on the target mesh, or provide other materials On it. In some cases, the target mesh has no specific scale ratio associated with it, for example, the creator can simply create a scene for viewers to watch. In other cases, however, the target mesh may have a specific scale ratio associated with it, for example, when a target mesh created from one creator needs to be compared to a topic mesh from one user, then two meshes It is very important to have an equivalent scale ratio. For example, if a target mesh is a piece of furniture or clothing, it will also have an associated scale ratio so that the viewer can use the same scale to compare his room, or its exact virtual avatar.

In some cases, the first server, or a second server in network communication with the first server, is configured or operable to provide a second user interface, which in turn allows the first user to create or enter a topic grid (step 15 ), the topic grid is generally compared to the target grid created by the creator. In doing so, the viewer can use the 3-D modeling program to create the theme grid, or more commonly the topic grid can be created for the viewer based on the data entered by the viewer. For example, a viewer can enter the size of a room. The same principle, the theme grid is compared to the target grid configured by the creator, that is, the theme grid is a room and the target grid is one piece located in the room. Furniture in the middle. Wherein, the social or chat function is as follows, the above-mentioned viewer is referred to as the first viewer and the first viewer invites the second viewer to participate in the media experience, that is, the second user interface is configured to allow the first A viewer sends an invitation to the second viewer to share the browsing session, ie, by email, text, text message or the like. In this way, when accepting the invitation to send, the first The first and second viewers can share a computer browsing session, moving together in a virtual environment and interacting with the virtual environment.

The next step is to determine if the topic grid is "fit" to the target grid (step 17). For example, one grid can be compared against another grid and objectively or subjectively determined whether the match is satisfactory. For example, when the theme grid is a room and the target grid is a piece of furniture, the furniture is properly textured to make it look like a viewer's favorite piece of furniture. Again, the furniture can be located in the room and the viewer can evaluate if the results are what they expect. In some cases, the result is binary, for example, yes or no. In other cases, the viewer can change or select the texture, texture and color of the furniture, and try different types, textures, textures and colors of furniture in the room. For example, the user can change a leather sofa to a leather sofa. Other embodiments can also be understood.

In any case, whether the "fit" decision is objective or subjective, after the "fit" process is completed, the user can receive his or her own input information (step 19). In a specific example, the user input can accept the target grid and the topic grid, and the user's results can be selectively converted into files that can be saved (step 21). For example, a user may like the way the sofa is placed in the room, and the user can save a picture or a three-dimensional model of the sofa in the room to show their family and friends. For example, a user can also post the picture or 3D model to a social network via an API.

In another example, the viewer may modify the target or topic grid (or both) (step 23). This is like a viewer modifying a sofa material (Texture/Coloring) to determine if different sofas will appear in the room for a better looking condition. In yet another example, a new topic grid or target grid may be provided by the system, and then the user may perform the process of comparison (step 25). For example, a viewer may try a completely different type of sofa, or you can try the same sofa, but in a different room location or a different room. Other implementation examples can also be understood. For example, it should be understood that the definition of the target and topic grids is arbitrary and the definitions are equally interchangeable.

The implementation of Figure 4 can be applied to VR or AR, but is typically preferably implemented in VR. Figure 5 refers to an implementation in a more specific AR. In flowchart 125 of Figure 5, the first step is to receive or create a topic grid (step 27). This theme grid can be created by the viewer or by the creator. For example, if a viewer wishes to watch a virtualized sofa in an actual house, the viewer can wear an AR headset to evaluate the comparison of the sofa position to the room wall in his own house. The theme grid may have been created by a creator, but was instantiated by a later viewer and compared to the wall of the room. Thus, new or other topic grids can be received or entered by subsequent viewers, or the current topic network can be modified (step 31). The new or added theme grid, or the modified theme grid, is further placed into the comparison environment (step 29). In this way, the viewer can finally visually see how the theme grid fits into the environment of his house.

Figure 6 shows an example of a topic grid 120 compared to a target grid 110, particularly in the context of an object such as a piece of furniture placed in an environment. Figure 7 shows a subject grid 140 compared to the target grid 130. A more recent case. In this case, the target grid 130 is the avatar of the viewer, which is configured to accurately represent the viewer's body. The theme grid 140 is a piece of clothing that is used to compare with the target grid 130, which is typically provided from a clothing provider, and such a grid is typically even available from the manufacturer because the clothing designer typically Design with 3-D modeling tools.

As mentioned above, in some cases, the creator will create a 3D environment regardless of whether the 3D environment will be compared or matched to the theme grid created by the viewer. These three-dimensional environments include theaters, educational applications, transactions, including shopping or retail transactions, and more. These implementations may still include, for example, a social networking component in which a viewer may share a 3D or virtual environment, typically a friend or teacher, with another viewer. It is understandable that some degree of "matching" may occur, but it also means that the viewer's avatar's theme grid matches the same theater seat, classroom seats, lab benches, stores, specialty stores, shopping malls, etc. In these cases, the viewer's avatar can be the size of the appropriate normal size relative to the size of the target grid configured by the creator.

There are several ways to create a target and theme mesh. For example, user data entry, that is, entering the appropriate height and weight, the system can create an avatar of the viewer with the appropriate level of accuracy. Browsers can have a user interface where they can set their avatars or make more precise adjustments. In a more precise way, scanning technology or camera image input can create a better grid, that is, better products, user avatars, buildings, buildings, rooms, shopping centers, shops, Models for storefronts, clothing, jewelry, and the like. Various other implementations can also be provided, including a user selecting a desired grid or portion of the grid from a page menu.

In some cases, for example, makeup needs to be placed on the viewer's body, that is, on the face of the viewer's avatar. Not only does the mesh need to be created when creating makeup, but it also requires texture, material or coloring to attach to the face of the avatar or the part that needs makeup. In this way, the mesh of the avatar can be appropriately changed according to the makeup attached.

In a particular case of building a building, Google's SketchUp or other 3-D building creation tools can be used, including the application of a 360-degree camera, and a 360° camera can create a model inside a room (or outdoor location).

The above implementation will also be understood, for example, a bridesmaid (model is an avatar or theme mesh) can be set to wear a bridesmaid's costume (first target grid) to appear in a church or other location of the wedding (second goal) grid).

In accordance with the principles of the present invention, systems and methods include various components that make it easy for an creator to create an environment. The end user or viewer can compare or match the environment created by the creator or place some theme meshes with the environment created by the creator. In this way, the creator usually includes a developer to create a data file relative to a target mesh using a creation tool. This target mesh is usually a model including texture/material/coloring. Of course, multiple models can be created and applied to an environment. The same way, when a theme grid needs When created, a viewer or an end user can be provided with a tool that they can use to create a theme grid, which is also a data file, which can be a texture/material/coloring model .

In one example, a database already exists that provides data for various items in stock. In accordance with the principles of the present invention, systems and methods can be implemented as "front ends" and implemented on a server, even on a server hosting a database.

In one example, the front end is implemented in a virtual reality (VR) system to provide a VR transaction and other environments. This environment can be implemented as a storefront in which various products are displayed, and users can walk around the store and select items. These items can interact with the viewer, that is, the viewer can place the items in the shopping cart, and some animations can be used to make the user's shopping process more interesting and interesting. However, the implementation of all systems and methods does not require details of storefronts and transactions. The system can simply allow the user to view the product or service item of interest.

When VR or AR is adopted, they can be implemented on some devices such as Oculus Rift, HTC Vive, Sony PlayStation VR, Samsung Gear VR, Google Cardboard, Microsoft Hololens and the like.

In addition to implementing VR, the system can also implement augmented reality or AR to enable viewers to view certain images or CG data in a real-life environment. For example, when a viewer is in a physical store In the middle, an AR can be overlaid on the background of the physical store. AR coverage can also be used in the viewer's own home or office, that is, the viewer can view the placement of virtual objects such as furniture placement, kitchen equipment placement, etc. at home. In another implementation, the same viewing effect can be simply displayed on a screen, that is, displayed in 3D without the use of a VR or AR device.

In addition to entering or creating a target mesh and a selectable topic mesh, other data input into the system includes the creation of a virtual reality environment data, GPS data, which is particularly important in implementing AR, it can be in an environment Navigate its users in a store. Other input data includes data entered by the user on the user interface regarding their avatar or environment. For example, user data entry "Yosemite" can be implemented in a national park of VR or AR.

In a VR or AR device, or in the cloud, processing incoming data is typically done by an additional computer. The output data is provided by a VR or AR screen, and when VR/AR is not used, the output data can be provided by a computer screen.

The described implementations have various technical advantages, including a more intuitive and easier to navigate user interface and environment. Visitors can trade or interact with friends as if they were within a few feet of a friend. These functions are implemented in a computer system with special design and settings, otherwise, as in real-world transactions, it is impossible.

For example, the above described subjectively or objectively determines whether a topic grid and a target grid are suitable for face-to-face shopping for each other. Very useful, that is, a viewer shopping with a friend. In a very useful implementation, a user can use the AR to make the theme grid look at the real physical environment, while his friends, connected to the social engine, can use the virtual reality environment to assist each other. In this case, the VR environment can include a video source so that the AR user can see the VR user.

Advantages of such systems and methods include the fact that systems and methods can reduce the computational cycle of the computer and the energy requirements of the battery to achieve the same user benefits. For example, the system and method can help viewers navigate through a very difficult and confusing environment and help viewers find where the product is located. Viewers can also trade with other viewers. Otherwise, users must use complex video technology and complex and cumbersome instruments to achieve the above user benefits and so on.

The system architecture diagram 100 is a template interface diagram displayed in the form of a module. Various applications 54A-54F are connected to the interface architecture, and various end users 82A-82F are also shown to be connected to the interface architecture. Application 54I involves different environments, etc., which are typically embodied in one or more different servers. However, in one implementation, several environments can be placed in one server or placed in a cloud architecture. Various applications 54I can be embodied, for example, commodity trading, real estate agents/homeowners, travel, hotels or resorts, entertainment, education and social applications.

The end user 82i typically communicates with the framework via a network communication device such as a smartphone, desktop computer, laptop, laptop or other similar device. Interface framework 100 includes a security module 56. A user management module 58, an entity management module 62, an application management module 64, a data storage module such as a database 66, a media load/load module 68, an open API module 76, and one or more Various VR/AR/3D trading (in some cases commercial trading) engines.

As further described in detail, the security module 56 protects the security of the system, which together with the user management module 58 enables the user to securely log into the application. The user management module 58 plays a role in establishing a user account and storing user information in the system. The entity management module 62 provides a management module for various entities, particularly the running application 54I, which manages products and services for merchant users. Data storage module 66 stores application and user data. Access to a database or data store is typically done by accessing executable statements using one or more databases. Media Import/Export 68 allows media to be imported and exported, including target meshes, topic meshes, virtual environments, and grids used to represent projects, as well as models used to import 3-D and 2-D or Picture tools, etc. The communication management module 72 provides network communication functions between a creator server, a server for implementing the systems and methods of the present patented system, an end user system, and a 3-D art creator, etc., for example, the module 72 can provide voice, Text and video communications. The transaction management module 74 provides the user with the ability to perform transactions, such as providing shopping cart, credit card or debit card processing. The open API module 76 provides functionality to access systems and methods based on the present principles through various social networking sites and the like. In addition, the open API allows users to ensure that they are properly and appropriately accepting social feeds or similar feeds to make users available. To promote virtual environments, products, or send invitations to friends to shop together, and so on. The open API further provides developers with an API to build applications on the platform. In this regard, it should be noted that an open API is a publicly available application development interface that provides developers with programmatic access to a proprietary software application. The API is a fixed requirement that governs the communication of one application with another. The VR, AR and 3-D engines 78 provide the functionality to create a virtual environment. For example, the engine 78 provides the creator with an easy tool that the creator can use to easily create a VR/AR/3-D storefront. The engine 78 further provides a simple tool for the viewer, which can be used by the viewer to easily create a viewer's avatar, or a topic grid of the viewer's object, which the viewer can thus The object is seen in the 3D environment created by the creator.

Figure 9 shows a module 150 of a system and method based on the present principles. As can be seen, various interfaces have been provided, such as an application owner 1 (interface 116), which can be a retailer, merchant or other product that provides real-life products and portrays the same product in a virtual environment. store. The owner 2 of an application has an interface 122 that provides an interface for the travel agency, teacher, educator, hotel owner, entertainer, etc. to enter the system. An end user interface 118 can enable an end user or other viewer to enter the desired computer environment and communicate in the environment, that is, the computer exchanges information with the end user's mobile device, such as a mobile phone, and can be downloaded by the user. Install or update software, etc.

The interface between the creator and the end user can be implemented in the environment layer 117, and the environment can be 3-D, VR or AR.

The upper layer is a variety of objects that the viewer or creator can manipulate, including 3-D model 98, video file 102, video stream file 108, or the like. Modules 106 in this layer provide additional 2-D and 3-D image and text objects. Depending on the implementation, the viewer or creator can create the engine 104 using a 3-D model. This engine can be used to create 3D models from text or 2-D images. The various components can be "gamified" by a game engine 96. The game engine also provides the appropriate interactive components for the target grid and theme grid. For example, with a game engine, a collider can be applied to the grid such that a topic grid abuts a target grid rather than passing through it.

Above the game engine is a build engine layer 94. A creator can use this engine to create one or more target meshes that are displayed to the viewer. Similarly, a browser can also use the engine to create a theme grid. A social engine 92 can be used to invite friends to shop together, share some functions, and the like, such as sharing a product to be purchased with a friend, or "buddy shopping." The social engine also enables end users to invite friends to share an experience, such as inviting friends to shop in a virtual world, going to a theater or theater, going to a resort, or enjoying an item together. For example, a user can shop in a mall with friends in a virtual world.

A communication engine 88 can be used by a viewer or creator for voice, video, text or chat communication or communication, and video conferencing functionality can also be provided in this engine. If you need to buy and sell transactions, the trading layer 86 can be used to make a payment from one party to another.

Finally, an application engine layer 85 can be used to perform the required functions so that the lower layer of the layer can communicate with the application, such as a database at the back end, such as a storefront backend database, a travel application or a gaming application. Backend and so on.

Through the system architecture, various applications 84A-84F can enter the system, including shops, shopping malls, houses, apartments, theaters, theaters, games (multiplayer) halls, classrooms, tourist attractions, hotels, adventure parks and Other travel projects, as well as demonstrations (such as how products work, how to assemble products, etc.) and so on.

It is understandable if the designer designs to place these layers in different locations or locations to achieve different goals.

Other components can also be included, such as media displays and video streaming engines to display text, images or videos, and video streams. A component engine can be used to provide a user interface so that a viewer or creator can assemble building modules to build real-world objects, such as a car, a boat or a lab.

In these ways, especially the creation engine, you can place a template or overlay on the game engine such as Unity, Unreal Engine, etc., so that the store owner can create a store, and allows the viewer to move the 3-D product model in the virtual shopping environment. The viewer can also see that the item is animated or presented to a shopping cart.

Figure 10 shows an implementation of a template framework 200 that is specifically biased toward augmented reality applications. Some modules in the framework are the same The modules in Figure 9 are identical and need not be repeated here. Applications at one end of the architecture, labeled as applications 124A-124F, are typically targeted at AR applications, although at some point some resources must be repurposed (reset or reprogrammed) to become VR applications. As part of augmented reality, the user is in a real-world environment, and a location-based service engine 126 can be used to provide the location of the user or item as well as other services. The engine can also interface with a service application engine 127. The service application engine 127 employs data from the location based service engine 126, which also provides the same data to the application 124I. The application 124 can also provide data in the opposite direction, ie, providing data to the service application engine. A location capture engine 132 can be used to obtain location information, such as through graphics recognition, GPS, Wi-Fi, and telemetry.

The social engine and communication engine have been described above in FIG.

Environment layer 144 provides an interface, in particular to provide an interface to the "real world." The environmental interface can be provided from an input such as a lens of a pair of glasses or a camera. A media layer 142 is used to provide visually visible components, such as CG objects, in the AR. The interactive engine layer 138 is above layer 142 so that the viewer can interact with or communicate with other viewers or creators about the objects in the AR display and the real environment in which they are located. In general, a user with a VR device will enter the virtual environment. This virtual environment can be set up as a commercial trading venue, a showroom, or a virtual experience. If there is no VR device, the viewer will usually be in a 3-D environment. In any case, systems and methods are available To enable end users to experience 3-D, VR, 3-D games, or by importing text messages or importing a 2-D/3-D self photo into a cloud or using a system including servers and software The VR system is used to create a viewer avatar. When a viewer imports their 2-D image into the system, the system can automatically create a three-dimensional model for the user. Other potential experiences include a virtual shopping experience, an experience of creating/building objects such as building a car, a boat, a room, a store, a lab or a business. The user can complete the above creation experience with the components provided in the virtual reality environment or the components imported by the user himself.

In general, a user with a VR device will enter a virtual environment that can be set up as a transaction (commercial), showroom, or virtual experience. If there is no VR device, the viewer will generally be in a 3D environment. The systems and methods in both cases can be implemented in the cloud or on servers and software to give end users the experience of 3-D, VR, or 3-D games, as well as users entering text or importing 2 -D/3-D photos to the VR system to create an avatar experience. In this case, if the user enters a 2-D photo, the system can automatically generate a 3-D model of the user for the user. Other potential experiences include a virtual shopping experience, a building/creation experience, whereby users can leverage real-world components or environments such as cars, boats, rooms, offices, stores, labs using components in a virtual environment or self-imported components. Or company, etc.

In one particular implementation, as shown in flowchart 350 of Figure 11, a new user can click on a "new user" button (step 228), which causes the new user to enter a registration/login page (step Step 232). When the registration is completed, the new user, in this case the user is an end user or a viewer, is asked if an avatar is to be created (step 234). In some implementations, the user or viewer can selectively create a new target avatar by entering body information on a user interface (244) (step 236). In this case, the user can import their photos (step 246). In some cases, users can choose different photos, but use the same body as their size. For example, a user can use the size of a movie star's face and use his own body size. Body size usually requires bust, waist, hips, thighs, ankles, sleeve lengths, and sometimes the size of the arm. In some cases, if the user has a garment that is very suitable, he can use the size of the garment as the source of his garment size rather than his own body size. In some cases, scanning a clothing-related number can bring up the clothing information and automatically import it into the system to be used to create an avatar. In any event, depending on the measured size, the system and method in accordance with the present principles will create an avatar or provide an optional avatar display for the user to select.

In addition, the user can select an existing avatar (step 238) or choose to change the existing avatar according to his or her body. In another implementation, the user may import their own avatars, and may alternatively choose to change or edit the avatar according to the size of his or her body (step 242). In any event, the end result is to create a 3-D theme avatar for the viewer so that the viewer can roam in the virtual environment (step 248). When using the size of the body, it is very useful for virtual "fitting", virtual wearing jewelry, and virtual makeup when purchasing clothing.

Users may be able to choose their hairstyle or something similar. The user can further use a 3-D model scanner or camera to get a 3-D image or model, and then import the 3-D model into the system, or the 3-D model scanner can automatically upload 3 -D images to the system or the cloud.

Figure 12 is a flow diagram 300 that the creator can use to conveniently create a 3-D environment for the viewer. At the initial visit, a creator can click on the "new user" button (step 182) and then the system can be registered (step 184). A user interface can be instantiated in which the user is asked if it is a creator (step 186). If the user is not the creator, click the "No" button or a similar button. The user can be redirected to the end user login (step 188) as if the method of Figure 11 were implemented.

If they click on them to be creators, such as merchants, they are redirected to a page where an option to create an environment (such as a store) is displayed (step 192). The system provides a variety of ways to create an environment or a store. When a store or an environment is not pre-established by the creator (step 194), the creator, which may include the business owner, may import their pre-established home or business, or may enable them to be built in a 3-D virtual system. Such a 3-D model (step 198) (by providing an appropriate user interface). In this case, various functions may be provided to the user to edit or personalize a pre-established model such as a store (step 202).

When not using a pre-built store, the designer can Import various 3-D model assets and use them to create a desired storefront or other environment. For example, the step of importing assets includes a 2-D picture (step 204), a 3-D picture (step 206), a 3-D model of a product or similar item (step 216), a video file (step 208), a video original (step 212) and so on. Text can also be used to display or describe the environment. It will be appreciated that such a selection can also be used as a step of editing and personalizing in a pre-established store. Typically, these steps can be provided to the creator so that they can configure or decorate the store to display a 3-D model of their product at a desired location, such as a shelf, so that the viewer can see the product and purchase the product. In implementing this process, the creator usually needs to access a database including a list of available items. When setting up the media experience, such as in a VR/AR environment, a server can provide a user interface that allows the creator to place items in the list of available goods into certain locations in the virtual environment, which the creator can also use. Create or import a 3-D model of the item. A user interface can be opened to allow viewers to enter the virtual environment, ie, the first and second viewers, and further allow the viewers to invite each other to access their virtual environment.

In addition to creating similar items such as buildings, cars, boats or chairs, the creator can imitate the real world with prefabricated components provided in a virtual or 3-D world, such as creating a personalized environment such as a room. Office or similar environment. The creator can also import a 3-D model.

For a certain product (step 222) or service (step 224), the store engine can be built and eventually completed (step 218). For example, these steps can include determining and establishing methods to make the viewers Easily walk or roam in a store or environment, provide software permissions to restrict the location where the viewer can walk or roam, and establish rules or regulations for the user to purchase or walk according to the user's ID or login credentials or the like.

If the environment is just to provide viewers with roaming, enjoyment or entertainment, the environment is not necessarily proportional to the viewer's avatar. The proportion of the environment can be chosen to be mainly to make the viewer feel comfortable, comfortable or comfortable in the virtual environment, such as preventing users from being disgusting. In some cases, the ratio is very important. For example, the viewer enters a virtual furniture store in a network virtual environment. The viewer needs to place the selected sofa in their living room to see if the furniture fits the size of the living room. Items in a 3-D environment, such as furniture, can have a proportion related to their entities, so that when furniture is placed in a virtual environment by a viewer, such as a 3-D model of their living room, the ratio of furniture to living room is It should be the same. This ratio accurately shows the style and location of the furniture in the living room. It should be noted that in this case, the user can use a mouse or other moving method to move the theme mesh (such as a sofa), such as a user can move a theme grid or CGI objects around another topic grid or CGI objects. Instead of causing non-physical visions that traverse each other, the scale can be stored as metadata or other data or methods of the target and/or subject grid.

Systems and methods in accordance with the principles of the present patent can be implemented "on top of" more complex three-dimensional environment creation tools such as Maya, Unity, Unreal Engine, CryEngine, and the like. The creator can quickly and easily create a suitable 3-D environment, including online environments and storefronts. The advantages of this implementation include the fact that the creator does not need to have strong software and modules. Type skills can create a desired environment.

Figure 13 is a flow diagram 400 showing the creation and import of a three dimensional model that can be applied to create most of the described 3-D and/or VR environments. The initial steps are registration and/or login (step 252). The creators are then asked if they want to use their own product or model (step 254). If the answer is yes, the creator can import the picture or 3-D model (step 258), such as importing the 3-D model into the VR engine via the 3-D import engine. This imported 2-D picture can be converted to a 3-D model (step 262), and the imported 3-D model can be used directly in a 3-D or VR environment as a 3-D model (step 264). . The model can be moved by the designer to the desired location (step 266). If the 3-D model represents a product to be sold, a textual description can be entered or imported to describe the model (step 268) such as SKUs, barcodes, QR codes or the like. The model can then be bound to a backend database (step 272) via an API to control and track the merchandise being sold in a sales list system, accounting system or similar system associated with the store. It should be noted that if the creator does not have his own model, the creator can select (or choose to purchase) the model into an online model marketplace for use in its store (step 256).

Figure 14 is a flow chart 450 showing a more detailed 3D product model creation method. Assuming that a creator has a 2-D or 3-D picture (step 274), the user can register and/or log in to a system in accordance with the method of the present patent (step 276). The user can indicate a desire to create a three-dimensional product model (step 278). If the user Representing a willingness to create a new product model, a model of the product can be created (step 282). A 2-D or 3-D product model can be imported (step 286), followed by a 3-D model creation engine to create a model placed in the environment based on the product model created in step 282 and 2-D/ Created from the 3-D product image (step 288). In particular, the 3-D model creation engine can package the created product model in a 2-D or 3-D product image, then texture the model, and finally give one or more colors to achieve the desired product model. .

Figure 15 shows a flow diagram 500 for establishing a service environment. After user registration/login (step 296), the creator will be asked if they want to use their own 3-D service model (step 298). This service model can include animated presentations, simulated images or text descriptions of the services provided, or a similar approach. If the creators do not have their own model, a marketplace can be provided to the creator, who can select a pre-made model in the market (step 302). If the creator has his own model, it is like the model imported in step 304. In this step, the imported may be a 3-D model or a 2-D image. When a 3-D model is imported, the 3-D model can be used directly (step 308). If a 2-D picture is imported, the picture can be transformed into a 3-D model by the 3D model creation engine (step 306). When all the models are created or imported, the designer can move or place the models in the way they want to showcase the services they provide.

Figure 16 shows a flow chart 550 showing a more detailed model for creating a product or service. Especially, when registering and / or boarding After recording (step 314), an image can be entered into the system (step 316). The image can be a 2-D image such as a photo or a 3-D image, such as a set of stereo photos. Machine learning (step 318) can be used to gradually learn the system, further improving its estimation and computation to create a 3-D model.

Machine learning (step 318) can be used to enable the system to gradually learn and improve its estimation and calculation/creation of 3-D models, and machine vision can also be used to evaluate and analyze the depth of the image so that it can be extracted from 2-D images. Create a 3-D model. Related steps include one or more pattern recognition/depth (thickness) calculations (step 322), three-dimensional model generation (step 324), and a final established 3-D model (step 326), the steps including the generated 3 The -D model applies texture and coloration.

Figure 18 shows a flow chart 650 showing a backend system integrated with existing e-commerce. Using this approach, products can be integrated into a 3-D or VR/AR display in accordance with the principles of this patent, and/or product information can be obtained from a database at the back end. For example, a 3-D product model can be entered into the system (step 352). A product ID can be assigned to the 3-D product model (step 354). For example, the product ID can include a SKU, a UPC barcode, an RFID identification, an EAN-13 identification, or other identifier. The 3-D model can then be imported into the appropriate engine (step 356). For example, a 3-D model can be entered into a trading engine, game engine or similar engine. Through an API (step 358), the engine is connected to a database of transactions. The API can be used to transfer product information to or from the engine (step 362). product information Can include a description of the product, price and stock. The engine can use the product information and then display the 3D product model and its virtual environment such as 3-D, VR, AR or similar virtual environment.

Flowchart 700 in Figure 19 shows a template of a viewer's process in the AR. First, the viewer registers/logs in to the system (step 366). At this point, different situations may arise, such as if the viewer finds a product on the website (step 367), he or she can select a location of the store using a user interface (step 368), such as user location tracking. A 3-D store will be displayed on the user's screen (step 373), and when the viewer is in the virtual store, the AR can direct the user to the location of the product (step 374).

At this point, there may be different situations, such as a user interface that can be used to display different colors or sizes of a product (step 370). Likewise, other products similar to the product can also be displayed (step 378). In some implementations, an in-store or online coupon or other promotional activity can be displayed (step 382). Viewers can add products to a virtual product lab. If the product is a piece of clothing, the viewer can add the product to a virtual fitting room (388), and if the size is accurate, the viewer can "try" the clothes with their avatar. The user can then checkout (step 392) and the product can be shipped to the user.

In another case, after step 366, the viewer can find the physical product in the store (step 369). The product code, RFID or other product identification code can be scanned (step 386). One A 3-D form of the product can be displayed in the AR display (step 386). Similar different situations as described above may occur, including activation of different versions, sizes, or colors of display products, activation of similar products, or the use of in-store or online coupons or promotions.

The viewer can try the product and/or use the avatar of the viewer to try the product. The viewer can purchase the product in the usual way, or can complete the transaction online and then take the product home or mail it to the viewer's address.

The implementation of AR provides consumers with a variety of benefits. Not only that, AR also offers many advantages to computers, such as when users can quickly find the desired item, as well as the location of the item or the location it is looking for. Efficient use of existing technology to save computer computing cycles and battery power.

Consumers in stores often find it difficult to find what they need in a large shopping mall or supermarket or department store. This situation has caused consumers to flock to online shopping because online shopping has a search engine. Retailers are also very difficult to introduce more services with shoppers in the store. In systems and methods in accordance with the principles of this patent, one or more of these problems can be solved. For example, systems and methods in accordance with the principles of this patent include hardware and software to provide multi-platform (mobile and desktop) and VR platforms to simulate real-world shopping malls or stores or to provide virtual shopping malls or stores, giving users a near real world The shopping experience, such as walking into a shopping mall or store, is received by sales people, watching product sales, testing products, watching product demonstrations and promotions. If the user does not have a VR device, the 3-D environment is the default state of the system and method.

Augmented reality AR implementations can be provided to shoppers in the store, thereby enabling shoppers to enter a three-dimensional virtual store that is a simulation of a precise physical store. Virtual stores allow shoppers to navigate through shopping centers or stores, helping users find goods quickly. The AR system can determine the location of the user based on GPS or other in-store or shopping center sensors, the Internet of Things or other technologies such as wireless networking, infrared, telemetry, usage and wearable tracking. The system thus allows users to search for products or product categories and to show the geographic location and path of products and categories. The shopper can then follow the displayed path to the location of the product or product category. The system can further provide viewers with discounts or other promotions or recommendations of interest to product shoppers. When the viewer finds the product (or before finding the product), she or he can invite their friend to the 3-D store to share the product. The way to invite friends can be text, voice, video, text messages, etc. These methods can be transmitted through the social engine. A shopper's friend can join a 3-D store and watch the same item with the shopper. For example, a shopper's friend can use a virtual reality helmet or earphone to see the same site as the shopper's AR device, but with different commanding heights. In this case, the physical scene displayed in the AR can be displayed in the VR by a 3-D model, or a computer generated description, or a video source. When using video, the viewer's location is the video source, the shopper's friend sees the same commanding height as the shopper, or a slightly different vantage point, simulating the shopper's friends standing with the shopper.

The AR system can be implemented on a separate system, or On the system coordinated with VR. The AR system can be implemented on 2-D or 3-D. For example, when bandwidth is limited, without a 3-D transaction system, the AR or VR can be implemented as a 2-D application. Typically, when implementing an AR, two additional engines, a location engine to determine the location of the user, and a service engine that provides user services based on the location of the user, will be provided. When not using VR, an AR system will typically only include the location engine and the service engine.

The viewer can have several options to display the item to their friends. For example, if the item is in a 3-D/VR store, viewers and their friends can see the item in the online 3-D/VR virtual environment and can discuss it in the system. If the item is not in an online 3-D/VR environment, or if the viewer expects a 'try' product, the viewer can use a 3-D scanner or a 3-D model to capture the booth to scan the product, either on its own, or on its own. The product is then automatically imported into the 3-D to share with friends.

The platform/system can provide a variety of features and features that have many benefits for both in-store and online shoppers, increasing online shopping and in-store consumer confidence, bridging online shopping and in-store shopping. difference.

This platform technology has multiple functions and advantages. First, the system can simulate a city or supermarket, or store for online shoppers. Online shoppers can see products and services in a virtual environment. For example, a viewer can walk into a virtual street and walk into a 3-D/VR mall or store through their mobile or desktop with VR or AR Ready. The viewer can enter a store and be greeted by a salesperson or watch a model. Viewers can enter a demo room or demo to watch a product demo or product promotion, watch or try a product. Visitors can walk into a theater to watch product promotions. You can watch a movie in the cinema. You can walk into a classroom to learn. Services such as travel products can be purchased. Web or in-store viewers can invite friends or relatives to a virtual store to see a product in real time, helping shoppers make purchase decisions. No matter how far the shoppers and their friends are, they can gather in the virtual store. Visitors can create their own virtual rooms or houses, and it is appropriate to place the furniture in the room. They can also put other items such as household items, home decorations, or DIY/home products into the room or house to see if they are suitable. The viewer can virtually attempt to build or assemble the product and, if necessary, determine the steps or complexity of the product assembly prior to purchase. A retailer's creator can build a virtual salesperson avatar to help online buyers. The virtual salesperson customer service is more user friendly than the current online chat customer service. Visitors can enter a 3-D virtual shopping mall or store to find a store or product through a mobile device, or compare prices (in some implementations, no VR/AR devices are required).

Retailers/creators can sign up and choose a pre-made virtual store or create a virtual store of their own, and the system can provide building modules that grab and drop to make this self-creation process very easy. The image/model of the product can be uploaded to the store and placed on the shelf. Stores can be refurbished in any way, as long as there is material/texture/coloring. create You can choose or create a customer service avatar in advance to help online shoppers. The creator/retailer can set up or build a product or service presentation room. They can use an API to deliver network product data. The creator/retailer can automatically convert the 2-D product image into a 3-D product image or model. Systems and methods in accordance with the principles of the present patent can be used in a three-dimensional virtual reality platform, with or without a payment system or component, enabling merchants to display only products or services without selling.

The above implementation provides some aspects of online shopping, and the same process and steps can be used for models created in 3D using AR. Additional implementation details specific to shopping or other applications are described below.

Figure 20 shows a method 750 that is related to 3-D, 3-D animation, or VR to display products and a 2-D trading website, and more specifically, on the front end of a 2-D trading website. 3-D, 3-D animation, or VR to display the product. In the first step, a trading website viewer registers and/or logs in (step 366). In some implementations, the 3-D model can be displayed directly at the front end of the website, or in a product gallery, in place of the regular product image (step 365). In this case, the viewer can click on the 3D model or 3-D animation, or the product name (step 374), and more product details will be displayed (step 376).

In some implementations, the 3-D model is not directly displayed (step 371). In this case, only the product picture or name is displayed, and if activated or can be clicked (step 368), the viewer is directed to a page labeled "View 3D" (step 372). When this After the button is activated, the product model, 3D model or 3-D animation will be displayed (step 376).

Figure 21 shows a method 800 that is similar to the model type in Figure 21, but which is assembled as part of the back end of the website. Again, the first step is a registration/login step, in which case the transaction website is logged with administrative privileges (step 378). A product portion can be activated (step 382) so that the user can select a product or service there. The user can select whether to expect to click to upload a model in a user interface, that is, upload a 3-D model or a 3-D animation, and the like. The user can then select the model to upload (step 386) and then upload the selected model) step 388). When the product or service model is uploaded, users with administrative privileges can see the 3-D product or service model on the user page and the page in the management backend, ie the product model or animation can be displayed on the front end, or on the front end and The back end (step 392). For a 3-D model is a test room model for furniture, home decor, home appliances or clothing, etc. On the management page (usually on the home page) the user can have a button section that allows a webmaster to upload 3-D of the test room The model, or a user-created environment is enabled, so that users can create their own labs in this environment.

In view of the above, it should be understood that certain systems and methods in accordance with the principles of the present patent are generally to a plurality of applications, while others are directed to particular applications. The implementations described above are often common to multiple applications, while the following sections generally describe specific application specific details.

Systems and methods used in accordance with the principles of the present invention, users You can import 3-D models of their prefabricated home or office, or build multiple types of buildings in a 3-D or VR environment, whether the product or service is being displayed, or being sold, such as commercial trading entertainment. For example, a business can create a virtual environment that represents their business to showcase their products and services. The company can build a virtual environment representing a showroom and office. Enterprises or users can build objects such as cars, boats and chairs. Creators and viewers can also create a personal home page or room or office space, or use a pre-made 3D building component in a 3-D or VR environment, or build or import components themselves to build any environment to simulate the real world.

The creator can have several ways to build the environment. A method of entering the size of a home or office allows the system to automatically generate a 3-D model of the home or office. The creator can also enter the size of the window or door, and the system can automatically generate doors and windows. In this way, the creator can place windows or doors (or other equipment) in their virtual room or office. In the second method, the creator can import/upload a pre-made 3-D model of his home or office. In a third way, the creator can import photos of their real-world home or office into the system, and a 3-D model creation engine as described above will automatically convert the image into a three-dimensional model. In some implementations, the creator may have the option of an edited 3-D model. In the fourth approach, the creator can create an ancillary environment in the system using the building components provided in the system or building components from external imports. The system offers users a variety of customization options, including allowing the creator to choose colors, styles, and more.

Enterprise users can log in as a business owner or entity, can create an avatar as an end user or build a business in 3-D, VR or AR. Just like an end user, enterprise users can import prefabricated 3-D models, including Stores, products, models/animations to show the services offered, etc., to build your own business. Business owners can also use a photo of their existing business to import a 3-D model into a 3D model generator or a 3-D model creation engine. As before, the user has the option to edit the 3-D model. The business owner can also use the 3D image of his business, and the 3D model generator can generate a 3-D model of this 3D image. For a better 3-D model, multiple images may be required. In other implementations, a 3D camera can be used to further enhance the 3-D model of the enterprise. In other implementations, a business or store may be built from building components in a virtual environment, or from an external building software program. As before, business owners can choose styles, colors and trademarks. Enterprise users, that is, company creators, can build 3-D or VR product companies, such as stores, shopping malls, or other types of businesses, such as restaurants, travel agencies, theaters, and games. There are various purposes including sales, promotional services, sales, to provide a user experience, and so on. For example, a restaurant owner may offer a virtual experience in their restaurant, including providing food, even if the viewer is not actually in the restaurant, but can fully experience the restaurant. For companies that require customers to assemble products in the real world, they can display packages of products in a virtual reality environment, and customers can have the experience of opening packages and assembling and disassembling products.

This description generally uses terms such as "browser" and "creator", although in some cases both entities create a grid that will be objectively or subjectively compared, each created or provided A grid.

In general, the browser or end user functions include: registering and uploading a 3-D model, or pre-selecting an existing avatar, the user can choose to enter his or her body size to adjust or create an avatar; the user avatar can enter a virtual environment. And walking around in its environment; avatars can also view or evaluate products or services, talk to avatars of virtual customer services, watch product demonstrations and test products, etc.; user avatars can also select products and display features and functions of the product; Product and payment; invite friends to shop with social network or engine, chat or talk with friends on social network in virtual network environment; open a product package, assemble or disassemble the product; create a virtual room/house, Simulate your own room/house, place furniture or other equipment in a virtual room/house for testing; decorate a virtual room or house; try or inspect the product before purchasing; try these products before purchasing clothes or other personal items.

In general, the merchant or creator features include: registering and selecting a pre-built store, or creating a new store; remodeling a pre-built or new store; importing a 2-D/3-D product image or model store, and placing the product on a shelf And decorating the store; pre-selecting or creating an avatar of customer service to help the customer; setting up or creating a showroom for the product or service; providing or using an API to transfer the product data on the line to other entities in the transaction chain; Convert 2-D product images to 3D product models; display product performance Show video; establish a product presentation room or product demonstration stand; provide a presentation to the viewer; create a product assembly or disassembly instruction video, for example, using 2-D or 3-D, using the product component module to show the user how to assemble or disassemble a product Create 2-D, 3-D, or VR product repair instructions to show users how to repair products; create stores with themes (as described in more detail below); import/create/set up avatars for sales or customer service; Display services such as restaurants, travel agencies, etc.

Now introduce various examples.

example

Online Shopping

The current online shopping experience is not ideal. The rate of return on items purchased online is high because users cannot test products before purchase, such as items in furniture or houses that cannot be determined to determine whether they are suitable for a particular location in a house or apartment, for assembly or For assembled products, users cannot determine the difficulty of assembly or assembly before purchasing.

There may be other reasons for the current unsatisfactory shopping experience. For example, in real life, it is very difficult to find a common leisure time to organize a group of friends to shop together. However, this mutual shopping experience is very much expected by consumers because a person's friends can help the shopper decide whether the item being purchased is worthwhile, or whether a product "sees good" on the shopper's body. The experience of “shopping around the mall” is generally a very enjoyable process for many shoppers. Although online shopping is more convenient, online shopping is generally a very isolated and lonely experience, which lacks the social nature of going to the mall with friends.

In addition, many online shopping sites are not intuitive to the user, and finding a particular desired item can be very difficult. Products that are typically searched in search engines typically produce thousands of results, and consumers are very difficult to choose which product to buy. In addition, for online retailers, launching new products is very difficult in a networked environment. If a new product is placed in a high-profile location, it will be better to promote new products.

Thus, in systems and methods in accordance with the principles of this patent, many or all of the requirements are set forth above. While having the convenience of online shopping, it also provides the function of friends shopping with each other. Visitors can invite friends to shop in a store or mall, or socialize in a cafe, or chat, or watch a movie together, just like in the real world. These viewers can communicate or talk via text, voice, video, or various meeting options. The disadvantage of not being able to try on clothes or other personal items in online shopping can be improved by graphically processing a theme grid relative to a target grid. Similar advantages are from testing whether a piece of furniture is suitable for the room with the viewer's home, trying on jewelry, trying on makeup, and so on. For example, a viewer can place furniture in the model of the viewer's three-dimensional living room, not only whether the size of the furniture is suitable for the living room, but also whether the appearance of the furniture such as color can be matched with the living room. The viewer, that is, the customer can virtually experience a product and service, thereby improving the customer's satisfaction with the purchase, and the possibility of converting from the customer's satisfaction to the purchase. In some cases, the service can be combined with the purchased features. For example, an interior designer can virtually “walk through” a room, suggesting which items and furniture are available. To be used to decorate the room. Users can virtually change the color or wallpaper of a room. In another example of shopping, a customer service representative, whether real or AI, can be implemented as an avatar in a virtual environment. The avatar can teach viewers how to dress in a particular way, how to make up, what jewelry May be interested and so on. The viewer can then purchase these items.

The benefits of such systems and methods in accordance with the principles of this patent are multiplied. For example, in virtual reality, items look particularly attractive, especially in the context of the reality of the avatar and the reality of the viewer. Consumers are more likely to get product education through virtual presentations or showrooms. Customers can “try” the product without breaking the product, and the customer can easily accept the loading and unloading training by virtually assembling or disassembling the product.

In general, systems and methods in accordance with the principles of the present invention can be applied to 3-D, VR, and AR experiences. In some cases, only one target mesh will be used, and a user's target mesh (constituting an avatar) may also be used. Such experiences include learning experiences, such as classrooms, labs, adventures, movies, or performances of life. The shopping or browsing experience may include a comparison of the creator's target grid to the viewer's body grid, such as generally including purchasing personal items such as clothes, shoes, bags, electronic devices, and the like. When the product to be purchased can be "tested" or placed in a "trial room", the target grid is required. Other examples include trying to assemble a product in VR and determining the difficulty of assembly. In a particularly enhanced implementation, component experience can be combined with tactile or other feedback (It can also be applied to other implementations) to enhance the user's experience.

Storefronts can be created and changed by creators such as shopkeepers and retailers. A viewer generally does not change the storefront, but can use certain animations or images to allow viewers to see items in a shopping cart, and so on.

When buying furniture, models of furniture (or any product) have been created that can be loaded while the viewer is shopping. For example, in an AR implementation, if a viewer wants to see how a sofa is displayed in their living room, the viewer may have loaded the 3-D model of the living room into their system, or the same from the cloud. The visit was obtained. For example, the creation of such a 3-D model can be accomplished by the methods described, including taking a 2-D and 3-D image, using a 3-D model generator or the like. The viewer can then scan the bar code or similar code of the sofa, and a 3-D model of the sofa will appear in their living room or at a table, ie, with an AR device such as Google Glass or Microsoft Hololens. Of course, in an AR, the actual couch can be located within a (virtual) image or model of the viewer's room. The scale can be stored in each grid, the living room and the sofa, so that when the sofa appears in the living room, the scale can be set correctly. In addition, the 3-D model of the living room can be obtained from the cloud, but the sofa in the display room as seen by the AR device is displayed, in particular, in the viewer's living room. Similar applications are in the VR environment and so on. It should be understood that various different implementation variations described above are within the scope of the systems and methods in accordance with the principles of the present invention. For example, a viewer may employ such systems and methods to test and select office furniture, store furniture, real The layout of the laboratory, the layout of the clean room, and so on.

In apparel applications, as described above, a "try-on" garment can be virtually virtualized using an appropriately sized avatar. Grids for clothing can be easily obtained from manufacturers, and garments are often used by clothing manufacturers to design garments using 3-D modeling software. In addition to clothing, this system can be used to try on shoes, cosmetics, hair, jewelry, cosmetics and handbags. In any case, the selected item can be placed in a shopping cart and the viewer/user can make payments through various online payment systems.

In some implementations, when setting up a mall, for the mall owner, if it is economical, the mall space can be used to rent to the store owner. A particularly good location may be close to the main store or entertainment venue. For example, a virtual mall may include a virtual game arcade where viewers can go to the game arcade to play games, play games with individuals or with others, or viewers can watch movies with other viewers in the movie theater.

The use of mall layouts provides significant technical advantages as a viewer or user can travel from one virtual storefront to another. Previously, users had to open a new browser window and navigate to the second virtual storefront, and the response at the time would be a heavy 2D experience, far less than the experience of a VR mall. However, in systems and methods in accordance with the present principles, the viewer or user saves considerable effort, thereby eliminating keystrokes, mouse clicks, and computational cycles and battery power (for battery powered devices).

In some implementations, the creator (or the creator's AI representative) can communicate with the customer in a virtual reality environment, either directly or through a grid. The customer service avatar provides assistance, and the service avatar is imported or created by the creator for customer service purposes. Merchants can communicate with customers in 3-D or VR or AR to answer sales and customer support questions.

The creator can provide a shopping experience with a theme. For example, a merchant can create an online environment suitable for their own products. For example, a merchant selling sports equipment may build a store with a track, a basketball court, a ski mountain, a surf beach, etc., so that customers can try (virtual) them. Sports products. A merchant selling Italian products can build a store in a simulated Italian background/city like Rome.

In a template implementation, the user interacts with the product in 3-D, VR or AR, where the user can be a shopper, and the following steps can be made, as seen in flowchart 1150 in FIG.

In a VR implementation, a user in a 2-D, 3-D, or VR online environment or game may enter a store or other product-enabled environment, such as a museum, classroom, etc. (step 502). The viewer can select a product of interest (step 504). Buttons or handles can also be attached to the product to allow the viewer to interact with the product, for example, with appropriate clicks, drags, or activations.

In an AR implementation, the viewer may be in a physical store (step 508) and may use, for example, a mobile device to capture information about a product of interest (step 512), such as a SKU or barcode, QR code. Information or the like. The product indication is displayed on the mobile device (step 514) and a button or handle is again provided to allow for a more convenient user interaction with the product.

In VR or AR, users or viewers can interact with products in a variety of ways. For example, the product itself can be interacted to view the internal structure or components of the product (step 516). For example, an exploded view can be provided. An animation can be provided to illustrate how to use the product (step 518). An animation can also be provided to demonstrate various product features (step 522).

As an example in animation, a step-by-step animation can be provided to illustrate how to assemble or disassemble a product (step 524). In any of the aspects depicted, for example, product interactions or animations can be added to a list of favorites, a shopping cart, a wish list, or a customized space or room in a 3-D model (step 526). The user can click on a product information button to get more information (step 528), for example, a more detailed description, price information, and inventory information, and the like.

A flowchart 1200 of a method in accordance with the principles of the present invention is shown in FIG. 23, for example, particularly for interaction between a shopper and a user in a 3-D, VR or AR group purchase or social shopping. In the first step, the user enters an online environment, for example, it can be 2-D, 3-D, VR, mall, etc. (step 534). The user can invite others to join in various ways (step 536).

Additionally, the user can be in a physical store or mall (step 538). The location of the user can be determined in various ways, such as global positioning system GPS, Wi-Fi over a wireless network, pattern or graphical recognition in an AR system, and the like (step 542). The user can then invite other users to join their AR session (step 544). Friend or home After receiving the invitation, the person can accept the invitation (step 546). The friend or family member, typically employing a VR device, can join and appear in the same store or mall as the inviter and in the same location (step 548). A friend or family member can join the shopping (step 552) process and can communicate with the inviter using voice, text or text messages, videos, etc. (step 554).

Figure 24A shows a flow chart 1250 of a method in accordance with the principles of the present invention. Again, the user enters one of the above stores (step 556), although in this case it is generally preferred to be in a VR or three dimensional environment. The viewer can select the desired garment (step 558) and may express the desire to try on the garment (step 562) by clicking or activating the button or other indication method. The system checks the user to determine if the user is registered (step 564). If not, as described above (step 566), the user can register and enter their body size as well as an avatar or other image. Subsequently, if the user has already registered, their avatar can be displayed (step 568). The item selected by the user can be presented to the user, or the user can re-select again (step 572), and the system can match the size of the garment according to the size of the user's body. An algorithm can be used to determine the type of garment fabric given, the stretch of the fabric, the size, etc., whether the garment is suitable for the user within a predetermined tolerance such as 5%, or 10%, etc. analogy. Users can enter preferences or user settings, whether they like loose clothes or tight clothes, and so on. If there is a match, the garment will be displayed on the user's body (step 578). If the fit is determined to be bad, for example, if the algorithm determines that the garment is not a good match, such as 20% of the ideal size set, the system will tell the user to select it. His costume (step 584).

If a match is finally achieved, the user will be given the option to see information about the garment, such as price, size, color option details, etc. (step 582). The user can then share a photo with their friends (step 586), invite the friend to view the fitting through the social engine (step 588), and/or can add clothing to the shopping cart, favorites list, wish list, etc. (steps 592).

In another implementation, the garment can be selected in the AR. This implementation is illustrated by flow chart 1275 of Figure 24B. Some details are the same as in Fig. 24A, and their description will not be repeated here. As shown in FIG. 24B, when a user is presented with a garment, the user can use the mobile device to scan or otherwise capture information about the garment (step 594), and the 3-D model information of the garment item can be displayed. On the mobile device (step 596). The remainder of the implementation is similar to Figure 24A, although in this case, the user can also try on the clothes in the physical store.

Figure 25 shows a flow diagram 1300 of using a real-time smart assistant in an online virtual environment, for example, in 3-D, VR or AR. In the first step, the user enters a virtual 2-D, 3-D, or VR store or mall or game, and if in the implementation of the AR, the user enters the physical store (step 602). At some point, the viewer can request sales or customer support via an appropriate button, text/text message, voice indicator, video indicator, etc. (step 604). Sales or customer support associated with the creator (retailer) can schedule sales or customer avatars, and the number of scheduled sales or customer avatars is based on the number of user requests. Determined by quantity. The sales or customer avatar near the requesting user may be invoked first (step 606). After the sales or customer service avatars greet, the user will use various means (step 608) to ask questions. Sales or customer avatars can receive questions via speech recognition, text/sms, videos, and more. If the avatar is manipulated by a real person, the avatar can provide various answers (step 618). If the avatar is purely virtual, for each question (step 614), the avatar program can search a database to find the corresponding answer (step 614). If a match is found, the avatar provides the answer to the viewer, or the user can be enumerated with several possible answers (step 622). The user then selects an answer or asks for further assistance (step 624). If extra help is needed, a real person can answer the question by communicating with the viewer in various ways, such as text/sms, voice, video, etc. through the avatar. The system can then terminate or provide a user survey to the viewer (step 628).

As mentioned above, theme shopping can be provided in 3-D, VR or AR. In this implementation, the user can be provided with a 3-D or VR store by theme of the product. For example, a store or mall can be placed in a virtual alpine ski resort, and luxury Italian products can be placed in a virtual boutique in Rome. In some implementations, a motion area can be provided to allow a viewer to try, test, or use a product, such as a field or area. In more complex implementations, ski shops can be virtually built in ski resorts, swimming pools built in swimwear stores, and the like. Users can enable capturing images and share their experience testing products in the theme store.

FIG. 26 illustrates a theme shopping flow diagram 1350. For example, a business user may select a store location/address (step 632). The store owner may be given an option to create a theme store (step 634). If the answer is no, the store owner or creator can proceed with the regular store process, which has been described elsewhere (step 636).

Whether it is a theme store or not, the creator can be given a choice to create a product test area (step 642). If the answer is no, a conventional store model as described elsewhere can be created (step 644). If the answer is yes, a test area, generally by subject, can be selected, created, or uploaded (step 646).

Figure 27 shows a flow chart 1400 for theme shopping from the perspective of the user. The user enters the theme of the shopping store, center, scene, game and mall, etc. (step 648). The user can, for example, enter a keyword for searching for a product (step 652), and in some cases, the user will select a theme store (step 654). The user can then try, test, or manipulate the products in the theme store, if provided, in the theme store or test area (step 656).

Additionally, if the user is in a physical store (step 658), the mobile device or the like can obtain the ID of the product (step 660) that is of interest to the user. An AR application may display the product in a theme virtual store, or in a test area (step 662), whether the user is in a physical store, the user may try, test or fiddle (step 664) the product in the theme store or trial area. The product can be added to a shopping cart, wish list, or shared (step 666). The topic can then It is stored the next time or shared in the social network (step 668). In this way, the user can be allowed to shop in the desired environment, but it is possible to shop in the same environment in each store of the store. In the final step, if the user decides to purchase the product from the theme environment, a payment step is performed (step 670).

Figure 28 is a flow chart 1450 of shopping AR specifically to a mall or shopping mall. In a first step, the user enters a physical mall or shopping mall with a mobile device such as an iPhone, iPad or wearable device (step 672). The AR application on the mobile device can identify the location of the user in various ways, such as pattern recognition, target recognition, scan flag, code, image, or AR ID, etc. (step 674). Additionally, the application can determine the location of the user by locating GPS and Wi-Fi or other positioning system (step 678).

In the first case, the AR application may identify the entire shopping mall (step 682), or identify a single store (step 684), an event or activity (step 688), and the like. After identifying the entire shopping mall, the user mobile device (step 688) will display the main promotions, events, events or announcements about the shopping center. If the identification is a single store, in-store promotions, or related to various event/activity announcements in the store, etc., may be displayed on the mobile device (step 692). If the identification is an activity/event, for example, via a QR code, barcode or the like, the event or activity information can be displayed on the screen of the mobile device (step 694).

The user can then click or select a promotion. Event or activity (step 696). An arrow or symbol on the mobile device will appear to direct the user to the location or location of the selected promotion, event, or activity (step 698).

Other specific AR applications are also understood (step 702).

In another implementation, the user may also search for products, events or events at the mall or shopping mall (step 676).

Flowchart 1500 in Figure 29 illustrates the implementation of a physical store using AR shopping. The mobile user enters a physical store (step 704). If the product is found by the user on the web, then the user can select a particular store/location (step 706). The 3-D model of the store may appear on the user's mobile device (step 708) and the user may be directed to the product portion within the virtual store (step 710). As noted above, different specifications or colors of the product may be displayed (step 712), similar products may be displayed (step 714), and online coupons and promotions may also be displayed to the user (step 716) and may be activated by the user. And use, and so on.

When the user finds a product in the store, the product code or the like can be scanned (step 718) and the product is displayed on the screen of the user's mobile device (step 719).

In any event, the product may be tried in a virtual or physical store (step 720) and a payment process may be provided if necessary (step 722).

Flowchart 1550 in Figure 30 illustrates another shopping implementation, but where the product relates to furniture, home décor, or equipment, and These products are tested in an environment in 3-D, virtual reality, AR, or the like. In the first step, the user enters a store or game in VR or 3-D (step 724). Additionally, the user may be a mobile device with an AR application (step 732) in a physical store (step 730) that may capture the store or product.

The application may ask the user if they have a space or room that they have customized (step 726). If not, the user may be prompted to create a new space or room with the desired size and/or shape (step 728). If the user already has a custom space/room, the space/room can also be downloaded or imported from the cloud or similar. The product can be moved into or out of the room or space subscribed by the viewer (step 734). The user may take other steps including moving, rotating or otherwise adapting the product to the customized space or room, as well as other items in the space or room. As mentioned above, the product can have a scale ratio that can be set, and both the viewer and the creator need to use the ratio. If the creator uses this scale to create a target mesh, the viewer can create a topic mesh with that scale. Thus, the product can be displayed in a 3-D, VR or AR environment correctly representing the true product size.

The user can click on the product to learn more product information (step 738). The space in which the product is placed and the product itself, can be captured as an image or 3-D model (step 740), shared with friends or family (step 742), added to various lists, including shopping carts, wish lists (step 744) in. As before, the user can pay for the purchase (step 746).

In various applications, including shopping and other applications, systems and methods in accordance with the principles of the present invention can implement an AR identification and location system that can be used to identify a location, object or product for display in an AR. In particular, current AR applications typically primarily determine location via GPS. However, GPS is often not accurate enough, especially where it is determined to be very close. Currently, there is no way to determine the location and identification of an object or product, so it is very difficult to use AR in shopping, events, activities, or other applications.

However, systems and methods in accordance with the present principles include positioning using wireless technologies (eg, Bluetooth, wireless network Wi-Fi, infrared, or mobile networks). Scanning techniques can also be used, for example with associated barcodes, QR codes, RFID, or specially designed images, patterns or numbers. If wireless technology is employed, the location or object has a means to transmit a wireless signal. The user's mobile device can automatically detect the wireless signal so that the location or object can be identified. If scanning techniques are employed, the location or object can be equipped with a barcode, QR code, RFID or other specially designed image, pattern, or code to facilitate scanning on the user's mobile device. Once the information is scanned into the AR system, the location and objects can be identified.

Flowchart 1600 in Figure 31 shows an AR identification and positioning system that can be used to identify a location, object, or product displayed in the AR. In a first step, the user uses their mobile device and AR device or system around the object (step 748). The user's mobile device can use wireless signals to obtain location (step 754), or can use scanning for various visions The image obtains location data (step 756). The various locations or objects recognizable by the AR application (step 750), and it may also access one or more databases or files to extract location information (step 752).

real estate

In the real estate or hotel industry, an owner or agent can load an image/3-D model of the real estate or hotel as well as the size and variety of requirements. If the owner or agent does not have a 3-D model or expects an interior design, an interior designer or 3-D artist can be hired to create or design a 3-D model. Interior designers or 3-D artists can receive information on the house or hotel and perform interior design according to the owner's requirements. The interior designer/3-D artist can then send the completed design to the owner or agent. If the design is approved, the owner can pay for the designer or 3-D artist. The property or hotel can be displayed for viewing or purchase for users in VR or 3-D, such as online or by other means.

In particular, reference is made to flowchart 850 of FIG. In a first step, an owner/agent logs into a suitable user interface (step 394) and the 3-D model or image can then be uploaded to the system or build (step 396). The owner/agent needs to decide if an interior designer or other 3-D artist is needed (step 318). If desired, a user interface is provided to the interior designer/three-dimensional artist (step 402). The interior designer/three-dimensional artist can receive or download the image/3-D model, as well as the size and requirements (step 404). The interior designer or 3-D artist then constructs (step 406) the 3-D model of the house or hotel.

For furniture or home décor, existing product models are also available To use (step 408), or to use a new model or upload model (step 412). The house or hotel can then use furniture and home décor (step 414). A payment engine (step 416) can be employed to enable the interior designer or three-dimensional artist to receive payment (step 418).

In one implementation, when the owner sells the house, he or she can upload an image of the house of the appropriate size so that the interior designer or 3-D artist can make a model of the house or room and then decorate the house with furniture and home décor products. Or the room, potential buyers can compare the original state of the property with a newly renovated property in 3-D or VR. This is especially useful if the original property is in a poor state, and potential buyers can see the potential of the property through 3-D or VR, thereby increasing confidence in buying the property. Buyers can also purchase furniture or home décor products in or out of the property through an interior designer or online transaction in the 3-D or VR described.

Owners can also use 3-D cameras to get a 3-D model of a house or room, or get a blueprint of the house from a government record in their city, so that an interior designer or 3-D artist can build a blueprint-based 3- D model.

User experiences related to real estate include real estate purchases and rentals, as well as roaming and exploring real estate.

It will be appreciated that there will be some implementation changes, for example, although various home purchase implementations are described above, it should be understood that the user may walk into an apartment or other leased virtual room model. With AR, users can be in a virtual house or apartment that they are interested in. See your own furniture or fixtures.

In a typical implementation, the object model can be targeted to a target model, where the model is usually a grid with appropriate materials, textures, and shading. In any case, the user may be allowed to navigate through the virtual property, or through the AR interface and using significantly enhanced information for the property in the store.

In social applications, friends may be invited to a meeting in VR, or friends may go shopping or otherwise get together, chat, voice, and video conferencing functions can be adopted and utilized to allow friends Communication and communication between us. You can connect to social networks through appropriate APIs, for example, on Facebook, Google+, Weibo, LinkedIn, etc.

The game industry can use this technology to get viewers into VR. Such games include sports events, video game competitions, social events, including political parties, and so on.

Education / classroom learning

In the field of education or classroom learning, referring to Figure 35, in the first step, an educator or other creator can select a classroom or building in 3-D or VR (step 790). Alternatively, a 3-D model of a classroom or school (step 792) can be uploaded. Similarly, classroom desks, laboratory tools, equipment, or decorations can also be selected from the system or uploaded to the classroom (step 796).

In another implementation, the creator or educator can upload an image of the classroom or school (step 794), a 3-D generation engine A three-dimensional model can be created automatically (step 798), and in addition, the 3-D artist can manually create (step 802) the model.

However, after creating the three-dimensional model, the model of the item is movable in the virtual environment and used to lay out the classroom (step 804). The educational material can then be selected and/or uploaded (step 806). Such textbooks include textbooks and other text, videos, 3D models, and so on. A suitable payment system can be in place to compensate the artist and other content creators (step 808).

The student can then enter the virtual classroom and purchase materials (step 810). Students can take other steps including watching videos, interacting with educators, performing projects, blending, assembling or disassembling objects, virtually going to ancient or remote places to experience historical events, watching movies, watching 3-D animations, etc. .

Figure 36 is a flow chart 1800. The flow chart shows the implementation of an education or classroom in augmented reality. In the first step, the teacher and student can enter a classroom with the mobile device (step 812). The system may provide appropriate images for the mobile device to scan (step 814), thereby enabling educators and students to register their presence through the mobile device. Various educational materials can then be displayed on the mobile device (step 816). It should be noted herein that particularly in this and other implementations, mobile devices include laptops, tablets, and smart phones. Similarly, the student can provide (step 818) various data such as selections in the classroom. Such data can also include, for example, reviewing detailed information, taking exams and quizzes, voting on a title, and doing this Discuss the questions, make comments, do experiments, interact with 3D objects, watch 3D animations to learn more about a topic, and get hands-on experience on the topic.

The target grid is generally associated with a virtual environment in this implementation, such as classrooms, labs, etc., and the subject grid is generally associated with a student's avatar or the like. If a communication engine is used, including voice and video conferencing, and the above social engine is used, a significant educational lever can be obtained.

It can be understood that the way of implementation is different. In addition to lectures, labs can be carried out, experiments and other projects or structures can be built or disassembled, and so on. Education or creators can enable the introduction of prefabricated 3-D models, or build a virtual classroom or lab in a network environment, in a network environment, in a cloud, etc., and so on. The course content can be constructed and delivered to the user, ie the student, and the course content can include text, voice, images, video, and the like. Individual or public courses can be set at the system level, at the local network level or in the cloud.

Travel/entertainment

A variety of travel-related applications may include those related to adventure and entertainment. For example, a user may go to a remote location, or may only go to a virtual game arcade to play a game, or watch a movie in a virtual cinema with friends or family. Before traveling, the user can enter a virtual resort or hotel room, for example, to determine if they are willing to travel to an entity and select a physical hotel represented by the virtual hotel. A virtual hotel or resort is configured to showcase the exact hotel or resort.

This virtual travel or adventure experience can make viewers imaginary Places and destinations of interest to the proposed tour, or to destinations where they are unable to travel. Such virtual experience can be used to understand destinations before traveling.

Friends can travel virtual together, and during the trip, they can visit virtual theaters or game halls. It will be appreciated that there are other embodiments, including those using social networks.

In one particular implementation, a travel agent can select or load an image/three-dimensional model of the travel destination. If the image is loaded, the 3D model generation engine can generate a 3-D image or model. The 3D artist industry can create 3-D models and the 3D artist industry can get the appropriate payment if needed. The 3-D model or image of the destination can also be displayed to the user for viewing. Users can also purchase travel packages, etc. in 3-D, VR and AR.

Figure 33 shows a flow chart 1650 in accordance with the principles of the invention. Flowchart 1650 shows a travel application in 3-D or VR. In the first step, the travel agency may select a 3-D model of an existing destination (step 758). In another implementation, or as another aspect of the first implementation, the travel agent or company may upload a 3-D model of the destination (step 760).

In either of these implementations, the user can view or purchase the travel service (step 768), and if necessary (step 774) the payment system can be used.

In another implementation, it can be a further step 758 or 760. Travel agency or company can upload destination image (steps 762). The image can be converted to a 3-D model by a three-dimensional model engine (step 764). In addition, the 3-D model can be constructed manually (step 766) in accordance with the image. The output of the 3D model engine is a model of the travel destination (step 770), which may be included in step 768, if the 3-D model was created by hand, the artist of the 3-D model may use the appropriate UI download or Upload the desired model (step 772). The process can continue with step 770.

Figure 34 shows a flow chart 1700 which is an augmented reality travel application. In a first step, a user with a mobile device arrives at a desired location or destination (step 776). The AR application can determine the location of the user via GPS or other wireless positioning system and technology (step 778). The AR application may have several options for the user to select from, including: location information, accommodation information, dining information, fuel information, vehicle rental information, or other promotional activities (step 780). The user can then select the information of interest to view (step 782). In addition, the user can be directed to the location of the selected option via the Global Positioning System (step 784). The user can also subscribe to the desired options using the AR application (step 786), for example, to reserve the displayed room. As before, the transaction can have a payment (step 788) system if needed.

In an implementation involving entertainment, an entertainment company or individual can build a 3-D or VR theme area such as an adventure park, game arcade, stage or theater. These venues can be built separately in a virtual environment or built into a 3-D or VR mall or shopping mall. Mobile devices Users can get a variety of entertainment options and may try or buy in 3-D or VR. An entertainment theme owner can engage the user by automatically sending an invitation to the viewer to view or play various items. In the case of a VR shopping area or mall, such an invitation may also be provided to a "passing" user or viewer. After the user or viewer has tried the entertainment, the user can pay for the entertainment content from the appropriate payment mechanism if needed.

Different implementation changes can be understood. For example, an image or 3D model of a hotel room or resort can be imported into a 3-D or VR system or cloud. The 3-D model engine used As described above, 2-D images can be converted into 3D models and imported into the system or cloud, making it easier for hotel and resort owners to build model rooms or houses. A 3-D image taken from a 3D camera can also be used for this purpose.

Entertainers can import videos into a 3-D or VR cloud or a server. The video stream can be used to deliver live performances to a VR or 3-D cloud or server. Entertainers can import or build custom 3-D or VR theaters.

Integration of products or services into 3-D or VR environments or gaming applications

Integrating a product or service into a 3-D or virtual reality environment, such as a game or other application, information about a product or service, etc., can be obtained from a manufacturer, retailer, or service provider. If the product or service does not have a 3-D model to represent the product or service, a 3D artist can request a model, or the creator (provider of the product or service) can choose a pre-made model from inventory. Once you get a 3-D model, a product or service The 3-D model can be used by designers or game developers. In the case of a game, the player can see a 3-D model of the product or service, then purchase the product or arrange the service. The game owner can activate the payment system and then transfer the information of the product or service to the owner of the product or service. Thus, each entity, including game owners, game developers, and 3D artists, can be paid.

In one particular implementation, as by flowchart 900 of FIG. Various entities 422 desire an initial step of allowing service transactions within the game to access the appropriate API as service information provided in the virtual environment. In Figure 37, entity 422 generally provides services, while entity 426 typically provides products, but it should be understood that in a given implementation, products and services may be provided by a given single entity. In some cases, the product or service provider may have a 3-D model suitable for the game that can be added directly to the game, and the game manufacturer 428 has an interface III. Another interface, interface I or interface II, can be used by the service or product provider as an interface to connect the designer or three-dimensional artist 424. The designer or 3D artist can create a 3-D model for the product or service in the game and provide it to the game creator 428 via interface III. Once the game maker 428 has obtained the desired model, the game 432 can be created. Subsequently, the game player 434 can play the game and purchase products or services in the game. The purchase order is transferred to product manufacturer 426 or service provider 422, respectively.

Flowchart 950 in Figure 38 shows more detail of interface I. In the first step, a retailer or other service provider has access to The user interface is to configure or access an API of systems and methods in accordance with the principles of the present patent for providing VR, AR, or 3-D functionality (step 436). Product or service information including images is loaded by hand or API (step 438). The designer or 3-D artist user interface 442 can be configured and used to enable the designer or 3-D artist to select a product/service and make a corresponding 3-D model (step 444). The designer or 3-D artist can then upload the 3-D model using its user interface for use in the game (step 448). In the process, the designer or 3-D artist can get paid at a specific time, as shown in the figure.

Figure 39 is a flow chart 1000 illustrating the steps of user interface II. In a first step, the manufacturer accesses or uses a user interface to begin providing product information (step 452). Product information, including 3-D models and images, is then loaded by hand or API (step 454). The designer or 3-D artist can then use the user interface 456 to select a product and build a 3-D model that matches the game engine based on the product and service information (step 458). The 3-D model can then be uploaded to the game factory creator (step 464).

FIG. 40 shows a flow chart 1050 that illustrates the steps of user interface III. In particular, at interface III, the game creator's user interface 468 can be accessed by the game creator and displayed as a 3-D model available in the game, such as on a web page (step 466). The game creator can then download the desired 3-D model (step 472) and integrate the 3-D model into the game (step 474). These models can then be linked to a backend trading system, such as a shopping cart system (step 476). With 3-D Orders for such products or services related to the model can be transmitted to the product or service provider 478, and in some cases, the product or service provider can pay the game producer a referral or advertising fee.

In another different implementation, the 3D model can be automatically configured and programmed. More specifically, the internal 3-D model objects can contain special properties whose properties can be used to add features and functions to the 3-D model so that the user/player can interact interactively using the 3-D model. Attributes can have different types, for example, attributes to determine avatars, products, advertising banners or decorative items, and the like. A computer program can read 3-D model objects and parse their properties, and then assign attribute-based functions to individual 3D model objects. Second, the functionality of the 3D model object can interface with a transaction (eg, commercial) API, with advertisers and other types of internal and external communications, for example, referring to flowchart 1100 in FIG. 41, a system can begin with A 3-D model (step 482), then various attributes can be assigned to the model (step 484). The model can be uploaded to the game engine and the system can then automatically check the model attributes (step 486). Depending on the nature, the system can automatically program the application (step 488) into each 3-D model. In this way, the three-dimensional model can automatically obtain various features and functions (step 492).

The system and method can be fully implemented on any number of computing devices. Generally, the instructions are placed on a computer readable medium, typically non-transitory, and such instructions are sufficient to allow a processor in the computing device to implement the methods of the present invention. The computer readable medium can be a hard disk drive or a solid state with state storage instructions that are added at runtime Loaded into random access memory. Inputs in the application, for example, from multiple users or from any one of the users, can be entered by any number of suitable computer input devices. For example, a user may input relevant computing data using a keyboard, mouse, touch screen, joystick, touchpad, other pointing device, or any other such computer input device, the data may also be by an inserted memory chip, a hard drive, Flash drive, flash, optical media, magnetic media, or any other type of file input. The storage medium output can be delivered to the user by way of a video graphics card or coupled to a display integrated graphics chipset that the user can see. In addition, the printer can also be used to output a hard copy of the results. In view of this description, any number of other tangible outputs can also be understood as contemplated and contemplated by the present invention. For example, the output can be stored on a memory chip, a hard drive, a flash drive, a flash memory, an optical medium, a magnetic medium, or any other type of output. It should also be noted that the present invention can be implemented in any number of different types of computing devices, such as personal computers, laptops, laptops, netbook computers, handheld computers, personal digital assistants, mobile phones, smart phones, A tablet computer that implements a computer and is designed for these purposes. In one implementation, a user with a smartphone or a wireless network connected device downloads an application from the server to their device over a wireless internet connection. Appropriate certification processes and secure transaction processing can be provided to the seller. The app can be downloaded via a mobile connection, a wireless network or other wireless network connection. The user can then run the application. Such a networked system can provide an implementation of a suitable computing environment in which multiple users provide independence. The input to the system and method. In the following systems, creating and using a virtual environment, including target and body meshes, is considered, and multiple inputs can allow multiple users to enter relevant data at the same time.

10a-10d‧‧‧Client devices

12‧‧‧ server

14‧‧‧Security module

16‧‧‧ module

18‧‧‧ database

22‧‧‧ module

24‧‧‧ module

50‧‧‧ system

Claims (17)

A method of configuring a server to provide a media experience, comprising: a. providing a first user interface on a first server operable to allow a creator to create a virtual environment, the virtual environment including at least one target grid b. providing, on the first server, or on a second server communicating with the first server over the network, a second user interface operable to allow the first viewer to log into the virtual environment and The virtual environment interacts with the virtual environment; c. wherein the second user interface is further configured to enable the first viewer to send an invitation to the second viewer to share the viewing session; d. The second viewer is enabled to share the viewing session and can move in the virtual environment, interacting with the first viewer in the virtual environment. The method of claim 1, wherein in the method on the first server or on the second server, the second user interface is further operable to enable the first viewer to construct a theme usable in the virtual environment grid. The method of claim 2, wherein the subject grid is an avatar. The method of claim 2, wherein the subject grid is a virtual environment. The method of claim 4, wherein the subject grid is a room or a building. The method of claim 1, wherein the second viewer is presented with the second user interface upon acceptance of the invitation. The method of claim 6, wherein the second user interface is further for causing the second viewer to establish a target grid for use in the virtual environment. The method of claim 2, wherein the second user interface is further operable to allow the first viewer to move the topic grid relative to the target grid. The method of claim 1, wherein the target grid has metadata associated therewith, the metadata having data indicating a scale ratio. The method of claim 9, wherein the subject grid has metadata associated therewith, the metadata having metadata indicating a scale ratio, and wherein the target grid and the topic grid are configured as The same size ratio is used, whereby the size of the theme mesh and the appearance of the model are correctly displayed relative to the target mesh. The method of claim 1, wherein the first viewer accesses the virtual environment method using a virtual reality device or an augmented reality device. The method of claim 11, wherein the second viewer accesses the virtual environment using the virtual reality device. Non-transitory computer readable medium comprising instructions for causing a computing environment to perform the method of claim 1 of the patent. A method of providing a media experience that can be The builder modifies on the server, including: a. accessing the database, the database including inventory data corresponding to the available items; b. providing a user interface operable to allow the creator to place the items in the inventory in a virtual Anywhere in the environment; c. wherein the user interface is operable to allow the creator to arbitrarily place a 3-D model of the item in the virtual environment, and to allow the creator to create or import a 3-D model of the item. The method of claim 14, wherein allowing the creator to create the item 3-D model comprises allowing the creator to import an image corresponding to the item into the three-dimensional model generation engine. The method of claim 14, further comprising adding a user interface, wherein the first viewer can access the virtual environment. The method of claim 16, wherein the user interface that causes the viewer to enter the virtual environment further allows the first viewer to invite the second viewer to access the virtual environment.