WO2024013707A1 - Improved system for playing extended reality (xr) with immersive experience in the consumer and professional spheres - Google Patents

Improved system for playing extended reality (xr) with immersive experience in the consumer and professional spheres Download PDF

Info

Publication number
WO2024013707A1
WO2024013707A1 PCT/IB2023/057215 IB2023057215W WO2024013707A1 WO 2024013707 A1 WO2024013707 A1 WO 2024013707A1 IB 2023057215 W IB2023057215 W IB 2023057215W WO 2024013707 A1 WO2024013707 A1 WO 2024013707A1
Authority
WO
WIPO (PCT)
Prior art keywords
kiosk
user
software
experience
virtual reality
Prior art date
Application number
PCT/IB2023/057215
Other languages
French (fr)
Inventor
Jacopo MERLI
Elisabetta Carrara
Marco Rossini
Original Assignee
Jmc Group S.R.L. A Socio Unico
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jmc Group S.R.L. A Socio Unico filed Critical Jmc Group S.R.L. A Socio Unico
Publication of WO2024013707A1 publication Critical patent/WO2024013707A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/20—Cooling means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/101—Collaborative creation, e.g. joint development of products or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/103—Workflow collaboration or project management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/109—Time management, e.g. calendars, reminders, meetings or time accounting
    • G06Q10/1093—Calendar-based scheduling for persons or groups
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0276—Advertisement creation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0621—Item configuration or customization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0623—Item investigation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0641—Shopping interfaces
    • G06Q30/0643—Graphical representation of items or shoppers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/04—Manufacturing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1601—Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays

Definitions

  • the present invention relatives to a system suitable for reproducing virtual realities with immersive experience in the consumer and business sectors.
  • the scope of the present invention is that of electronic systems employed to provide users with the ability to access extended reality (XR) experience either online or offline.
  • XR extended reality
  • object an object, product, solution, or service
  • the invention can represent and provide experience to objects other than those involved in the process of purchasing an object, product, or solution, but also for consulting and analyzing them (e.g., at the design stage), which can also involve procedures for use (e.g., training for using industrial machinery).
  • the system of the invention can be used for the mechanical analysis of a component, such as machinery in general or a prosthesis, by browsing the catalog with the touchscreen of the kiosk or through artificial intelligence. Afterward, through the headset, the same component will be seen in virtual or augmented reality to configure it, open it (mechanically), evaluate its individual parts, and display the related information.
  • a component such as machinery in general or a prosthesis
  • the system of the invention offers, to the user of services in the consumer and business sectors, the possibility of experiencing in immersive form the feelings, impressions, and stimuli normally experienced in the real relationship with the object, interacting with it inside its environment of use.
  • the system of the invention offers the advantage of contextualizing the object and the surrounding scenario related to the user’s physical person allowing the user to move around the environment that hosts the object and preferably also interact with a virtual figure of one or more users, such as the seller or the designer of the object or a trainer.
  • the system of the invention also offers the important advantage of being easily and quickly accessible, when compared to the current systems that require long start-up times (e.g., for connecting all devices, turning on the computer and all its peripherals), complex configurations, and a number of other devices, such as mice, cameras, headsets and more depending on the experience.
  • FIGS. 1 and 1 B are perspective and side views, respectively, of an embodiment of the kiosk, belonging to the system object of the invention, in the version with two screens;
  • FIG. 1A shows the microphone 16 and the webcam 15 components in detail
  • FIG. 2 shows the system in Figure 1 with a detail of the two main constituent units of the kiosk, i.e. , the headset or head-mounted display and the computer;
  • Figure 3 is an exploded view of the system in Figure 2;
  • FIG. 3A is a section view of the fixing technique of the 24-inch and 43- inch modules
  • FIG. 4 shows an embodiment of the system of the invention, in the version with a single 43-inch screen
  • FIG. 5 is an exploded view of the system in Figure 4.
  • FIG. 8 shows the operation diagram of the system of the invention in the single-user and multi-user modes, with their respective modes of use;
  • FIG. 8A shows a diagram of some examples of real-time applications compatible with the system of the invention
  • FIG. 9 shows a version, but not the only version, of the system interface of the invention in the version shown in Figure 1 ;
  • FIG. 10 shows the operating diagram of the system in Figure 1 ;
  • FIG. 11 shows the flowchart of the user’s actions according to the interventions of the cloud server, kiosk, and virtual reality device of the system of the invention
  • FIG. 12 shows the embodiment of the system of the invention in single- user mode (only one person accesses the immersive experience);
  • FIG 13 shows the system in Figure 12, in the multi-user version (two or more people access the same immersive experience);
  • FIG 14 shows the system in Figure 13, in the version using artificial intelligence (represented with an avatar);
  • FIG. 16 shows the detail of the ventilation devices placed on the chassis of the kiosk and on the internal power supply
  • FIG. 17 shows the detail of the ventilation devices near the computer and thus inside the kiosk.
  • the system of the invention is suitable for displaying digital content by means of XR or network technologies.
  • the system comprises the kiosk 1 in Figure 1 , the virtual reality display device or headset 4, the preferably industrialgrade PC 8 (i.e., designed to withstand high temperatures and long stress cycles), and possibly the server 27 for the online operation of the system and the network 23 as shown in the diagram in Figure 10.
  • the kiosk 1 in Figure 1 the virtual reality display device or headset 4
  • the preferably industrialgrade PC 8 i.e., designed to withstand high temperatures and long stress cycles
  • server 27 for the online operation of the system and the network 23 as shown in the diagram in Figure 10.
  • the kiosk was created to make Extended Reality - XR technologies more accessible and easy to use for companies, in particular for the B2B market.
  • system of the invention is suitable for a wide variety of scenarios and contexts of use not necessarily limited only to the following sectors:
  • Points of sale and pop-up stores airport lounges, yacht clubs, golf clubs, shopping malls, events, and showrooms.
  • the invention can also represent and provide experience to objects other than those involved in the process of purchasing an object, product, or solution, but also for consulting and analyzing them (e.g., at the design stage).
  • Maintenance and training processes which may also involve operating procedures (e.g., training to operate industrial machinery) delivered in complete safety compared to field training.
  • the invention concerns a pre-configured system ready to be used by the user simply and intuitively to exploit the potential of XR technologies.
  • the two main constituent modules of the kiosk 1 shown in Figure 2 are an industrial-grade computer 8 and a virtual reality display device 4 provided with a headset and possible controllers that allow the user to interact and move inside the virtual reality experience and, with some types of headsets, also inside augmented reality (AR) experiences and content.
  • the headset 4 has the function of immersing the user in various experiences, by virtue of the sensors and electronics installed in it.
  • the kiosk 1 is compatible with various models and brands of headsets on the market according to the user’s needs.
  • the virtual reality display device 4 is selected together with the kiosk user, understanding the needs and verifying the compatibility of the devices with the software applications.
  • a headset model HTC VIVE FOCUS 3 made by HTC Corp. (Taiwan) may be suitable for the invention.
  • the headset can, in turn, be connected to two controllers 5 and 6, one for each of the user’s hands, which are used to interact with objects in virtual reality, such as for moving objects, opening car doors, turning on lights and the like, having sensory feedback (haptic, auditory and visual) through vibrations, sounds and visual effects played in virtual reality (VR).
  • VR virtual reality
  • the immersive interaction in virtual reality can take place with the help of different types of controls, suitable for allowing even free hand interaction, such as tracing devices (e.g., cameras built into the headset which allow controller-free experiences) and of the more advanced and innovative wearable type (e.g., gloves for capturing movements in VR with respective haptic feedback).
  • tracing devices e.g., cameras built into the headset which allow controller-free experiences
  • wearable type e.g., gloves for capturing movements in VR with respective haptic feedback
  • the headset 4 is wirelessly connected to the kiosk 1 , preferably with a Wi-Fi connection. Thus, there are no cables that could interfere with or get in the way of the user’s movements.
  • the computer 8 is the result of an assembly designed and created by the kiosk manufacturer and is preferably, but not necessarily, industrial grade, so that it can offer maximum full-time (h 24/7) performance, even when subjected to the highest graphics processing loads, thus overcoming the limitations of standard solutions that require long booting times and complex configurations that inherently have several points of failure.
  • One feature of the kiosk 1 is its modularity so that two configurations can be mounted.
  • the first one shown in Figure 1 , which comprises two 24-inch landscape-mode touch displays 9, 10, and a backlit shelf 11 for storing the virtual reality device 4 after use.
  • the second configuration shown in Figures 4 and 5, provides a 43-inch touch display 12 in portrait mode instead of the elements mentioned in the previous configuration.
  • the displays and their supporting frames 33 are slightly titled, preferably by 5 degrees from the vertical, to promote the ergonomics of the product and consequently its use (Figure 1A).
  • Figure 3A shows a section view explaining the technology which allows the interchangeability of the modules 9, 10, and 12.
  • threaded inserts 34 are electro-welded around the perimeter of the displays 9, 10, and the frame 35 for their installation. Then the displays 9, 10 are fixed with brackets 36 to the threaded inserts 34. Finally, the plate 33 is fixed containing the displays fixed to the frame 35, which has holes 37 into which the threaded inserts 34 present on the perimeter of the plate 33 of the displays are inserted. Bolts or alternatively flange nuts 38 are used to fix it all.
  • the main hardware devices for completing the system are a Near Field Communication - NFC or barcode reader 13, which can be configured according to the user’s needs (e.g., to read badges and restrict the use of applications to the user), two Wi-Fi antennas 14a, 14b to ensure a wireless connection between the kiosk 1 and the virtual reality display device 4, and the standard instrumentation needed to make video calls and conference calls, such as a webcam 15, a microphone 16 and speakers 17 ( Figure 1 ). These three elements are integrated into the kiosk 1 according to studies conducted on user needs.
  • the webcam 15 is positioned above the two displays 9, 10 and is tilted by 15 sexagesimal degrees from the floor, a magnitude calculated based on percentiles of human heights, both male and female so that it can ensure adequate filming of individuals operating in front of the kiosk.
  • the camera can be used for security purposes, for conducting meetings, capturing user gestures, or acquiring video or images, e.g., for creating the avatar for VR with facial features that match those of the user.
  • the microphone 16 is placed just above the camera 15 so that it is aesthetically unobtrusive and can still pick up the users’ voices.
  • the speakers 17 are located on the side walls in the upper part of the kiosk 1 . This choice of placement is not random and is designed to ensure good sound quality. The two speakers are directed out from the side walls and arranged in an elevated position and therefore they ensure good audio propagation around the entire kiosk. To ensure good sound quality even in environments with slight background noise, an amplifier is provided, shown in Figure 1 C, and placed at the rear of kiosk 1 . This gives the user the option of being able to select the volume level that best suits the environment in which the kiosk is installed (see Figure 15).
  • the hardware devices mentioned above are connected to each other in the manner shown in Figure 6 by means of proprietary cables to the computer, excluding the headset 4 and any virtual reality controllers. Indeed, the wireless connection is preferable for these components, so as to avoid harmful interference to the user’s experience in Virtual Reality - VR, who cannot see the real-world surroundings precisely because he/she is immersed in a virtual world.
  • the system of the invention comprises, as shown in Figure 6, a Central Processing Unit - CPU with a base frequency of at least 3 GHz (GigaHertz) which can reach the frequency of 4.70 GHz when subjected to the highest workloads.
  • a Random Access Memory - RAM which is strongly recommended with at least 32 GB of capacity although 128 GB is recommended
  • the memory capacity of such a device is recommended to be at least 12 GB.
  • the GPU plays a key role in content consumption.
  • virtual reality processes content by means of real-time 3D rendering. Selecting a performance GPU offers a smooth experience, free from lag, delays, or crashes caused by high computational demands.
  • the GPU temperature after six hours of use remains reasonably low (94 degrees Celsius), within the standard safe temperature range for proper operation of the component without it overheating.
  • a configuration with at least 32 GB is recommended for the RAM; the higher the RAM value, the higher the processing capacity. Furthermore, a Wi-Fi card 18 is mounted to ensure a stable connection between the PC and the virtual reality headset. The result of all this is a PC assembled in-house to ensure the best performance when using content, especially for virtual reality content.
  • the company can upload its multimedia content and virtual experiences in two different modes, depending on the availability of the network connection: offline or online.
  • offline there are several installation options related to software applications.
  • the first option involves installing the various applications locally, then transferring them by means of external peripheral devices (e.g., USB pen drives, external hard drives) into the PC 8 of the kiosk 1 .
  • external peripheral devices e.g., USB pen drives, external hard drives
  • This procedure does not require any kind of Internet connection.
  • the cloud server 27 in Figure 10 can be used to access applications and install them on PCs.
  • a network connection is required when downloading applications.
  • the resources which can be executed without a network connection range from software already available to the organization/user purchasing the kiosk, customized VR experiences created by the kiosk supplier company for the user, or by means of the 3DFrame software from the company Vection Technologies designed for VR experiences.
  • the kiosk 1 is a virtual reality display device, as shown in Figure 10 it is compatible with other online collaboration software which takes advantage of such technology.
  • the 3DFrame software in the case of a single user still offers the possibility of accessing the network to download content such as 3D models and other material (e.g., such as textures, HDRI, etc.) while in the case of multiuser mode, the network is used for the collaboration of multiple users within virtual reality experiences.
  • the user can interact with the kiosk 1 to display virtual objects with colors and content not present in reality, such as adding a car in an office or showroom, configuring materials and colors as desired, and then checking how this result behaves in the environment using virtual reality.
  • virtual stores can be created, as if they were real so that products can always be accessible to potential customers, who can view the goods displayed by means of virtual reality.
  • Training 20 users interact with the object to explore it and gain more knowledge about how that object or parts of it function or interact with other objects/other components. For example, by having a 3D model of an engine, it is possible to dissect it, see the exploded view of the components, and understand how the engine works. It is also possible to view and interact with assembly/disassembly and maintenance training of a given 3D model of a product.
  • the diagram in Figure 8 also shows the single-user operating mode 22, which through the network 23 allows access to the respective content 24.
  • Figure 8 also shows the multi-user operating mode 25, in which, collaboration 26 with other users can also be implemented again through the network 23.
  • the virtual work area can be understood ( Figure 10).
  • the users 3a, 3b, 3c are connected to the network 23 having an active part in this work area.
  • the connection to the network connects the user 3b of the kiosk 1 to the users 3a, 3c of the other virtual reality compatible platforms, i.e. , a PC without a headset and a PC provided with a headset.
  • a PC without a headset a PC without a headset
  • a PC provided with a headset Obviously, the user, if provided with a headset, will enjoy a more immersive experience than the PC user in desktop mode but this does not affect the content, which is the same for everyone.
  • the user’s experience of using the kiosk is then proposed ( Figure 11 ), traced from the first use.
  • the user scans the ID badge on the appropriate NFC or QR reader (as a function of the technology used by the badge), the cloud verifies the data and related permissions of the badge, and if these are valid, the user has access to use the kiosk.
  • the kiosk must be used for VR and video conferencing, the webcam 15 and microphone 16 of the kiosk 1 are activated, whereas if only the display of virtual reality content is needed, the headset 4 along with the corresponding controllers are activated.
  • the user can begin to use, with the possibility of falling into the scenarios mentioned earlier, and in the case of the presence of a network connection, additional content can also be added in real time.
  • the dedicated application for this purpose e.g., Cisco Webex
  • content is viewed in a multi-user context.
  • the headset 4 is put back on the support shelf 11 , thus leaving the system ready for the next use.
  • FIGS 12, 13, and 14 illustrate possible variants of virtual experience.
  • the variant shown in Figure 12 describes a single user’s experience, i.e., what the user sees during the immersive experience.
  • the kiosk 1 is connected with other kiosks or remote users to provide a multi-user virtual reality experience. Therefore, the system of the invention, shown in this variant, can also include a plurality of kiosks 1 or other devices equipped with the 3DFrame software which connect, also remotely, to other kiosks located elsewhere, so that the users can share the same immersive experience.
  • an artificial intelligence function e.g., in the form of an avatar 7
  • the avatar 7 interacts with the user 3 as human-to-human and no longer human-to-machine, behaving, for example, like the seller of the car 2, asking questions and making suggestions interactively with the user.
  • the artificial intelligence system can be equipped with a camera to analyze the surrounding environment and its occupants for analysis and interaction. For example, it can recognize the gender and age of the user to adapt the interactive model with the user.
  • the kiosk 1 enables realtime consumption of content of interest to users, for example, in the retail environment by virtue of the software and virtual reality display device.
  • the solution of the kiosk 1 is integrated and ready to use, consequently when using the kiosk it is necessary to execute the software, and configure in a few simple steps the asset the external user is interested in, and this new ad hoc configured solution is ready to be displayed in virtual reality. Therefore, it is essential, in addition to possessing software 40, 41 , 42, 43, 44, which offers the possibility of configuring assets (e.g., cars, clothing, etc.) by means of virtual reality, to also have a hardware system which allows maximum ease of use for everyone, in which all the wiring and configuration part of the devices is already done.
  • virtual reality headsets with inherent wireless technology have been selected so that the user can have the best possible experience without having to perform any operation on the hardware, as the same is always available for use.
  • the kiosk is essential as this device, being standalone, offers the possibility to be placed and configured in a wide range of scenarios.
  • the solution of the kiosk allows an integrated and complete, ready-to-use solution; indeed, it is possible to introduce kiosks in stores (or other businesses listed above) to improve and make the user experience more efficient: conventionally, PCs connected to a monitor, a keyboard, a mouse, a virtual reality headset and controllers (with associated cables) are used. All these elements are chaotic and not optimized to ensure a good user experience, which is why the kiosk, by integrating all components, is efficient, to the point that it can boast a dedicated space within the store.
  • the basic interface of the kiosk (supplied with the purchase of the kiosk) shown in Figure 9 in the version with the two 24-inch displays, offers the virtual reality experience in the upper display 9, while there is the home screen in the lower display 10, in which it is possible to see the charge status of the headsets 4, a launcher section with all the installed applications, settings and advertising and calendar screens for booking the slot of use of the product.
  • the kiosk buyers also have the option, as an additional package, to request a specific experience based on their needs or to upload their own application to the kiosk.
  • the main advantage of the system of the invention is that it provides a ready-to-use platform for any virtual reality experience, whatever the context of the application content.
  • the applications 40 created ad hoc for the user are also compatible with the system of the invention, like the applications 41 already available to the user, as well as the software 43, 44 for VR, designed for use online in a multi-user context or offline, respectively.
  • the 3DFrame software 42 developed by Vection Technologies, can also be used.
  • the main advantage for the company/user is to have a chameleonic device ready to adapt to any situation.
  • the kiosk 1 can be placed in a retail store to provide enhanced customer experiences, as well as used in the office as a support tool during planning, having the ability to view content on a 1 :1 scale and interact with it.
  • the kiosk of the system of the invention is characterized by a design aimed at making it suitable for office environments, e.g., such as the meeting room shown in Figure 15.
  • the decibel values generated by the kiosk during the tests are following those outlined in the reference standards.
  • the kiosk in order to be used in an office environment, the kiosk must maintain a decibel value within regulatory limits for a long time, withstand high computation and graphics processing loads throughout the day, not overheat, and ensure that all internal components are not damaged by any heat fluxes generated by the high processing load of the CPU, GPU, and RAM.
  • the kiosk 1 must not give off too much heat, and the maximum use temperature of the internal components is respected and thus not exceeded.
  • the Graphics Card Unit - GPU also reaches lower temperature values than the threshold (usually 100°C) above which inconveniences could occur. Ventilation is made possible by the two fans 29a and 29b in Figure 16, located on the back of the device and providing good airflow and a low noise index. There is also a fan 30 arranged on the side, dedicated to expelling internally generated heat. By having more air entering than exiting, the internal temperature stabilizes and stays within limits to ensure proper operation of the entire device.
  • the virtual reality experience can be used for design, planning, analysis, evaluation, prototyping, training, service, marketing, operations support, consultation (e.g., to see what an object looks like), and more.
  • the upgradability of the kiosk indeed, this object can be configured with future hardware components not yet on the market which will allow for greatly increased graphics performance and XR software experience.
  • components such as CPUs, GPUs, RAM and storage, industrial computers, webcams, speakers, microphones, and displays are interchangeable with better performing future models.
  • the system of the invention allows the delivery of virtual reality experiences, as it is a standalone product containing all the hardware devices required to execute virtual reality applications.
  • the XR experience display system is suitable for various contexts and market scenarios by virtue of its compatibility with virtual, augmented, and Al reality software applications.
  • the kiosk of the system of the invention consists of hardware components wired with proprietary cables, except for the virtual reality headset, which must have wireless connection technology in order to ensure a better experience during the consumption of the demo.
  • the same kiosk is a modular type with two different possible configurations.
  • the standard configuration provides two 24-inch displays, with the option of removing this configuration to install one which contains a 43-inch display.
  • the threaded fasteners must be unscrewed to perform the change operation.
  • the kiosk has the part in which the displays are placed tilted by 5 sexagesimal degrees, so it can ensure better usability.
  • the same kiosk has an NFC or QR reader system to scan badges, enabling use to authorized users; this feature is relevant in office settings for design review or virtual reality viewing of confidential material under development by accessing internal servers.
  • the kiosk comes with the required devices for consuming multimedia content and making video conferences: speakers equipped with an amplifier, webcam, and microphone. Because of its features, the kiosk can conduct video conferences in virtual reality, by virtue of the 3DFrame software developed by the company Vection Technologies.
  • the kiosk of the system of the invention is an integrated and ready-to-use product because each device is already connected and functioning, consequently sparing all the complex configuration part to be able to enter virtual realities.
  • the computer 8 placed inside the kiosk 1 is preferably industrial grade and equipped with state-of-the-art components to ensure maximum performance h 24/7.
  • at least one Central Processing Unit - CPU with a base frequency of 3 GHz (GigaHertz) is required, which can reach the frequency of 4.70 GHz when subjected to the highest workloads.
  • the memory capacity of the card is recommended to be at least 12 GB.
  • the kiosk of the system of the invention is found to be suitable for office environments because the noise level is within the limits; furthermore, the temperatures of the internal components are within the ranges provided by the manufacturers’ data sheets. This peculiarity is made possible by the fans installed both inside the kiosk and on the kiosk chassis, which are designed and positioned to ensure proper airflow.
  • the kiosk 1 is a real-time modification tool in all the contexts aimed at third-party users interested in asset configuration.
  • the system consisting of the kiosk and the 3D Frame software application makes it possible to be able to change the physical properties of the assets (such as color, texture, and material) and have immediate feedback on the result virtually, so that the asset can be viewed realistically and from all points of view.
  • the kiosk contributes to the immediacy of action because, as mentioned earlier, it is an integrated, ready- to-use system.
  • the kiosk of the system of the invention also allows for different types of virtual reality applications; indeed, existing applications, ad hoc applications at the request of the end user, 3D Frame software, and all types of software designed to enable a virtual reality experience are executed.
  • the kiosk 1 itself has both internal storage (by virtue of the previously mentioned storage) and cloud storage, by virtue of the network connection. In both cases, the space is abundantly oversized to ensure product longevity without having to worry about maintenance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • General Engineering & Computer Science (AREA)
  • Tourism & Hospitality (AREA)
  • Development Economics (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Educational Administration (AREA)
  • Game Theory and Decision Science (AREA)
  • Manufacturing & Machinery (AREA)
  • Educational Technology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A system for reproducing virtual realities with immersive experience in the consumer and business sectors. The system comprises a kiosk (1) equipped with a PC (8) and a device (4) for displaying virtual realities (XR technologies), wherein said virtual realities consist of either software (40) designed according to the user's needs, or software (41) already available to the user, or software (42) for the creation and display of Presentation (19), Training (20) and Collaboration (21) scenarios, or software (43, 44) for multi-user use (25) in on-line and off-line modes, respectively. The system of the invention allows the simplified delivery of experiences by means of XR technologies, as it is a standalone product containing all the necessary hardware devices to execute extended reality (XR) applications.

Description

IMPROVED SYSTEM FOR PLAYING EXTENDED REALITY (XR) WITH IMMERSIVE EXPERIENCE IN THE CONSUMER AND PROFESSIONAL
SPHERES
BACKGROUND OF THE INVENTION
The present invention relatives to a system suitable for reproducing virtual realities with immersive experience in the consumer and business sectors.
The scope of the present invention is that of electronic systems employed to provide users with the ability to access extended reality (XR) experience either online or offline.
Any system with traditional instrumentation offers, at best, a photograph or a 3D image of the product, which, for obvious reasons, cannot convey to the users the feeling that they would have if the analysis and evaluation experience were made directly at the seller’s facility, where - moreover - often not all models and configurations of the product are present. In addition, the feelings about interior and exterior dimensions, ergonomics and roominess, relative bulk, and capacity are very limited.
For this reason, users completely miss the impression of the actual experience of the process of analyzing and evaluating an object, product, solution, or service (hereinafter object), within the scope of the scene in which it takes place.
Furthermore, although implicit reference is made in the description to (typically consumer) sales processes, the field of application must also be understood to extend to professional use, to simplify the access process to the consumption of the invention, for collaboration, training, and contextualization purposes, in real-time and multi-user modes.
SUMMARY OF THE INVENTION
The invention can represent and provide experience to objects other than those involved in the process of purchasing an object, product, or solution, but also for consulting and analyzing them (e.g., at the design stage), which can also involve procedures for use (e.g., training for using industrial machinery).
For example, the system of the invention can be used for the mechanical analysis of a component, such as machinery in general or a prosthesis, by browsing the catalog with the touchscreen of the kiosk or through artificial intelligence. Afterward, through the headset, the same component will be seen in virtual or augmented reality to configure it, open it (mechanically), evaluate its individual parts, and display the related information.
These and other objects are accomplished with the system in claim 1. Preferred embodiments of the invention will be apparent from the remaining claims.
In comparison with the prior art systems, the system of the invention offers, to the user of services in the consumer and business sectors, the possibility of experiencing in immersive form the feelings, impressions, and stimuli normally experienced in the real relationship with the object, interacting with it inside its environment of use.
Indeed, the system of the invention offers the advantage of contextualizing the object and the surrounding scenario related to the user’s physical person allowing the user to move around the environment that hosts the object and preferably also interact with a virtual figure of one or more users, such as the seller or the designer of the object or a trainer.
The system of the invention also offers the important advantage of being easily and quickly accessible, when compared to the current systems that require long start-up times (e.g., for connecting all devices, turning on the computer and all its peripherals), complex configurations, and a number of other devices, such as mice, cameras, headsets and more depending on the experience.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other objects, advantages, and characteristics result from the following description of preferred embodiments of the system of the invention illustrated, by way of non-limiting examples, in the figures of the accompanying drawings.
In these:
- Figures 1 and 1 B are perspective and side views, respectively, of an embodiment of the kiosk, belonging to the system object of the invention, in the version with two screens;
- Figure 1A shows the microphone 16 and the webcam 15 components in detail;
- Figure 1 C shows the detail of the amplifier connected to the speakers;
- Figure 2 shows the system in Figure 1 with a detail of the two main constituent units of the kiosk, i.e. , the headset or head-mounted display and the computer;
- Figure 3 is an exploded view of the system in Figure 2;
- Figure 3A is a section view of the fixing technique of the 24-inch and 43- inch modules;
- Figure 4 shows an embodiment of the system of the invention, in the version with a single 43-inch screen;
- Figure 5 is an exploded view of the system in Figure 4;
- Figure 6 is the wiring diagram of the system hardware components in the preceding figures;
- Figure 7 shows the usage pattern of the system of the invention in the offline and on-line modes;
- Figure 8 shows the operation diagram of the system of the invention in the single-user and multi-user modes, with their respective modes of use;
- Figure 8A shows a diagram of some examples of real-time applications compatible with the system of the invention;
- Figure 9 shows a version, but not the only version, of the system interface of the invention in the version shown in Figure 1 ;
- Figure 10 shows the operating diagram of the system in Figure 1 ;
- Figure 11 shows the flowchart of the user’s actions according to the interventions of the cloud server, kiosk, and virtual reality device of the system of the invention;
- Figure 12 shows the embodiment of the system of the invention in single- user mode (only one person accesses the immersive experience);
- Figure 13 shows the system in Figure 12, in the multi-user version (two or more people access the same immersive experience);
- Figure 14 shows the system in Figure 13, in the version using artificial intelligence (represented with an avatar);
- Figure 15 shows an example of an office environment compatible with the use of the invention;
- Figure 16 shows the detail of the ventilation devices placed on the chassis of the kiosk and on the internal power supply;
- Figure 17 shows the detail of the ventilation devices near the computer and thus inside the kiosk.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The system of the invention is suitable for displaying digital content by means of XR or network technologies. The system comprises the kiosk 1 in Figure 1 , the virtual reality display device or headset 4, the preferably industrialgrade PC 8 (i.e., designed to withstand high temperatures and long stress cycles), and possibly the server 27 for the online operation of the system and the network 23 as shown in the diagram in Figure 10.
The kiosk was created to make Extended Reality - XR technologies more accessible and easy to use for companies, in particular for the B2B market.
Through this system, companies can easily exploit XR technologies to expand their presentation spaces, provide a better and more comprehensive experience to users, communicate directly with users and co-workers also remotely, extend their brand presence, or optimize business processes. For example, solutions for expanding presentation spaces make it possible to display assets and products immediately and at full scale but in a virtual context with no need for the assets and products to be physically present at the display location.
It is apparent that such a solution lends itself very well to all the sectors in which large machinery or numerous components or objects requiring large space to be displayed are involved. The experience for users is thus complete because it is possible to interact directly with the components as well as display them and also to display them individually.
Therefore, the system of the invention is suitable for a wide variety of scenarios and contexts of use not necessarily limited only to the following sectors:
Points of sale and pop-up stores, airport lounges, yacht clubs, golf clubs, shopping malls, events, and showrooms.
Displaying fashion and retail-related content by means of virtual reality (VR) in showrooms, pop-up stores, events, and shopping malls.
Configuration for the customization of content in showrooms, pop-up stores, and events; currently the users who would like to evaluate (or configure for practice), e.g., their favorite car model online have at their disposal the touch button (on mobile systems, such as tablets and smartphones) or the mouse on traditional computers, through which to choose the car model and customize it with the desired accessories, favorite color and so forth. However, the described system with traditional instrumentation can offer, at best, a photograph or a 3D image of the car, which, for obvious reasons, cannot convey to the users the feeling that they would have if the analysis and evaluation experience were made directly at the seller’s facility, where - moreover - often not all models and configurations of the product are present. In addition, the feelings about interior and exterior dimensions, ergonomics and roominess, relative bulk, and capacity are very limited. Through XR kiosks, on the other hand, it is possible to configure one or more full-scale products or environments and be able to interact with them through XR technologies.
Virtual showrooms.
The invention can also represent and provide experience to objects other than those involved in the process of purchasing an object, product, or solution, but also for consulting and analyzing them (e.g., at the design stage).
Maintenance and training processes, which may also involve operating procedures (e.g., training to operate industrial machinery) delivered in complete safety compared to field training.
The prior art regarding the use of virtual reality technologies is complicated to understand for any user who is unfamiliar with technology and for the many people who have never tried virtual reality.
To better understand this factor, it is necessary to consider not only the technology itself, but also the connecting part of the aforesaid headset with the PC, an operation provided for some headset models or for loading experiences created by third parties not affiliated with the headset manufacturing companies onto the device. Thus, the invention concerns a pre-configured system ready to be used by the user simply and intuitively to exploit the potential of XR technologies.
To this end, the two main constituent modules of the kiosk 1 shown in Figure 2 are an industrial-grade computer 8 and a virtual reality display device 4 provided with a headset and possible controllers that allow the user to interact and move inside the virtual reality experience and, with some types of headsets, also inside augmented reality (AR) experiences and content. The headset 4 has the function of immersing the user in various experiences, by virtue of the sensors and electronics installed in it.
The kiosk 1 is compatible with various models and brands of headsets on the market according to the user’s needs. The virtual reality display device 4 is selected together with the kiosk user, understanding the needs and verifying the compatibility of the devices with the software applications. For example, a headset model HTC VIVE FOCUS 3 made by HTC Corp. (Taiwan) may be suitable for the invention. The headset can, in turn, be connected to two controllers 5 and 6, one for each of the user’s hands, which are used to interact with objects in virtual reality, such as for moving objects, opening car doors, turning on lights and the like, having sensory feedback (haptic, auditory and visual) through vibrations, sounds and visual effects played in virtual reality (VR).
The immersive interaction in virtual reality, based on the type of headset used, can take place with the help of different types of controls, suitable for allowing even free hand interaction, such as tracing devices (e.g., cameras built into the headset which allow controller-free experiences) and of the more advanced and innovative wearable type (e.g., gloves for capturing movements in VR with respective haptic feedback).
The headset 4 is wirelessly connected to the kiosk 1 , preferably with a Wi-Fi connection. Thus, there are no cables that could interfere with or get in the way of the user’s movements.
The computer 8, on the other hand, is the result of an assembly designed and created by the kiosk manufacturer and is preferably, but not necessarily, industrial grade, so that it can offer maximum full-time (h 24/7) performance, even when subjected to the highest graphics processing loads, thus overcoming the limitations of standard solutions that require long booting times and complex configurations that inherently have several points of failure.
One feature of the kiosk 1 is its modularity so that two configurations can be mounted. The first one, shown in Figure 1 , which comprises two 24-inch landscape-mode touch displays 9, 10, and a backlit shelf 11 for storing the virtual reality device 4 after use. The second configuration, shown in Figures 4 and 5, provides a 43-inch touch display 12 in portrait mode instead of the elements mentioned in the previous configuration. In both of these configurations, the displays and their supporting frames 33 are slightly titled, preferably by 5 degrees from the vertical, to promote the ergonomics of the product and consequently its use (Figure 1A). Figure 3A shows a section view explaining the technology which allows the interchangeability of the modules 9, 10, and 12. Starting from the plate 33 in Figure 3, shaped specifically for the size of the displays, threaded inserts 34 are electro-welded around the perimeter of the displays 9, 10, and the frame 35 for their installation. Then the displays 9, 10 are fixed with brackets 36 to the threaded inserts 34. Finally, the plate 33 is fixed containing the displays fixed to the frame 35, which has holes 37 into which the threaded inserts 34 present on the perimeter of the plate 33 of the displays are inserted. Bolts or alternatively flange nuts 38 are used to fix it all.
The main hardware devices for completing the system are a Near Field Communication - NFC or barcode reader 13, which can be configured according to the user’s needs (e.g., to read badges and restrict the use of applications to the user), two Wi-Fi antennas 14a, 14b to ensure a wireless connection between the kiosk 1 and the virtual reality display device 4, and the standard instrumentation needed to make video calls and conference calls, such as a webcam 15, a microphone 16 and speakers 17 (Figure 1 ). These three elements are integrated into the kiosk 1 according to studies conducted on user needs. The webcam 15 is positioned above the two displays 9, 10 and is tilted by 15 sexagesimal degrees from the floor, a magnitude calculated based on percentiles of human heights, both male and female so that it can ensure adequate filming of individuals operating in front of the kiosk. The camera can be used for security purposes, for conducting meetings, capturing user gestures, or acquiring video or images, e.g., for creating the avatar for VR with facial features that match those of the user.
The microphone 16 is placed just above the camera 15 so that it is aesthetically unobtrusive and can still pick up the users’ voices. The speakers 17 are located on the side walls in the upper part of the kiosk 1 . This choice of placement is not random and is designed to ensure good sound quality. The two speakers are directed out from the side walls and arranged in an elevated position and therefore they ensure good audio propagation around the entire kiosk. To ensure good sound quality even in environments with slight background noise, an amplifier is provided, shown in Figure 1 C, and placed at the rear of kiosk 1 . This gives the user the option of being able to select the volume level that best suits the environment in which the kiosk is installed (see Figure 15).
The hardware devices mentioned above are connected to each other in the manner shown in Figure 6 by means of proprietary cables to the computer, excluding the headset 4 and any virtual reality controllers. Indeed, the wireless connection is preferable for these components, so as to avoid harmful interference to the user’s experience in Virtual Reality - VR, who cannot see the real-world surroundings precisely because he/she is immersed in a virtual world.
Furthermore, VR technologies require high performance from the internal components of the PC. For this reason, the system of the invention comprises, as shown in Figure 6, a Central Processing Unit - CPU with a base frequency of at least 3 GHz (GigaHertz) which can reach the frequency of 4.70 GHz when subjected to the highest workloads. There is a Random Access Memory - RAM, which is strongly recommended with at least 32 GB of capacity although 128 GB is recommended, and a Graphic Processing Unit - GPU with a base clock frequency of at least 1.365 GHz that can go up to 1.665 GHz if the GPU is subjected to the highest processing loads. Furthermore, the memory capacity of such a device is recommended to be at least 12 GB.
In particular, the GPU plays a key role in content consumption. Indeed, virtual reality processes content by means of real-time 3D rendering. Selecting a performance GPU offers a smooth experience, free from lag, delays, or crashes caused by high computational demands. Furthermore, the GPU temperature after six hours of use remains reasonably low (94 degrees Celsius), within the standard safe temperature range for proper operation of the component without it overheating.
A configuration with at least 32 GB is recommended for the RAM; the higher the RAM value, the higher the processing capacity. Furthermore, a Wi-Fi card 18 is mounted to ensure a stable connection between the PC and the virtual reality headset. The result of all this is a PC assembled in-house to ensure the best performance when using content, especially for virtual reality content.
Companies that purchase the kiosk 1 in the configuration in Figure 1 can use the content already on the application at the time of purchase, as shown in the diagram in Figure 8. The application is the 3DFrame software developed by corporate Vection Technologies headquartered in Casalecchio di Reno (BO), which offers basic features of simplifying VR access, demo demos, a calendar for booking kiosk use, and multimedia content.
Furthermore, the company can upload its multimedia content and virtual experiences in two different modes, depending on the availability of the network connection: offline or online. In the first case scenario, i.e., offline, there are several installation options related to software applications. The first option involves installing the various applications locally, then transferring them by means of external peripheral devices (e.g., USB pen drives, external hard drives) into the PC 8 of the kiosk 1 . This procedure does not require any kind of Internet connection. Alternatively, the cloud server 27 in Figure 10 can be used to access applications and install them on PCs. Obviously, in the presence of a cloud platform, a network connection is required when downloading applications.
In terms of the types of applications in this category, the resources which can be executed without a network connection range from software already available to the organization/user purchasing the kiosk, customized VR experiences created by the kiosk supplier company for the user, or by means of the 3DFrame software from the company Vection Technologies designed for VR experiences.
Remaining in the realm of applications that require a network connection to run, we find again 3DFrame software, as multiple users, remotely connected from anywhere in the world, can get together to collaborate in virtual reality. In addition, using Cisco’s Webex platform, it is possible to switch to virtual reality during a meeting, with up to sixteen people, as long as all users have the software installed on their PCs and the meeting host has an active license.
As the kiosk 1 is a virtual reality display device, as shown in Figure 10 it is compatible with other online collaboration software which takes advantage of such technology. The 3DFrame software in the case of a single user still offers the possibility of accessing the network to download content such as 3D models and other material (e.g., such as textures, HDRI, etc.) while in the case of multiuser mode, the network is used for the collaboration of multiple users within virtual reality experiences.
Both the cases (both single-user and multi-user) offer the different content possibilities outlined in Figure 8: Presentation 19, Training 20, and Collaboration 21.
In the Presentation 19 scenario, the user can interact with the kiosk 1 to display virtual objects with colors and content not present in reality, such as adding a car in an office or showroom, configuring materials and colors as desired, and then checking how this result behaves in the environment using virtual reality. Furthermore, virtual stores can be created, as if they were real so that products can always be accessible to potential customers, who can view the goods displayed by means of virtual reality.
In the Training 20 scenario, users interact with the object to explore it and gain more knowledge about how that object or parts of it function or interact with other objects/other components. For example, by having a 3D model of an engine, it is possible to dissect it, see the exploded view of the components, and understand how the engine works. It is also possible to view and interact with assembly/disassembly and maintenance training of a given 3D model of a product.
In the Collaboration 21 mode, on the other hand, by virtue of virtual reality, users are projected into a virtual environment in which they can cooperate with other users, again within virtual reality, with the ability to interact with contents in scenes and with other users represented in the form of avatars. For example, with the kiosk placed within a design office located in a specific geographic area (e.g., Europe), it is possible to display a project to designers in all geographic areas (e.g., Europe, Africa, Asia, America) by means of virtual reality so that they can collaborate, analyze and make decisions together, all these users being present in the same environment and being able to view and interact with the same content. Consequently, if a user moves a component in the scene, all other users can see this action. The diagram in Figure 8 also shows the single-user operating mode 22, which through the network 23 allows access to the respective content 24. Figure 8 also shows the multi-user operating mode 25, in which, collaboration 26 with other users can also be implemented again through the network 23. Once the kiosk 1 is connected to the server 27, it is possible to interact with it by means of dedicated applications or by conducting virtual reality meetings through integration with Cisco Webex.
In this specific multi-user case, in which a network is created, the virtual work area can be understood (Figure 10). The users 3a, 3b, 3c are connected to the network 23 having an active part in this work area. By taking advantage of the potential of the network, it is possible to connect from different parts of the world and with different peripheral devices, having the same content display results. The connection to the network connects the user 3b of the kiosk 1 to the users 3a, 3c of the other virtual reality compatible platforms, i.e. , a PC without a headset and a PC provided with a headset. Obviously, the user, if provided with a headset, will enjoy a more immersive experience than the PC user in desktop mode but this does not affect the content, which is the same for everyone.
The user’s experience of using the kiosk is then proposed (Figure 11 ), traced from the first use. The user scans the ID badge on the appropriate NFC or QR reader (as a function of the technology used by the badge), the cloud verifies the data and related permissions of the badge, and if these are valid, the user has access to use the kiosk.
At this point, there are two scenarios, based on the user’s needs. If the kiosk must be used for VR and video conferencing, the webcam 15 and microphone 16 of the kiosk 1 are activated, whereas if only the display of virtual reality content is needed, the headset 4 along with the corresponding controllers are activated. At this point, the user can begin to use, with the possibility of falling into the scenarios mentioned earlier, and in the case of the presence of a network connection, additional content can also be added in real time.
If the need arises to conduct immersive experiences in video conferences, after connecting to the network, the dedicated application for this purpose (e.g., Cisco Webex) is executed and content is viewed in a multi-user context. When these experiences are finished, the headset 4 is put back on the support shelf 11 , thus leaving the system ready for the next use.
Figures 12, 13, and 14 illustrate possible variants of virtual experience.
The variant shown in Figure 12 describes a single user’s experience, i.e., what the user sees during the immersive experience.
In the variant shown in Figure 13, the kiosk 1 is connected with other kiosks or remote users to provide a multi-user virtual reality experience. Therefore, the system of the invention, shown in this variant, can also include a plurality of kiosks 1 or other devices equipped with the 3DFrame software which connect, also remotely, to other kiosks located elsewhere, so that the users can share the same immersive experience.
According to the system variant of the invention shown in Figure 14, a multiuser experience is again achieved, with the addition of an artificial intelligence function (e.g., in the form of an avatar 7), guiding experiences such as organizing a trip and the like, or buying a car, a house, or other. The avatar 7 interacts with the user 3 as human-to-human and no longer human-to-machine, behaving, for example, like the seller of the car 2, asking questions and making suggestions interactively with the user. The artificial intelligence system can be equipped with a camera to analyze the surrounding environment and its occupants for analysis and interaction. For example, it can recognize the gender and age of the user to adapt the interactive model with the user.
From these examples of use, it is apparent that the kiosk 1 enables realtime consumption of content of interest to users, for example, in the retail environment by virtue of the software and virtual reality display device. Indeed, the solution of the kiosk 1 is integrated and ready to use, consequently when using the kiosk it is necessary to execute the software, and configure in a few simple steps the asset the external user is interested in, and this new ad hoc configured solution is ready to be displayed in virtual reality. Therefore, it is essential, in addition to possessing software 40, 41 , 42, 43, 44, which offers the possibility of configuring assets (e.g., cars, clothing, etc.) by means of virtual reality, to also have a hardware system which allows maximum ease of use for everyone, in which all the wiring and configuration part of the devices is already done. Furthermore, virtual reality headsets with inherent wireless technology have been selected so that the user can have the best possible experience without having to perform any operation on the hardware, as the same is always available for use.
The advantage of real-time changes, both at the hardware and software level, is respectively brought by the XR Kiosk with the 3D Frame software; for example, users interested in a motor vehicle can set the color of their liking by means of the software. Therefore, to visualize the result, the kiosk is essential as this device, being standalone, offers the possibility to be placed and configured in a wide range of scenarios. So, the solution of the kiosk allows an integrated and complete, ready-to-use solution; indeed, it is possible to introduce kiosks in stores (or other businesses listed above) to improve and make the user experience more efficient: conventionally, PCs connected to a monitor, a keyboard, a mouse, a virtual reality headset and controllers (with associated cables) are used. All these elements are chaotic and not optimized to ensure a good user experience, which is why the kiosk, by integrating all components, is efficient, to the point that it can boast a dedicated space within the store.
The basic interface of the kiosk (supplied with the purchase of the kiosk) shown in Figure 9 in the version with the two 24-inch displays, offers the virtual reality experience in the upper display 9, while there is the home screen in the lower display 10, in which it is possible to see the charge status of the headsets 4, a launcher section with all the installed applications, settings and advertising and calendar screens for booking the slot of use of the product. The kiosk buyers also have the option, as an additional package, to request a specific experience based on their needs or to upload their own application to the kiosk.
The main advantage of the system of the invention is that it provides a ready-to-use platform for any virtual reality experience, whatever the context of the application content.
As shown in Figure 8A, the applications 40 created ad hoc for the user are also compatible with the system of the invention, like the applications 41 already available to the user, as well as the software 43, 44 for VR, designed for use online in a multi-user context or offline, respectively. Finally, the 3DFrame software 42, developed by Vection Technologies, can also be used.
The main advantage for the company/user is to have a chameleonic device ready to adapt to any situation. Indeed, the kiosk 1 can be placed in a retail store to provide enhanced customer experiences, as well as used in the office as a support tool during planning, having the ability to view content on a 1 :1 scale and interact with it.
The kiosk of the system of the invention is characterized by a design aimed at making it suitable for office environments, e.g., such as the meeting room shown in Figure 15. Indeed, the decibel values generated by the kiosk during the tests are following those outlined in the reference standards. In particular, in order to be used in an office environment, the kiosk must maintain a decibel value within regulatory limits for a long time, withstand high computation and graphics processing loads throughout the day, not overheat, and ensure that all internal components are not damaged by any heat fluxes generated by the high processing load of the CPU, GPU, and RAM. Furthermore, the kiosk 1 must not give off too much heat, and the maximum use temperature of the internal components is respected and thus not exceeded.
As mentioned earlier, the Graphics Card Unit - GPU also reaches lower temperature values than the threshold (usually 100°C) above which inconveniences could occur. Ventilation is made possible by the two fans 29a and 29b in Figure 16, located on the back of the device and providing good airflow and a low noise index. There is also a fan 30 arranged on the side, dedicated to expelling internally generated heat. By having more air entering than exiting, the internal temperature stabilizes and stays within limits to ensure proper operation of the entire device.
Inside the kiosk 1 , there are additional internal fans 31 , shown in Figure 17, located near the ventilation grille of the computer in order to continue to feed cool air to the GPU and ensure its optimal operation during longer time periods. As shown in Figure 16, there is also an internal fan 32 on the general power supply of the kiosk because it is a passive component that, as per the respective datasheet, needs a fan to dissipate the generated heat.
To keep the performance of the kiosk at a high level, there is also a system that allows the removal of the aforementioned fans, so that the filter can be replaced when it is loaded with dust, which is capable of affecting the performance and safety of the components placed inside the kiosk.
Modifications may be made to the invention as described above and illustrated in the accompanying figures in order to make variants, which nevertheless fall within the scope of the following claims.
Thus, for example, the virtual reality experience can be used for design, planning, analysis, evaluation, prototyping, training, service, marketing, operations support, consultation (e.g., to see what an object looks like), and more.
Finally, it is important to mention the upgradability of the kiosk; indeed, this object can be configured with future hardware components not yet on the market which will allow for greatly increased graphics performance and XR software experience. As a result, components such as CPUs, GPUs, RAM and storage, industrial computers, webcams, speakers, microphones, and displays are interchangeable with better performing future models. The system of the invention allows the delivery of virtual reality experiences, as it is a standalone product containing all the hardware devices required to execute virtual reality applications.
The XR experience display system is suitable for various contexts and market scenarios by virtue of its compatibility with virtual, augmented, and Al reality software applications.
The kiosk of the system of the invention consists of hardware components wired with proprietary cables, except for the virtual reality headset, which must have wireless connection technology in order to ensure a better experience during the consumption of the demo.
The same kiosk is a modular type with two different possible configurations. The standard configuration provides two 24-inch displays, with the option of removing this configuration to install one which contains a 43-inch display. The threaded fasteners must be unscrewed to perform the change operation. The kiosk has the part in which the displays are placed tilted by 5 sexagesimal degrees, so it can ensure better usability.
The same kiosk has an NFC or QR reader system to scan badges, enabling use to authorized users; this feature is relevant in office settings for design review or virtual reality viewing of confidential material under development by accessing internal servers.
The kiosk comes with the required devices for consuming multimedia content and making video conferences: speakers equipped with an amplifier, webcam, and microphone. Because of its features, the kiosk can conduct video conferences in virtual reality, by virtue of the 3DFrame software developed by the company Vection Technologies.
The kiosk of the system of the invention is an integrated and ready-to-use product because each device is already connected and functioning, consequently sparing all the complex configuration part to be able to enter virtual realities.
The computer 8 placed inside the kiosk 1 is preferably industrial grade and equipped with state-of-the-art components to ensure maximum performance h 24/7. As a result, at least one Central Processing Unit - CPU with a base frequency of 3 GHz (GigaHertz) is required, which can reach the frequency of 4.70 GHz when subjected to the highest workloads. There is a Random Access Memory - RAM, which is strongly recommended with at least 32 GB of capacity although 128 GB is recommended, and a Graphic Processing Unit - GPU with a base clock frequency equal to 1.365 GHz that can go up to 1.665 GHz if the GPU is subjected to the highest processing loads. Furthermore, the memory capacity of the card is recommended to be at least 12 GB.
As a result of tests conducted internally, the kiosk of the system of the invention is found to be suitable for office environments because the noise level is within the limits; furthermore, the temperatures of the internal components are within the ranges provided by the manufacturers’ data sheets. This peculiarity is made possible by the fans installed both inside the kiosk and on the kiosk chassis, which are designed and positioned to ensure proper airflow.
Thus, the kiosk 1 is a real-time modification tool in all the contexts aimed at third-party users interested in asset configuration. Indeed, the system consisting of the kiosk and the 3D Frame software application makes it possible to be able to change the physical properties of the assets (such as color, texture, and material) and have immediate feedback on the result virtually, so that the asset can be viewed realistically and from all points of view. The kiosk contributes to the immediacy of action because, as mentioned earlier, it is an integrated, ready- to-use system.
The kiosk of the system of the invention also allows for different types of virtual reality applications; indeed, existing applications, ad hoc applications at the request of the end user, 3D Frame software, and all types of software designed to enable a virtual reality experience are executed. The kiosk 1 itself has both internal storage (by virtue of the previously mentioned storage) and cloud storage, by virtue of the network connection. In both cases, the space is abundantly oversized to ensure product longevity without having to worry about maintenance.

Claims

1. A system for reproducing extended realities (XR) with immersive experience in the consumer and business sectors, characterized in that it comprises a kiosk (1 ) equipped with a PC (8) and an XR device (4) for ensuring quick access to virtual experiences through either software (40) designed according to the user’s needs or software (41 ) already available to the user, or 3DFrame software (42) for creating and displaying Presentation (19), Training (20) and Collaboration (21 ) scenarios, or software (43, 44) for multi-user use (25) in online and offline modes, respectively.
2. The system according to claim 1 , characterized in that it involves the realtime use of said software (40, 41 , 42, 43, 44) with the possibility of configuring assets and interacting with them by means of said extended reality.
3. The system according to claim 2, characterized in that said PC (8) is provided with a CPU having a base frequency of at least 3 GHz, a RAM of at least 32 GB, and a GPU of at least 1.365 GHz to allow high performance during the consumption of XR experiences.
4. The system according to claim 3, characterized in that said kiosk (1 ) further comprises one or more displays (9, 10, 12), an NFC reader (13), a webcam (15), a microphone (16), and an amplifier (39) for amplifying the sound of said speakers (17) to deliver multi-sensory and complete XR experiences of various types.
5. The system according to claim 4, characterized in that it further provides two Wi-Fi antennas (14A, 14B) for putting the kiosk (1 ) into communication with said display device (4) in a wireless mode.
6. The system according to one of claims 2 to 5, characterized in that said kiosk (1 ) promotes a real-time experience in which the configuration and exploration of assets are implemented with the possibility for the user to change their physical properties, receiving immediate virtual feedback in the form of said assets displayed realistically, with the possibility of accessing presentations, training and collaborative virtual environments.
7. The system according to claim 1 , characterized in that said kiosk (1 ) respects the noise and temperature values for use in the office sector.
8. The system according to claim 7, characterized in that said kiosk (1 ) has rear ventilation fans (29a, 29b) and a side ejection fan (30) of the internal heat, suitable for increasing the air level entering the kiosk (1 ), so as to stabilize its internal temperature.
9. The system according to claim 8, characterized in that it further provides an internal fan (32) cooling the general power supply of the kiosk (1).
10. The system according to claim 9, characterized in that said PC (8) is provided with internal fans (31 ) located near the respective ventilation grille.
11. The system according to one or more of the preceding claims, characterized in that said kiosk (1 ) is provided with a frame (35) equipped with holes (37) for housing threaded inserts (34) fixing the plates (33) of said displays (9, 10, 12) in order to make the product configurable for various uses.
12. The system according to claim 11 , characterized in that said frame (35) is tilted by 5 degrees from the vertical in order to promote usability.
PCT/IB2023/057215 2022-07-15 2023-07-14 Improved system for playing extended reality (xr) with immersive experience in the consumer and professional spheres WO2024013707A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
IT202200014920 2022-07-15
IT102022000014920 2022-07-15
IT202300014481 2023-07-11
IT102023000014481 2023-07-11

Publications (1)

Publication Number Publication Date
WO2024013707A1 true WO2024013707A1 (en) 2024-01-18

Family

ID=87889956

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2023/057215 WO2024013707A1 (en) 2022-07-15 2023-07-14 Improved system for playing extended reality (xr) with immersive experience in the consumer and professional spheres

Country Status (1)

Country Link
WO (1) WO2024013707A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170103160A1 (en) * 2015-10-12 2017-04-13 Milsco Manufacturing Company, A Unit Of Jason Incorporated Customer Comfort Optimization Method, Apparatus, and System
US20170111520A1 (en) * 2015-10-15 2017-04-20 Civiq Smartscapes, Llc Method and apparatus for power and temperature control of compartments within a personal communication structure (pcs)
KR20200125300A (en) * 2019-04-26 2020-11-04 차봇모빌리티 주식회사 Kiosk apparatus and system for providing comparative estimation and subscription service of car insurance product
US20200401576A1 (en) * 2019-06-18 2020-12-24 Tmrw Foundation Ip & Holding S. À R.L. Interacting with real-world items and corresponding databases through a virtual twin reality

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170103160A1 (en) * 2015-10-12 2017-04-13 Milsco Manufacturing Company, A Unit Of Jason Incorporated Customer Comfort Optimization Method, Apparatus, and System
US20170111520A1 (en) * 2015-10-15 2017-04-20 Civiq Smartscapes, Llc Method and apparatus for power and temperature control of compartments within a personal communication structure (pcs)
KR20200125300A (en) * 2019-04-26 2020-11-04 차봇모빌리티 주식회사 Kiosk apparatus and system for providing comparative estimation and subscription service of car insurance product
US20200401576A1 (en) * 2019-06-18 2020-12-24 Tmrw Foundation Ip & Holding S. À R.L. Interacting with real-world items and corresponding databases through a virtual twin reality

Similar Documents

Publication Publication Date Title
US11823256B2 (en) Virtual reality platform for retail environment simulation
CN103019505B (en) The method and apparatus setting up user's dedicated window on multiusers interaction tables
US20080215994A1 (en) Virtual world avatar control, interactivity and communication interactive messaging
CN106875244A (en) A kind of virtual reality purchase method, device and electronic equipment
JP2010535362A (en) Monitoring the opinions and reactions of users in the virtual world
US9741062B2 (en) System for collaboratively interacting with content
CN107079186A (en) enhanced interactive television experience
CN105536247A (en) Video gaming device with image identification
JP7415390B2 (en) Purchased product management system, user terminal, server, purchased product management method, and program
JP2021103526A (en) Information providing device, information providing system, information providing method, and information providing program
JP6714306B1 (en) Distribution system, live distribution device, live distribution method and program
KR20200008706A (en) Production system and production method for virtual reality contents
KR102121107B1 (en) Method for providing virtual reality tour and record media recorded program for implement thereof
WO2024013707A1 (en) Improved system for playing extended reality (xr) with immersive experience in the consumer and professional spheres
TWM559476U (en) System device with virtual reality and mixed reality house purchase experience
JP2023018884A (en) Program, information processing device, and method
US20030135408A1 (en) Methods and apparatus for providing an interactive backdrop with real time variable advertising content
JP2019207509A (en) Moving image creation system, method, and program
WO2022176428A1 (en) Sales method, program, information processing device, and terminal
CN110956688A (en) Three-dimensional free display interaction method for commodities
Brown et al. Towards a service framework for remote sales support via augmented reality
KR102566312B1 (en) Method and apparatus for providing platform album service
WO2023181562A1 (en) Virtual item control system, virtual item control method, and virtual item control program
WO2021131363A1 (en) Object management system, object management method, and object management program
Freitas KAVE-Kinect Cave: design, tools and comparative analysis with other VR technologies

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23764381

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)