BG2910U1 - Interactive system for computer-assisted contactless testing of clothing and accessories by adaptating the tools of augmented reality and usage of a system of depth cameras - Google Patents

Abstract

The main application of the proposed interactive system for computer-assisted contactless testing of clothing and accessories by adapting tools of the augmented reality and using a depth camera system is primarily, but not exclusively, in the field of fashion, commerce, advertising and entertainment. The system offered allows the advertising of target products, incl. and through the means of customized advertising allowing the potential customer to see how it would look in the corresponding advertised offered item as well as in a combination of one or more advertised and / or available items, as based on artificial intelligence, the system offers the customer the best size, design, colour of the clothing, corresponding to the specifics of their figure and desire. The offered system also allows making a purchase of the item preferred by the user as well as its payment and delivery, incl. and without the intervention of an employee of the commercial site and regardless of the time of realization of the selected items and purchase, as well as the manufacturer or supplier.

Description

Field of technology

Multimedia interactive device

BACKGROUND OF THE INVENTION

Attempts to create computer-based systems to support customers' choice when buying clothes and accessories marked their development with the massification of computer technology, and especially after the advent of color CRT monitors and graphics operating systems. Through the process of supporting the customer's choice, retailers ensure an increase in sales volumes and a reduction in returned goods, and customers receive a higher degree of satisfaction with their purchase.

The main approaches for the realization of computer-assisted choice of customers when buying clothes and accessories, and in particular in the visualization of the selected product and how it would fit the customer, are mainly aimed at:

1. Visualization of a photo of a garment (with or without a mannequin) (Sources: 1111p: // \\\\\\ 2.Iim.com/bg_bg/ index.html. Http://www.explosi.bg/)

2. Visualization of a photograph of a garment superimposed as a still position and combined with a video stream from a camera or a photograph of the face in which the client "adjusts" on the photograph to see what it will look like (Source:

https://www.youtube.com/watch?v=5TZmQPdhpak. http://zugara.com/virtual-dressing-room-technology/virtual-dressing-room-kiosk.

https://www.youtube.com/watch?v=L9EwtGUTNzO)

3. Upload a user photo, where the system morphs the body parts to replace the head and visible parts of an existing photo with a mannequin

4. Visualization of a clothing model with an option for viewing / rotating by the client (Source: https: //www.magictoolbox.coin/magic360/)

5. Building a typical computer-generated avatar, which "dresses" with computer-generated clothes - entirely in a computer environment (Source:

https://www.youtube.com/watch?v=ZDJsZla9SYQ. https://www.youtube.com/watch?v=mWO InW-Zvs)

6. Construction of a customized (based on body type) computer-generated avatar, which "dresses" with computer-generated clothes - entirely in a computer environment (source:

http: // www. trimirror.com/en/ about /)

7. Construction of a customized (with actually measured or entered customer data) computer-generated avatar, which "dresses" with computer-generated clothes - entirely in a computer environment (Source:

https://www.youtube.com/watch?v=pMladTwgHoI)

8. Generating and superimposing a computer-generated garment on a video stream from a camera with an image of the client based on the silhouette of the client's body (in a controlled environment) (Source:

https://www.voutube.com/watch?v=BD4TCfECxsU&feature=voutu.be)

9. Generating and superimposing a computer-generated garment on a video stream from a camera with an image of the client based on a virtual skeleton of the client obtained from depth cameras (in a controlled environment) (Source:

https: //www.youtube.com/watch? v = ljbvnk 1Т 4vQ)

10. Generating and superimposing a computer-generated garment on a video stream from a camera with an image of the client based on the silhouette of the client's body (in an uncontrolled environment) (Source:

https://www.youtube.com/watch?v=u a-iFzQFuQ)

4711

11. Generating and superimposing a computer-generated garment on a video stream from a camera with an image of the client based on a virtual skeleton of the client, obtained from depth cameras (in an uncontrolled environment) (source:

https://www.youtube.com/watch?v=gz5ynrfJSDM. https://www.youtube.com/watch?v=IB2jutoT4Eo.

https://www.youtube.com/watch?v=h3vP3hdwf6B.

And although based on these approaches there are a number of commercial solutions, at the moment none of them solves the problem with the realism of the depicted garment, so that the client has the feeling that he sees himself with a real garment on himself.

The main problems of the existing solutions are related to the fact that the client cannot be seen from all sides (solutions related to approach 1, 2 and 3), the simulation leaves a feeling for a computer game and not for a real try on clothes, solutions related to approach 4, 5, 6 and 7), or the presence of a significant discrepancy between the customer's movement and the computer-generated garment (solutions related to approaches 8,9,10 and 11), as well as the lack of realistic behavior of the garment, according to the specifics of the specific figure of the client (solutions related to approach 1, 2, 3, 4, 5, 8, 9, 10 and 11).

Another key aspect of the issue is related to the reporting of the actual physical data of the client, in view of the correct visualization of the computer-generated garment or accessory, incl. and to determine the appropriate size of garment or accessory. The main approaches for measuring the physical measurement data of the client are related to the use of:

1. Physical means for assisted or independent contact measurement

2. Means for calculating estimated dimensions based on a two-dimensional photographic image (photo)

3. Means for non-contact measurement of body dimensions by creating a digital model

Despite the advancement of the technology for contactless measurements, the most accurate remain the so-called contact measurements, as the alternative methods are characterized by a certain level of mathematical approximation, extrapolation and modeling, as well as by the presence of measurement errors due to the specific technology used.

Regarding the measurement by 3D modeling, the main approaches are related to the use of:

1. Photogrammetric methods based on high quality photographic images

2. Creating a 3 D depth map using one or more depth sensors (type RGB-D, ToF or other applicable technology)

In order to achieve high accuracy of the created ZD model, these approaches require the use of multiple sensors or cameras, performing a series of multiple images processed with appropriate algorithms to create relatively accurate ZD models. This approach usually requires a certain period of time and does not allow the creation and tracking of the 3D model of the person in real time.

Technical essence of the utility model

The presented interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras solves a number of technical problems that have long faced workers in the field and helps to meet long-standing needs. previous approaches and solutions related to the construction of computer-assisted systems for contactless testing of clothes and accessories.

Although in its complexity there is no similar solution to the offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras, the closest solutions and components related to individual aspects of the system are as follows:

1. Regarding the approach for visualization of the selected product and how it would stand for the specific customer, as part of the systems for realization of computer-assisted customer selection at purchase

4712 of clothing and accessories, the closest solutions are related to generating and superimposing a computer-generated garment on a video stream from a camera with a client image based on the client's virtual skeleton obtained from depth cameras (in an uncontrolled environment). The main problem with existing solutions is that they mainly use stand-alone depth sensors (such as MS Kinect or similar), which are designed and optimized for the purpose of computer game consoles (eg MS Xbox) and are therefore characterized by a specific range of action (0.80 sh. - 4.00 sh. Source: https://blogs.msdn.microsoft.com/kinectforwindows/2012/01/20/near-mode-what-it-is-and-isnt/) , limited shooting angle of objects (43 ° vertically and 57 ° horizontally - source: https: // msdn.microsoft.com/en-us/library/ij 131033, aspx), which reflects extremely high error values when measuring the Z coordinate with increasing distance (source: Maimone, A., Fuchs, E Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras. https://www.researchgate.net/publication/ 221221196_Encumbrance-free_tele-presence_system_with_realtime_3D_capture_and_display_using_commodity_depth_cameras Fig. 5). It should be noted that although in practice there are solutions that use depth cameras located above the floor (on the ceiling), these systems are again characterized by shooting in a single plane, and their use is aimed at security activities indoors. (source: Gasparrini, S., Cippitelli, E., Spinsante, S., Gambi, E. A Depth-Based Fall Detection System Using a Kinect® Sensor.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3958279/).

In this regard, the proposed interactive system for computer-assisted contactless testing of clothing and accessories by adapting the tools of augmented reality and using a system of depth cameras, introduces a new approach combining a system of depth cameras, at least one of which is located above the floor (on the ceiling), which allows shooting of at least two planes, one of which is necessarily a top view and serves as a compensatory for the Z coordinate of the frontally located depth chamber. The adoption of such an approach allows to apply a specific compensatory approach, which allows through a system of specific parameters to compensate for errors in the measurement of depth chambers in one of the dimensions (mainly Z), which in turn allows building an accurate ZD model and virtual skeleton of the subject.

2. Regarding the non-contact measurement of the body, the closest solution to the offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras is the use of means for non-contact measurement of body dimensions by creating a TA digital model through the methods of TA scanning. This approach is usually characterized by the use of a large number of optical (source: http://www.aniwaa.com/3d-body-scanning/) and depth cameras (source: https://www.artec3d.com/3d-scanner / shapifybooth). laser measurements (source: https://www.youtube.com/watch?v= AzBlzxUgrg) or using a combination of sensors and a rotating platform (source: http://www.styku.com/). In addition, the proposed approach requires time (min. 10 s) to create the model and perform the measurements (source: https://www.artec3d.com/3dscanner/shapifybooth).

In this regard, the proposed interactive system for computer-assisted contactless testing of clothing and accessories by adapting the tools of augmented reality and using a system of depth cameras, introduces a new approach, building on the approach adopted by Microsoft to create a virtual skeleton (source: https : //msdn.microsoft.com/en-us/library/jj 131025.aspx) by using a system of depth cameras, at least one of which is located above the floor (ceiling) eliminates errors in depth measurement and subsequent generating the so-called "Virtual skeleton". Unlike similar solutions for non-contact measurement, the construction of a virtual skeleton using a system of depth cameras, at least one of which is located above the floor (ceiling) allows its generation and tracking in real time (up to 30 frames in second - source: https://i.stack.imgm.com/QQiI4.pngn https://stackoverflow.com/ guestions / 12978415 / all-frames-from-kinect-at-30fps), which in turn allows implementation on

4713 anthropometric measurement of established anthropometric relations between key points in the skeletal structure and key points of the physical body, taking into account the contour measurements of the depth chambers along the X and Y axes.

The technical essence of the offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras, is locked in the visualization of a computer-generated virtual garment superimposed on high-resolution video stream. the generated layer of digital information is created using adapted augmented reality tools and based on a customized (with non-contact measured customer data) computer-generated invisible avatar, which "dresses" with computer-generated clothing, taking into account the specifics of matter and its behavior, such as the adjustment of the computer-generated layers, incl. and augmented reality is based on a unified virtual skeleton obtained from multiple depth chambers, at least one of which is located above the floor (the virtual skeleton also serves as an augmented reality marker).

The interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system , includes at least: a high-quality video camera 3 placed at the level of human vision and capturing the space in front of the interactive installation connected via a computer system 5 to a device for visualizing 6 the captured space in front of the interactive installation, while maintaining the actual ratio between the components of the captured objects, a system of at least two depth cameras, the first camera 2 being located near the high quality video camera 3 of the space in front of the interactive installation and directed in the same shooting direction as it, and the second k the camera 1 is positioned so as to allow the capture of a target area from above, the system of depth cameras through a specialized hardware and / or software module 4 is connected to a computer system 5 for processing incoming information from the cameras, rendering the 3D model of items selected or selected by the customer, combining them into a single video stream for visualization of the connected visualization device 5, in real time or with a delay of not more than 0.033 s. The system of at least two depth cameras, at least one of them (element 2) is located near the camera for high-quality video camera (element 3) of the space in front of the interactive installation and is directed in the same shooting direction as it, and at least one of which (element 1) is located so as to allow the capture of a target area with a view from above, as the incoming information from the depth cameras (elements 1 and 2) enters a specialized hardware and / or software module (element 4), in which through applying a specially developed algorithm, the real positions of key points of the human anatomy are calculated by eliminating the errors in the measurement with the depth chambers, and on this basis the module (element 4) allows non-contact measurement of various visually visible parts of the human anatomy, including and to determine customer measurement data. These key points in human anatomy are also used as specialized markers to coordinate and visualize computer-generated objects, including, but not limited to, augmented, virtual, mixed, and other types of realities created using local, distributed devices. or remote computing power, regardless of the type of computing device (CPU, GPU or other applicable), the technology of use (microprocessor, nanotechnology or other applicable) and the architecture of the computing device itself (RISC, CISC or other applicable).

At the same time, the offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras, allows a combination of multilayer video streams and computer-generated images, incl. and transparent ones, allowing the visualization of the realistic behavior of selected clothes and accessories on the real figure of the specific user, which is realized by creating a customized transparent computer avatar, by adapting

4714 of a basic avatar, built on statistical anthropometric information, by contactless reading of the shape and key dimensions of the real human body, incl. and key measurement data, using the component of the interactive installation presented in claim 2.

Explanation of the attached figure

Figure 1 presents the relationship between the individual elements of the interactive system for computer-assisted contactless testing of clothing and accessories by adapting the tools of augmented reality and using a system of depth cameras

Examples of implementation of the utility model

The offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras is a complex component system that includes at least: a camera for high quality video recording 3, placed on the level of human vision and capturing the space in front of the interactive installation connected via a computer system 5 to a device for visualizing 6 the captured space in front of the interactive installation, while maintaining the actual ratio between the components of the captured objects, a system of at least two depth cameras 2 is located near the high-quality video camera 3 of the space in front of the interactive installation and is directed in the same shooting direction as it, and the second camera 1 is located so as to allow shooting of a target area with a view from above, such as the system of depth cameras through a specialized hardware and / or software module 4 is connected to a computer system 5 for processing incoming information from the cameras, rendering the ID model of selected or selected by the customer items, combining them into a single video stream for visualization of the connected visualization device 5 , in real time or with a delay of not more than 0,033 s.

Application (use) of the utility model

The offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras can be built mainly, but not only in the form of a stand-alone installation, as its individual components can be installed both in the main body of the installation and in the room in which it is located. The use of standard components with general purpose and / or specialized application, which are manufactured by more than one manufacturer and are offered for free retail, allow the smooth construction and operation in an uncontrolled environment of the offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras.

The main application of the offered interactive system for computer-assisted contactless testing of clothes and accessories by adapting the tools of augmented reality and using a system of depth cameras is basic, but not only in the field of fashion, trade, advertising and entertainment. The offered system can be located as part of a store network or in the form of a free-standing installation outside a physical store, as its main purpose is to advertise certain target products, incl. and by the customized advertising method, allowing the potential customer to see what it would look like in the respective advertised item, as well as in a combination of one or more advertised and / or available items, the visualization being performed by a real-time photorealistic image. Based on the available artificial intelligence, the system offers the customer the best size of clothing, corresponding to the specifics of his figure, and also allows visualization of how he would look when choosing a different size from the offered items. The offered system also allows the purchase of an item liked by the consumer, as well as its payment and delivery, incl. and without the intervention of an employee of the commercial site and regardless of the time of the selection and purchase.

Claims (1)

Claims 1. An interactive system for computer-assisted contactless testing of clothing and accessories by adapting the augmented reality tools and using a system of depth cameras, characterized in that it includes at least a camera for high-quality video recording (3), placed at the level of human vision and capturing the space in front of the interactive installation connected via a computer system (5) with a device for visualization (6) of the captured space in front of the interactive installation, while maintaining the real ratio between the components of the captured objects, a system of at least two depth cameras, the first camera (2) being located close to the high-quality video camera (3) of the space in front of the interactive installation and directed in the same shooting direction as it, and the second camera (1) being positioned so as to allow shooting a target area with a view from above, as the system of depth cameras through the spa A hardware and / or software module (4) is connected to a computer system (5) for processing the incoming information streams from the cameras, rendering the LED model of selected or customer-selected items, combining them into a single video stream for visualization of the connected device. visualization (5), real-time or up to 0.033 s delay.