KR102183574B1 - System for constructing of virtual reality shoppingmall using photographed image data and method thereof - Google Patents

Abstract

The present invention constructs a virtual reality space using images taken from an offline store according to the location or change of an exhibited product, and reconfigures a virtual reality shopping mall by linking products arranged in the configured virtual reality space with pre-stored product information. It relates to a system and method for constructing a virtual reality shopping mall using a possible offline photographed video.
In order to solve the above problems, the system for constructing a virtual reality shopping mall using an offline photographed video object according to the present invention is a virtual reality shopping mall by receiving a plurality of cameras that are installed in an offline store to photograph a video and a video photographed from the plurality of cameras. A system for building a virtual reality shopping mall including a shopping mall building server for constructing a shopping mall building server, wherein the shopping mall building server comprises: a product registration unit for storing and managing product information on exhibited products; An image generation unit for receiving an image photographed from the camera and generating a panoramic image using the received image; A product matching unit detecting a product included in the panoramic image generated by the image generating unit and matching the detected product with product information stored in the product registration unit; A VR shopping mall building unit that establishes a shopping mall in a virtual reality space by linking the panoramic image generated by the image generation unit with the product information matched by the product matching unit, and a virtual reality space built by the VR shopping mall building unit according to an input event It characterized in that it is configured to include a VR space search unit that displays.

Description

Construction system and method of virtual reality shopping mall using offline filmed images {SYSTEM FOR CONSTRUCTING OF VIRTUAL REALITY SHOPPINGMALL USING PHOTOGRAPHED IMAGE DATA AND METHOD THEREOF}

The present invention relates to a system and method for constructing a virtual reality shopping mall using offline photographed images, and more particularly, to configure a virtual reality space using images taken from an offline store according to the location or change of an exhibited product, and The present invention relates to a system and method for constructing a virtual reality shopping mall using an offline photographed image capable of reconfiguring a virtual reality shopping mall by linking products arranged in a real space with product information stored in advance.

The existing offline shopping method, in which customers visit an actual offline store and purchase the desired product, has been changed to the Internet shopping method using a computer or a home shopping mall method using TV commercials, and in addition to this, various methods of cyber shopping malls are provided. Shopping malls have become commonplace.

Shopping malls do not exist only on the Internet, but are a concept that existed in offline stores. In other words, a cyber shopping mall is a form in which a buyer purchases a product through a website built on the Internet and performs payment (credit card, e-money, cash, etc.) Say.

These cyber shopping malls pursue a unique service type and use a sales strategy that highlights differentiation from other cyber shopping malls, and due to the convenience of access, the development of purchase payment methods, and the development of the delivery system, shopping related to purchases, etc. In the field, it has occupied a weight equal to that of offline shopping.

In line with this trend, virtual reality shopping malls that provide 3D images of products for sale are attracting attention in order to increase the accessibility of consumers to products.

As one of the technologies for providing a virtual reality shopping mall, a mobile online shopping mall viewed with a VR camera has been disclosed in Korean Patent Publication No. 10-2017-0085260.

The technology is characterized by photographing and producing a target product or image (including) and advertisement of a shopping mall with a VR camera, synthesizing and editing the left/right images, and then uploading the VR stereoscopic image to the corresponding online shopping mall. do.

In addition, as a technology for providing a shopping mall based on an augmented reality, a system for providing shopping information based on an augmented reality and a control method thereof are disclosed in Korean Patent Publication No. 10-2017-0042258.

The above technology implements virtual reality by including tags related to the product or store in a 3D panoramic image of the store on the road view of the product or street displayed in the store, thereby realizing the view encountered in a department store, shopping mall, street, etc. While providing, additional information related to products or stores provided offline can be delivered through tags.

The above technologies have the advantage of encouraging product purchase by providing a product in a three-dimensional form in a virtual space, and providing a product design, size, practicality, and usage method.

However, in the reality that the cycle of replacement and change of products shown in virtual reality or symptom reality is getting shorter and shorter, such as the products displayed in accordance with the consumer's demands are constantly changing and the products on display are replaced according to product upgrades. A conventional virtual shopping mall including technologies is a method of disposing and providing a product produced as a stereoscopic image in a virtual online space, and when a product is replaced or changed, the virtual online space needs to be edited.

In other words, if a product displayed in a virtual space is changed or replaced, it must be newly uploaded as a product of the corresponding stereoscopic image. The technology for editing the virtual online space is complicated, takes a long time, and is difficult to build. Due to the high cost, there are many difficulties in continuous maintenance.

KR 10-2017-0085260 A (2017. 07. 24.) KR 10-2017-0042258 A (2017. 04. 18.)

The present invention was created in order to solve the problems of the prior art, and the problem to be solved in the present invention is a virtual reality by linking the product image included in the video photographed in the offline store and the product information of the corresponding product previously stored. It is to provide a system and method for constructing a virtual reality shopping mall using an offline photographed image capable of constructing a shopping mall.

In addition, it is to provide a system and method for constructing a virtual reality shopping mall using offline photographed images, in which a virtual shopping mall space can be expanded using a relationship with images photographed in an offline store.

In order to solve the above problems, the system for constructing a virtual reality shopping mall using an offline photographed video object according to the present invention is a virtual reality shopping mall by receiving a plurality of cameras that are installed in an offline store to photograph a video and a video photographed from the plurality of cameras. A system for building a virtual reality shopping mall including a shopping mall building server for constructing a shopping mall building server, wherein the shopping mall building server comprises: a product registration unit for storing and managing product information on exhibited products; An image generation unit for receiving an image photographed from the camera and generating a panoramic image using the received image; A product matching unit detecting a product included in the panoramic image generated by the image generating unit and matching the detected product with product information stored in the product registration unit; A VR shopping mall building unit that establishes a shopping mall in a virtual reality space by linking the panoramic image generated by the image generation unit with the product information matched by the product matching unit, and a virtual reality space built by the VR shopping mall building unit according to an input event It characterized in that it is configured to include a VR space search unit that displays.

Here, the shopping mall building server further includes an image updating unit for comparing the panoramic image generated by the image generating unit with a previously stored panoramic image and transmitting an update signal to the image generating unit when the panoramic image generated by the image generating unit is changed. And the image generation unit receives an image photographed from the camera based on an update signal of the image update unit, and regenerates a panoramic image using the received image, and the VR shopping mall construction unit is in the image generation unit. A shopping mall is constructed in a virtual reality space by connecting the reproduced panoramic image and product information matched by the product matching unit.

In addition, the product matching unit may include a product matching module for matching the product image included in the generated panoramic image with product information stored in the product registration unit; It may be configured to include a product information display module that displays product information on a product selected according to a selection event, and a product purchase execution module that performs a purchase procedure for the product displayed on the product display module.

In addition, the product information may include a product identification number, a stereoscopic image, a classification, a name, a price, a standard, a remaining amount, and a description.

In addition, a method of constructing a virtual reality shopping mall using an offline photographed image according to the present invention includes: a panoramic image generation step of generating a panoramic image by receiving an image photographed from a camera and connecting regions overlapping in the received image; Product information matching step of matching product information corresponding to the product with the product included in the panoramic image generated in the panoramic image generation step, and the product matched in the product information matching step with the panoramic image generated in the panoramic image generation step It includes a VR shopping mall building step of building a shopping mall in a virtual reality space using information, and when the panorama image generated in the panorama image generation step is changed by comparing it with a previously stored panorama image, it is generated in the panorama image generation step And a VR shopping mall product management step of configuring a virtual reality space using the created panoramic image and the product information matched in the product information matching step to manage the placed products.

According to the present invention, since it is possible to construct a virtual reality shopping mall space for exhibited products by using photographed images of an offline store, there is an advantage of saving time and cost required for construction.

In addition, the virtual reality shopping mall space can be automatically constructed according to changes, replacements, and location movements of exhibited products, which greatly reduces the time the product is applied to the virtual reality shopping mall, and makes it easier to maintain the virtual reality shopping mall. There is this.

In addition, the VR shopping mall space is immediately changed due to the change of the display products in the offline store, and the products in the VR shopping mall space and the exhibition products in the offline store are matched and reflected, so that the availability of products can be checked in the VR shopping mall space. , Since the product displayed in the VR shopping mall space is a product placed in the real space, it has the advantage of being able to purchase immediately regardless of inventory.

1 is a schematic configuration diagram of a system for constructing a virtual reality shopping mall using an offline photographed image according to the present invention.
2 is a block diagram of a shopping mall building server applied to a system for building a virtual reality shopping mall using offline photographed images according to the present invention.
3 is an interface of an embodiment displayed by a product information display module in a system for building a virtual reality shopping mall using an offline photographed image according to the present invention.
4 is a payment processing interface of an embodiment displayed by a product purchase execution module in a system for building a virtual reality shopping mall using an offline photographed image according to the present invention.
5 is a VR shopping mall interface generated by a VR shopping mall construction unit in a system for building a virtual reality shopping mall using an offline photographed image according to the present invention.
6 is a flowchart illustrating a method of constructing a virtual reality shopping mall using an offline photographed image according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

The present invention constructs a virtual reality space using images taken from an offline store according to the location or change of an exhibited product, and reconfigures a virtual reality shopping mall by linking products arranged in the configured virtual reality space with pre-stored product information. It relates to a system and method for constructing a virtual reality shopping mall using a possible offline photographed video.

1 is a schematic configuration diagram of a system for building a virtual reality shopping mall using an offline photographed image according to the present invention.

Referring to FIG. 1, a system for constructing a virtual reality shopping mall using an offline photographed image according to the present invention includes a camera 100, a shopping mall building server 200, and a terminal 300.

The camera 100 is installed in an offline store to photograph the offline store and transmits the photographed video. In this case, the offline store may be composed of a store that directly sells products such as a department store or a shopping center, or a service store such as a ticket, gift certificate, lottery, health, medical, etc. that sells a service for a product.

The camera 100 may be configured as a device that can easily generate a panoramic image, and is one of an omni-directional camera capable of shooting 360° (360° VR camera) or a camera capable of moving vertically and horizontally (such as a CCTV camera). Optionally can be applied.

In addition, the camera 100 may be configured as a plurality of cameras to capture the entire area of the offline store, and may be arranged so that an area photographed by one selected camera and a part of an area photographed by a neighboring camera overlap.

The shopping mall building server 200 receives the image photographed from the camera 100 to build a virtual reality shopping mall, and sells and manages products for the built virtual reality shopping mall.

In this case, the shopping mall building server 200 and the camera 100 may each include a communication unit to communicate with an internal component or at least one external terminal through a wired/wireless communication network. Here, as the wireless Internet technology, wireless LAN (WLAN), Wibro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), High Speed Downlink Packet Access (HSDPA), IEEE 802.16, Long Term Evolution (Long Term Evolution: LTE), Wireless Mobile Broadband Service (WMBS), and the like may be included. In addition, as short-range communication technologies, Bluetooth, Wi-Fi, RFID (Radio Frequency Identification), infrared communication (Infrared Data Association: IrDA), UWB (Ultra Wideband), ZigBee, neighboring Magnetic field communication (Near Field Communication: NFC), ultrasonic communication (Ultra Sound Communication: USC), Visible Light Communication (VLC), BLE (Bluetooth Low Energy) may be included. In addition, as a wired communication technology, power line communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cable, and the like may be included.

FIG. 2 is a diagram showing the configuration of a shopping mall building server applied to a system for building a virtual reality shopping mall using an offline photographed image according to the present invention.

2, the shopping mall building server 200 includes a product registration unit 210, an image generation unit 220, a product matching unit 230, a VR shopping mall construction unit 240, and a VR space search unit 250. And an image update unit 260.

In the product registration unit 210, product information on the exhibited product is input and stored and managed. Product information may include product identification number, stereoscopic image, classification, name, price, standard, remaining amount, and instructions.

The identification number may be made of a product serial number or product number.

In addition, the three-dimensional image refers to an image taken at 360° of the product, and after acquiring a 2D image taken from multiple angles of the product, analyzing the multiple arrays by reflecting each absolute position of the acquired 2D image , This is a 3D model acquired image processed by image rendering using an algorithm of mathematical 3D matching theory.

Here, the product information may further include a 2D image used to generate a stereoscopic image.

When a new product is released or a product is changed due to an upgrade, the product registration unit 210 generates and registers and stores product information for the corresponding product.

The image generation unit 220 receives an image photographed from the camera 100 and generates a panoramic image using the received image.

In this case, when the camera 100 for photographing an image is configured as an omnidirectional camera (360° VR camera) capable of shooting 360°, the image generator 220 directly converts the image captured from the omnidirectional camera to a panoramic image. In the case of a camera (CCTV camera, etc.) that can be generated and photographed vertically and horizontally, the image generation unit 220 generates a panoramic image by overlapping an overlapping area among images captured by a CCTV camera. .

That is, since a part of the image captured by the CCTV camera includes an overlapping area, a partially overlapped area exists in each image captured by the plurality of cameras 100.

The image generator 220 generates a panoramic image by synthesizing the image captured by the camera 100 based on the overlapped area. In this case, the panoramic image may be configured to include a 360° region of top, bottom, left and right.

The panoramic image generated by the image generator 220 includes the entire area of the offline store. That is, all products displayed in offline stores are included in the generated panoramic image.

The product matching unit 230 detects a product included in the panoramic image generated by the image generating unit 220 and performs a function of matching the detected product with product information stored in the product registration unit 210.

The product matching unit 230 includes a product matching module 231, a product information display module 232, and a product purchase execution module 233.

The product matching module 231 matches the product image included in the generated panoramic image with product information stored in the product registration unit 210.

That is, the product matching module 231 must specify a product area in the panoramic image, detect a product image in the specified area, and match the detected product image with product information registered in the product registration unit 210. , The method of detecting a product in the panoramic image may be selected from a method using a product image, a method using characters marked on the product, or a method using a combination of the product image and marked characters.

Among the methods of detecting a product in a panoramic image, the method using a product image compares the 3D image or 2D image of the product registered in the product registration unit 210 to determine the position where the product is placed, and the determined area is an area of interest. After setting to, it may be configured to detect a product by removing a background area for the shape of the product image in the set region of interest.

In addition, the method of using the product image is to compare the image pattern and RGB value of the set region of interest with the image pattern and RGB value of the product information registered in the product registration unit 210, and the position and product of the displayed product in the panoramic image. Matching information.

Among the methods of detecting a product in a panoramic image, the method using the text marked on the product is to use the name and price of the product information.After determining the location where the product is placed, setting the determined area as the region of interest, the set above It may be configured to detect a product by detecting a character for the product name or price in the region of interest.

In this case, the method using the characters marked on the product may be configured to be detected using a guide sticker (a table in which the name and price, etc. are listed) displayed around the product or a character written on the packaging of the product.

The product matching module 231 detects and specifies an image or text of a product photographed from a panoramic image, and the specified image or text is matched with pre-stored product information.

In addition, the method of using the product image and the displayed text in combination is a method of searching and matching product information that simultaneously satisfies the image and text of the exhibited product, and the image of the previously stored product information and the product image included in the text and panoramic image And a method of matching characters.

Such a method of using the product image and the displayed text in combination has an advantage of minimizing the occurrence of errors when matching the displayed product with previously stored product information.

In this way, when a product is detected, the product matching module 231 converts and stores matching information including a position in the panoramic image for the detected product into a database.

The product information display module 232 displays product information on a selected product according to a selection event.

That is, product information about the product is displayed according to the user's selection (for example, an event of selecting a product with a mouse).

3 is a diagram showing an interface of an embodiment displayed by a product information display module in a system for constructing a virtual reality shopping mall using an offline photographed image according to the present invention.

Referring to FIG. 3, when a product is selected from a panoramic image, a product information interface displayed by a product information display module is displayed in a foreground area of the panoramic image.

The product information interface is expressed and arranged as a graphic user interface (GUI), and a product information column for a selected product and an operation button for rotating/enlarging a three-dimensional shape are displayed, respectively.

If necessary, the product information interface may further include additional information for guiding the operation form, usage form, and internal structure of the product, and may be configured to guide detailed information on the product.

The product information display module 232 displays on the display device pre-stored product information matched with the product included in the panoramic image according to the selection event. Accordingly, the user selects in the VR shopping mall space with only the captured image of the offline store. Product information is displayed according to the event, so it is easy to build a VR shopping mall space.

In addition, the VR shopping mall space is immediately changed due to the change of the products displayed in the offline store, and the products in the VR shopping mall space and the products displayed in the offline store are matched and reflected, so that the availability of products can be checked in the VR shopping mall space. In addition, since the product displayed in the VR shopping mall space is a product placed in the real space, it has the advantage of being able to purchase immediately regardless of inventory.

The product purchase execution module 233 performs a purchase procedure for the product displayed on the product display module.

4 is a diagram illustrating a payment processing interface displayed by a product purchase execution module in the system for constructing a virtual reality shopping mall using an offline photographed image according to the present invention.

That is, FIG. 4 is an interface displayed according to selection of a purchase icon displayed on the product information interface of FIG. 3, and is a payment processing interface for performing a purchase procedure for the selected product.

The procedure for payment processing may be performed through a separate payment processing server, and may be performed based on user log information, payment information, and delivery information of the payment processing server.

Accordingly, the payment processing server for processing payment may further include a database for recording a log of a user, a database for recording billing and payment information, and the like. Such a payment processing server may be performed through a series of procedures with a process for processing payment in connection with an external financial institution, or may be configured to be performed through a separate payment processing process in the shopping mall building server 200. .

If necessary, the product purchase execution module 233 may be configured to discount a price for a selected product including a discount rate and quantity for the selected product, or to purchase a plurality of single products.

In addition, as shown in FIG. 4, after moving the product selected by the shopping cart icon to the shopping cart, the payment processing interface may be configured to perform a lump sum payment for the products included in the shopping cart.

The VR shopping mall building unit 240 performs a function of building a shopping mall in a virtual reality space by linking the panoramic image generated by the image generating unit 220 with product information matched by the product matching unit 230.

The virtual reality space is a world of virtual reality in which a 3D realization algorithm and its language are used to create reality-like buildings, animals, and sculptures in cyberspace without being restricted by reality.

In the present invention, the VR shopping mall construction unit 240 includes the distance (depth) from the camera to the object for the panoramic image generated by the image generation unit 220, the location of the camera 100 that photographed the image, and the size of the offline store. , A 3D virtual reality shopping mall is created based on an area in which the offline store is divided into several areas and the relationship with adjacent stores.

At this time, the VR shopping mall construction unit 240 matches each product (object) included in the generated virtual reality space shopping mall with product information to establish a connection.

In addition, the VR shopping mall building unit 240 may be configured such that a virtual space can be expanded. That is, the VR shopping mall building unit 240 divides the offline store into several areas and connects the divided areas to establish one VR shopping mall. It may be configured to be connected with an expansion space such as a space or a play space, or a VR shopping mall space for a neighboring store.

When the extended space is configured, of course, the virtual reality shopping mall may be configured to be movable in connection with the extended space.

The virtual reality space shopping mall generated by the VR shopping mall building unit 240 is generated based on a photographed image of an actual offline store, and provides a feeling as if visiting an actual offline store.

5 is a diagram illustrating a VR shopping mall interface generated by a VR shopping mall construction unit in a system for building a virtual reality shopping mall using an offline photographed image according to the present invention.

A space movement selection icon 241 for space movement is displayed on the VR shopping mall interface representing the virtual reality space shopping mall generated by the VR shopping mall construction unit 240, and the spatial movement selection icon 241 is selected. Screen change is made by moving.

The VR space search unit 250 displays a virtual reality space built in the VR shopping mall construction unit according to an input event.

The virtual reality space shopping mall created by the VR shopping mall building unit 240 may be displayed on a head mounted display (HMD) or a monitor of the terminal 300.

When the virtual reality shopping mall is displayed on the monitor of the terminal 300, space movement, expansion, conversion, and product selection may be performed through an input event using an input device such as a mouse.

In addition, when a virtual reality shopping mall is displayed on the HMD, spatial movement, expansion, conversion, and product selection may be performed based on the movement of the HMD.

According to the above configuration, it is possible to construct a virtual reality shopping mall space for the displayed products by using the photographed video of the offline store, so that time and cost required for construction can be saved, and the constructed virtual reality shopping mall is an offline store. According to the structure of the product and the products on display, it has the advantage of creating a situation like visiting an offline store.

In such a configuration, when the displayed products are changed or replaced, the products included in the built virtual reality shopping mall must also be changed or replaced.

Accordingly, in the present invention, an image update unit 260 is included.

The image update unit 260 compares the panorama image generated by the image generation unit 220 with a previously stored panorama image, and when the panorama image generated by the image generation unit 220 is changed, the image generation unit 220 ) To transmit the update signal.

Receiving the update signal, the image generator 220 receives an image captured from the camera 100, and regenerates a panoramic image by connecting regions overlapping in the received image.

In addition, the VR shopping mall building unit 240 connects the panoramic image reproduced by the image generating unit 220 and product information matched by the product matching unit 230 to establish a shopping mall in a virtual reality space.

In this case, the product registration unit 210 should store product information on the replaced or changed product in advance.

Here, the image update unit 260 may be configured to compare a previously stored panorama image with a generated panorama image and transmit an update signal to the image generation unit 220 when the image is changed, but an artificial event, for example, For example, an update signal may be configured to be transmitted to the image generator 220 according to a user's update signal event. That is, it may be configured to build a shopping mall in a virtual reality space according to an input event of a manager.

According to this, a virtual reality shopping mall space can be automatically constructed according to changes, replacements, and location movements of exhibited products, thereby significantly reducing the time that products are applied to the virtual reality shopping mall, and making it easier to maintain the virtual reality shopping mall. There is an advantage.

The terminal 300 is a user terminal that is provided with services provided by the shopping mall building server 200, and includes a smart phone, a portable terminal, a mobile terminal, and a personal digital assistant. : PDA), PMP (Portable Multimedia Player) terminal, Telematics terminal, Navigation terminal, Personal Computer, Notebook computer, Slate PC, Tablet PC, Ultrabook (ultrabook), wearable devices, e.g., smartwatch, glass-type terminals, HMD (Head Mounted Display), etc.), Wibro terminals, Internet Protocol Television (IPTV) ) Terminals, smart TVs, digital broadcasting terminals, AVN (Audio Video Navigation) terminals, A/V (Audio/Video) systems, Flexible Terminals, and Head Mounted Disaply (HMD) terminals. Can be applied.

Next, a method of constructing a virtual reality shopping mall using an offline photographed image according to the present invention will be described.

6 is a flowchart illustrating a method of constructing a virtual reality shopping mall using an offline photographed image according to the present invention.

6, the method of constructing a virtual reality shopping mall using an offline photographed image according to the present invention includes an image generation step (S10), a product information matching step (S20), a VR shopping mall construction step (S30), and a VR shopping mall product. It comprises a management step (S40).

1. Image creation step (S10)

The image generating step S10 is a step of generating a panoramic image by receiving an image photographed from a camera and connecting regions overlapping in the received image.

That is, in the image generation step (S10), a video image photographed from a plurality of cameras 100 installed in an offline store is received by the shopping mall building server 200, and the shopping mall building server 200 uses the received video material to perform a panorama. This is the step of creating an image.

In the image generation step (S10), a panoramic image is generated using two algorithms according to an image photographed by the camera 100.

That is, when the camera 100 is a 360° VR camera (an omni-directional camera), the image generation step S10 generates a panoramic image by correcting the image captured from the 360° VR camera, and the camera 100 ) Is a one-way camera (CCTV camera), a panoramic image is created by overlapping the overlapping areas.

Since the image captured from the 360° VR camera has geometric distortion when converted into a flat image, a camera calibration process is required to accurately estimate parameters that cause distortion to correct the distortion.

Existing techniques for correcting distortion of fisheye lens images perform three-dimensional correction using a grid pattern. This is because it is practically impossible to shoot accurately from the front based on the center of the grid pattern.

An image photographed with a fisheye lens has a relatively shorter focal length than an image photographed with a general lens. On the other hand, a wide viewing angle can be secured with a view angle of 180° or more, and information including a lot of information can be obtained from the center.

The central portion of the geometrically distorted image is relatively enlarged than the image before the distortion, and the peripheral portion is small and blurred. Due to this characteristic, the geometrically distorted spiral pattern image has smaller and blurred dots toward the periphery.

Therefore, when the camera 100 is a 360° VR camera (an omnidirectional camera), in the image generation step (S10), in order to match the points between the images, the brightness value of each point of the geometrically distorted spiral pattern image is used. Thus, the center of each point is detected, and the central image and the peripheral image are corrected to generate a panoramic image.

In addition, when the camera 100 is a CCTV camera, in the imaging step (S10), an overlapping area is detected in images received from the cameras, and the overlapping area is overlapped by using a pattern or RGB value to create a panoramic image. Will be created.

In the present invention, the camera 100 is configured as a 360° VR camera (an omnidirectional camera) to receive a video from the shopping mall building server 200 through real-time shooting, and the shopping mall building server 200 is the received 360° VR video material. By generating a panoramic image by using, it is possible to generate a panoramic image more quickly while reducing the number of cameras installed.

2. Product information matching step (S20)

The product information matching step (S20) is a step of matching a product included in the panoramic image generated in the panoramic image generating step with product information corresponding to the product.

That is, in the product information matching step (S20), the product included in the panoramic image generated while the product information is registered in the product registration unit 210 and the product information registered in the product registration unit 210 are matched and connected. Step.

3. VR shopping mall construction step (S30)

The VR shopping mall building step (S30) is a step of constructing a shopping mall in a virtual reality space using the panorama image generated in the panorama image generating step and the product information matched in the product information matching step.

The VR shopping mall construction step (S30) includes a distance (depth) from a camera to an object with respect to the panoramic image generated by the image generation unit 220 through the VR shopping mall construction unit 240, and a camera 100 that photographs the image. It creates a 3D virtual reality shopping mall based on the location of, the size of the offline store, the area divided into several areas of the offline store, and the relationship with adjacent stores, as well as the VR shopping mall space built by panoramic images. It can be constructed by further including an extended space.

4. VR shopping mall product management step (S40)

In the VR shopping mall product management step (S40), when the panoramic image generated in the panoramic image generating step is changed by comparing it with a previously stored panoramic image, the panoramic image generated in the panoramic image generating step and the product matched in the product information matching step This is the stage of managing the deployed products by configuring a virtual reality space using information.

That is, the VR shopping mall product management step (S40) is performed when a product displayed in an offline store is changed or replaced, and is a step of managing to rearrange the replaced or changed virtual reality space in the VR shopping mall space. This is the step of making the product appear on the VR shopping mall.

The VR shopping mall product management step S40 may be configured to be performed whenever the panorama image generated in the panorama image generation step is changed by comparing it with a previously stored panorama image.

According to the present invention, it is possible to construct a virtual reality shopping mall space for exhibited products using photographed images of an offline store, thereby reducing the time and cost required for construction, and changing, replacing, and moving the displayed product. The virtual reality shopping mall space can be automatically constructed according to the like, so that the time for which the product is applied to the virtual reality shopping mall can be greatly reduced, and maintenance of the virtual reality shopping mall is easy.

Although the preferred embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and it should be understood that the scope of the present invention extends to the scope substantially equal to the embodiment of the present invention. Various modifications may be made by those of ordinary skill in the technical field to which the present invention pertains within the scope not departing from the spirit of the invention.

100: camera 200: shopping mall building server
210: product registration unit 220: image generation unit
230: Product matching department 240: VR shopping mall construction department
250: VR space search unit 260: Image update unit
300: terminal

Claims (5)

In the construction system of a virtual reality shopping mall comprising a plurality of cameras installed in an offline store to photograph an image, and a shopping mall building server configured to construct a virtual reality shopping mall by receiving images taken from the plurality of cameras,
The shopping mall building server,
Product registration unit for storing and managing product information including product identification number, stereoscopic image, classification, name, price, standard, remaining amount, and instructions including product serial number or product number for exhibited products;
An image generator configured to receive an image of all products displayed in an offline store through the camera, and generate a panoramic image using the received image;
A product matching unit detecting a product included in the panoramic image generated by the image generating unit and matching the detected product with product information stored in the product registration unit;
A VR shopping mall building unit for constructing a shopping mall in a virtual reality space by connecting the panoramic image generated by the image generating unit and product information matched by the product matching unit; And
A VR space search unit that displays a virtual reality space built by the VR shopping mall construction unit according to an input event;
It is composed including,
The shopping mall building server,
An image that compares the panoramic image generated by the image generation unit with a previously stored panoramic image, and transmits an update signal to the image generation unit when the panoramic image generated by the image generation unit is changed due to change, replacement, or location movement of an exhibited product. Update unit;
It is configured to further include,
The image generation unit,
Receives an image photographed from the camera based on an update signal of the image update unit, and regenerates a panoramic image using the received image,
The VR shopping mall building unit,
In a state in which product information on the replaced or changed product is previously stored in the product registration unit, a shopping mall is established in a virtual reality space by connecting the panoramic image reproduced in the image generating unit and the product information matched in the product matching unit,
When selecting a product from the panoramic image, the product information interface displayed by the product information display module is displayed in the foreground area of the panoramic image,
In the product information interface,
Product information for the selected product;
An operation button for rotating/enlarging the stereoscopic image; And
Additional information guiding the product's operation form, usage form, and internal structure;
Is included,
A space movement selection icon for space movement is displayed on the VR shopping mall interface expressing the virtual reality space shopping mall created by the VR shopping mall construction unit,
The VR shopping mall building unit,
When constructing a VR shopping mall by dividing the offline store into several areas and connecting the divided areas so that the virtual space can be expanded, a pre-product exhibition space, a rest area, or a play area for the products displayed It is configured to be movable in connection with the expansion space including the and the VR shopping mall space of the neighboring store,
The panoramic image generated by the image generating unit,
When the camera is a 360° VR camera, in order to match the points between the images, the center of each point is detected using the brightness value of each point of the geometrically distorted spiral pattern image, and the center image and the peripheral image are corrected to be generated. , When the camera is a CCTV camera, it is generated by detecting an overlapping area in images received from the cameras, and overlapping the overlapping area using a pattern or RGB value,
The product matching unit,
The product image included in the generated panorama image is matched with product information stored in the product registration unit, and the image or text of the product captured in the panorama image is detected and specified, and the specified image or text is previously stored. A product matching module to match information;
Product information display that displays pre-stored product information matched with the product included in the panorama image according to the selected event so that product information is displayed according to the selected event in the VR shopping mall space with only the video taken from the offline store. module; And
A product purchase module configured to enable multiple purchases for a single product and a price discount for a selected product, including the discount rate and quantity for the product displayed on the product information display module, and perform a purchase procedure for the selected product. ;
Including,
The product matching module detects a product in the panoramic image,
To search and match product information that satisfies the image and text of the exhibited product at the same time,
A background area for the shape of the product image in the set area of interest after determining the position where the product is placed by comparing the 3D image or 2D image of the product registered in the product register, setting the determined area as the area of interest A method using a product image to detect a product by removing it; And
To determine the position where the product is placed, to set the region of interest indicated on the guide sticker displayed around the product or the text on the packaging of the product as the region of interest, and then detect the text of the product name or price in the set region of interest. The method using the characters on the product;
Minimizes the occurrence of errors when matching the displayed product and previously stored product information by using a combination of
As the virtual reality shopping mall space is automatically established according to the change, replacement and location of the exhibited products, the time for the product to be applied to the virtual reality shopping mall is reduced, and the VR shopping mall space is immediately reduced by the change of the displayed products in the offline store. It is possible to check whether the product is in stock in the VR shopping mall space as it is changed and the products in the VR shopping mall space and the products displayed in the offline store are matched and reflected.
The camera is installed in an offline store, and is composed of a plurality of cameras so as to photograph the offline store, transmit the photographed video, and shoot the entire area of the offline store, and a camera adjacent to the area photographed by one selected camera Are arranged so that part of the area covered by is overlapping,
The shopping mall building server and the camera each include a communication unit to communicate with an internal component or at least one external terminal through a wired or wireless communication network,
Among the product information stored and managed in the product register, the 3D image acquires a 2D image photographed from multiple angles of the product, analyzes the multiple arrays by reflecting each absolute position of the acquired 2D image, and then performs a mathematical 3D registration. It is a 3D model acquired image processed by image rendering using the algorithm of the theory, and the product information further includes a 2D image used to generate a stereoscopic image,
The VR shopping mall building unit,
For the panoramic image generated by the image generation unit, based on the distance from the camera to the object, the location of the camera where the video was photographed, the size of the offline store, the area divided into several areas of the offline store, and the relationship with adjacent stores, etc. It creates a 3D virtual reality shopping mall,
The virtual reality shopping mall created by the VR shopping mall building unit is displayed on a head mounted display (HMD) or a monitor of the terminal, but when the virtual reality shopping mall is displayed on the monitor of the terminal, an input device of a mouse is used. Space movement, expansion, conversion and product selection are made through input events, and when a virtual reality shopping mall is displayed on the HMD, space movement, expansion, conversion and product selection are made based on the movement of the HMD.
A panoramic image generation step of receiving a video object photographed from a plurality of cameras installed in an offline store at a shopping mall building server, and generating a panorama image using the received video object; The product included in the panoramic image generated in the panoramic image generating step and the product included in the panoramic image generated in the panoramic image generation step to match and connect the product included in the panoramic image generated while the product information is registered in the product registration unit with the product information registered in the product registration unit. Product information matching step of matching product information corresponding to the product; For the panoramic image generated by the image generation unit through the VR shopping mall building unit, the distance from the camera to the object, the location of the camera where the video was photographed, the size of the offline store, the area divided into several areas and adjacent stores VR that builds a shopping mall in a virtual reality space using the panorama image generated in the panorama image generation step and the product information matched in the product information matching step to create a three-dimensional virtual reality shopping mall based on the relationship with Shopping mall construction step; And when the panoramic image generated in the panoramic image generating step is changed by comparing it with a previously stored panoramic image, a virtual reality space is generated using the panoramic image generated in the panoramic image generating step and the product information matched in the product information matching step. VR shopping mall product management step of managing the configured and arranged products; The shopping mall is built through
In the panoramic image generation step, a panoramic image is generated using two algorithms according to an image photographed from a camera, and when the camera is a 360° VR camera, the panoramic image generation step is performed by the 360° VR camera. A panoramic image is created by correcting a video object, and if the camera is a CCTV camera that photographs one direction, a panoramic image is created by overlapping the overlapping area, and if the camera is a 360° VR camera, the points between the images are matched. To do this, the center of each point is detected by using the brightness value of each point of the geometrically distorted spiral pattern image, and a panoramic image is generated by correcting the central image and the peripheral image. If the camera is a CCTV camera, the cameras It detects an overlapping area in the images received from and creates a panoramic image by overlapping the overlapping area using a pattern or RGB value, and in the VR shopping mall construction step, the product displayed in the offline store is changed or replaced. It is performed in the case, and the product displayed in the offline store is displayed in the VR shopping mall so that the replaced or changed virtual reality space is rearranged in the VR shopping mall space, and the panorama image generated in the panorama image generation step is compared with the previously stored panorama image. A system for constructing a virtual reality shopping mall using an offline photographed video, characterized in that it is performed every time there is a change. delete delete delete delete

Images