WO2016098091A1

WO2016098091A1 - Network integrated interactive display system with augmented reality

Info

Publication number: WO2016098091A1
Application number: PCT/ID2015/000003
Authority: WO
Inventors: Michel Budi WIRJATMO
Original assignee: Pt. Wir Asia
Priority date: 2014-12-15
Filing date: 2015-08-27
Publication date: 2016-06-23

Abstract

Disclosed is a system and method for retrieving and displaying additional information with respect to an object. A user uses a client device to capture an image of the object. Image data is transmitted to a server which recognizes the object and retrieves and returns related content which is then displayed to the user.

Description

NETWORK INTEGRATED INTERACTIVE DISPLAY SYSTEM WITH

AUGMENTED REALITY

Technical Field of Invention

This invention is related to network integrated interactive display system with augmented reality used by user to recognize an object (such as: product) in an area/location.

Background of Invention

As we have already known, the development of recognition of a product through advertisement has grown rapidly. Display media such as television, cellular telephone and other display media, plays an important role to recognize a whole product when the user desires to carry out further recognition out of a product.

Producers compete to bring near their product to consumers using any ways, however, most of them are using media that are considered conventional and commonly known, namely an advertisement that is one directional (i.e.: non interactive / passive participation) by way of looking through traditional or conventional media such as television, or newspaper.

Looking at an existing example happing in a shopping area, with a well-known particular supermarket, therefore, it is expected that the supermarket provides enough information about its products offered to the customers. The purpose is clear, namely the visiting customer are able to obtain information about the particular product or as far as communicating about the facilities offered by the supermarket . At the same time, Information provided by the supermarket to customers must be targeted in order to gain maximum impact .

Commonly seen in large supermarkets, advertisements in such area are provided utilizing hanging televisions, so that consumers and customers entering the supermarket only see the product advertisement.

Such solution is surely not advantageous because apart from only in one directional, customer seeing the advertisement are not directly engaged and worst, cannot recognize the information of product shown as they are not of their personal interest. Other forms of solution to promote product recognition, generally, are also utilizing a sensor device used only to recognize the price of a product, or to trigger another non-interactive audio and/or visual content located near that product .

This is also disadvantageous since the customers can only either see the printed price, without being able to recognize further the products or seeing another similar advertisement like the ones they see in other conventional media ..

Although the product packaging contains basic information, however, additional information regarding the product, for example on-going discount, or lottery or any other detail information will have a role to bring the product closer to customer, or even- information related to supermarket such as price discount, events to be held in the supermarket .

The disadvantageous things mentioned above, by means of this invention, will be settled by applying a device that is furnished by an interactive display, user can put forth a desired product, with the interactive device that can recognize the product, user may obtain information about the product utilizing interactive technology like augmented reality where the information content is delivered after the recognition process by means of computer; to be then displayed by means of display device (such as television or cellular phone screen) in augmented reality.

Utilizing also some media that are already widely used at present, the impact will be greater if the technology is combined with Augmented Reality technology. The Augmented Reality itself is a technology that combines several digital elements such as 3D animation, video, text, voice, and figure into the real world. With the utilization of this Augmented Reality technology in display media that can be used for marketing and advertisement, therefore, indirectly user will get an advertised product to be hold in front of a camera thus triggering the digital content displaying the information of such product shown on the display device. Indirectly, this will make an advertiser to get further interaction between the user in this case prospective buyer with the product, because the user has hold/taken the product to interact in the screen, increasing better opportunity for he/she to gain interest to purchase the product.

To reach different spots (for example: supermarket) this interactive device will be deployed and customer is able to access the interactive device to recognize the desired products with interactive technology such as augmented reality . Moreover, by means of mobile display such as cellular phone display, user may use it similarly like those of the interactive display screen. Integrated with wireless network, product recognition by means of mobile display (e.g. Cellular phone) and receives back in the form of data of product with augmented reality and the existing content set by brand owner will be very practical and portable .

Brief Description of Invention

This invention is related to network integrated interactive display system with augmented reality used by user to recognize a product in an area / a location.

This invention aims to facilitate the user in recognizing an object (such as product) by means of device by obtaining information about the particular object (e.g.,: product) with augmented reality only by showing the object to sensor or camera, therefore, the pre-set contents such as product description or campaign or any other contents will be displayed by augmented reality by means of display screen that is placed in particular spots and also in cellular phone that can be carried in an area / a location.

This invention shall be a network-integrated interactive display system, a device installed with interactive technology such as Augmented Reality in Display Media and system to add or delete content via online (centralized) in which all of such things are controlled by means of a server. Client's computer shall be used as a facility to process the figure captured by sensor or camera and to process the figure to be sent in the form of digital content into the server .

Server containing database and software will be triggered by the presence of digital content and resend it to the client's computer in the form of information on data of product into the client's computer and has been accompanied by interactive technology like augmented reality.

By means of display screen in the client's computer, the user seems to interact directly (live) with the object (or product) shown to the sensor or camera and the display screen directly shows the data of such object that is adhered to the object wherever the object moves.

Network feature enables the user to access by means of cellular phone other than using the display screen in which the position is specified.

For the same method using camera that is integrated to the mobile device such as cellular phone, the user may enjoy a direct interaction (live) from cellular phone.

Moreover, management or owner of location/area in such territory may also utilize as communication tool such as to advertise facility offerings for customer directly through the display media be it on screen or mobile device such as cellular phone. Brief Description of Figure

Figure 1. Is the schematic of components embedded in the Augmented Reality in Display Media Figure 2. Is the schematic of integrated device of Augmented Reality in Display Media in the Augmented Reality Content Management System

Figure 3. Is the schematic of work method of object recognition and data/content storage and delivery of Augmented Reality in the Interactive Display System

Figure 4. is the schematic of content data flow of Augmented Reality

Figure 5. is the perspective view of Augmented Reality in Display Media device in shelving

Figure 6. is the perspective view of Augmented Reality in Display Media device in pillar

Figure 7. is the perspective view of Augmented Reality in Display Media device in booth Figure 8. is the perspective view of Augmented Reality in Display Media device in robot toy

Complete Description of Invention

This invention is related to the network-integrated interactive display system with interactive technology such as augmented reality used by user to recognize an object (e.g. Product) in an area / a location.

The Augmented Reality in Display Media device has main components namely server, network modules, client's computer, and mobile device, here represented by cellular phone.

In the client's computer, there are components such as screen panel, touch screen, processor, camera, network modules, speaker, and sensor, where the components have their respective functions.

Screen panel has the function to display visualization of digital content, touch screen panel is used as an input device that operates by the presence of touch in screen display using fingers, processor is used to control and operate the screen panel and owns or has been connected to memory for storage of digital content that will be presented in at least one display digital panel as well as data or content and operative function, camera that functions as a figure catcher (product packaging) that will be recognized as tracking functioning as a trigger to activate the digital content in Augmented Reality technology, network module used as a communication device with the Content Management System, and sensor that is used to detect user standing in front of the Augmented Reality in Display Media device.

To use this device, user is obliged to face a product packaging or other items that can be used as a trigger to activate the Augmented Reality system existed in this device. When the object has been faced in front of the camera, therefore, the camera will start its function to capture and recognize the figure shown or shape of the object by means of client's computer existed in every sensor and then sent in the form of digital data into the server . Server that recognizes the digital data triggers and calls the digital content that has been prepared in the database and server will send the digital content (such as product info, animation, or others) to the client's computer and visualizes the content through the display screen by using Augmented Reality technology so that digital content is united with product packaging.

The implementation of this Augmented Reality in Display Media device may be applied to various spots such as shelving, pillar/wall, booth, or even a more mobile placement platform such as robot. The placement of this device may be adjusted to requirements.

Moreover, with the application of augmented reality usable in mobile devices such as cellular phone, with the application of wireless network by means of digital content module detected by the mobile device, the application that has been furnished by augmented reality can be activated for the same thing with the display screen applicable only in one area/location when it is connected to the captured Wi-Fi signal.

With the existence of this application in various mobile device such as cellular phone, in this case includes smartphones, tablets, etc., it will facilitate mobile user to access the contents and use it to recognize the particular object for more interactive contents and information utilizing augmented reality.

On the other side, with the existence of this network integrated system, business owner may use it to deliver information to be given to the masses by means of display screen and mobile devices phone simultaneously.

This network integrated interactive display with augmented reality will not run without any software containing the Content Management System that functions to arrange image tracking data and digital content in the form of 2D/3D model, 2D/3D animation, digital text, video, image/photograph, and audio. All of these data will be stored in a database server; therefore not all of them has to be stored in the Augmented Reality in Display Media device.

Not only as storage media of data of Augmented Reality, but this server is also able to function to store statistical data obtained from each Augmented Reality in Display Media such as data on status of Augmented Reality in Display Media if experiencing problems (on/off status of device) , data on user interaction with the specific object that can be seen based on time (hour, day, week, month, and year) , data on user interaction with a product based on location (urban village, sub district, city, regency, and province), and other statistical data.

Claims

Network integrated interactive display system with augmented reality shall include:

at least one server or more;

a client's computer shall include at least one screen panel ,

at least one touch screen panel. Touch screen panel is used as an input device that operates with the touch in screen display by using fingers,

at least one camera. The camera that functions as a figure capture pre-defined object to be recognized as tracking; functioning as a trigger to activate the digital content with Augmented Reality technology, at least one internal or external speaker,

at least one sensor used to detect the user located in front of the Augmented Reality in Display Media device,

at least one processor to control and operate the screen panel and owns or has been attached with memory to save digital content that will be presented in at least one display digital panel as well as data or content and operative function;

at least one network module used as a communication device with the Augmented Reality Content Management System, at least one cellular phone used as facility for user to be able to access directly the content of Augmented Reality on mobile;

Network integrated interactive display system with augmented reality as mentioned in claim number one (1) where the display in screen panel will be placed in a booth and portable (mobile) such as shelfing rack, pillar, booth, and other mobile platform such as robot.

Network integrated interactive display system with augmented reality as mentioned in claim number one (1) where the applied mobile devices such as cellular phone shall include the cellular device across cellular programs connected to mobile network and/or Wi-Fi signal captured in the area / location.

Network integrated interactive display system with augmented reality as mentioned in claim number one (1) where there is at least one server placed in the data center, number of server depends on the number of Augmented Reality in Display Media used and also depends on the spread of Augmented Reality in Display Media device in several locations.

Work method of network integrated interactive display with augmented reality in the client's computer, shall include:

to activate standby condition mode in media. Display is in standby mode to detect when there is an individual in front of the device, automatically the device will call the individual standing in front of it with audio and/or visual; to capture figure of object using camera directed by user, therefore, software of Augmented Reality in Display Media will recognize figure and/or shape of the object captured by using the camera; to display digital content in the form of Augmented Reality of a product in the Augmented Reality in Display Media device;

6. Work method of network integrated interactive display with augmented reality in the server, shall include: to receive digital content of figure and/or shape of the object (such as packaging of a product) from the client's computer to undertake authentication in the user's authentication module; to carry out addition, editing, deleting and saving tracking materials (figure of the object) ; to carry out addition, editing, deleting and saving digital content permanently that has passed tracking material; to arrange schedule of digital content update; to arrange the remote function to each Augmented Reality in Display Media in the client's computer;