KR20230138813A - System and method for providing metaverse fairs based on analysis of buyer's intersests - Google Patents

Abstract

According to embodiments of the present invention, the present invention relates to a system and a method for providing a metaverse exhibition by analyzing the interest of a buyer. The method of the present invention comprises the steps of: transmitting, to an electronic device, metaverse exposition data to display an accessible metaverse pavilion in a metaverse exposition including a metaverse space on the electronic device accessing the metaverse exposition; obtaining pre-input information on a buyer obtained before entering the metaverse pavilion, arranging a plurality of booths in an array customized for the buyer on the basis of the pre-input information; transmitting the metaverse exposition data to display an arranged booth array to the electronic device; obtaining visit input data and facial image data of the buyer during a visit to a metaverse booth, and generating intermediate matching information on the basis of the visit input data and facial image data of the buyer; determining the buyer of interest on the basis of the pre-input information and the intermediate matching information; and transmitting information on the determined buyer of the interest to a corresponding participating company. Therefore, the satisfaction of a participating company can increase.

Description

Metaverse fair provision system and method based on buyer interest analysis {SYSTEM AND METHOD FOR PROVIDING METAVERSE FAIRS BASED ON ANALYSIS OF BUYER'S INTERSESTS}

Embodiments of the present application relate to technology for providing a metaverse exhibition, and more specifically, to a system and method for providing a customized metaverse exhibition to buyers by analyzing the interests of buyers.

After COVID-19, the holding of expos, which require an unspecified number of people to gather in one space, has generally declined.

To overcome this, attempts to replace virtual fairs in online spaces have increased, but the following limitations exist.

Some existing virtual exhibition methods provided web-based platforms. On this web-based platform, products are pre-checked based on images, text-based consultations are requested and conducted, and general video meetings between buyers and companies are linked.

The main function of the expo is to easily discover new companies from specific concepts and spaces even without prior information. This is because it saves buyers time, money, and effort in searching. For participating companies, the key point is that they can promote their companies and products more efficiently through a schedule of intensive visits by buyers of various nationalities and backgrounds to areas that need to be promoted to an unspecified number of people. Therefore, it is difficult to satisfy the most core needs of buyers and participating companies by providing image information in advance or hosting a video conference afterwards.

Some other existing virtual exhibition methods have only improved in the visual effect area by modeling the virtual exhibition in 3D space.

Accordingly, there is an increasing demand from fair stakeholders, such as government agencies and various organizations that promote small and medium-sized enterprises and exports, for a new type of fair that can more effectively replace the existing online/offline fair.

Korean Patent Publication No. 10-2020-0068084 (2020.06.15.)

In order to solve the above-mentioned problems, this application implements an exhibition space according to the theme desired by buyers, deploys and utilizes artificial intelligence (AI) counselors, provides realistic product demonstration and experience in the metaverse space, We aim to provide a system and method for providing a metaverse exhibition based on buyer interest analysis that can recommend and sort booths based on buyer interest.

The method of analyzing buyer interest and providing a metaverse exhibition according to one aspect of the present application may be performed by a server. The method includes transmitting metaverse expo data for displaying metaverse pavilions that can be entered in a metaverse expo consisting of a metaverse space to an electronic device that has accessed the metaverse expo; Obtaining pre-input information of buyers obtained before entering the metaverse hall, and arranging a plurality of booths in an arrangement customized to the buyer based on the pre-input information; Transmitting metaverse expo data for displaying the arranged booth arrangement to the electronic device; Obtaining a buyer's visit input data and face image data during a visit to the Metaverse booth, and generating intermediate matching information based on the buyer's visit input data and face image data; determining buyers of interest based on the pre-input information and intermediate matching information; And it may also include a step of delivering information on the determined interested buyer to the relevant participating company.

In one embodiment, the step of arranging a plurality of booths in an arrangement customized to the buyer based on the pre-input information includes obtaining the pre-input information of the buyer through the buyer's electronic device before entering the metaverse hall, For each of the plurality of booths included in the metaverse pavilion, calculating a buyer's pre-matching score based on the buyer's pre-input information, determining the placement ranking of each booth according to the buyer's pre-matching score, And it may include arranging a plurality of booths according to the determined arrangement order. The step of calculating the buyer's dictionary matching score includes sorting similar keywords to the buyer keyword entered by the buyer in the seller keyword set and the expo keyword set, and quantifying the similarity between the sorted similar keywords and the buyer keyword. It may also include calculating the buyer's preliminary matching score for participating companies related to the keyword.

In one embodiment, the step of arranging a plurality of booths according to the arrangement order may be arranging the booths of the corresponding pavilion based on at least one of the location of the buyer and the direction of movement of the buyer on the metaverse exhibition.

In one embodiment, the method includes rearranging the remaining booth array of the entered metaverse pavilion based on intermediate matching information; And it may further include transmitting metaverse expo data for displaying the updated booth arrangement to the electronic device.

In one embodiment, the intermediate matching information includes an intermediate matching score, and the step of generating the intermediate matching information includes inputting facial image data of the buyer captured while interacting with the metaverse fair from the buyer's electronic device and receiving the buyer's visit input data, calculating the buyer's present factor and explicit factor based on at least one of the buyer's visit input data and the buyer's face image data, and the buyer's present factor and It may also include calculating the intermediate matching score of buyers for companies participating in the Metaverse Expo based on external factors.

In one embodiment, the implicit factor includes biometric information, and the step of calculating the implicit factor and the explicit factor of the buyer based on at least one data of the buyer's visit input data and the buyer's face image data includes the face Measuring one or more biosignals of heart rate and heart rate variability of the buyer from image data; It may also include generating biometric information of the buyer based on the biometric signals.

In one embodiment, measuring one or more biosignals of the buyer's heart rate and heart rate variability may include detecting a face area from the face image data; extracting a G-channel signal from the face area; Converting the G-channel signal to the frequency domain; and measuring one or more biosignals of heart rate and heart rate variability based on the converted frequency signal.

In one embodiment, the biometric information includes one or more of a high-frequency region of heart rate, a low-frequency region, and stress level, and the step of generating the biometric information of the buyer based on the biosignal includes using a pre-stored band pass filter. Classifying the frequency region of the heart rate into a high frequency region and a low frequency region using; And based on the ratio of the high-frequency area and the low-frequency area, it may include calculating the buyer's stress level while interacting with the metaverse fair based on the ratio of the high-frequency area and the low-frequency area.

In one embodiment, the appearance factor includes one or more of the buyer's visit information, emotion information, concentration information, and gaze information while interacting with the metaverse exhibition, the buyer's visit input data, and the buyer's face. Calculating the buyer's intrinsic factors and explicit factors based on at least one piece of video data includes calculating the visit information based on the buyer's visit input data entered for a contract at the Metaverse Expo; calculating emotional information of the buyer from the facial image data; calculating gaze information of the buyer from the face image data; and calculating concentration information of the buyer based on one or more of gaze information and biometric information, and the emotion information.

In one embodiment, the emotional information includes positive or negative, and the step of calculating the emotional information of the buyer from the facial image data includes extracting facial landmarks from the input image and using a method previously learned to infer the buyer's emotion. The buyer's emotion information may be calculated using an emotion classification model.

In one embodiment, calculating the gaze information of the buyer from the face image data includes detecting a face area in the face image data of the buyer and extracting a plurality of landmarks from the face area; Detecting the buyer's eye portion and pupil portion based on a plurality of landmarks; Estimating the reference direction of gaze by estimating the direction of the buyer's face based on a plurality of landmarks; calculating the position of the buyer's gaze based on at least one of the gaze reference direction, eye position, and pupil position; and calculating at least one of gaze fixation time and gaze fixation frequency based on the object area displayed on the screen of the electronic device and the calculated gaze position.

In one embodiment, the concentration information includes a concentration or immersion value, and calculating the concentration information of the buyer determines the degree to which the buyer is immersed in a specific object based on at least one of the gaze information and biometric information. Quantifying steps; When the emotional information includes positivity, calculating an immersion value using a numerical value; And when the emotional information includes negativity, it may include calculating a concentration value using a numerical value.

In one embodiment, the step of determining the placement ranking based on the intermediate matching score determines the placement ranking based on the degree of similarity to the participating company being visited. If the buyer's emotion is positive during the visit, the degree of similarity is The higher the similarity, the higher the rank assigned, and if the buyer's emotion is negative during the visit, the lower the degree of similarity, the higher the rank may be assigned.

In one embodiment, the step of transmitting metaverse expo data for displaying an updated booth arrangement to the electronic device may involve the buyer's electronic device exiting the visited metaverse booth to calculate an intermediate matching score.

In one embodiment, the step of determining the buyer of interest may include acquiring post-interest information of the buyer after exiting the metaverse and determining the buyer of interest based on the pre-input information, intermediate matching information, and post-interest information. there is.

In one embodiment, the step of determining an interested buyer based on the pre-input information, intermediate matching information, and post-interest information includes determining the buyer's interest based on at least one of a pre-matching score, an intermediate matching score, and a post-matching score. It may also include calculating a final matching score, and using the final matching score to determine buyers interested in the participating company. The final matching score may be calculated based on the values of the pre-matching score, intermediate matching score, and post-matching score, and a preset weight for each score.

In one embodiment, a higher weight may be assigned to the intermediate matching score or the post-matching score than the pre-matching score.

A computer-readable recording medium according to another aspect of the present application may record a program for performing the method of providing a metaverse exhibition by analyzing buyer interest according to the above-described embodiments.

According to another aspect of the present application, a system that induces buyers to purchase by analyzing buyers' interest in objects on the Metaverse Expo consists of a Metaverse Pavilion that can be entered at the Metaverse Expo consisting of a Metaverse space. displayed on an electronic device accessed, acquire face image data of the buyer by filming the buyer's face while the buyer interacts with the metaverse fair, and visit input data entered while interacting with the metaverse fair. Metaverse expo data for displaying metaverse pavilions that can be entered at a metaverse expo consisting of an acquired electronic device and a metaverse space to the electronic device that accesses the metaverse expo is transmitted to the electronic device, and entry into the metaverse pavilion is performed. Obtain previously obtained buyer's pre-input information, arrange a plurality of booths in an arrangement customized to the buyer based on the pre-input information, and send metaverse expo data for displaying the arranged booth arrangement to the electronic device. transmits, acquires the buyer's visit input data and face image data during the visit to the Metaverse booth, generates intermediate matching information based on the buyer's visit input data and face image data, and uses the pre-input information and intermediate matching information It may also include a server configured to determine interested buyers based on the information, and to deliver information on the determined interested buyers to the participating companies.

The metaverse exhibition provision system based on buyer interest analysis according to one aspect of the present application arranges booths based on the pre-matching score, intermediate matching score, and/or post-matching score between buyers and participating companies. In particular, the system can increase buyer convenience by analyzing buyers' needs more accurately using intermediate matching scores based on the buyer's activity data and emotional data while visiting the booth within the metaverse space.

In addition, the system can increase the satisfaction of participating companies with the fair by conveying the interest of buyers with a high degree of matching to participating companies so that participating companies do not miss out on those buyers.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

In order to more clearly explain the technical solutions of the embodiments of the present invention or the prior art, drawings necessary in the description of the embodiments are briefly introduced below. It should be understood that the drawings below are for illustrative purposes only and not for limiting purposes of the present specification. Additionally, for clarity of explanation, some elements may be shown in the drawings below with various modifications, such as exaggeration or omission.
Figure 1 is a schematic diagram of a metaverse exhibition providing system based on buyer interest analysis, according to one aspect of the present application.
Figure 2 is a schematic block diagram of an electronic device according to an embodiment of the present application.
Figure 3 is a schematic block diagram of a server according to an embodiment of the present application.
Figure 4 shows a metaverse exhibition displayed on an electronic device, according to an embodiment of the present application.
Figure 5 shows a booth arrangement to be provided to multiple interested buyers, according to an embodiment of the present application.
Figure 6 is a schematic diagram of a process for calculating a dictionary matching score according to an embodiment of the present application.
Figure 7 is a layout diagram of an exhibition booth according to an embodiment of the present application.
Figure 8 is a diagram explaining the buyer's metaverse exhibition collection data according to an embodiment of the present application.
Figure 9 is a flowchart of a process for calculating gaze information using a gaze tracking model, according to an embodiment of the present application.
Figure 10 is a flowchart of a process for calculating heart rate information from a face image, according to an embodiment of the present application.
Figure 11 is a flowchart of a method for providing a metaverse exhibition based on buyer interest analysis, according to another aspect of the present application.

Hereinafter, embodiments of the present application will be examined in detail with reference to the drawings.

However, this disclosure is not intended to limit the disclosure to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of the disclosure. . In connection with the description of the drawings, similar reference numbers may be used for similar components.

In this specification, expressions such as “have,” “may have,” “includes,” or “may include” refer to the corresponding features (e.g., numerical values, functions, operations, steps, parts, elements and/or components). It refers to the presence of components such as etc.) and does not exclude the presence or addition of additional features.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Expressions such as “first,” “second,” “first,” or “second” used in various embodiments may modify various elements regardless of order and/or importance, and limit the elements. I never do that. The above expressions can be used to distinguish one component from another. For example, the first component and the second component may represent different components, regardless of order or importance.

The expression “configured to” used in this specification may mean, for example, “suitable for,” “having the capacity to,” depending on the situation. ,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured (or set) to” may not necessarily mean “specifically designed to” in terms of hardware. Instead, in some situations, the expression “device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” refers to a processor dedicated to performing those operations (e.g., an embedded processor), or executing one or more software programs stored on a memory device. By doing so, it may mean a general-purpose processor (eg, CPU or application processor) capable of performing the corresponding operations.

In this specification, “metaverse” is a compound word of ‘Meta’, meaning processing and abstraction, and ‘Universe’, meaning the real world, and refers to the existing virtual reality environment. As a more advanced concept than the term, it refers to a virtual space that provides the same or similar experience as the non-real world such as the web and the Internet integrated with the real world. The metaverse may, for example, be implemented as a space that provides augmented or mixed reality to users, but is not limited to this.

Figure 1 is a schematic diagram of a metaverse exhibition providing system based on buyer interest analysis, according to one aspect of the present application.

Referring to FIG. 1, the buyer interest analysis-based metaverse exhibition providing system 1 includes a first electronic device 110; electronic device 110; telecommunication networks (120); and server 130.

The buyer interest analysis-based metaverse exhibition providing system (1) according to embodiments may be entirely hardware, entirely software, or may have aspects that are partly hardware and partly software. For example, a system may collectively refer to hardware equipped with data processing capabilities and operating software for running it. In this specification, terms such as “unit,” “module,” “device,” or “system” are intended to refer to a combination of hardware and software driven by the hardware. For example, the hardware may be a computing device capable of processing data, including a central processing unit (CPU), a graphics processing unit (GPU), or another processor. Additionally, software may refer to a running process, object, executable, thread of execution, program, etc.

The telecommunication network 120 provides a wired/wireless telecommunication path through which the electronic device 110, the electronic device 110, and the server 130 can transmit and receive data with each other. The telecommunication network 120 is not limited to a communication method according to a specific communication protocol, and an appropriate communication method may be used depending on the implementation. For example, when configured as an Internet Protocol (IP)-based system, the telecommunications network 120 may be implemented as a wired and/or wireless Internet network, and may include an electronic device 110, an electronic device 110, and a server ( When 130) is implemented as a mobile communication terminal, the telecommunication network 120 may be implemented as a wireless network such as a cellular network or a wireless local area network (WLAN) network.

The system 1 acquires and obtains the buyer's pre-input data prior to entering the Metaverse fair through the buyer's electronic device 110, and various aspects of raw data about the buyer interacting with the Metaverse fair. The raw data and the buyer's post-entry data input after leaving the Metaverse Expo are provided to the server 130 through the telecommunication network 120, and information is provided to the user of the electronic device 110 from the server 130. We can also provide customized exhibition models.

In addition, the system 1 provides participating company data for implementing the Metaverse Expo through the electronic device 110 of the participating companies to the server 130 through the telecommunication network 120, and the server 130 Information on buyers who are interested in the user of the electronic device 110 may be provided.

The electronic device 110 is implemented as a terminal device configured to transmit/receive data to and from the server 130 through the telecommunication network 120. The electronic device 110 includes at least one processor capable of processing data, a memory for storing data, and a communication unit for transmitting/receiving data. The electronic device 110 may be, for example, a laptop computer, notebook, other computing device, tablet, cellular phone, smart phone, smart watch, smart glasses, head mounted display (HMD), other mobile device, or other wearable device. It may be possible.

Figure 2 is a schematic block diagram of an electronic device according to an embodiment of the present application.

Referring to FIG. 2, the electronic device 110 includes a memory 111, a communication unit 112, a processor 113, a camera 114, an input device 115, and an output device 116.

The memory 111 is connected to the processor 113 and can store data such as basic programs for operation of the processor 113, application programs, setting information, and information generated by operations of the processor 113. The memory 111 may be comprised of volatile memory, non-volatile memory, or a combination of volatile memory and non-volatile memory. Additionally, the memory 111 may provide stored data according to the request of the processor 113.

In certain embodiments, the memory 111 may store a program (or application) for performing a metaverse exhibition providing system or method based on buyer interest analysis. The memory 111 provides stored data related to programs for executing the system and method according to instructions of the processor 113.

Additionally, the memory 111 may permanently or temporarily store data received from the communication unit 112, data captured by the camera 114, and data generated or processed by the processor 113. In the specific embodiments, the memory 111 may store metaverse exhibition data received through the communication unit 112, buyer's face image data captured by the camera 114, or computational data generated by the processor 113. It may be possible.

The memory 111 includes HDD (Hard Disk Drive), SSD (Solid State Drive), CD (Compact Disc), RAM (Random Access Memory), Rom (Read Only Memory), and various types of data that store data permanently, semi-permanently, or temporarily. It may also include other storage devices.

The communication unit 112 is connected to the processor 113 to transmit and receive data, and can transmit and receive data with external devices such as another electronic device 110 or the server 130. All or part of the communication unit 113 may be referred to as a transmitter, receiver, transceiver, communication unit, communication modem, or communication circuit. The transceiver 112 uses wired access systems and wireless access systems such as the IEEE (institute of electrical and electronics engineers) 802.xx system, IEEE Wi-Fi system, 3rd generation partnership project (3GPP) system, and 3GPP LTE (long term evolution) system. , 3GPP 5GNR (new radio) system, 3GPP2 system, and Bluetooth can support at least one of various wireless communication standards.

The processor 113 may be configured to implement the procedures and/or methods proposed in the present invention. The processor 113 may determine at least one executable operation of the electronic device 110 based on information determined or generated using a data analysis algorithm. And, the processor 113 can control the components of the electronic device 111 to perform the determined operation. To this end, the processor 113 may request, retrieve, receive, or utilize data from the memory 111, and the electronic device 110 may execute a predicted operation or an operation determined to be desirable among the at least one executable operation. ) components can be controlled.

When linkage with an external device such as the server 130 is necessary to perform a determined operation, the processor 113 generates a control signal to control the external device and generates a control signal to the external device through the communication unit 112. control signals can be transmitted.

The processor 113 visits booths at the Metaverse Expo and controls the overall operations of the electronic device 110 to implement interactive tasks between participating companies and buyers. In the specific embodiments, at least one processor 113 obtains image data about the buyer through the camera 114. Additionally, at least one processor 113 may be further configured to acquire biometric information through predetermined processing.

The processor 113 may be implemented with at least one processor. The processor 113 may include a CPU (Central processing unit) or a neuromorphic processor designed to be advantageous for artificial neural network calculations by incorporating neurons and synapses of the human brain.

The camera 114 captures a subject and obtains an image frame, such as a sperm image or video. The acquired image frame may be displayed on the display of the output device 116, transmitted to an external device through the communication unit 112, processed by the processor 113, or stored in the memory 111.

The camera 114 may be comprised of one or more photographing units. The imaging unit may generate an image of the subject in response to wavelengths. The camera 114 may include an RGB sensor to acquire RGB images. Additionally, the camera 114 may further include an IR sensor or a depth sensor to further acquire an IR image or depth image.

Camera 114 may acquire an image including the face of a user (eg, buyer). Depending on the embodiment, the camera 114 may acquire an image including the buyer's face and upper body.

The input device 115 is a component configured to receive commands related to input from a user (eg, a buyer or participating company). The input device 115 may include a touch unit or other input unit.

The touch unit is a component that uses a part of the buyer's body or another object as a pointing object to input buyer commands. The touch unit may include, but is not limited to, a pressure-sensitive or electrostatic sensor. The other input units include, for example, buttons, keyboards, dials, switches, sticks, etc.

The output device 116 is a component that displays information stored and/or processed in the electronic device 110 and may include, for example, an LCD, OLED, or flexible screen, but is not limited thereto.

Although the input device 115 and the output device 116 are separated in FIG. 2, in many embodiments, the input device 115 and the output device 116 are implemented as a single component to receive input and output information. It can be. For example, the input device 115 and the output device 116 may be a touch panel implemented as a touch screen forming a layer structure with a screen. The touch input is input by a pointing object (eg, including the buyer's body, a tool, etc.).

Server 130 may be in electrical communication with multiple electronic devices 110 . The server 130 is a plurality of computer systems or computer software implemented as network servers. Here, a network server is a computer system and computer that is connected to a sub-device that can communicate with other network servers through a computer network such as a private intranet or the Internet, receives a request to perform a task, performs the task, and provides a performance result. Refers to software (network server program). However, in addition to these network server programs, it should be understood as a broad concept that includes a series of application programs operating on a network server and, in some cases, various databases built within it.

Figure 3 is a schematic block diagram of a server according to an embodiment of the present application.

Referring to FIG. 3, the server 130 includes a memory 131, a communication unit 132, and a processor 133. Since the components 131, 132, and 133 of FIG. 3 are similar to the components 111, 112, and 113 of FIG. 2, the differences will be mainly described.

The memory 131 may permanently or temporarily store data received from the communication unit 132 and data generated or processed by the processor 133. In certain embodiments, the memory 131 may store history data that accumulates information about previously visited booths and information about products (or product groups) of interest during existing visits.

Processor 133 may model a metaverse fair. Additionally, the processor 133 may perform various operations to increase buyer convenience or increase satisfaction of participating companies.

Figure 4 shows a metaverse exhibition displayed on an electronic device, according to an embodiment of the present application.

Referring to FIG. 4, the server 130 transmits metaverse expo data so that the metaverse expo is displayed on the device 110 that accesses the server 130.

The metaverse exhibition data includes exhibition data describing the exhibition. The expo data is obtained from the electronic device 110 of the organizer, operator, or participating company of the expo.

In some embodiments, the expo data may include one or more of expo name information, expo organizer information, expo details, expo banner information, participating company badge information, and billboard information.

The expo details include information on pavilions, booths, or products that make up the expo.

The metaverse expo data includes metaverse data that implements information of the expo data in the metaverse space.

The metaverse data is data for implementing the metaverse space, and renders virtual spaces such as halls, pavilions, and zones, or virtual objects such as products and booths in two or three dimensions. It is used to display a model on the electronic device 110.

The metaverse data may include area information displaying at least some information of the exhibition data. The area information may include a display position, area location, size, etc. of at least some information of the exhibition data.

For example, as shown in Figure 4, the metaverse data includes an expo name area (A) for displaying exposition name information, an exposition details area (B) for displaying exposition details, and an exposition banner information displaying It may include one or more area information among an expo banner area (C) for displaying information on an expo, an expo organizer area (D) for displaying expo organizer information, and a billboard area (E) for displaying billboard information.

The server 130 may generate such metaverse exhibition data and transmit it to the buyer's electronic device 110 through the network 120. Then, the electronic device 110 that has received the metaverse exhibition data may display the screen of FIG. 4. Due to this, buyers can view online booths on the Metaverse Fair, allowing buyers and the Metaverse Fair to interact. In this process, the system 1 may obtain the buyer's raw data from various aspects. The use of raw data according to this interaction is described in more detail with reference to FIGS. 8 to 10 below.

Additionally, the server 130 may generate metaverse expo data capable of demonstrating realistic products and transmit it to the buyer's electronic device 110.

In one example, if the participating company's product is a general industrial product, the server 130 generates metaverse expo data that enables image and video-based 360-degree product confirmation for virtual objects of general industrial products at the metaverse expo and It can also be transmitted to the buyer's electronic device 110.

In addition, if the participating company's product is a non-tangible product such as content or software, the server 130 allows the buyer to actually experience it by selecting a virtual object indicating the non-tangible product. Download data for the corresponding product may also be transmitted to the electronic device 110 of . The virtual object may be a packaging box, document, or other object representing a non-tangible product. The download data may be data of a purchased product or data of a different version (eg, beta version data).

Through this Metaverse expo data, buyers can interact more deeply with the Metaverse expo platform.

Figure 5 shows a booth arrangement to be provided to multiple interested buyers, according to an embodiment of the present application.

Referring to FIG. 5, the server 130 is configured to analyze the interest of buyers and provide a metaverse exhibition customized to the buyers.

Every expo has an industry group and theme, such as edtech, home appliances, sporting goods, and interior fairs. Within a trade fair, it is common to group similar companies and product groups into “halls.”

The server 130 may modularize the metaverse pavilion and corporate booth to provide a metaverse exhibition consisting of a metaverse pavilion and corporate booth customized for buyers. The Metaverse Hall responds to buyers’ areas of interest.

Additionally, the server 130 may analyze the interest of the buyer and provide a Metaverse exhibition having a Metaverse pavilion with booths installed in an arrangement customized to the buyer. Multiple booths may be installed in the metaverse pavilion corresponding to the above fields of interest. The plurality of booths may be modularized and arranged based on buyer interest.

For example, as shown in FIG. 5, server 130 may provide buyers with a first interest, a second interest, a third interest, and a fourth interest with booth(s) associated with the first interest, booth(s) associated with the second interest, A metaverse exhibition consisting of booth(s), booth(s) related to third interests, and booth(s) related to fourth interests may be provided.

The system 1 may receive exhibition application data through the buyer's electronic device 110 in order to provide a metaverse view corresponding to the buyer's interest. The exhibition application data includes information on the buyer's field of interest. The buyer's field of interest information may also be used as the buyer's prior input information.

In some embodiments, the expo application data may be transmitted to the server 130 when a buyer clicks on a field of interest when applying for the expo through the electronic device 110. Then, the server 130 may generate metaverse expo data including expo data describing pavilions corresponding to the field of interest selected by the buyer.

In some embodiments, booth locations where a plurality of booths are installed in each metaverse hall may be designated. That is, the array range in which a plurality of booths can be installed may be specified in advance. Like the order of placement within a designated range, which booths are placed where within the designated range is based on buyer interest.

As a result, the system 1 can provide a metaverse exhibition in which unnecessary tubes to users are removed, unlike offline fairs where tubes are fixed and booths of no interest to buyers are installed.

The server 130 analyzes the interest of buyers, determines the placement ranking of a plurality of booths according to the possibility of buyers being interested in each booth, and reflects the arrangement of the plurality of booths according to the placement ranking of the Metaverse Expo. Data may also be transmitted to the buyer's electronic device 110.

Analyzing the buyer's level of interest includes calculating the buyer's preliminary matching score. In this case, the server 130 calculates the buyer's pre-matching score, determines the placement order of the plurality of booths based on the buyer's pre-matching score, and may install a plurality of booths according to the placement order.

The buyer's pre-matching score indicates the likelihood that the buyer will be interested in a plurality of booths installed inside the Metaverse pavilion to be entered soon, a product group in an individual booth, or a specific product.

The buyer's pre-matching score is calculated using the buyer's information pre-entered before entering the Metaverse pavilion. The pre-matching score may be calculated to have a higher value as the possibility increases, but is not limited to this.

Figure 6 is a schematic diagram of a process for calculating a dictionary matching score according to an embodiment of the present application.

Referring to FIG. 6, the server 130 may obtain and store an exhibition keyword set and a seller keyword set.

The expo keyword set includes keywords representing pavilions of the expo. At the Metaverse Expo, each pavilion may correspond to an expo keyword. For example, when the server 130 acquires the expo keyword set including leisure, sports, and outdoor in FIG. 6, it will provide a metaverse expo including a leisure-themed pavilion, a sports-themed pavilion, and an outdoor pavilion. It may be possible.

The seller keyword set includes keywords representing participating companies that set up booths and participated in the fair. At the Metaverse Expo, each booth can respond to seller keywords. For example, when the server 130 acquires a set of seller keywords including camping tent, export company, and top 20 popular items in FIG. 6, one or more booths related to camping tent, one or more booths related to export company, You could also offer a Metaverse expo that includes one or more booths related to the top 20 popular items.

Additionally, the server 130 may obtain a set of buyer keywords from a plurality of buyers. The buyer keyword set includes keywords indicating prior information and expectations of interest to each buyer. As shown in FIG. 6, the server 130 may obtain a set of buyer keywords including camping, Asia, and MZ generation from one or more buyers.

The server 130 may sort similar expo keywords and/or seller keywords for each buyer keyword entered by the buyer. The server 130 may sort similar keywords in the buyer keyword set and the seller keyword set, or may sort similar keywords in the expo keyword set, seller keyword set, and buyer keyword set. As a result of sorting, a group of similar keywords may be formed.

The server 130 may sort similar keywords using various word similarity analysis methods.

In one example, the server 130 may use a keyword model that infers the meaning of a word to sort keywords with similar meanings into similar keywords. The degree of similarity of keywords may be calculated based on the value embedding the corresponding meaning.

In another example, the server 130 may calculate a similar relationship by statistics on the frequency with which buyer keywords, seller keywords, and expo keywords are mentioned together in text sets such as documents and pages. The server 130 may search for other keywords similar to the entered buyer keyword using the similar relationship.

Additionally, the server 130 may calculate a buyer's preliminary matching score based on the sorting results of similar keywords.

In one embodiment, the server 130 calculates the buyer's preliminary matching score for participating companies associated with the similar keyword based on the similarity between the group of similar keywords sorted as similar for each buyer keyword and the corresponding buyer keyword. It may be possible. Through the similarity between similar keywords (e.g., seller keywords) and buyer keywords, the likelihood that buyers will be interested in the participating company's booth is quantified. The higher the similarity between a buyer's keyword and a similar keyword (e.g., a seller's keyword), the higher the pre-matching score is calculated, making it more likely that the buyer will be interested in the seller's booth.

The server 130 assigns a higher placement priority to the booth as the possibility of interest in the booth increases (i.e., the higher the preliminary matching score). Booths with high pre-matching scores are assigned high placement ranks.

The server 130 arranges a plurality of booths according to the placement order so that buyers preferentially visit booths with a high placement order.

Figure 7 is a layout diagram of an exhibition booth according to an embodiment of the present application.

Referring to FIG. 7, arranging a plurality of booths according to the arrangement order may mean arranging the booths of the corresponding pavilion based on at least one of the location of the buyer and the direction of movement of the buyer on the metaverse exhibition.

The closer the booth is to the buyer's location, the more likely the buyer will visit first. The server 130 may place booths with high pre-matching scores closer to buyers. The location of the buyer on the metaverse expo is expressed by the location of the buyer avatar.

Additionally, the closer the booth is installed to the starting point in the buyer's direction of movement, the more likely it is that the buyer will visit first. The buyer's direction of movement depends on the array range of the designated booth, the entrance and exit of the metaverse tube. The starting point may be the entrance to the metaverse tube.

In some embodiments, when the direction of movement of buyers is specified in the metaverse pavilion, the plurality of booths may be arranged based on the direction of movement of buyers. For example, in Figure 7, the booth above pre-matching n, which has the lowest immediate matching score, is closer to the buyer's location than some booths with higher scores, but may also be installed at the point furthest from the starting point in the direction of travel. . In other words, the booth above pre-matching n may be installed in the last placement order along the direction of movement.

In one embodiment, the server 130 may respond to the buyer's electronic device 110 entering the metaverse pavilion and arrange a plurality of booths in the visited metaverse pavilion based on a prior matching score. In this case, when the buyer accesses the Metaverse pavilion he or she wishes to visit through the electronic device 110, the server 130 sends the Metaverse exhibition data reflecting the results arranged based on the prior matching score to the buyer's electronic device. You can also send it to (110). Then, a plurality of booths arranged according to the preliminary matching score may be displayed on the electronic device 110 of the buyer entering the metaverse hall.

Additionally, the server 130 may change the placement order of at least one booth according to the designation of the exhibition operator. The placement ranking of at least one booth may have a higher ranking than the placement ranking based on the prior matching score. Then, the placement order of the remaining booths may naturally be re-determined to a later priority.

In one embodiment, the server 130 changes the placement ranking of the booth of a specific participating company that has paid a fee to the platform or organizer to be higher than the placement ranking according to the pre-matching score, and creates a plurality of booths according to the changed placement ranking. You can also place . The costs may also include advertising costs.

In Figure 7, the advertising company has a lower ranking than the pre-matching 4th place, but may be assigned a higher ranking than the pre-matching 4th place. As a result, multiple booths may be placed at the advertising company's booth so that buyers visit the booth ranked 4th in pre-matching first.

In one embodiment, the server 130 may calculate the participation index of participating companies in the expo platform and select companies recommended by the organizer based on the participation index.

The participation index quantifies the degree of active participation in the expo platform, such as continuously accessing the expo platform, continuously registering products, registering buyer response scenarios, or conducting frequent online meetings with actual buyers.

The server 130 may select these high-ranking participating companies using a preset participation threshold. Participating companies with a participation index higher than the participation threshold may be selected.

The server 130 may change the placement ranking of the booths of participating companies selected as companies recommended by the organizer to be higher than the placement ranking based on the prior matching score. Companies recommended by the organizer may receive a higher placement ranking even if they do not pay any fees to the organizer.

In some embodiments, a location for a booth of a company recommended by the organizer may be designated in the metaverse pavilion. The location may be a location that is more likely to be exposed to buyers than other booths. As shown in Figure 7, it may be located in the center of the metaverse pipe.

By this booth arrangement operation, when a first buyer and a second buyer with different prior interest information in the system 1 enter the same metaverse hall, the booth displayed on the electronic device 110 of the first buyer The arrangement and the arrangement of the booth displayed on the electronic device 110 of the second buyer may be at least partially different.

Additionally, analyzing the buyer's level of interest includes calculating the buyer's intermediate matching score. In this case, the server 130 may calculate the intermediate matching score of the buyer and rearrange a plurality of booths based on the intermediate matching score of the buyer or based on the pre-matching score of the buyer and the intermediate matching score of the buyer. there is.

Interacting with the Metaverse Expo includes visiting. The buyer's median matching score is a value indicating the likelihood that the buyer will proceed with a contract for an object (eg, a booth, product group, or specific product) while the buyer interacts with the Metaverse Expo. The degree to which interest has increased during the actual visit is analyzed as the intermediate matching score. By using the intermediate matching score, you can check whether interest in the participating company has increased while visiting the actual booth. The intermediate matching score may be calculated to have a higher value as the possibility increases, but is not limited to this.

The buyer's intermediate matching score may be calculated from data collected at the Metaverse Fair, including information describing changes in the buyer while the buyer interacts with the Metaverse Fair. The metaverse exhibition collection data may be used to generate detailed information about the buyer and stored in the memory 133.

Figure 8 is a diagram explaining the buyer's metaverse exhibition collection data according to an embodiment of the present application.

Referring to FIG. 8, the buyer's metaverse exhibition collection data may include implicit factors and explicit factors.

The implicit factor is information that describes changes in the buyer's internal state while interacting with the Metaverse Expo. The implicit factors may include biometric information.

The external factors are information that describes changes in the buyer's external state while interacting with the Metaverse Expo. The externality factor may include one or more of visitation information, emotion information, concentration information, and gaze information.

At a fair, buyers perform actions similar to shopping, selecting products and entering into a purchase contract. The visit information is information that describes the buyer's behavior pattern. The server 130 calculates visit information based on the buyer's visit input data entered for a purchase contract at the Metaverse Expo. In some embodiments, the visit information may include product selection information and/or purchase information.

Product selection information represents product selection details. Product selection on the metaverse expo may be implemented by selecting an area containing a product object on the screen.

In some embodiments, the product selection information may include the number of clicks by the buyer on the product or product group, the number of items placed in the shopping cart, whether the product is registered, the number of product registrations, whether the stock notification is registered, and/or the number of inventory notification registrations, etc. It may be possible. Additionally, the product selection information may include products or product groups frequently checked by the buyer. The product or product group frequently checked by the buyer may be a product group to which products clicked more than a preset threshold belong.

Purchase information represents the purchase details of the product for which a contract is desired at the expo or the purchase details of the product contracted at the expo. In some embodiments, the purchase information may include actual purchase status, purchase date (or expected purchase date), purchase quantity, and/or purchase number.

The server 130 calculates emotional information from facial image data captured while interacting with the metaverse exhibition. Interacting with the Metaverse expo includes communicating with participating companies at the Metaverse booth via electronic communication, viewing product objects displayed inside the Metaverse booth, product description content, etc.

The emotional information includes a first type of emotional information indicating an approximate emotional state of the buyer and/or a second type of emotional information indicating a specific emotional state of the buyer. In some embodiments, the first type of emotional information includes positive or negative. For example, the first type of emotional information may be classified as positive, negative, or neutral. Additionally, in some embodiments, the second type of emotional information may include joy, surprise, sadness, anger, fear, discomfort, calmness, and/or other emotions.

In one embodiment, the server 130 may classify the buyer's face using a facial recognition-based emotion classification model.

The face recognition-based emotion classification model is learned to infer the buyer's emotion by extracting facial landmarks from the input image. In some embodiments, the face recognition-based emotion classification model is a machine learning model that is learned to classify each of the first and second types of emotions expressed on a buyer's face included in input image data. The number of the seven types of second types of emotions described above is merely illustrative, and the types and numbers may change depending on the embodiments.

The emotion classification model includes an artificial neural network. When the buyer's face image data is input, the artificial neural network is trained to output a probability corresponding to each preset emotion. The emotion classification model may be composed of, for example, an artificial neural network including a Convolutional Neural Network (CNN) or other neural networks.

The server 130 calculates gaze information by recognizing the face direction and pupil direction of the buyer from facial image data captured while interacting with the metaverse exhibition. In some embodiments, the gaze information may include at least one of gaze position, gaze fixation time, and gaze fixation frequency. The process of calculating this gaze information is described in more detail with reference to FIG. 9 below.

The server 130 calculates biometric information by measuring heart rate and/or heart rate variability from facial image data captured while interacting with the Metaverse Expo. In some embodiments, the biometric information may include one or more of a high-frequency region of heart rate, a low-frequency region, and the buyer's stress level. The process of calculating such biometric information will be described in more detail with reference to FIG. 7 below.

The server 130 may calculate buyer concentration information from facial image data captured while interacting with the metaverse exhibition. The concentration information describes the degree of concentration (i.e., concentration value) or the degree of immersion (i.e., immersion value). Concentration is a state in which buyers are absorbed in a specific object or action more rationally under negative emotions. Flow is a state of being immersed in a specific object or action more naturally under positive emotions.

In certain embodiments, the server 130 may calculate concentration information based on one or more of gaze information and biometric information, and the emotion information. In one example, the server 130 quantifies the degree to which a person is immersed in a specific object or action based on gaze information and/or biometric information. A numerical value based on emotional information is used to calculate the concentration value or immersion value.

In some embodiments, the server 130 may use a pre-learned concentration determination model to quantify the degree of immersion in a specific object or action and classify the degree of immersion in a specific object or action into two or more groups. The concentration determination model includes an artificial neural network, and the degree to which one is absorbed in a specific object or action may be quantified as a probability. Based on the probability, it may be classified into high/low group or high/normal/low group.

Figure 9 is a flowchart of a process for calculating gaze information using a gaze tracking model, according to an embodiment of the present application.

Referring to FIG. 9, the server 130 may detect a face area from the buyer's face image data and extract a plurality of landmarks from the face area.

In some embodiments, the server 130 may previously store and use a machine learning model for detecting face areas and extracting landmarks. The machine learning model is composed of an artificial neural network including CNN, and may be a model that detects a face area in an input image and infers the location of a facial landmark in the detected face area.

The server 130 may detect the buyer's eye area and pupil area based on a plurality of landmarks. Detection results can also be calculated based on the positions of the eyes and pupil. The location may be calculated as coordinates on the plane of the face image. The server 130 may track the positions of the eyes and pupils by processing a series of facial image data expressing continuous movements of the eyes and pupils.

Additionally, the server 130 may estimate the reference direction of gaze by estimating the head pose of the buyer based on a plurality of landmarks. The reference direction of gaze is the direction in which the buyer's eyes/pupils point toward the front of the face. The buyer's eyes/pupils generally have a gaze range that includes the gaze reference direction.

Then, the server 130 may calculate the position of the buyer's gaze based on the reference direction of gaze, eye position, and/or pupil position. In some embodiments, the server 130 may calculate the position of the buyer's gaze based further on the relative position between the camera 114 and the buyer's face and/or the relative position between the output device 116 and the buyer's face. there is. The server 130 may calculate pupil movement through correction between the screen and the pupil. To this end, the server 130 may perform a calibration task before calculating the gaze position.

Additionally, the server 130 may calculate gaze fixation time and/or gaze fixation frequency based on the object area displayed on the screen of the electronic device 110 and the calculated gaze position.

Gaze fixation time refers to the time a buyer's pupil gazes at an object on the Metaverse Expo between eye blinks. The gaze fixation frequency indicates the frequency with which the buyer's pupils gaze at an object on the Metaverse Expo.

The server 130 may calculate the object area and gaze position on the screen of the electronic device 110 for each unit of time. The server 130 may calculate the gaze fixation time and gaze fixation frequency based on the time and frequency at which the buyer's gaze is located within the object area.

The server 130 extracts a G-channel signal from the patch image showing the detected face area, converts the G-channel signal to the frequency domain by performing FFT processing, and calculates the heart rate and/or heart rate based on the converted signal. Variability can also be calculated. Changes in blood flow are expressed through the G-channel. When converted to the frequency domain, frequencies outside the heart rate range can be filtered out. The server 130 may measure heart rate and/or heart rate variability using a signal in the heart rate range using a preset standard (eg, NN interval (N-N interval) or RR interval (R-R interval) value).

Figure 10 is a flowchart of a process for calculating heart rate information from a face image, according to an embodiment of the present application.

Referring to FIG. 10, the server 130 detects a face area from facial image data captured while interacting with the metaverse exhibition. The face image data consists of pixels of RGB channels. A patch image consisting of a face area includes pixels representing skin. Values for each RGB channel are calculated across all pixels of the patch image. When channel-specific values are tracked across all pixels, a 1-D signal for each channel is generated, consisting of channel-specific values and pixels.

Additionally, the server 130 may measure the head pose. The facial direction may be expressed as a pitch axis, a roll axis, and a yaw axis.

The server 130 extracts each color signal from the patch image through R, G, and B channels, and combines each color signal into a single rPPG signal. In some embodiments, the single rPPG signal may be generated using a plane-orthogonal to skin (POS) method. The POS method includes filtering intensity variations by projecting RGB signals on a plane orthogonal to the skin tone vector determined in RGB space; and combining the projection result signals into a one-dimensional signal by adding weights to them. The weight may be determined by the standard deviation ratio of the two signals. This ensures that the rPPG signal in one dimension contains the maximum amount of heart rate components.

In some embodiments, the POS method may further include converting the extracted rPPG signal and facial direction into the frequency domain using Fast Fourier Transform (FFT) and filtering the extracted rPPG signal to the frequency of a human heartbeat.

The 3-axis facial orientation result vector may be converted to a single average value. This average value may be removed from the raw rPPG spectrum after amplitude normalization. In this case, frequency components with high values in the face orientation spectrum are attenuated in the rPPG spectrum, and then frequencies outside the human heartbeat range are removed from the spectrum.

In some embodiments, the POS method may further include a wideband/narrowband filtering step. The strongest frequency component in the spectrum of the resulting signal is used to determine the passband range of the narrow-bandpass filter. The bandwidth of this passband range depends on the peak detection result. The higher the robustness of peak detection, the wider the bandwidth can be. This filter can also be applied to the extracted raw rPPG to generate a noise-free BVP.

Server 130 may calculate heart rate and/or heart rate variability based on the rPPG signal (raw or filtered). For example, the server 130 may measure heart rate and/or heart rate variability through NN interval (N-N interval) or RR interval (R-R interval) values.

Additionally, the server 130 may classify the frequency region of the heart rate into a high frequency region (HF) and a low frequency region (LF) using a pre-stored band pass filter. The server 130 can determine the buyer's stress based on the ratio of the low-frequency area to the high-frequency area (LF/HF ratio). This is based on general research results showing that when a person is stressed, the sympathetic nervous system is activated and the heart rate increases.

In alternative embodiments, the server 130 may determine stress using the ratio of the high-frequency area to the low-frequency area (HF/LF ratio) instead of the ratio of the low-frequency area to the high-frequency area (LF/HF ratio).

The ratio value may also be used as a stress value. The server 130 may determine that the stress level is normal when the ratio of the low-frequency area to the high-frequency area is between 0.5 and 2.0, and in other cases, it may determine that the stress is high.

The heart rate standard for distinguishing between the high frequency region (HF) and the low frequency region (LF) can be set in various ways. For example, the section with a heart rate of 120 BPM or more is referred to as the high frequency region (HF), and the other sections are referred to as the low frequency region (LF). can be distinguished.

The server 130 calculates the buyer's intermediate matching score based on implicit factors including biometric information and explicit factors including one or more of visit information, emotional information, concentration information, and gaze information for objects in the metaverse booth.

The object may be an online booth at the Metaverse Expo displayed on the screen of the electronic device 110, a product object placed within the online booth, or product content.

In one embodiment, weights may be set in advance for each item of information included in the implicit factor and the explicit factor. Then, the buyer's intermediate matching score is calculated based on the value of each information item and its weight. Items within the same information may be given the same weight, or may be given different individual weights.

In one embodiment, information items of the implicit factor may be assigned a higher weight than items of the explicit factor. This is because implicit reactions have more influence in measuring buyers' interest in a product.

For example, the intermediate matching score is a weighted sum according to the weight of each item, where equal weights are given to items in the same information and the weight value of the implicit factor is set to be the largest, as shown in the following table. It can also be calculated through .

information item Visit (action) information emotional information concentration information gaze information biometric information reflected value number of clicks Positive emotions or not degree of concentration line-of-sight coordinate value High frequency and low frequency values Number of products of interest added Negative feelings or not degree of immersion gaze duration Stressed or not? Add to Cart … … … … … weight 20% 20% 20% 10% 30%

The server 130 calculates a buyer's matching score based on the buyer's intermediate matching score or based on the buyer's preliminary matching score and the buyer's intermediate matching score, and determines the placement ranking based on the intermediate matching score, Depending on the placement ranking, you can rearrange all booths except those you have already interacted with.

The remaining booths have a lower placement rank than the booths that exited the metaverse hall entered, and are booths scheduled to be visited after exiting the specific booth. The process of re-arranging the remaining booths according to the matching score is similar to the process of initially arranging a plurality of booths according to the pre-matching score, so detailed description is omitted.

In one embodiment, determining the placement ranking based on the intermediate matching score may mean determining the placement ranking based on the degree of similarity to the participating company being visited. If the buyer's sentiment is positive during the visit, the higher the similarity, the higher the rank assigned. If the buyer's sentiment is negative during the visit, the lower the similarity, the higher the rank assigned.

In one embodiment, the server 130 may re-arrange the remaining booths in response to the buyer's electronic device 110 leaving the metaverse booth where the interaction results used to calculate the intermediate matching score occurred. there is.

In this case, when the buyer completes visiting a specific booth through the electronic device 110 and leaves, the server 130 sends the metaverse expo data reflecting the re-arrangement results based on the intermediate matching score to the buyer's electronic device. You can also send it to (110). If the arrangement ranking of the remaining booths according to the matching score is different from the arrangement ranking of the remaining booths according to the pre-matching score, the buyer's electronic device 110 displays the remaining booths re-arranged in a different arrangement order than before entering the specific booth. It may be displayed.

In this way, the system 1 reacts in real time to the results of interaction with the Metaverse fair and updates the arrangement of the booth, thereby reflecting the changing status of buyers in real time in the structure of the Metaverse fair. As a result, while interacting with the Metaverse Expo, buyers first visit booths in which their interest deepens, and do not need to visit booths in which their interest fades, saving them the trouble of visiting all booths.

The booth's product content may include live commerce videos and/or advertising videos displaying real-time promotions. The server 130 may transmit product content of the booth to the electronic device 110 of a buyer visiting the booth.

When a buyer's level of interest is calculated through face recognition and eye tracking in real time while eye-shopping while looking at the shelves in the booth, it is automatically determined whether to expose additional content for the product based on the level of interest. For example, recommended products for product groups that were mainly viewed and had high interest within the last 5, 10, or 30 minutes may be broadcast in real time from the live commerce zone within the booth.

In one embodiment, the product content may be transmitted to the electronic device when the buyer's emotional information is positive. When a buyer looks at a product with positive emotions (e.g., joy, surprise, curiosity), a short-form advertisement for the product may be sent in the form of a pop-up. If the buyer's emotion is normal or negative, the live commerce video or advertising video may not be transmitted.

In one embodiment, the live commerce video or advertising video may be a film of an avatar's movements in the space inside the booth.

A live commerce video may be a live commerce zone preset inside the booth where the promotional actions of an NPC avatar are broadcast in real time.

An advertising video may be a real-time broadcast of the advertising behavior of a model NPC avatar inside a booth.

These metaverse shooting results may be transmitted to the electronic device 110 through real-time streaming.

Additionally, the server 130 may include a consultation module configured to generate conversation content in response to the buyer's consultation input text input through the buyer's electronic device 110. The counseling module operates as an artificial intelligence counselor and may be implemented with at least one processor 133.

The consultation module may execute one or more computer programs to generate conversation content. The interactive content is text with context and is an interactive response to buyer input data.

The computer program may be, for example, a natural language program (including, for example, Nature Language Processing (NLP) and/or Nature Language Understanding (NLU) programs) that supports machine learning or artificial intelligence, or various non-humanoid chat programs. . For example, the consultation module may have a chatbot program installed to generate and provide conversation content.

In one embodiment, the counseling module may generate conversation content through a machine learning based chatbot. Machine learning-based chatbots consist of a conversation model that produces output text appropriate for the input text based on large-scale training samples. The training sample may include response scenarios from participating companies. The machine learning-based chatbot may be composed of, for example, seq2seq model, pLSA, etc., but is not limited thereto.

The server 130 may use a keyword included in at least one of the buyer input data input to the consultation module and the conversation content data output from the consultation module to calculate a dictionary matching score. In one example, the keywords of the buyer input data input to the consultation module may be used as the buyer keywords in FIG. 6 to calculate a dictionary matching score. Additionally, keywords in the conversation content data output from the consultation module can be used as buyer keywords or seller keywords to calculate a preliminary matching score. As a result, buyers can receive a Metaverse exhibition that reflects their prior expectations through conversation with the consultation module.

Additionally, the server 130 determines a buyer who is interested in the target participating company based on the preliminary matching score and the intermediate matching score, and delivers the interest information of this interested buyer to the target participating company. The interest information of the interested buyer may include at least part of the metaverse exhibition collection data. For example, the interested buyer's interest information may include purchase information.

The server 130 may deliver a list of interested buyers to target participating companies. The list of interested buyers may include interest information for each interested buyer.

In one embodiment, analyzing the buyer's interest includes calculating the buyer's post-matching score. In this case, determining the buyer of interest may mean determining the buyer of interest based on at least one of a pre-matching score, an intermediate matching score, and a post-matching score.

The buyer's post-matching score indicates the likelihood that the buyer will be interested in or maintain interest in a specific participating company after visiting the booth of that participating company.

The buyer's post-matching score is calculated based on post-input information describing the buyer's follow-up interests. The post-matching score may be calculated to have a higher value as the possibility increases, but is not limited to this.

In one embodiment, post-input information may include whether a contract is concluded, whether a quotation is requested, whether an MOU is concluded, whether a prototype is requested, whether additional consultation is requested, and whether additional materials are requested. The degree to which each item is required may be quantified as a post-matching score.

Assumption values may be set in advance for each score item. Then, the server 130 may determine a buyer of interest using the final matching score calculated based on the value of each score and the corresponding weight. If the final matching score is calculated to be higher than a preset interest threshold, the server 130 may determine this buyer as an interested buyer.

In one embodiment, the intermediate matching score or the post-matching score may be assigned a higher weight than the pre-matching score. This is because the buyer's interest generated while participating in the fair has a more important impact on the contract than the interest expected prior to participation.

For example, the final matching score for determining the buyer of interest may be calculated through a weighted sum in which the weight value of the preliminary matching score is set to be the smallest, as shown in the following table.

Buyer Interest Index (Pre) Matching Score (Medium) Matching Score (Post) follow-up score 20% 40% 40%

The system (1) can provide information on which groups of buyers have a high level of interest in the target participating company by utilizing the intermediate interest or follow-up interest of buyers that have a more important influence. As a result, the system (1) allows participating companies to contact or conduct sales first without missing interested buyers after the fact. It will be apparent to those skilled in the art that the system 1 may include other components not described herein. For example, it may further include a data input device, an output device such as display and/or printing, a network, a network interface, and a protocol.

Figure 11 is a flowchart of a method for providing a metaverse exhibition based on buyer interest analysis, according to another aspect of the present application.

The method of providing a Metaverse expo based on buyer interest analysis of FIG. 11 is performed by at least one processor to provide a customized Metaverse expo by analyzing buyers' interest in booths or products of participating companies on the Metaverse expo. In certain embodiments, the method may be performed by the buyer interest analysis-based metaverse exhibition providing system 1 of FIG. 1.

Referring to FIG. 11, the method includes transmitting (e.g., by the server 130 of FIG. 1) metaverse expo data for displaying the metaverse expo to the electronic device (S1101). do. The metaverse expo data includes metaverse data for implementing the expo model in the metaverse space, and expo data. The expo data may include one or more of the following: expo name information, expo organizer information, expo details, expo banner information, participating company badge information, and billboard information. Because of this, buyers can view online booths or products on the Metaverse fair, allowing buyers and the Metaverse fair to interact.

The metaverse expo data of step S1101 is for displaying metaverse pavilions that can be entered at a metaverse expo consisting of a metaverse space, and the metaverse expo data is used to display the metaverse pavilion before entering at least one metaverse pavilion. You can also display a screen. For example, as shown in FIG. 4, a screen displaying one or more metaverse halls that can be entered may be displayed using the metaverse exhibition data of step S1101. The buyer's electronic device 110 accesses at least one metaverse pavilion of the metaverse expo, and the buyer enters the pavilion.

In addition, the method may include the step of obtaining pre-input information of buyers obtained before entering the metaverse pavilion, and arranging a plurality of booths in an arrangement customized to the buyer based on the pre-input information (S1110). there is.

In one embodiment, the step (S1110) includes obtaining the buyer's pre-input information through the buyer's electronic device 110 before entering the metaverse pavilion, for each of the plurality of booths included in the metaverse pavilion. , calculating a buyer's pre-matching score based on the buyer's pre-input information, determining a placement ranking of each booth according to the buyer's pre-matching score, and arranging a plurality of booths according to the determined placement ranking. It may also include steps.

In one embodiment, the pre-input information may include one or more of interest field information of the buyer's exhibition application data, the buyer's consultation input text, and conversation content in response to the consultation input text.

Since the buyer's consultation input text and conversation content were described above with reference to the consultation module, detailed descriptions are omitted.

In step S1110, at least part of the inquiry/answer text with the artificial intelligence counselor before entering the metaverse hall may be used as pre-input information.

In one embodiment, the step of calculating the buyer's pre-matching score includes sorting similar keywords to the buyer keyword entered by the buyer in a seller keyword set and an exhibition keyword set, and between the sorted similar keywords and the buyer keyword. It may also include the step of calculating the buyer's preliminary matching score for participating companies associated with similar keywords by quantifying the similarity.

In one embodiment, the step of arranging a plurality of booths according to the placement order may mean arranging the booths of the corresponding pavilion based on at least one of the location of the buyer and the direction of movement of the buyer on the metaverse exhibition.

In some embodiments, when the direction of movement of buyers is specified in the metaverse pavilion, the plurality of booths may be arranged based on the direction of movement of buyers.

The method includes transmitting metaverse expo data for displaying the arranged booth arrangement to the electronic device (S1120).

In one embodiment, the server 130 responds to the buyer's electronic device 110 entering the metaverse pavilion and arranges the plurality of booths of the visited metaverse pavilion based on the prior matching score. may be transmitted to the electronic device (S1120).

Buyers who entered the Metaverse pavilion after steps (S1110, S1120) may visit at least some of the multiple booths.

The method includes obtaining a buyer's visit input data and face image data during a visit to the Metaverse booth, and generating intermediate matching information based on the buyer's visit input data and face image data (S1130).

In one embodiment, the intermediate matching information may include an intermediate matching score. The step of generating the intermediate matching information includes receiving, from the electronic device 110, the buyer's face image data captured while interacting with the metaverse fair and the buyer's visit input data entered, the buyer's visit input Calculating implicit factors and explicit factors of a buyer based on at least one of the data and face image data of the buyer, and mediating buyers for participating companies in the Metaverse Expo based on the implicit factors and explicit factors of the buyer A step of calculating a matching score may also be included.

The implicit factor includes information describing changes in the buyer in terms of the internal body during interaction with the metaverse fair.

In one embodiment, the underlying factors may include biometric information describing cardiac motor changes. The biometric information may include at least one of a high-frequency region of heart rate, a low-frequency region, and the buyer's stress level.

The external factors include information describing changes in buyers in various aspects during interaction with the Metaverse Expo.

In one embodiment, the appearance factor may include one or more of the buyer's emotion information, concentration information, and gaze information.

One or more of emotion information, concentration information, gaze information, and biometric information is calculated based on the image data (S1130).

In one embodiment, the appearance factor may further include visitation information. The visit information is generated based on the buyer's visit input data entered into the electronic device 110 while interacting with the Metaverse Expo.

The description of this information and the process of generating it are described above with reference to FIGS. 8 to 10, and detailed description will be omitted.

In one embodiment, the intermediate matching score of the buyer may be calculated based on values for each information item of implicit factors and explicit factors, and a preset weight for each information item.

In one embodiment, items of the implicit factor may be assigned a higher weight than items of the explicit factor.

The explanation of this intermediate matching score and the process of calculating it are described above with reference to Table 1, so detailed explanation will be omitted.

In alternative embodiments, the method includes rearranging the remaining booth array of the entered metaverse pavilion based on intermediate matching information (S1140); And it may further include transmitting metaverse expo data for displaying the updated booth arrangement to the electronic device 110 (S1150).

The re-arranging step (S1140) includes determining the placement ranking of the remaining booths based on the intermediate matching score, and re-arranging the remaining booths excluding the already interacted booths according to the placement ranking. do.

In one embodiment, determining the placement ranking based on the intermediate matching score may mean determining the placement ranking based on the degree of similarity to the participating company being visited. If the buyer's sentiment is positive during the visit, the higher the similarity, the higher the rank assigned. If the buyer's sentiment is negative during the visit, the lower the similarity, the higher the rank assigned.

In one embodiment, the step of transmitting metaverse expo data for displaying the updated booth arrangement to the electronic device is in response to the buyer's electronic device exiting the visited metaverse booth to calculate an intermediate matching score. It may be something that is being done.

In response to the buyer completing a visit to a specific booth and leaving through the electronic device 110, the server 130 rearranges the remaining booths and sends the metaverse expo data reflecting the rearrangement results to the buyer's electronic device. It can also be transmitted to (110) (S1150). Then, buyers may be provided with an updated booth arrangement upon exiting.

Additionally, the method includes a step of determining interested buyers based on the pre-input information and intermediate matching information (S1160), and delivering information on the determined interested buyers to the participating companies (S1170).

In one embodiment, the step (S1160) may be to acquire post-interest information of buyers after leaving the metaverse and determine buyers of interest based on the post-interest information.

In one embodiment, the step (S1160) calculates the final matching score of the buyer based on at least one of the pre-matching score, the intermediate matching score, and the post-matching score, and uses the final matching score to provide the participating company with the final matching score. It may be determining which buyers are interested in the product.

In one embodiment, the final matching score may be calculated based on the pre-matching score, intermediate matching score, post-matching score, and a preset weight for each score.

In one embodiment, post-input information may include whether a contract is concluded, whether a quotation is requested, whether an MOU is concluded, whether a prototype is requested, whether additional consultation is requested, and whether additional materials are requested.

In one embodiment, the intermediate matching score or the post-matching score may be assigned a higher weight than the pre-matching score.

In alternative embodiments, some of the operations of the server 130 may be performed on the electronic device 110.

In one embodiment, calculating a pre-matching score, calculating an intermediate matching score, and/or calculating a post-matching score may be performed in the electronic device 110. In this case, the results of the calculation performed in the electronic device 110 may be transmitted to the server 130 and subsequent operations may proceed.

According to these embodiments of the metaverse exhibition provision system and method based on buyer interest analysis, it goes beyond simply checking products in advance based on images, requesting and conducting text-based consultations, and linking general video meetings between buyers and companies. , provides an advanced exhibition platform model that comprehensively utilizes data on buyers’ emotions and interests. As a result, the system can provide customized services by more accurately analyzing the needs of buyers and participating companies based on the buyer's activity data and emotional data while viewing products and visiting booths in the metaverse space. As a result, the system can directly contribute to increasing sales by accurately providing interested buyer information to participating companies.

When implementing embodiments of the present invention using hardware, application specific integrated circuits (ASICs) or digital signal processors (DSPs), digital signal processing devices (DSPDs), or programmable PLDs (PLDs) configured to perform the embodiments of the present application. logic devices), field programmable gate arrays (FPGAs), etc. may be included in the components of the present application.

The operations of the system and method for providing a metaverse exhibition based on buyer interest analysis according to the embodiments of the present application described above may be at least partially implemented as a computer program and recorded on a computer-readable recording medium. For example, implemented with a program product comprised of a computer-readable medium containing program code, which can be executed by a processor to perform any or all steps, operations, or processes described.

The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Additionally, computer-readable recording media may be distributed across computer systems connected to a network, and computer-readable codes may be stored and executed in a distributed manner. Additionally, functional programs, codes, and code segments for implementing this embodiment can be easily understood by those skilled in the art to which this embodiment belongs.

The present invention examined above has been described with reference to the embodiments shown in the drawings, but these are merely illustrative examples, and those skilled in the art will understand that various modifications and modifications of the embodiments are possible therefrom. However, such modifications should be considered within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the attached patent claims.

111: electronic device 111: memory
112: Transmitter and receiver 113: Processor
114: camera 115: input device
116: output device 120: telecommunication network
130: server 131: memory
132: Transmitter and receiver 133: Processor

Claims (19)

In a method of providing a metaverse exhibition by analyzing buyer interest, performed by a server,
Transmitting metaverse expo data for displaying metaverse pavilions that can be entered at a metaverse expo consisting of a metaverse space to an electronic device that has accessed the metaverse expo;
Obtaining pre-input information of buyers obtained before entering the metaverse hall, and arranging a plurality of booths in an arrangement customized to the buyer based on the pre-input information;
Transmitting metaverse expo data for displaying the arranged booth arrangement to the electronic device;
Obtaining a buyer's visit input data and face image data during a visit to the Metaverse booth, and generating intermediate matching information based on the buyer's visit input data and face image data;
determining buyers of interest based on the pre-input information and intermediate matching information; and
A method including the step of delivering information on the determined interested buyer to the participating company.
The method of claim 1, wherein the step of arranging a plurality of booths in an arrangement customized to the buyer based on the pre-input information comprises:
Obtaining the buyer's pre-entered information through the buyer's electronic device before entering the Metaverse hall,
For each of the plurality of booths included in the metaverse pavilion, calculating a buyer's pre-matching score based on the buyer's pre-input information,
Determining the placement ranking of each booth according to the buyer's pre-matching score, and
It includes the step of arranging a plurality of booths according to the determined arrangement ranking,
The step of calculating the buyer's pre-matching score is,
Sorting similar keywords to the buyer keyword entered by the buyer in the seller keyword set and the expo keyword set, and
A method comprising calculating the buyer's preliminary matching score for participating companies associated with the similar keywords by quantifying the similarity between the sorted similar keywords and the buyer keywords.
The method of claim 2, wherein the step of arranging a plurality of booths according to the arrangement order comprises:
A method characterized by arranging the booth of the relevant pavilion based on at least one of the location of the buyer and the direction of movement of the buyer on the metaverse expo.
The method of claim 1, wherein the method comprises:
Re-arranging the remaining booth arrangement of the entered metaverse pavilion based on intermediate matching information; and
The method further comprising transmitting metaverse expo data to the electronic device for displaying an updated booth arrangement.
According to claim 4,
The intermediate matching information includes an intermediate matching score,
The step of generating the intermediate matching information is,
Receiving the buyer's face image data captured while interacting with the Metaverse Expo and the buyer's visit input data from the buyer's electronic device;
Calculating the buyer's internal factors and external factors based on at least one of the buyer's visit input data and the buyer's face image data, and
A method comprising calculating an intermediate matching score of a buyer for a participating company in the Metaverse Expo based on the buyer's implicit and explicit factors.
The method of claim 5,
The implicit factors include biometric information,
The step of calculating the buyer's internal factors and external factors based on at least one of the buyer's visit input data and the buyer's face image data,
measuring one or more biosignals of the buyer's heart rate and heart rate variability from the facial image data; and
A method comprising generating biometric information of the buyer based on the biosignal.
The method of claim 6, wherein the step of measuring one or more biosignals of the buyer's heart rate and heart rate variability comprises:
Detecting a face area from the face image data;
extracting a G-channel signal from the face area;
Converting the G-channel signal to the frequency domain; and
A method comprising measuring one or more biosignals of heart rate and heart rate variability based on the converted frequency signal.
The method of claim 6,
The biometric information includes one or more of the high-frequency region of heart rate, low-frequency region, and stress level,
The step of generating biometric information of the buyer based on the biosignal,
Classifying the frequency region of the heart rate into a high-frequency region and a low-frequency region using a pre-stored band-pass filter; and
Characterized in that it comprises calculating a stress level of the buyer while interacting with the metaverse fair based on the ratio of the high-frequency area and the low-frequency area based on the ratio of the high-frequency area and the low-frequency area.
The method of claim 6,
The appearance factor includes one or more of the buyer's visit information, emotion information, concentration information, and gaze information while interacting with the metaverse exhibition,
The step of calculating the buyer's internal factors and external factors based on at least one of the buyer's visit input data and the buyer's face image data,
calculating the visit information based on the buyer's visit input data entered for a contract on the Metaverse Expo;
calculating emotional information of the buyer from the facial image data;
calculating gaze information of the buyer from the face image data; and
A method comprising calculating concentration information of the buyer based on one or more of gaze information and biometric information, and the emotion information.
The method of claim 9,
The emotional information includes positive or negative,
The step of calculating the buyer's emotional information from the facial image data includes:
A method characterized by extracting facial landmarks from an input image and calculating the buyer's emotion information using an emotion classification model previously learned to infer the buyer's emotion.
The method of claim 9,
The step of calculating the buyer's gaze information from the face image data includes:
Detecting a face area from the buyer's face image data and extracting a plurality of landmarks from the face area;
Detecting the buyer's eye portion and pupil portion based on a plurality of landmarks;
Estimating the reference direction of gaze by estimating the direction of the buyer's face based on a plurality of landmarks;
calculating the position of the buyer's gaze based on at least one of the gaze reference direction, eye position, and pupil position; and
A method comprising calculating at least one of gaze fixation time and gaze fixation frequency based on the object area displayed on the screen of the electronic device and the calculated gaze position.
The method of claim 9,
The concentration information includes a concentration or immersion value,
The step of calculating the buyer concentration information is,
Quantifying the degree to which the buyer is immersed in a specific object based on at least one of the gaze information and biometric information;
When the emotional information includes positivity, calculating an immersion value using a numerical value; and
When the emotional information includes negativity, a method comprising calculating a concentration value using a numerical value.
According to claim 4,
The step of determining the placement ranking based on the intermediate matching score is,
The placement ranking is determined based on the degree of similarity to the participating company being visited. If the buyer's emotion is positive during the visit, the higher the degree of similarity, the higher the rank is assigned. If the buyer's emotion is negative during the visit, the degree of similarity is assigned. A method characterized in that the lower the rank, the higher the rank is assigned.
The method of claim 4, wherein transmitting metaverse expo data for displaying the updated booth arrangement to the electronic device comprises:
Characterized in that the method is performed in response to the buyer's electronic device exiting the visited Metaverse booth to calculate an intermediate matching score.
The method of claim 1, wherein the step of determining the interested buyer comprises:
A method characterized by obtaining post-interest information of buyers after exiting the metaverse and determining buyers of interest based on the pre-input information, intermediate matching information, and post-interest information.
The method of claim 15, wherein determining interested buyers based on the pre-input information, intermediate matching information, and post-interest information comprises:
calculating the buyer's final matching score based on at least one of the pre-matching score, the intermediate matching score, and the post-matching score; and
A step of determining buyers interested in the participating company using the final matching score,
The final matching score is calculated based on the values of the pre-matching score, intermediate matching score, and post-matching score, and a preset weight for each score.
The method of claim 16,
A method characterized in that the intermediate matching score or the post-matching score is assigned a higher weight than the pre-matching score.
A computer-readable recording medium on which a program for performing a method of providing a metaverse exhibition by analyzing buyer interest according to any one of claims 1 to 17 is recorded.
In a system that induces buyers to purchase by analyzing their interest in objects on the Metaverse Expo,
A Metaverse pavilion that can be entered at a Metaverse expo consisting of a Metaverse space is displayed on an electronic device accessing the Metaverse expo, and the buyer's face is filmed while the buyer is interacting with the Metaverse expo, thereby creating an image of the buyer's face. An electronic device for acquiring data and obtaining visit input data entered while interacting with the Metaverse Expo, and
Transmitting metaverse expo data to display metaverse pavilions that can be entered at a metaverse expo consisting of a metaverse space to the electronic device that accesses the metaverse expo,
Obtain the buyer's pre-input information obtained before entering the metaverse hall, and place a plurality of booths in an arrangement customized to the buyer based on the pre-input information,
Transmitting metaverse exhibition data to display the arranged booth arrangement to the electronic device,
Obtaining the buyer's visit input data and face image data during the visit to the Metaverse booth, and generating intermediate matching information based on the buyer's visit input data and face image data,
Determine interested buyers based on the pre-input information and intermediate matching information, and
Including a server configured to deliver information on determined interested buyers to the participating companies,
system.

Images