KR102610737B1 - Service providing robot for vehicle display shop and method of operating thereof - Google Patents

Abstract

The present invention relates to a service provision robot at a vehicle exhibition store and a method of operating the same to provide customized services to customers. A service providing robot at a vehicle exhibition store according to an embodiment of the present invention includes a first service determination unit that determines an object of interest for the user based on the user's state, and a service for the object of interest based on the state of the user. It may include a second service determination unit that determines a start time for providing content, and a service provision unit that provides the service content to the user according to the start time.

Description

Service provision robot for vehicle display store and its operating method {SERVICE PROVIDING ROBOT FOR VEHICLE DISPLAY SHOP AND METHOD OF OPERATING THEREOF}

The present invention relates to a service provision robot at a vehicle exhibition store and a method of operating the same to provide customized services to customers.

Recently, as robot-related technology has developed rapidly, the development of robots with necessary functions in various industrial fields is actively underway. In particular, robots are recognized for their great utility in the manufacturing field, which requires simple mechanical repetitive tasks.

Meanwhile, the demand for robots is increasing in various industrial fields because robots have many strengths in terms of work accuracy, efficiency, and economic feasibility, even in somewhat complex tasks other than mechanical work.

The purpose of the present invention is to provide a service provision robot at a vehicle exhibition store that can actively provide services to users visiting the exhibition store and a method of operating the same.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

A service providing robot at a vehicle exhibition store according to an embodiment of the present invention includes a first service determination unit that determines an object of interest for the user based on the user's state, and a service for the object of interest based on the state of the user. It may include a second service determination unit that determines a start time for providing content, and a service provision unit that provides the service content to the user according to the start time.

A store service provision system for a vehicle exhibition store according to an embodiment of the present invention includes the service provision robot; And it may include at least one vehicle that communicates with the service providing robot and provides status information.

A method of operating a service providing robot at a vehicle exhibition store according to an embodiment of the present invention includes determining an object of interest of the user based on the user's state, and service content for the object of interest based on the state of the user. It may include determining a starting time to provide and providing the service content to the user according to the starting time.

The service provision robot and its operating method at a vehicle exhibition store according to an embodiment of the present invention can provide services suitable for the user's various tastes and interests by actively identifying the user's location, behavior, voice, etc. without separate manipulation by the user. You can.

Additionally, by learning the user's response to service provision, it is possible to provide services of improved quality.

Figure 1 is a diagram showing an example of a place where a service providing robot according to an embodiment of the present invention is applied.
Figure 2 is a block diagram showing a service providing robot according to an embodiment of the present invention.
Figure 3 is a flowchart showing a method of operating a service providing robot according to an embodiment of the present invention.
Figure 4 is a flowchart showing step S2 shown in Figure 3 in more detail.
Figure 5 is a flowchart showing step S3 shown in Figure 3 in more detail.
Figure 6 is a flowchart showing step S7 shown in Figure 3 in more detail.
Figure 7 is a flowchart showing step S8 shown in Figure 3 in more detail.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by the term. Additionally, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present application. No.

Figure 1 is a diagram showing an example of a place where a service providing robot according to an embodiment of the present invention is applied.

Referring to FIG. 1, the service providing robot 100 can be applied to an exhibition store (or vehicle exhibition store, 10) that provides general customers with the opportunity to display and directly experience vehicles. The exhibition store 10 is only an example, and the present invention can be applied to stores that display or sell various products.

In the exhibition store 10, vehicles for display (vehicle 1 to vehicle 3) are displayed at a specific location, and the user 50 can enter the exhibition store 10 to directly see and experience the vehicle of interest. The structure of the exhibition store 10, the location and number of vehicles, etc. are merely examples.

As shown in FIG. 1, the service providing robot 100 is located at the entrance of the exhibition store 10 or waits at a random location within the exhibition store 10, and then when the user 50 enters the exhibition store 10. I can recognize what is entering within me. The service providing robot 100 may determine which vehicle (eg, vehicle 3) the user 50 is interested in by determining the location and/or gaze of the recognized user 50.

As shown in FIG. 1 , if the vehicle in which the user 50 is interested is vehicle 3, the service providing robot 100 may prepare service content for vehicle 3. Additionally, the service providing robot 100 may move near the user 50 .

The service providing robot 100 may determine the starting time to provide the prepared service content by considering the time the user stays near vehicle 3, reaction, and/or experiential behavior, and when the starting time arrives, the user (50 ) can provide appropriate service content according to the reaction and experience behavior.

The service providing robot 100 can continuously track the location, gaze, reaction, and experience behavior of the user 50, and the end point at which provision of service content will end according to the tracked information of the user 50. can be determined, and the provision of service content can be terminated when the end point arrives. In addition, the service providing robot 100 can learn the user's reaction by detecting the user's reaction to the provision of service content through information collected through a camera and information collected through a microphone and forming a database.

The service provision robot 100 can collect status information of the vehicle (e.g., information on whether the trunk, door, etc. is open or closed) by communicating with the vehicles in the exhibition store 10, and is a store service provision system together with the vehicles. can be configured.

According to the service providing robot 100 according to an embodiment of the present invention, the location, behavior, voice, etc. of the user 50 are actively identified without any separate manipulation by the user 50, and the various tastes and interests of the user 50 are provided. We can provide services suitable for you.

Additionally, by learning the user's (50) response to service provision, it is possible to provide services of improved quality.

Figure 2 is a block diagram showing a service providing robot according to an embodiment of the present invention.

Referring to FIG. 2, the service providing robot 100 includes a sensor unit 110, a user recognition unit 120, a first service determination unit 130, a second service determination unit 140, and a service creation unit 150. , It may include a service provision unit 160, a third service determination unit 170, and a user response learning unit 180. Each component of the service providing robot 100 may be implemented as hardware, software, or a combination thereof. Each configuration of the service providing robot 100 is merely illustrative, and some configurations may be omitted or added as needed. For example, a main body that forms the appearance of the service providing robot 100, a rotating wheel for moving the service providing robot 100, a rotating drive unit for changing the sensing direction of various sensors included in the sensor unit 110, etc. Additional configurations may be included.

The sensor unit 110 may include at least one sensor for collecting various types of information from the outside of the service providing robot 100. The sensor unit 110 includes a camera capable of capturing external images, a microphone capable of collecting external voices, a distance sensor capable of detecting the distance to an external object, a service provision robot 100 in the exhibition store 10, and a user. It may include a position sensor capable of detecting the position of (50). Here, the distance sensor may be at least one of an infrared sensor, an ultrasonic sensor, and a Time of Flight (ToF) sensor. In addition, location sensors are used in a variety of applications such as base stations, Wi-Fi, inertial navigation, high-sensitivity GNSS (Global Navigation Satellite System), UWB (Ultra Wide Band), RFID (Radio Frequency Identification), pseudosatellites, ultrasound, infrared, geomagnetic fields, cameras, etc. Positioning of the service providing robot 100 and the user 50 can be performed using physical resources. For example, the service providing robot 100 may first determine the location of the service providing robot 100 and determine the relative position of the user 50 from the service providing robot 100 to determine the exact location of the user 50. You can.

In addition, the sensor unit 110 communicates with each vehicle (vehicle 1 to vehicle 3) to collect sensing information from sensors (opening and closing sensors of the trunk, door, bonnet, etc.) installed in the vehicles (vehicle 1 to vehicle 3). You can.

The sensor unit 110 may provide sensing information collected from at least one sensor in another configuration (eg, the user recognition unit 120).

The user recognition unit 120 allows the user 50 to enter a designated area (for example, within a certain range from any vehicle or within a certain range of the service providing robot 100) based on the sensing information provided from the sensor unit 110. It is possible to recognize whether the user 50 has entered, and when the user 50's entry is recognized, the operation of the first service determination unit 130 and the second service determination unit 140 can be activated by entering the preparation mode from the standby mode.

The service providing robot 100 can operate in three modes: standby mode, preparation mode, and service mode. The standby mode periodically checks whether the user 50 enters the exhibition store 10 in an idle state. means a mode, and the preparation mode means a mode that prepares a service suitable for the user 50 by identifying the location, behavior, voice, etc. of the user 50 in the exhibition store 10, and the service mode refers to the provision of the determined service. It refers to the mode of providing services from the start point to the end point of service provision.

The first service determination unit 130 may determine an object of interest of the user 50 based on the user's 50 status. Here, the object of interest may include a car model of interest in which the user 50 is interested and a segment of interest that refers to a specific part among various parts of the car model of interest. Depending on the embodiment, if the segment of interest cannot be specified, the object of interest may consist only of the vehicle type of interest.

Specifically, the first service determination unit 130 may determine the type of vehicle of interest to the user 50 based on the location of the user 50 and the time spent at the location. For this purpose, the first service determination unit 130 may receive the location of the user 50 from the position sensor of the sensor unit 110, and the exhibition store 50 and vehicles (vehicle 1 to vehicle 3) stored in advance ) From the map information storing the location of the user 50 and the current location of the user 50, which of the vehicles (vehicle 1 to vehicle 3) is the current location of the user 50 close to and for a certain period of time at the adjacent location The type of vehicle of interest (eg, vehicle 3) of the user 50 may be determined based on analysis (eg, location tracking) of whether the user 50 stays there for more than 10 seconds.

Additionally, the first service determination unit 130 may track the gaze of the user 50 and the time the gaze lasts to determine the section of interest in which the user 50 is interested. The first service determination unit 130 may receive an image of the face of the user 50 from the camera of the sensor unit 110, analyze the gaze direction of the user 50 in the image, and detect vehicles (vehicles). The interest that the user 50 is interested in based on analyzing which part of the vehicle type of interest the user 50's gaze direction is directed to from map information storing the locations of vehicles 1 to 3) (i.e., gaze tracking) The category (eg, rear camera, AVM (Around View Monitor) camera) can be determined.

For example, when there are a plurality of vehicles close to the user 50 or areas where the user's gaze is directed (when the user 50 is located at the front of the Santa Fe and looks at the grill for 3 seconds and the headlamps for more than 10 seconds) , the first service determination unit 130 may determine that the headlamp, which is the part where the user 50's gaze lasts the longest, is the highest priority object of interest.

The first service determination unit 130 may provide the vehicle model of interest and the sector of interest to the service creation unit 150.

The second service determination unit 140 may determine a starting time to provide service content for an object of interest based on the state of the user 50.

Specifically, the second service determination unit 140 may determine the degree of interest according to the user 50's gaze tracking result and the user 50's reaction. The second service determination unit 140 determines how long the gaze toward a specific object of interest lasts according to the gaze tracking described above and determines the text (STT; Speech to Text) converted from the voice collected from the microphone of the sensor unit 110. The level of interest can be judged based on how many adjectives or exclamations (i.e., meaning reactions, e.g., 'Wow, this is good, it's a little different this time, it's pretty', etc.) are included.

If the gaze on a specific object of interest continues for more than a certain period of time (e.g., 15 seconds) (hereinafter referred to as gaze condition), or if the text according to STT contains adjectives or interjections more than a certain number of times (e.g., 3 times) ( Hereinafter, reaction conditions), the second service determination unit 140 may immediately control the service creation unit 150 to provide the corresponding service content to the service provision unit 160.

In addition, the second service determination unit 140 provides services based on sensing information from sensors (opening and closing sensors of trunk, door, bonnet, etc.) installed on vehicles (vehicles 1 to 3) collected from the sensor unit 110. You can decide the starting point for . The second service determination unit 140 determines whether the user 50 operates the experience item (trunk, door, bonnet, etc.) according to the sensing information of the sensor unit 110, and determines whether the user 50 operates the experience item. In the case of manipulation (hereinafter referred to as an experience recognition condition), the service creation unit 150 can immediately control the service content for the experience item to be provided to the service provider 160.

If any one of the gaze condition, reaction condition, and experience recognition condition is not satisfied even after a certain period of time (e.g., 30 seconds) has elapsed since the second service determination unit 140 is activated, the second service determination unit 140 Can control the service content generated by the service creation unit 150 to be processed into a suggestion form and provided to the service provision unit 160.

That is, the second service determination unit 140 can determine the start time of providing the service created by the service creation unit 150, and the service creation unit 150 may provide the service according to the start time to the service provision unit 160. ) can be transmitted.

The service creation unit 150 may collect service content based on the vehicle type and segment of interest provided by the first service determination unit 130. To this end, the service creation unit 150 may communicate with a service content database (not shown) internal or external to the service providing robot 100 to collect service content corresponding to the vehicle type and sector of interest. Here, service content may refer to information such as features, advantages, usage, specifications, etc. about the object of interest (e.g., the trunk of vehicle 3) according to the vehicle type and segment of interest. The service content database (not shown) may be a rule base-based learning model.

In addition, the service creation unit 150 delivers the service content to the service provider 160 at the start time under the control of the second service determination unit 140, or processes the service content into a proposal form to provide the service provider 160 ) can be transmitted.

The service provider 160 may provide the service content received from the service creation unit 150 to the user 50. At this time, voice display and/or screen display may be used as a method of providing information to the user 50, and for this purpose, the service provider 160 may include a speaker and/or display.

As described above, the service provider 160 can receive two types of service content. That is, when at least one of the gaze conditions, reaction conditions, and experience recognition conditions is satisfied, the service provider 160 provides original service content (e.g., 'Isn't the headlamp pretty? This headlamp has ~~ features). and the design is specialized in ~~', etc.) can be received.

However, if any of the gaze conditions, reaction conditions, and experience recognition conditions are not satisfied, the service provider 160 provides service content processed in the form of a suggestion (for example, 'Are you interested in a headlamp?' ‘Can you explain the characteristics of the headlamp?’, ‘Are you interested in the rear of the vehicle? Among the trunk, rear lamp, and rear camera, which part are you interested in?’, etc.). Thereafter, the service provider 160 receives the original service content (e.g., features of a headlamp) from the service creation unit 150 according to the user 50's selection input through the user interface and provides the user 50 can be provided to.

Additionally, the service provider 160 may end provision of service content under the control of the third service determination unit 170.

The third service determination unit 170 may determine the end point of service provision based on the video and audio provided from the camera and microphone of the sensor unit 110, respectively.

The third service determination unit 170 recognizes the face of the user 50 included in the video, extracts facial expression feature points on the recognized face, and then extracts emotions corresponding to the facial expression feature points using an emotion/emotion classifier. You can. The third service determination unit 170 may classify these emotions into positive or negative categories and store them. For example, emotions such as laughter, surprise, irritation, and depression can be extracted depending on facial expression features. Laughter and surprise can be classified into the positive category, and irritation and depression can be classified into the negative category.

The third service determination unit 170 may recognize the voice of the user 50 included in the voice, extract the text (STT), and then extract a reaction corresponding to the text using an adjective/exclamation classifier. The third service determination unit 170 may classify these reactions into positive or negative categories and store them.

The third service determination unit 170 may determine the end point of service provision based on the emotions and reactions stored in each positive and negative category.

For example, the third service determination unit 170 determines whether the number of emotions and reactions stored in the negative category exceeds a certain number (e.g., 3) or when the number of emotions and reactions stored in the positive category is compared to the number of emotions and reactions stored in the negative category. When the ratio of the number of emotions and reactions stored in exceeds a certain ratio (e.g., double), it can be determined as the end point for service provision. Accordingly, the third service determination unit 170 can control the service provider 160 to immediately end provision of the corresponding service content.

In addition, even while the service provider 160 is providing the service, the object of interest, degree of interest, and experience item tracking operations of the first and second service determination units 130 and 140 may continue, and the object of interest may change or the experience item may be changed. In case of change, the corresponding service determination unit may control other service determination units and the service creation unit 150 so that a preparation mode for the changed object of interest or experience item can be executed.

The user response learning unit 180 may learn user responses based on user responses before service provision and user responses after service provision.

The user response before service provision may include a user response to a specific object of interest according to the vehicle type and segment of interest of the first service determination unit 130, a user selection of a service proposal from the service provision unit 160, etc.

The user reaction after service provision may include emotions and reactions of each positive and negative category stored by the third service determination unit 170.

The user response learning unit 180 receives user responses before service provision and user responses after service provision, and uses a learning algorithm for machine learning to accumulate and learn user responses regarding the type and time of service provision. , various parameters for the operation of the service providing robot 100 can be modified.

For example, if the user 50's selection is not continuously received for the service proposal of the service provider 160, the user response learning unit 180 may change the service proposal method (e.g., the advantage in the feature description proposal). Change to explanation suggestion) The service creation unit 150 can be controlled.

Alternatively, if the ratio of emotions and reactions in the negative category stored by the third service determination unit 170 is excessively high, the user response learning unit 180 changes the order of service content provided by the service provider 160. (e.g., change the order from Features->Advantages->Specs to Advantages->Specs->Features).

The parameters that the user response learning unit 180 can change according to the learning results are not limited to this and may include a certain time and a certain number of times, which are the criteria for judgment, as well as the distance from the user 50, output voice, and screen interface. there is.

Figure 3 is a flowchart showing a method of operating a service providing robot according to an embodiment of the present invention. Figure 4 is a flowchart showing step S2 shown in Figure 3 in more detail. Figure 5 is a flowchart showing step S3 shown in Figure 3 in more detail. Figure 6 is a flowchart showing step S7 shown in Figure 3 in more detail. Figure 7 is a flowchart showing step S8 shown in Figure 3 in more detail.

Referring to FIGS. 3 to 7, the overall operating method of the service providing robot 100 is shown in FIG. 3, and each of FIGS. 4 to 7 shows specific steps among the steps in FIG. 3 in more detail.

First, in FIG. 3, it is assumed that the service providing robot 100 is in standby mode.

The user recognition unit 120 allows the user 50 to enter a designated area (for example, within a certain range from any vehicle or within a certain range of the service providing robot 100) based on the sensing information provided from the sensor unit 110. It is possible to recognize whether the user 50 has entered, and when the user 50's entry is recognized, the operation of the first service determination unit 130 and the second service determination unit 140 can be activated by entering the preparation mode from the standby mode ( S1).

The first service determination unit 130 determines the vehicle type of interest of the user 50 based on the location of the user 50 and the time spent at the location, and tracks the gaze of the user 50 to determine the type of vehicle of interest to the user 50. By determining the interest segment you have, you can determine the vehicle type of interest and the object of interest according to the interest segment (S2).

In Figure 4, step S2 may include steps S21 to S24.

The first service determination unit 130 may receive the location of the user 50 from the position sensor of the sensor unit 110, and the locations of the exhibition store 50 and vehicles (vehicle 1 to vehicle 3) stored in advance. From the map information that stores and the current location of the user 50, the current location of the user 50 determines which of the vehicles (vehicle 1 to vehicle 3) is close to the vehicle and the proximity location for a certain period of time (e.g., 10 seconds). Location tracking can be performed to analyze whether there is an abnormal stay (S21).

The first service determination unit 130 may determine the vehicle type (eg, vehicle 3) of interest to the user 50 based on the location tracking result (S21).

The first service determination unit 130 may receive an image of the face of the user 50 from the camera of the sensor unit 110, analyze the gaze direction of the user 50 in the image, and detect vehicles (vehicles). From the map information storing the positions of vehicles 1 to 3), gaze tracking can be performed to analyze which part of the vehicle type of interest the user 50's gaze direction is toward (S23).

The first service determination unit 130 may determine the area of interest (eg, rear camera, AVM (Around View Monitor) camera) in which the user 50 is interested based on the gaze tracking result (S24).

Again in FIG. 3, in parallel with step S2, the second service determination unit 140 transfers the service content of the service creation unit 150 to the service provision unit 160 to determine the starting point of time to provide it to the user 50. (S3).

In Figure 5, step S3 may include steps S31 to S35.

The second service determination unit 140 determines how long the gaze toward a specific object of interest continues according to the gaze tracking described above (S31), and adds an adjective to the text converted from the voice collected from the microphone of the sensor unit 110. It is possible to determine (i.e., reaction tracking) how many exclamations are included (S32). The second service determination unit 140 may determine the degree of interest based on the gaze tracking result and the reaction tricking result (S33).

The second service determination unit 140 may determine (i.e., experience item tracking) whether the user 50 operates an experience item (trunk, door, bonnet, etc.) based on the sensing information of the sensor unit 110. (S34).

If the gaze on a specific object of interest continues for more than a certain period of time (e.g., 15 seconds) or if the text according to STT contains adjectives or interjections more than a certain number of times (e.g., 3 times) (i.e., if the text has a high degree of interest) case), the second service determination unit 140 determines the current time as the start time (S35) and immediately controls the service creation unit 150 to deliver the corresponding service content to the service provider 160 (S35) Yes in S6).

If the user 50 manipulates the experience item, the second service determination unit 140 determines the current time as the start time (S35), and immediately the service creation unit 150 creates service content for the experience item. It can be controlled to transmit it to the service provider 160 (Yes in S6).

If any one of the gaze condition, reaction condition, and experience recognition condition is not satisfied even after a certain period of time (e.g., 30 seconds) has elapsed since the second service determination unit 140 is activated, the second service determination unit 140 The service provider determines that a certain amount of time (e.g., 10 seconds) has elapsed from the current time as the start time (S35), processes the service content generated by the service creation unit 150 into a proposal form, and sends the service content to the service provider 160 at the start time. ) can be controlled to be transmitted (Yes in S6).

Referring back to FIG. 3 , the service creation unit 150 may collect and generate service content based on the vehicle type and segment of interest provided by the first service determination unit 130 (S4). To this end, the service creation unit 150 may communicate with a service content database (not shown) internal or external to the service providing robot 100 to collect service content corresponding to the vehicle type and sector of interest.

Additionally, the service creation unit 150 may temporarily store service content or service content processed in the form of a suggestion and may be in a service standby state (S5). This is a different state from the standby mode described above and corresponds to a part of the preparation mode.

The service creation unit 150 delivers the service content to the service provider 160 at the start time under the control of the second service determination unit 140, or processes the service content into a proposal form and sends it to the service provider 160. Can be passed on (Yes in S6).

The service provider 160 may provide the service content received from the service creation unit 150 to the user 50 (S7). At this time, voice display and/or screen display may be used as a method of providing information to the user 50, and for this purpose, the service provider 160 may include a speaker and/or display.

In Figure 6, step S7 may include steps S71 to S74.

As described above, the service provider 160 can receive two types of service content. That is, when at least one of the gaze conditions, reaction conditions, and experience recognition conditions is satisfied, the service provider 160 provides original service content (e.g., 'Isn't the headlamp pretty? This headlamp has ~~ features). ', and the design is specialized for ~~', etc.) can be received and provided to the user (50) (S74).

However, if any of the gaze conditions, reaction conditions, and experience recognition conditions are not satisfied, the service provider 160 provides service content processed in the form of a suggestion (for example, 'Are you interested in a headlamp?' Would you like to explain the features of the headlamp?', 'Are you interested in the rear of the vehicle? Among the trunk, rear lamp, and rear camera, which part are you interested in?', etc.) are delivered to the user (50). (S71).

Thereafter, when the user 50 inputs a choice to refuse service provision through the user interface (No in S72), the service provision unit 160 transmits this result to the user response learning unit 180 and provides the service. The robot 100 may enter standby mode.

When the user 50 enters the selection to allow service provision through the user interface (Yes in S72), the service provider 160 generates the original service content (e.g., head lamp) from the service creation unit 150. characteristics) can be delivered and provided to the user 50 (S73).

Referring again to FIG. 3, the third service determination unit 170 may determine the end point of service provision based on the video and audio provided from the camera and microphone of the sensor unit 110, respectively (S8).

In Figure 7, step S8 may include steps S81 to S86.

The third service determination unit 170 analyzes the video to recognize the face of the user 50 included in the video (S81), extracts facial expression feature points on the recognized face (S82), and then uses the emotion/emotion classifier to determine the facial expression. Emotions corresponding to feature points can be extracted (S83). The third service determination unit 170 may classify these emotions into positive or negative categories and store them.

The third service determination unit 170 analyzes the voice, recognizes the voice of the user 50 included in the voice (S84), extracts the text (S85), and creates a reaction corresponding to the text using an adjective/interjection classifier. Can be extracted (S86). The third service determination unit 170 may classify these reactions into positive or negative categories and store them.

Referring again to FIG. 3, the third service determination unit 170 may determine the end point of service provision based on the emotions and reactions stored in each positive and negative category (S8). Additionally, the user response learning unit 180 may learn user responses based on user responses before service provision and user responses after service provision (S8).

The user response before service provision may include a user response to a specific object of interest according to the vehicle type and segment of interest of the first service determination unit 130, a user selection of a service proposal from the service provision unit 160, etc.

The user reaction after service provision may include emotions and reactions of each positive and negative category stored by the third service determination unit 170.

The user response learning unit 180 receives user responses before service provision and user responses after service provision, and can modify various parameters for the operation of the service provision robot 100 using a learning algorithm for machine learning. there is.

The number of emotions and reactions stored in the negative category exceeds a certain number (e.g., 3), or the ratio of the number of emotions and reactions stored in the negative category to the number of emotions and reactions stored in the positive category is a certain ratio ( For example, if it exceeds 2 times), the third service determination unit 170 may determine the current time as the end point for service provision. Accordingly, the third service determination unit 170 can control the service provision unit 160 to immediately end provision of the corresponding service content (negation of S9). At this time, the service providing robot 100 may enter standby mode so as not to provide service to the user (S10).

The number of emotions and reactions stored in the negative category does not exceed a certain number (e.g., 3), and the ratio of the number of emotions and reactions stored in the negative category to the number of emotions and reactions stored in the positive category is a certain ratio. If it does not exceed (eg, twice), the third service determination unit 170 may determine the point in time at which the service provision unit 160 completes provision of service content as the end point for service provision. Accordingly, when the provision of the service content by the service provider 160 is completed (S9 affirmative), the third service determination unit 170 uses the user recognition unit 120 to prepare for service provision to the user. ) can be controlled to track the object of interest of the user.

In addition, when the user 50 leaves a certain range (e.g., within 2 meters from the vehicle) from the location of the vehicle type of interest within the preparation mode or service mode of the service providing robot 100, the first service determination unit 130 Can immediately stop additional operations and control the service providing robot 100 to enter standby mode.

The steps of the method or algorithm described in connection with the embodiments disclosed herein may be implemented directly in hardware, software modules, or a combination of the two executed by a processor. Software modules may reside in a storage medium such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, or CD-ROM. An exemplary storage medium is coupled to a processor, the processor capable of reading information from and writing information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and storage medium may reside as separate components within the user terminal.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

10: Exhibition store
50: user
100: Service provision robot

Claims (20)

A sensor unit including at least one sensor for collecting information from the outside;
a first service determination unit that determines an object of interest for the user based on the user's status;
a second service determination unit that determines a starting time to provide service content for the object of interest based on the user's status and sensing information provided from the sensor unit;
a third service determination unit that determines an end point of providing the service content based on the video and audio for the user;
a user recognition unit that identifies whether the user has entered a designated area based on the sensing information provided from the sensor unit; and
A service provider that provides the service content to the user according to the start time,
The user recognition unit,
Based on the sensing information provided from the sensor unit, and based on identifying that the user has entered the designated area, change from standby mode to ready mode, activating the first service determination unit and the second service determination unit; ,
The third service judgment department,
A service providing robot at a vehicle exhibition store that determines an end point of providing the service content based on video and audio for the user while providing the service content. According to paragraph 1,
The first service determination unit is a service providing robot that determines a vehicle type of interest based on the user's location and the time spent at the location. According to paragraph 1,
The first service determination unit is a service providing robot that determines a section of interest by tracking the user's gaze and the time the gaze lasts. According to paragraph 1,
The second service determination unit is a service providing robot that determines whether the gaze condition for whether the user's gaze continues for more than a certain time and the reaction condition for whether the user's voice contains an adjective or exclamation more than a certain number of times are satisfied. . According to paragraph 4,
The second service determination unit is a service providing robot that determines whether the user satisfies the experience recognition condition for manipulating the experience item. According to clause 5,
The second service determination unit controls the service provider to immediately provide the service content to the user when at least one of the gaze condition, the reaction condition, and the experience recognition condition is satisfied. According to clause 5,
The second service determination unit, when any one of the gaze condition, the reaction condition, and the experience recognition condition is not satisfied, controls the service provider unit to provide service content processed in the form of a suggestion to the user. According to paragraph 1,
Collects the service content for the object of interest, and delivers the service content to the service provider at the start time under the control of the second service determination unit, or processes the service content into a proposal form and delivers it to the service provider. A service providing robot further comprising a service creation unit. According to clause 8,
The service providing unit provides service content processed in the form of the suggestion to the user, and provides the service content according to the user's selection input through a user interface. delete According to paragraph 1,
The third service judgment department,
A service providing robot that recognizes the user's face included in the video, extracts facial expression feature points, extracts emotions corresponding to the facial expression feature points using an emotion/emotion classifier, classifies the emotions into positive or negative categories, and stores them. . According to paragraph 1,
The third service judgment department,
A service providing robot that recognizes the user's voice included in the voice, extracts text, extracts a reaction corresponding to the text using an adjective/exclamation classifier, classifies the reaction into a positive or negative category, and stores the reaction. According to clause 12,
The third service determination unit determines whether the number of emotions and reactions stored in the negative category exceeds a certain number, or the ratio of the number of emotions and reactions stored in the negative category to the number of emotions and reactions stored in the positive category is A service provision robot that controls the service provider to immediately terminate provision of the service content when a certain ratio is exceeded. According to paragraph 1,
A service providing robot further comprising a user response learning unit configured to modify at least one parameter by accumulating and learning user responses based on user responses before service provision and user responses after service provision. The service providing robot of claim 1; and
At least one vehicle that communicates with the service providing robot to provide status information,
The service providing robot is,
Based on the sensing information provided from the sensor unit, and based on identifying that the user has entered the designated area, changing from the standby mode to the ready mode, activating the first service determination unit and the second service determination unit,
A store service providing system, configured to determine an end point of providing service content, based on video and audio for the user, while providing service content. Based on the sensing information provided from the sensor unit, and based on identifying that the user has entered the designated area, changing from the standby mode to the preparation mode and activating the first service determination unit and the second service determination unit;
determining an object of interest for the user based on the user's status;
determining a starting time to provide service content for the object of interest based on the user's status and sensing information provided from the sensor unit;
providing the service content to the user according to the start time; and
A method of operating a service providing robot in a vehicle exhibition store, comprising determining an end point of providing the service content based on video and audio for the user while providing the service content. According to clause 16,
The step of determining the object of interest of the user is,
determining a vehicle type of interest based on the user's location and the time spent at the location; and
A method of operating a service providing robot including the step of determining a section of interest by tracking the user's gaze and the time the gaze lasts. According to clause 16,
The step of determining the starting point is,
determining whether the gaze condition of whether the user's gaze continues for more than a certain period of time and the reaction condition of whether the user's voice includes an adjective or an exclamation more than a certain number of times are satisfied; and
A method of operating a service providing robot including the step of determining whether the user satisfies an experience recognition condition for manipulating an experience item. According to clause 18,
The step of determining the starting point is,
A method of operating a service providing robot comprising the step of controlling to immediately provide the service content to the user when at least one of the gaze condition, the reaction condition, and the experience recognition condition is satisfied. According to clause 18,
The step of determining the starting point is,
A method of operating a service providing robot that controls providing service content processed in the form of a suggestion to the user when any one of the gaze condition, the reaction condition, and the experience recognition condition is not satisfied.

Images