US20140170619A1 - Apparatus and method for creating artificial feelings - Google Patents

Apparatus and method for creating artificial feelings Download PDF

Info

Publication number
US20140170619A1
US20140170619A1 US13/714,344 US201213714344A US2014170619A1 US 20140170619 A1 US20140170619 A1 US 20140170619A1 US 201213714344 A US201213714344 A US 201213714344A US 2014170619 A1 US2014170619 A1 US 2014170619A1
Authority
US
United States
Prior art keywords
feeling
value
group
probability
distribution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US13/714,344
Other versions
US9324245B2 (en
Inventor
Ho Seok Ahn
Dong Wook Lee
Woong Hee Shon
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Korea Institute of Industrial Technology KITECH
Original Assignee
Korea Institute of Industrial Technology KITECH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Korea Institute of Industrial Technology KITECH filed Critical Korea Institute of Industrial Technology KITECH
Priority to US13/714,344 priority Critical patent/US9324245B2/en
Assigned to KOREA INSTITUTE OF INDUSTRIAL TECHNOLOGY reassignment KOREA INSTITUTE OF INDUSTRIAL TECHNOLOGY ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AHN, HO SEOK, LEE, DONG WOOK, SHON, WOONG HEE
Publication of US20140170619A1 publication Critical patent/US20140170619A1/en
Application granted granted Critical
Publication of US9324245B2 publication Critical patent/US9324245B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00Teaching not covered by other main groups of this subclass
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/004Artificial life, i.e. computing arrangements simulating life
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63HTOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H3/00Dolls

Definitions

  • FIG. 1 is a schematic view illustrating a conventional robot's feeling expression method.
  • one feeling distribution is expressed as one generative probability model having a plurality of Gaussian distributions G1 or a Gaussian mode
  • the feeling space is expressed as a plurality of generative probability models which define a volume space.
  • the volume space might be expressed as a plurality of feeling values which are largest at the coordinate of the basic feeling and get gradually reduced as it gets farther from the coordinate of the basic feeling.
  • the feeling value is an intensity which represents feeling. For example, when assuming that there exists only feeling of pure happiness, the intensity of a specific happiness might be different in terms of such happiness.
  • the feeling value is designed to express the magnitude of such happiness.
  • Gaussian curve can define the personalities of a robot.
  • ⁇ j,m (k) is a m-th Gaussian distribution of a j-th feeling distribution or a weight value of a Gaussian mode at the time of k
  • M j (k) is the number of Gaussian modes that the j-th feeling distribution has at the time of k
  • ⁇ j,m (k) is a central value of the m-th Gaussian mode of the j-th feeling distribution at the time of k
  • ⁇ j,m (k) is a bandwidth of the m-th Gaussian mode of the j-th feeling distribution at the time of k
  • x(k) is the internal state input value at the time of k.
  • the Gaussian mode means one Gaussian curve.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Feedback Control In General (AREA)

Abstract

An apparatus and a method for creating combined artificial feelings to which different basic feelings like a human being's combined feeling are reflected. The apparatus for creating an artificial feeling comprises a feeling value group creation part which is configured to create a feeling value group having a feeling value which gradually decreased as it gets farther from a basic feeling assigned to a machine apparatus on an internal state coordinate system of a machine apparatus; and a feeling creation part which is configured to create, as a feeling of a machine apparatus, a group of a feeling value of each feeling value group that a coordinate of an internal state input value of the machine apparatus indicates on the internal state coordinate system.

Description

    BACKGROUND OF THE INVENTION
  • 1. Field of the Invention
  • The present invention relates to an apparatus and a method for creating artificial feelings, and particularly to an apparatus and a method for creating artificial feelings which make it possible to actually express the feelings of a machine apparatus such as a robot, etc. like human being's combined feelings in such a way that the current feelings of a robot are created in the forms of the groups consisting of feeling values of different feeling value groups having a plurality of basic feelings.
  • 2. Description of Related Art
  • The feelings of a machine apparatus such as a robot, etc. is generally limited to the feelings created at a specific position of a feeling space with a certain number of feelings which is previously set in accordance with an input via a sensor.
  • FIG. 1 is a schematic view illustrating a conventional robot's feeling expression method.
  • The calculation of the current feeling value of a robot is needed in order to make a robot express its feelings. Emotion, in other words, feelings is rarely decided in a specific form of feeling like happiness or sadness. Though a human being currently feels happiness, part of another feeling such as a feeling of surprise and a feeling of anger appears in a combined form, inevitably reflected. In other words, the expression of a feeling comes from a result of the reflection of very combined and detailed feelings. In order to implement an actual feeling expression in a robot, the feeling values adapted to a robot might be expressed in a form of vectors which reflects different and detailed feelings such as happiness, sadness, surprise, anger, etc.
  • As shown in FIG. 1, a feeling and a feeling expression corresponding to the feeling are mapped on a certain position of a fixed dimension space with the aid of a 2-dimensionally or a 3-dimensionally fixed space so as to express the feelings of a robot. The feeling values can be expressed and calculated as a vector value corresponding to a certain position in the space.
  • In other words, feelings are mapped on multiple points in a vector space, and a feeling expression corresponding to each feeling is mapped at 1:1. If a specific feeling vector is given, one feeling which is closest among the feelings mapped on the specific feeling vector and in the vector spaces is selected, and the feeling mapped at 1:1 with the selected feeling can be finally expressed.
  • Since there is a limit in manually mapping the feeling and the feeling expression corresponding to the feeling on a number of coordinates in the vector space, the conventional method of FIG. 1 is directed to selecting a small number of coordinates and to mapping a feeling corresponding to each coordinate and a feeling expression behavior corresponding to the feeling and then to analyzing a feeling value of a robot and to selecting a feeling of the closest coordinate, thus performing a feeling expression.
  • For example, a feeling value 1 {happiness 1, sadness 0, surprise 0, anger 0} is set to be expressed at a coordinate of 1 in a 4-dimensional vector space, and when a feeling value 2 {happiness ¾, sadness ¼, surprise 0, anger 0} and a feeling value 3 {happiness ¾, sadness 0, surprise ¼, surprise 0} are closer to a coordinate of 1 than the coordinate which expresses another feeling, the feeling values 1, 2, 3 all perform the feeling expressions set in the coordinate of 1.
  • In the above mentioned way, the conventional method is configured in such a way that though the internally, actually created feeling values differ from each other, the most similar one among the feeling values mapped on the coordinate of 1 is selected as the selected feeling value. Since the feeling expression behavior is selected based on the feeling values of the same coordinate, the types of the expressions appearing by way of an expression organ are same.
  • Referring to FIG. 2, it will be described in more details. FIG. 2 is a schematic view for explaining a procedure for creating feelings of a robot from a feeling state input value in the conventional art. In FIG. 2, a 1-dimensional feeling coordinate system is used for simplified explanation.
  • It is assumed that a basic feeling of happiness is set on the coordinate of 1 of the feeling coordinate system, and a basic feeling of sadness is set on the coordinate of −1. When an input value is 0.3, the basic feeling closest to 0.3 is extracted. As shown in FIG. 2, since 0.3 is closer to 1 than −1, the basic feeling of happiness is extracted. Since the basic feeling of happiness is on the coordinate of 1 of the feeling coordinate system, the feeling of the robot finally becomes the coordinate of 1.
  • According to the above described feeling creation method, though the input value is 0.5, the final feeling of the robot will become the coordinate of 1 like the input value is 0.3.
  • As for the feeling expression apparatus which expresses with eyes, a mouth, a gesture, etc. by receiving the feelings of the robot created by the above described feeling creation method, though the feeling state input values are different like 0.3, 0.5, etc., the same coordinate of 1 is expressed as the feeling of the robot. So, though the input values are different in most of the conventional robot, the same feelings are expressed.
  • In terms of the feelings of a human being, happiness could be combined with other feelings such as sadness, surprise, etc. So, the degree of the happiness appears different depending on how much another feeling such as sadness, surprise, etc. is reflected in the happiness. There might also be a difference in the expression of feelings depending on the degree of happiness.
  • Since what the robot's feeling expression most needs is a feeling expression which is closest to the expression of a human being, a combined feeling should be created, which combined feeling needs a combined feeling of a human being in order for a robot to express the feeling which is closest to that of a human being.
  • The Korean patent publication number 2007-0061054 discloses a robot and a method which are directed to creating a plurality of feelings such as a first feeling and a second feeling; however the first feeling disclosed in the above mentioned publication comes from a feeling (surprise feeling, fear feeling, etc.) created in the form of a robot's feeling without external evaluations based on the information received from a sensor part, and the second feeling comes from a feeling (feelings of happiness, anger, rejection, neutral position, sadness, partial fear, etc.) created by means of an evaluation based on a list standard of a database in addition to the information from a sensor and an influence evaluation of time, etc. In other words, the above mentioned patent publication does not disclose combined feelings which are closest to the combined feelings of a human being, so the above described patent publication has the limits like the technologies of FIGS. 1 and 2.
  • SUMMARY OF THE INVENTION
  • Accordingly, it is an object of the present invention to provide an apparatus and a method for creating artificial feelings which make it possible to more actually implement a feeling expression of a robot like a combined feeling of a human being in such a way to create the current feelings of a robot in the forms of the groups consisting of feeling values of different feeling value groups having a plurality of basic feelings.
  • The technical problems to be resolved by the present invention are not limited to the above mentioned technical problems, and the other technical problems not mentioned might be clearly understood to an ordinary person skilled in the art.
  • To achieve the above objects, there is provided an apparatus for creating an artificial feeling, comprising a feeling value group creation part which is configured to create a feeling value group having a feeling value which gradually decreases as it gets farther from a basic feeling assigned to a machine apparatus on an internal state coordinate system of a machine apparatus; and a feeling creation part which is configured to create, as a feeling of a machine apparatus, a group of a feeling value of each feeling value group that a coordinate of an internal state input value of the machine apparatus indicates on the internal state coordinate system.
  • In the embodiment, the feeling value group is either one Gaussian distribution having a weight value, a central value and a distribution or a Gaussian distribution with which a plurality of Gaussian distributions are combined.
  • In the embodiment, the feeling value group creation part is configured to form one feeling value group by combining a plurality of feeling value groups of the same basic feeling.
  • In the embodiment, each feeling value of the group of the feeling value is expressed in a form of probability.
  • In the embodiment, the feeling e(k) of the machine apparatus can be expressed like equation 1 below:

  • e(k)=[P(e 1 |x(k))P(e 2 |x(k)) . . . P(e j |x(k)) . . . P(e J |x(k))]T  [Equation 1]
  • where J means the number of basic feelings, and ej means the feeling value of the j-th basic feeling, and in the equation 1, the feeling probability distribution or feeling value group of P(x(k)|ej) with respect to the j-th basic feeling on the n-dimensional internal state coordinate system can be decided like the following equation 2:
  • P ( x ( k ) | e j ) = m = 1 M j ( k ) ω j , m ( k ) · η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) here , η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) = 1 ( 2 π ) n 2 Σ j , m ( k ) 1 2 · exp { - 1 2 [ x ( k ) - μ j , m ( k ) ] T Σ j , m - 1 ( k ) [ x ( k ) - μ j , m ( k ) ] } [ Equation 2 ]
  • where ωj,m(k) is a weight value of a m-th Gaussian mode of a j-th feeling distribution at the time of k, and Mj(k) is the number of Gaussian modes that the j-th feeling distribution has at the time of k, and μj,m(k) is a central value of the m-th Gaussian mode of the j-th feeling distribution at the time of k, and Σj,m(k) is a bandwidth of the m-th Gaussian mode of the j-th feeling distribution at the time of k, and x(k) is the internal state input value at the time of k, and [ ]T is a transposed matrix, and
  • the probabilistic expression (probability value) of each feeling value of the feeling probability distribution or feeling value group P(x(k)|ej) can be calculated using Bayes's rule like the following equation 3:
  • P ( e j | x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) P ( x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) i = 1 J P ( x ( k ) | e i ) P ( e i ) [ Equation 3 ]
  • where P(ej|x(k)) which is a probability value of the j-th feeling is used as a posterior probability, and P(x(k)|ej) is used as a likelihood function, and P(ej) is a probability in which each feeling value can be selected as the prior probability of ej, and the sum of the prior probability of each feeling value is 1.
  • To achieve the above objects, there is provided a method of calculating an artificial feeling, comprising creating a feeling value group having a feeling value which gradually decreases as it gets farther from a basic feeling assigned to a machine apparatus on an internal state coordinate system of a machine apparatus; and creating, as a feeling of a machine apparatus, the group of a feeling value of each feeling value group that a coordinate of an internal state input value of the machine apparatus indicates on the internal state coordinate system.
  • In the embodiment, the step of creating the feeling value group comprises creating one feeling value group in such a way to combine a feeling value group of a plurality of basic feelings by adapting GMM (Gaussian Mixture Model).
  • In the embodiment, the step of creating, as a feeling of a machine apparatus, the group of a feeling value of each feeling value group comprises converting each feeling value to be expressed in a form of probability by adapting Bayes' rule to the feeling value of the group.
  • In the embodiment, the step of creating, as an artificial feeling, the group of a feeling value of each feeling value group comprises creating, as a feeling of a machine apparatus, e(k) which can be defined by the equation 1.
  • According to another aspect of the present invention, there is provided a computer readable recording medium which can be read by a computer with a program. Here, the program represents a certain means (software) for executing a method for creating feelings of a machine apparatus.
  • Advantageous Effects
  • According to the present invention, it is advantageous to provide an apparatus and a method for creating artificial feelings which are directed to creating the feelings of a robot in the forms of a group of feeling values of a feeling value group consisting of a plurality of basic feelings as compared to the way that the feelings of a robot are created only with a basic feeling value.
  • In addition, according to the present invention, an apparatus and a method for creating artificial feelings are provided, which can create a combined feeling to which various basic feelings like a human being's combined feelings are reflected in a combined form. The thusly created combined feelings can be used when more actually implementing the expression of feelings of a robot.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a schematic view illustrating a feeling space of a conventional robot.
  • FIG. 2 is a schematic view illustrating a procedure that the feelings of a robot are created from a feeling state input value in the conventional art.
  • FIG. 3 is a schematic block diagram illustrating an artificial feeling creation apparatus according to an embodiment of the present invention.
  • FIG. 4 is a schematic view illustrating a feeling probability distribution which can be adapted to a feeling creation apparatus of a robot of FIG. 3.
  • FIG. 5 is a schematic view for explaining a feeling creation procedure when the same basic feeling is added as a feeling creation method which can be adapted to a feeling creation apparatus of a robot of FIG. 3.
  • FIG. 6 is a schematic view for explaining an operation of a feeling creation apparatus of a robot according to the present invention.
  • FIG. 7 is a flow chart of a feeling creation method of a robot according to an embodiment of the present invention.
  • FIG. 8 is a schematic view for explaining a decision procedure of a feeling probability according to the present invention.
  • DESCRIPTION OF SPECIFIC EMBODIMENTS
  • Hereinafter the apparatus and method for creating artificial feelings according to the present invention will be described with reference to the accompanying drawings.
  • FIG. 3 is a schematic block diagram illustrating an artificial feeling creation apparatus according to an embodiment of the present invention. FIG. 4 is a schematic view illustrating a feeling probability distribution which can be adapted to a feeling creation apparatus of a robot of FIG. 3.
  • FIG. 4 shows a coordinate representing an internal state space which consists of openness to experience, conscientiousness, extraversion, agreeableness and neuroticism which are classified in accordance with FFM (Five Factor Model) in the field of psychology. The conical configurations with the same textures of FIG. 4 each represent a specific feeling. One conical configuration represents one feeling probability distribution, and as shown in FIG. 4, a probability distribution curve representing feelings called G1 is drawn over closer to a basic internal state axis of a robot which is called extraversion and agreeableness.
  • As shown in FIG. 3, the feeling creation apparatus of a robot comprises a feeling value group creation part 110 and a feeling creation part 130. The feeling creation apparatus comprises a feeling value group creation part 110 which is configured to create a feeling value group having a feeling value which gradually decreases it gets farther from the basic feeling assigned to the robot in an internal state space of the robot, and feeling creation part 130 which is configured to create, as the feeling of a robot, the group of the feeling value of each feeling value that the coordinate of the internal state input value of the robot indicates in the internal state space.
  • In other words, as shown in FIG. 4, the feeling value group creation part 110 serves to create the feeling distribution of a conical model, and the feeling creation part 130 serves to create the group of the feeling value by way of an externally inputted value and the feeling distribution created by the feeling value group creation part 110.
  • The feeling value group creation part 110 serves to set one feeling distribution in an internal state space as shown in FIG. 4 in a different way as compared to when one feeling is set at one coordinate of the feeling coordinate as shown in FIG. 2.
  • Here, one feeling distribution is expressed as one generative probability model having a plurality of Gaussian distributions G1 or a Gaussian mode, and the feeling space is expressed as a plurality of generative probability models which define a volume space. At this time, the volume space might be expressed as a plurality of feeling values which are largest at the coordinate of the basic feeling and get gradually reduced as it gets farther from the coordinate of the basic feeling. In addition, the feeling value is an intensity which represents feeling. For example, when assuming that there exists only feeling of pure happiness, the intensity of a specific happiness might be different in terms of such happiness. The feeling value is designed to express the magnitude of such happiness.
  • The basic feelings are happiness, sadness, anger, surprise, fear, etc. that the user can assign to a robot. Such basic feelings are supposed to be set on the internal state space coordinate system of the robot.
  • Here the feeling value group can be set in different ways. The configuration of the volume space created from such feeling value group might express the personality and tendency of a robot. For example, as shown in FIG. 4, it might be set that the conical configurations with the same textures have the same feelings, and the conical configuration might differ from each robot, so it can be the natural personality of the robot. It is preferable that when setting the feeling value groups, the feeling value can be set to be the highest on the coordinate of the basic feeling, and the feeling value can be set to get gradually reduced as it gets farther from the coordinate of the basic feeling, which features are advantageous in terms of the natural expression of the feelings, but they are not limited thereto.
  • The feeling value group can be created in accordance with the internal state input values. There might be an initial external stimulation in order for the robot to express a certain feeling. Such stimulation is detected by a feeling-related sensor installed in a robot, and a result of the detection of the sensor is processed in a form of data which can be expressed on the coordinate system of the internal space state that the robot has. The thusly processed data can become an internal state input value. As it is configured that the feeling value group can change in accordance with an internal state input value, a dynamic feeling expression can be made possible.
  • The basic feeling might be configured to have the same basic feelings in multiple numbers. At this time, the feeling value group creation part 110 might form one feeling value group by combining the feeling value group of the same basic feelings.
  • In case of a robot to which a learning ability is assigned, the feeling coordinates of happiness might change, be added and deleted by means of the learning. For example, as shown in FIG. 5, the coordinate of 0.5 representing happiness might be added with the aid of the learning in addition to the coordinate of 1 of happiness. The feeling value group creation part 110 serves to create a feeling value group for respective happiness. In this case, since there are two same feelings, they can be combined and expressed one feeling value group.
  • FIG. 5 is a schematic view for explaining a feeling creation procedure when the same basic feeling is added as a feeling creation method which can be adapted to a feeling creation apparatus of a robot of FIG. 3.
  • FIG. 5 is a schematic view illustrating the occasion that each feeling value group forms a Gaussian cure about the coordinate of the basic feeling while showing a state that the Gaussian curve of the coordinate of 1 of happiness and the Gaussian curve of 0.5 of happiness which is newly added are combined by means of GMM (Gaussian Mixture Model).
  • The feeling creation part 130 serves to create, as the feeling of a robot, the group of the feeling value of each feeling value group that the coordinate of the internal state input value of a robot indicates on the coordinate system of the internal state space. The procedure in which the group of the feeling value of each feeling value group is created as a feeling of a robot will be described below in more details.
  • In the present embodiment of the present invention, the feeling value group is expressed in a form of Gaussian curve; however it can be expressed in forms of various curves which satisfy the condition of the earlier described volume space, provided that when using Gaussian curve, etc. which has the same distribution about the central value (feeling value, etc. with respect to basic feelings), the distribution of the feeling value defined about the reference feeling on the internal state coordinate of the robot is uniformly formed, so the calculation of the feeling vector representing the feelings of the robot can be made easier, but it is not limited thereto.
  • FIG. 6 is a view illustrating a feeling model which has two feeling probability distributions (feeling value group) with respect to different feelings on the 1-dimensional feeling space according to an embodiment of the present invention while showing an operation of the feeling creation apparatus for a robot. Referring to FIG. 6, the feeling model will be described below with the aid of a feeling curve positioned in a 1-dimensional internal state space for the sake of simplified descriptions.
  • As shown in FIG. 6, the coordinate of 1 of happiness and the coordinate of −1 of sadness are assigned to the 1-dimensional coordinate, and there is assigned a feeling probability distribution which defines Gaussian curve with respect to each basic feeling.
  • If the internal state input value is 0.3, the coordinate of 1 of happiness was the feeling of the robot in the conventional art; however in the feeling creation part 130 according to the present embodiment, the group {4, 1} of the feeling value of each feeling value group that the coordinate of 0.3 indicates becomes the feeling of a robot in the present invention. For the sake of the above expression, it is needed to previously set the order of elements of the group. In the present embodiment, it is set like {feeling value of happiness, feeling value of sadness}.
  • According to the present embodiment, it is possible to reliably create a combined feeling in which the magnitude of feeling of happiness is 4, and the magnitude of feeling of sadness is 1 without expressing the feeling of a robot with the coordinate of 1 which simply expresses happiness. At this time, since the combined feeling changes depending on the configuration of Gaussian curve, Gaussian curve can define the personalities of a robot.
  • In addition, when one internal state input value represents a plurality of input values in the feeling probability distribution, one feeling value can be extracted by selecting one of the largest value, the lowest value or a value closet to the mean value. Consequently, when the internal state coordinate system is expanded in a form of multiple dimensions, only one of the input values that the internal state input value indicates can be extracted per internal state.
  • Meanwhile the feeling creation part 130 can help express, in a form of probability, the feeling value of the group which is each element of the combined feelings in consideration with the simplified operations of the feeling expression apparatus of a robot which defines the feelings in accordance with an input. The probability expression means that the feeling value is expressed in a form of percentage or in a form of fraction of the total sum of 1. For example, if the feeling of a robot is {4, 1}, it might be expressed as {80, 20} in a form of percentage, and it can be expressed as {⅘, ⅕} in a form of fraction of the total sum of 1.
  • The behavior of the feeling creation apparatus of a robot according to the present invention will be described with the aid of mathematical equations.
  • A basic feeling is set on the internal state coordinate system. The feeling value of each feeling is decided by means of an internal state input value and a feeling probability distribution of each basic feeling, and the group of them defines a combined feeling. The probability distribution (feeling value group) of each feeling might be defined using GMM (Gaussian Mixture Model), and the probability expression of each feeling value can be calculated using Bayes' rule. The details thereon are described as below.
  • 1) Definition of Internal State Vector x(k) which is a Creation Input Value of Feeling Value Group
  • The feeling of a robot is created based on an internal state input value (vector) on the internal state coordinate system. The internal state input value x(k) is decided as the current state of a robot by a surrounding environment or an input from a sensor, which value might be decided in many different ways. As an example, it can be decided by NEO PI-R (the Revised NEO Personality Inventory), but they are not limited thereto.
  • The internal state input value x(k) has the same dimension as the number of the axes of the internal state coordinate system. For example, when it has an internal state of a 5-dimensional space using FFM (Five Factor Model in psychology), the internal sate input value x(k) is a 5th vector, and the state of the robot is decided by means of each value with the aid of the values of openness to experience, conscientiousness, agreeableness and neuroticism.
  • 2) Definition of Feeling Vector which is a Feeling of a Robot
  • The final feeling is not simply expressed with only one feeling value, but is created with a combined value to which the probability values of different feeling values are reflected. The combined feeling can be expressed in a form of feeling vector e(k), and if a J-number of feelings is used, the feelings of a robot can be expressed with the following equation 1.

  • e(k)=[P(e 1 |x(k))P(e 2 |x(k)) . . . P(e j |x(k)) . . . P(e J |x(k))]T  [Equation 1]
  • where J means the number of basic feelings, and ej means the feeling value of the j-th basic feeling. For example, they may be e1=happiness, e2=surprise, e3=sadness, e4=love, e5=disgust, e6=fear, e7=angry. Since [ ]T is a transposed matrix and is just one of the expression ways, so it might be deleted.
  • 3) Setting of Feeling Probability Distribution P(x(k)|ej)
  • The probability distribution with respect to each basic feeling is arranged on the internal state coordinate system. The probability distribution with respect to each basic feeling uses GMM (Gaussian Mixture Model). With it, it is possible to make dynamic the distribution of each basic feeling, and each feeling value can be independently calculated. The feeling probability P(x(k)|ej) (feeling probability, feeling value group) with respect to a j-th basic feeling on the n-dimensional internal state coordinate system can be decided by the following equation 2.
  • P ( x ( k ) | e j ) = m = 1 M j ( k ) ω j , m ( k ) · η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) here , η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) = 1 ( 2 π ) n 2 Σ j , m ( k ) 1 2 · exp { - 1 2 [ x ( k ) - μ j , m ( k ) ] T Σ j , m - 1 ( k ) [ x ( k ) - μ j , m ( k ) ] } [ Equation 2 ]
  • In addition, ωj,m(k) is a m-th Gaussian distribution of a j-th feeling distribution or a weight value of a Gaussian mode at the time of k, and Mj(k) is the number of Gaussian modes that the j-th feeling distribution has at the time of k, and μj,m(k) is a central value of the m-th Gaussian mode of the j-th feeling distribution at the time of k, and Σj,m(k) is a bandwidth of the m-th Gaussian mode of the j-th feeling distribution at the time of k, and x(k) is the internal state input value at the time of k. The Gaussian mode means one Gaussian curve.
  • 4) Calculation of P(ej|x(k)) which is a Probabilistic Expression of a Feeling Value
  • The probabilistic expression (probability value) of each feeling value can be calculated with the mathematic equation 3 using Bayes rule, and P(ej|x(k)) which is the probability value of the j-th feeling can be used as the posterior probability.
  • P ( e j | x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) P ( x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) i = 1 J P ( x ( k ) | e i ) P ( e i ) [ Equation 3 ]
  • where P(x(k)|ej) can be used as a likelihood function, and P(ej) is a probability (probability expression of feeling value) in which each feeling value can be selected as the prior probability of ej, and the sum of the prior probability of each feeling value is 1.
  • According to the above described three equations 1 to 3, when a j-number of basic feelings is assigned, the feeling vector e(k) of the J-dimension with a j-number of P(ej|x(k)) can be decided. Here, e(k) becomes a combined feeling to which a j-number of feeling values is actually reflected in a form of probabilistic expression.
  • FIG. 1 is a flow chart illustrating a method for creating the feelings of a robot according to an embodiment of the present invention.
  • First of all, the feeling value group having a feeling value which gradually decreases as it get farther from the coordinate of the basic feeling assigned to the robot is created in multiple numbers in the internal state coordinate system of the robot (S510).
  • The feeling value group can be created by adapting GMM (Gaussian Mixture Model) to the operation conducted by means of the feeling value group creation part 110. Afterward, P(x(k)|ej), which is the feeling value group, can be created after the equation 2 is conducted.
  • Next, the group of the feeling value of each feeling value group that the coordinate of the internal state input value indicates of the robot on the internal state coordinate system can be created as a feeling of a robot (S530).
  • Here, the internal state input value can match with the value outputted from an input part (not shown) of the robot. In this case, the input part might be a certain means which is configured to detect external environment information of a robot and outputs an internal state input value to match with one dot or a specific coordinate of the internal state coordinate system or a certain construction part matching with the means.
  • Bayes' rule can be adapted to the feeling value of the group in terms of the operation conducted in the feeling creation part 130, so each feeling value can be further converted, thus expressing each feeling value in a form of probability. P(ej|x(k)) which is a probabilistic expression of the feeling value can be outputted after the equation 3 is conducted.
  • The method for creating the feelings of a robot according to the present can be recorded on a computer-readable recording medium in a form of a program that a computer can execute.
  • While the present invention has been described with respect to the specific embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the following claims.
  • INDUSTRIAL APPLICABILITY
  • The present invention can adapted to an apparatus for creating a machine apparatus such as a robot, etc., and can be used in order to create an input value of a feeling expression apparatus which can express feelings in different ways.

Claims (9)

What is claimed is:
1. An apparatus for creating an artificial feeling, the apparatus comprising:
a feeling value group creation part which is configured to create a feeling value group having a feeling value which gradually decreases as it gets farther from a basic feeling assigned to a machine apparatus on an internal state coordinate system of a machine apparatus; and
a feeling creation part which is configured to create, as a feeling of a machine apparatus, a group of a feeling value of each feeling value group that a coordinate of an internal state input value of the machine apparatus indicates on the internal state coordinate system.
2. The apparatus of claim 1, wherein the feeling value group is either one Gaussian distribution having a weight value, a central value and a distribution or a Gaussian distribution with which a plurality of Gaussian distributions are combined.
3. The apparatus of claim 2, wherein the feeling value group creation part is configured to form one feeling value group by combining a plurality of feeling value groups of the same basic feeling.
4. The apparatus of claim 3, wherein each feeling value of the group of the feeling value is expressed in a form of probability.
5. The apparatus of claim 3, wherein the feeling of the machine apparatus is expressed by e(k) defined by the following equation 1:

e(k)=[P(e 1 |x(k))P(e 2 |x(k)) . . . P(e j |x(k)) . . . P(e J |x(k))]T,
wherein J means the number of basic feelings, and ej means the feeling value of the j-th basic feeling,
wherein the feeling probability distribution or feeling value group of P(x(k)|ej) with respect to the j-th basic feeling on the n-dimensional internal state coordinate system is defined by the following equation 2:
P ( x ( k ) | e j ) = m = 1 M j ( k ) ω j , m ( k ) · η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) , wherein : η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) = 1 ( 2 π ) n 2 Σ j , m ( k ) 1 2 · exp { - 1 2 [ x ( k ) - μ j , m ( k ) ] T Σ j , m - 1 ( k ) [ x ( k ) - μ j , m ( k ) ] }
ωj,m(k) is a weight value of a m-th Gaussian mode of a j-th feeling distribution at the time of k;
Mj(k) is the number of Gaussian modes that the j-th feeling distribution has at the time of k;
μj,m(k) is a central value of the m-th Gaussian mode of the j-th feeling distribution at the time of k;
Σj,m(k) is a bandwidth of the m-th Gaussian mode of the j-th feeling distribution at the time of k;
x(k) is the internal state input value at the time of k;
[ ]T is a transposed matrix; and
the probabilistic expression (probability value) of each feeling value of the feeling probability distribution or feeling value group P(x(k)|ej) is calculated using Bayes's rule as the following equation 3:
P ( e j | x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) P ( x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) i = 1 J P ( x ( k ) | e i ) P ( e i )
wherein:
P(ej|x(k)) is a probability value of the j-th feeling used as a posterior probability;
P(x(k)|ej) is used as a likelihood function;
P(ej) is a probability in which each feeling value can be selected as the prior probability of ej; and
the sum of the prior probability of each feeling value is 1.
6. A method of creating an artificial feeling, the method comprising:
creating a feeling value group having a feeling value which gradually decreases as it gets farther from a basic feeling assigned to a machine apparatus on an internal state coordinate system of a machine apparatus; and
creating, as a feeling of a machine apparatus, the group of a feeling value of each feeling value group that a coordinate of an internal state input value of the machine apparatus indicates on the internal state coordinate system.
7. The method of claim 6, wherein the step of creating the feeling value group comprises:
creating one feeling value group in such a way to combine a feeling value group of a plurality of basic feelings by adapting GMM (Gaussian Mixture Model).
8. The method of claim 6, wherein the step of creating, as a feeling of a machine apparatus, the group of a feeling value of each feeling value group comprises:
converting each feeling value to be expressed in a form of probability by adapting Bayes' rule to the feeling value of the group.
9. The method of claim 6, wherein the step of creating, as an artificial feeling, the group of a feeling value of each feeling value group comprises:
creating, as a feeling of a machine apparatus, e(k) which is defined by the following equation 1:

e(k)=[P(e 1 |x(k))P(e 2 |x(k)) . . . P(e j |x(k)) . . . P(e J |x(k))]T,
wherein J means the number of basic feelings, and ej means the feeling value of the j-th basic feeling, and
wherein the feeling probability distribution or feeling value group of P(x(k)|ej) with respect to the j-th basic feeling on the n-dimensional internal state coordinate system is defined by the following equation 2:
P ( x ( k ) | e j ) = m = 1 M j ( k ) ω j , m ( k ) · η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) , wherein : η ( x ( k ) | μ j , m ( k ) , Σ j , m ( k ) ) = 1 ( 2 π ) n 2 Σ j , m ( k ) 1 2 · exp { - 1 2 [ x ( k ) - μ j , m ( k ) ] T Σ j , m - 1 ( k ) [ x ( k ) - μ j , m ( k ) ] }
ωj,m(k) is a weight value of a m-th Gaussian mode of a j-th feeling distribution at the time of k;
Mj(k) is the number of Gaussian modes that the j-th feeling distribution has at the time of k;
μj,m(k) is a central value of the m-th Gaussian mode of the j-th feeling distribution at the time of k;
Σj,m(k) is a bandwidth of the m-th Gaussian mode of the j-th feeling distribution at the time of k;
x(k) is the internal state input value at the time of k;
[ ]T is a transposed matrix; and
the probabilistic expression (probability value) of each feeling value of the feeling probability distribution or feeling value group P(x(k)|ej) is calculated using Bayes's rule as defined by the following equation 3:
P ( e j | x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) P ( x ( k ) ) = P ( x ( k ) | e j ) P ( e j ) i = 1 J P ( x ( k ) | e i ) P ( e i ) ,
wherein:
P(ej|x(k)) is a probability value of the j-th feeling used as a posterior probability;
P(x(k)|ej) is used as a likelihood function;
P(ej) is a probability in which each feeling value can be selected as the prior probability of ej; and
the sum of the prior probability of each feeling value is 1.
US13/714,344 2012-12-13 2012-12-13 Apparatus and method for creating artificial feelings Expired - Fee Related US9324245B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/714,344 US9324245B2 (en) 2012-12-13 2012-12-13 Apparatus and method for creating artificial feelings

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/714,344 US9324245B2 (en) 2012-12-13 2012-12-13 Apparatus and method for creating artificial feelings

Publications (2)

Publication Number Publication Date
US20140170619A1 true US20140170619A1 (en) 2014-06-19
US9324245B2 US9324245B2 (en) 2016-04-26

Family

ID=50931329

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/714,344 Expired - Fee Related US9324245B2 (en) 2012-12-13 2012-12-13 Apparatus and method for creating artificial feelings

Country Status (1)

Country Link
US (1) US9324245B2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140172163A1 (en) * 2012-12-13 2014-06-19 Korea Institute Of Industrial Technology Apparatus and method for selecting lasting feeling of machine
US20140198991A1 (en) * 2013-01-17 2014-07-17 Fuji Xerox Co., Ltd Image processing apparatus, image processing method, and computer-readable medium
US9324245B2 (en) * 2012-12-13 2016-04-26 Korea Institute Of Industrial Technology Apparatus and method for creating artificial feelings
CN111949591A (en) * 2020-08-12 2020-11-17 珠海格力电器股份有限公司 Demonstrator display method, demonstrator, processor and movable equipment

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9956687B2 (en) * 2013-03-04 2018-05-01 Microsoft Technology Licensing, Llc Adapting robot behavior based upon human-robot interaction

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020156751A1 (en) * 2000-03-24 2002-10-24 Tsuyoshi Takagi Method for determining action of robot and robot
US6484068B1 (en) * 2001-07-24 2002-11-19 Sony Corporation Robot apparatus and method for controlling jumping of robot device
US20020183896A1 (en) * 2000-10-11 2002-12-05 Satoko Ogure Robot apparatus and its control method
US6560511B1 (en) * 1999-04-30 2003-05-06 Sony Corporation Electronic pet system, network system, robot, and storage medium
US20030144764A1 (en) * 2001-02-21 2003-07-31 Jun Yokono Operational control method, program, and recording media for robot device, and robot device
US20040015265A1 (en) * 2002-03-18 2004-01-22 Yasuharu Asano Robot apparatus and method for controlling the operation thereof
US20040093118A1 (en) * 2000-12-06 2004-05-13 Kohtaro Sabe Robot apparatus and method and system for controlling the action of the robot apparatus
US7065490B1 (en) * 1999-11-30 2006-06-20 Sony Corporation Voice processing method based on the emotion and instinct states of a robot
US20070150099A1 (en) * 2005-12-09 2007-06-28 Seung Ik Lee Robot for generating multiple emotions and method of generating multiple emotions in robot
US20070168894A1 (en) * 2006-01-16 2007-07-19 Fujitsu Limited Method and device for supporting verification, and computer product
US20090248372A1 (en) * 2008-03-25 2009-10-01 Electronics And Telecommunications Research Institute Method of modeling composite emotion in multidimensional vector space
US20090285456A1 (en) * 2008-05-19 2009-11-19 Hankyu Moon Method and system for measuring human response to visual stimulus based on changes in facial expression
US20130216098A1 (en) * 2010-09-17 2013-08-22 Tokyo Institute Of Technology Map generation apparatus, map generation method, moving method for moving body, and robot apparatus
US20140093849A1 (en) * 2012-10-01 2014-04-03 Korea Institute Of Industrial Technology Apparatus and method for learning emotion of robot
US20140095570A1 (en) * 2012-09-28 2014-04-03 Korea Institute Of Industrial Technology Apparatus and method for calculating internal state for artificial emotion
US20140172163A1 (en) * 2012-12-13 2014-06-19 Korea Institute Of Industrial Technology Apparatus and method for selecting lasting feeling of machine
US20140172771A1 (en) * 2012-12-13 2014-06-19 Korea Institute Of Industrial Technology Apparatus and method for selecting motion signifying artificial feeling

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9324245B2 (en) * 2012-12-13 2016-04-26 Korea Institute Of Industrial Technology Apparatus and method for creating artificial feelings

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6560511B1 (en) * 1999-04-30 2003-05-06 Sony Corporation Electronic pet system, network system, robot, and storage medium
US7065490B1 (en) * 1999-11-30 2006-06-20 Sony Corporation Voice processing method based on the emotion and instinct states of a robot
US20020156751A1 (en) * 2000-03-24 2002-10-24 Tsuyoshi Takagi Method for determining action of robot and robot
US20020183896A1 (en) * 2000-10-11 2002-12-05 Satoko Ogure Robot apparatus and its control method
US20040093118A1 (en) * 2000-12-06 2004-05-13 Kohtaro Sabe Robot apparatus and method and system for controlling the action of the robot apparatus
US20030144764A1 (en) * 2001-02-21 2003-07-31 Jun Yokono Operational control method, program, and recording media for robot device, and robot device
US6484068B1 (en) * 2001-07-24 2002-11-19 Sony Corporation Robot apparatus and method for controlling jumping of robot device
US20040015265A1 (en) * 2002-03-18 2004-01-22 Yasuharu Asano Robot apparatus and method for controlling the operation thereof
US20070150099A1 (en) * 2005-12-09 2007-06-28 Seung Ik Lee Robot for generating multiple emotions and method of generating multiple emotions in robot
US20070168894A1 (en) * 2006-01-16 2007-07-19 Fujitsu Limited Method and device for supporting verification, and computer product
US20090248372A1 (en) * 2008-03-25 2009-10-01 Electronics And Telecommunications Research Institute Method of modeling composite emotion in multidimensional vector space
US20090285456A1 (en) * 2008-05-19 2009-11-19 Hankyu Moon Method and system for measuring human response to visual stimulus based on changes in facial expression
US20130216098A1 (en) * 2010-09-17 2013-08-22 Tokyo Institute Of Technology Map generation apparatus, map generation method, moving method for moving body, and robot apparatus
US20140095570A1 (en) * 2012-09-28 2014-04-03 Korea Institute Of Industrial Technology Apparatus and method for calculating internal state for artificial emotion
US20140093849A1 (en) * 2012-10-01 2014-04-03 Korea Institute Of Industrial Technology Apparatus and method for learning emotion of robot
US20140172163A1 (en) * 2012-12-13 2014-06-19 Korea Institute Of Industrial Technology Apparatus and method for selecting lasting feeling of machine
US20140172771A1 (en) * 2012-12-13 2014-06-19 Korea Institute Of Industrial Technology Apparatus and method for selecting motion signifying artificial feeling

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Ahn et al, "Can We Teach What Emotions a Robot Should Express?", 12 October 2012, International Conference on Intelligent Robots and Systems, pp. 1407-1412 *
Ahn et al., "A Behavior Combination Generating Method for Reflecting Emotional Probabilities using Simulated Annealing Alogorithm", 03 August 2011, 20th IEEE Internation Symposium on Robot and Human Interactive Communications, pp 192-197. *
Ahn et al., "Natural changing of emotional expression by considering correlation of behavior history", 17 June 2011, Consumer Electronics (ISCE), IEEE 15th International Symposium on Consumer Electronics, pp.369-372, doi: 10.1109/ISCE.2011.5973851, URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5973851&isnumber=5973167 *
Lee et al., "A General Behavior Generation Module for Emotional Robots Using Unit Behavior Combination Method", 2 Ocotober 2009, The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp. 375-380. *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140172163A1 (en) * 2012-12-13 2014-06-19 Korea Institute Of Industrial Technology Apparatus and method for selecting lasting feeling of machine
US9211645B2 (en) * 2012-12-13 2015-12-15 Korea Institute Of Industrial Technology Apparatus and method for selecting lasting feeling of machine
US9324245B2 (en) * 2012-12-13 2016-04-26 Korea Institute Of Industrial Technology Apparatus and method for creating artificial feelings
US20140198991A1 (en) * 2013-01-17 2014-07-17 Fuji Xerox Co., Ltd Image processing apparatus, image processing method, and computer-readable medium
US9098949B2 (en) * 2013-01-17 2015-08-04 Fuji Xerox Co., Ltd Image processing apparatus, image processing method, and computer-readable medium
CN111949591A (en) * 2020-08-12 2020-11-17 珠海格力电器股份有限公司 Demonstrator display method, demonstrator, processor and movable equipment

Also Published As

Publication number Publication date
US9324245B2 (en) 2016-04-26

Similar Documents

Publication Publication Date Title
US8494982B2 (en) Emotion model, apparatus, and method for adaptively modifying personality features of emotion model
US8972313B2 (en) Apparatus and method for learning emotion of robot
US9324245B2 (en) Apparatus and method for creating artificial feelings
JP6020161B2 (en) Graph creation program, information processing apparatus, and graph creation method
US20210397954A1 (en) Training device and training method
US10818063B2 (en) Repurposing existing animated content
KR20130091364A (en) Apparatus and method for generating emotion of robot
JP5624100B2 (en) Artificial emotion learning apparatus and method
EP2112621A2 (en) Apparatus for forming good feeling of robot and method therefor
JP6201212B2 (en) Character generating apparatus and program
Shen et al. Simulating realistic human motion trajectories of mid-air gesture typing
EP1901209A2 (en) System and method for generating robot genome
CN108229640B (en) Emotion expression method and device and robot
US9117168B2 (en) Apparatus and method for calculating internal state for artificial emotion
Ajili et al. Expressive motions recognition and analysis with learning and statistical methods
Algabri Artificial Intelligence And ChatGPT
KR101284642B1 (en) Apparatus for generating emotion of robot
US9037526B2 (en) Apparatus and method for selecting motion signifying artificial feeling
CN106845391B (en) Atmosphere field identification method and system in home environment
JP5927797B2 (en) Robot control device, robot system, behavior control method for robot device, and program
KR20100070773A (en) Personality transition apparatus and method, and behavior diversification apparatus and method using that, and its robot system
US8407173B2 (en) System and method for comparing system features
JP5521240B1 (en) Apparatus and method for selecting an artificial emotional expression operation
JP5076138B2 (en) Pattern generation method, program and apparatus using self-organizing map
JP6692271B2 (en) Multitask processing device, multitask model learning device, and program

Legal Events

Date Code Title Description
AS Assignment

Owner name: KOREA INSTITUTE OF INDUSTRIAL TECHNOLOGY, KOREA, R

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AHN, HO SEOK;LEE, DONG WOOK;SHON, WOONG HEE;SIGNING DATES FROM 20121204 TO 20121205;REEL/FRAME:029467/0162

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Expired due to failure to pay maintenance fee

Effective date: 20200426