CN110379234A - A kind of study coach method and device - Google Patents

A kind of study coach method and device Download PDF

Info

Publication number
CN110379234A
CN110379234A CN201910668360.1A CN201910668360A CN110379234A CN 110379234 A CN110379234 A CN 110379234A CN 201910668360 A CN201910668360 A CN 201910668360A CN 110379234 A CN110379234 A CN 110379234A
Authority
CN
China
Prior art keywords
information
children
robot
emotional state
image information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910668360.1A
Other languages
Chinese (zh)
Inventor
张小杰
邓小兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Genius Technology Co Ltd
Original Assignee
Guangdong Genius Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Genius Technology Co Ltd filed Critical Guangdong Genius Technology Co Ltd
Priority to CN201910668360.1A priority Critical patent/CN110379234A/en
Publication of CN110379234A publication Critical patent/CN110379234A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174Facial expression recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Toys (AREA)

Abstract

The invention belongs to robot fields, disclose a kind of study coach method and device, and method includes: to receive the image information for the children that robot is acquired and sent at interval of preset duration;It is identified to obtain the emotional state of children according to image information;According to emotional state, judge whether children need to teach, if so, obtaining the corresponding guidance information of emotional state, and guidance information is sent to robot;Receive the problem of machine human hair is sent information;Judge whether solvable problem information, if it is not, problem information to be sent to the intelligent terminal bound with robot, if so, obtaining and returning to answer information to robot, makes robot output answer information.The facial expression and movement that the present invention passes through acquisition children, to judge whether children need study coach, and when needing study coach, actively inquire whether children want help, prevent children when the comparison to think over a problem is irritated, forget to seek help to robot, to realize when parent has no time, actively teaches children's study.

Description

A kind of study coach method and device
Technical field
The invention belongs to robotic technology field, in particular to a kind of study coach method and device.
Background technique
Nowadays many parent's work are more and more busier, so that parent's long-time in the preschool education and developmental process of child In default conditions, the company and love for how giving children's high quality are always to perplex the problem of parent.
And with the fast development of artificial intelligence and robot industry, the appearance of AI children robot, largely more It has mended parents to regret to what child can not accompany, parents " can accompany " in another way in child's body side.
Currently, children robot function on the market is also relatively simple, daily chat can only be carried out with children, plays and draws this Story, ancient poetry, nursery rhymes and cartoon etc.;Children's study can not be also actively taught, and actively Children's Solving can not be helped to learn Problem reduces the usage experience of user.
Summary of the invention
The object of the present invention is to provide a kind of study coach method and device, can initiative recognition go out whether children need to learn It teaches, and actively gives children when needing to teach and teach accordingly, improve the intelligence of robot.
Technical solution provided by the invention is as follows:
On the one hand, a kind of study coach method is provided, comprising:
Receive the image information for the children that robot is acquired and sent at interval of preset duration, wherein described image information Facial expression information and action message including the children;
It is identified to obtain the emotional state of the children according to described image information;
According to the emotional state, judge whether the children need to teach, if so, it is corresponding to obtain the emotional state Guidance information, and the guidance information is sent to the robot;
Receive the problem of robot is sent information;
Judge whether solvable described problem information, is tied up if it is not, being then sent to described problem information with the robot Fixed intelligent terminal makes the robot export the answer letter if so, obtaining and returning to answer information to the robot Breath.
It is further preferred that it is described according to the emotional state, judge whether the children need to teach and specifically includes:
Record the read-around ratio that the emotional state is default emotional state;
When the read-around ratio is greater than preset times, judge that the children need to teach.
It is further preferred that described judge whether that solvable described problem information specifically includes:
Described problem information is segmented, keyword is extracted;
It is scanned in preset exam pool according to the keyword;
When searching the topic with the Keywords matching in the exam pool, solvable described problem information is judged;
When not searching the topic with the Keywords matching in the exam pool, judgement can not answer described problem letter Breath.
It is further preferred that the emotional state for identifying to obtain the children according to described image information specifically includes:
By described image information input into preset identification model;
If the identification model can recognize described image information, emotional state is exported;
If the unidentified described image information out of the identification model, calculates the characteristic value of described image information;
Matched emotional state is found in preset feature database according to the characteristic value.
It is further preferred that the creation method of the feature database are as follows:
A large amount of different image informations are collected, described image information includes the facial expression information and movement letter of the children Breath;
Obtain the corresponding status information of described image information;
Extract the feature in each image information;
The characteristic value of the feature is calculated, the characteristic value is that the letter of presetting digit capacity and data are composed;
Establish the corresponding relationship between the characteristic value and the status information;
According to the corresponding pass between the characteristic value, the status information and the characteristic value and the status information System, creates the feature database.
On the other hand, a kind of study coach device is also provided, comprising:
Signal dispatcher module, the image information for the children for acquiring and sending at interval of preset duration for receiving robot, Wherein, described image information includes the facial expression information and action message of the children;
Identification module obtains the emotional state of the children for identifying according to described image information;
Judgment module, for judging whether the children need to teach according to the emotional state;
It obtains module and obtains the corresponding guidance information of the emotional state if needing to teach for the children;
The signal dispatcher module, for the guidance information to be sent to the robot;
The signal dispatcher module, for receiving the problem of robot is sent information;
The judgment module, for judging whether solvable described problem information;
The signal dispatcher module, for when judgement can not answer described problem information, described problem information to be sent To the intelligent terminal bound with the robot;
The acquisition module, for when judging solvable described problem information, obtaining the corresponding solution of described problem information Answer information;
The signal dispatcher module makes robot output institute for returning to the answer information to the robot State answer information.
It is further preferred that the judgment module includes:
Recording unit, for recording the read-around ratio that the emotional state is default emotional state;
Judging unit, for judging that the children need to teach when the read-around ratio is greater than preset times.
It is further preferred that the judgment module includes:
Participle unit extracts keyword for segmenting described problem information;
Search unit, for being scanned in preset exam pool according to the keyword;
Judging unit, for judging solvable when searching the topic with the Keywords matching in the exam pool Described problem information;
The judging unit, for judging when not searching the topic with the Keywords matching in the exam pool Described problem information can not be answered.
It is further preferred that the identification module includes:
Model recognition unit is used for described image information input into preset identification model;If the identification model It can recognize described image information, then export emotional state;
Characteristic value matching unit, if calculating described image for the unidentified described image information out of the identification model The characteristic value of information;And matched emotional state is found in preset feature database according to the characteristic value.
It is further preferred that further including creation module, the creation module includes:
Collector unit, for collecting a large amount of different image informations, described image information includes the facial table of the children Feelings information and action message;
Acquiring unit, for obtaining the corresponding status information of described image information;
Extraction unit, for extracting the feature in each image information;
Computing unit, for calculating the characteristic value of the feature, the characteristic value is the letter and data group of presetting digit capacity It closes;
Unit is established, the corresponding relationship for establishing between the characteristic value and the status information;
Creating unit, for according to the characteristic value, the status information and the characteristic value and the status information Between corresponding relationship, create the feature database.
Compared with prior art, a kind of study coach method and device provided by the invention has the advantages that this Invention to judge whether children need study coach, and is needing study coach by the facial expression and movement of acquisition children When, it actively inquires whether children want help, prevents children when the comparison to think over a problem is irritated, forget to seek help to robot, To realize when parent has no time, children's study is actively taught, in addition, can also make children by exchanging with robot, alleviation The current anxiety of children, irritated mood.
Detailed description of the invention
Below by clearly understandable mode, preferred embodiment is described with reference to the drawings, to a kind of study coach method and Above-mentioned characteristic, technical characteristic, advantage and its implementation of device are further described.
Fig. 1 is a kind of flow diagram of the first embodiment of study coach method of the present invention;
Fig. 2 is a kind of flow diagram of the second embodiment of study coach method of the present invention;
Fig. 3 is a kind of flow diagram of the 3rd embodiment of study coach method of the present invention;
Fig. 4 is a kind of structural schematic block diagram of one embodiment of study coach device of the present invention.
Drawing reference numeral explanation
100, signal dispatcher module;200, identification module;210, model recognition unit;220, characteristic value matching unit; 300, judgment module;310, recording unit;320, judging unit;330, participle unit;340, search unit;400, mould is obtained Block;500, creation module;510, collector unit;520, acquiring unit;530, extraction unit;540, computing unit;550, it establishes Unit;560, creating unit.
Specific embodiment
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, Detailed description of the invention will be compareed below A specific embodiment of the invention.It should be evident that drawings in the following description are only some embodiments of the invention, for For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other Attached drawing, and obtain other embodiments.
It should be appreciated that when used in this manual, term " includes " indicates the Expressive Features, entirety, step, behaviour Make, the presence of element and/or component, but one or more other features, entirety, step, operation, element, component are not precluded And/or the presence or addition of set.
To make simplified form, part related to the present invention is only schematically shown in each figure, they are not represented Its practical structures as product.In addition, there is identical structure or function in some figures so that simplified form is easy to understand Component only symbolically depicts one of those, or has only marked one of those.Herein, "one" is not only indicated " only this ", can also indicate the situation of " more than one ".
The first embodiment provided according to the present invention, as shown in Figure 1, a kind of study coach method, comprising:
S100 receives the image information for the children that robot is acquired and sent at interval of preset duration, wherein described image Information includes the facial expression information and action message of the children;
S200 identifies to obtain the emotional state of the children according to described image information;
S300 judges whether the children need to teach according to the emotional state, if so, obtaining the emotional state Corresponding guidance information, and the guidance information is sent to the robot;
S400 receives the problem of robot is sent information;
S500 judges whether solvable described problem information;
Described problem information if it is not, is then sent to the intelligent terminal bound with the robot by S600;
S700 makes the robot export the answer and believes if so, obtain and return to answer information to the robot Breath.
Specifically, for student in learning process, robot can acquire the image information of a children at interval of certain time, The image information of acquisition should include the facial expression information and action message of children.The facial expression information and action message of children It can reflect out the current emotional state of children.
Camera and infrared detection module are provided in robot.Robot can sense children's by infrared detection module Orientation shoots camera face children to adjust the shooting angle of camera.Illustratively, children are on desk when learning, Robot can be placed on desk, robot by infrared detection module detect children be located at desk where after, Robot can be moved into the right opposite of children position, and the image information of children is then acquired by camera.
After robot collects the image information of children, which is sent to server, server is according to the figure As information identifies to obtain the emotional state of children, such as emotional state is normal or anxiety, agitation, doubt.As children have wrinkle When the expression of eyebrow, then when illustrating that children are more irritated or children's expression is more serious, illustrate that children are likely encountered problem, or Children have the movement of grappling fixture or support hair it is slow-witted when, then illustrate that children are likely to be at doubt state.
The emotional state for the children that server is obtained according to identification, judges whether children need to teach, and such as works as and identifies When child is more irritated in learning process or compares anxiety or is in doubt state, then it is assumed that children need study coach, that is, work as When emotional state is irritated, anxiety or feels uncertain, then judge that children need to teach, when the mood of children is more gentle, then it is assumed that youngster Child does not need to teach.If judging, children need to teach, and it is corresponding that server searches the emotional state in the preset database Guidance information, and the guidance information is sent to robot.Guidance information refers to that guidance children say the information for the problem encountered, Different emotional states can correspond to different guidance informations, and each emotional state can correspond to one or more different guidance letters Breath, when emotional state corresponds to multiple guidance informations in the preset database, can randomly choose a guidance information and be sent to machine Device people.For example, when identifying that children are in doubt state, guidance information can for " you meet difficulty? ", " be There is no to inscribe? ", " you think deeply what problem? " Deng.
After robot receives the guidance information of server transmission, the guidance information can be converted to natural language (i.e. people The language that can be identified), and exported by voice mode, actively to inquire whether children need study coach, prevent children from thinking When the comparison that problem is thought is irritated, forget to seek help to robot, can not only teach children's study, and can be relieved children's anxiety, Irritated mood.
Children can input current puzzlement to robot, will not solve after the guidance information for hearing robot voice output Topic, the problem of not understanding the problems such as information, after the problem of robot receives children's input information, which is sent To server, the solvable problem is judged whether by server, if solvable, corresponding answer information is searched for and is sent to machine Device people exports the answer information from robot to children, and robot can select different output sides according to the type of answer information Formula is such as shown by screen or voice mode is exported.
If server judgement can not answer the problem, which can be sent to the intelligence bound with robot eventually End, the intelligent terminal of binding can be the intelligent terminal of parent, be also possible to the intelligent terminal of teacher, to be returned by parent or teacher Corresponding answer is returned to server, then robot is sent to by server and shows children.
Wherein, server judges whether that the method for the solvable problem information specifically includes:
Described problem information is segmented, keyword is extracted;
It is scanned in preset exam pool according to the keyword;
When searching the topic with the Keywords matching in the exam pool, solvable described problem information is judged;
When not searching the topic with the Keywords matching in the exam pool, judgement can not answer described problem letter Breath.
Specifically, after server receives the problem of robot is sent information, which is handled, if problem Information is inputted with voice mode, then problem information is converted to text information, then carries out word segmentation processing to text information, Keyword is extracted, keyword can be one or more.
Exam pool has been stored in advance on server, after extracting keyword, has been existed according to the one or more keywords extracted It scans for matching in exam pool, if may search for, judges the solvable problem information, and the problem information can be directly acquired Solve a problem thinking and course of solving questions, the thinking that then will solve a problem and course of solving questions be sent to robot, and robot can be first to children's exhibition Show thinking of solving a problem, give the certain prompt of solving a problem of children, allow children receive solve a problem thinking after think independently again, avoid Child depends on robot unduly to solve a problem;After certain interval of time, server is believed according to the image that robot acquires again When breath judges children there is still a need for teaching, the information for exporting course of solving questions is sent to robot, robot again exports course of solving questions To children, to carry out study coach to children.
If search in exam pool less than topic with Keywords matching, judgement can not answer the problem information.It is preferred that Ground, when search is less than matched topic in exam pool, server also can return to prompt information to robot.Robot output mentions Show information, to prompt children to re-enter problem, or robot is allowed directly to shoot the picture for obtaining the problem.For example, prompt Information can be " I does not understand problem, directly me can be allowed to have a look at problem ".When problem is given robot and is seen by children, robot The picture that acquisition problem can be shot by the way that the camera of eye is arranged in, is then sent to server for picture.
After server receives the picture of problem, scan for matching in preset exam pool further according to the picture, with into One step is confirmed whether the solvable problem, if can search for, then it is assumed that solvable, if can not search, then it is assumed that can not answer. Server can store the picture and stem text of topic, for example, geometry class is inscribed when constructing exam pool simultaneously according to topic types Mesh can store the stem content and graphic pictures of topic simultaneously;Picture talk topic, can store simultaneously topic stem content and The picture of picture.By text and the matched mode of the dual search of picture, the accuracy rate of search can be improved, and then it is auxiliary to improve study Effect is led, and increases the intelligence of robot, improves the usage experience of user.
The present invention to judge whether children need study coach, and is being needed by the facial expression and movement of acquisition children It when wanting study coach, actively inquires whether children want help, prevents children when the comparison to think over a problem is irritated, forget to machine Device people seeks help, with realize when parent has no time, actively teach children's study, in addition, can also make children by with robot The current anxiety of children, irritated mood are alleviated in exchange.
The second embodiment provided according to the present invention, as shown in Fig. 2, a kind of study coach method, comprising:
S100 receives the image information for the children that robot is acquired and sent at interval of preset duration, wherein described image Information includes the facial expression information and action message of the children;
S200 identifies to obtain the emotional state of the children according to described image information;
S310 records the read-around ratio that the emotional state is default emotional state;
S320 judges that the children need to teach when the read-around ratio is greater than preset times;
S330 obtains the corresponding guidance information of the emotional state when the children need to teach, and by the guidance Information is sent to the robot;
S400 receives the problem of robot is sent information;
S500 judges whether solvable described problem information;
Described problem information if it is not, is then sent to the intelligent terminal bound with the robot by S600;
S700 makes the robot export the answer and believes if so, obtain and return to answer information to the robot Breath.
Specifically, server when judging whether children need to teach, can record mood according to the emotional state identified State is that the read-around ratio of default emotional state then judges that children need to teach when read-around ratio is greater than preset times.Example Such as, the preset times set when the emotional state that server continuously identifies three times is default emotional state, are then recognized as 2 times Study coach is needed for children.
Default emotional state can be that server is pre-set, is also possible to be arranged by parent, presets emotional state It can be one or more, when default emotional state is multiple, as long as emotional state is any one in default emotional state It is a, then it is recorded.When record, it need to record and the number that emotional state is default emotional state continuously occur.
The numerical value of preset times can acquire the interval duration of image information according to robot to be set, to avoid bothering The thinking processes of children.Assuming that parent wishes that robot is thought deeply after ten minutes in children, then gives children and teaches and help, and machine Once then preset times are 5 times within a length of 2 minutes when device people acquires the interval of image information, if robot acquisition image information Once then preset times are 2 times within a length of 5 minutes when interval.
Assuming that preset times are 5 times, if identifying that obtained emotional state is default emotional state to the 4th time for the first time, But the emotional state that the 5th identification obtains is not default emotional state, then the emotional state recorded is default emotional state Read-around ratio be 4 times, be less than preset times 5 times, then illustrate that children oneself solve study problem, do not need robot Guidance illustrate the also unresolved study problem of children oneself if read-around ratio is greater than default read-around ratio, need to teach.
This programme is the read-around ratio of default emotional state by recording emotional state, and is judged according to the read-around ratio Whether children, which need, is taught, and before giving children and teaching, is supplied to children and is reasonably thought independently the time, prevent from bothering children Thinking processes, and then improve children's study effect.
The 3rd embodiment provided according to the present invention, as shown in figure 3, a kind of study coach method, comprising:
S100 receives the image information for the children that robot is acquired and sent at interval of preset duration, wherein described image Information includes the facial expression information and action message of the children;
S210 is by described image information input into preset identification model;
If the S220 identification model can recognize described image information, emotional state is exported;
If the unidentified described image information out of the S230 identification model, calculates the characteristic value of described image information;
S240 finds matched emotional state according to the characteristic value in preset feature database;
S300 judges whether the children need to teach according to the emotional state, if so, obtaining the emotional state Corresponding guidance information, and the guidance information is sent to the robot;
S400 receives the problem of robot is sent information;
S500 judges whether solvable described problem information;
Described problem information if it is not, is then sent to the intelligent terminal bound with the robot by S600;
S700 makes the robot export the answer and believes if so, obtain and return to answer information to the robot Breath.
Specifically, preparatory trained identification model is stored on server, server is receiving robot transmission After image information, which is input in identification model and is identified, if identification model may recognize that the image information, Recognition result (emotional states of children) are then exported, if identification model does not identify the image information, calculate image information Then characteristic value finds matched emotional state according to characteristic value in preset feature database.
Identification model is the model algorithm of some open sources, image information (the i.e. every figure largely marked by input Corresponding status information is labelled on picture) it is obtained by training, the training process of identification model is the prior art, no longer superfluous herein It states.After being trained by the image information largely marked to identification model, the weight parameter in identification model be can determine, Therefore, image information can be identified by trained identification model.
But it since current model algorithm degree of intelligence is limited, does not identify out all images, therefore, works as identification When model can not identify the image information, then corresponding emotional state is matched in feature database using the matched method of characteristic value. In actual use, it is lower also there is robot acquired image clarity, it is lower according to the clarity at this time The calculated characteristic value of image is smaller, when being matched in the database with this feature value, it is understood that there may be can not be matched to mood Therefore the case where state, can not fully rely on characteristic value matching to obtain emotional state, need identification model and characteristic value Matched method is combined to obtain the emotional state of children, to improve discrimination, and then prevent the feelings that can not identified Condition.
Wherein, the creation method of feature database are as follows:
A large amount of different image informations are collected, described image information includes the facial expression information and movement letter of the children Breath;
Obtain the corresponding status information of described image information;
Extract the feature in each image information;
The characteristic value of the feature is calculated, the characteristic value is that the letter of presetting digit capacity and data are composed;
Establish the corresponding relationship between the characteristic value and the status information;
According to the corresponding pass between the characteristic value, the status information and the characteristic value and the status information System, creates the feature database.
Specifically, the image information of a large amount of different users is first collected, likewise, image information includes the facial expression of user Information and action message.Then the corresponding status information of each image information is obtained, status information can be by being manually entered.Pass through again Python language extracts the feature in image information, then forms the feature extracted with the letter and data of presetting digit capacity It is stored, the letter and data of the presetting digit capacity are the characteristic value of this feature.In the present solution, by feature with 192 letters It is stored in the database (feature database) of ES search engine with the form of data, and establishes the pass between characteristic value and status information System, further according to the relationship of foundation, status information is stored in features described above library, to complete the creation of feature database.
In practical identification process, when identification model is unidentified goes out image information, then image is believed by Yolo algorithm Breath is cut, and is extracted the target image in image information, i.e., is removed unwanted background image, and target figure is then calculated As corresponding characteristic value.It further according to this feature value, is searched in ES, to be matched to object feature value, then according to target signature It is worth corresponding status information, obtains the corresponding emotional state of image information.
In the present solution, by the feature for extracting each image, and stored in the form of characteristic value, and matched searching for It is scanned in the process also by the mode of characteristic value, search accuracy rate can be improved.
It should be understood that in the above embodiments, the size of each step number is not meant that the order of the execution order, it is each to walk Rapid execution sequence should determine that the implementation process without coping with the embodiment of the present invention constitutes any limit with function and internal logic It is fixed.
The fourth embodiment provided according to the present invention, as shown in figure 4, a kind of study coach device, comprising:
Signal dispatcher module 100, the image letter for the children for acquiring and sending at interval of preset duration for receiving robot Breath, wherein described image information includes the facial expression information and action message of the children;
Identification module 200 obtains the emotional state of the children for identifying according to described image information;
Judgment module 300, for judging whether the children need to teach according to the emotional state;
It obtains module 400 and obtains the corresponding guidance information of the emotional state if needing to teach for the children;
Signal dispatcher module 100, for the guidance information to be sent to the robot;
Signal dispatcher module 100, for receiving the problem of robot is sent information;
Judgment module 300, for judging whether solvable described problem information;
Signal dispatcher module 100, for when judgement can not answer described problem information, described problem information to be sent to The intelligent terminal bound with the robot;
Module 400 is obtained, for when judging solvable described problem information, obtaining the corresponding answer of described problem information Information;
Signal dispatcher module 100 makes described in the robot output for returning to the answer information to the robot Answer information.
Specifically, for student in learning process, robot can acquire the image information of a children at interval of certain time, The image information of acquisition should include the facial expression information and action message of children.The facial expression information and action message of children It can reflect out the current emotional state of children.
Camera and infrared detection module are provided in robot.Robot can sense children's by infrared detection module Orientation shoots camera face children to adjust the shooting angle of camera.Illustratively, children are on desk when learning, Robot can be placed on desk, robot by infrared detection module detect children be located at desk where after, Robot can be moved into the right opposite of children position, and the image information of children is then acquired by camera.
After robot collects the image information of children, which is sent to server, server is according to the figure As information identifies to obtain the emotional state of children, such as emotional state is normal or anxiety, agitation, doubt.As children have wrinkle When the expression of eyebrow, then when illustrating that children are more irritated or children's expression is more serious, illustrate that children are likely encountered problem, or Children have the movement of grappling fixture or support hair it is slow-witted when, then illustrate that children are likely to be at doubt state.
The emotional state for the children that server is obtained according to identification, judges whether children need to teach, and such as works as and identifies When child is more irritated in learning process or compares anxiety or is in doubt state, then it is assumed that children need study coach, that is, work as When emotional state is irritated, anxiety or feels uncertain, then judge that children need to teach, when the mood of children is more gentle, then it is assumed that youngster Child does not need to teach.If judging, children need to teach, and it is corresponding that server searches the emotional state in the preset database Guidance information, and the guidance information is sent to robot.Guidance information refers to that guidance children say the information for the problem encountered, Different emotional states can correspond to different guidance informations, and each emotional state can correspond to one or more different guidance letters Breath, when emotional state corresponds to multiple guidance informations in the preset database, can randomly choose a guidance information and be sent to machine Device people.For example, when identifying that children are in doubt state, guidance information can for " you meet difficulty? ", " be There is no to inscribe? ", " you think deeply what problem? " Deng.
After robot receives the guidance information of server transmission, the guidance information can be converted to natural language (i.e. people The language that can be identified), and exported by voice mode, actively to inquire whether children need study coach, prevent children from thinking When the comparison that problem is thought is irritated, forget to seek help to robot, can not only teach children's study, and can be relieved children's anxiety, Irritated mood.
Children can input current puzzlement to robot, will not solve after the guidance information for hearing robot voice output Topic, the problem of not understanding the problems such as information, after the problem of robot receives children's input information, which is sent To server, the solvable problem is judged whether by server, if solvable, corresponding answer information is searched for and is sent to machine Device people exports the answer information from robot to children, and robot can select different output sides according to the type of answer information Formula is such as shown by screen or voice mode is exported.
If server judgement can not answer the problem, which can be sent to the intelligence bound with robot eventually End, the intelligent terminal of binding can be the intelligent terminal of parent, be also possible to the intelligent terminal of teacher, to be returned by parent or teacher Corresponding answer is returned to server, then robot is sent to by server and shows children.
The present invention to judge whether children need study coach, and is being needed by the facial expression and movement of acquisition children It when wanting study coach, actively inquires whether children want help, prevents children when the comparison to think over a problem is irritated, forget to machine Device people seeks help, with realize when parent has no time, actively teach children's study, in addition, can also make children by with robot The current anxiety of children, irritated mood are alleviated in exchange.
Preferably, judgment module 300 includes:
Recording unit 310, for recording the read-around ratio that the emotional state is default emotional state;
Judging unit 320, for judging that the children need to teach when the read-around ratio is greater than preset times.
Specifically, server when judging whether children need to teach, can record mood according to the emotional state identified State is that the read-around ratio of default emotional state then judges that children need to teach when read-around ratio is greater than preset times.Example Such as, the preset times set when the emotional state that server continuously identifies three times is default emotional state, are then recognized as 2 times Study coach is needed for children.
Default emotional state can be that server is pre-set, is also possible to be arranged by parent, presets emotional state It can be one or more, when default emotional state is multiple, as long as emotional state is any one in default emotional state It is a, then it is recorded.When record, it need to record and the number that emotional state is default emotional state continuously occur.
The numerical value of preset times can acquire the interval duration of image information according to robot to be set, to avoid bothering The thinking processes of children.Assuming that parent wishes that robot is thought deeply after ten minutes in children, then gives children and teaches and help, and machine Once then preset times are 5 times within a length of 2 minutes when device people acquires the interval of image information, if robot acquisition image information Once then preset times are 2 times within a length of 5 minutes when interval.
Assuming that preset times are 5 times, if identifying that obtained emotional state is default emotional state to the 4th time for the first time, But the emotional state that the 5th identification obtains is not default emotional state, then the emotional state recorded is default emotional state Read-around ratio be 4 times, be less than preset times 5 times, then illustrate that children oneself solve study problem, do not need robot Guidance illustrate the also unresolved study problem of children oneself if read-around ratio is greater than default read-around ratio, need to teach.
This programme is the read-around ratio of default emotional state by recording emotional state, and is judged according to the read-around ratio Whether children, which need, is taught, and before giving children and teaching, is supplied to children and is reasonably thought independently the time, prevent from bothering children Thinking processes, and then improve children's study effect.
Preferably, judgment module 300 includes:
Participle unit 330 extracts keyword for segmenting described problem information;
Search unit 340, for being scanned in preset exam pool according to the keyword;
Judging unit 320, for when searching the topic with the Keywords matching in the exam pool, judgement can be solved Answer described problem information;
Judging unit 320, for when not searching the topic with the Keywords matching in the exam pool, judgement to be not Solvable described problem information.
Specifically, after server receives the problem of robot is sent information, which is handled, if problem Information is inputted with voice mode, then problem information is converted to text information, then carries out word segmentation processing to text information, Keyword is extracted, keyword can be one or more.
Exam pool has been stored in advance on server, after extracting keyword, has been existed according to the one or more keywords extracted It scans for matching in exam pool, if may search for, judges the solvable problem information, and the problem information can be directly acquired Solve a problem thinking and course of solving questions, the thinking that then will solve a problem and course of solving questions be sent to robot, and robot can be first to children's exhibition Show thinking of solving a problem, give the certain prompt of solving a problem of children, allow children receive solve a problem thinking after think independently again, avoid Child depends on robot unduly to solve a problem;After certain interval of time, server is believed according to the image that robot acquires again When breath judges children there is still a need for teaching, the information for exporting course of solving questions is sent to robot, robot again exports course of solving questions To children, to carry out study coach to children.
If search in exam pool less than topic with Keywords matching, judgement can not answer the problem information.It is preferred that Ground, when search is less than matched topic in exam pool, server also can return to prompt information to robot.Robot output mentions Show information, to prompt children to re-enter problem, or robot is allowed directly to shoot the picture for obtaining the problem;For example, prompt Information can be " I does not understand problem, directly me can be allowed to have a look at problem ".When problem is given robot and is seen by children, robot The picture that acquisition problem can be shot by the way that the camera of eye is arranged in, is then sent to server for picture.
After server receives the picture of problem, scan for matching in preset exam pool further according to the picture, with into One step is confirmed whether the solvable problem, if can search for, then it is assumed that solvable, if can not search, then it is assumed that can not answer. Server can store the picture and stem text of topic, for example, geometry class is inscribed when constructing exam pool simultaneously according to topic types Mesh can store the stem content and graphic pictures of topic simultaneously;Picture talk topic, can store simultaneously topic stem content and The picture of picture.By text and the matched mode of the dual search of picture, the accuracy rate of search can be improved, and then it is auxiliary to improve study Effect is led, and increases the intelligence of robot, improves the usage experience of user.
Preferably, identification module 200 includes:
Model recognition unit 210 is used for described image information input into preset identification model;If the identification mould Type can recognize described image information, then exports emotional state;
Characteristic value matching unit 220, if calculating the figure for the unidentified described image information out of the identification model As the characteristic value of information;And matched emotional state is found in preset feature database according to the characteristic value.
Specifically, preparatory trained identification model is stored on server, server is receiving robot transmission After image information, which is input in identification model and is identified, if identification model may recognize that the image information, Recognition result (emotional states of children) are then exported, if identification model does not identify the image information, calculate image information Then characteristic value finds matched emotional state according to characteristic value in preset feature database.
Identification model is the model algorithm of some open sources, image information (the i.e. every figure largely marked by input Corresponding status information is labelled on picture) it is obtained by training, the training process of identification model is the prior art, no longer superfluous herein It states.After being trained by the image information largely marked to identification model, the weight parameter in identification model be can determine, Therefore, image information can be identified by trained identification model.
But it since current model algorithm degree of intelligence is limited, does not identify out all images, therefore, works as identification When model can not identify the image information, then corresponding emotional state is matched in feature database using the matched method of characteristic value. In actual use, it is lower also there is robot acquired image clarity, it is lower according to the clarity at this time The calculated characteristic value of image is smaller, when being matched in the database with this feature value, it is understood that there may be can not be matched to mood Therefore the case where state, can not fully rely on characteristic value matching to obtain emotional state, need identification model and characteristic value Matched method is combined to obtain the emotional state of children, to improve discrimination, and then prevent the feelings that can not identified Condition.
It preferably, further include creation module 500, creation module 500 includes:
Collector unit 510, for collecting a large amount of different image informations, described image information includes the face of the children Expression information and action message;
Acquiring unit 520, for obtaining the corresponding status information of described image information;
Extraction unit 530, for extracting the feature in each image information;
Computing unit 540, for calculating the characteristic value of the feature, the characteristic value is the letter and data of presetting digit capacity It is composed;
Unit 550 is established, the corresponding relationship for establishing between the characteristic value and the status information;
Creating unit 560, for being believed according to the characteristic value, the status information and the characteristic value and the state Corresponding relationship between breath creates the feature database.
Specifically, the image information of a large amount of different users is first collected, likewise, image information includes the facial expression of user Information and action message.Then the corresponding status information of each image information is obtained, status information can be by being manually entered.Pass through again Python language extracts the feature in image information, then forms the feature extracted with the letter and data of presetting digit capacity It is stored, the letter and data of the presetting digit capacity are the characteristic value of this feature.In the present solution, by feature with 192 letters It is stored in the database (feature database) of ES search engine with the form of data, and establishes the pass between characteristic value and status information System, further according to the relationship of foundation, status information is stored in features described above library, to complete the creation of feature database.
In practical identification process, when identification model is unidentified goes out image information, then image is believed by Yolo algorithm Breath is cut, and is extracted the target image in image information, i.e., is removed unwanted background image, and target figure is then calculated As corresponding characteristic value.It further according to this feature value, is searched in ES, to be matched to object feature value, then according to target signature It is worth corresponding status information, obtains the corresponding emotional state of image information.
In the present solution, by the feature for extracting each image, and stored in the form of characteristic value, and matched searching for It is scanned in the process also by the mode of characteristic value, search accuracy rate can be improved.
It should be noted that above-described embodiment can be freely combined as needed.The above is only of the invention preferred Embodiment, it is noted that for those skilled in the art, in the premise for not departing from the principle of the invention Under, several improvements and modifications can also be made, these modifications and embellishments should also be considered as the scope of protection of the present invention.

Claims (10)

1. a kind of study coach method characterized by comprising
Receive the image information of children that robot is acquired and sent at interval of preset duration, wherein described image information includes The facial expression information and action message of the children;
It is identified to obtain the emotional state of the children according to described image information;
According to the emotional state, judge whether the children need to teach, the emotional state is corresponding to draw if so, obtaining Information is led, and the guidance information is sent to the robot;
Receive the problem of robot is sent information;
Judge whether solvable described problem information, if it is not, then by described problem information be sent to the robot binding Intelligent terminal makes the robot export the answer information if so, obtaining and returning to answer information to the robot.
2. a kind of study coach method according to claim 1, which is characterized in that it is described according to the emotional state, sentence Whether the children of breaking, which need to teach, specifically includes:
Record the read-around ratio that the emotional state is default emotional state;
When the read-around ratio is greater than preset times, judge that the children need to teach.
3. a kind of study coach method according to claim 1 or 2, which is characterized in that described to judge whether solvable institute Problem information is stated to specifically include:
Described problem information is segmented, keyword is extracted;
It is scanned in preset exam pool according to the keyword;
When searching the topic with the Keywords matching in the exam pool, solvable described problem information is judged;
When not searching the topic with the Keywords matching in the exam pool, judgement can not answer described problem information.
4. a kind of study coach method according to claim 1 or 2, which is characterized in that described according to described image information The emotional state that identification obtains the children specifically includes:
By described image information input into preset identification model;
If the identification model can recognize described image information, emotional state is exported;
If the unidentified described image information out of the identification model, calculates the characteristic value of described image information;
Matched emotional state is found in preset feature database according to the characteristic value.
5. a kind of study coach method according to claim 4, which is characterized in that the creation method of the feature database are as follows:
A large amount of different image informations are collected, described image information includes the facial expression information and action message of the children;
Obtain the corresponding status information of described image information;
Extract the feature in each image information;
The characteristic value of the feature is calculated, the characteristic value is that the letter of presetting digit capacity and data are composed;
Establish the corresponding relationship between the characteristic value and the status information;
According to the corresponding relationship between the characteristic value, the status information and the characteristic value and the status information, wound Build the feature database.
6. a kind of study coach device characterized by comprising
Signal dispatcher module, the image information for the children for acquiring and sending at interval of preset duration for receiving robot, wherein Described image information includes the facial expression information and action message of the children;
Identification module obtains the emotional state of the children for identifying according to described image information;
Judgment module, for judging whether the children need to teach according to the emotional state;
It obtains module and obtains the corresponding guidance information of the emotional state if needing to teach for the children;
The signal dispatcher module, for the guidance information to be sent to the robot;
The signal dispatcher module, for receiving the problem of robot is sent information;
The judgment module, for judging whether solvable described problem information;
The signal dispatcher module, for when judgement can not answer described problem information when, by described problem information be sent to The intelligent terminal of the robot binding;
The acquisition module, for when judging solvable described problem information, obtaining the corresponding answer letter of described problem information Breath;
The signal dispatcher module makes the robot export the solution for returning to the answer information to the robot Answer information.
7. a kind of study coach device according to claim 6, which is characterized in that the judgment module includes:
Recording unit, for recording the read-around ratio that the emotional state is default emotional state;
Judging unit, for judging that the children need to teach when the read-around ratio is greater than preset times.
8. a kind of study coach device according to claim 6 or 7, which is characterized in that the judgment module includes:
Participle unit extracts keyword for segmenting described problem information;
Search unit, for being scanned in preset exam pool according to the keyword;
Judging unit, for judging described in solvable when searching the topic with the Keywords matching in the exam pool Problem information;
The judging unit, for when not searching the topic with the Keywords matching in the exam pool, judgement can not Answer described problem information.
9. a kind of study coach device according to claim 6 or 7, which is characterized in that the identification module includes:
Model recognition unit is used for described image information input into preset identification model;If the identification model can be known Other described image information, then export emotional state;
Characteristic value matching unit, if calculating described image information for the unidentified described image information out of the identification model Characteristic value;And matched emotional state is found in preset feature database according to the characteristic value.
10. a kind of study coach device according to claim 9, which is characterized in that it further include creation module, the creation Module includes:
Collector unit, for collecting a large amount of different image informations, described image information includes that the facial expression of the children is believed Breath and action message;
Acquiring unit, for obtaining the corresponding status information of described image information;
Extraction unit, for extracting the feature in each image information;
Computing unit, for calculating the characteristic value of the feature, the characteristic value be presetting digit capacity letter and data combination and At;
Unit is established, the corresponding relationship for establishing between the characteristic value and the status information;
Creating unit, for according between the characteristic value, the status information and the characteristic value and the status information Corresponding relationship, create the feature database.
CN201910668360.1A 2019-07-23 2019-07-23 A kind of study coach method and device Pending CN110379234A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910668360.1A CN110379234A (en) 2019-07-23 2019-07-23 A kind of study coach method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910668360.1A CN110379234A (en) 2019-07-23 2019-07-23 A kind of study coach method and device

Publications (1)

Publication Number Publication Date
CN110379234A true CN110379234A (en) 2019-10-25

Family

ID=68255265

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910668360.1A Pending CN110379234A (en) 2019-07-23 2019-07-23 A kind of study coach method and device

Country Status (1)

Country Link
CN (1) CN110379234A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110853430A (en) * 2019-11-20 2020-02-28 深圳创维-Rgb电子有限公司 Learning tutoring method and device based on smart home and storage medium
CN111967328A (en) * 2020-07-16 2020-11-20 深圳市沃特沃德股份有限公司 Method for tutoring operation, intelligent drawing robot and server
CN113084794A (en) * 2020-01-08 2021-07-09 重庆第二师范学院 Robot for thinking training of children
CN113377982A (en) * 2021-06-22 2021-09-10 读书郎教育科技有限公司 System and method for instructing students to tell picture story

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103902046A (en) * 2014-04-10 2014-07-02 深圳市中兴移动通信有限公司 Intelligent prompting method and terminal
CN106650718A (en) * 2016-12-21 2017-05-10 远光软件股份有限公司 Certificate image identification method and apparatus
CN107861684A (en) * 2017-11-23 2018-03-30 广州视睿电子科技有限公司 Write recognition methods, device, storage medium and computer equipment
CN108334583A (en) * 2018-01-26 2018-07-27 上海智臻智能网络科技股份有限公司 Affective interaction method and device, computer readable storage medium, computer equipment
CN108434757A (en) * 2018-05-25 2018-08-24 深圳市零度智控科技有限公司 intelligent toy control method, intelligent toy and computer readable storage medium
CN108733209A (en) * 2018-03-21 2018-11-02 北京猎户星空科技有限公司 Man-machine interaction method, device, robot and storage medium
CN108877357A (en) * 2018-06-21 2018-11-23 广东小天才科技有限公司 A kind of exchange method and private tutor's machine based on private tutor's machine
CN109346068A (en) * 2018-08-16 2019-02-15 北京云迹科技有限公司 Robot voice exchange method and device
CN109446968A (en) * 2018-10-22 2019-03-08 广东小天才科技有限公司 A kind of method and system based on mood regularized learning algorithm situation
CN109522059A (en) * 2018-11-28 2019-03-26 广东小天才科技有限公司 A kind of program invocation method and system
CN109817203A (en) * 2019-02-19 2019-05-28 广东小天才科技有限公司 A kind of method and system of interactive voice
CN109844735A (en) * 2016-07-21 2019-06-04 奇跃公司 Affective state for using user controls the technology that virtual image generates system
CN109871807A (en) * 2019-02-21 2019-06-11 百度在线网络技术(北京)有限公司 Face image processing process and device
CN110009941A (en) * 2019-04-12 2019-07-12 广东小天才科技有限公司 A kind of method and system of robot assisted study

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103902046A (en) * 2014-04-10 2014-07-02 深圳市中兴移动通信有限公司 Intelligent prompting method and terminal
CN109844735A (en) * 2016-07-21 2019-06-04 奇跃公司 Affective state for using user controls the technology that virtual image generates system
CN106650718A (en) * 2016-12-21 2017-05-10 远光软件股份有限公司 Certificate image identification method and apparatus
CN107861684A (en) * 2017-11-23 2018-03-30 广州视睿电子科技有限公司 Write recognition methods, device, storage medium and computer equipment
CN108334583A (en) * 2018-01-26 2018-07-27 上海智臻智能网络科技股份有限公司 Affective interaction method and device, computer readable storage medium, computer equipment
CN108733209A (en) * 2018-03-21 2018-11-02 北京猎户星空科技有限公司 Man-machine interaction method, device, robot and storage medium
CN108434757A (en) * 2018-05-25 2018-08-24 深圳市零度智控科技有限公司 intelligent toy control method, intelligent toy and computer readable storage medium
CN108877357A (en) * 2018-06-21 2018-11-23 广东小天才科技有限公司 A kind of exchange method and private tutor's machine based on private tutor's machine
CN109346068A (en) * 2018-08-16 2019-02-15 北京云迹科技有限公司 Robot voice exchange method and device
CN109446968A (en) * 2018-10-22 2019-03-08 广东小天才科技有限公司 A kind of method and system based on mood regularized learning algorithm situation
CN109522059A (en) * 2018-11-28 2019-03-26 广东小天才科技有限公司 A kind of program invocation method and system
CN109817203A (en) * 2019-02-19 2019-05-28 广东小天才科技有限公司 A kind of method and system of interactive voice
CN109871807A (en) * 2019-02-21 2019-06-11 百度在线网络技术(北京)有限公司 Face image processing process and device
CN110009941A (en) * 2019-04-12 2019-07-12 广东小天才科技有限公司 A kind of method and system of robot assisted study

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110853430A (en) * 2019-11-20 2020-02-28 深圳创维-Rgb电子有限公司 Learning tutoring method and device based on smart home and storage medium
CN110853430B (en) * 2019-11-20 2022-02-15 深圳创维-Rgb电子有限公司 Learning tutoring method and device based on smart home and storage medium
CN113084794A (en) * 2020-01-08 2021-07-09 重庆第二师范学院 Robot for thinking training of children
CN111967328A (en) * 2020-07-16 2020-11-20 深圳市沃特沃德股份有限公司 Method for tutoring operation, intelligent drawing robot and server
CN113377982A (en) * 2021-06-22 2021-09-10 读书郎教育科技有限公司 System and method for instructing students to tell picture story

Similar Documents

Publication Publication Date Title
CN110379234A (en) A kind of study coach method and device
CN109176535B (en) Interaction method and system based on intelligent robot
CN109940627B (en) Man-machine interaction method and system for picture book reading robot
Wibberley Getting to grips with bricolage: A personal account.
MacDonald Finding a critical perspective in grounded theory
CN110059614A (en) A kind of intelligent assistant teaching method and system based on face Emotion identification
CN106097793B (en) Intelligent robot-oriented children teaching method and device
CN105894873A (en) Child teaching method and device orienting to intelligent robot
CN109710748B (en) Intelligent robot-oriented picture book reading interaction method and system
WO2021077382A1 (en) Method and apparatus for determining learning state, and intelligent robot
CN109885595A (en) Course recommended method, device, equipment and storage medium based on artificial intelligence
CN109637207A (en) A kind of preschool education interactive teaching device and teaching method
CN111475656B (en) Visual question-answering method and system based on external knowledge aggregation
WO2021047185A1 (en) Monitoring method and apparatus based on facial recognition, and storage medium and computer device
CN111027486A (en) Auxiliary analysis and evaluation system and method for big data of teaching effect of primary and secondary school classroom
CN107016046A (en) The intelligent robot dialogue method and system of view-based access control model displaying
CN113506624A (en) Autism child cognitive ability assessment intervention system based on layer-by-layer generalization push logic
CN110245253B (en) Semantic interaction method and system based on environmental information
CN117055724B (en) Working method of generating teaching resource system in virtual teaching scene
CN109754653B (en) Method and system for personalized teaching
Boncompagni Wittgenstein on Forms of Life
Ramík et al. From visual patterns to semantic description: A cognitive approach using artificial curiosity as the foundation
CN110728604B (en) Analysis method and device
CN111695496A (en) Intelligent interactive learning method, learning programming method and robot
Wang et al. RUC at MediaEval 2019: Video Memorability Prediction Based on Visual Textual and Concept Related Features.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20191025

RJ01 Rejection of invention patent application after publication