CN109508687A - Man-machine interaction control method, device, storage medium and smart machine - Google Patents
Man-machine interaction control method, device, storage medium and smart machine Download PDFInfo
- Publication number
- CN109508687A CN109508687A CN201811418691.1A CN201811418691A CN109508687A CN 109508687 A CN109508687 A CN 109508687A CN 201811418691 A CN201811418691 A CN 201811418691A CN 109508687 A CN109508687 A CN 109508687A
- Authority
- CN
- China
- Prior art keywords
- interactive object
- angle
- facial angle
- face
- voice messaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/4401—Bootstrapping
- G06F9/4418—Suspend and resume; Hibernate and awake
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Manipulator (AREA)
Abstract
This application discloses man-machine interaction control method, device, storage medium and smart machines, are related to human-computer interaction technique field.Interaction habits in this method based on user and robot collect the voice messaging of interactive object in the awake state；According to the collected image comprising the interactive object, the facial angle of the interactive object is determined；If the facial angle is greater than predetermined angle, determination does not need to respond the voice messaging.In this way, realizing that the judgement for whether having interaction to be intended to collected voice does not respond collected voice when being judged as not interactive intention according to facial angle, the accuracy rate of the speech recognition of robot can be improved, reduce to interference caused by user security risk.
Description
Technical field
This application involves human-computer interaction technique field more particularly to man-machine interaction control method, device, storage medium and intelligence
It can equipment.
Background technique
Interactive voice has the characteristics that natural interaction, to using crowd not specially require, i.e. child old man or illiterate
As long as people can normally speak and use the function.So many robots use the function of interactive voice to come for user
Respective services are provided.
Currently, robot in interactive process, has feedback as long as receiving voice, can not differentiate the voice is to be directed to
Robot or other people.For example, robot in interactive object interactive process, if there is pair of other people and interactive object
Words, robot can also respond after identifying the voice of the dialogue for dialogue.It can be seen that robot can misidentify voice simultaneously
It makes a response, user is interfered.
Summary of the invention
The embodiment of the present application provides a kind of man-machine interaction control method, device, storage medium and smart machine, for solving
The more low problem of the exposure rate and utilization rate of robot in the prior art.
In a first aspect, the embodiment of the present application provides a kind of man-machine interaction control method, this method comprises:
In the awake state, the voice messaging of interactive object is collected；
According to the collected image comprising the interactive object, the facial angle of the interactive object is determined；
If the facial angle is greater than predetermined angle, determination does not need to respond the voice messaging.
Second aspect, the application also provide a kind of man-machine interaction control device, and described device includes:
Voice messaging acquisition module, in the awake state, collecting the voice messaging of interactive object；
Facial angle determining module, for determining the interaction according to the collected image comprising the interactive object
The facial angle of object；
Control module determines if being greater than predetermined angle for the facial angle and does not need to carry out the voice messaging
Response.
The third aspect, the application also provide a kind of smart machine, including at least one processor；And with described at least one
The memory of a processor communication connection；Wherein, the memory is stored with the finger that can be executed by least one described processor
It enables, described instruction is executed by least one described processor, so that at least one described processor is able to carry out the application implementation
Any man-machine interaction control method that example provides.
Fourth aspect, present invention also provides a kind of computer storage mediums, wherein the computer storage medium storage
There are computer executable instructions, the computer executable instructions are for making computer execute any people in the embodiment of the present application
Machine interaction control method.
Man-machine interaction control method, device and storage medium provided by the present application.User and robot are based in the application
Interaction habits collect the voice messaging of interactive object in the awake state；According to collected comprising the interactive object
Image, determine the facial angle of the interactive object；If the facial angle is greater than predetermined angle, determination is not needed to described
Voice messaging is responded.In this way, the judgement for whether having interaction to be intended to collected voice realized according to facial angle, sentencing
Break for it is no interaction be intended to when, collected voice is not responded, can be improved the accuracy rate of the speech recognition of robot,
It reduces to interference caused by user security risk.
Other features and advantage will illustrate in the following description, also, partly become from specification
It obtains it is clear that being understood and implementing the application.The purpose of the application and other advantages can be by written explanations
Specifically noted structure is achieved and obtained in book, claims and attached drawing.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please are not constituted an undue limitation on the present application for explaining the application.In the accompanying drawings:
Fig. 1 is the application scenarios schematic diagram of the human-computer interactive control in the embodiment of the present application；
Fig. 2 is one of the flow diagram of man-machine interaction control method in the embodiment of the present application；
Fig. 3 be the embodiment of the present application in man-machine interaction control method in extract characteristic point one of schematic diagram；
Fig. 4 is the two of the schematic diagram of the extraction characteristic point of the man-machine interaction control method in the embodiment of the present application；
Fig. 5 is the man-machine interaction control device schematic diagram in the embodiment of the present application；
Fig. 6 is the structural schematic diagram according to the smart machine of the application embodiment.
Specific embodiment
In order to keep human-computer interaction more accurate, the interference to user is reduced, provides a kind of man-machine friendship in the embodiment of the present application
Mutually control method, apparatus and storage medium.In order to better understand technical solution provided by the embodiments of the present application, here to the party
The some nouns and basic principle that case uses, which are done, to be briefly described:
Noun explanation:
Wake up: when user executes specific feature operation using robot, robot is in wake-up states, machine under the state
The interaction of device people and interactive object is executed according to specific business.
Facial angle: when human-computer interaction, plane opposed robots where the face of user's (i.e. interactive object of robot)
The angle of plane where facial.
Principle explanation:
During user uses robot, it is generally proximal to robot and issues voice in face of robot.Also lead to when interaction
It is often in face of robot interactive.Human-computer interaction based on people is accustomed to, and in the embodiment of the present application under robot wake-up states, adopts
Collect image, and carries out image analysis to determine the facial angle of interactive object, if facial angle is larger, then it represents that interactive object
Voice not is issued towards robot namely simultaneously there is interaction to be intended to for non-sum robot, so robot can not be to the language of user
Sound is fed back.
Further, right with reference to the accompanying drawing for the ease of further understanding technical solution provided by the embodiments of the present application
This is illustrated.
As shown in Figure 1, being the application scenarios schematic diagram of man-machine interaction control method provided by the embodiments of the present application, the application
It include user 10, robot 11 in scene, robot 11 can collect the voice messaging of interactive object in wake-up states；It adopts
Collect the ambient image in front of robot, Face datection is carried out to acquired image；To the face detected, facial characteristics is extracted
Point, with the facial angle of the determination interactive object；If the facial angle is greater than predetermined angle, determination is not needed to institute's predicate
Message breath is responded.If not needing to respond voice messaging, it is not necessary that collected voice messaging is uploaded to service
Device is parsed.
Correspondingly, if facial angle is less than or equal to predetermined angle, then it represents that the voice that interactive object issues is and machine
The voice that people has interaction to be intended to, then respond the voice, i.e., collected voice messaging be uploaded to server and solved
Analysis, and the feedback based on server, respond the voice messaging.
It should be noted that the robot that the robot in the embodiment of the present application can be display screen is also possible to not have
There is the robot of display screen, the function of providing can be the various functions of all trades and professions.
Man-machine interaction control method provided by the embodiments of the present application is described further below, as shown in Fig. 2, this method
The following steps are included:
Step 201: in the awake state, collecting the voice messaging of interactive object；
Step 202: according to the collected image comprising the interactive object, determining the face angle of the interactive object
Degree；
When it is implemented, may have more people in ambient enviroment, then by any personal accomplishment interactive object just it needs to be determined that side
Case.Due to usual interactive object be all from the closer people of robot, so, if comprising more in acquired image in the application
A target object determines each target object at a distance from smart machine according to acquired image；It will be apart from nearest target pair
As being determined as the interactive object, and determine the facial angle of the interactive object.
When it is implemented, Image Acquisition can trigger after collecting voice messaging；Surrounding image can also be acquired in real time.
Step 203: if the facial angle is greater than predetermined angle, determination does not need to respond the voice messaging.
It may include the voice for not being directed to interactive object when it is implemented, not needing to respond the voice messaging
It conversates, meanwhile, if there is screen in robot, any feedback information is not also shown on the screen.
Correspondingly, determining needs believe the voice if it is determined that the facial angle is less than or equal to the predetermined angle
Breath is responded.
When it is implemented, predetermined angle can be according to actual scene and needing to set.It under normal circumstances, can be by predetermined angle
It is set as 45 degree.
In this way, when it is implemented, can determine whether voice messaging is to hand in the embodiment of the present application by facial angle
The voice messaging that mutual object has interaction to be intended to, and then to determine whether to respond the voice messaging.Still with the example in background technique
For son, explanation is compared to the effect of the embodiment of the present application:
Robot in interactive object interactive process, if there is other people and interactive object are talked with, at this time interactive object
Head usually can all be turned to other people and go to talk in face of other people, so that the facial angle of interactive object be caused to change.At this moment, though
Right robot collects voice messaging, but robot is able to confirm that the facial angle of interactive object is larger, and then determines interaction
The voice messaging of object is issued not directed to robot, so not responding to collected voice messaging.In this way, machine
Although device people identifies voice, the voice of each identification can't can be made a response, but interactive object is not handed over
The voice being mutually intended to is without response, so as to avoid interfering user, improving the accuracy of human-computer interaction.
Certainly, in the specific implementation, however it is not limited to the scene of above-mentioned human-computer interaction.When it is implemented, and user
Before interaction, robot goes to judge the facial angle of interactive object as long as collecting voice, not when facial angle is larger
Voice can be responded.
Further, in one embodiment, when people's natural interaction, it will usually face other side.When other side is mobile, hand over
Sight can also be moved and be spoken in face of other side when mutually.Such natural interactive style, which can allow, to be felt each other and is respected and relaxed
It is suitable.In the embodiment of the present application, when human-computer interaction, can intelligence tracking sound source, approach interacting for natural way to realize.
It is specific implementable are as follows: according to the corresponding Sounnd source direction of the voice messaging, to adjust smart machine towards the Sounnd source direction；Acquisition
The image of the smart machine ambient enviroment.In this way, people can not only be allowed by interacting towards Sounnd source direction by tracking sound source
Machine interaction is subsequent to search interactive object according to the image more naturally, the image of audio direction can also be got in time,
Convenient for whether interacting intention with the voice of determining interactive object.
When it is implemented, if towards still including multiple target objects in acquired image after Sounnd source direction, according to adopting
The image collected determines each target object at a distance from smart machine；It will be determined as the interaction apart from nearest target object
Object, and determine the facial angle of the interactive object.
For example, A is on the left of robot, and B is on the right side of robot when existing simultaneously multiple interactive object A and B.A issues language
When sound, towards the Sounnd source direction of A, the image of A can be collected.And then whether the voice for determining that A is issued has interactive intention, so as to
Determine whether to respond the voice of A in subsequent.Similarly, when B issues voice, towards the Sounnd source direction of B, in this direction
Be conducive to collect the image of B, then judge whether B has interactive intention.
Further, the face feature point of interactive object can be extracted；Then according to the face feature point of extraction, institute is determined
State the facial angle of interactive object.It is specific implementable for one of following scheme:
Scheme one:
In the embodiment of the present application in order to guarantee to determine facial orientation and robot direction angle accuracy, can use
The method of deep learning is completed.When it is implemented, aforementioned face feature point includes for describing face profile as shown in Figure 3
Characteristic point, the face may include eye, eyebrow, mouth, nose, face.Training sample can be first acquired, including various angles in the sample
Facial image extracts characteristic point by human face characteristic point extractive technique.Such as, model can be extracted by training characteristics point, including
But be not limited to convolutional neural networks etc..Then model is extracted by training sample training characteristics point to extract face feature point.It
Afterwards, it by the characteristic point for being used to describe face profile of extraction, inputs to the pre- trained angle of deep learning method that first passes through and divides
Model is analysed, the facial angle of the interactive object is obtained.Wherein angle analysis model is also to be trained according to great amount of samples
Come, the training stage can require the accuracy rate of identification, so can be obtained using the accuracy of this method identification facial angle
To guarantee.
For example, angle analysis model provides angle of the face relative to tri- directions robot yaw, pitch, roll.Its
In, the face that yaw is equivalent to interactive object is equivalent to the face of interactive object relative to angle (i.e. pitch angle), the pitch of X-axis
Angle (i.e. head is around the angle that vertical axis rotates), roll relative to Y-axis are equivalent to folder of the face of interactive object relative to Z axis
Angle (i.e. the angle of head deviation shoulder).When it is implemented, can be mainly using the angle relative to the direction pitch.
To sum up, the model established by deep learning method, its accuracy for analyzing result can after the model training is good
To reach 99%, the accuracy for the facial angle for going analysis to obtain with the model of high accuracy is therewith with regard to high, then can be quasi-
True determines when to respond voice, when does not respond to voice.
Scheme two:
It, can also be using simple side other than the above-mentioned model analysis facial angle trained using deep learning method
Formula determines.Specifically, face feature point may include nose profile point and face mask point: then with nose profile point be boundary
Point obtains the face area of nose two sides according to face mask point；The ratio for calculating two sides face area, according to preset ratio
With the corresponding relationship of angle, facial angle of the corresponding angle of ratio calculated as interactive object is determined.For example, such as Fig. 4 institute
Show, extracts naris position and nasal bone position feature point and face mask characteristic point.The line orientations constituted along nasal features point
Face is divided into left side of the face and right side face, then calculates the area of left and right sides face.When due to face not against robot, then
The area of face side is bigger, and other side area is smaller, it is possible to which ratio determines facial angle according to area.
Further, in the embodiment of the present application, in order to further improve the accuracy of human-computer interaction, the embodiment of the present application
In, before determining the facial angle of interactive object, it is also necessary to judge whether image is clear image, if clear image and people
Face angle is greater than predetermined angle, and just determination does not respond collected voice messaging.Below to determining whether clearly to scheme
The effect of picture is illustrated:
When whether execute the facial image determined in acquired image before confirming facial angle is clear image,
Can based on determine acquired image whether be clear image as a result, come determine the need for further to the image into
Row analysis is to determine facial angle.If blurred picture, then it represents that user is likely to be in the state of fast moving, with robot into
The intention of row interaction is weaker, does not need the determination for carrying out facial angle in this case, to save due to determining facial angle consumption
The process resource taken.It is specific:
An image can be therefrom selected with multiple images of continuous acquisition after collecting voice messaging, determined whether it is clear
Clear image, an image is then reselected if blurred picture to determine whether for clear image, until from the more of continuous acquisition
It opens until finding clear image in image or determining that multiple images of continuous acquisition are blurred picture.If continuous acquisition arrives
Multiple images be blurred picture, then collected voice messaging is not responded；
If finding clear image, the subsequent determination for carrying out facial angle, when determining that facial angle is greater than predetermined angle,
Determination does not respond collected voice messaging；Otherwise when determining that facial angle is less than or equal to predetermined angle and is, to adopting
The voice messaging collected is responded.
Whether clearly judge for image, can be trained to obtain training pattern according to machine learning method, then
Go to judge whether image is clear based on this model.
Based on identical inventive concept, the embodiment of the present application also provides a kind of man-machine interaction control device.As shown in figure 5,
The device includes:
Voice messaging acquisition module 501, in the awake state, collecting the voice messaging of interactive object；
Facial angle determining module 502, for determining the friendship according to the collected image comprising the interactive object
The facial angle of mutual object；
Control module 503, if for the facial angle be greater than predetermined angle, determine do not need to the voice messaging into
Row response.
Further, the control module, be also used to if it is determined that the facial angle be less than or equal to the predetermined angle,
It determines and needs to respond the voice messaging.
Further, described device further include:
Towards adjustment module, after the voice messaging for collecting interactive object, the face of the interactive object is determined
Before angle, according to the corresponding Sounnd source direction of the voice messaging, smart machine is adjusted towards the Sounnd source direction；
Image capture module, for acquiring the image of the smart machine ambient enviroment.
Further, facial angle determining module is specifically used for:
Extract the face feature point of interactive object；
According to the face feature point of extraction, the facial angle of the interactive object is determined.
Further, the face feature point includes the characteristic point for describing face profile, and the face include eye, eyebrow
Hair, mouth, nose, face；
Facial angle determining module is specifically used for the characteristic point for being used to describe face profile that will be extracted, and inputs to logical in advance
The angle analysis model for crossing the training of depth learning method, obtains the facial angle of the interactive object.
Further, the face feature point, including nose profile point and face mask point:
Facial angle determining module is specifically used for obtaining nose two according to face mask point using nose profile point as separation
The face area of side；The ratio for calculating two sides face area determines calculating according to the corresponding relationship of preset ratio and angle
Facial angle of the corresponding angle of ratio as interactive object.
Further, facial angle determining module is specifically used for:
If determining each target object and intelligence according to acquired image comprising multiple target objects in acquired image
The distance of energy equipment；
It will be determined as the interactive object apart from nearest target object, and determine the facial angle of the interactive object.
Further, facial angle determining module before the facial angle for being also used to determine the interactive object, determines institute
Stating image is clear image.
In the man-machine interaction control method for describing the application illustrative embodiments, after device, next, introducing root
According to the smart machine of the another exemplary embodiment of the application.
Person of ordinary skill in the field it is understood that the various aspects of the application can be implemented as system, method or
Program product.Therefore, the various aspects of the application can be with specific implementation is as follows, it may be assumed that complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
In some possible embodiments, at least one processing can be included at least according to the smart machine of the application
Device and at least one processor.Wherein, memory is stored with program code, when program code is executed by processor, so that
Processor executes in the man-machine interaction control method according to the various illustrative embodiments of the application of this specification foregoing description
The step of.For example, processor can execute step 201-203 as shown in Figure 2.
The smart machine 130 of this embodiment according to the application is described referring to Fig. 6.The intelligence that Fig. 6 is shown
Equipment 130 is only an example, should not function to the embodiment of the present application and use scope bring any restrictions.
As shown in fig. 6, smart machine 130 is showed in the form of universal intelligent equipment.The component of smart machine 130 can wrap
Include but be not limited to: at least one above-mentioned processor 131, above-mentioned at least one processor 132, the different system components of connection (including
Memory 132 and processor 131) bus 133.
Bus 133 indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller,
Peripheral bus, processor or the local bus using any bus structures in a variety of bus structures.
Memory 132 may include the readable medium of form of volatile memory, such as random access memory (RAM)
1321 and/or cache memory 1322, it can further include read-only memory (ROM) 1323.
Memory 132 can also include program/utility 1325 with one group of (at least one) program module 1324,
Such program module 1324 includes but is not limited to: operating system, one or more application program, other program modules and
It may include the realization of network environment in program data, each of these examples or certain combination.
Smart machine 130 can also be communicated with one or more external equipments 134 (such as keyboard, sensing equipment etc.), also
Can be enabled a user to one or more equipment interacted with smart machine 130 communication, and/or with make the smart machine
The 130 any equipment (such as router, modem etc.) that can be communicated with one or more of the other smart machine are led to
Letter.This communication can be carried out by input/output (I/O) interface 135.Also, smart machine 130 can also be suitable by network
Orchestration 136 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network, such as because of spy
Net) communication.As shown, network adapter 136 is communicated by bus 133 with other modules for smart machine 130.It should
Understand, although not shown in the drawings, other hardware and/or software module can be used in conjunction with smart machine 130, including but unlimited
In: microcode, device driver, redundant processor, external disk drive array, RAID system, tape drive and data
Backup storage system etc..
In some possible embodiments, the various aspects of man-machine interaction control method provided by the present application can also be real
It is now a kind of form of program product comprising program code, when program product is run on a computing device, program code
For making computer equipment execute the human-computer interaction according to the various illustrative embodiments of the application of this specification foregoing description
Step in control method, for example, computer equipment can execute step 201-203 as shown in Figure 2.
Program product can be using any combination of one or more readable mediums.Readable medium can be readable signal Jie
Matter or readable storage medium storing program for executing.Readable storage medium storing program for executing for example may be-but not limited to-electricity, magnetic, optical, electromagnetic, infrared
The system of line or semiconductor, device or device, or any above combination.The more specific example of readable storage medium storing program for executing is (non-
The list of exhaustion) include: electrical connection with one or more conducting wires, portable disc, hard disk, random access memory (RAM),
Read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, the read-only storage of portable compact disc
Device (CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
The program product for human-computer interactive control of presently filed embodiment can be read-only using portable compact disc
Memory (CD-ROM) and including program code, and can run on intelligent devices.However, the program product of the application is unlimited
In this, in this document, readable storage medium storing program for executing can be any tangible medium for including or store program, which can be referred to
Enable execution system, device or device use or in connection.
Readable signal medium may include in a base band or as the data-signal that carrier wave a part is propagated, wherein carrying
Readable program code.The data-signal of this propagation can take various forms, including --- but being not limited to --- electromagnetism letter
Number, optical signal or above-mentioned any appropriate combination.Readable signal medium can also be other than readable storage medium storing program for executing it is any can
Read medium, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or
Program in connection.
The program code for including on readable medium can transmit with any suitable medium, including --- but being not limited to ---
Wirelessly, wired, optical cable, RF etc. or above-mentioned any appropriate combination.
Can with any combination of one or more programming languages come write for execute the application operation program
Code, programming language include object oriented program language-Java, C++ etc., further include conventional process
Formula programming language-such as " C " language or similar programming language.Program code can be fully in smart machine
It is upper to execute, partly execute on intelligent devices, as an independent software package executes, partially part exists on intelligent devices
It executes on other smart machines or is executed on other smart machines or server completely.It is being related to other smart machines
In situation, other smart machines can pass through the network of any kind --- even including local area network (LAN) or wide area network (WAN)-
It is connected to smart machine.
It should be noted that although being referred to several unit or sub-units of device in the above detailed description, this stroke
It point is only exemplary not enforceable.In fact, according to presently filed embodiment, it is above-described two or more
The feature and function of unit can embody in a unit.Conversely, the feature and function of an above-described unit can
It is to be embodied by multiple units with further division.
In addition, although describing the operation of the application method in the accompanying drawings with particular order, this do not require that or
Hint must execute these operations in this particular order, or have to carry out shown in whole operation be just able to achieve it is desired
As a result.Additionally or alternatively, it is convenient to omit multiple steps are merged into a step and executed by certain steps, and/or by one
Step is decomposed into execution of multiple steps.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although the preferred embodiment of the application has been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications can be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the application range.
Obviously, those skilled in the art can carry out various modification and variations without departing from the essence of the application to the application
Mind and range.In this way, if these modifications and variations of the application belong to the range of the claim of this application and its equivalent technologies
Within, then the application is also intended to include these modifications and variations.
Claims (10)
1. a kind of man-machine interaction control method, which is characterized in that the described method includes:
In the awake state, the voice messaging of interactive object is collected；
According to the collected image comprising the interactive object, the facial angle of the interactive object is determined；
If the facial angle is greater than predetermined angle, determination does not need to respond the voice messaging.
2. the method according to claim 1, wherein the method also includes:
If it is determined that the facial angle is less than or equal to the predetermined angle, determines and need to respond the voice messaging.
3. the method according to claim 1, wherein determining institute after collecting the voice messaging of interactive object
Before the facial angle for stating interactive object, further includes:
According to the corresponding Sounnd source direction of the voice messaging, smart machine is adjusted towards the Sounnd source direction；
Acquire the image of the smart machine ambient enviroment.
4. the method according to claim 1, wherein determining the facial angle of the interactive object, comprising:
Extract the face feature point of the interactive object；
According to the face feature point of extraction, the facial angle of the interactive object is determined.
5. according to the method described in claim 4, it is characterized in that, the face feature point includes for describing face profile
Characteristic point, the face include eye, eyebrow, mouth, nose, face；
According to the face feature point of extraction, the facial angle of the interactive object is determined, comprising:
By the characteristic point for being used to describe face profile of extraction, the pre- angle analysis for first passing through the training of deep learning method is inputed to
Model obtains the facial angle of the interactive object.
6. according to the method described in claim 4, it is characterized in that, the face feature point, including nose profile point and face
Profile point:
According to the face feature point of extraction, the facial angle of the interactive object is determined, comprising:
Using nose profile point as separation, the face area of nose two sides is obtained according to face mask point；
The ratio for calculating two sides face area determines that the ratio calculated is corresponding according to the corresponding relationship of preset ratio and angle
Facial angle of the angle as interactive object.
7. any method in -6 according to claim 1, which is characterized in that according to it is collected include the interactive object
Image, determine the facial angle of the interactive object, comprising:
If in acquired image including multiple target objects, according to acquired image, determine that each target object is set with intelligence
Standby distance；
It will be determined as the interactive object apart from nearest target object, and determine the facial angle of the interactive object.
8. described the method according to claim 1, wherein before the facial angle for determining the interactive object
Method further include:
Determine that described image is clear image.
9. a kind of man-machine interaction control device, which is characterized in that described device includes:
Voice messaging acquisition module, in the awake state, collecting the voice messaging of interactive object；
Facial angle determining module, for determining the interactive object according to the collected image comprising the interactive object
Facial angle；
Control module, if being greater than predetermined angle for the facial angle, determination does not need to respond the voice messaging.
10. a kind of computer-readable medium, is stored with computer executable instructions, which is characterized in that the computer is executable
Instruction is for executing the method as described in any claim in claim 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811418691.1A CN109508687A (en) | 2018-11-26 | 2018-11-26 | Man-machine interaction control method, device, storage medium and smart machine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811418691.1A CN109508687A (en) | 2018-11-26 | 2018-11-26 | Man-machine interaction control method, device, storage medium and smart machine |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109508687A true CN109508687A (en) | 2019-03-22 |
Family
ID=65750639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811418691.1A Pending CN109508687A (en) | 2018-11-26 | 2018-11-26 | Man-machine interaction control method, device, storage medium and smart machine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109508687A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070016A (en) * | 2019-04-12 | 2019-07-30 | 北京猎户星空科技有限公司 | A kind of robot control method, device and storage medium |
| CN110187766A (en) * | 2019-05-31 | 2019-08-30 | 北京猎户星空科技有限公司 | A kind of control method of smart machine, device, equipment and medium |
| CN110253595A (en) * | 2019-06-21 | 2019-09-20 | 北京猎户星空科技有限公司 | A kind of smart machine control method and device |
| CN110310657A (en) * | 2019-07-10 | 2019-10-08 | 北京猎户星空科技有限公司 | A kind of audio data processing method and device |
| CN110730115A (en) * | 2019-09-11 | 2020-01-24 | 北京小米移动软件有限公司 | Voice control method and device, terminal and storage medium |
| CN110850971A (en) * | 2019-10-25 | 2020-02-28 | 智亮君 | Handshake interaction method and system between hand model and intelligent mirror and storage medium |
| CN111126163A (en) * | 2019-11-28 | 2020-05-08 | 星络智能科技有限公司 | Intelligent panel, interaction method based on face angle detection and storage medium |
| CN111176744A (en) * | 2020-01-02 | 2020-05-19 | 北京字节跳动网络技术有限公司 | Electronic equipment control method, device, terminal and storage medium |
| CN111341350A (en) * | 2020-01-18 | 2020-06-26 | 南京奥拓电子科技有限公司 | Man-machine interaction control method and system, intelligent robot and storage medium |
| CN112634872A (en) * | 2020-12-21 | 2021-04-09 | 北京声智科技有限公司 | Voice equipment awakening method and device |
| CN114727120A (en) * | 2021-01-04 | 2022-07-08 | 腾讯科技（深圳）有限公司 | Method and device for acquiring live broadcast audio stream, electronic equipment and storage medium |
| CN115242569A (en) * | 2021-04-23 | 2022-10-25 | 海信集团控股股份有限公司 | Man-machine interaction method and server in intelligent home |
| CN115494960A (en) * | 2022-11-15 | 2022-12-20 | 中国信息通信研究院 | Signal processing method and device, controlled machine and multi-user multi-machine interface system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096373A (en) * | 2016-06-27 | 2016-11-09 | 旗瀚科技股份有限公司 | The exchange method of robot and user and device |
| CN107297745A (en) * | 2017-06-28 | 2017-10-27 | 上海木爷机器人技术有限公司 | voice interactive method, voice interaction device and robot |
| US20170368688A1 (en) * | 2016-06-27 | 2017-12-28 | Qihan Technology Co., Ltd. | Method and apparatus for adjusting interactive directions of robots |
| CN108733208A (en) * | 2018-03-21 | 2018-11-02 | 北京猎户星空科技有限公司 | The I-goal of smart machine determines method and apparatus |
| CN108766438A (en) * | 2018-06-21 | 2018-11-06 | Oppo广东移动通信有限公司 | Man-machine interaction method, device, storage medium and intelligent terminal |
-
2018
- 2018-11-26 CN CN201811418691.1A patent/CN109508687A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096373A (en) * | 2016-06-27 | 2016-11-09 | 旗瀚科技股份有限公司 | The exchange method of robot and user and device |
| US20170368688A1 (en) * | 2016-06-27 | 2017-12-28 | Qihan Technology Co., Ltd. | Method and apparatus for adjusting interactive directions of robots |
| CN107297745A (en) * | 2017-06-28 | 2017-10-27 | 上海木爷机器人技术有限公司 | voice interactive method, voice interaction device and robot |
| CN108733208A (en) * | 2018-03-21 | 2018-11-02 | 北京猎户星空科技有限公司 | The I-goal of smart machine determines method and apparatus |
| CN108766438A (en) * | 2018-06-21 | 2018-11-06 | Oppo广东移动通信有限公司 | Man-machine interaction method, device, storage medium and intelligent terminal |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070016A (en) * | 2019-04-12 | 2019-07-30 | 北京猎户星空科技有限公司 | A kind of robot control method, device and storage medium |
| CN110187766A (en) * | 2019-05-31 | 2019-08-30 | 北京猎户星空科技有限公司 | A kind of control method of smart machine, device, equipment and medium |
| CN110253595B (en) * | 2019-06-21 | 2022-01-21 | 北京猎户星空科技有限公司 | Intelligent equipment control method and device |
| CN110253595A (en) * | 2019-06-21 | 2019-09-20 | 北京猎户星空科技有限公司 | A kind of smart machine control method and device |
| CN110310657A (en) * | 2019-07-10 | 2019-10-08 | 北京猎户星空科技有限公司 | A kind of audio data processing method and device |
| CN110310657B (en) * | 2019-07-10 | 2022-02-08 | 北京猎户星空科技有限公司 | Audio data processing method and device |
| CN110730115A (en) * | 2019-09-11 | 2020-01-24 | 北京小米移动软件有限公司 | Voice control method and device, terminal and storage medium |
| CN110730115B (en) * | 2019-09-11 | 2021-11-09 | 北京小米移动软件有限公司 | Voice control method and device, terminal and storage medium |
| US11335345B2 (en) | 2019-09-11 | 2022-05-17 | Beijing Xiaomi Mobile Software Co., Ltd. | Method for voice control, terminal, and non-transitory computer-readable storage medium |
| CN110850971A (en) * | 2019-10-25 | 2020-02-28 | 智亮君 | Handshake interaction method and system between hand model and intelligent mirror and storage medium |
| CN111126163A (en) * | 2019-11-28 | 2020-05-08 | 星络智能科技有限公司 | Intelligent panel, interaction method based on face angle detection and storage medium |
| CN111176744A (en) * | 2020-01-02 | 2020-05-19 | 北京字节跳动网络技术有限公司 | Electronic equipment control method, device, terminal and storage medium |
| CN111341350A (en) * | 2020-01-18 | 2020-06-26 | 南京奥拓电子科技有限公司 | Man-machine interaction control method and system, intelligent robot and storage medium |
| CN112634872A (en) * | 2020-12-21 | 2021-04-09 | 北京声智科技有限公司 | Voice equipment awakening method and device |
| CN114727120A (en) * | 2021-01-04 | 2022-07-08 | 腾讯科技（深圳）有限公司 | Method and device for acquiring live broadcast audio stream, electronic equipment and storage medium |
| CN114727120B (en) * | 2021-01-04 | 2023-06-09 | 腾讯科技（深圳）有限公司 | Live audio stream acquisition method and device, electronic equipment and storage medium |
| CN115242569A (en) * | 2021-04-23 | 2022-10-25 | 海信集团控股股份有限公司 | Man-machine interaction method and server in intelligent home |
| CN115242569B (en) * | 2021-04-23 | 2023-12-05 | 海信集团控股股份有限公司 | Man-machine interaction method and server in intelligent home |
| CN115494960A (en) * | 2022-11-15 | 2022-12-20 | 中国信息通信研究院 | Signal processing method and device, controlled machine and multi-user multi-machine interface system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109508687A (en) | Man-machine interaction control method, device, storage medium and smart machine | |
| Kessous et al. | Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis | |
| US11393206B2 (en) | Image recognition method and apparatus, terminal, and storage medium | |
| US10664060B2 (en) | Multimodal input-based interaction method and device | |
| Hou et al. | Signspeaker: A real-time, high-precision smartwatch-based sign language translator | |
| US10860839B2 (en) | Method and apparatus for generating training data for human face recognition, device and computer storage medium | |
| US20210174034A1 (en) | Computer vision based sign language interpreter | |
| Wöllmer et al. | LSTM-modeling of continuous emotions in an audiovisual affect recognition framework | |
| Von Agris et al. | Recent developments in visual sign language recognition | |
| CN111432989A (en) | Artificially enhanced cloud-based robot intelligence framework and related methods | |
| KR102133728B1 (en) | Device, method and readable media for multimodal recognizing emotion based on artificial intelligence | |
| Turk et al. | Perceptual interfaces | |
| CN109063587A (en) | data processing method, storage medium and electronic equipment | |
| Kang et al. | Development of head detection and tracking systems for visual surveillance | |
| CN112667068A (en) | Virtual character driving method, device, equipment and storage medium | |
| Minotto et al. | Multimodal multi-channel on-line speaker diarization using sensor fusion through SVM | |
| WO2020244074A1 (en) | Expression interaction method and apparatus, computer device, and readable storage medium | |
| Alshamsi et al. | Automated facial expression and speech emotion recognition app development on smart phones using cloud computing | |
| CN109948450A (en) | A kind of user behavior detection method, device and storage medium based on image | |
| Johansson et al. | Opportunities and obligations to take turns in collaborative multi-party human-robot interaction | |
| US11216067B2 (en) | Method for eye-tracking and terminal for executing the same | |
| CN110253595A (en) | A kind of smart machine control method and device | |
| US20230143452A1 (en) | Method and apparatus for generating image, electronic device and storage medium | |
| CN115757706A (en) | Active speaker detection using image data | |
| CN106502382A (en) | Active exchange method and system for intelligent robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190322 |