WO2017215297A1 - Système interactif en nuage, robot intelligent multicognitif, et procédé d'interaction cognitive associés - Google Patents

Système interactif en nuage, robot intelligent multicognitif, et procédé d'interaction cognitive associés Download PDF

Info

Publication number
WO2017215297A1
WO2017215297A1 PCT/CN2017/076274 CN2017076274W WO2017215297A1 WO 2017215297 A1 WO2017215297 A1 WO 2017215297A1 CN 2017076274 W CN2017076274 W CN 2017076274W WO 2017215297 A1 WO2017215297 A1 WO 2017215297A1
Authority
WO
WIPO (PCT)
Prior art keywords
recognition
signal
cloud
pressure
intelligent robot
Prior art date
Application number
PCT/CN2017/076274
Other languages
English (en)
Chinese (zh)
Inventor
刘若鹏
舒良轩
Original Assignee
深圳光启合众科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳光启合众科技有限公司 filed Critical 深圳光启合众科技有限公司
Publication of WO2017215297A1 publication Critical patent/WO2017215297A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00—Manipulators not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00—Manipulators not otherwise provided for
    • B25J11/0005—Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means

Definitions

  • the invention relates to a robot, in particular to a cloud interactive system and a multi-sense intelligent robot and a sensing interaction method thereof.
  • the traditional intelligent electronic pet is a virtual pet running on a smart platform. This type of electronic pet lacks realism, and the user can only feel its presence in the virtual world through the display device. There is also a class of toy electronic pets with interactive functions. Although the authenticity is strong, it is limited by the processing ability and other factors, and the intelligence displayed is often limited.
  • family companion robots can interact with humans in simple interactions, such as imitation of movements and imitation of sounds, but their interaction behavior is far from that of real pets.
  • the ability of the robot to perceive complex external parameters and to interact and process accordingly is relatively weak.
  • the technical problem to be solved by the present invention is to provide a cloud interactive system and a multi-perceptive intelligent robot and a sensing interaction method thereof, which have multiple sensing capabilities and have stronger processing and interaction capabilities.
  • the invention provides a multi-sense intelligent robot with cloud interaction function, which cooperates with an external cloud server.
  • the intelligent robot includes: a password recognition processing unit for performing local password recognition on the externally input voice signal and generating a password recognition processing result; and a local image recognition processing unit for The externally input scene image performs local image recognition and generates a local image recognition result; the pressure signal recognition processing unit is configured to identify and process the external pressure signal and generate a pressure-sensing emotional signal; and a cloud recognition unit for using the voice signal Sending to the cloud server and performing at least one of cloud speech recognition and cloud semantic understanding by the cloud server, and receiving a cloud speech recognition processing result sent by the cloud server; and sending the scene image to the The cloud server performs facial recognition by the cloud server, and receives a cloud face recognition result sent by the cloud server; the controller is configured to perform at least the password recognition processing result and the cloud voice recognition processing result according to the cloud server And performing, by the at least one of the local image recognition result and the cloud face recognition result and/or the pressure-
  • the multi-sense intelligent robot further includes an identification selecting unit configured to determine an externally input voice signal, thereby selecting to transmit the externally input voice signal to the password recognition processing unit. Or transmitting to the cloud recognition unit, and/or judging the externally input scene image, thereby selecting whether to transmit the externally input scene image to the local image recognition processing unit or to the cloud recognition unit.
  • an identification selecting unit configured to determine an externally input voice signal, thereby selecting to transmit the externally input voice signal to the password recognition processing unit. Or transmitting to the cloud recognition unit, and/or judging the externally input scene image, thereby selecting whether to transmit the externally input scene image to the local image recognition processing unit or to the cloud recognition unit.
  • the multi-sense intelligent robot further includes a voice collection unit for obtaining an externally input voice signal.
  • the voice collection unit is a microphone, and the number of the microphones is two, which are respectively installed at the left and right ears of the intelligent robot.
  • the multi-sense intelligent robot further includes a preset password storage unit, where the preset password storage unit is configured to store preset password data; and the password recognition processing unit is configured to The password data is set to perform local password recognition on the voice signal and generate a password recognition processing result.
  • the multi-sense intelligent robot further includes a voiceprint recognition unit, configured to perform identity verification according to the pre-stored voiceprint data before performing the recognition process on the voice signal.
  • the multi-sense intelligent robot further includes an image acquisition unit for capturing more than one scene image of the external input.
  • the multi-sense intelligent robot further includes a face image acquiring unit configured to acquire a face image having the recognized feature point from the externally input scene image; the local image recognition processing unit For performing local image recognition on the face image with the identified feature points The image recognition result is obtained by the cloud recognition unit; the cloud recognition unit is configured to send the face image having the identification feature point to the cloud server for cloud face recognition.
  • the face image acquiring unit is further configured to: after acquiring a face image having the recognized feature point from the externally input scene image, excluding the face image not having the recognized feature point.
  • the multi-sense intelligent robot further includes a preset image storage unit configured to store preset image data; the local image recognition processing unit is configured to use the preset image data to The externally input scene image performs local image recognition and generates a local image recognition result.
  • the multi-sense intelligent robot further includes a pressure signal acquisition unit for acquiring an external pressure signal.
  • the pressure signal acquisition unit is a resistive pressure sensor.
  • the pressure signal acquisition unit includes a pressure sensing chip array distributed on a surface of the intelligent robot and an analog to digital conversion circuit connected to the pressure sensing chip array, the pressure sensing The chip array senses the pressure change of the surface of the intelligent robot and converts it into a pressure analog signal, which converts the pressure analog signal into a pressure digital signal.
  • the pressure signal recognition processing unit includes: a pressure type determining unit, configured to calculate a pressure change rate of the external pressure signal, according to the pressure change rate and a preset change threshold ratio Determining a type of the external pressure signal; a pressure position determining unit configured to determine a pressure generating position based on the external pressure signal; and a pressure sensing type emotion signal generating unit configured to generate a position and an external pressure signal according to the pressure The type is compared with a preset map list, and a pressure-aware emotion signal corresponding to the pressure generation position and the type of the external pressure signal is generated.
  • the pressure signal recognition processing unit further includes a connection with the pressure type determining unit and the pressure sensing type emotion signal generating unit, respectively, for storing a preset change threshold and a preset mapping list. Data storage unit.
  • the multi-sense intelligent robot further includes a motion sensing unit coupled to the controller, configured to sense a motion state of the smart robot to generate a motion state parameter.
  • the motion sensing unit is a gravity acceleration sensor, a gyroscope or a tilt sensor mounted on the torso of the smart robot.
  • the multi-sense intelligent robot further includes a network determining unit, configured to determine a connection state of the smart robot and the cloud server, and generate a connection state according to the connection state. The result of the network judgment.
  • the intelligent robot and the cloud server are connected through a wireless network interface.
  • the controller is configured with an impact model, the password recognition processing result, the cloud voice recognition processing result, the local image recognition result, the cloud face recognition result, the The pressure-aware emotional signal, the motion state parameter is an input parameter of the impact model, and the impact model outputs the interaction decision according to the input parameter.
  • the controller activates the intelligent robot in response to a start command.
  • the startup instruction is included in a voice signal
  • the password recognition processing unit or the cloud recognition unit is further configured to identify a startup instruction in the voice signal
  • the startup instruction includes
  • the pressure signal recognition processing unit is further configured to identify a start command in the external pressure signal
  • the start command is included in a wireless signal
  • the smart robot further includes a wireless communication unit and a wireless signal An identification unit, the wireless communication unit is configured to receive an externally transmitted wireless signal, and the wireless signal identification unit is configured to identify a startup instruction in the wireless signal.
  • the invention also provides a method for perceptual interaction of a multi-sense intelligent robot with cloud interaction function, which comprises: performing local password recognition on an externally input voice signal and generating a password recognition processing result, or transmitting the voice signal
  • Receiving at least one of cloud speech recognition and cloud semantic understanding by the cloud server receiving a cloud speech recognition processing result sent by the cloud server; performing local image recognition on the externally input scene image and generating a local Image recognition result, or transmitting the scene image to the cloud server for face recognition and receiving the cloud face recognition result returned by the cloud server; identifying and processing the external pressure signal and generating a pressure-sensing emotion signal; And making the intelligent robot according to at least one of the password recognition processing result and the cloud speech recognition processing result, at least one of the local image recognition result and the cloud face recognition result, and/or a pressure-aware emotion signal Interactive decision The execution of the interactive decision.
  • the method for perceptual interaction further includes: determining, by the externally input voice signal, whether to perform local password recognition on the externally input voice signal or transmitting to the cloud server, and/or Determining the externally input scene image to select whether to perform local image recognition on the externally input scene image or to transmit to the cloud server.
  • the method for perceptual interaction further includes obtaining an externally input voice message. number.
  • the method for perceptual interaction further includes storing a preset password data, and performing local password recognition on the externally input voice signal and generating a password recognition processing result according to the preset password data Local password recognition is performed on the voice signal and a password recognition processing result is generated.
  • the local password recognition is performed on the externally input voice signal and the password recognition processing result is generated, or the voice signal is sent to the cloud server, and further includes a pre-stored voiceprint.
  • the data authenticates the voice signal.
  • the method for perceptual interaction further includes capturing more than one scene image of an external input.
  • the method for perceptual interaction further includes acquiring a face image having the identified feature point from more than one scene image; and performing local image recognition on the externally input scene image and generating a local image recognition result
  • the step of performing local image recognition on the face image having the identification feature point and generating a local image recognition result; and the step of transmitting the scene image to the cloud server for face recognition is to have the identification feature point
  • the face image is sent to the cloud server for cloud face recognition.
  • the step of acquiring the face image having the recognized feature point from the externally input scene image further includes excluding the face image not having the recognized feature point.
  • the method for perceptually interacting further includes storing the preset image data, and performing local image recognition on the externally input scene image and generating a local image recognition result according to the preset image data Performing local image recognition on the face image having the identification feature point and generating a local image recognition result.
  • the method before the sending the voice signal or the scene image to the cloud server, the method further includes: determining whether the network status is normal, and sending the voice signal or the scene image to the cloud when the network is normal. server.
  • the method for perceptual interaction further includes acquiring an external pressure signal.
  • the step of performing an identification process on the external pressure signal and generating the pressure-sensing emotion signal includes: calculating a pressure change rate of the external pressure signal, according to the pressure change rate and a preset change threshold Aligning determines the type of the external pressure signal; based on the external pressure signal a constant pressure generating position; and comparing the type of the pressure generating position and the external pressure signal with a preset mapping list, and generating a pressure sensing type emotion signal corresponding to the pressure generating position and the type of the external pressure signal.
  • the method for perceptual interaction further includes storing a preset change threshold and a preset mapping list.
  • determining the type of the external pressure signal as tapping if the pressure change rate is greater than a preset first change threshold, determining the type of the external pressure signal as tapping; otherwise, determining the type of the external pressure signal as a stroke .
  • determining the type of the external pressure signal as a beat comprises: if the pressure change rate is greater than the first change The threshold is less than or equal to the second change threshold, and the type of the pressure signal is determined to be a slight tap; and if the pressure change rate is greater than the second change threshold, the type of the pressure signal is determined to be a hard tap.
  • the calculating a pressure change rate of the external pressure signal is: calculating a duration value of the external pressure signal, and selecting the preset according to the preset time period within the continuous time value
  • the digital signal corresponding to the time period calculates a pressure change rate according to the preset time period and the digital signal corresponding to the preset time period.
  • the preset time period is 0.5-1.5 seconds.
  • the method for perceptual interaction further includes sensing a motion state of the smart robot to generate a motion state parameter.
  • the interactive decision includes an emotional expression location and an emotional expression instruction.
  • the emotion expression part includes an upper limb, a lower limb, a trunk, a head, a face, and/or a mouth of the intelligent robot;
  • the emotion expression instruction includes executing a corresponding motion instruction, and playing a corresponding prompt Voice and / or display the corresponding prompt information.
  • the action instruction comprises a mechanical action command and/or a facial expression command.
  • the mechanical action command includes action type information, action amplitude information, action frequency information, and/or action duration information corresponding to the emotion expression portion.
  • the method for perceptual interaction further includes: in response to a startup command The smart robot is activated; the start command is included in a voice signal, in an external pressure signal, or in a wireless signal.
  • the invention also provides a cloud interaction system, comprising the above-mentioned multi-aware intelligent robot with cloud interaction function and a cloud server, and the intelligent robot performs wireless communication with the cloud server.
  • the invention has the following significant advantages: by configuring a plurality of sensing devices, comprehensively acquiring environmental signals and making interactive decisions, thereby improving the interactive capability of the robot.
  • the cloud recognition unit communicates with external processing resources, which improves the processing power of the robot and makes more complex interactive decisions possible.
  • FIG. 1 is a system block diagram of a multi-sense intelligent robot with cloud interaction function according to a first embodiment of the present invention.
  • FIG. 2 is a system block diagram of a multi-sense intelligent robot with cloud interaction function according to a second embodiment of the present invention.
  • FIG. 3 is a flow chart of a method for perceptual interaction of a multi-sense intelligent robot with cloud interaction function according to an embodiment of the invention.
  • FIG. 4 is a flowchart of a cloud voice recognition method according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of a cloud speech recognition method according to another embodiment of the present invention.
  • FIG. 6 is a flow chart of a cloud speech recognition method according to another embodiment of the present invention.
  • Fig. 7 is a schematic view showing the use of a pressure sensor according to an embodiment of the present invention.
  • FIG. 8 is a flow chart of a haptic sensing method according to an embodiment of the present invention.
  • FIG. 9 is a flow chart of a haptic sensing method according to another embodiment of the present invention.
  • Figure 10 is a schematic diagram of an impact model in accordance with an embodiment of the present invention.
  • FIG. 11 is a flow chart of a face recognition method according to an embodiment of the present invention.
  • FIG. 12 is a flowchart of a face recognition method according to another embodiment of the present invention.
  • FIG. 13 is a flowchart of a face recognition method according to another embodiment of the present invention.
  • FIG. 14 is a block diagram showing the structure of a pressure signal processing unit of the multi-sense intelligent robot shown in FIG. 1.
  • Embodiments of the present invention describe a multi-sense intelligent robot with cloud interaction function and an interactive method thereof, and the method and system are particularly suitable for a home companion robot. It will of course be understood that the method and system are also applicable to other robots with high interaction requirements, such as commercial service robots.
  • the robot's processing and decision-making ability is improved by giving the robot multiple sensing functions and performing interactive decision-making and motion control based on these sensing functions.
  • the intelligent robot 100 of the present embodiment includes a voice collection unit 101, an image acquisition unit 102, a pressure signal acquisition unit 103, a motion sensing unit 104, a password recognition processing unit 105, a local image recognition processing unit 106, and a pressure.
  • the various components can be connected to controller 108 as needed.
  • the cloud identification unit 110 is configured to communicate with the external cloud server 200.
  • the power management unit 111 is for supplying power to the entire smart robot 100.
  • the power management unit 111 provides a stably adapted power supply to each unit through the DC-DC module. At the same time, the power management unit 111 can configure the overload protection circuit to avoid overloading of the motion actuator.
  • the cloud identification unit 110 can communicate with the cloud server 200 in a variety of ways.
  • the cloud server 200 can be a cluster of one server or multiple servers, and the manufacturer of the smart robot 100 can set up a cloud server or obtain a service interface provided by a network provider.
  • the cloud identification unit 110 can communicate with the cloud server through a wireless local area network that accesses the Internet. Alternatively, the cloud identification unit 110 can also communicate with the cloud server via the mobile internet.
  • the voice collection unit 101 is configured to collect voice signals from the environment.
  • An embodiment of the voice acquisition unit 101 is a microphone that can acquire voice signals.
  • the microphone can be mounted at the left and right ears of the head of the smart robot 100.
  • the two microphones of the two ears are used as a voice input source, and the collected sound information is converted into a voice signal in the form of an electrical signal.
  • the voice signal is audio information of a natural language, and needs to be subjected to noise reduction, filtering, and the like.
  • a microphone employing an intelligent digital array noise canceling pickup having two noise reduction modes reduces noise by up to 45 dB.
  • the microphones are respectively placed at the ears of the penguin, and the acquired audio signals are dispersed to ensure the accuracy of the acquired audio signals. Sex and integrity.
  • the voice collection unit 101 may also have a voice pre-processing function.
  • the externally input voice signal may be affected by factors such as environment, scene, relative position, etc., and needs to perform modulation, demodulation, voice noise reduction, audio amplification, and the like on the audio information. Pretreatment. Among them, voice noise reduction can use DSP noise reduction algorithm to reduce noise, which can remove background noise, suppress external vocal interference, suppress echo, and suppress reverberation.
  • the DSP noise reduction algorithm has a strong ability to suppress both steady-state and non-steady-state noise as well as mechanical noise.
  • the combination of dual microphone and voice pre-processing eliminates noise almost completely, while ensuring the clarity and naturalness of normal speech and output without delay.
  • the preprocessed speech signal is transmitted through the harness to the identification selection unit 109 located in the intelligent robot cavity for processing.
  • the voice signal contains various passwords that the robot is interested in. For example, the password of the robot is called, and the robot is allowed to complete the passwords for running, jumping, and the like.
  • the identification selection unit 109 receives the speech signal and determines an appropriate speech recognition unit in accordance with a predetermined policy.
  • speech recognition refers to extracting text content through a series of sound algorithms according to the input sound signal.
  • the two voice recognition modes provided in the embodiment of the present invention include local identification and cloud recognition. After the identification selection unit 109 determines a specific voice recognition mode, the voice signal is sent to the corresponding identification unit, and the processing result is received.
  • the Local identification is to send a voice signal to the password recognition processing unit 105.
  • the cloud recognition is sent to the cloud server 200 by the cloud recognition unit 110 and at least one of cloud speech recognition and cloud semantic understanding is performed by the cloud server 200, and the cloud speech recognition processing result sent by the cloud server 200 is received.
  • the identification selection unit 109 can set a plurality of types of predetermined policies, for example, designating an identification unit in a voice signal, or performing local recognition by default, and then performing cloud recognition, or vice versa.
  • the choice of strategy can reduce the time of useless identification and improve the efficiency of intelligent robots. For example, in general, the processing efficiency of local recognition is higher than the processing efficiency of cloud recognition. Therefore, the voice signal is usually locally identified and then cloud-recognized.
  • the identification selection unit 109 determines whether to transmit a voice signal to the cloud server 200 for cloud recognition based on the password recognition processing result. Further, the identification selecting unit 109 determines whether the voice signal is successfully recognized by the local password according to the result of the password recognition processing, and if so, performs subsequent processing, for example, responding to the password; if not, transmitting the voice signal to the cloud server 200 for cloud identification. In another example, the identification selection unit 109 determines whether to perform local password recognition of the speech signal based on the result of the cloud speech recognition processing.
  • the identification selecting unit 109 determines whether the voice signal is successfully recognized by the cloud according to the cloud voice recognition processing result, and if so, performs subsequent processing, for example, responding to the password, and if not, the voice signal is locally password-recognized.
  • the identification selection unit 109 can perform the above selection operation autonomously. In another embodiment, the identification selection unit 109 can perform the above selection operation under the control of the controller 108.
  • the password recognition processing unit 105 executes locally, reads the voice signal from the controller 108, compares the predefined password data with the voice signal, and executes an appropriate processing module based on the comparison result.
  • the password recognition processing unit 105 also returns the recognition processing result to the controller 108.
  • the predefined password data can be understood as a series of voice signals stored locally, and the processing modules of the voice signals are integrated in the password recognition processing unit 105.
  • These processing modules are implemented by software or circuit form. For example, enter the greeting password "Hello", which corresponds to the Q&A module and gives an answer "Hello.” Of course, these processing modules can be integrated or implemented separately.
  • the exemplary description herein is not intended to limit the invention itself.
  • the intelligent robot may further include a preset password storage unit 115 for storing preset password data.
  • the password recognition processing unit 105 can perform local password recognition on the voice signal according to the preset password data and generate a password recognition processing result.
  • Cloud recognition can be one of cloud speech recognition and cloud semantic understanding or a combination of the two, and the cloud processing performs corresponding processing according to the extracted language information.
  • many Internet companies provide online cloud function services such as online speech recognition and semantic understanding. By accessing the APIs provided by these companies, they can obtain corresponding services. For example, if a voice signal of "Beijing-Hankou flight inquiry" is sent to an online flight service provider, the flight service provider performs speech recognition, speech analysis, semantic understanding, etc. on the voice signal, thereby obtaining a voice signal.
  • the logical meaning according to the logical meaning, returns to the current flight information of Hankou in Beijing, and returns the cloud speech recognition processing result to the controller 108.
  • the smart robot 100 can be normally in a standby or hibernation state, waiting for the user's activation (eg, a voice call).
  • the voice collection unit 101 collects a voice signal
  • the password recognition processing unit 105 or the cloud recognition unit 110 can recognize a startup command in the voice signal and transmit it to the controller 108, and the controller 108 responds to the startup command,
  • the intelligent robot 100 starts working.
  • the controller 108 can cause the intelligent robot 100 to start working under other conditions.
  • the controller 108 causes the smart robot 100 to start working in response to the user's on/off button.
  • the startup command can also be included in the wireless signal.
  • the intelligent robot includes a wireless communication unit for receiving an externally transmitted wireless signal and a wireless signal recognition unit (not shown) for identifying a startup command in the wireless signal.
  • FIG. 2 is a system block diagram of an intelligent robot with cloud interactive function according to a second embodiment of the present invention.
  • the smart robot shown in FIG. 2 adds the voiceprint identifying unit 113 and the network determining unit 114 as compared with the intelligent robot structure shown in FIG.
  • the voiceprint recognition unit 113 can be connected to the voice collection unit 101 and the controller 108, and the voiceprint recognition unit 113 is configured to perform identity verification on the person who sends the voice signal according to the pre-stored voiceprint data, wherein the voiceprint data can be stored locally. Also stored in the cloud (such as Cloud Server 200).
  • the voiceprint recognition allows the intelligent robot to respond only to the sound signals of specific people, thereby increasing the safety of the intelligent robot.
  • the network judging unit 114 can determine the connection state of the smart robot 100 and the cloud server 200 between the controller 108 and the cloud recognizing unit 110 and generate a network judgment result based on the connection state. For this reason, before the voice signal is sent to the cloud server 200 for cloud identification processing, the current network state is obtained first, and the voice signal is sent to the cloud server 200 for recognition processing only when the network judgment result is that the network is normal.
  • the existing network connection technologies have wireless and wired connections. Considering the characteristics that intelligent robots need to move, the preferred method is wireless connection, which is connected to the Internet through WIFI or Bluetooth.
  • the controller 108 determines whether to call another recognition processing unit based on the recognition processing result of the current recognition processing unit.
  • the intelligent robot 100 integrates offline password recognition and cloud online recognition, and can determine an applicable identification unit and an execution sequence according to actual scenarios or other strategies, and expands the scope of use of the robot.
  • the cloud recognition processing function can be extended as needed to enhance the intelligence of the intelligent robot.
  • FIG. 4 shows a flow chart of one embodiment of a cloud speech recognition method.
  • the cloud speech recognition method includes steps 410-460.
  • an externally input speech signal is obtained.
  • an externally input sound signal is received through a microphone mounted on the body part of the intelligent robot.
  • a microphone employing an intelligent digital array noise canceling pickup having two noise reduction modes reduces noise by up to 45 dB.
  • the microphones are respectively placed at the ears of the penguin-shaped intelligent robot, and the accuracy and integrity of the acquired audio signals are ensured by dispersing the collected sound signals.
  • the voice signal is sent to the cloud server to perform cloud recognition processing.
  • cloud software services and cloud voice storage functions to realize cloud speech recognition and cloud semantic understanding, to ensure that voice signals are recognized to the maximum extent and to obtain corresponding services according to the language information extracted from voice signals. information.
  • many Internet companies currently provide cloud software function services such as online speech recognition and semantic understanding. By accessing the APIs provided by these companies, they can obtain corresponding services.
  • step 430 it is determined whether the voice signal is capable of cloud recognition processing.
  • the cloud speech recognition result of step 420 is judged. If the recognition is successful, the password is responded to and executed in step 460. Otherwise, step 440 is performed to perform local password recognition processing.
  • a local password recognition process is performed.
  • the local password recognition process is a supplement to the cloud recognition. After the cloud identification fails, the local password recognition process is started, the password stored in the local and the input password are compared, the corresponding processing module is called, and the processing result is obtained.
  • step 450 it is determined whether the password can be identified. In this step, if the password recognition process is successful, it is determined based on the processing result that the actuator is restarted. If the password recognition process fails, no action is taken.
  • the password is responded to.
  • an actuator that drives an intelligent robot performs mechanical actions or provides information.
  • the actuator can include a speaker, a display, and a moving component for playing voice prompt information, displaying text or graphics, and performing mechanical actions. For example, answer the user's greeting message, or answer the question according to the pre-edited question and answer list, or do some simple actions according to the user's request.
  • FIG. 5 is a flow chart showing another embodiment of the cloud speech recognition method of the present invention. As shown in FIG. 5, the cloud speech recognition method includes steps 510-560.
  • the cloud speech recognition method shown in FIG. 5 and the cloud speech recognition method shown in FIG. 4 differ only in the execution order.
  • the local password recognition is first performed.
  • Figure 4 is the opposite. Only steps 520-550 that are different from FIG. 4 are described herein.
  • step 520 a local password recognition process is performed.
  • the comparison is performed according to the pre-stored password and the entered password, and the corresponding processing module is called, and the password recognition processing result is obtained.
  • step 530 it is determined whether the speech signal can be identified.
  • the password recognition processing result of step 520 is judged. If the recognition is successful, it is determined that the execution mechanism is restarted, and the processing is performed to step 560, otherwise step 540 is performed.
  • the voice signal is sent to the cloud server for cloud identification.
  • the cloud software service and cloud voice storage function are used to realize cloud speech recognition and cloud semantic understanding, to ensure that the voice signal is recognized to the maximum extent and to obtain corresponding service or information according to the language information extracted from the voice signal.
  • many Internet companies currently provide cloud software function services such as online speech recognition and semantic understanding. By accessing the APIs provided by these companies, they can obtain corresponding services.
  • step 550 it is determined whether the voice signal is capable of cloud recognition processing.
  • the cloud speech recognition result of step 540 is judged. If the recognition is successful, it is determined that the execution mechanism is restarted, and the process is performed in step 560. If the cloud recognition processing fails, no action is taken.
  • FIG. 6 is a flow chart showing another embodiment of the cloud speech recognition method of the present invention.
  • the cloud interaction method includes steps 610-670.
  • step 640 "determining the cloud network status" is added.
  • the voice signal is submitted to the cloud server for identification processing. This implementation is to improve the efficiency of cloud recognition and reduce network latency.
  • the recognition execution priority may also be determined based on a predefined preference policy. For example, it can be determined by fuzzy matching whether those voice signals are first sent to the cloud server for processing, or are processed locally first. For another example, the processing priority may be determined by enumeration, and the local processing password information is relatively limited, and the voice information not in the range is sent to the cloud server for processing.
  • the voice signal before the voice signal is sent to the server to perform cloud recognition, the voice signal is pre-processed, including pre-processing the voice modulation, demodulation, voice noise reduction, and audio amplification.
  • the person who sent the voice signal may also be authenticated according to the pre-stored voiceprint data.
  • the image acquisition unit 102 is configured to capture more than one scene image of an external input.
  • An example of image acquisition unit 102 is a camera.
  • the camera can be mounted on the eyes of the smart robot 100.
  • the image acquisition unit 102 may continuously acquire images, or may acquire one or several frames of images at regular intervals, depending on the specific occasion.
  • the acquired scene image is processed by the harness transmission to the identification selection unit 109 located in the intelligent robot cavity.
  • the recognition selection unit 109 receives the scene image and determines an appropriate image recognition unit according to a predetermined policy.
  • the two image recognition modes provided in the embodiment of the present invention include local recognition and cloud recognition.
  • the identification selection unit 109 determines a specific image recognition mode, and then sends the scene image to the corresponding identification unit, and receives the processing result.
  • the local recognition is to transmit the scene image to the ontology image recognition processing unit 106.
  • the cloud recognition is sent to the cloud server 200 through the cloud recognition unit 110 and the face recognition is performed by the cloud server 200, and the face recognition result sent by the cloud server 200 is received.
  • the identification selection unit 109 can set a plurality of types of predetermined policies, for example, specifying a recognition unit in the scene image, or defaulting first Line local identification, then perform cloud recognition, or vice versa.
  • the choice of strategy can reduce the time of useless identification and improve the efficiency of intelligent robots.
  • the processing efficiency of local recognition is higher than the processing efficiency of cloud recognition. Therefore, the scene image is usually locally identified and then cloud-recognized.
  • the identification selection unit 109 determines whether to send the scene image to the cloud server 200 for cloud recognition based on the local image recognition result.
  • the identification selecting unit 109 determines whether the scene image is successfully recognized by the local password according to the local image recognition result, and if so, performs subsequent processing; if not, sends the scene image to the cloud server 200 for cloud recognition.
  • the recognition selection unit 109 determines whether to perform local image recognition on the scene image based on the cloud face recognition processing result. Further, the identification selecting unit 109 determines whether the scene image is successfully recognized by the cloud according to the cloud face recognition result, and if so, performs subsequent processing, and if not, performs local image recognition on the scene image.
  • the intelligent robot 100 further includes a face image acquisition unit 116 that connects the image acquisition unit 102 and the recognition selection unit 109 for externally input scene images.
  • the face image acquisition unit 116 may include an algorithm capable of making a preliminary selection. This algorithm is designed to capture images of faces that have identified feature points while removing images that are not recognizable or obscured. If the face images having the recognition feature points are not acquired, the face image acquisition unit 116 excludes the face images that do not have the recognition feature points, and notifies the image acquisition unit 102 to continue capturing the scene images.
  • the local image recognition processing unit 106 can perform local image recognition on the face image having the recognized feature point and generate a local image recognition result.
  • the cloud recognition unit 110 may also send the face image having the identification feature point to the cloud server 200 and perform face recognition by the cloud server 200 and receive the cloud face recognition result sent by the cloud server 200. This operation can save processing resources and transfer resources.
  • the intelligent robot 100 further includes a preset image storage unit 117 for storing preset image data.
  • the local image recognition processing unit 106 can perform local image recognition on the face image having the recognized feature point according to the preset image data and generate a local image recognition result.
  • a feature of this embodiment is that some complex operations and processing can be done without utilizing the internal resources of the intelligent robot, but rely on an external server.
  • the processing steps performed by the intelligent robot 100 on the captured scene image are preliminary processing, in order to select a face image from which a face exists, and then send the face image and the face recognition request to the cloud.
  • the server 200 requests execution of face recognition.
  • the cloud server 200 is equipped with a program for executing a face recognition algorithm, which can respond to a face recognition request.
  • the feature points are analyzed on the image and compared with the face database to obtain face recognition information.
  • the face recognition algorithm in the cloud server 200 can use a known algorithm, and is not expanded in detail here.
  • the face image acquiring unit 116 it is further determined whether a scene image includes a face image having the recognized feature point, and if so, the face image is acquired, and if not, the image capturing unit 102 is notified to continue capturing the scene. image.
  • the cloud identification unit 110 can transmit the face image to the cloud server 200 through the wireless local area network accessing the Internet.
  • the cloud server 200 can obtain and establish a face library of family members in advance for comparison identification.
  • the cloud recognition unit 110 may also transmit a face image to the cloud server via the mobile internet.
  • Commercial service robots typically use cloud server 200 to store a face library of sufficient capacity and to provide sufficiently powerful processing resources.
  • FIG. 11 is a flowchart of a face recognition method according to an embodiment of the present invention. As shown in FIG. 11, the face recognition method includes steps 1110-1170.
  • step 1110 more than one scene image of the external input is captured.
  • an external image is captured by the image acquisition unit 102 installed in the intelligent robot.
  • the cameras as the image acquisition unit 102 are respectively placed at the eyes of the intelligent robot in the form of a penguin.
  • a face image having the recognized feature point is acquired from the externally input scene image.
  • the face image recognition unit 106 acquires a face image having the recognition feature point from the scene image.
  • the face image having the identification feature point is transmitted to the cloud server 200.
  • Cloud-based software services and cloud face storage are used to realize cloud face recognition and ensure that face images are recognized to the maximum extent.
  • many Internet companies currently provide cloud software function services such as online face recognition, and access to the APIs provided by these companies can obtain corresponding services.
  • step 1140 it is determined whether the face image is capable of cloud recognition processing.
  • the cloud face recognition result of step 1130 is determined. If the recognition is successful, the process proceeds to step 1170, otherwise step 1150 is performed to perform local image recognition processing.
  • a local image recognition process is performed.
  • the local image recognition process is a supplement to the cloud recognition. After the cloud recognition fails, the local image recognition process is started, and the face image having the recognized feature points is transmitted to the local image recognition processing unit 106.
  • the local image recognition processing unit 106 performs local image recognition on the face image having the recognized feature point based on the preset image data and generates a local image recognition result.
  • step 1160 it is determined whether the face image can be recognized. In this step, if the image If the recognition process is successful, then proceed to step 1170. If the image recognition processing fails, no operation is performed.
  • step 1170 the recognition result is saved.
  • the recognition results can be used by controller 108 along with other results.
  • FIG. 12 is a flowchart of a face recognition method according to another embodiment of the present invention. As shown in FIG. 12, the face recognition method includes steps 1210-1270.
  • the face recognition method shown in FIG. 12 and the face recognition method shown in FIG. 11 differ only in the execution order.
  • the local image is first performed.
  • Recognition processing, and then cloud face recognition processing Figure 11 is the opposite. Only steps 1230-1250 that differ from FIG. 11 are described herein.
  • step 1230 local image recognition processing is performed.
  • the local image recognition processing is to transmit the face image having the recognition feature point to the local image recognition processing unit 106.
  • the local image recognition processing unit 106 performs local image recognition on the face image having the recognized feature point based on the preset image data and generates a local image recognition result.
  • step 1240 it is determined whether the face image can be recognized. In this step, if the image recognition processing is successful, then proceed to step 1270. If the image recognition processing fails, step 1250 is executed to perform cloud face recognition processing.
  • the face image with the identified feature points is transmitted to the cloud server 200.
  • Cloud-based software services and cloud face storage are used to realize cloud face recognition and ensure that face images are recognized to the maximum extent.
  • many Internet companies currently provide cloud software function services such as online face recognition, and access to the APIs provided by these companies can obtain corresponding services.
  • step 1260 it is determined whether the face image is capable of cloud recognition processing.
  • the cloud face recognition result of step 1250 is determined. If the recognition is successful, the process proceeds to step 1270, otherwise no operation is performed.
  • step 1270 the recognition result is saved.
  • the recognition results can be used by controller 108 along with other results.
  • FIG. 13 is a flowchart of a face recognition method according to another embodiment of the present invention.
  • the face recognition method includes steps 1310-1380.
  • step 1350 is added to "determine the cloud network status".
  • the cloud network is normal, the face image is submitted to the cloud server for identification processing. This implementation is to improve the efficiency of cloud recognition and reduce network latency.
  • the recognition execution priority may also be determined based on a predefined preference policy. For example, it is possible to determine which face images are first sent to the cloud server for processing and which must be processed locally by means of fuzzy matching. For another example, the processing priority may be determined by enumeration, and the locally processed image information is relatively limited, and the face images not in the range are sent to the cloud server for processing.
  • the pressure signal acquisition unit 103 is for sensing an external pressure signal of the surface of the intelligent robot.
  • the pressure signal acquisition unit unit 103 typically includes a thin film pressure sensor sheet array and an analog to digital (A/D) conversion circuit.
  • the membrane pressure sensor can be distributed in the area of the front chest, forelegs, head and back of the intelligent robot.
  • the film pressure sensor of this solution has adhesive on the back and is directly attached to a certain part of the body of the intelligent robot.
  • a long strip sensor can be mounted on the back, front chest, abdomen and/or forelegs of the intelligent robot to sense the force state in the strip area.
  • a square sensor is mounted on the head of the intelligent robot to sense the force state in the block area.
  • the film pressure sensor in the present embodiment is preferably a resistive pressure sensor.
  • the pressure sensor patch array is used to acquire an external pressure signal and transmit the pressure signal to an analog to digital conversion circuit.
  • the pressure sensor sheet array can use an ultra-thin resistive pressure sensor as an external force detecting device, and the sensor converts the pressure applied in the thin film region into a change in the resistance value, thereby obtaining a signal corresponding to the pressure information.
  • the larger the external pressure, the lower the resistance value, the change of the resistance value changed by the external pressure is converted into the voltage or current change by the internal circuit of the sensor, and the value of the voltage or current is converted into an analog signal output to the analog-to-digital conversion circuit. .
  • the analog to digital conversion circuit converts the external pressure signal into a digital signal and transmits it to the controller 108.
  • the controller 108 can pass these signals to the pressure signal recognition processing unit 107 for processing.
  • the pressure signal acquisition unit 103 may directly connect the pressure signal recognition processing unit 107 to directly transmit its signal to the pressure signal recognition processing unit 107.
  • the pressure signal recognition processing unit 107 is for acquiring an external pressure signal and processing it to generate a pressure-sensing emotion signal.
  • 14 is a block diagram showing the structure of a pressure signal recognition processing unit of the multi-sense type intelligent robot shown in FIG. 1.
  • the pressure signal recognition processing unit 107 includes a pressure type determination unit 205, a pressure position determination unit 206, and a pressure sensing type. Emotion signal generating unit 207, data storage unit 208.
  • the pressure type determining unit 205 is configured to calculate a time value and a pressure change rate of the external pressure signal, and determine the type of the external pressure signal according to the pressure change rate and the preset change threshold.
  • the smart robot 100 can be normally in a standby or hibernation state, waiting for the user's touch to start. Such as starting a smart machine
  • the instructions of the person 100 can be included in the pressure signal.
  • the pressure signal recognition processing unit 107 is for identifying a start command in the pressure signal.
  • the pressure position determining unit 206 is configured to determine a pressure generating position based on an external pressure signal.
  • the pressure-aware emotion signal generating unit 207 is configured to compare the pressure generating position and the type of the external pressure signal with a preset mapping list, and generate a pressure-sensing emotion signal corresponding to the pressure generating position and the type of the external pressure signal.
  • the controller 108 compares the received pressure-aware emotion signal with a preset mapping list to generate an emotion expression part and an emotion expression instruction corresponding to the pressure-aware emotion signal.
  • the emotion expression instructions herein are used to control the execution of corresponding mechanical actions, play corresponding prompt voices, and/or display corresponding prompt information.
  • the intelligent robot 100 further includes a data storage unit 208 connected to the pressure type determining unit 205 and the pressure sensing type emotion signal generating unit 207, respectively, for storing a preset change threshold. And a list of preset mappings.
  • FIG. 7 is a schematic view of the use of a pressure sensor.
  • the pressure sensor 700 includes a pressure sensitive layer 703 and an adhesive layer 702 through which the pressure sensor can be attached at any position of the smart robot housing 701.
  • the size and area of the pressure sensor can also be adjusted according to actual needs.
  • FIG. 8 is a flowchart of a haptic sensing method according to an embodiment of the present invention.
  • the haptic sensing method of the present embodiment includes steps 801-806.
  • step 801 an external pressure signal is acquired to convert the external pressure signal into a digital signal.
  • the sensing component is attached to each part of the body of the intelligent robot for acquiring the pressure signal at each part.
  • the acquired pressure signal is converted into a digital signal for subsequent processing.
  • a time value for the duration of the external pressure signal is calculated, and a rate of pressure change is calculated based on the time value and the digital signal.
  • a preset time period of 0.5-1.5 seconds is selected, and the change of the pressure signal (ie, the applied external force change) in the time period is calculated, and the two are The ratio of the difference to the time period is taken as the rate of change of pressure.
  • a period of 0.5-1.5 seconds is sufficient for the sensor to capture an accurate change in applied force to capture changes in the digital signal.
  • the applied external force at 1 second is 100 Newtons
  • the applied area is 0.026 square meters, passing 100/0.026 ⁇ 3846 Newtons per square meter, and 3846 Newtons per square meter is the value characterized by the rate of change of pressure.
  • step 803 the pressure change rate is compared to a preset first change threshold.
  • the pressure change rate is compared with a preset first change threshold, and the type of the external pressure signal is determined according to the comparison result.
  • step 804 it is determined that the type of the external pressure signal is determined to be tapping.
  • step 805 it is determined that the type of the external pressure signal is determined to be a stroke.
  • the pressure change rate in the above example is 3,846 Newtons per square meter. If the preset change threshold is greater than the value, it can be determined as a tap, otherwise it is a stroke.
  • step 806 the pressure generation position and the type of the external pressure signal are compared with a preset mapping list, and an emotion expression part and an emotion expression instruction corresponding to the type of the pressure generation position and the external pressure signal are generated, thereby triggering the emotion. expression.
  • the preset mapping list stores the mapping relationship between the pressure generating position, the type of the external pressure signal, and the robot feedback.
  • the mapping relationship is as shown in Table 1 below:
  • the emotion expression part and the emotion expression instruction are generated according to the pressure generation position and the type of the external pressure signal.
  • Emotional expression instructions are used to characterize robot feedback types, such as robot feedback in the above table.
  • the robot's executive mechanism can be triggered to perform certain actions and expressions, thereby expressing some anthropomorphic emotions such as happiness, anger, depression, and the like.
  • the actuator of the emotional expression may include various parts of the robot body, speakers mounted on the body of the robot, a display, and the like. For example, the action of dancing and dancing is performed by hands and feet, or the corresponding sounds are played by the sound synthesizing device and the speaker, or some emoticons, prompts, etc. are displayed through the display, or feedback is combined in several ways.
  • the intelligent robot can make different feedback according to different parts and the type of external force applied thereon, so that the intelligent robot is more anthropomorphic.
  • Step 9 is a flow chart of a haptic sensing method according to another embodiment of the present invention.
  • the haptic sensing method includes steps 901-907. Steps 901-902 are the same as steps 801-802 of FIG. 8, and are not described herein again.
  • step 903 the pressure change rate is compared with a preset first change threshold and a second change threshold. In this step, the pressure change rate and the first change threshold and the second change threshold are respectively compared. If the pressure signal change is greater than the second change threshold, step 904 is performed, if the pressure change rate is greater than the first change threshold and less than or equal to the second change If the threshold is reached, step 905 is performed, otherwise step 906 is performed.
  • steps 904, 905, 906 it is determined that the type of the external pressure signal is force tapping, tapping and touching, respectively.
  • Table 2 below is a new mapping table.
  • step 907 the type of the pressure generating position and the external pressure signal are compared with the preset mapping list, and an emotion expression part and an emotion expression instruction corresponding to the type of the pressure generating position and the external pressure signal are generated.
  • FIG. 9 the description of the second change threshold is added, so that the tapping is divided into hard tapping and slight tapping, which increases the diversity of intelligent robot processing and feedback, making it more anthropomorphic.
  • FIG. 8 and FIG. 9 are merely exemplary descriptions of the haptic sensing method of the present invention, and the types of pressure types should not be limited to the three types mentioned above, all passing signal change rates. The type of pressure determined in comparison with the preset change threshold should be within the scope of the present invention.
  • the present invention emphasizes that an emotional expression part and an emotional expression instruction are generated by a combination of a pressure type and a pressure position, wherein the emotional expression part and the emotional expression instruction are used to trigger various forms of emotional expression, and various pressure positions and pressure types can be defined.
  • the mapping relationship with the control signals (as in the above table), all of these definitions and implementations should be included in the scope of the present invention.
  • a person skilled in the art can make some reasonable modifications in the spirit of the present invention, and such modifications are also included in the scope of the present invention.
  • the pressure is transmitted to the tactile sensing units of the robot through the pressure sensing unit attached to various parts of the intelligent robot body, and each of the tactile senses is sensed by the tactile sense.
  • an emotion expression part and an emotion expression instruction are generated, and the control signal is used to drive the robot to make various emotion expressions.
  • a plurality of actuators 112 such as motors, speakers, displays, etc., are mounted on the body of the robot, such as hands, feet, chest, back, head, etc., and these components are electrically connected to the controller 108 and in accordance with the received emotions.
  • the expression site and the emotional expression instruction make corresponding emotional expressions.
  • the motion sensing unit 104 is configured to sense a motion state of the smart robot 100 to generate a motion state parameter.
  • Examples of the motion sensing unit 104 include a gravity acceleration sensor, a gyroscope, or a tilt sensor mounted on the robot's torso to measure data of acceleration and angular velocity during robot motion in real time.
  • the data of the motion sensing unit 104 is output to the controller 108.
  • the controller 108 acquires real-time data of the motion parameters through the motion sensing unit 104, and adjusts the motion by the adjustment algorithm.
  • the controller 108 senses the motion parameter such as the acceleration of the gravity sensor as the feedback, or uses the gyroscope or the tilt sensor mounted on the torso of the robot to sense the motion state of the robot, and the like as feedback.
  • the pattern recognition algorithm recognizes the current motion state and adjusts the motion through feedback to ensure the stability of the motion.
  • the controller 108 calculates the inclination of the robot through the motion sensing unit 104, and the simulation identifies whether it is in a state to fall; if it is close to the boundary of the fall, the joint is adjusted by feedback to avoid the occurrence of a fall.
  • the controller 108 is connected to the voice collection unit 101, the image acquisition unit 102, the pressure signal acquisition unit 103, the motion sensing unit 104, the password recognition processing unit 105, the local image recognition processing unit 106, the pressure signal recognition processing unit 107, and the identification selection unit 109. , cloud identification unit 110 and actuator 112.
  • the controller 108 can acquire a voice signal, a scene image or a face image, an external pressure signal, and a motion state parameter for controlling the overall operation of the robot.
  • the controller 108 can instruct the voice collection unit 101, the image acquisition unit 102, the pressure signal acquisition unit 103, the motion sensing unit 104 to capture external information, or the command password recognition processing unit 105, the local image recognition processing unit 106, and the pressure signal recognition processing.
  • Unit 107 begins to work to obtain the desired password recognition result, local image recognition result, pressure-aware emotion signal, and the like.
  • the controller 108 instructs the cloud identification unit 110 and the cloud server 200 to communicate to transmit data that needs further processing to the cloud server 200, and obtains from the cloud server 200.
  • Results such as cloud speech recognition results, cloud face recognition results.
  • Controller 108 can command, for example, actuator 112 to perform the corresponding action.
  • the controller 108 may make an intelligent robot according to a combination of at least one of the password recognition processing result and the cloud speech recognition processing result, at least one of the local image recognition result and the cloud face recognition result, and one or more of the pressure-sensing emotion signals.
  • An interactive decision of 100, and the controller 108 optionally adjusts the motion of the intelligent robot 100 based on the motion state parameters.
  • the smart robot 100 can be normally in a standby or hibernation state, waiting for the user's activation (eg, vocal call or tap waking).
  • the voice collection unit 101 collects the voice signal and transmits it to the controller 108.
  • the controller 108 sends it to the password recognition processing unit 105, it can recognize the start command in the voice signal, in response to the start command. This turns on the robot to start working.
  • the controller 108 responds to the pressure signal, thereby turning on the smart robot 100 to start working. It will of course be understood that the controller 108 can turn on the smart robot 100 under other conditions. For example, the controller 108 turns on the smart robot 100 in response to a user's switch button.
  • the real-time data about the motion, voice, and the like of the smart robot 100 obtained by each sensing device can also be transmitted to the cloud server 200 through the cloud recognition unit 110, thereby implementing the cloud server 200 to monitor the operation of the smart robot 100.
  • the cloud identification unit 110 and the cloud server 200 Through the connection of the cloud identification unit 110 and the cloud server 200, data is transmitted to the cloud for processing, and the real-time processing capability of the system can be improved.
  • the controller 108 is the core of the intelligent robot 100, and is mainly responsible for collecting signals and data of each sensing device, and analyzing and processing the signals and data, thereby performing interaction and motion decision.
  • the controller 108 can be internally configured with an impact model as shown in FIG. 10, and the input parameters are at least one of a password recognition processing result and a cloud speech recognition processing result, at least one of a local image recognition result and a cloud face recognition result, and a pressure-sensing emotion signal.
  • the impact model can make an interactive decision based on this, and the command executing agency 112 interacts to achieve interaction with the outside world.
  • the impact model can be a training model built on an artificial intelligence algorithm.
  • the training model can obtain the algorithm parameters of the training model according to the artificial intelligence algorithm, and the actual input parameters and the output parameters that the developer desires corresponding to the actual input parameters are used as training.
  • the controller 108 is capable of recognizing one of the processing result and the cloud speech recognition processing result, one of the local image recognition result and the cloud face recognition result, and the feeling of stress Obtaining emotional information of the user in the cognitive emotion signal and the motion state parameter, and determining the emotion type of the intelligent robot according to the emotional information of the user, and then determining the emotion of the intelligent robot corresponding to the emotion type according to the mapping list pre-stored in the impact model
  • the expression part and the emotion expression instruction and finally control the emotion expression part to execute the emotion expression instruction.
  • the facial expression of the user is determined based on the facial image of the user, and the emotional information of the user is determined according to the facial expression of the user. For example, when the facial expression of the user is a smile, the emotional information of the user is happy, and the emotional type of the intelligent robot determined according to the emotional information of the user is hi.
  • the volume and sound frequency of the user are acquired by the voice collecting unit 101, and the emotion information of the user is determined according to the volume and sound frequency of the user. For example, when the volume of the user is less than the first preset value, and the voice frequency of the user is less than the second preset value, it is determined that the emotional information of the user is sad, and the emotional type of the intelligent robot determined according to the emotional information of the user is sad.
  • the emotion information of the user is acquired by the pressure signal acquisition unit 103 and/or the motion sensing unit 104, and the emotion type of the intelligent robot is determined according to the emotion information of the user. For example, when the pressure signal acquisition unit 103 detects that the user embraces the smart robot, it is determined that the emotion type of the smart robot is hi; and, for example, when the pressure signal acquisition unit 103 and the motion sensing unit 104 detect that the user shakes the smart robot 100 with force, Determine the emotional type of the intelligent robot as anger.
  • the type of emotion includes joy, anger, sadness and/or music.
  • one type of emotion corresponds to at least one part of the emotional expression.
  • the emotion expression instruction corresponds to the emotion expression part; the emotion expression instruction is an action instruction and/or a facial expression instruction.
  • the emotional expression site includes a forelimb, a hind limb, a torso, a head and/or a face
  • the hind limb includes a leg and a foot.
  • the emotion expression part corresponding to the emotion type is a forelimb
  • the emotion expression instruction corresponding to the emotion type is a fore limb swinging up and down
  • the facial expression expression may also be performed at the same time, such as facial expression joy.
  • the emotion expression parts corresponding to the emotion type are a forelimb, a trunk, a right leg, and a right foot
  • the emotion expression instruction corresponding to the emotion type is that the forelimb is unfolded.
  • the torso is tilted slightly to the left, the right leg swings back and forth, and the right foot is stomped, and facial expressions can also be expressed at the same time, such as facial expressions showing anger.
  • the emotion expression part corresponding to the emotion type is a head
  • the emotion expression instruction corresponding to the emotion type is that the head is turned to the shoulder position, and the head is lowered
  • the facial expression can also be performed simultaneously. Expressions such as facial expressions with sad expressions.
  • the emotion expression part corresponding to the emotion type is a forelimb and a torso
  • the emotion expression instruction corresponding to the emotion type is that the forelimb swings up and down and the trunk swings left and right, and At the same time, facial expressions are expressed, such as a happy expression on the face.
  • the action command includes action type information, action amplitude information, action frequency information, and/or action duration information corresponding to the emotion expression portion.
  • the corresponding emotional expression part is the forelimb
  • the corresponding action type information is up and down swing
  • the action amplitude information refers to the amplitude of the fore limb swinging up and down
  • the action frequency information refers to the fore limb swinging up and down.
  • the frequency for example, once per second
  • the action duration information refers to the total length of time during which the forelimbs are controlled to swing up and down.
  • the type of emotion expression instruction can be sound.
  • the audio information corresponding to the emotion type is the joyous call; the audio information corresponding to the emotion type is the angry call; the audio type of the emotion type is the sad call; the emotion type is the audio corresponding to the music.
  • Information is the call of joy.
  • the intelligent robot actively acquires the emotional information of the user, determines the emotional type of the intelligent robot according to the emotional information of the user, and determines the emotional expression part and the emotional expression of the intelligent robot corresponding to the emotional type of the intelligent robot according to the pre-stored mapping list. Directing, controlling the emotional expression part to execute the emotional expression instruction, thereby actively sensing the emotional change of the external user and determining the emotional type of the intelligent robot through the emotional information of the user, and expressing the emotion of the intelligent robot through the physical motion of the intelligent robot, thereby The interaction between the intelligent robot and the user is improved, the emotional expression of the intelligent robot is improved, the interest is enhanced, and the user experience is improved.
  • Another part of the interactive decision is to perform actions based on the password.
  • the intelligent robot will walk in the direction of the user. Or when the user instructs the intelligent robot to sit down, shake his head, etc., the intelligent robot responds.
  • FIG. 3 is a flow chart showing a method for perceptual interaction according to an embodiment of the present invention. The method can be performed in the system shown in Figures 1 and 2, or in other systems.
  • a method for sensing interaction of an intelligent robot according to this embodiment includes the following steps:
  • step 301 speech recognition is performed.
  • step 302 face recognition is performed.
  • the externally input scene image is processed to generate a local image recognition result, or the externally input scene image is transmitted to the cloud server for face recognition and receives the cloud face recognition result returned by the cloud server.
  • an external pressure signal is identified and a pressure-aware emotional signal is generated.
  • determining the type of the external pressure signal and the pressure generating position calculating the duration value and the pressure change rate of the external pressure signal, determining the type of the external pressure signal according to the ratio of the pressure change rate and the preset change threshold value, and according to The external pressure signal determines a pressure generating position; the pressure generating position and the type of the external pressure signal are compared with a preset mapping list, and a pressure sensing type emotion signal corresponding to the type of the pressure generating position and the external pressure signal is generated.
  • step 304 an interactive decision of the intelligent robot is made according to at least one of the password recognition processing result and the cloud speech recognition processing result, at least one of the local image recognition result and the cloud face recognition result, and the pressure-aware emotion signal, thereby triggering the interactive decision.
  • the intelligent robot and the sensing interaction method thereof improve the interaction ability of the robot by configuring a plurality of sensing devices, comprehensively acquiring environmental signals and performing interactive decision making.
  • the cloud recognition unit communicates with external processing resources, which improves the processing power of the robot and makes more complex interactive decisions possible.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)

Abstract

L'invention porte également sur un système interactif en nuage, sur un robot intelligent multicognitif (100) de celui-ci et sur un procédé d'interaction cognitive associé. Le robot intelligent (100) possède divers dispositifs cognitifs (101, 102 103 et 104) pour la détection de la voix, de l'image, du stress et du mouvement et est doté de capacités de reconnaissance vocale, de reconnaissance faciale, de détection du stress et de reconnaissance des émotions. Le robot intelligent effectue des décisions interactives au moyen d'une combinaison de divers résultats de reconnaissance de façon à répondre à des comportements humains. Le robot intelligent est également capable de communiquer avec une ressource de traitement externe par l'intermédiaire d'une unité de reconnaissance de nuages (110), améliorant ainsi la capacité de traitement du robot, et rendant possible une prise de décision interactive de complexité accrue.
PCT/CN2017/076274 2016-06-15 2017-03-10 Système interactif en nuage, robot intelligent multicognitif, et procédé d'interaction cognitive associés WO2017215297A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610422085.1A CN107511832A (zh) 2016-06-15 2016-06-15 云端互动系统及其多感知型智能机器人和感知互动方法
CN201610422085.1 2016-06-15

Publications (1)

Publication Number Publication Date
WO2017215297A1 true WO2017215297A1 (fr) 2017-12-21

Family

ID=60663695

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/076274 WO2017215297A1 (fr) 2016-06-15 2017-03-10 Système interactif en nuage, robot intelligent multicognitif, et procédé d'interaction cognitive associés

Country Status (2)

Country Link
CN (1) CN107511832A (fr)
WO (1) WO2017215297A1 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110096996A (zh) * 2019-04-28 2019-08-06 深圳前海达闼云端智能科技有限公司 生物信息识别方法、装置、终端、系统及存储介质
CN110363278A (zh) * 2019-07-23 2019-10-22 广东小天才科技有限公司 一种亲子互动方法、机器人、服务器及亲子互动系统
WO2020133405A1 (fr) * 2018-12-29 2020-07-02 深圳市大疆创新科技有限公司 Procédé et dispositif de commande d'un robot à télécommande au sol
CN111931484A (zh) * 2020-07-31 2020-11-13 于梦丽 一种基于大数据的数据传输方法
CN113119138A (zh) * 2021-04-16 2021-07-16 中国科学技术大学 基于物联网的助盲机器人系统及方法
CN113524212A (zh) * 2021-06-29 2021-10-22 智动时代(北京)科技有限公司 一种三体机器人组成方法
CN114260919A (zh) * 2022-01-18 2022-04-01 华中科技大学同济医学院附属协和医院 一种智能机器人
CN115795278A (zh) * 2022-12-02 2023-03-14 广东元一科技实业有限公司 一种智能铺布机控制方法、装置及电子设备

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108687768B (zh) * 2018-04-02 2022-08-05 深圳臻迪信息技术有限公司 一种涉水机器人以及涉水机器人信息输入方法
CN110405777B (zh) * 2018-04-28 2023-03-31 深圳果力智能科技有限公司 一种机器人的交互控制方法
CN109036392A (zh) * 2018-05-31 2018-12-18 芜湖星途机器人科技有限公司 机器人交互系统
CN108839036A (zh) * 2018-07-05 2018-11-20 四川长虹电器股份有限公司 家庭智能健康监护机器人
CN110852133A (zh) * 2018-07-27 2020-02-28 宝时得科技（中国）有限公司 自动行走设备、及其控制方法、控制装置及计算机设备
CN110969051A (zh) * 2018-09-29 2020-04-07 上海小蚁科技有限公司 基于图像传感器系统的人脸识别方法及图像传感器系统
CN109108984B (zh) * 2018-10-22 2021-03-26 杭州任你说智能科技有限公司 一种实体机器人接入云端语音平台的方法及实体机器人
CN109571494A (zh) * 2018-11-23 2019-04-05 北京工业大学 情绪识别方法、装置及宠物机器人
CN109623837A (zh) * 2018-12-20 2019-04-16 北京子歌人工智能科技有限公司 一种基于人工智能的伴侣机器人
CN109605373A (zh) * 2018-12-21 2019-04-12 重庆大学 基于机器人的语音交互方法
CN111604920B (zh) * 2020-06-02 2022-06-07 南京励智心理大数据产业研究院有限公司 一种基于素质教育的陪伴成长机器人
CN112829754B (zh) * 2021-01-21 2023-07-25 合众新能源汽车股份有限公司 车载智能机器人和车载智能机器人的运行方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102778879A (zh) * 2012-07-30 2012-11-14 西安润基投资控股有限公司 一种家居三元智能化系统及实现家居智能化控制的方法
CN104268684A (zh) * 2014-09-23 2015-01-07 桂林驰讯科技有限公司 基于智能终端的智能化巡查系统
CN105184058A (zh) * 2015-08-17 2015-12-23 李泉生 一种悄悄话机器人
CN105652875A (zh) * 2016-03-29 2016-06-08 苏州倍特罗智能科技有限公司 一种用于病房的带有识别模块机器人系统
CN205989331U (zh) * 2016-06-15 2017-03-01 深圳光启合众科技有限公司 云端互动系统及其多感知型智能机器人

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102778879A (zh) * 2012-07-30 2012-11-14 西安润基投资控股有限公司 一种家居三元智能化系统及实现家居智能化控制的方法
CN104268684A (zh) * 2014-09-23 2015-01-07 桂林驰讯科技有限公司 基于智能终端的智能化巡查系统
CN105184058A (zh) * 2015-08-17 2015-12-23 李泉生 一种悄悄话机器人
CN105652875A (zh) * 2016-03-29 2016-06-08 苏州倍特罗智能科技有限公司 一种用于病房的带有识别模块机器人系统
CN205989331U (zh) * 2016-06-15 2017-03-01 深圳光启合众科技有限公司 云端互动系统及其多感知型智能机器人

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020133405A1 (fr) * 2018-12-29 2020-07-02 深圳市大疆创新科技有限公司 Procédé et dispositif de commande d'un robot à télécommande au sol
CN110096996A (zh) * 2019-04-28 2019-08-06 深圳前海达闼云端智能科技有限公司 生物信息识别方法、装置、终端、系统及存储介质
CN110096996B (zh) * 2019-04-28 2021-10-22 达闼机器人有限公司 生物信息识别方法、装置、终端、系统及存储介质
CN110363278A (zh) * 2019-07-23 2019-10-22 广东小天才科技有限公司 一种亲子互动方法、机器人、服务器及亲子互动系统
CN111931484A (zh) * 2020-07-31 2020-11-13 于梦丽 一种基于大数据的数据传输方法
CN113119138A (zh) * 2021-04-16 2021-07-16 中国科学技术大学 基于物联网的助盲机器人系统及方法
CN113524212A (zh) * 2021-06-29 2021-10-22 智动时代(北京)科技有限公司 一种三体机器人组成方法
CN114260919A (zh) * 2022-01-18 2022-04-01 华中科技大学同济医学院附属协和医院 一种智能机器人
CN114260919B (zh) * 2022-01-18 2023-08-29 华中科技大学同济医学院附属协和医院 一种智能机器人
CN115795278A (zh) * 2022-12-02 2023-03-14 广东元一科技实业有限公司 一种智能铺布机控制方法、装置及电子设备
CN115795278B (zh) * 2022-12-02 2023-08-04 广东元一科技实业有限公司 一种智能铺布机控制方法、装置及电子设备

Also Published As

Publication number Publication date
CN107511832A (zh) 2017-12-26

Similar Documents

Publication Publication Date Title
WO2017215297A1 (fr) Système interactif en nuage, robot intelligent multicognitif, et procédé d'interaction cognitive associés
JP6816925B2 (ja) 育児ロボットのデータ処理方法及び装置
CN205989331U (zh) 云端互动系统及其多感知型智能机器人
EP3624442A1 (fr) Robot et son procédé de fonctionnement
US11948241B2 (en) Robot and method for operating same
EP3623118A1 (fr) Dispositif de reconnaissance d'émotions, robot l'incluant et serveur l'utilisant
CN111163906B (zh) 能够移动的电子设备及其操作方法
US20180257236A1 (en) Apparatus, robot, method and recording medium having program recorded thereon
JP2018072876A (ja) 感情推定システム、感情推定モデル生成システム
WO2002099545A1 (fr) Procede de commande d'unite d'interface homme-machine, dispositif robotise et procede de commande associe
CN111002303B (zh) 识别装置、机器人、识别方法和存储介质
US20180376069A1 (en) Erroneous operation-preventable robot, robot control method, and recording medium
CN110737335B (zh) 机器人的交互方法、装置、电子设备及存储介质
JP6891601B2 (ja) ロボットの制御プログラム、ロボット装置、及びロボットの制御方法
KR20220130000A (ko) 인공지능 아바타에 기초한 인터랙션 서비스 방법 및 장치
WO2016206644A1 (fr) Moteur et système de commande de robot
US11938625B2 (en) Information processing apparatus, information processing method, and program
JP2018075657A (ja) 生成プログラム、生成装置、制御プログラム、制御方法、ロボット装置及び通話システム
KR102519599B1 (ko) 멀티모달 기반의 인터랙션 로봇, 및 그 제어 방법
KR20200101221A (ko) 사용자 입력 처리 방법 및 이를 지원하는 전자 장치
JP7414735B2 (ja) 複数のロボットエフェクターを制御するための方法
JP7024754B2 (ja) 制御装置、ロボット、制御方法およびプログラム
US20210201139A1 (en) Device and method for measuring a characteristic of an interaction between a user and an interaction device
JP4735965B2 (ja) 遠隔地間対話システム
CN205334503U (zh) 具有人脸识别功能的机器人

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17812412

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17812412

Country of ref document: EP

Kind code of ref document: A1