CN113793605A

CN113793605A - Autonomous mobile device voice control method, device, equipment and readable storage medium

Info

Publication number: CN113793605A
Application number: CN202110940990.7A
Authority: CN
Inventors: 陈茂勇; 丁民权; 刘阳; 林兴宝; 岳鹏飞; 于昊田; 高超; 李大亮
Original assignee: Ecovacs Robotics Suzhou Co Ltd
Current assignee: Ecovacs Robotics Suzhou Co Ltd
Priority date: 2021-08-17
Filing date: 2021-08-17
Publication date: 2021-12-14

Abstract

After the autonomous mobile equipment collects a second voice signal, at least two working areas are determined according to the second voice signal, and tasks indicated by the second voice signal are executed to the working areas in sequence. By adopting the scheme, a plurality of working areas can be indicated to the autonomous mobile equipment through one voice signal, and a user interacts with the autonomous mobile equipment through natural language so as to realize extremely simple control on the autonomous mobile equipment, so that the accuracy is high and the process is simple.

Description

Autonomous mobile device voice control method, device, equipment and readable storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for voice control of an autonomous mobile device.

Background

With the development of artificial intelligence technology, various autonomous mobile devices increasingly enter people's lives, such as logistics robots, floor sweeping robots, mowing robots, greeting robots and the like.

The conventional autonomous mobile device is controlled by a mobile phone Application (APP), a physical key arranged on the autonomous mobile device, a remote controller, and the like. When the APP is used for controlling the autonomous mobile equipment, the APP needs to be opened, the process is complicated, and the difficulty is relatively high for the old; the physical keys can only simply control the self-owned mobile equipment, such as starting, stopping and the like; the remote control is easily lost, and when the user cannot find the remote control, the autonomous mobile device cannot be controlled.

Therefore, how to conveniently and quickly control the autonomous mobile device is regarded as a problem to be solved urgently.

Disclosure of Invention

The application provides a voice control method, a voice control device, equipment and a readable storage medium for autonomous mobile equipment, wherein a plurality of working areas are indicated to the autonomous mobile equipment through a voice signal, so that the autonomous mobile equipment can execute tasks on the plurality of working areas in sequence, the accuracy is high, and the process is simple.

In a first aspect, an embodiment of the present application provides an autonomous mobile device voice control method, which is applied to an autonomous mobile device, and the method includes:

collecting a first voice signal;

when the first voice signal is matched with a wake-up instruction of the autonomous mobile device, waking up a voice control function of the autonomous mobile device;

collecting a second voice signal in the voice control function awakening state;

determining at least two working areas according to the second voice signal;

and executing the tasks indicated by the second voice signals for the at least two working areas in sequence.

In a second aspect, an embodiment of the present application provides an autonomous mobile device voice control method, which is applied to an autonomous mobile device, and the method includes:

collecting a second voice signal;

determining a working area according to the second voice signal;

determining a driving path according to the current position and the working area;

closing the working module and advancing to the working area according to the driving path;

and if the mobile terminal moves to the working area, starting the working module to execute the task indicated by the second voice signal.

In a third aspect, an embodiment of the present application provides an autonomous mobile device control apparatus, including:

the acquisition module is used for acquiring a first voice signal;

a processing module for waking up a voice control function of the autonomous mobile device when the first voice signal matches a wake-up instruction of the autonomous mobile device;

the acquisition module is also used for acquiring a second voice signal in the voice control function awakening state;

the processing module is further configured to determine at least two working areas according to the second voice signal;

and the execution module is used for sequentially executing the tasks indicated by the second voice signals to the at least two working areas.

In a fourth aspect, an embodiment of the present application provides an autonomous mobile device control apparatus, including:

the acquisition module is used for acquiring a second voice signal;

the processing module is used for determining a working area according to the second voice signal; determining a driving path according to the current position and the working area;

the driving module is used for closing the working module and driving the working module to the working area according to the driving path;

and the execution module is used for starting the working module to execute the task indicated by the second voice signal if the mobile terminal moves to the working area.

In a fifth aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory and a computer program stored on the memory and executable on the processor, when executing the computer program, causing the electronic device to carry out the method according to the first aspect or the various possible implementation manners of the first aspect.

In a sixth aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory and a computer program stored on the memory and executable on the processor, when executing the computer program, causing the electronic device to implement the method as described above for the second aspect or various possible implementation manners of the second aspect.

In a seventh aspect, this application embodiment provides a computer-readable storage medium, in which computer instructions are stored, and when executed by a processor, the computer instructions are configured to implement the method according to the first aspect or each possible implementation manner of the first aspect.

In an eighth aspect, embodiments of the present application provide a computer-readable storage medium, in which computer instructions are stored, and when executed by a processor, the computer instructions are used to implement the method according to the second aspect or various possible implementation manners of the second aspect.

In a ninth aspect, embodiments of the present application provide a computer program product comprising a computer program, which when executed by a processor, implements the method according to the first aspect or the various possible implementations of the first aspect.

In a tenth aspect, embodiments of the present application provide a computer program product including a computer program, which when executed by a processor implements the method according to the second aspect or various possible implementations of the second aspect.

According to the voice control method, the voice control device, the voice control equipment and the readable storage medium of the autonomous mobile equipment, after the autonomous mobile equipment collects the second voice signal, at least two working areas are determined according to the second voice signal, and tasks indicated by the second voice signal are executed on the working areas in sequence. By adopting the scheme, the plurality of working areas can be indicated to the autonomous mobile equipment through one voice signal, so that the autonomous mobile equipment can execute tasks on the plurality of working areas in sequence, and a user interacts with the autonomous mobile equipment through natural language so as to realize extremely simple control on the autonomous mobile equipment, and the autonomous mobile equipment has high accuracy and simple process.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1A is a schematic diagram of an implementation environment of an autonomous mobile device voice control method provided in an embodiment of the present application;

fig. 1B is a schematic structural diagram of a sweeping robot provided in the embodiment of the present application;

fig. 1C is a schematic structural view of a sound signal collecting apparatus of the autonomous mobile device;

fig. 1D is a schematic structural diagram of another embodiment of an autonomous mobile device provided by an embodiment of the present application;

FIG. 1E is a flow chart of voice control of an autonomous mobile device according to an embodiment of the present application;

fig. 1F is another voice control flow diagram of an autonomous mobile device provided by an embodiment of the present application;

fig. 1G is another voice control flow diagram of an autonomous mobile device provided by an embodiment of the present application;

fig. 2 is a flow chart of an autonomous mobile device voice control method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a speech recognition process in an embodiment of the present application;

FIG. 4 is a schematic view of a carpet area;

FIG. 5 is a schematic view of a furniture identification process;

FIG. 6 is a schematic view of a process for identifying a door;

FIG. 7 is a schematic diagram of an autonomous mobile device and voice recognition server synchronization process;

FIG. 8A is a schematic diagram of determining the location of a sound source relative to the center of a microphone array;

fig. 8B is a schematic diagram of an identified microphone array and autonomous mobile device body;

FIG. 8C is a schematic diagram of a process for training a speech recognition model and recognizing speech;

FIG. 9 is a flow chart of autonomous mobile device voice control logic as provided by an embodiment of the present application;

fig. 10 is another flowchart of an autonomous mobile device voice control method provided by an embodiment of the present application;

FIG. 11 is a further flowchart of an autonomous mobile device voice control method provided by an embodiment of the present application;

FIG. 12 is a further flowchart of an autonomous mobile device voice control method provided by an embodiment of the present application;

fig. 13 is a schematic diagram of an autonomous mobile device voice control apparatus according to an embodiment of the present application;

fig. 14 is a schematic diagram of another autonomous mobile device voice control apparatus provided in an embodiment of the present application;

fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

With the progress of science and technology, robots have advanced into the lives of more and more people and play an important role in the lives of people. At present, the robot can be operated through a host physical key, a mobile phone application program (APP), a remote controller, and the like. These methods of operation all have drawbacks. Therefore, the mode of controlling the robot through voice is widely popular among users due to being more intelligent. After the robot with the voice recognition function collects the voice signal, the voice signal is recognized and relevant tasks are executed.

However, the current voice commands can only control the robot to start, execute tasks, stop, charge, etc. The robot cannot be precisely controlled. For example, if a user wants a robot to perform a task in a specific area, the robot needs to be transported to the specific area and then voice-controlled. For another example, if some areas are forbidden areas, such as a toilet, the user must close the toilet while the robot is working to prevent the robot from entering the forbidden areas.

For another example, sometimes a user wants a robot to perform tasks on a plurality of areas, the user needs to move the robot to one of the areas, and after the robot performs the tasks on the area, the user needs to move the robot to another area. Although some robots are capable of recognizing a work area indicated by a user, it is a prerequisite that the speech uttered by the user can indicate only one work area. If the user wants to execute tasks on a plurality of areas, after the robot executes the tasks each time, the user needs to indicate the next working area through voice, and the process is troublesome.

Based on this, embodiments of the present application provide a method, an apparatus, a device, and a readable storage medium for voice control of an autonomous mobile device, so that a plurality of working areas are indicated to the autonomous mobile device through a voice signal, and the autonomous mobile device can sequentially execute tasks for the plurality of working areas, and has high accuracy and a simple process.

Fig. 1A is a schematic diagram of an implementation environment of an autonomous mobile device voice control method according to an embodiment of the present application. Referring to fig. 1A, the implementation environment includes an autonomous mobile device, such as a sweeper robot, a self-moving air cleaning robot, a robotic lawnmower, a window cleaning robot, a solar panel cleaning robot, a housekeeper robot, an unmanned aerial Vehicle, an Automated Guided Vehicle (AGV), a security robot, a guest greeting robot, a nursing robot, and the like.

The autonomous mobile equipment is provided with a sound signal acquisition device such as a microphone and the like, and can acquire a voice signal sent by a user. And after the autonomous mobile equipment collects the voice signal, the autonomous mobile equipment identifies the voice signal to obtain a voice instruction and executes a task indicated by the voice instruction. In practice, the autonomous mobile device itself may recognize the voice signal. Or, the autonomous mobile device establishes a network connection with a voice recognition server (not shown in the figure), and after acquiring the voice signal, the autonomous mobile device sends the voice signal to the voice recognition server, so that the voice recognition server recognizes the voice signal, and sends the recognized voice command to the autonomous mobile device.

Next, the structure of the autonomous moving apparatus will be described in detail by taking the autonomous moving apparatus as a sweeping robot as an example.

Fig. 1B is a schematic structural diagram of a sweeping robot provided in the embodiment of the present application. Hereinafter, the sweeping robot is simply referred to as a robot, and please refer to fig. 1B, "→" represents a propagation direction of the voice signal. The robot comprises a robot shell 1, a driving element, a convex structure 2 and a voice signal acquisition device 3; wherein, the driving element is arranged in the robot shell 1 and is used for driving the robot shell 1 to move; the protruding structure 2 is arranged on the upper surface 10 of the robot housing 1, and the voice signal collecting device 3 is arranged on the protruding structure 2.

Referring to fig. 1B again, the robot housing 1 includes a top plate, an annular side plate, and a bottom plate, the top plate, the annular vertical plate, and the bottom plate are enclosed and assembled to form an accommodation chamber, and a control unit and a driving element are disposed in the accommodation chamber. In addition, the robot also comprises functional elements such as a driving wheel 6, an edge brush 7, a rolling brush or a fan and the like which are arranged on the robot shell 1, wherein the driving wheel 6 is used for driving the robot to run under the action of the driving element, the edge brush 7 and the rolling brush clean a working surface after receiving signals of a control unit, and the fan is used for forming a negative pressure cavity in the dust box so as to suck dust, sundries and the like on the working surface into the dust box for dust removal. It should be noted that the structure and the working principle of the functional elements are basically the same as those of the existing sweeping robot, and those skilled in the art can completely implement the functional elements based on the existing technology, so that the description is omitted here.

The upper surface 10 of the top plate of the robot housing 1 is convexly provided with a protruding structure 2. In some embodiments, the raised structures 2 and the top plate are integrally formed. In other embodiments, the protrusion structure 2 and the top plate are separately formed, and then the protrusion structure 2 is fixedly connected to the upper surface 10 of the top plate by bonding, screwing, or the like. The protruding structure 2 is provided with a sound signal collecting device 3.

In the invention, the sound signal collecting device is arranged on the convex structure 2 which is convexly arranged on the upper surface 10 of the robot shell 1, so that the sound signal collecting device 3 is far away from a noise source of the robot, the interference of the noise emitted by the robot to the sound signal collecting device 3 is reduced, and the robot can more accurately collect a user voice control command. The user voice control instruction comprises the steps of starting to sweep the floor, playing music, stopping sweeping, charging and the like, and a person skilled in the art can set corresponding functions according to the actual requirement of the robot.

Fig. 1C is a schematic structural diagram of a sound signal collecting apparatus of an autonomous mobile device. Referring to fig. 1C, the sound signal collecting device 3 includes a Microphone (MIC). In detail, in some embodiments, the sound signal collection device 3 includes a PCB board 30 (printed circuit board), a shock-absorbing housing 31, and a microphone chip 32; wherein, the damping cover 31 is arranged on the PCB 30 and encloses with the PCB 30 to form an external package of the sound signal collecting device 3 with a containing cavity, the microphone chip 32 is arranged in the containing cavity, and the central area of the top of the damping cover 31 is provided with a sound pickup hole 310 for communicating the outside with the containing cavity. The PCB 30 is in communication connection with the microphone chip 32 and the control unit of the robot, the microphone chip 32 collects external sound signals from the sound pickup hole 310 and transmits the sound signals to the control unit through the PCB 30, and the control unit controls the robot to execute user voice control instructions contained in the sound signals.

It should be noted that, on the one hand, the damping housing 31 of the sound signal collecting device 3 can reduce the influence of the vibration generated during the operation of the robot on the sound signal collecting device 3, on the other hand, the damping housing 31 can absorb the noise from the robot itself, and the sound pickup hole 310 is opened in the central area of the top of the damping housing 31 and only collects the sound signal from the top (usually, the voice control command sent by the user). Especially for the sweeping robot, the sweeping robot generally works on the ground, and the user sends out voice control from a high position, the sound pickup hole 310 positioned in the center area of the top of the shock absorption housing 31 can easily collect the voice signal of the voice control of the user, and the noise sent by the robot can be blocked by the shock absorption housing 31 surrounding the sound pickup hole 310, so that the interference of the noise on the signal collected by the voice signal collecting device 3 can be reduced. In other embodiments, the shock absorbing housing 31 comprises a shock absorbing foam, it being understood that the foam can not only block noise from the robot itself from entering the sound pickup hole 310, but also absorb some of the noise.

With continued reference to fig. 1C, the sound signal collecting device 3 further includes a waterproof and dustproof film 33, and the waterproof and dustproof film 33 is disposed on the vibration-damping housing 31 and covers the sound-pickup hole 310 to prevent water or dust from falling onto the microphone chip 32 through the sound-pickup hole 310 and affecting the sound signal collecting effect of the microphone chip 32.

With reference to fig. 1C, in this embodiment, the sound signal collecting device 3 further includes an upper cover 34, and the upper cover presses the shockproof cover 31 onto the PCB, and is fixedly connected to the protruding structure 2 or the distance sensor 3 through a connecting member such as a screw (not shown), so as to achieve a fixed connection relationship between the sound signal collecting device 3 and the robot housing 1, so as to prevent the sound signal collecting device 3 from falling off from the robot housing 1 during the robot driving process. In addition, a sound pickup hole is formed in the top center area of the upper cover 34 at a position corresponding to the sound pickup hole of the damper housing 31.

Further, in order to enhance the capability of the sound signal collecting device 3 to collect the sound signal, the sound signal propagation path is ensured to be short and wide as much as possible, and in some embodiments, the above purpose is achieved by limiting the aperture-hole depth ratio of the sound pickup hole 310, specifically, the aperture-hole depth (d2) ratio of the sound pickup hole 310 is greater than 1 as much as possible. In a more specific embodiment, the aperture (d1) and depth (d2) ratio of pickup hole 310 is greater than 2: 1.

In order to enable the robot to better collect the voice signals controlled by the user, in some embodiments, the robot comprises at least three voice signal collecting devices 3, and the voice signal collecting devices 3 are distributed uniformly in a ring shape. A plurality of sound signal collection devices 3 of annular equipartition can the balanced collection to the sound signal of transmitting from each angle to guarantee the accuracy and the continuity of the user speech control signal who gathers.

Fig. 1D is a schematic structural diagram of another embodiment of an autonomous mobile device according to an embodiment of the present disclosure. Referring to fig. 1D, the robot includes three sound signal collecting devices 3, and the three sound signal collecting devices 3 are uniformly distributed in a ring shape, that is, the three sound signal collecting devices 3 are located on a circle, the distance from each sound signal collecting device 3 to the center of the circle is the radius of the circle, and the central angle between two adjacent sound signal collecting devices 3 is 120 ° (degrees). In order to optimize the sound signal collecting capability of the plurality of sound signal collecting devices 3, the diameter of the circle in which the at least three sound signal collecting devices 3 are uniformly distributed in a ring shape is in the range of 60mm to 100 mm.

In other embodiments, the robot comprises three sound signal collection devices 3, the three sound signal collection devices 3 are distributed in a triangle, and one of the three sound signal collection devices 3 is located in front of the upper surface 10 of the robot housing 1 relative to the other two. The three sound signal collecting devices 3 may be uniformly distributed in a ring shape, that is, the three sound signal collecting devices 3 are located on a circumscribed circle of a triangle, and a central angle between two adjacent sound signal collecting devices 3 is 120 ° (degrees).

Of course, in other embodiments, the three sound signal collecting devices 3 do not need to be uniformly distributed in a ring shape, and only need to be distributed in a front-to-back arrangement manner. The advantage of this kind of arrangement is that, when sweeping robot went forward, the voice control instruction that the user sent because of transmission delay in media such as air, a small amount of sound signals only can be gathered to the anterior sound signal collection device 3 of robot housing 1 upper surface 10, and most sound signals need be gathered by the sound signal collection device 3 that is located the rear portion, sets up sound signal collection device 3 more at the rear portion can be better gather sound signals, guarantees the accuracy of the sound signal of gathering.

Further, in order to optimize the effect of collecting the sound signal by the sound signal collecting device 3, in some embodiments, a model selection standard of the sound signal collecting device 3 is further provided, specifically: an omnidirectional digital microphone is selected, the Signal-to-noise ratio (SNR) of the omnidirectional digital microphone is larger than 64dB (A), the sensitivity is guaranteed to be minus 26+3dBFS, the Acoustic Overload Point (AOP) is guaranteed to be 120Db SPL, and the Total Harmonic Distortion (THD) is preferably lower than 0.5% at 94dB SPL @1 kHz.

Further, in some embodiments, the robot further comprises a distance sensor 4, the distance sensor 4 is disposed on the robot housing 1 and is used for measuring the distance between the obstacle and the robot in front of the moving direction of the robot, so that when the distance between the obstacle and the robot reaches a set threshold value, the robot can stop moving or change the moving path to prevent the robot and the obstacle from colliding. In other embodiments, the distance sensor 4 is rotatably disposed on the robot housing 1, and can rotate 360 degrees relative to the robot housing to detect the layout of furniture, walls, etc. in the workspace, then map the workspace, and work according to the mapped map to improve the work efficiency.

The distance sensor 4 includes DTOF and LDS. In some embodiments, the distance sensor 4 is disposed on the protruding structure 2, and the sound signal collecting device 3 is disposed on the distance sensor 4. Therefore, the distance sensor 4 and the sound signal acquisition device 3 can utilize the convex structure 2 without independently arranging the convex structure, so that the structure of the robot can be simplified as much as possible, and the manufacturing cost of the robot can be reduced.

In other embodiments, the protruding structure 2 comprises a distance sensor 4, i.e. the distance sensor 4 is directly arranged on the upper surface of the robot housing 1 to form a protruding structure 2, and the sound signal collecting device 3 is arranged on the distance sensor 4, i.e. the sound signal collecting device 3 is arranged on the protruding structure 2 formed by the distance sensor 4. Distance sensor 4 directly sets up and forms protruding structure 2 on the upper surface of robot housing 1, and sound signal collection system 3 utilizes its distance sensor 4 self characteristics protruding setting on robot housing 1, need not to set up protruding structure in addition, and overall structure is simple, and is with low costs.

On the other hand, the distance sensor 4 is arranged on the upper surface 10 of the robot shell 1, so that other structures of the robot can be avoided well, and the position of an obstacle can be sensed accurately. The sound signal collecting device 3 can be far away from noise-generating components such as a driving motor, a rolling brush, an edge brush 7 and a fan of the robot as far as possible, and can reduce the interference of the noise generated by the robot to the sound signal.

In other embodiments, the robot further includes a sound signal playing device 5, the sound signal playing device 5 may be a speaker (horn), the sound signal playing device 5 is disposed on the robot housing 1, and the sound signal playing device 5 is in communication with a control unit of the robot, and the control unit sets a broadcast operation mode of the robot, such as playing music. After the user controls the robot to enter the broadcast working mode through the remote controller or the APP, the music stored in the control unit is played through the sound signal playing device 5.

In order to prevent the sound signal played by the sound signal playing device 5 from interfering with the sound signal collecting device 3 collecting the voice-controlled sound signal emitted by the user, the sound collecting hole 310 of the sound signal collecting device 3 and the sound emitting hole of the sound signal playing device 5 are oriented in different directions in some embodiments. More specifically, the sound pickup hole 310 of the sound signal collection device 3 is oriented perpendicular to the upper surface 10 of the robot housing 1, and the sound emission hole of the sound signal playing device 5 is oriented perpendicular to the outer vertical surface 11 of the robot housing 1, that is, the sound pickup hole 310 of the sound signal collection device 3 and the sound emission hole of the sound signal playing device 5 are oriented at an angle of 90 ° (degrees).

Note that, normally, the upper surface 10 and the outer vertical surface 11 of the robot housing 1 are disposed perpendicular to each other, but naturally, when the sound collecting hole 310 of the sound signal collecting device 3 and the sound emitting hole of the sound signal playing device 5 are oriented in different directions, the upper surface 10 and the outer vertical surface 11 of the robot housing 1 are disposed at different angles.

Further, in some embodiments, the sound signal playing device 5 is located at the front of the robot housing 1, and the sound signal collecting device 3 is located at the rear of the robot housing 1. In other embodiments, the sound signal playing device 5 is located at the rear of the robot housing 1, and the sound signal collecting device 3 is located at the rear of the robot housing 1. The division standard of the front and rear parts of the robot housing 1 is based on the shape of the robot housing 1 to divide it into two in front and rear, wherein the area located at the front side of the robot housing 1 is the front part and the area located at the rear side of the robot housing 1 is the rear part. For example: taking the embodiment shown in fig. 1C as an example, the circular robot housing 1 is divided into a front semicircular area and a rear semicircular area in the front-rear direction, the front semicircular area defining a front portion, and the rear semicircular area defining a rear portion.

It can be understood that one of the sound signal collection device 3 and the sound signal playing device 5 is located in the front of the robot housing 1, and the other is located in the rear of the robot housing 1, so that a distance between the two is kept far enough, thereby further reducing the interference of the sound signal played by the robot to the sound signal collection device 3, the robot can collect the voice control instruction of the user more accurately and execute the instruction accurately, and then better use experience can be provided for the user.

Furthermore, in order to reduce the interference of the sound signal played by the robot itself on the sound signal acquisition device 3, in some embodiments, the robot further includes a sound signal recovery device, the sound signal recovery device is in communication connection with the control unit of the robot and the sound signal playing device 5, and is used for recovering the sound signal of the sound signal playing device 5, the control unit receives the sound signal recovered by the sound signal recovery device, filters the recovered sound signal from the sound signal acquired by the sound signal acquisition device 3, and transmits an instruction contained in the filtered sound signal to the execution element, so as to control the execution element to execute the instruction.

In some embodiments, the sound signal extraction device includes a filtered extraction circuit electrically connected to the control unit of the robot body by a wire and electrically connected to the sound signal playing device by a wire.

In addition to the sound signal collecting device, in some embodiments, the robot further includes a sound signal noise reduction device, which is in communication with the sound signal collecting device 3 and the control unit, and is configured to perform noise reduction processing on the sound signal collected by the sound signal collecting device 3 to eliminate a noise or invalid sound signal portion of the collected sound signal.

In addition to the robot, the present invention also provides a control method suitable for the robot to eliminate invalid sound signals collected by the sound signal collecting device 3, especially to eliminate interference of sound signals emitted by the robot itself to signal collection of the sound signals. For example, please refer to fig. 1E.

Fig. 1E is a flow chart of voice control of an autonomous mobile device according to an embodiment of the present application. The embodiment comprises the following steps:

s1, collecting the sound signal by using the sound signal collecting device 3;

the sound signal collected by the sound signal collection device 3 mainly includes a voice control instruction of the robot from the user, for example, the robot collects the sound signal included in the voice control of the user by using the sound signal collection device 3 such as the sound signal collection device 3. However, in practice, the robot may also generate sound signals during operation, or the robot itself may also have the capability of generating sound signals, for example, the robot may play music, read books, etc. during operation or in a shutdown state, since the sound signal collection device 3 mainly functions to collect voice control of the user, these sound signals generated by the robot itself are herein referred to as "invalid sound signals". Based on this, in order to eliminate the interference of the invalid sound signals on the collected signals of the sound signal collecting device 3, the control method of the robot of the present invention further includes the following steps:

and S2, filtering the sound signals played by the robot in the sound signals collected by the sound signal collecting device 3 to obtain effective sound signals.

Fig. 1F is another flow chart of voice control of an autonomous mobile device according to an embodiment of the present application. Referring to fig. 1F, in some embodiments, the method for implementing step S2 in the control method includes the following steps:

s20, acquiring the sound signal played by the robot as an invalid sound signal;

s21, filtering the invalid sound signal from the sound signal collected by the sound signal collecting device 3 to obtain a valid sound signal.

In detail, a sound signal playing device 5 is provided in the robot, the sound signal playing device 5 may be a speaker (horn), the sound signal playing device 5 is provided on the robot housing 1, and the sound signal playing device 5 is in communication connection with a control unit of the robot, the control unit is provided with an operating mode of the robot, such as playing music, and after a user controls the robot to enter the control mode through a remote controller or APP, the music stored in the control unit is played through the sound signal playing device 5.

The robot further comprises a sound signal recovery device, the sound signal recovery device is in communication connection with the control unit of the robot and the sound signal playing device 5 and is used for recovering the sound signals of the sound signal playing device 5, the control unit receives the sound signals recovered by the sound signal recovery device, filters the recovered sound signals from the sound signals collected by the sound signal collection device 3, transmits instructions contained in the filtered sound signals to an execution element, and controls the execution element to execute the instructions.

Fig. 1J is another flow chart of voice control of an autonomous mobile device provided by an embodiment of the present application. In this embodiment, the method for implementing step S2 in the control method includes the following steps:

s20', judging whether the robot is in the broadcasting working mode;

s21', if yes, acquiring the sound signal played by the robot in the playing working mode as an invalid sound signal;

s22', the invalid sound signal is filtered from the sound signal collected by the sound signal collecting device 3 to obtain a valid sound signal.

In addition, in other embodiments, in the control method of the present invention, after the sound signal is collected by the sound signal collecting device 3, noise reduction processing is performed on the sound signal, and then the sound signal played by the robot in the sound signal is filtered to obtain an effective sound signal, so as to further eliminate the influence of other sound signals except the user voice control instruction.

After obtaining the valid sound signal from step S2, the control method performs the following steps:

and S3, executing the control instruction contained in the effective sound signal to realize the voice interaction between the robot and the user, thereby improving the use experience of the user.

The application scenarios are exemplified as follows:

firstly, when the floor sweeping robot is sweeping the ground, a user sends a voice control command of playing music, and the robot starts playing the stored music after acquiring the command. Of course, the user can also request the required music according to the audio data stored in the robot, and the voice control command only needs to contain the music name.

And secondly, when the current sweeping robot is in a halt state or a standby state, the user sends a sweeping voice control instruction, and the robot starts to sweep the ground according to a preset route after acquiring the instruction.

And thirdly, when the current floor sweeping robot is sweeping the ground and playing music at the same time, the user sends a voice control instruction of 'stopping playing music', and the robot collects the instruction and filters out invalid sound signals generated by playing music and then stops playing music.

Fig. 2 is a flowchart of an autonomous mobile device voice control method according to an embodiment of the present application. The execution subject of this embodiment is an autonomous mobile device, and this embodiment includes:

201. a first speech signal is collected.

202. When the first voice signal matches a wake-up instruction of the autonomous mobile device, waking up a voice control function of the autonomous mobile device.

The autonomous mobile device is provided with a sound signal acquisition device, and the sound signal acquisition device is a microphone, a microphone array and the like. When the voice control function of the autonomous mobile equipment is not awakened, the sound signal acquisition device continuously acquires a first voice signal in the surrounding environment, identifies the first voice signal, and if the first voice signal is matched with the awakening instruction of the autonomous mobile equipment, awakens the voice control function of the autonomous mobile equipment; and if the first voice signal is not matched with the awakening instruction of the autonomous mobile equipment, keeping the voice control function in a waiting awakening state.

When the voice control function is awakened, the autonomous mobile equipment acquires a second voice signal and then sends the second voice signal to the voice recognition server at the cloud end, so that the voice recognition server determines whether the second voice signal is matched with the control instruction or not; when the voice control function is in a waiting awakening state, the autonomous mobile equipment locally identifies whether the acquired first voice signal is matched with the awakening instruction.

203. And acquiring a second voice signal in the voice control function awakening state.

After the voice control function of the autonomous mobile device is awakened, the autonomous mobile device can acquire voice signals in the surrounding environment by using the sound signal acquisition device. For example, a second speech signal uttered by the user is collected.

204. At least two working areas are determined from the second speech signal.

And when the autonomous mobile equipment has the voice recognition function, the autonomous mobile equipment recognizes the second voice signal, so that at least two working areas are determined. Or the autonomous mobile equipment sends the second voice signal to the voice recognition server, and the voice recognition server determines at least two working areas and indicates the working areas to the autonomous mobile equipment. For example, the text content corresponding to the second voice signal is "clean xiaoming room and study", and then at least two working areas are xiaoming room and study; if the text content corresponding to the second voice signal is "clean all carpet areas", the work area is a plurality of areas including a carpet centered on the carpet.

Fig. 3 is a schematic diagram of a speech recognition process in an embodiment of the present application. Referring to fig. 3, in the voice recognition process, the autonomous mobile device acquires a second voice signal, performs noise reduction processing on the second voice signal, and uploads the noise-reduced second voice signal to the voice recognition server. And the voice recognition server carries out semantic recognition on the second voice signal to obtain a voice instruction, and returns the voice instruction to the autonomous mobile equipment. And after the autonomous mobile equipment receives the voice command, executing the task indicated by the voice command. Tasks may be cleaning, mopping, mowing, purifying air, etc.

It should be noted that, although fig. 3 illustrates an example in which the autonomous mobile device collects the second voice signal. However, the embodiment of the present application is not limited, and in other feasible implementation manners, the user may further open the client on the terminal device and then send out the second voice signal, the terminal device collects the second voice signal, performs noise reduction processing on the second voice signal, and uploads the second voice signal after the noise reduction processing to the voice recognition server.

205. And executing the tasks indicated by the second voice signals for the at least two working areas in sequence.

The autonomous mobile device performs tasks to the work areas in sequence according to a certain order. For example, the autonomous mobile device randomly orders the work areas to obtain a random queue, and sequentially executes tasks on the work areas according to the order indicated by the random queue.

For another example, if the user desires to perform tasks such as cleaning on a plurality of work areas and the user desires to preferentially perform the cleaning task on one of the work areas, the second voice signal may be issued in order of priority. And the autonomous mobile equipment determines the sequence of the occurrence of each working area in the at least two working areas in the second voice signal. And then, sequentially executing the tasks indicated by the second voice signals to the at least two working areas according to the sequence.

Taking the autonomous mobile device as an example of the air purifying robot, if the user wants to purify the air in the study room and the baby room and wants to preferentially purify the air in the baby room, the second voice signal is "please purify the baby room and the study room". And the autonomous mobile equipment determines that the working areas are the baby room and the study room according to the second voice signal, and the baby room is preferred. At this time, even if the study room is closer to the autonomous mobile device and the baby room is farther from the autonomous mobile device, the autonomous mobile device advances to the baby room in advance, and after air purification of the baby room is completed, the study room is advanced to purify air of the study room.

By adopting the scheme, the autonomous mobile equipment executes tasks to a plurality of working areas in sequence according to the priority, so that the user requirements are met to a great extent, and the autonomous mobile equipment is more humanized.

For another example, there are many working areas and a certain distance exists between the working areas, and if the autonomous mobile device executes tasks on the working areas according to a random sequence, the energy consumption is large and the time consumption is long. For this purpose, after determining a plurality of working areas, the autonomous mobile device determines a distance between itself and each of the at least two working areas. Then, sequencing the at least two working areas according to the sequence from near to far to obtain a queue; and executing the tasks indicated by the second voice signals to the at least two working areas in sequence according to the queues.

For example, if the second voice signal is "clear all carpet areas", the autonomous mobile device determines all areas containing carpet from the environment map, for example, please refer to fig. 4. Fig. 4 is a schematic view of a carpet area.

Referring to fig. 4, there are 3 carpet areas, carpet area 41, carpet area 42, and carpet area 43. The autonomous mobile device 40 determines that the carpet area 41 is closest to it, followed by the carpet area 42, and finally the carpet area 43. Thus, the sweeping sequence is carpet area 41, carpet area 42, and carpet area 43.

By adopting the scheme, the autonomous mobile equipment executes tasks in each working area according to the sequence of the distance from near to far, so that the energy consumption can be reduced to the maximum extent and the speed can be increased.

It should be noted that although the above description is directed to operating an autonomous mobile device to perform tasks on multiple work areas via voice operation. However, the embodiments of the present application are not limited, and in other possible implementations, the autonomous mobile device may be operated in a voice operation to perform tasks on a single work area.

According to the voice control method of the autonomous mobile device, after the autonomous mobile device collects the second voice signal, at least two working areas are determined according to the second voice signal, and tasks indicated by the second voice signal are executed on the working areas in sequence. By adopting the scheme, the plurality of working areas can be indicated to the autonomous mobile equipment through one voice signal, so that the autonomous mobile equipment can sequentially execute tasks on the plurality of working areas, and a user interacts with the autonomous mobile equipment through natural language so as to realize extremely simple control on the autonomous mobile equipment, and the autonomous mobile equipment has high accuracy and simple process.

Optionally, in the above embodiment, during the process that the autonomous mobile device sequentially executes the tasks indicated by the second voice signal to the at least two working areas, after the task is executed to one of the at least two working areas, before the autonomous mobile device moves to the next working area, the autonomous mobile device stops executing the task.

And after the autonomous mobile equipment determines at least two working areas, the autonomous mobile equipment sequentially executes tasks to the working areas according to a certain sequence. If the distance between two adjacent working areas is long, the working module can be closed in the process that the autonomous mobile equipment travels from one working area to the other working area according to the travel path. That is, the autonomous moving apparatus does not perform tasks such as cleaning, mowing, and the like while traveling on the travel path.

Referring again to fig. 4, the second voice signal is used to clean the carpet area 43, the carpet area 42 and the carpet area 41. The autonomous mobile device 40 travels from the current location to the altar area 43. In this process, the operation module of the autonomous mobile apparatus 40 is in a turned-off state. The work modules are closed on the travel path during travel from the carpet area 43 to the carpet area 42 after the autonomous moving apparatus 40 finishes cleaning the carpet area 43, and during travel from the carpet area 42 to the carpet area 41 after the autonomous moving apparatus 40 finishes cleaning the carpet area 42, and the travel path is shown by a dotted arrow in the figure.

By adopting the scheme, the working module does not need to be started in the process of moving the autonomous mobile equipment from one working area to the next working area, so that the energy is saved and the traveling speed is increased.

Optionally, in the above embodiment, before the autonomous mobile device moves from the current location to the first working area, or before the autonomous mobile device moves from the current working area to the next working area, it is further determined whether the length of the travel path is greater than a preset threshold. And if the length of the running path is greater than the preset length, closing the working module and advancing to the working area according to the running path. And if the length of the driving path is less than or equal to the preset length, the driving module advances to the working area in the opening state.

For example, sometimes the travel path between two work areas is relatively short, e.g., the length of the travel path between a bedroom and a living room is almost negligible. In order to avoid the damage caused by frequent switching of the working modules, when the driving path is short, the working modules do not need to be closed.

Optionally, in the above embodiment, when the autonomous mobile device determines at least two working areas according to the second voice signal, first, the area category is determined according to the second voice signal. And then, determining at least two working areas from the area set corresponding to the environment map according to the area types.

In the embodiment of the application, the autonomous mobile device constructs an environment map when in a completely unknown environment, or receives the environment map sent by other robots, and then performs region segmentation on the environment map by using a partition algorithm and the like, so that a plurality of working regions are obtained, and the environment map can represent each working region, such as a kitchen, a toilet, a bedroom and the like. In addition, the environment map can represent the actual positions of different objects in the environment, so that the autonomous mobile equipment can judge the placing states of the objects in each working area.

When the user wants to clean a work area of a certain category, the information of the category is added into the first voice command. For example, "clean all bedrooms". The autonomous mobile device only cleans each bedroom after recognizing the voice signal.

For another example, "cleaning furniture in the living room", the area type indicated by the voice signal is "living room". Thus, the autonomous mobile device determines the living room from the environment map. Further, the voice signal indicates the target object "furniture". Thus, the autonomous mobile device determines furniture in the living room, such as a sofa, a tea table, and the like. For each piece of furniture, a zone containing the piece of furniture is determined, centred on the piece of furniture, and the zone is swept.

For another example, "cleaning the bed bottom of all bedrooms", the area type indicated by the voice signal is "bedroom". Thus, the autonomous mobile device determines the bedroom from the environment map. Further, the voice signal also indicates the target object "bed". Thus, the autonomous mobile device continues to determine the position of the bed within each bedroom. For each bed, a zone containing the bed is determined, centered on the bed, and the zone is swept.

As another example, "begin to clean the sofa and dining table area," the behavior of the autonomous mobile device is: and (4) walking to the position of the sofa, drawing a rectangular frame which is larger than the sofa by taking the center of the sofa as an original point, and taking the rectangular frame as a working area for cleaning. And then, moving to the area where the dining table is located, drawing a rectangular frame larger than the dining table as the original point at the center of the dining table as a working area, and cleaning.

As another example, "start to clean the sofa area," the behavior of the autonomous mobile device is: the user walks to the position of the sofa, draws a rectangular frame larger than the sofa by taking the center of the sofa as an original point, and cleans the sofa by taking the rectangular frame as a working area.

By adopting the scheme, the user can control the autonomous mobile equipment through voice to only carry out tasks such as cleaning on a certain type of working area, and the intelligentization is high. Furthermore, a target object can be indicated in the voice signal, so that the autonomous mobile equipment can perform tasks such as cleaning on a local area, and the intelligence of the autonomous mobile equipment is further improved.

Optionally, in the above embodiment, the autonomous mobile device may divide the environment map into a plurality of working areas according to the environment map, the position information of the object in the environment map, or the position information of the door in the environment map, so as to obtain the area set. And then updating the identification of each working area in the area set, and sending updating information to the voice recognition server so that the voice recognition server updates the identification of each working area.

Illustratively, a camera or other shooting device is installed on the autonomous mobile device. Fig. 5 is a schematic diagram of a furniture identification process. Referring to fig. 5, the autonomous mobile device constructs an environment map or continuously captures images during traveling to perform image acquisition. After the image is acquired, the image is preprocessed, wherein the preprocessing comprises one or more of contrast enhancement, lossless amplification, feature extraction and the like. Then, the autonomous mobile device performs AI recognition on the preprocessed image by using a pre-deployed training model so that the training model outputs recognition results such as the type and position coordinates of furniture in the image, and maps the recognition results from the (three-dimensional, 3D) environment map to a two-dimensional (two-dimensional, 2D) environment map to be saved and displayed. The training model is, for example, an AI model trained using various furniture as a sample.

Fig. 6 is a schematic view of a process of identifying a door. The difference from fig. 5 is that the training model is an AI model that is trained in advance using various gates as samples, and when an image is input to the training model, the recognition result such as the position coordinates of the gate is output, and the recognition result is mapped from the 3D environment map to the 2D environment map and stored and displayed.

After the autonomous mobile device acquires the 2D environment map, the recognition results of furniture and doors in the 2D environment map are fused, and the 2D map is partitioned by using a partition algorithm, so that the 2D environment map is divided into a plurality of areas. And then, the autonomous mobile equipment sends the partition result to an APP server and a voice recognition server, the APP server sends the partition result to the terminal equipment, and the terminal equipment is provided with an APP for controlling the autonomous mobile equipment. And after receiving the partitioning result, the terminal equipment displays the partitioning result. For example, please refer to fig. 7.

Fig. 7 is a schematic diagram of an autonomous mobile device and voice recognition server synchronization process. Referring to fig. 7, after being partitioned, the area set of the environment map includes an area 1, an area 2, and an area 3. The user can carry out custom editing on the identification of each area at the client. For example, the above-described respective identifications of the area 1, the area 2, and the area 3 are modified in order to be a small and clear room, a living room, and a kitchen. And then, the terminal equipment sends the update information to the APP server, and the APP server sends the update information to the autonomous mobile equipment. And after the autonomous mobile equipment receives the updating information, updating the identification of each working area in the local environment map.

In addition, after the autonomous mobile device updates the identification of each working area, the autonomous mobile device also synchronously updates the identification of the working area to the voice recognition server. And after receiving the update information from the APP server, the autonomous mobile equipment discovers that the identification of the working area changes. And then, the autonomous mobile equipment updates the local area and simultaneously sends the updating information to the voice recognition server, so that the voice recognition server updates and stores the identification of each working area. The user is then able to interact with the autonomous mobile device according to the custom name.

For example, the user says: "clean a small and clear room", the behavior of the autonomous mobile device is: walk to a custom-named "Xiaoming" room area and sweep this area.

For another example, the user says: "clean living room and kitchen", the behavior of the autonomous mobile device is: and walking to an area named as a 'living room' by definition according to the shouting sequence of the user, cleaning the area, and then walking to an area named as a 'kitchen' by definition and cleaning.

By adopting the scheme, the identification of each working area stored on the voice recognition server is consistent with the identification of the corresponding working area stored on the autonomous mobile equipment, and the accuracy of voice recognition of the voice recognition server is improved.

In the above embodiment, when the second voice signal indicates the area type, the autonomous mobile device determines, according to the area type, a working area that meets the area type from a pre-constructed area set. However, the embodiment of the present application is not limited, and in other possible implementations, when the area indicated by the area category is a specific area and the location of the specific area may be different each time the voice signal is sent down, the autonomous mobile device may further determine the working area in real time according to the area category. For example, the specific area is an area with water damage in the home, and obviously, the area with water damage in the home is different at different times. In this case, the autonomous mobile apparatus acquires an image using a camera or the like, and identifies the image to determine at least two working areas that match the area category.

For example, the second voice signal is "check the home for water and dry up". And after the autonomous mobile equipment collects the voice signal and carries out semantic recognition, continuously shooting images and recognizing the images in the advancing process, and wiping the places with water if the places with water exist in the images. Then, the user continues to move and take an image, and the wiping is performed every time a place where water is present is recognized.

For another example, the second voice signal is "mopping the kitchen in a greasy place". The autonomous mobile equipment acquires the voice signal and performs semantic recognition, continuously shoots images and recognizes the images in the kitchen in the process of traveling, and mopping is performed after a place with oil stains is recognized every time.

By adopting the scheme, the purpose of executing the task on the specific area is realized.

Optionally, in the above embodiment, when the autonomous mobile device sequentially executes the tasks indicated by the second voice signal to the at least two work areas, the autonomous mobile device may determine an operation manner according to the area type, and execute the tasks to the two indicated work areas at a time according to the operation manner.

For example, when the user utters the second voice signal, the specific operation mode may not be indicated, but the operation mode may be autonomously determined and executed by the autonomous mobile device. For example, the user says: "cleaning greasy dirt in kitchen". After the autonomous mobile equipment continuously collects images to determine an oil stain area, determining that the operation mode is as follows: adding cleaning liquid and increasing mopping frequency. Thereafter, the autonomous mobile device sprays cleaning liquid in the greasy dirt area and strongly drags the floor.

For another example, the user says: "clean water stain in living room". After the autonomous mobile equipment continuously collects images to determine a water stain area, if the water in the water stain area is more, determining that the operation mode is as follows: mopping is performed three times. Thereafter, the autonomous mobile device mops the floor three times in the water-affected area. If the water in the water stain area is less, determining the operation mode as follows: mopping is performed once. Thereafter, the autonomous mobile device mops the water spot area once.

By adopting the scheme, the autonomous mobile equipment automatically determines a more appropriate operation mode, and the purpose of improving the task execution efficiency is achieved.

Optionally, in the above embodiment, it is assumed that the autonomous mobile device is located in an initial region when acquiring the second voice signal, and the initial region is not any working region in the second voice signal. Then, before the autonomous moving apparatus moves from the initial area to the working area, the task performance of the initial area is recorded. And then, executing tasks on the work areas in sequence. And after the task is executed, determining whether the task is not executed on the initial area according to the record. And if the autonomous mobile equipment does not execute the task to the initial area, returning to the initial area and executing the task.

By adopting the scheme, after the autonomous mobile equipment executes the task in the area indicated by the second voice signal, the autonomous mobile equipment returns to the initial area to continue executing the task, so that the task cannot be completed in the initial area.

Optionally, in the above embodiment, the second speech signal may further include task parameters and the like. For example, the second speech signal is: the method comprises the steps of cleaning a small and clear room and a study room in a strong mopping mode, respectively cleaning for 10 minutes, mopping an oil stain area twice, and the like.

In the above embodiments, to prevent the autonomous mobile device from continuously recognizing the voice signal when the user has no interaction requirement, the voice control function of the autonomous mobile device is usually in a silent state. The voice control function of the autonomous mobile device can only be woken up after the user issues a specific wake-up word. While the voice control function is in the silence state, the autonomous mobile device may be stationary or operating.

During the operation of the autonomous mobile device, noise is generated due to the rotation of the motor and the like, and the noise is likely to interfere with the accuracy of the autonomous mobile device in recognizing the voice signal. The embodiment of the application can avoid the defects. The working state after the autonomous mobile device is awakened is called a second working state, the working state before the autonomous mobile device is awakened is called a first working state, and the volume of sound generated by the autonomous mobile device in the second working state is smaller than the volume of sound generated in the first working state. After the autonomous mobile device acquires the second voice command in the first working state, if the first voice signal is matched with the awakening command of the autonomous mobile device, the autonomous mobile device automatically switches to the second working state, namely the autonomous mobile device switches to the second working state by reducing output power consumption and the like, and acquires the second voice signal in the second working state.

Illustratively, when a microphone is installed on an autonomous mobile device, the microphone is installed in a place where noise is lowest and stability is sought. Moreover, the awakening model is trained through a large number of samples, so that the awakening rate of the autonomous mobile equipment in various running states is improved. Then, when the voice control function of the autonomous mobile device is awakened in the operation state, the autonomous mobile device reduces the volume of the noise generated by itself by changing the operation state of itself. And then, the user sends a voice control command through normal volume, and the autonomous mobile equipment receives and executes a corresponding task.

For example, the autonomous mobile device is a sweeping robot, and when the autonomous mobile device works in the first working state, the traveling speed is 0.2 m/s. And the user sends a first voice signal, and if the first voice signal is matched with the awakening instruction, the autonomous mobile equipment is switched to a second working state, the traveling speed is 0.1 m/s, and the generated noise is low. And then, the user sends out a second voice signal, and the autonomous mobile equipment acquires the second voice signal and executes a related task. The volume of the second voice signal may be less than the volume of the first voice signal.

In addition, in order to ensure that the autonomous mobile device can be awakened in a high-noise environment, the awakening rate of the autonomous mobile device in the running state can be increased in advance through multiple trainings and algorithms. For example, when the autonomous mobile device is within 5 meters of the user and in the first working state, the normal voice wake-up rate can reach 85%. After awakening, the autonomous mobile equipment is switched to a first working state, and the voice recognition accuracy rate in the first working state almost reaches the same level of the intelligent sound box.

By adopting the scheme, after the autonomous mobile equipment is awakened, the running state is automatically changed so as to reduce the noise generated by the autonomous mobile equipment and improve the accuracy of subsequent voice recognition.

Optionally, in the above embodiment, when the first voice signal matches with the wake-up instruction of the autonomous mobile device, the autonomous mobile device determines the sound source position of the first voice signal. And then the autonomous mobile equipment controls the autonomous mobile equipment to be switched from a first position to a second position according to the position of a sound source, the distance between a microphone and the position of the sound source when the autonomous mobile equipment is in the second position is smaller than the distance between the microphone and the position of the sound source when the autonomous mobile equipment is in the first position, and the microphone is arranged on the autonomous mobile equipment.

Illustratively, the intelligent voice technology has been widely applied to the fields of human-computer interaction, intelligent control, online service and the like, and along with the expansion of more application scenes, the intelligent voice technology has become the most convenient and effective means for people to acquire and communicate information, and the intelligent voice technology comprises a voice recognition technology and a voice synthesis technology. Sound source localization is a method for localizing a sound source based on a microphone array, and the implementation method can be divided into directional wave velocity formation and time delay estimation. The intelligent voice technology, the microphone sound source positioning technology and the autonomous mobile device are combined, so that a very rich application scene can be designed, for example, a voice instruction is given to the autonomous mobile device to execute a task, the autonomous mobile device is in voice interaction with the autonomous mobile device to obtain corresponding guidance, the autonomous mobile device is controlled to turn according to a sound source, and the like.

In the field of autonomous mobile devices, a microphone array is generally designed on the autonomous mobile device to receive sound source information for sound source positioning, and the autonomous mobile device is controlled to turn to the sound source positioning direction according to positioning, so that the interest of interaction and the accuracy of next voice recognition are increased. The disadvantage of this kind of application is that the positioning error of the microphone array is large, and the error is generally about ± 45 °, so that the root cause of the error is that the estimation accuracy of the time difference of the sound source reaching the microphone is not enough. Due to the existence of errors, the effect of steering the sound source by the autonomous mobile device may be inaccurate, resulting in poor use experience.

Therefore, in the embodiment of the application, after the voice control function of the autonomous mobile device is awakened, the autonomous mobile device can adjust the pose, so that the microphone on the autonomous mobile device is close to the user, and the accuracy of voice acquisition is improved. And accurately controlling the autonomous mobile equipment to turn to a speaker, namely to a user by utilizing a sound source positioning technology, a voice recognition technology and an AI recognition technology. In the process, the autonomous mobile device captures a sound source through the microphone array, and the sound source is identified as a set awakening word after signal conversion. Thereafter, the position of the sound source relative to the microphone array, and hence the body, is determined, thereby determining the approximate rotation angle, i.e., the approximate location of the sound source. And finally, the autonomous mobile equipment rotates according to the rotation angle, and the specific position of the sound source is accurately judged in the rotation process by combining AI identification, so that the autonomous mobile equipment is controlled to stop at the position facing the user.

Next, how the autonomous mobile device turns to the sound source localization direction will be described in detail.

First, a first position is determined, the first position being a position of a sound source relative to a center of the microphone array.

After the user sends out voice, the autonomous mobile equipment picks up the voice signal through the microphone array, and processes the voice signal by the computing unit to obtain a voice recognition result. If the voice recognition result is matched with the awakening word, determining a first position; and if the voice recognition result is not matched with the awakening word, keeping the state of waiting for awakening.

Fig. 8A is a schematic diagram of determining the position of a sound source relative to the center of a microphone array. Referring to fig. 8A, the microphone array includes 6 microphones respectively located at S1-S6, the 6 microphones are uniformly distributed on a circle with a radius of L1, and an origin O of the spatial coordinate system is a center of the microphone array. After the sound source emits sound, the time length of the sound reaching each microphone is different. Thus, the autonomous mobile device may determine the first location based on the speed of propagation of sound, time delays, locations of microphones, and the like. The time delay refers to a difference between time lengths of sounds received by different microphones.

It should be noted that, although 6 microphones are taken as an example here, the embodiments of the present application are not limited. In other possible implementations, more or fewer microphones may be provided.

Second, a second location is determined from the first location and a rotation angle is determined from the second location, wherein the second location is a location of the sound source relative to the center of the autonomous mobile device.

Generally, the microphone array is located at a fixed position of the body of the autonomous mobile device, and after the first position is determined, the autonomous mobile device can determine the second position according to the position of the microphone array and the first position.

Fig. 8B is a schematic diagram of an exact microphone array and autonomous mobile device body. Referring to fig. 8B, the center of the body is the center of the great circle, and the center of the body and the center of the microphone array are not coincident, but the relative positions of the two are known. Thus, after the autonomous mobile device determines the first location, the second location can be determined. After the second position is determined, a rotation angle can be determined, wherein the rotation angle is the rotation angle of the autonomous mobile device in the process from the first position posture to the second position posture.

The first pose is the pose of the autonomous mobile device before wake-up, the second pose is the pose of the microphone of the autonomous mobile device towards the user, and the second pose can also be understood as the pose of the autonomous mobile device facing the user. The pose of the autonomous mobile device against the user is: the camera which is autonomously moved faces the pose of the user.

And finally, rotating according to the rotation angle.

In the rotating process, in order to avoid the influence of too fast rotation on the AI identification effect and too slow rotation on the user experience, the electronic equipment divides the rotating angle into a first angle and a second angle; rotating at a first speed within the first angle and at a second speed within the second angle, the first speed being greater than the second speed. The second angle is, for example, α degrees.

In the rotating process, assuming that the rotating angle is theta, firstly rapidly rotating theta-alpha degrees, and then uniformly rotating alpha degrees, wherein alpha is less than theta. And in the uniform-speed rotation process, continuously shooting images by using a camera and carrying out AI identification. If the user is identified, stopping rotation; if the user is not recognized, the rotation is stopped after rotating for alpha degrees. Where α is related to the statistical error of the autonomous mobile device, which may be 30 degrees, 60 degrees, etc. The embodiments of the present application are not limiting.

Taking the autonomous mobile device as a sweeping robot as an example, referring to fig. 8B again, the sweeping robot functional component includes a camera, a microphone array, a laser ranging sensor, an infrared receiving sensor, a side brush, a driving wheel, and the like. In addition, the sweeping robot further comprises an edge sensor, a falling prevention sensor, a dust suction fan, a motion motor, a rolling brush, a calculation storage unit, a battery module, a wifi module and the like which are not shown in the figure. Under the voice control using scene, the sweeping robot is in any working state, a user sends a voice awakening instruction to the sweeping machine, the sweeping robot stops working at present, the sweeping robot turns to the person sending the awakening instruction, and the next step of interaction command of the user is waited.

For example, the sweeping robot is sweeping in the living room, the user sends out the awakening words "small Q, small Q", the sweeping robot suspends the sweeping work, turns to the user, and simultaneously, the voice broadcast responds "i am", waits for further instructions of the user, such as "please leave the living room to sweep in another room", the sweeping robot can voice broadcast responds "good", and simultaneously leave the living room to enter other rooms such as a bedroom and the like to continue sweeping.

In the voice interaction experience, the sweeping robot is required to accurately recognize the awakening instruction of the user, and simultaneously, the sweeping robot can accurately and quickly turn to the user to wait for the next control instruction. If the position of the voice command sender cannot be accurately positioned, the interactive scene becomes very poor, and the first condition is that the positioning is not accurate, and the robot turns to other directions and does not accurately face the voice operator; the second case is the direction of the voice operator, but the rotation process is slow, the action duration is long, and the interaction experience is poor.

Therefore, the sweeping robot stops sweeping work, and in the process of turning to a user, a first position of a sound source relative to the center of the microphone array is determined, and then a second position and a rotation angle are determined according to the first position and the position of the microphone array relative to the machine body. And then, rapidly rotating the theta-alpha degree, then uniformly rotating the alpha degree, and continuously shooting images by using a camera and carrying out AI identification in the uniform rotation process. If the user is identified, stopping rotation; if the user is not recognized, the rotation is stopped after rotating for alpha degrees.

For example, in fig. 8B, when the user is positioned on the right side of the sweeping robot, the rotation angle θ is 180 degrees. If alpha is 60 degrees, the autonomous mobile equipment firstly rotates 120 degrees clockwise rapidly, then rotates at a constant speed and collects images, and if the rotation reaches 170 degrees, the autonomous mobile equipment identifies a user according to the images, and stops rotating. If the user is not identified, the rotation is stopped after the uniform rotation of 60 degrees.

By adopting the scheme, the steering action of the autonomous mobile equipment is more accurate by combining sound source positioning, voice recognition and AI recognition. In addition, in the rotating process, the actions of firstly rotating rapidly and then slowly rotating at a constant speed according to the rotating angle are smoother, and AI identification is more accurate.

In the above embodiment, the speech recognition technology is a pattern recognition based on the speech characteristic parameters, and the speech recognition server can classify the input speech according to a certain pattern, and further find out the best matching result according to the judgment criterion. The schematic frame diagram is shown in fig. 8C.

FIG. 8C is a schematic diagram of a process for training a speech recognition model and recognizing speech. Referring to fig. 8C, in the training process, the input speech signal is preprocessed and then feature extracted, model training is performed using the extracted features, and the speech recognition model is generated and then stored.

The trained voice recognition model is deployed on a voice recognition server. The voice signal sent by the user is preprocessed and then subjected to feature extraction, and the voice recognition server inputs the extracted features into the voice recognition model so as to obtain a voice recognition result.

Fig. 9 is a flow chart of autonomous mobile device voice control logic provided by an embodiment of the present application. The embodiment comprises the following steps:

901. the autonomous mobile device is in a first operating state and the voice control function is in a wait-for-wake state.

902. The autonomous mobile device acquires a first voice signal.

Illustratively, the user utters a first voice signal, and the autonomous mobile device acquires the first voice signal using a sound signal acquisition device or the like.

903. Whether the first voice signal is matched with the awakening instruction of the autonomous mobile equipment or not is judged, if the first voice signal is matched with the awakening instruction, the autonomous mobile equipment is successfully awakened and the step 904 is executed, and if the first voice signal is not matched with the awakening instruction, the autonomous mobile equipment is not awakened, and the autonomous mobile equipment executes the step 911.

In this step, the autonomous mobile device determines whether the first voice signal matches the wake-up command by using its own voice recognition function, or the autonomous mobile device sends the first voice signal to the voice recognition server, and the voice recognition server determines whether the first voice signal matches the wake-up command.

904. The autonomous mobile device switches from a first operating state to a second operating state.

For example, the autonomous mobile device in the second operating state has less power consumption and less noise than the first operating state, for example, the driving wheel rolls normally and the rest of the sounding components operate normally in the first operating state; and in the second working state, the driving wheel stops rolling, and the rest sounding components reduce the running power. For another example, in the second operating state, the driving wheel rolls at a reduced speed, and the remaining sound-producing components reduce the operating power.

905. The autonomous mobile device determines whether a second voice signal is acquired within a preset time length, and if the autonomous mobile device acquires the second voice signal within the preset time length, the step 906 is executed; if the autonomous mobile device does not acquire the second voice signal within the preset duration, step 912 is executed.

906. The autonomous mobile device determines whether the second voice signal was successfully parsed.

Illustratively, whether the autonomous mobile device itself or the voice server parses the second voice signal, if the autonomous mobile device itself or the voice server parses the second voice signal successfully, step 907 is executed; if the second speech signal is not successfully analyzed, step 913 is executed.

907. The autonomous mobile device determines whether the parsed result matches the control instruction, if so, go to step 908; if the parsed result does not match the control instruction, step 914 is executed.

Illustratively, the autonomous mobile device or voice recognition server determines whether the parsed result is a task of self-cleaning, mowing, or the like.

908. The autonomous mobile device determines whether its own status meets the requirement for executing the task, and if the own status meets the requirement for executing the task, the autonomous mobile device executes step 909; if the self status does not satisfy the requirement of executing the task, step 915 is executed.

Illustratively, the autonomous mobile device determines whether its own power level, remaining space of the dust box, remaining water level in the water box, etc., meet the mission requirements.

909. The task is performed and the user is voice fed back, after which step 910 is performed.

For example, an autonomous mobile device starts traveling and says to the user: "good, i.e., clean cubicle and study".

910. And finishing the voice interaction of the current round, and enabling the voice control function to enter a waiting awakening state.

911. The autonomous mobile device continues to operate in the first operating state, and after a preset time period, step 910 is executed.

912. The autonomous mobile device feeds back the command timeout and resumes the first operational state, after which step 910 is performed.

Illustratively, an autonomous mobile device issues to a user: voice feedback of "voice interaction timeout please re-wake up". At the same time, the autonomous mobile device re-enters the first operating state. Thereafter, step 910 is performed.

913. The autonomous mobile device feeds back the user's intent without understanding and resumes the first operational state, after which step 910 is performed.

Illustratively, an autonomous mobile device issues to a user: "not receive correct command, please wake up again"; or "i don't hear clearly what you say, please wake up again" voice feedback. Meanwhile, the autonomous mobile device reenters the first operating state. Thereafter, step 910 is performed.

914. The autonomous mobile device continues to operate in the second operating state and performs voice interaction, answering, and the like with the user. After the preset time period, step 910 is executed.

Illustratively, an autonomous mobile device issues to a user: "this task is too difficult for me to perform, please change the description," ask you how you are under the bed to clean the bedroom, "guide the user to interact with it to clarify the user's intent.

915. The autonomous mobile device voice feedback fails to perform the task and continues to operate in the second operating state.

Illustratively, an autonomous mobile device says to the user: "i need charge can carry out the task", "i need return to the base station moisturizing", "please clear up the dirt box", etc. and continue to work in the second operating condition. After a preset time period, step 910 is executed.

Or, the autonomous mobile device: "I want to go to charge, go to clean bedroom again after I's completion of charging", and move to the basic station by oneself and charge in order to maintain self-condition. And then, the autonomous mobile equipment performs tasks such as cleaning again.

By adopting the scheme, the autonomous mobile equipment judges the state of the autonomous mobile equipment before executing the task, determines whether to execute the task immediately or to execute the task after charging and water replenishing according to the state of the autonomous mobile equipment, and can avoid interruption and the like in the task executing process.

In the above embodiment, after the voice control function is awakened and the preset duration elapses, the voice control function enters the wait-for-awakening state again. For example, after the voice control function is awakened and the second voice signal is not collected after a preset time period, the voice control function automatically enters a wait-to-awaken state. And if the instruction is executed for one time, the system automatically enters a waiting wakeup state.

By adopting the scheme, the false triggering of surrounding voice to the autonomous mobile equipment can be avoided.

Optionally, in the above embodiment, the second voice signal may further indicate a task exclusion zone. The autonomous mobile device determines a task exclusion area from the environment map and determines at least two working areas from areas outside the task exclusion area.

If the task exclusion zone is less or more easily delineated relative to the work area, the user may indicate the task exclusion zone in the second speech signal. And the autonomous mobile equipment determines a task forbidden zone according to the second voice signal, and then takes other zones as working zones and executes tasks. For example, the user says: if the bed bottom is not cleaned, the autonomous mobile equipment cleans the area outside the bed bottom.

By adopting the scheme, the difficulty of issuing the voice command by the user is reduced.

In the above embodiment, the voice signal is mainly used for controlling the work area. However, the embodiments of the present application are not limited. In other feasible implementation manners, the autonomous mobile device can be controlled to operate other electronic devices in the home through voice control, fixed-point patrol, monitoring, watching, protecting and other functions of the autonomous mobile device can be realized through voice control, the autonomous mobile device can be controlled to automatically find objects through voice control, and the like. These scenes will be described below.

Firstly, the scene that the autonomous mobile equipment controls other electronic equipment in the home is realized through voice control. For example, please refer to fig. 10. Fig. 10 is another flowchart of an autonomous mobile device voice control method according to an embodiment of the present application. The embodiment comprises the following steps:

1001. the autonomous mobile device is in a first operating state and the voice control function is in a wait-for-wake state.

1002. The user awakens the voice control function through the first voice signal and sends out a second voice signal.

For example, the autonomous mobile device may further automatically switch to the second working state after waking up the voice control function, which may be referred to the above description and is not described herein again.

1003. And the autonomous mobile equipment obtains a control instruction according to the second voice signal, and the control instruction is used for instructing the autonomous mobile equipment to control the specified equipment.

Illustratively, the autonomous mobile device parses the second voice signal itself, or, parses the voice signal by the voice recognition server to obtain the control instruction. The control instructions are for instructing the autonomous mobile device to control a designated device within a designated area. The designated device is, for example, an air conditioner, a refrigerator, a curtain, or other household appliance or household appliance.

1004. The autonomous mobile device moves to a designated area to complete control of the designated device.

For example, in the power-on state of the autonomous mobile device, the user is located within a voice signal acquisition range of the autonomous mobile device, for example, the user and the autonomous mobile device are simultaneously in an object, the autonomous mobile device is 5 meters away from the user, and the like. The second voice signal sent by the user is' turn on the main horizontal air conditioner and refrigerate at 25 ℃. After the autonomous mobile equipment plans a driving path according to an environment map and enters a main lying position, a hardware remote control module of the autonomous mobile equipment is used for turning on an air conditioner of the main lying position, setting the mode to be a refrigeration mode and setting the temperature to be 25 ℃.

By adopting the scheme, the value-added function is realized by combining voice control with other hardware and algorithms of the autonomous mobile equipment, so that the autonomous mobile equipment is more intelligent.

And secondly, realizing a scene of fixed-point patrol of the autonomous mobile equipment through voice control. For example, please refer to fig. 11. Fig. 11 is yet another flowchart of an autonomous mobile device voice control method provided by an embodiment of the present application. The embodiment comprises the following steps:

1101. the autonomous mobile device is in a first operating state and the voice control function is in a wait-for-wake state.

1102. The user awakens the voice control function through the first voice signal and sends out a second voice signal.

1103. And the autonomous mobile equipment obtains a control instruction according to the second voice signal, wherein the control instruction is used for instructing the autonomous mobile equipment to patrol at a fixed point.

Illustratively, the autonomous mobile device parses the second voice signal itself, or, parses the voice signal by the voice recognition server to obtain the control instruction. The control instructions are used for instructing the automatic mobile device to patrol, monitor or attend to at a fixed point.

1104. The autonomous mobile device performs a fixed point patrol, monitoring, or care.

For example, the second voice signal emitted by the user is "go dad's room patrol". The autonomous mobile equipment plans a driving path according to the environment map, enters a father room, then moves to a monitoring point set in advance, starts a camera to shoot a video, and transmits the video back to a client on a mobile phone of a user, so that the function of monitoring the old across rooms is realized.

By adopting the scheme, the functions of home fixed-point patrol, monitoring of a specified area, nursing of a specific room and the like of the autonomous mobile equipment according to the intention of a user are realized through voice control.

And finally, controlling the scene of automatic object searching of the autonomous mobile equipment through voice. For example, refer to fig. 12. Fig. 12 is yet another flowchart of an autonomous mobile device voice control method provided by an embodiment of the present application.

The embodiment comprises the following steps:

1201. the autonomous mobile device is in a first operating state and the voice control function is in a wait-for-wake state.

1202. The user awakens the voice control function through the first voice signal and sends out a second voice signal.

1203. And the autonomous mobile equipment obtains a control instruction according to the second voice signal, wherein the control instruction is used for instructing the autonomous mobile equipment to search for the target object.

1204. The autonomous mobile device determines whether the environment map is marked with the position coordinates of the target object, and if the environment map is marked with the target object, the autonomous mobile device executes the step 1205; if no target object is marked in the environmental subway, step 1208 is executed.

Illustratively, the acquired image is input into an AI training model during the working process of the autonomous mobile equipment so as to obtain the position coordinates, the name, the category and the like of the object, and the information is recorded in an environment map for subsequent intelligent object finding. In order to prevent the client interface from being messy, the position coordinates of the objects and the like may not be displayed on the environment map. When the user issues an instruction to find the target object, the client displays the position of the target object in the environment map, and the like.

1205. The autonomous mobile device asks the user whether the user needs to find the target object now through voice, and if the user feeds back that the target object is found now, step 1206 is executed; if the user feedback does not require to now find the target object, step 1207 is performed.

1206. The user is guided to the target object location and the location of the target object is displayed on the environment map.

1207. Displaying the location of the target object on the environment map, and voice-prompting the user to: the location of the target object has been displayed at the client.

And searching the position of the target object found in the running process in the environment map.

1208. And voice prompting the user: if the target object is not found, please change a description mode.

For example, the first speech signal is: "please help me find socks". After the autonomous mobile device recognizes the first voice signal, it determines whether the position coordinates of the sock have been locally marked. If the socks are not marked, the user is prompted that the socks cannot be found. If the position coordinates of the sock are marked, an inquiry voice is sent out: "whether to look for socks now". If the answer of the user is a positive answer such as "ok", "good", etc., then, the autonomous mobile device travels to guide the user to the location where the sock is located, and displays the location coordinates of the sock on the interface of the client environment map. If the user's response is a negative response such as "not required," the autonomous mobile device need only instruct the client to display the location coordinates of the socks.

By adopting the scheme, the AI training model is trained through machine learning, and the AI training model is utilized to identify the specific object, so that a user can search some specific objects through the voice-controlled autonomous mobile equipment, and marks the specific objects on a map or brings the user to the object site, thereby realizing the purpose of intelligent object searching, expanding the application of the autonomous mobile equipment and improving the intelligence of the autonomous mobile equipment.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Fig. 13 is a schematic diagram of an autonomous mobile device voice control apparatus according to an embodiment of the present disclosure. The autonomous mobile device voice control apparatus 1300 includes: an acquisition module 1301, a processing module 1302 and an execution module 1303.

An acquisition module 1301, configured to acquire a first voice signal;

a processing module 1302, configured to wake up a voice control function of the autonomous mobile device when the first voice signal matches a wake-up instruction of the autonomous mobile device;

the collecting module 1301 is further configured to collect a second voice signal;

the processing module 1302 is configured to determine at least two working areas according to the second voice signal;

and the executing module 1303 is configured to sequentially execute the tasks indicated by the second voice signals to the at least two working areas.

In a possible design, the executing module 1303 is configured to determine an appearance sequence of each of the at least two working areas in the second speech signal; and executing the tasks indicated by the second voice signals to the at least two working areas in sequence according to the sequence.

In one possible design, the executing module 1303 is configured to determine a distance between the autonomous mobile apparatus and each of the at least two work areas; sequencing the at least two working areas according to the sequence from near to far to obtain a queue; and executing the tasks indicated by the second voice signals to the at least two working areas in sequence according to the queues.

In a possible design, the executing module 1303 is configured to stop executing the task after the task is executed on one of the at least two work areas and before the next work area is reached.

In one possible design, the processing module 1302 is configured to determine a region category according to the second speech signal; and determining the at least two working areas from the area set corresponding to the environment map according to the area types.

In a possible design, when the processing module 1302 determines the at least two working areas from the environment map according to the area category, the processing module is configured to determine an area containing the target object from the environment map by taking the target object as a center to obtain the at least two working areas when the area category indicates the target object.

Referring to fig. 13 again, in a possible design, the above-mentioned autonomous mobile device voice control apparatus 1300 further includes: a transceiver module 1304.

Before the processing module 1302 determines at least two working areas according to the second voice signal, the processing module is further configured to divide the environment map into a plurality of working areas according to the environment map, the position information of the object in the environment map, or the position information of the door in the environment map to obtain the area set, and update the identifier of each working area in the area set;

the transceiver module 1304 is configured to send update information to the speech recognition server, so that the speech recognition server updates the identifier of each working area.

In a possible design, the processing module 1302 is further configured to control the autonomous mobile apparatus to switch from a first operating state to a second operating state when the first voice signal matches a wake-up command of the autonomous mobile apparatus, where a volume of sound generated by the autonomous mobile apparatus in the second operating state is smaller than a volume of sound generated in the first operating state, and the wake-up command is used to wake up a voice control function of the autonomous mobile apparatus; the collecting module 1301 is configured to collect the second voice signal in the second working state.

In one possible design, the processing module 1302 is further configured to determine a sound source location of the first voice signal when the first voice signal matches a wake-up instruction of the autonomous mobile device; the autonomous mobile equipment is controlled to be switched from a first position to a second position according to the position of the sound source, the distance between the microphone and the position of the sound source when the autonomous mobile equipment is in the second position is smaller than the distance between the microphone and the position of the sound source when the autonomous mobile equipment is in the first position, and the microphone is arranged on the autonomous mobile equipment.

In one possible design, the processing module 1302 is configured to determine a rotation angle according to the sound source position when the autonomous mobile apparatus is controlled to switch from the first posture to the second posture according to the sound source position, wherein the rotation angle is used for indicating an angle that the autonomous mobile apparatus needs to rotate when the autonomous mobile apparatus is switched from the first posture to the second posture; dividing the rotation angle into a first angle and a second angle; rotating at a first speed within the first angle and at a second speed within the second angle, the first speed being greater than the second speed.

In a feasible design, when the first voice signal is matched with a wake-up instruction of the autonomous mobile device, the processing module 1302 is further configured to control the voice control function to enter a wait wake-up state after a preset time period, after controlling the autonomous mobile device to switch from the first working state to the second working state according to the sound source position.

In one possible design, the executing module 1303 is configured to determine whether the self-status of the autonomous mobile device meets the requirement of performing the task; and if the self state does not meet the requirement of executing the task, maintaining the autonomous mobile equipment.

In one possible design, the processing module 1302 is configured to determine the forbidden task zone from the environment map and determine the at least two work areas from areas other than the forbidden task zone when the second voice signal indicates the forbidden task zone.

In one possible design, the processing module 1302 is configured to determine a region category according to the second speech signal; collecting an image; and determining the area corresponding to the area type from the image to obtain the at least two working areas.

In a possible design, the executing module 1303 is configured to determine an operation manner according to the area category; and executing tasks to the at least two working areas in sequence according to the operation mode.

In a possible design, the processing module 1302 is further configured to determine whether the task is completed in an initial area after the task indicated by the second voice signal is sequentially executed in the at least two working areas, where the initial area is located when the autonomous mobile apparatus acquires the second voice signal; and if the task is not executed in the initial area, returning to the initial area to execute the task.

The voice control apparatus for autonomous mobile device provided in the embodiment of the present application may perform the actions of the autonomous mobile device in the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.

Fig. 14 is a schematic diagram of another autonomous mobile device voice control apparatus according to an embodiment of the present application. The autonomous mobile device voice control apparatus 1400 includes: acquisition module 1401, processing module 1402, travel module 1403, and execution module 1404.

An acquisition module 1401, configured to acquire a second voice signal;

a processing module 1402, configured to determine a working area according to the second voice signal; determining a driving path according to the current position and the working area;

a travel module 1403, configured to turn off the work module and travel to the work area according to the travel path;

an executing module 1404, configured to start the working module to execute the task indicated by the second voice signal if the autonomous mobile device travels to the working area.

In a possible design, the driving module 1403 is configured to determine whether the length of the driving path is greater than a preset length, and if the length of the driving path is greater than the preset length, close the working module and advance to the working area according to the driving path.

In one possible design, after the execution module 1404 starts the work module to perform the task indicated by the second voice signal, it is further configured to determine whether the task is completed for an initial area, where the autonomous mobile device was located when the autonomous mobile device collected the second voice signal; and if the task is not executed in the initial area, returning to the initial area to execute the task.

In a possible design, when the processing module 1402 determines the working area according to the second voice signal, it is configured to determine the area category according to the second voice signal; collecting an image; and determining a region corresponding to the region type from the image to obtain the working region.

Fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 8, the electronic device 1500 is, for example, the autonomous mobile device described above, and the autonomous mobile device 1500 includes:

a processor 1501 and memory 1502;

the memory 1502 stores computer instructions;

the processor 1501 executes the computer instructions stored by the memory 1502 such that the processor 1501 executes the autonomous mobile device voice control method implemented by the autonomous mobile device as described above.

For a specific implementation process of the processor 1501, reference may be made to the above method embodiments, which have similar implementation principles and technical effects, and this embodiment is not described herein again.

Optionally, the electronic device 1500 further comprises a communication component 1503. The processor 1501, the memory 1502, and the communication section 1503 may be connected by a bus 1504.

Embodiments of the present application also provide a computer-readable storage medium having stored therein computer instructions, which when executed by a processor, are used to implement the autonomous mobile device voice control method implemented by the autonomous mobile device as described above.

Embodiments of the present application also provide a computer program product, which contains a computer program that, when executed by a processor, implements the autonomous mobile device voice control method implemented by the autonomous mobile device as described above.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings and described above, and that various modifications and changes can be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An autonomous mobile device voice control method, applied to an autonomous mobile device, the method comprising:

collecting a first voice signal;

when the first voice signal matches a wake-up instruction of the autonomous mobile device, waking up a voice control function of the autonomous mobile device;

collecting a second voice signal in the voice control function awakening state;

determining at least two working areas according to the second voice signal;

2. The method of claim 1, wherein the performing the task indicated by the second voice signal for the at least two work areas in sequence comprises:

determining the sequence of the occurrence of each working area in the at least two working areas in the second voice signal;

and executing the tasks indicated by the second voice signals to the at least two working areas in sequence according to the sequence.

3. The method of claim 1, wherein the performing the task indicated by the second voice signal for the at least two work areas in sequence comprises:

determining a distance between the autonomous mobile device and each of the at least two work areas;

sequencing the at least two working areas according to the sequence from near to far to obtain a queue;

and executing the tasks indicated by the second voice signals to the at least two working areas in sequence according to the queue.

4. A method according to any of claims 1-3, wherein said performing the task indicated by the second speech signal for the at least two work areas in sequence comprises:

and stopping executing the task before moving to the next working area after executing the task on one of the at least two working areas.

5. The method according to any of claims 1-3, wherein said determining at least two working regions from said second speech signal comprises:

determining a region type according to the second voice signal;

and determining the at least two working areas from the area set corresponding to the environment map according to the area types.

6. The method of claim 5, wherein determining the at least two work areas from the environmental map based on the area categories comprises:

and when the area type indicates a target object, determining an area containing the target object from the environment map by taking the target object as a center so as to obtain the at least two working areas.

7. The method of claim 5, wherein prior to determining at least two working regions from the second speech signal, further comprising:

dividing the environment map into a plurality of working areas according to the environment map, the position information of objects in the environment map or the position information of doors in the environment map so as to obtain the area set;

updating the identification of each working area in the area set;

and sending update information to a voice recognition server so that the voice recognition server updates the identification of each working area.

8. The method according to any one of claims 1-3, wherein said collecting a second voice signal in said voice control function awake state comprises:

when the first voice signal is matched with a wake-up instruction of the autonomous mobile device, controlling the autonomous mobile device to be switched from a first working state to a second working state, wherein the volume of sound generated by the autonomous mobile device in the second working state is smaller than that of the sound generated in the first working state, and the wake-up instruction is used for waking up a voice control function of the autonomous mobile device;

and acquiring the second voice signal in the second working state.

9. The method of claim 8, further comprising:

determining a sound source location of the first voice signal when the first voice signal matches a wake-up instruction of the autonomous mobile device;

the autonomous mobile equipment is controlled to be switched from a first position to a second position according to the position of the sound source, the distance between the microphone and the position of the sound source when the autonomous mobile equipment is in the second position is smaller than the distance between the microphone and the position of the sound source when the autonomous mobile equipment is in the first position, and the microphone is arranged on the autonomous mobile equipment.

10. The method of claim 9, wherein the controlling the autonomous mobile device to switch from a first pose to a second pose as a function of the sound source position comprises:

determining a rotation angle according to the sound source position, wherein the rotation angle is used for indicating an angle which needs to be rotated when the autonomous mobile device is switched from the first pose to the second pose;

dividing the rotation angle into a first angle and a second angle;

rotating at a first speed within the first angle and at a second speed within the second angle, the first speed being greater than the second speed.

11. The method of claim 9, wherein after controlling the autonomous mobile device to switch from the first operating state to the second operating state based on the location of the sound source when the first voice signal matches a wake-up command of the autonomous mobile device, further comprising:

and after the preset time length, controlling the voice control function to enter a waiting awakening state.

12. The method according to any one of claims 1-3, wherein prior to performing the task indicated by the second voice signal for the at least two work areas in sequence, further comprising:

determining whether the self-state of the autonomous mobile device meets the requirement of executing a task;

and if the self state does not meet the requirement of executing the task, maintaining the autonomous mobile equipment.

13. The method according to any of claims 1-3, wherein said determining at least two working regions from said second speech signal comprises:

when the second voice signal indicates a task forbidden area, the task forbidden area is determined from an environment map, and the at least two working areas are determined from areas outside the task forbidden area.

14. The method according to any of claims 1-3, wherein said determining at least two working regions from said second speech signal comprises:

determining a region type according to the second voice signal;

collecting an image;

and determining the region corresponding to the region type from the image to obtain the at least two working regions.

15. The method of claim 14, wherein performing the tasks indicated by the second voice signal for the at least two work areas in sequence comprises:

determining an operation mode according to the region type;

and executing tasks to the at least two working areas in sequence according to the operation mode.

16. The method according to any one of claims 1-3, wherein after performing the task indicated by the second voice signal for the at least two work areas in sequence, further comprising:

determining whether a task is performed for an initial area, the initial area being an area where the autonomous mobile device is located when acquiring the second voice signal;

and if the task is not executed in the initial area, returning to the initial area to execute the task.

17. An autonomous mobile device voice control apparatus, comprising:

the acquisition module is used for acquiring a first voice signal;

a processing module to wake up a voice control function of the autonomous mobile device when the first voice signal matches a wake-up instruction of the autonomous mobile device;

18. An electronic device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein execution of the computer program by the processor causes the electronic device to carry out the method of any one of claims 1 to 16.

19. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 16.