WO2019169591A1

WO2019169591A1 - Method and device for voice interaction

Info

Publication number: WO2019169591A1
Application number: PCT/CN2018/078362
Authority: WO
Inventors: 魏建宾; 余尚春
Original assignee: 华为技术有限公司
Priority date: 2018-03-07
Filing date: 2018-03-07
Publication date: 2019-09-12
Also published as: CN111819626A

Abstract

Provided in an embodiment of the present application is a method for voice interaction, comprising: a voice equipment determining the number of people within a space in which the voice equipment is located; when the voice equipment determines that the number of people in the space is one, the voice equipment entering a non-awakening voice interaction mode. Compared with the existing technology, awakening actions on a smart equipment may be reduced when the smart equipment must be operated by means of using the solution provided by using the present invention.

Description

Method and device for voice interaction

Technical field

The embodiments of the present invention relate to the field of communications technologies, and in particular, to a voice prompting method and apparatus.

Background technique

Artificial intelligence technology is widely used in in-vehicle smart devices, but the smart car devices in the current market are equipped with voice control functions, but all need to wake up before use (such as "Xiao Chi Hello"), and the recognition is not sensitive enough. The common phenomenon of too complicated interaction has become one of the biggest problems for users of car smart rearview mirrors and other products.

If you do not use wake-up words to wake up, there are cases of frequent false wake-ups, especially when chatting with other people, the device mistakenly thinks that it is responding to the instructions issued by it, which is very embarrassing. At present, some manufacturers propose to adopt the "wake-up word + speech semantic recognition" integration method to achieve zero-interval, zero-delay, seamless docking between wake-up words and voice manipulation, abandon the traditional one-question-answer form, and reduce user voice control. Steps and complicated wake-up actions. For the user, when using each function of the smart device, whether it is first wake up and then answer the question or wake up the question and answer integration, because there will be a wake-up action, the interaction between the in-vehicle device and the user is too complicated.

Summary of the invention

Embodiments of the present application provide a method and apparatus for voice prompting, which can reduce the wake-up action to the smart device when the smart device needs to be operated.

In one aspect, the embodiment of the present invention provides a method for voice interaction, including: a voice device determining a number of people in a space where the voice device is located; and when the voice device determines that the number of people in the space is one, the voice device Enter the wake-free voice interaction mode.

In a possible design, the voice device determines the number of people in the space where the voice device is located, including: the voice device according to one or more of voiceprint information, iris information, portrait information, fingerprint information, and sensing data. Judging the number of people in the space where the voice device is located. The voice device recognizes the number of people in the space where the voice device is located in a plurality of ways, and comprehensively judges the accuracy of identifying the number of people in the space where the voice device is located.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the voiceprint information, including: the voice device collects the first voice in the space; and the voice device determines that the voice device is received. Whether the second voice is received within the first time period after the first voice, the second voice and the first voice have different voiceprint characteristics; if the voice device is not in the first time Receiving the second voice within the segment determines that there is one person in the space. It is a common way for a voice device to identify the number of people in the space where the voice device is located through different voiceprints. In this example, there is one person in the space.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the voiceprint information, including: the voice device collects a first voice in the space; if the first voice is not Describe a specific instruction, the voice device determines whether a second voice is received within a first time period after receiving the first voice, and the second voice and the first voice have different voiceprint characteristics And if the voice device does not receive the second voice within the first time period, determining that there is a person in the space. The voice device further determines whether the second voice is received within the first time period after receiving the first voice, by determining that the collected first voice is not a specific command, and can more accurately determine the space in the space where the voice device is located. Number of people. The first voice in this article can be the first voice.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the voiceprint information, including: the voice device collects a first voice in the space; if the first voice is not Describe a specific instruction, the voice device determines whether a second voice is received within a first time period after receiving the first voice, and the second voice has a different voiceprint with the first voice Characteristic; if the voice device receives the second voice within the first time period, the smart voice device determines that there are multiple people in the space. The voice device further determines whether the second voice is received within the first time period after receiving the first voice, by determining that the collected first voice is not a specific command, and can more accurately determine the space in the space where the voice device is located. Number of people.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the iris information, including: the voice device obtains an image recognized by the iris through a camera in a space where the voice device is located; the voice device Determining whether there is different iris information in the image; when the voice device determines that there is different iris information, the voice device determines that there are multiple people in the space where the voice device is located; when the voice device determines that there is only one type In the case of iris information, the voice device determines that the space in which the voice device is located is one person. In this example, the voice device determines the number of people in the space where the voice device is located according to the iris information, and adds a method for determining the number of people in the space where the voice device is located.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the portrait information, including: the voice device obtains portrait information through a camera in a space where the voice device is located; the voice device Determining whether there is different portrait information in the image; when the voice device determines that there is different portrait information, the voice device determines that there are multiple people in the space where the voice device is located; when the voice device determines that there is only one type In the case of portrait information, the voice device determines that the space in which the voice device is located is one person. In this example, the voice device determines the number of people in the space where the voice device is located according to the portrait information, and adds a method for determining the number of people in the space where the voice device is located.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the fingerprint information, including: the voice device obtains fingerprint information by using a fingerprint identification device in a space where the voice device is located; the voice device Determining whether there is different fingerprint information in the image; when the voice device determines that there is different fingerprint information, the voice device determines that there are multiple people in the space where the voice device is located; when the voice device determines that there is only one type of In the case of fingerprint information, the voice device determines that the space in which the voice device is located is one person. In this example, the voice device determines the number of people in the space where the voice device is located according to the fingerprint information, and adds a method for determining the number of people in the space where the voice device is located.

In a possible design, the voice device determines the number of people in the space where the voice device is located according to the sensing data, including: the voice device obtains the sensing data through the sensing device in the space where the voice device is located; the voice device determines Whether there is different sensing data in the image; when the voice device determines that there is different sensing data, the voice device determines that there are multiple people in the space where the voice device is located; when the voice device determines that there is only one sensing In the case of data, the voice device determines that the space in which the voice device is located is one person. In this example, the voice device determines the number of people in the space where the voice device is located according to the sensing data, and adds a method for determining the number of people in the space where the voice device is located.

In a possible design, after the voice device enters the wake-free voice interaction mode, the method further includes: the voice device receives a third voice, the third voice does not include an awakening word; Performing a function corresponding to the third voice. In this example, after the voice device enters the wake-free voice interaction mode, the third voice that does not include the wake-up word is recognized, and the corresponding function is executed, so that the number of voice interactions of the wake-up word can be reduced.

In a possible design, when the voice device determines that there are multiple people in the space, the voice device enters a wake-up voice interaction mode; the voice device receives an wake-up word or a fourth voice including an wake-up word; The voice device enters a voice interaction mode or voice recognition and performs a function corresponding to the fourth voice. In this example, the voice device enters the wake-up voice interaction mode, and the voice interaction based on the wake-up word can be implemented.

In one possible design, the space in which the voice device is located is an enclosed space, a semi-enclosed space, or an open space.

In one possible design, the semi-enclosed space or open space is a spherical space having a radius of communication distance of the voice device.

In a possible design, if the radius of the closed space is less than or equal to the communication distance of the voice device, the space in which the voice device is located is the closed space.

On the other hand, the embodiment of the invention provides a voice device, which has the function of realizing the behavior of the voice device in the actual method. The functions may be implemented by hardware or by corresponding software implemented by hardware. The hardware or software includes one or more modules corresponding to the functions described above.

In still another aspect, an embodiment of the present invention provides a computer storage medium for storing computer software instructions for use in the voice device, including a program designed to perform the above aspects.

Compared with the prior art, the solution provided by the present invention can reduce the wake-up action on the smart device when the smart device needs to be operated.

DRAWINGS

1 is a system architecture diagram of a mobile vehicle communication system according to an embodiment of the present invention;

2 is a functional block diagram of an exemplary vehicle 12 according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a terminal according to an embodiment of the present disclosure;

4 is a schematic structural view of the audio circuit 305 of FIG. 3;

FIG. 5 is a flowchart of voice interaction according to an embodiment of the present invention;

FIG. 6 is a flowchart of determining a number of people in a vehicle according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of interaction between a person and a vehicle-mounted smart rearview mirror according to an embodiment of the present invention; FIG.

FIG. 8 is another schematic diagram of interaction between a person and a vehicle-mounted smart rearview mirror according to an embodiment of the present invention; FIG.

FIG. 9 is a schematic diagram of a vehicle-mounted smart speaker according to an embodiment of the present invention; FIG.

FIG. 10 is a schematic diagram of a vehicle-mounted smart TV according to an embodiment of the present invention; FIG.

FIG. 11 is a schematic diagram of a home smart speaker according to an embodiment of the present invention; FIG.

FIG. 12 is a schematic diagram of interaction between a person and a home smart television according to an embodiment of the present invention; FIG.

Figure 13 is a schematic diagram of two enclosed spaces provided by an embodiment of the present invention;

FIG. 14 is a schematic structural diagram of a processor according to an embodiment of the present invention.

Detailed ways

In the following, the terms "first" and "second" are used for descriptive purposes only, and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, "multiple" means two or more unless otherwise stated.

The detailed description below describes various features and functions of the disclosed systems and methods with reference to the drawings. In the figures, similar symbols identify similar components unless the context indicates otherwise. The illustrative system and method embodiments described herein are not intended to be limiting. It will be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a variety of different configurations, all of which are contemplated herein.

Referring to FIG. 1, FIG. 1 is a system architecture diagram of a mobile vehicle communication system according to an embodiment of the present invention. Among other things, communication system 10 includes vehicle 12, one or more wireless carrier systems 14, terrestrial communication network 16, computer 18, and call center 20. It should be understood that the disclosed methods can be used with any number of different systems and are not specifically limited to the operating environments shown herein. Likewise, the architecture, construction, setup, and operation of system 10, as well as its individual components, are generally known in the art. Thus, the following paragraphs simply provide an overview of an example communication system 10, and other systems not shown herein can also use the disclosed methods.

The vehicle 12 can be implemented on a car or in the form of a car. However, the example system can also be implemented on other vehicles or in the form of other vehicles, such as cars, trucks, motorcycles, buses, boats, airplanes, helicopters, lawn mowers, snow shovels, recreational vehicles, amusement park vehicles. Other equipment such as agricultural equipment, construction equipment, trams, golf carts, trains and trams. In addition, the robotic device can also be used to perform the methods and systems described herein.

Some vehicle hardware 28 is shown in FIG. 1, including an information communication unit 30, a microphone 32, one or more buttons or other control inputs 34, an audio system 36, a visual display 38, and a global position system (global position system, GPS) module 40 and a plurality of vehicle security modules (VSMs) 42. Some of these devices can be directly connected to an information communication unit, such as microphone 32 and button 34, while others use one or more network connections to make an indirect connection, such as communication bus 44 or entertainment bus 46. Examples of suitable network connections include controller area network (CAN), media oriented systems transport (MOST), local interconnect network (LIN), local area network (LAN) And other suitable connections, such as Ethernet or conforming to the International Organization for Standardization (ISO), the Society of Automotive Engineers (SAE), and the Institute of Electrical and Electronics Engineers (institute of electrical) And electronics engineers, IEEE) other connections to the standards and regulations, just to name a few.

The information communication unit 30 may be an original equipment manufacturer (OEM) installation (embedded) or an accessory market device installed in the vehicle and capable of wirelessly sounding and/or wirelessly networked on the wireless carrier system 14. Or data communication. This enables the vehicle to communicate with the call center 20, other information-enabled vehicles, or some other entity or device. The information communication unit preferably uses radio broadcasts to establish a communication channel (sound channel and/or data channel) with the wireless carrier system 14 such that sound and/or data transmissions can be transmitted and received over the channel. By providing voice and data communications, the messaging unit 30 enables the vehicle to provide a variety of different services, including those associated with navigation, telephony, emergency, diagnostics, infotainment, and the like. Data can be transmitted over a data connection (e.g., via packet data over a data channel, or via a voice channel using techniques known in the art). For combined services including voice communication (eg, having a field advisor or voice response unit at call center 20) and data communication (eg, providing GPS location data or vehicle diagnostic data to call center 20), the system can utilize the sound A single call on the channel and switching between sound and data transmission on the sound channel as needed, which can be done using techniques known to those skilled in the art. In addition, the short message service SMS can be used to send and receive data (eg, a packet data protocol (PDP)); the information communication unit can be configured to be mobile terminated and/or initiated, or configured to apply termination and/or initiate.

The information communication unit 30 utilizes cellular communication according to a global system for mobile communication (GSM) or code division multiple access (CDMA) standard, and thus includes standards for voice communication (eg, hands-free calling). A set of cells 50, a wireless modem for data transmission, an electronic processing device 52, one or more digital memory devices 54, and a dual antenna 56. It should be understood that the modem can be implemented by software stored in the information communication unit and executed by the processor 52, or it can be a separate hardware component located inside or outside the information communication unit 30. The modem can use any number of different standards or protocols (eg EVDO (CDMA2000 1xEV-DO, EVDO), CDMA, general packet radio service (GPRS) and enhanced data rate for GSM) (enhanced data rate for GSM) Evolution, EDGE)) to run. Wireless networking between the vehicle and other networked devices can also be performed using the information communication unit 30. To this end, the information communication unit 30 can be configured to wirelessly communicate according to one or more wireless protocols (eg, any of IEEE 802.11 protocols, worldwide interoperability for microwave access (WiMAX), or Bluetooth) . When used for packet switched data communication such as transmission control protocol/Internet protocol (TCP/IP), the information communication unit can be configured to have a static IP address or can be set to be from the network. Another device (such as a router) or automatically receives the assigned IP address from a network address server.

Processor 52 can be any type of device capable of processing electronic instructions, including microprocessors, microcontrollers, host processors, controllers, vehicle communication processors, and application specific integrated circuits (ASICs). It can be a dedicated processor for the information communication unit 30 only or can be shared with other vehicle systems. Processor 52 executes various types of digital storage instructions, such as software or firmware programs stored in memory 54, which enables the information communication unit to provide a wide variety of services. For example, processor 52 can execute a program or process data to perform at least a portion of the methods discussed herein.

The information communication unit 30 can be used to provide a range of vehicle services, including wireless communication with other portions from the vehicle. Such services include: turn-by-turn direct 1 ns and other navigation-related services provided in conjunction with GPS-based vehicle navigation module 40; airbag deployment notifications and interface modules with one or more collision sensors (eg, The main body control module (not shown) combines the services provided by other emergency or roadside rescues. Diagnostic reports using one or more diagnostic modules. And entertainment related services, where music, web pages, movies, television shows, video games, and/or other information are downloaded by the infotainment module and stored for current or later playback. The services listed above are by no means an exhaustive list of all the capabilities of the messaging unit 30, but merely an enumeration of some of the services that the messaging unit can provide. Moreover, it should be understood that at least some of the above modules can be implemented in the form of software instructions stored inside or outside of the information communication unit 30, which may be hardware components located inside or outside the information communication unit 30, or they may be integrated with each other. / or shared, or integrated and / or shared with other systems located throughout the vehicle, just to name a few possibilities. Where the VSMs 42 located outside of the information communication unit 30 are in operation, they can exchange data and commands with the information communication unit 30 using the vehicle bus 44.

The GPS module 40 receives radio signals from the GPS satellites 60. From these signals, the GPS module 40 is able to determine the location of the vehicle that is used to provide the vehicle driver with navigation and other location-associated services. The navigation information can be presented on display 38 (or other display within the vehicle) or can be rendered in a language, such as when providing steering navigation. A navigation module (which may be part of the GPS module 40) within the dedicated vehicle can be used to provide navigation services, or some or all of the navigation services can be completed via the information communication unit 30, where the location information is sent to a remote location to facilitate Provide navigation maps, map annotations (points of interest, restaurants, etc.), route calculations, etc. for vehicles. The location information can be provided to call center 20 or other remote computer system, such as computer 18, for other purposes, such as fleet management. And, new or updated map data can be downloaded from the call center 20 to the GPS module 40 via the information communication unit 30.

In addition to the audio system 36 and the GPS module 40, the vehicle 12 can include other vehicle security modules VSM in the form of electronic hardware components. The other vehicle security modules VSM 42 are located throughout the vehicle, typically receiving input from one or more sensors. And use the sensed inputs to perform diagnostics, monitoring, control, reporting, and/or other functions. Each of the VSMs 42 is preferably connected to other VSMs via a communication bus 44, also to the information communication unit 30, and can be programmed to run vehicle system and subsystem diagnostic tests. For example, one VSM 42 can be an engine control module (ECM) that controls various aspects of engine operation (eg, fuel ignition and ignition timing), and the other VSM 42 can be one that regulates the powertrain of the vehicle or The operating powertrain control module of the plurality of components, and the other VSM 42 can be a body control module that manages various electrical components (like the power door locks and headlights of the vehicle) located throughout the vehicle. According to one embodiment, the engine control module is equipped with on board diagnostics (OBD) features that provide a large amount of real-time data, such as data received from various sensors (including vehicle emission sensors), and provide a standardized series Diagnostic Trouble Code (DTS), which diagnoses fault codes allows technicians to quickly identify and repair faults within the vehicle. As will be appreciated by those skilled in the art, the VSMs mentioned above are merely examples of some of the modules that may be used within the vehicle 12, and many other modules are also possible.

The vehicle electronics 28 also includes a plurality of vehicle user interfaces that provide vehicle occupants with means for providing and/or receiving information, including a microphone 32, buttons 34, an audio system 36, and a visual display 38. As used herein, the term "vehicle user interface" broadly includes any suitable form of electronic device, including hardware and software components that are located on a vehicle and that enable the vehicle user to communicate with components of the vehicle. Or communicate through the components of the vehicle. The microphone 32 provides an audio input to the information communication unit to enable the driver or other occupant to provide voice commands and perform an hands-free escort via the wireless carrier system 14. For this purpose, it can be connected to an in-vehicle automated sound processing unit that utilizes human machine interface (HMI) technology known in the art. Button 34 allows a manual user to input to messaging unit 30 to initiate a wireless telephone call and provide other data, response or control inputs. Separate buttons can be used to initiate an emergency call as well as a regular service call to call center 20. The audio system 36 provides audio output to the vehicle occupant and can be a dedicated stand-alone system or part of the host vehicle audio system. In accordance with the particular embodiment illustrated herein, audio system 36 is operatively coupled to vehicle bus 44 and entertainment bus 46, and is capable of providing amplitude modulation (AM), frequency modulation (FM), and satellite broadcast, digital. Digital versatile disc (DVD) and other multimedia features. This functionality can be provided in conjunction with or separately from the infotainment module described above. Visual display 38 is preferably a graphical display, such as a touch screen on a dashboard or a heads-up display that is reflected from a windshield, and can be used to provide a variety of input and output functions. Various other vehicle user interfaces can also be utilized, as the interface in Figure 1 is merely an example of a particular implementation.

The wireless carrier system 14 is preferably a cellular telephone system comprising a plurality of cellular towers 70 (only one shown), one or more mobile switching centers (MSCs) 72, and a wireless carrier system 14 coupled to the terrestrial network 16. Any other networking components required. Each of the cellular towers 70 includes transmit and receive antennas and base stations, and base stations from different cellular towers are directly connected to the MSC 72 or to the MSC 72 via intermediate devices (e.g., base station controllers). Cellular system 14 may implement any suitable communication technology including, for example, analog technologies (e.g., an advanced mobile phone system (AMPS)) or newer digital technologies (e.g., CDMA (e.g., CDMA2000) or GSM/GPRS). As will be appreciated by those skilled in the art, various cellular tower/base station/MSC settings are possible and can be used with wireless system 14. For example, the base station and the cellular tower can be co-located at the same location, or they can be located farther from each other, each base station can respond to a single cellular tower or a single base station can serve each cellular tower, and each base station can be coupled to a single MSC, which is merely an example Give a small set of possible settings.

In addition to using the wireless carrier system 14, different wireless carrier systems in the form of satellite communications can be used to provide one-way or two-way communication with the vehicle. This can be done using one or more communication satellites 62 and an uplink transmitting station 64. The one-way communication can be, for example, a satellite broadcast service in which program content (news, music, etc.) is received by the transmitting station 64, packaged for uploading, and then transmitted to the satellite 62, which broadcasts the program to the user. The two-way communication can be, for example, a satellite telephone service that relays telephone communications between the vehicle 12 and the station 64 using the satellite 62. If used, such a satellite phone can be attached to or used in place of the wireless carrier system 14.

The terrestrial network 16 may be a conventional land-based radio communication network that is coupled to one or more fixed telephones and that connects the wireless carrier system 14 to the call center 20. For example, terrestrial network 16 may include a public switched telephone network (PSTN), such as a PSTN that is used to provide wired telephone, packet switched data communications, and Internet infrastructure. One or more portions of terrestrial network 16 can be accessed using standard wired networks, fiber optic or other optical networks, cable networks, power lines, other wireless networks (eg, wireless local area networks (WLAN)), or providing broadband wireless access. (broadband wireless access, BWA) network and any combination thereof to implement. The terrestrial network 16 may also include one or more short message service centers (SMSCs) for storing, uploading, converting, and/or transmitting short messages (SMS) between the sender and the receiver. ). For example, the SMSC can receive an SMS message from the call center 20 or a content provider (eg, an external short message entity or ESME), and the SMSC can transmit the SMS message to the vehicle 12 (eg, a mobile terminal device). SMSCs and their functions are known to the skilled person. Moreover, call center 20 need not be connected via terrestrial network 16, but may include a wireless telephone device such that it can communicate directly with a wireless network (e.g., wireless carrier system 14).

Computer 18 can be one of a plurality of computers that are accessible via a private or public network, such as the Internet. Each such computer 18 can be used for one or more purposes, such as a vehicle that can access a web server via the information communication unit 30 and the wireless carrier 14. Other such accessible computers 18 can be, for example, a service center computer in which diagnostic information and other vehicle data can be uploaded from the vehicle via the information communication unit 30; the vehicle owner or other user is a client for use, for example, for the following purposes Computer: accessing or receiving vehicle data, or setting or configuring user parameters, or controlling the functionality of the vehicle; or third party library, whether by communicating with the vehicle 12 or the call center 20, or communicating with both, vehicle data Or other information is provided to or from the third party library. The computer 18 can also be used to provide an internet connection, such as a domain name server (DNS) service, or as a means of assigning an IP address to a vehicle using a dynamic host configuration protocol (DHCP) or other suitable protocol. 12 network address server.

The call center 20 is designed to provide a variety of different system backend functions to the vehicle electronics 28, and according to the exemplary embodiment shown herein, the call center 20 typically includes one or more switches 80, servers 82, databases 84. On-site consultant 86, and automatic voice response system (VRS) 88, all of which are known in the prior art. These various call center components are preferably coupled to each other via a wired or wireless local area network 90. Switch 80 can be a private branch exchange (PBX) that routes incoming signals such that voice transmissions are typically sent to field consultant 86 via a regular telephone or to automated voice response system 88 using VoIP. The on-site advisory phone can also use voice over Internet phone (VoIP), as indicated by the dashed line in Figure 1. VoIP and other data communications through switch 80 are implemented via a modem (not shown) connected between switch 80 and network 90. Data transfer is passed to server 82 and/or database 84 via a modem. The database 84 is capable of storing account information such as user authentication information, vehicle identifiers, data profile records, behavioral patterns, and other related user information. Data transmission can also be performed by a wireless system, such as 802.1lx, GPRS, and the like. In addition, short message service (SMS) can be used to send and/or receive data (eg, PDP); and call center 20 can be configured to terminate and/or initiate mobile, or configured to terminate and/or initiate applications. Although the illustrated embodiment has been described as being used with a human-controlled call center 20 using a field advisor 86, it will be appreciated that the call center can instead use VRS 88 as an automated advisor, or VRS 88 and on-site consultant 86. The combination can be used.

2 is a functional block diagram of an example vehicle 12 provided by an embodiment of the present invention. Components coupled to or included in vehicle 12 may include propulsion system 102, sensor system 104, control system 106, peripherals 108, power source 110, computing device 111, and user interface 112. Computing device 111 can include a processor 113 and a memory 114. Computing device 111 may be part of a controller or controller of vehicle 12. The memory 114 can include instructions 115 that the processor 113 can run, and can also store map data 116. The components of the vehicle 12 can be configured to operate in a manner interconnected with each other and/or with other components coupled to the various systems. For example, power source 110 can provide power to all components of vehicle 12. Computing device 111 can be configured to receive data from, and control, propulsion system 102, sensor system 104, control system 106, and peripherals 108. Computing device 111 can be configured to generate a display of images on user interface 112 and receive input from user interface 112.

In other examples, vehicle 12 may include more, fewer, or different systems, and each system may include more, fewer, or different components. Moreover, the systems and components shown may be combined or divided in any number of ways.

The propulsion system 102 can be used to provide power motion to the vehicle 12. As shown, the propulsion system 102 includes an engine/engine 118, an energy source 120, a transmission 122, and a wheel/tire 124.

Engine/engine 118 may be or include any combination of internal combustion engine, electric motor, steam engine, and Stirling engine. Other engines and engines are also possible. In some examples, propulsion system 102 can include multiple types of engines and/or engines. For example, a gas-electric hybrid car may include a gasoline engine and an electric motor. Other examples are possible.

Energy source 120 may be a source of energy that is fully or partially powered to engine/engine 118. That is, the engine/engine 118 can be used to convert the energy source 120 to mechanical energy. Examples of energy source 120 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source(s) 120 may additionally or alternatively include any combination of fuel tanks, batteries, capacitors, and/or flywheels. In some examples, energy source 120 may also provide energy to other systems of vehicle 12 .

Transmission 122 can be used to transfer mechanical power from engine/engine 118 to wheel/tire 124. To this end, the transmission 122 can include a gearbox, a clutch, a differential, a drive shaft, and/or other components. In the example where the transmission 122 includes a drive shaft, the drive shaft includes one or more shafts for coupling to the wheel/tire 124.

The wheel/tire 124 of the vehicle 12 can be configured in a variety of forms, including a single wheeled vehicle, a bicycle/motorcycle, a tricycle, or a car/truck four wheel form. Other wheel/tire forms are also possible, such as those that include six or more wheels. The wheel/tire 124 of the vehicle 12 can be configured to rotate differentially relative to the other wheels/tires 124. In some examples, the wheel/tire 124 can include at least one wheel that is fixedly attached to the transmission 122 and at least one tire that is coupled to the driving surface and that is coupled to the edge of the wheel. Wheel/tire 124 may comprise any combination of metal and rubber, or a combination of other materials.

Propulsion system 102 may additionally or alternatively include components in addition to those shown.

Sensor system 104 may include a number of sensors for sensing information regarding the environment in which vehicle 12 is located. As shown, the sensors of the sensor system include a GPS 126, an inertial measurement unit (IMU) 128, a radio detection and radar ranging (RADAR) unit 130, a laser ranging (LIDAR) unit 132, a camera 134, and Actuator 136 to modify the position and/or orientation of the sensor. Sensor system 104 may also include additional sensors including, for example, sensors that monitor the internal system of vehicle 12 (eg, O2 monitor, fuel gauge, oil temperature, etc.). Sensor system 104 may also include other sensors.

The GPS module 126 can be any sensor for estimating the geographic location of the vehicle 12. To this end, the GPS module 126 may include a transceiver that estimates the position of the vehicle 12 relative to the earth based on satellite positioning data. In an example, computing device 111 can be used in conjunction with map data 116 to use GPS module 126 to estimate the location of a lane boundary on a road on which vehicle 12 can travel. The GPS module 126 can take other forms as well.

IMU 128 may be used to sense changes in position and orientation of vehicle 12 based on inertial acceleration and any combination thereof. In some examples, the combination of sensors can include, for example, an accelerometer and a gyroscope. Other combinations of sensors are also possible.

The RADAR unit 130 can be viewed as an object detection system for detecting the characteristics of an object using radio waves, such as the distance, height, direction or speed of the object. The RADAR unit 130 can be configured to transmit radio waves or microwave pulses that can bounce off any object in the course of the wave. The object may return a portion of the energy of the wave to a receiver (eg, a dish or antenna), which may also be part of the RADAR unit 130. The RADAR unit 130 can also be configured to perform digital signal processing on the received signal (bounce from the object) and can be configured to identify the object.

Other systems similar to RADAR have been used on other parts of the electromagnetic spectrum. One example is LIDAR (Light Detection and Ranging), which can use visible light from a laser instead of radio waves.

The LIDAR unit 132 includes a sensor that uses light to sense or detect objects in the environment in which the vehicle 12 is located. In general, LIDAR is an optical remote sensing technique that can measure the distance to a target or other attribute of a target by illuminating the target with light. As an example, LIDAR unit 132 can include a laser source and/or a laser scanner configured to emit laser pulses, and a detector for receiving reflections of the laser pulses. For example, the LIDAR unit 132 can include a laser range finder that is reflected by a rotating mirror and scans the laser around the digitized scene in one or two dimensions to acquire distance measurements at specified angular intervals. In an example, LIDAR unit 132 may include components such as light (eg, laser) sources, scanners and optical systems, photodetectors, and receiver electronics, as well as position and navigation systems.

In an example, the LIDAR unit 132 can be configured to image an object using ultraviolet (UV), visible, or infrared light, and can be used for a wide range of targets, including non-metallic objects. In one example, a narrow laser beam can be used to map physical features of an object with high resolution.

In an example, wavelengths in the range of from about 10 microns (infrared) to about 250 nanometers (UV) can be used. Light is typically reflected via backscattering. Different types of scattering are used for different LIDAR applications such as Rayleigh scattering, Mie scattering and Raman scattering, and fluorescence. Based on different kinds of backscattering, as an example, LIDAR can thus be referred to as Rayleigh laser RADAR, Mie LIDAR, Raman LIDAR, and sodium/iron/potassium fluorescent LIDAR. Appropriate combinations of wavelengths may allow remote mapping of objects, for example by looking for wavelength dependent changes in the intensity of the reflected signal.

Three-dimensional (3D) imaging can be achieved using both a scanned LIDAR system and a non-scanning LIDAR system. "3D gated viewing laser radar" is an example of a non-scanning laser ranging system that uses a pulsed laser and a fast gating camera. Imaging LIDAR can also use high speed detector arrays that are typically built on a single chip using complementary metal oxide semiconductor (CMOS) and hybrid complementary metal oxide semiconductor/charge coupled device (CCD) fabrication techniques. And modulating the sensitive detector array to perform. In these devices, each pixel can be locally processed by high speed demodulation or gating such that the array can be processed to represent an image from the camera. Using this technique, thousands of pixels can be acquired simultaneously to create a 3D point cloud representing the object or scene detected by the LIDAR unit 132.

A point cloud can include a set of vertices in a 3D coordinate system. These vertices may be defined, for example, by X, Y, Z coordinates and may represent the outer surface of the object. The LIDAR unit 132 can be configured to create a point cloud by measuring a large number of points on the surface of the object, and can output the point cloud as a data file. As a result of the 3D scanning process of the object through the LIDAR unit 132, the point cloud can be used to identify and visualize the object.

In one example, the point cloud can be rendered directly to visualize the object. In another example, a point cloud may be converted to a polygonal or triangular mesh model by a process that may be referred to as surface reconstruction. Example techniques for converting a point cloud to a 3D surface may include a Delaunay triangulation, an alpha shape, and a rotating sphere. These techniques include building a network of triangles on existing vertices of a point cloud. Other example techniques may include converting a point cloud to a volumetric distance field, and reconstructing such an implicit surface as defined by a moving cube algorithm.

Camera 134 can be used to capture any camera (eg, a still camera, video camera, etc.) of an image of the environment in which vehicle 12 is located. To this end, the camera can be configured to detect visible light, or can be configured to detect light from other portions of the spectrum, such as infrared or ultraviolet light. Other types of cameras are also possible. Camera 134 can be a two-dimensional detector or can have a three-dimensional spatial extent. In some examples, camera 134 can be, for example, a distance detector configured to generate a two-dimensional image indicative of the distance from camera 134 to several points in the environment. To this end, camera 134 can use one or more distance detection techniques. For example, camera 134 can be configured to use structured light technology in which vehicle 12 illuminates an object in the environment with a predetermined light pattern, such as a grid or checkerboard pattern, and uses camera 134 to detect reflections from a predetermined light pattern of the object. . Based on the distortion in the reflected light pattern, the vehicle 12 can be configured to detect the distance of a point on the object. The predetermined light pattern may include infrared light or light of other wavelengths.

Actuator 136 can be configured, for example, to modify the position and/or orientation of the sensor. Sensor system 104 may additionally or alternatively include components in addition to those shown.

Control system 106 can be configured to control the operation of vehicle 12 and its components. To this end, control system 106 can include steering unit 138, throttle 140, braking unit 142, sensor fusion algorithm 144, computer vision system 146, navigation or routing system 148, and obstacle avoidance system 150.

Steering unit 138 may be any combination of mechanisms configured to adjust the direction or direction of advancement of vehicle 12.

The throttle 140 may be any combination of mechanisms configured to control the operating speed and acceleration of the engine/engine 118 and thereby control the speed and acceleration of the vehicle 12.

Brake unit 142 may be any combination of mechanisms configured to decelerate vehicle 12. For example, the brake unit 142 can use friction to slow the wheel/tire 124. As another example, the braking unit 142 can be configured to regeneratively convert the kinetic energy of the wheel/tire 124 into a current. Brake unit 142 can take other forms as well.

Sensor fusion algorithm 144 may include, for example, an algorithm (or a computer program product that stores the algorithm) that computing device 111 may operate. Sensor fusion algorithm 144 can be configured to accept data from sensor 104 as an input. The data may include, for example, data representing information sensed at the sensors of sensor system 104. Sensor fusion algorithm 144 may include, for example, a Kalman filter, a Bayesian network, or another algorithm. The sensor fusion algorithm 144 may also be configured to provide various ratings based on data from the sensor system 104, including, for example, an assessment of individual objects and/or features in the environment in which the vehicle 12 is located, an assessment of a particular situation, and/or An assessment based on the likely impact of a particular situation. Other evaluations are also possible.

Computer vision system 146 may be any system configured to process and analyze images captured by camera 134 to identify objects and/or features in the environment in which vehicle 12 is located, such as lane information, traffic, for example Signals and obstacles. To this end, computer vision system 146 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, or other computer vision techniques. In some examples, computer vision system 146 may additionally be configured as a mapping environment, following an object, estimating the speed of an object, and the like.

Navigation and route control system 148 may be any system configured to determine the driving route of vehicle 12. The navigation and route control system 148 can additionally be configured to dynamically update the driving route while the vehicle 12 is in operation. In some examples, navigation and route control system 148 can be configured to combine data from sensor fusion algorithm 144, GPS module 126, and one or more predetermined maps to determine a driving route for vehicle 12.

The obstacle avoidance system 150 can be any system configured to identify, evaluate, and avoid or otherwise cross obstacles in the environment in which the vehicle 12 is located.

Control system 106 may additionally or alternatively include components in addition to those shown.

Peripheral device 108 can be configured to allow vehicle 12 to interact with external sensors, other vehicles, and/or users. To this end, peripheral device 108 can include, for example, wireless communication system 152, touch screen 154, microphone 156, and/or speaker 158.

Wireless communication system 152 can be any system configured to be wirelessly coupled to one or more other vehicles, sensors, or other entities, either directly or via a communication network. To this end, the wireless communication system 152 can include an antenna and chipset for communicating with other vehicles, sensors, or other entities, either directly or through an air interface. The chipset or the entire wireless communication system 152 can be arranged to communicate in accordance with one or more other types of wireless communications (e.g., protocols) such as those described in Bluetooth, IEEE 802.11 (including any IEEE 802.11 revision). Communication protocol, cellular technology (such as GSM, CDMA, universal mobile telecommunications system (UMTS), EV-DO, WiMAX or long term evolution (LTE)), ZigBee, dedicated short-range communication (dedic short) Range communications, DSRC) and radio frequency identification (RFID) communications, and the like. Wireless communication system 152 can take other forms as well.

Touch screen 154 can be used by a user to enter commands into vehicle 12. To this end, the touch screen 154 can be configured to sense at least one of a position and a movement of a user's finger via a capacitive sensing, a resistive sensing, or a surface acoustic wave process or the like. The touch screen 154 may be capable of sensing finger movement in a direction parallel to the touch screen surface or in the same plane as the touch screen surface, in a direction perpendicular to the touch screen surface, or in both directions, and may also be capable of sensing application to The level of pressure on the surface of the touch screen. Touch screen 154 may be formed from one or more translucent or transparent insulating layers and one or more translucent or transparent conductive layers. Touch screen 154 can take other forms as well.

Microphone 156 can be configured to receive audio (eg, a voice command or other audio input) from a user of vehicle 12. Similarly, the speaker 158 can be configured to output audio to a user of the vehicle 12.

Peripheral device 108 may additionally or alternatively include components in addition to those shown.

The power source 110 can be configured to provide power to some or all of the components of the vehicle 12. To this end, the power source 110 can include, for example, a rechargeable lithium ion or lead acid battery. In some examples, one or more battery packs can be configured to provide power. Other power materials and configurations are also possible. In some examples, power source 110 and energy source 120 can be implemented together, as in some all-electric vehicles.

Processor 113 included in computing device 111 may include one or more general purpose processors and/or one or more special purpose processors (eg, image processors, digital signal processors, etc.). Insofar as the processor 113 includes more than one processor, such processors can work individually or in combination. Computing device 111 may implement the function of controlling vehicle 12 based on input received through user interface 112.

The memory 114, in turn, can include one or more volatile storage components and/or one or more non-volatile storage components, such as optical, magnetic, and/or organic storage devices, and the memory 114 can be fully or partially coupled to the processor 113. integrated. Memory 114 may include instructions 115 (eg, program logic) executable by processor 113 to perform various vehicle functions, including any of the functions or methods described herein.

The components of the vehicle 12 can be configured to operate in a manner interconnected with other components internal and/or external to their respective systems. To this end, the components and systems of the vehicle 12 can be communicatively linked together via a system bus, network, and/or other connection mechanism.

FIG. 3 is a schematic structural view of a vehicle according to an embodiment of the present invention. The terminal 300 (taking the vehicle as an example) includes a processor 301, a memory 302, a camera 303, an RF circuit 304, an audio circuit 305, a speaker 306, a microphone 307, an input device 308, other input devices 309, a display screen 310, and a touch. The control panel 311, the display panel 312, the output device 313, and the power source 314 and the like. The display screen 310 is composed of at least a touch panel 311 as an input device and a display panel 312 as an output device. It should be noted that the terminal structure shown in FIG. 3 does not constitute a limitation on the terminal, and may include more or less components than those illustrated, or combine some components, or split some components, or different. The component arrangement is not limited herein.

The components of the terminal 300 will be specifically described below with reference to FIG. 3:

The radio frequency (RF) circuit 304 can be used for transmitting and receiving information or during the call, and receiving and transmitting the signal. For example, if the terminal 300 is an in-vehicle device, the terminal 300 can send the downlink information sent by the base station through the RF circuit 304. After receiving, it is transmitted to the processor 301 for processing; in addition, data related to the uplink is transmitted to the base station. Generally, RF circuits include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like. In addition, RF circuitry 304 can also communicate with the network and other devices via wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (code division) Multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), e-mail, short messaging service (SMS), and the like.

The memory 302 can be used to store software programs and modules, and the processor 301 executes various functional applications and data processing of the terminal 300 by running software programs and modules stored in the memory 302. The memory 302 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (for example, a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored. Data (such as audio data, video data, etc.) created according to the use of the terminal 300, and the like. Moreover, memory 302 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

Other input devices 309 can be used to receive input numeric or character information, as well as to generate key signal inputs related to user settings and function control of terminal 300. Specifically, other input devices 309 may include, but are not limited to, a physical keyboard, function keys (eg, volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, light rats (light mice are touches that do not display visual output) One or more of a sensitive surface, or an extension of a touch sensitive surface formed by a touch screen. The other input device 309 may further include a sensor built in the terminal 300, such as a gravity sensor, an acceleration sensor, etc., and the terminal 300 may also use the parameter detected by the sensor as input data.

The display screen 310 can be used to display information input by the user or information provided to the user as well as various menus of the terminal 300, and can also accept user input. In addition, the display panel 312 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. The touch panel 311 is also called a touch screen or a touch sensitive screen. Etc., it is possible to collect contact or non-contact operations on or near the user (for example, the user can use any suitable object or accessory such as a finger or a stylus on the touch panel 311 or in the vicinity of the touch panel 311, or Including the somatosensory operation; the operation includes a single point control operation, a multi-point control operation and the like, and drives the corresponding connection device according to a preset program. It should be noted that the touch panel 311 may further include two parts: a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation and posture of the user, and detects a signal brought by the touch operation, and transmits a signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into the processor 301. The information that can be processed is transmitted to the processor 301, and the commands sent from the processor 301 can also be received and executed. In addition, the touch panel 311 can be implemented by using various types such as resistive, capacitive, infrared, and surface acoustic waves, and the touch panel 311 can be implemented by any technology developed in the future. In general, the touch panel 311 can cover the display panel 312, and the user can cover the display panel 312 according to the content displayed by the display panel 312 (including but not limited to a soft keyboard, a virtual mouse, a virtual button, an icon, etc.). The operation is performed on or near the touch panel 311. After the touch panel 111 detects the operation thereon or nearby, the touch panel 111 transmits to the processor 301 to determine the user input, and then the processor 301 provides the display panel 312 according to the user input. Corresponding visual output. Although in FIG. 3, the touch panel 311 and the display panel 312 are used as two independent components to implement the input and output functions of the terminal 300, in some embodiments, the touch panel 311 can be integrated with the display panel 312. To implement the input and output functions of the terminal 300.

The RF circuit 304, the speaker 306, and the microphone 307 can provide an audio interface between the user and the terminal 300. The audio circuit 305 can transmit the converted audio data to the speaker 306 and convert it into a sound signal output by the speaker 306. On the other hand, the microphone 307 can convert the collected sound signal into a signal, which is received by the audio circuit 305. It is then converted to audio data, which is then output to RF circuitry 304 for transmission to a device such as another terminal, or audio data is output to memory 302 for processor 301 to perform further processing in conjunction with the content stored in memory 302. In addition, the camera 303 can acquire image frames in real time and transmit them to the processor 301 for processing, and store the processed results to the memory 302 and/or present the processed results to the user via the display panel 312.

Processor 301 is the control center of terminal 300, which connects various portions of the entire terminal 300 using various interfaces and lines, by running or executing software programs and/or modules stored in memory 302, and recalling data stored in memory 302. The various functions and processing data of the terminal 300 are executed to perform overall monitoring of the terminal 300. It should be noted that the processor 301 may include one or more processing units; the processor 301 may further integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface (UI) And the application, etc., the modem processor mainly handles wireless communication. It can be understood that the above modem processor may not be integrated into the processor 301.

The terminal 300 may further include a power source 314 (for example, a battery) for supplying power to the respective components. In the embodiment of the present invention, the power source 314 may be logically connected to the processor 301 through the power management system, thereby managing charging, discharging, and the like through the power management system. And power consumption and other functions.

In addition, there are components not shown in FIG. 3, for example, the terminal 300 may further include a Bluetooth module, a sensor, and the like, and details are not described herein again.

As shown in FIG. 4, the audio circuit 305 may specifically include a digital signal processing (DSP) and a codec 401 module, wherein the codec sub-module implements analog to digital/digital to analog (AD/DA) conversion. The DSP sub-module implements the processing of the speech algorithm.

The following takes the terminal as an in-vehicle device as an example. As shown in Figure 5, the booting process of the car is as follows:

S401, the car is turned on, and the surrounding sound is collected. In this step, the vehicle collects the surrounding sound through the microphone 307 in FIG.

In S402, the car machine determines the number of people in the car through voiceprint recognition technology. Voiceprint recognition is a kind of biometric technology, also known as speaker recognition. There are two types, namely speaker recognition and speaker confirmation. This article refers to the confirmation of the speaker. Voiceprint recognition is the conversion of sound signals into electrical signals, which are then identified by a computer. Specifically in this paper, how many people in the car can be identified by the voiceprint recognition technology by judging how many different voiceprints are in the car. The so-called voiceprint is a sound wave spectrum that carries speech information displayed by an electroacoustic instrument. The generation of human language is a complex physiological and physical process between the human language center and the vocal organs. The vocal organs used by people in speech--tongue, teeth, throat, lungs, and nasal cavity vary greatly in size and shape. , so the soundprints of any two people are different. Each person's speech acoustic characteristics are both relatively stable and variability, not absolute and immutable. This variation can come from physiology, pathology, psychology, simulation, camouflage, and also related to environmental disturbances. However, since each person's vocal organs are different, in general, people can still distinguish the voices of different people or judge whether they are the same person's voice. Voiceprint is a way of identifying, and the number of people in the car can also be identified by one or more of, for example, iris information, facial information, fingerprint information, infrared, sensing data, and voice inquiries.

For example, an image recognized by the iris can be obtained by photographing the camera on the vehicle, and the vehicle judges whether there are many people beside the vehicle by judging whether there is a different iris.

For another example, an image of the face recognition can be obtained by photographing the camera on the vehicle, and the vehicle determines whether there are many people beside the vehicle by judging whether there are different facial characteristics.

For another example, it is possible to identify whether there is a different fingerprint on the vehicle by setting a fingerprint recognition device on the door handle of the vehicle, and whether there are many people on the vehicle by whether there are different fingerprints.

For another example, the number of people in the vehicle can be determined by an infrared sensor on the vehicle or in the vehicle.

For another example, the number of people in the car can be determined by sensing data. Specifically, the number of people in the car can be determined by detecting the number of people by the pressure sensor of the seat on the car.

For another example, the number of people in the car can be judged by voice inquiry. Specifically, the car machine can determine how many people are in the car by asking how many people in the car, and then the passengers in the car or the driver answering.

The above various ways of identifying the number of people in the car can be implemented in various ways.

For example, although voiceprint recognition only recognizes one person, other ways of identifying the number of people in the car are recognized as multiple people, and then it is handled in a multi-person manner.

S403, if the vehicle determines that there is only one person in the vehicle (ie, only the driver), go to step 404, otherwise go to step 405.

S404, the vehicle enters a wake-free interaction mode. The wake-free interaction mode here refers to the interaction mode that does not require wake-up words. For example, the driver can say "navigation home" to the car. Subsequently, in response to this wake-free command, the vehicle will perform an operation of navigating home.

S405, the vehicle enters a wake-up question and answer interaction mode. The wake-and-answer interaction mode here refers to the human-computer interaction that needs to wake up words. For example, the driver or the passenger in the car can wake up the car and say "Hello, Xiaochi". Subsequently, in response to this awakened command, the vehicle responded with a reply saying "What can help you?". After that, the driver or the passenger in the car can say "Please navigate home." Thereafter, the vehicle performs an operation of navigating home in response to the instruction. For example, the driver or the passenger in the car can directly speak the voice of the wake-up word with the car, such as "Xiaochi I am going to the airport"; among them, "Xiaochi" is the wake-up word. Subsequently, the vehicle performs an operation of navigating to the airport in response to this command.

In the embodiment of the invention, the vehicle determines whether the number of people in the vehicle determines whether to activate the wake-free mode, thereby reducing the number of times the user in the vehicle interacts with the vehicle through the wake-up word, thereby improving the user experience.

How to determine the number of people in the car, the following way. For details, refer to the method flowchart shown in FIG. 6.

S501, user A speaks the first voice message. The voice of the first user A is collected, and the voiceprint feature of the user A is marked. The first voice information refers to the first voice message received by the voice device just after voice recognition is turned on.

S502. The vehicle determines whether the first voice information is a specific instruction of the vehicle. Here, the vehicle determines whether the first voice information is a specific instruction of the vehicle, and refers to whether the vehicle determines whether there is a strong command related to the function of the vehicle. Specifically, some specific instructions may be pre-stored in the local or cloud database of the vehicle; when the vehicle receives a voice, it may be determined whether the voice completely matches the pre-stored instruction in the database or the matching degree is greater than a certain threshold. If the match is complete or the match is high, the voice belongs to the device strong related instruction (pre-defined), which is a specific instruction. For example, the vehicle judges whether there are instruction words of "immediately" and "navigation" in the first piece of voice information. If both exist, the vehicle determines that the first voice message is a vehicle-specific command. If the vehicle determines that only the word "navigation" exists in the first piece of voice information, the car machine determines that the first piece of voice information is not a specific instruction of the car machine. If the vehicle determines that the first voice message is a device-specific command, then S503 is performed. If the vehicle determines that the first piece of voice information is not a device-specific instruction, then S504 is performed. For example, the "immediate navigation to a certain place" is a device-related command, and it is likely to ask the car equipment; for example, "Is it good to sleep last night", it is not a strong related instruction, more likely to ask next to People.

S503, when the vehicle determines that the first voice information is a device-specific instruction, the vehicle performs a related voice response. S502 and S503 are optional.

S504. If the vehicle determines that the first voice information is not a device-specific instruction, the vehicle displays the command content on the display screen. Among them, this step is optional. When the embodiment does not include S504, if the vehicle determines that the first piece of voice information is not a specific instruction, then S505 is performed.

If the above S502 and S503 are not executed, S504 will not be executed.

If S502 and S503 are executed, S504 may or may not be executed.

S505, the car is delayed by X seconds. Specifically, X can be 3 seconds, or 4 seconds, or 5 seconds. The delay here may be to determine whether an answer of other users in the car is received within the first time period after receiving the first piece of voice information. The first voice message and the answers of other users in the car have different voiceprint information.

S506, the car machine uses voiceprint technology to determine whether there are other users in the car to answer. If there is no user answer, it proceeds to S507. Otherwise, go to S508.

S507, if there is no other user answering in the car, the car machine feedbacks the voice response to the user A, and records that there is only one person in the car. The voice response that the car machine feeds back to User A can be "OK, and will perform XX operations for you."

S508, if there is an answer from a user other than the user A in the car, the car machine gives up the voice response and records that there are many people in the car. That is, if the second user B's voice reply is collected by the voiceprint recognition technology during the delayed voice response, the voice response is discarded, and there are many people in the mark.

In the embodiment of the present invention, the vehicle determines that there are several people in the vehicle, and then records the number of people in the vehicle in the vehicle, so that the vehicle can determine whether to adopt the wake-up word according to several people in the vehicle, thereby improving the vehicle and the person. The efficiency of the interaction.

The space in which an intelligent voice device (eg, a car) is located is an enclosed space, a semi-enclosed space, or an open space. The enclosed space may be an enclosed space formed by a continuous curved surface. The semi-enclosed space may be a semi-enclosed space composed of a non-closed curved surface, for example, a space formed by a room in which a room door is opened. The open space can be an open space or a space that is not enclosed by any spatial surface. The voice device may specifically be an intelligent voice device, and is used for implementing voice input (ie, receiving external voice and converting the voice into an electrical signal), voice recognition, and implementing a voice required function.

Specifically, the semi-enclosed space or the open space may be a spherical space having a radius of communication distance of the intelligent voice device.

If the radius of the closed space is less than or equal to the communication distance of the smart voice device, the space where the smart voice device is located is the closed space. The radius of the enclosed space here may refer to the distance of half of the longest side of the enclosed space.

As shown in Figure 13, the car is an enclosed space. The car has a radius of half the length of the car. When the communication distance of the smart voice device in the vehicle is the radius of the ball C1, the radius of the car is exactly equal to the radius of the ball C1, and the space in which the smart voice device is located is the closed space.

When the communication distance of the intelligent voice device in the car is the radius of the ball C2, since the radius of the ball C2 is larger than the radius of the ball C1 (ie, the radius of the car), the space where the intelligent voice device is located is the closed space surrounded by the vehicle body. Rather than being a spherical space with a radius of communication distance (radius of C2) of the intelligent voice device.

There are many types of car equipment, including car smart rearview mirrors, car smart speakers and car smart TVs. The following uses examples to describe how the wake-up words are specifically used in the embodiments of the present invention.

As shown in Figure 7, the vehicle's smart rearview mirror is turned on and begins to collect ambient sound. In this step, the in-vehicle smart rearview mirror collects the surrounding sound through the microphone 307 in FIG.

The car smart rearview mirror determines the number of people in the car through voiceprint recognition technology. Voiceprint recognition is a kind of biometric technology, also known as speaker recognition. There are two types, namely speaker recognition and speaker confirmation. Different voiceprint recognition techniques are used for different tasks and applications. For example, when narrowing the scope of criminal investigation, it may be necessary to identify the technology, and the bank transaction needs to confirm the technology. Voiceprint recognition is the conversion of sound signals into electrical signals, which are then identified by a computer. Specifically in this paper, voiceprint recognition technology can be used to identify and confirm how many people are in the car.

In this example, the vehicle-mounted smart rearview mirror determines that there is only one person in the car (ie, only the driver), and the vehicle-mounted smart rearview mirror enters the wake-free interaction mode. The wake-free interaction mode here refers to the interaction mode that does not require wake-up words. For example, the driver can say "navigation to the airport" with the on-board smart rearview mirror. Subsequently, the in-vehicle smart rearview mirror will perform navigation to the airport in response to this wake-free command. Optionally, the onboard smart rearview mirror can report "Start to navigate to the airport for you" in response to the driver's instructions.

As shown in Fig. 8, if the vehicle intelligent rearview mirror determines that there are many people in the vehicle, the vehicle intelligent rearview mirror enters a wake-up question and answer interaction mode. The wake-and-answer interaction mode here refers to the human-computer interaction that needs to wake up words. For example, the driver or the passenger in the car can wake up the car smart rearview mirror and say "Hello, Xiaochi". Subsequently, the in-vehicle smart rearview mirror responded to a reply in response to a reply saying "What can help you?". After that, the driver or the passenger in the car can say "I am going to the airport." Thereafter, the in-vehicle smart rearview mirror performs an operation of navigating to the airport in response to the instruction. Optionally, the onboard smart rearview mirror can report "Start to navigate to the airport for you" in response to the driver's instructions.

As shown in Figure 9, it is a schematic diagram of the internal structure of a car. Specifically, the car smart speaker is integrated into the car center control panel. In FIG. 9, the in-vehicle smart speaker is different from the existing speaker, and the in-vehicle smart speaker includes a sound collecting device (for example, a microphone). The driver of the car or the passenger in the car can communicate with the car smart speaker via the microphone and speaker on the car smart speaker. The car smart speaker uses the voiceprint recognition technology to determine the number of people in the car.

After the car smart speaker judges whether there are one or more people in the car, whether and how to use the wake-up words can refer to the processing method of the car smart rearview mirror. I will not repeat them here.

As shown in Figure 10, it is a schematic diagram of the internal structure of a car. Specifically, the car smart TV is integrated on the top of the car. In FIG. 10, the in-vehicle smart TV differs from the existing television in that the in-vehicle smart television includes a sound pickup device (for example, a microphone). The driver of the car or the passenger in the car can communicate with the car smart TV via the microphone and speaker on the car smart TV. The car smart TV uses the voiceprint recognition technology to determine the number of people in the car.

After the car smart TV judges whether there are one or more people in the car, whether and how to use the wake-up words can refer to the processing method of the car smart rearview mirror. I will not repeat them here.

The car smart TV and car smart speaker in the car can be used in the home field, specifically corresponding to the home smart speaker and home smart TV.

As shown in Figure 11, it is a schematic diagram of a home smart speaker. In FIG. 11, the in-vehicle smart speaker is different from the existing speaker, and the in-vehicle smart speaker includes a sound collecting device (for example, a microphone). The home can communicate with the home smart speaker through the microphone and speaker on the home smart speaker. The car smart speaker judges the number of people in the home through voiceprint recognition technology.

After the home smart speaker judges one or more people in the home, whether and how to use the wake-up words, refer to the processing method of the vehicle-mounted smart rearview mirror of the above embodiment. I will not repeat them here.

As shown in Figure 12, it is a schematic diagram of a home smart TV. In FIG. 12, the in-vehicle smart TV differs from the existing television in that the in-vehicle smart television includes a sound pickup device (for example, a microphone). People in the home can communicate with the home smart TV via microphones and speakers on the home smart TV. The car smart TV judges the number of people in the home through voiceprint recognition technology.

After the family smart TV judges one or more people in the home, whether and how to use the wake-up words can refer to the processing method of the car smart rearview mirror. For example, when it is judged that there are two people in the home in FIG. 12, the home smart television enters a wake-up question and answer interaction mode. The wake-and-answer interaction mode here refers to the human-computer interaction that needs to wake up words. For example, a mother or daughter can wake up the family smart TV and say "Hello, TV." Subsequently, in response to this awakened command, the home smart TV can respond to a reply saying "What can help you?". After that, my mother or daughter can say "I want to watch the live broadcast." Thereafter, the home smart television performs an operation of watching a live television broadcast in response to the instruction. Alternatively, the home smart TV can broadcast "Start selecting a live channel for you" in response to a mother or daughter's instruction.

The embodiment of the present invention further discloses an intelligent voice device. Specifically, the smart voice device may include: a processor 301, configured to determine a number of people in a space where the smart voice device is located; and determine a number of people in the space. For a moment, the smart voice device is controlled to enter a wake-free voice interaction mode. The wake-free voice interaction method here refers to a voice interaction method without using a wake-up word. For example, the user can say "navigation to XX place", the voice device (for example, smart voice device) will perform navigation to the XX place operation without replying "What can help you?"

The smart voice device further includes:

a collector (eg, a microphone 307) for collecting a first voice in the space;

The processor 301 is configured to determine the number of people in the space, including:

The processor is configured to determine whether the first voice is a specific instruction of the smart voice device; if the first voice is not a specific instruction of the smart voice device, delay X seconds; determine whether there is The first voice has a second voice with different voiceprint characteristics; when it is determined that the second voice command has a different voiceprint characteristic with the first voice, determining that there are multiple people in the space;

Where X is a positive number greater than zero.

The smart voice device further includes:

a collector (eg, a microphone 307) for collecting a first voice in the space;

The processor 301 is configured to determine the number of people in the closed space, including:

The processor is configured to determine whether the first voice is a specific instruction of the smart voice device; if the first voice is not a specific instruction of the smart voice device, delay X seconds; determine whether there is The first voice has a second voice with different voiceprint characteristics; when it is determined that the second voice command does not have a different voiceprint characteristic with the voice, it is determined that there is only one person in the closed space;

Where X is a positive number greater than zero.

The space in which the intelligent voice device is located is an enclosed space, a semi-enclosed space or an open space.

The semi-enclosed space or open space is a spherical space having a radius of communication distance of the intelligent voice device.

If the radius of the closed space is less than or equal to the communication distance of the smart voice device, the space where the smart voice device is located is the closed space.

As shown in Figure 14, it is a block diagram of the internal implementation of a processor. As can be seen from the figure, the processor includes 4 high-speed processing cores and 4 low-speed processing cores. Each of the four high-speed processing cores and a corresponding secondary cache are combined to form a high-speed core processing area. Each of the four low-speed processing cores and a corresponding second-level buffer are combined to form a low-speed core processing area. Here, the high speed processing core may refer to a processing core having a processing frequency of 2.1 GHz (hertz). Here, the low speed processing core may refer to a processing core having a processing frequency of 1.7 GHz (hertz).

All of the steps performed by processor 301 are performed by a high speed processing core or a low speed processing core.

In addition to the high-speed processing core, the low-speed processing core and the corresponding secondary cache, there are other components. For example, a modem baseband portion; a radio frequency transceiver connection for processing a baseband portion of the radio frequency signal; a display subsystem coupled to the display; an image signal processing subsystem coupled to the external CPU; and a single channel DDR controller coupled to the DDR memory ; embedded multimedia card interface with embedded multimedia card; USB interface connected with personal computer; SDIO input and output interface connected with short-distance communication module; UART interface connected with Bluetooth, GPS; I2C interface connected with sensor; Smart card interface of smart card SIM card interface. And a video processing subsystem, a Sensor Hub subsystem, a low-power microcontroller, a high-resolution video codec, a dual-security engine, an image processor, and an image processing unit formed by a second-level cache. There is also a coherent bus that is placed inside the CPU to connect all the interfaces and processing units in the CPU.

It can be understood that, in order to implement the above functions, the above terminal and the like include hardware structures and/or software modules corresponding to each function. Those skilled in the art should readily appreciate that the technical solutions of the embodiments of the present application can be implemented in a combination of hardware or hardware and computer software, in combination with the unit and algorithm steps of the examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

The embodiment of the present application may perform the division of the function modules on the terminal or the like according to the foregoing method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it may occur in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center Transfer to another website site, computer, server, or data center by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL), or wireless (eg, infrared, wireless, microwave, etc.). The computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media. The usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. . Therefore, the scope of protection of the present application should be determined by the scope of the claims.

Claims

A method for voice interaction, comprising:

The voice device determines the number of people in the space where the voice device is located;

When the voice device determines that the number of people in the space is one, the voice device enters a wake-free voice interaction mode.
The method according to claim 1, wherein the voice device determines the number of people in the space where the voice device is located, including:

The voice device determines the number of people in the space where the voice device is located according to one or more of voiceprint information, iris information, portrait information, fingerprint information, and sensing data.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the voiceprint information, including:

The voice device collects a first voice in the space;

The voice device determines whether a second voice is received within a first time period after receiving the first voice, and the second voice and the first voice have different voiceprint characteristics;

If the voice device does not receive the second voice within the first time period, it is determined that there is one person in the space.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the voiceprint information, including:

The voice device collects a first voice in the space;

If the first voice is not the specific instruction, the voice device determines whether a second voice is received within a first time period after receiving the first voice, the second voice and the second voice A voice has different voiceprint characteristics;

If the voice device does not receive the second voice within the first time period, it is determined that there is one person in the space.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the voiceprint information, including:

The voice device collects a first voice in the space;

If the first voice is not the specific instruction, the voice device determines whether a second voice is received within a first time period after receiving the first voice, the second voice and the The first voice has different voiceprint characteristics;

If the voice device receives the second voice within the first time period, the smart voice device determines that there are multiple people in the space.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the iris information, including:

The voice device obtains an image recognized by the iris through a camera in a space where the voice device is located;

The voice device determines whether there is different iris information in the image;

When the voice device determines that there is different iris information, the voice device determines that there are multiple people in the space where the voice device is located;

When the voice device determines that there is only one type of iris information, the voice device determines that the voice device is located in a space.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the portrait information, including:

The voice device obtains portrait information by using a camera in a space where the voice device is located;

The voice device determines whether there is different portrait information in the image;

When the voice device determines that there is different portrait information, the voice device determines that there are multiple people in the space where the voice device is located;

When the voice device determines that there is only one type of portrait information, the voice device determines that the voice device is located in a space.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the fingerprint information, including:

The voice device obtains fingerprint information by using a fingerprint identification device in a space where the voice device is located;

The voice device determines whether there is different fingerprint information in the image;

When the voice device determines that there is different fingerprint information, the voice device determines that there are multiple people in the space where the voice device is located;

When the voice device determines that there is only one type of fingerprint information, the voice device determines that the voice device is located in a space.
The method according to claim 2, wherein the voice device determines the number of people in the space where the voice device is located according to the sensing data, including:

The voice device obtains the sensing data by using a sensing device in a space where the voice device is located;

The voice device determines whether there is different sensing data in the image;

When the voice device determines that there is different sensing data, the voice device determines that there are multiple people in the space where the voice device is located;

When the voice device determines that there is only one type of sensing data, the voice device determines that the voice device is located in a space.
The method according to any one of claims 1 to 9, wherein after the voice device enters the wake-free voice interaction mode, the method further includes:

The voice device receives a third voice, and the third voice does not include an awakening word;

The voice device identifies and performs a function corresponding to the third voice.
The method according to any one of claims 1 to 9, wherein the method further comprises:

When the voice device determines that there are multiple people in the space, the voice device enters a wake-up voice interaction mode;

Receiving, by the voice device, an wake-up word or a fourth voice including an awakening word;

The voice device enters a voice interaction mode or voice recognition and performs a function corresponding to the fourth voice.
The method according to any one of claims 1 to 11, wherein the space in which the voice device is located is a closed space, a semi-enclosed space or an open space.
The method according to claim 12, wherein the semi-enclosed space or the open space is a spherical space having a radius of a communication distance of the voice device.
The method according to claim 12, wherein if the radius of the closed space is less than or equal to the communication distance of the voice device, the space in which the voice device is located is the closed space.
A voice device, comprising:

a collector, configured to collect information in a space where the voice device is located;

The processor is configured to determine the number of people in the space according to the information collected by the collector; and when the number of people in the space is determined to be one, the voice device is controlled to enter a wake-free voice interaction mode.
The voice device according to claim 15, wherein the processor is configured to determine the number of people in the space where the voice device is located according to the information collected by the collector, including:

The processor is configured to determine, according to one or more of the voiceprint information, the iris information, the portrait information, the fingerprint information, and the sensing data, the number of people in the space where the voice device is located.
The voice device according to claim 16, wherein the processor is configured to determine the number of people in the space where the voice device is located according to the voiceprint information, including:

The collector is configured to collect a first voice in the space;

The processor is specifically configured to determine whether the second voice is received within a first time period after receiving the first voice, where the second voice and the first voice have different voiceprint characteristics; If the voice device does not receive the second voice within the first time period, it is determined that there is a person in the space.
A speech device according to claim 16 wherein:

The collector is specifically configured to collect the first voice in the space;

The processor is configured to determine the number of people in the space, including:

The processor, configured to determine, if the first voice is not the specific instruction, determining whether a first time period after receiving the first voice is received is different from the first voice The second voice of the voiceprint characteristic determines that there are a plurality of people in the space when it is determined that the second voice command has a different voiceprint characteristic with the first voice.
The voice device according to claim 7, wherein the collector is configured to collect the first voice in the space;

The processor is configured to determine the number of people in the closed space, including:

The processor, configured to determine, if the first voice is not the specific instruction, determining whether a first time period after receiving the first voice is received is different from the first voice The second voice of the voiceprint characteristic determines that there is one person in the closed space when it is determined that the second voice is not received between the first time periods.
The voice device according to claim 16, wherein the processor is configured to determine, according to the iris information, the number of people in the space where the voice device is located, including:

The processor is specifically configured to obtain an iris-recognized image by using a camera in a space where the voice device is located; determine whether there is different iris information in the image; and when determining different iris information, the processor Determining that there are multiple people in the space where the voice device is located; when it is determined that there is only one type of iris information, the processor determines that the space in the space where the voice device is located is one person.
The voice device according to claim 16, wherein the processor is configured to determine, according to the portrait information, the number of people in the space where the voice device is located, including:

The processor is specifically configured to obtain portrait information by using a camera in a space where the voice device is located; determine whether there is different portrait information in the image; and when determining that there is different portrait information, the processor determines There are a plurality of people in the space where the voice device is located; when it is judged that there is only one type of portrait information, the processor determines that the space in which the voice device is located is one person.
The voice device according to claim 16, wherein the processor is configured to determine, according to the fingerprint information, the number of people in the space where the voice device is located, including:

The processor is specifically configured to obtain fingerprint information by using a fingerprint identification device in a space where the voice device is located; determine whether there is different fingerprint information in the image; and when determining different fingerprint information, the processor Determining that there is more than one person in the space where the voice device is located; when determining that there is only one type of fingerprint information, the processor determines that the space in which the voice device is located is one person.
The voice device according to claim 16, wherein the processor is configured to determine, according to the sensing data, the number of people in the space where the voice device is located, including:

The processor is specifically configured to obtain the sensing data by using a sensing device in a space where the voice device is located; determine whether there is different sensing data in the image; and when determining that different sensing data is determined, the processor determines There are multiple people in the space where the voice device is located; when it is determined that there is only one type of sensing data, the processor determines that the space in which the voice device is located is one person.
A speech device according to claims 15-23, characterized in that

The collector is further configured to receive a third voice, where the third voice does not include an awakening word;

The processor is further configured to: identify and perform a function corresponding to the third voice.
A speech device according to claims 15-23, characterized in that

The processor is further configured to: when the voice device determines that there are multiple people in the space, the voice device enters a wake-up voice interaction mode;

The collector is further configured to receive an awakening word or receive a fourth voice including an awakening word;

The processor is further configured to: control the voice device to enter a voice interaction mode, or perform voice recognition and perform a function corresponding to the fourth voice.
The voice device according to any one of claims 15 to 25, wherein the space in which the voice device is located is a closed space, a semi-enclosed space or an open space.
The voice device according to claim 26, wherein the semi-enclosed space or the open space is a spherical space having a radius of a communication distance of the voice device.
The voice device according to claim 26, wherein if the radius of the closed space is less than or equal to the communication distance of the voice device, the space in which the voice device is located is the closed space.