WO2021217572A1 - 车内用户定位方法、车载交互方法、车载装置及车辆 - Google Patents

车内用户定位方法、车载交互方法、车载装置及车辆 Download PDF

Info

Publication number
WO2021217572A1
WO2021217572A1 PCT/CN2020/088227 CN2020088227W WO2021217572A1 WO 2021217572 A1 WO2021217572 A1 WO 2021217572A1 CN 2020088227 W CN2020088227 W CN 2020088227W WO 2021217572 A1 WO2021217572 A1 WO 2021217572A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
vehicle
car
mapping relationship
voice command
Prior art date
Application number
PCT/CN2020/088227
Other languages
English (en)
French (fr)
Inventor
赵勤嵩
殷明睿
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20933541.3A priority Critical patent/EP4134949A4/en
Priority to CN202080004901.1A priority patent/CN112655000B/zh
Priority to PCT/CN2020/088227 priority patent/WO2021217572A1/zh
Publication of WO2021217572A1 publication Critical patent/WO2021217572A1/zh
Priority to US17/976,424 priority patent/US20230038039A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/037—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
    • B60R16/0373—Voice control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/10—Interpretation of driver requests or demands
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/16—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
    • G01S5/22—Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/593—Recognising seat occupancy
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0881—Seat occupation; Driver or passenger presence
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/089—Driver voice
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo or light sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/54—Audio sensitive means, e.g. ultrasound
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/047—Prioritizing desires of multiple occupants, e.g. when setting climate control or driving behaviour
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/21—Voice
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/80—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using ultrasonic, sonic or infrasonic waves
    • G01S3/8006—Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics

Definitions

  • the embodiments of the present application relate to the field of artificial intelligence, and in particular to a method for positioning a user in a car, a method for in-vehicle interaction, an in-vehicle device, and a vehicle.
  • Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.
  • artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.
  • Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theories.
  • Smart cockpit is a mainstream application in the field of artificial intelligence, and the intelligentization of vehicles and machines is an important aspect of smart cockpit research.
  • the car machine can realize the information communication between people and car, car and the outside world, and realize a variety of interactive functions, such as sending and receiving text messages, making phone calls, navigation, voice control, playing music, online movies, etc., greatly enhancing the user experience.
  • interactive functions such as sending and receiving text messages, making phone calls, navigation, voice control, playing music, online movies, etc., greatly enhancing the user experience.
  • multiple display screens such as the back of the seat
  • users in various positions can enjoy the video.
  • the way of using mechanical buttons or touch screens for interaction has been difficult to meet the needs of users.
  • the way of voice interaction can reduce user operations, the user issues voice commands, the voice assistant locates the sound source, and only opens the display screen corresponding to the user at the sound source location, which can avoid interference with others and protect the privacy of the user.
  • the space in the car is small, the user’s voice is noisy, and the requirements for positioning accuracy are high. If multiple users in the car are talking at the same time, the existing method cannot accurately locate the user who issued the voice command, and thus cannot perform the corresponding operation. , Affect the interactive experience.
  • the present application provides a method for locating a user in a vehicle, an on-vehicle interaction method, an on-vehicle device, and a vehicle, which can realize the positioning of the user when there are multiple sound sources in the vehicle.
  • a method for locating users in a car including: acquiring a sound signal collected by a microphone in a car; when a first voice command is recognized from the sound signal, determining the first user who issued the first voice command; and acquiring The mapping relationship between the in-vehicle user and the in-vehicle location; the in-vehicle location of the first user is determined according to the mapping relationship between the in-vehicle user and the in-vehicle location.
  • the microphone in the vehicle may be one microphone or multiple microphones.
  • the multiple microphones may be a microphone array.
  • the microphone array can be set as required, for example, the microphone array can be a uniform circular array or a uniform linear array.
  • the first voice command may be a wake-up word
  • the wake-up word may be used to wake up the voice assistant and activate subsequent processing procedures.
  • the manner of determining the first user who issued the first voice command may include multiple manners.
  • the embodiment of the present application does not limit the manner of determining the first user.
  • the first user corresponding to the first voice command may be determined according to the mapping relationship between the user and the voice command.
  • different voice commands are set for different users in advance, and the voice commands are bound to the users, that is, a mapping relationship is established.
  • a mapping relationship For example, user A establishes a mapping relationship with the voice command "turn on A's speaker”, and user B establishes a mapping relationship with the voice command "turn on B's speaker”. If the first voice command is "turn on A's speaker”, it is determined that the first user who issued the first voice command is user A.
  • the mapping relationship between in-vehicle users and in-vehicle locations can be the location mapping relationships of all users in the vehicle, or one user in the vehicle, or the location mapping relationships of some users in the vehicle.
  • the method for acquiring the mapping relationship between the in-vehicle user and the in-vehicle location may include multiple methods.
  • the mapping relationship between the user in the car and the position in the car may be pre-stored, or it may be obtained from other devices, such as a cloud server, or it may be updated in real time.
  • the embodiment of the present application does not limit the method for obtaining the mapping relationship between the user in the car and the position in the car.
  • the method for determining the mapping relationship between the in-vehicle user and the in-vehicle location may include multiple methods.
  • fingerprint collectors can be set at different locations in the car.
  • the fingerprint is collected by the fingerprint collector, and the user corresponding to the fingerprint collected by the fingerprint collector is determined according to the mapping relationship between the fingerprint characteristics of the user and the user. Since different fingerprint collectors correspond to different positions in the car, a mapping relationship between users and positions in the car can be established.
  • the fingerprint collector A corresponds to the position A in the car
  • the fingerprint A is collected by the fingerprint collector A
  • the fingerprint A is determined to correspond to the user A through fingerprint identification, and then the user A and the position A in the car are mapped.
  • the embodiment of the present application does not limit the method for determining the mapping relationship between the user in the car and the position in the car.
  • the user who needs to interact is determined based on the voice command, and then the position in the car of the user who needs to interact is determined according to the mapping relationship between the user in the car and the position in the car, even when multiple users are talking at the same time. Being able to locate the location of the user who needs to interact is conducive to achieving one-to-one interaction and improving user experience.
  • determining the first user who issued the first voice command includes: performing voiceprint feature matching on the first voice command, and identifying the first user who issued the first voice command.
  • the mapping relationship between the user in the vehicle and the location in the vehicle is determined based on the face image collected by the vehicle camera.
  • the face image collected by the vehicle-mounted camera determines the mapping relationship between the user and the vehicle without the user's cooperation, which can avoid disturbing the user and improve the user experience.
  • the mapping relationship between the user in the car and the position in the car is determined based on the face image collected by the on-board camera, including: obtaining user information; according to user information and face The image mapping relationship determines the user corresponding to the face image; the user's in-vehicle position is determined according to the position and/or angle of the vehicle-mounted camera.
  • the user information acquisition method may include multiple methods.
  • user information may be pre-stored, or obtained from other devices, such as a cloud server, or updated in real time.
  • the embodiment of the application does not limit the method for obtaining user information.
  • the mapping relationship between the user information and the face image the user corresponding to the face image is determined, that is, the face recognition is performed on the face image, and the user corresponding to the face image is determined.
  • the vehicle-mounted camera may be an in-vehicle camera or an outside-vehicle camera.
  • the user's in-vehicle position is determined by the mapping relationship between the position of the in-vehicle camera and the in-vehicle position.
  • mapping relationship between the position of the camera and the position in the vehicle can also be understood as the mapping relationship between different cameras and the position in the vehicle.
  • the location of the user captured by different cameras is the location in the vehicle corresponding to the camera.
  • the user's in-vehicle position is determined by the mapping relationship between the rotation angle of the in-vehicle camera and the in-vehicle position.
  • the camera is located on the rear view mirror of the car, and the rotation angle of the camera corresponds to the position in the car.
  • the corresponding position in the car is the middle of the car.
  • the position of the user captured by the camera is the middle of the car;
  • the rotation angle of the camera When it is the angle A, the corresponding position in the car is the position A in the car.
  • the position of the user captured by the camera is the position A in the car.
  • the position in the car can be indicated by the seat in the car.
  • the position/or angle of the camera can be set according to the seat, which can reduce the amount of calculation during the positioning process and increase the processing speed.
  • the mapping relationship between the user in the vehicle and the location in the vehicle is determined based on the facial image collected by the vehicle camera, including: based on the mapping relationship between user information and facial images , Determine the user corresponding to the face image, and determine the user's in-vehicle position according to the user's relative position in the lens of the vehicle-mounted camera.
  • the user's in-vehicle position is determined according to the relative position of the user in the lens of the vehicle-mounted camera, that is, the user's in-vehicle position is determined according to the image information collected by the vehicle-mounted camera.
  • the vehicle-mounted camera is located on the rear-view mirror of the vehicle, and all users in the vehicle can be put into the viewfinder in one shot.
  • the user corresponding to the face image and its relative position in the lens are determined through face recognition, and the in-vehicle position corresponding to the user is determined according to the relative position.
  • the vehicle-mounted camera is located on the B-pillar outside the vehicle, and the vehicle-mounted cameras can be set on the B-pillars on both sides.
  • the face image collected by the camera on the left B-pillar the user corresponding to the face image and the relative position in the lens are determined through face recognition. For example, if the relative position of the user in the camera is on the right side, the user gets in the car from the front door on the left side of the vehicle, and the position in the car corresponding to the user may be on the driver's seat.
  • the position in the car can be indicated by the seat in the car, so that the relative position in the lens can be mapped to the seat in the car, which can reduce the amount of calculation during the positioning process and increase the processing speed.
  • the mapping relationship between the user in the vehicle and the location in the vehicle is updated regularly.
  • mapping relationship between the users in the car and the position in the car is updated regularly, which can improve the accuracy of user positioning.
  • the mapping relationship between the in-car user and the in-car location is updated regularly, including: determining the current in-vehicle location corresponding to the mapping relationship between the in-vehicle user and the in-vehicle location
  • the user obtains the current face image collected by the vehicle camera, and determines the current user corresponding to the current face image according to the mapping relationship between the user information and the face image; when the user corresponding to the current position in the vehicle is different from the current user, the vehicle is updated
  • the mapping relationship between in-car users and in-vehicle locations is updated regularly, including: determining the current in-vehicle location corresponding to the mapping relationship between the in-vehicle user and the in-vehicle location.
  • mapping relationship between the user in the car and the position in the car it is determined that there is no user in the current car position, and the current face image collected by the on-board camera is obtained.
  • the mapping relationship between the user information and the face image the current face image corresponding to the current face image is determined.
  • the mapping relationship between the user in the car and the position in the car it is determined that the user corresponding to the current position in the car is user A, the current face image collected by the on-board camera is obtained, and the current person is determined according to the mapping relationship between the user information and the face image.
  • the current user corresponding to the face image is user B; when user A is different from user B, a mapping relationship between user B and the current position in the car is established, and the mapping relationship between the user in the car and the position in the car is updated.
  • the method further includes: performing sound source localization according to the sound signal to obtain at least one sound source position; and according to the first user's in-vehicle position and at least one sound source position Correct the position in the car of the first user.
  • a sound source localization method based on a microphone array can be used to localize the sound source of the sound signal.
  • the sound source localization method may be the MUSIC algorithm.
  • the microphone array can be set as required, for example, the microphone array can be a uniform circular array or a uniform linear array.
  • the accuracy of the position in the vehicle of the first user can be further improved through sound source localization, and the user who needs to interact can also be accurately located in a noisy environment in the vehicle.
  • correcting the in-vehicle position of the first user according to the in-vehicle position of the first user and the position of the at least one sound source includes: comparing the position of the at least one sound source with the position of the first user The position of the sound source closest to the position in the car of a user is determined as the position in the car of the first user after correction.
  • the sound source position within the angular range of the at least one sound source position is taken as the corrected in-vehicle position of the first user.
  • the sound source position closest to the value among the at least one sound source position is taken as the in-vehicle position of the first user after correction.
  • an in-vehicle interaction method which includes: acquiring a sound signal collected by a microphone in a car; when a first voice command is recognized from the sound signal, determining the first user who issued the first voice command; The mapping relationship between the user and the in-vehicle location determines the in-vehicle location of the first user according to the mapping relationship between the in-vehicle user and the in-vehicle location; and executes the first voice command according to the in-vehicle location of the first user.
  • the microphone in the vehicle may be one microphone or multiple microphones.
  • the multiple microphones may be a microphone array.
  • the microphone array can be set as required, for example, the microphone array can be a uniform circular array or a uniform linear array.
  • the first voice command may be a wake-up word
  • the wake-up word may be used to wake up the voice assistant and activate subsequent processing procedures.
  • the first user corresponding to the first voice command may be determined according to the mapping relationship between the user and the voice command.
  • different voice commands are set for different users in advance, and the voice commands are bound to the users, that is, a mapping relationship is established.
  • a mapping relationship For example, user A establishes a mapping relationship with the voice command "turn on A's speaker”, and user B establishes a mapping relationship with the voice command "turn on B's speaker”. If the first voice command is "turn on A's speaker”, it is determined that the first user who issued the first voice command is user A.
  • the user who needs to interact is determined based on the voice command, and then the position in the car of the user who needs to interact is determined according to the mapping relationship between the user in the car and the position in the car, even when multiple users are talking at the same time. It can be located to the location of the user who needs to interact, and can execute voice commands according to the user's location to achieve one-to-one interaction and improve user experience.
  • executing the first voice command according to the position in the vehicle of the first user includes: controlling the position of the projection interface according to the position in the vehicle of the first user.
  • the projection interface can be used to display the projected image.
  • the position of the projection interface can be controlled by the rotation angle of the projection component.
  • the projection assembly may include a projection frame and a steering motor.
  • the projection frame can be used for projection, for example, to display the three-dimensional image of the voice assistant.
  • the three-dimensional image is displayed on the projection interface, that is to say, the position of the three-dimensional image can be controlled according to the position in the vehicle of the first user.
  • the first voice command is not necessarily a command for controlling the position of the projection interface, that is, the first voice command can be any voice command.
  • the first voice command may be adjusting the temperature of the air conditioner, and executing the first voice command includes adjusting the temperature of the air conditioner and controlling the position of the projection interface.
  • the projection image can be displayed through the projection interface, and the position of the projection interface can be adjusted according to the position in the car, so that the projection image can face the user who needs to interact and provide the user with a one-to-one interactive experience.
  • executing the first voice command according to the position in the vehicle of the first user includes: controlling a speaker corresponding to the position in the vehicle of the first user.
  • the first voice command may be to play music.
  • Controlling the speaker corresponding to the position in the vehicle of the first user may include turning on the speaker and playing corresponding music.
  • executing the first voice command according to the position in the vehicle of the first user includes: controlling the air conditioner corresponding to the position in the vehicle of the first user.
  • the first voice command is to turn on the air conditioner.
  • Controlling the air conditioner corresponding to the position in the vehicle of the first user may include turning on the air conditioner corresponding to the position in the vehicle of the first user.
  • an in-vehicle device which includes: a storage unit for storing the mapping relationship between a user in the car and a position in the car; a processing unit for: acquiring the sound signal collected by the microphone in the car; When the first voice command is recognized, determine the first user who issued the first voice command; obtain the mapping relationship between the user in the car and the position in the car; determine the position in the car of the first user according to the mapping relationship between the user in the car and the position in the car .
  • the processing unit is specifically configured to: perform voiceprint feature matching on the first voice command, and identify the first user who issued the first voice command.
  • the mapping relationship between the user in the vehicle and the location in the vehicle is determined based on the face image collected by the vehicle camera.
  • the mapping relationship between the user in the car and the position in the car is determined according to the position of the camera and the facial image collected by the camera, including: obtaining user information; according to user information
  • the mapping relationship with the face image is used to determine the user corresponding to the face image; the user's position in the car is determined according to the position and/or angle of the camera.
  • the processing unit is further configured to: perform sound source localization according to the sound signal to obtain at least one sound source position; The position corrects the position in the car of the first user.
  • the processing unit is specifically configured to: determine the position of the sound source that is closest to the position in the vehicle of the first user among the at least one sound source position as the corrected second position. The location of a user in the car.
  • the processing unit is further configured to execute the first voice command according to the position in the vehicle of the first user.
  • the processing unit is specifically configured to: control the position of the projection interface according to the position in the vehicle of the first user.
  • a vehicle including the device in the third aspect.
  • an in-vehicle device including an input and output interface, a processor, and a memory.
  • the processor is used to control the input and output interface to send and receive information
  • the memory is used to store a computer program
  • the processor is used to call and run the computer program from the memory, so that the vehicle-mounted device executes the method in the first aspect or the second aspect.
  • the foregoing device may be a terminal device/server, or a chip in the terminal device/server.
  • the aforementioned memory may be located inside the processor, for example, may be a cache in the processor.
  • the above-mentioned memory may also be located outside the processor so as to be independent of the processor, for example, the internal memory (memory) of the device.
  • first aspect includes any one of the implementation manners of the first aspect
  • second aspect includes any one of the implementation manners of the second aspect
  • third aspect includes any one of the implementation manners of the third aspect.
  • a computer program product includes: computer program code, which when the computer program code runs on a computer, causes the computer to execute the methods in the foregoing aspects.
  • the above-mentioned computer program code may be stored in whole or in part on a first storage medium, where the first storage medium may be packaged with the processor, or may be packaged separately with the processor. There is no specific limitation.
  • a computer-readable medium stores program code, and when the computer program code runs on a computer, the computer executes the methods in the above-mentioned aspects.
  • a chip in an eighth aspect, includes a processor and a data interface, and the processor reads instructions stored in a memory through the data interface, and executes the methods in the foregoing aspects.
  • the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory.
  • the processor is used to execute the methods in the above aspects.
  • Figure 1 is a schematic diagram of a vehicle provided by an embodiment of the present application.
  • Fig. 2 is a schematic structural diagram of a microphone array provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of the energy spectrum provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of the structure of the vehicle-mounted system provided by an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of a method for locating users in a vehicle provided by an embodiment of the present application
  • FIG. 6 is a schematic structural diagram of an in-vehicle hardware configuration provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of correcting the position in the vehicle of a first user provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a vehicle-mounted interaction method provided by an embodiment of the present application.
  • Fig. 9 is a schematic structural diagram of a projection frame provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a method for updating a user's in-vehicle location according to an embodiment of the present application
  • FIG. 11 is a schematic flowchart of state transition of seats in a vehicle provided by an embodiment of the present application.
  • FIG. 12 is a schematic flowchart of an initialization process for updating seats in a vehicle provided by an embodiment of the present application.
  • FIG. 13 is a schematic flowchart of a verification process for updating seats in a vehicle provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a vehicle-mounted system provided by an embodiment of the present application.
  • FIG. 15 is a schematic block diagram of a vehicle-mounted device provided by an embodiment of the present application.
  • FIG. 16 is a schematic block diagram of a vehicle-mounted device provided by an embodiment of the present application.
  • Fig. 1 is a functional block diagram of a vehicle 100 provided by an embodiment of the present invention.
  • the vehicle 100 may include various subsystems, such as a travel system 102, a sensor system 104, a control system 106, one or more peripheral devices 108 and a power supply 110, a computer system 112, and a user interface 116.
  • the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements.
  • each of the subsystems and elements of the vehicle 100 may be wired or wirelessly interconnected.
  • the travel system 102 may include components that provide power movement for the vehicle 100.
  • the propulsion system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels/tires 121.
  • the sensor system 104 may include several sensors that sense information about the environment around the vehicle 100.
  • the sensor system 104 may include a positioning system 122 (the positioning system may be a GPS system, a Beidou system or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and Camera 130.
  • the sensor system 104 may also include sensors of the internal system of the monitored vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and identification are key functions for the safe operation of the autonomous vehicle 100.
  • the control system 106 controls the operation of the vehicle 100 and its components.
  • the control system 106 may include various components, including a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.
  • the computer vision system 140 may be operable to process and analyze the images captured by the camera 130 in order to identify objects and/or features in the surrounding environment of the vehicle 100.
  • the objects and/or features may include traffic signals, road boundaries and obstacles.
  • the computer vision system 140 may also be operated to process and analyze the images captured by the camera 130 in order to identify users and/or features in the internal environment of the vehicle 100.
  • the computer vision system 140 may use object recognition algorithms, face recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision technologies.
  • control system 106 may add or alternatively include components other than those shown and described. Alternatively, a part of the components shown above may be reduced.
  • the vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through peripheral devices 108.
  • the peripheral device 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and/or a speaker 152.
  • the peripheral device 108 provides a means for the user of the vehicle 100 to interact with the user interface 116.
  • the onboard computer 148 may provide information to the user of the vehicle 100.
  • the user interface 116 can also operate the on-board computer 148 to receive user input.
  • the on-board computer 148 can be operated through a touch screen.
  • the peripheral device 108 may provide a means for the vehicle 100 to communicate with other devices located in the vehicle.
  • microphone 150 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 100.
  • the on-board computer 148 can be operated by receiving a user's voice command or other audio input.
  • the speaker 152 may output audio to the user of the vehicle 100.
  • the power supply 110 may provide power to various components of the vehicle 100. Part or all of the functions of the vehicle 100 are controlled by the computer system 112.
  • the computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer readable medium such as a data storage device 114.
  • the computer system 112 may also be multiple computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.
  • the processor 113 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor may be a dedicated device such as an ASIC or other hardware-based processor.
  • FIG. 1 functionally illustrates the processor, the memory, and other elements of the computer 110 in the same block, those of ordinary skill in the art should understand that the processor, the computer, or the memory may actually include may or may not Multiple processors, computers, or memories stored in the same physical enclosure.
  • the memory may be a hard disk drive or other storage medium located in a housing other than the computer 110. Therefore, a reference to a processor or computer will be understood to include a reference to a collection of processors or computers or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described here, some components such as steering components and deceleration components may each have its own processor that only performs calculations related to component-specific functions .
  • the processor may be located away from the vehicle and wirelessly communicate with the vehicle.
  • some of the processes described herein are executed on a processor disposed in the vehicle and others are executed by a remote processor, including taking the necessary steps to perform a single manipulation.
  • the data storage device 114 may include instructions 115 (eg, program logic), which may be executed by the processor 113 to perform various functions of the vehicle 100, including those described above.
  • the data storage device 114 may also contain additional instructions, including sending data to, receiving data from, interacting with, and/or performing data on one or more of the propulsion system 102, the sensor system 104, the control system 106, and the peripheral device 108. Control instructions.
  • the data storage device 114 may also store data, such as road maps, route information, the location, direction, and speed of the vehicle, and other such vehicle data, as well as other information. Such information may be used by the vehicle 100 and the computer system 112 during the operation of the vehicle 100 in autonomous, semi-autonomous, and/or manual modes. In the embodiment of the present application, the data storage device 114 may also store related information of the user. Such information can be used by the vehicle 100 and the computer system 112 during human-computer interaction.
  • the user interface 116 is used to provide information to or receive information from a user of the vehicle 100.
  • the user interface 116 may include one or more input/output devices in the set of peripheral devices 108, such as a wireless communication system 146, an in-vehicle computer 148, a microphone 150, and a speaker 152.
  • the computer system 112 may control the functions of the vehicle 100 based on inputs received from various subsystems (for example, the travel system 102, the sensor system 104, and the control system 106) and from the user interface 116. For example, the computer system 112 may utilize input from the user interface 116 in order to control the peripheral device 108 for human-computer interaction. In some embodiments, the computer system 112 is operable to provide control of many aspects of the vehicle 100 and its subsystems.
  • one or more of these components described above may be installed or associated with the vehicle 100 separately.
  • the data storage device 114 may exist partially or completely separately from the vehicle 100.
  • the above-mentioned components may be communicatively coupled together in a wired and/or wireless manner.
  • FIG. 1 should not be construed as a limitation to the embodiment of the present invention.
  • the aforementioned vehicle 100 may be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawn mower, recreational vehicle, playground vehicle, construction equipment, tram, golf cart, train, and trolley, etc.
  • the embodiments of the invention are not particularly limited.
  • the method for locating users in a vehicle provided by the embodiments of the present application can be applied to all scenarios where users in a vehicle need to be located. Specifically, the method for locating users in a car provided by the embodiments of the present application can be applied to scenes that require locating users in a car, such as music playback, air conditioning control, seat heating, and seat adjustment.
  • the two commonly used application scenarios are briefly introduced below.
  • the speakers 152 in FIG. 1 may be distributed in multiple locations in the vehicle. Specifically, speakers in different locations can be controlled to play different music respectively to realize the function of partitioned music playback, so that users in different locations can turn on/off the speakers or play different music as needed. For example, when a user needs to play music, a voice command can be issued.
  • the voice assistant can locate the user who issued the voice command and control the speaker near the user according to the voice command, for example, turn on the speaker and play corresponding music.
  • locating the user who issued the voice command may be performed by the method for locating the user in the vehicle in the embodiment of the present application. Even when multiple users are speaking at the same time in the car, the user positioning method in the car in the embodiment of the present application can accurately locate the user who issued the voice command, and then control the speaker near the user according to the voice command. This does not affect the normal communication of other users, and at the same time improves the accuracy of user positioning in the car, avoids interference with others, and meets the needs of different users.
  • the vehicle 100 in FIG. 1 may also include air conditioners, and the air conditioners may be distributed in multiple locations in the vehicle.
  • the air conditioners in different locations can be controlled to adjust the temperature of different locations, and the function of zoned air conditioner control can be realized, so that users in different locations can adjust the nearby temperature as needed.
  • a voice command can be issued.
  • the voice assistant can locate the user who issued the voice command, and control the air conditioner near the user according to the voice command, and adjust the temperature of the air conditioner according to the user's needs.
  • locating the user who issued the voice command may be performed by the method for locating the user in the vehicle in the embodiment of the present application. Even when multiple users are speaking at the same time in the car, the user positioning method in the car in the embodiment of the present application can accurately locate the user who issued the voice command, and control the air conditioner near the user according to the voice command. This does not affect the normal communication of other users, and at the same time improves the accuracy of user positioning in the car, avoids interference with others, and meets the needs of different users.
  • the user positioning method in the car provided by the embodiments of this application can be used to locate users who need to use the corresponding function, and enable the corresponding function as needed to improve The user’s interactive experience.
  • the MUSIC algorithm uses the orthogonality of the signal subspace and the noise subspace to calculate the energy distribution of the signal at different angles, so as to locate the position where the signal peak appears. When multiple people speak at different angles at the same time, the algorithm will generate multiple peaks.
  • FIG. 2 shows a schematic structural diagram of a microphone array.
  • M is 4, that is, a microphone array is composed of 4 microphones with a spacing of d.
  • the signal received by the microphone array can be as shown in formula (1):
  • X(t) is the received signal
  • t is the time
  • S(t) is the signal strength of the signal source on the array element
  • A is the influence of different microphones on the received signal, which is a function of angle ⁇
  • ⁇ is the received signal.
  • N(t) is noise.
  • the covariance matrix is composed of signal subspace and noise subspace, which can be specifically shown in formula (2):
  • R(t) is the covariance matrix
  • H is the matrix conjugate transpose
  • ⁇ 2 is the noise power
  • I is the identity matrix of order M*M
  • R S is the correlation matrix of the signal.
  • the estimated value of the maximum likelihood can be obtained, as shown in formula (3):
  • R x is the estimated value of the maximum likelihood
  • N is the number of samples.
  • a( ⁇ ) is the response of the microphone to the signal source whose azimuth angle is ⁇
  • P music ( ⁇ ) is the energy spectrum, which includes one or more peaks, and the position of the sound source is obtained by searching for the peaks. That is to say, at the same time, multiple sound sources speaking at the same time can be displayed in the energy spectrum.
  • Figure 3 shows a schematic diagram of the energy spectrum when multiple people speak at the same time. In Figure 3, two people are speaking at 0° and 60° relative to the microphone array at the same time.
  • a neural network can be composed of neural units.
  • a neural unit can refer to an arithmetic unit that takes x s and intercept 1 as inputs.
  • the output of the arithmetic unit can be as shown in formula (5):
  • s 1, 2,...n, n is a natural number greater than 1
  • W s is the weight of x s
  • b is the bias of the neural unit.
  • f is the activation function of the neural unit.
  • the activation function is used to perform non-linear transformation of the features in the neural network and convert the input signal in the neural unit into an output signal.
  • the output signal of the activation function can be used as the input of the next convolutional layer.
  • the activation function can be a sigmoid function.
  • a neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit.
  • the input of each neural unit can be connected with the local receptive field of the previous layer to extract the characteristics of the local receptive field.
  • the local receptive field can be a region composed of several neural units.
  • Deep neural network also known as multi-layer neural network
  • the DNN is divided according to the positions of different layers.
  • the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer.
  • the first layer is the input layer
  • the last layer is the output layer
  • the number of layers in the middle are all hidden layers.
  • the layers are fully connected, that is to say, any neuron in the i-th layer must be connected to any neuron in the i+1th layer.
  • DNN looks complicated, it is not complicated in terms of the work of each layer. Simply put, it is the following linear relationship expression: in, Is the input vector, Is the output vector, Is the offset vector, W is the weight matrix (also called coefficient), and ⁇ () is the activation function.
  • Each layer is just the input vector After such a simple operation, the output vector is obtained Due to the large number of DNN layers, the coefficient W and the offset vector The number is also relatively large.
  • DNN The definition of these parameters in DNN is as follows: Take the coefficient W as an example, suppose that in a three-layer DNN, the linear coefficients from the fourth neuron in the second layer to the second neuron in the third layer are defined as The superscript 3 represents the number of layers where the coefficient W is located, and the subscript corresponds to the output third-level index 2 and the input second-level index 4.
  • the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as
  • the neural network can use the backpropagation (BP) algorithm to modify the values of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, forwarding the input signal to the output will cause error loss, and the parameters in the initial neural network model are updated by backpropagating the error loss information, so that the error loss is converged.
  • the back-propagation algorithm is a back-propagation motion dominated by error loss, and aims to obtain the optimal parameters of the neural network model, such as the weight matrix.
  • the sound source positioning system is usually used to locate the user in the car.
  • the sound source localization system mainly includes a sound source localization system based on a microphone array and a sound source localization system based on the human hearing mechanism.
  • the sound source localization based on the microphone array refers to the use of microphones to pick up sound signals, and the position information of one or more sound sources is obtained by analyzing and processing each output signal of the microphone array.
  • the above solution cannot distinguish which user or several users are speaking.
  • the system cannot determine the direction of interaction and cannot provide a one-to-one interactive experience.
  • Fig. 4 shows a schematic structural diagram of a vehicle-mounted system in an embodiment of the present application.
  • the in-vehicle system 200 in FIG. 4 includes a voice assistant component 210, a voice processing unit 220, and a database 230.
  • the voice assistant component 210 includes a microphone 211.
  • the microphone 211 may refer to an energy conversion device that converts a sound signal into an electric signal.
  • the microphone 211 may be a microphone array.
  • the microphone array refers to an array composed of multiple microphones at a certain distance, for example, a linear array or a circular array.
  • the sound signal collected by the microphone 211 may be transmitted to the voice processing unit 220 for processing.
  • the voice assistant component 210 may further include a steering motor 212.
  • the steering motor 212 may be a stepping motor that controls rotation.
  • the steering motor 212 may be a stepping motor used to control the rotation of the 3D projection assembly.
  • the steering motor 212 can receive the position information sent by the positioning module 222 and rotate accordingly.
  • the voice processing unit 220 includes a voice recognition module 221 and a positioning module 222.
  • the voice recognition module 221 is used for recognizing voice commands in the voice signal collected by the microphone.
  • the voice command may be a wake-up word.
  • the wake word can be used to activate the processing flow of the system 200.
  • the user who issued the voice command is determined according to the mapping relationship between the voice command and the user. In other words, different users can set different voice commands.
  • the voice processing unit 220 may further include a voiceprint recognition module 223.
  • the voiceprint recognition module 223 is used to recognize the user according to the sound wave characteristics of the speaking user.
  • the voice signal may be sent to the voiceprint recognition module 223.
  • the voiceprint recognition module 223 may be used to perform voiceprint recognition on the voice command according to the mapping relationship between the user and the voiceprint feature of the user, and determine the user who issued the voice command.
  • voiceprint recognition includes: voice signal processing, voiceprint feature extraction, voiceprint comparison, and discriminative decision-making.
  • voiceprint feature extraction refers to the extraction and selection of acoustic or language features that are highly separable and stable to the speaker's voiceprint.
  • voiceprint comparison and discriminative decision-making refer to voiceprint pattern matching (voiceprint pattern recognition). Specifically, the extracted feature data of the voiceprint and the feature template (voiceprint feature) stored in the database 230 are searched and matched. For example, neural network methods, nearest neighbor methods, etc. can be used.
  • voiceprint recognition module 223 may be deployed in a vehicle, or the voiceprint recognition module 223 may also be deployed in the cloud.
  • voiceprint recognition module 223 is only an example.
  • the voiceprint recognition module 223 is used to recognize the user by collecting the voice of the speaking user.
  • the voiceprint recognition module 223 may be in any form. There are no restrictions.
  • the positioning module 222 may determine the in-vehicle position of the user who issued the voice command according to the mapping relationship between the in-vehicle user and the in-vehicle position, that is, the position of the user who needs to interact. Wherein, the mapping relationship between the user in the vehicle and the location in the vehicle may be stored in the database 230.
  • the positioning module 222 may feed back the position information to the voice assistant component 210, so that the voice assistant component 210 performs subsequent operations.
  • the steering motor 212 is controlled to rotate.
  • the localization module 222 can also be used to implement sound source localization and obtain at least one sound source location.
  • the voice recognition module may transmit the sound signal collected by the microphone to the positioning module 222 for use in realizing sound source positioning.
  • Sound source localization refers to locating the direction and distance of one or more sound sources based on sound signals.
  • the sound source localization can be implemented using methods such as the MUSIC algorithm and the time difference of arrival (TDOA) method.
  • the positioning module 222 may correct the position of the user who issued the voice command according to the position of the vehicle interior obtained by the voiceprint recognition module 223 and the position of at least one sound source obtained by the sound source positioning, so as to obtain a more accurate vehicle interior position ⁇ that needs to be interacted. .
  • positioning module 222 may be deployed in a vehicle, or the positioning module 222 may also be deployed in the cloud.
  • the positioning module 222 is only an example.
  • the positioning module 222 may be used to locate the user's position by collecting the voice signal of the speaking user.
  • the sound source positioning method may be in any form, which is not described in this embodiment of the application. Any restrictions.
  • the system 200 may further include an image acquisition unit 250 and an image processing unit 240.
  • the image acquisition unit 250 includes a camera 251.
  • the camera 251 can have basic functions such as video capture/transmission or static image capture. After the image is captured by the lens, the image is processed by the photosensitive component circuit and control component in the camera and converted into a digital signal for subsequent signals. deal with.
  • the image collection unit 250 may send the collected face image to the image processing unit 240.
  • the image processing unit 240 includes a face recognition module 241.
  • the face recognition module 241 is used for recognizing the user according to the face image of the user. Specifically, the face recognition module 241 may be used to determine the user corresponding to the face image according to the mapping relationship between the user and the facial features of the user. Wherein, the mapping relationship between the user and the facial features of the user may be stored in the database 230.
  • face recognition includes: face image preprocessing, face image feature extraction, and matching and recognition.
  • face image preprocessing refers to the process of processing the image based on the result of face detection and ultimately serving the feature extraction. For example, due to various conditions and random interference, the original image acquired by the system cannot be used directly. It is necessary to perform image preprocessing such as gray-scale correction and noise filtering in the early stage of image processing.
  • the aforementioned face feature extraction method may include: a knowledge-based representation method and a representation method based on algebraic features or statistical learning.
  • the aforementioned face image matching and recognition refers to the process of searching and matching the extracted feature data of the face image with the feature template (face feature) stored in the database 230; by setting a preset threshold, when the similarity exceeds If the threshold is preset, the result of the matching is output.
  • Face recognition is to compare the facial features to be recognized with the facial feature templates that have been obtained, and judge the users corresponding to the faces based on the degree of similarity.
  • the aforementioned face recognition module 241 may be deployed in a vehicle, or the aforementioned face recognition module 241 may also be deployed in the cloud.
  • the face recognition module 241 is only an example.
  • the face recognition module 241 is used to identify the user by collecting images of the user.
  • the face recognition module 241 may be in any form, which is not described in this embodiment of the application. Any restrictions.
  • the image processing unit 240 further includes a mapping module 242.
  • the mapping module 242 may establish a mapping relationship between the user's in-vehicle location and the user, and store the mapping relationship between the in-vehicle user and the in-vehicle location in the database 230.
  • the position in the vehicle of the user may be determined according to the angle and/or position of the camera 251.
  • the image processing unit 240 may also be used to periodically update the mapping relationship between the user in the vehicle and the location in the vehicle.
  • the database 230 may include a user information database 231.
  • the user information database 231 may be used to store the mapping relationship between the voiceprint features of the user and the user and the mapping relationship between the facial features of the user and the user.
  • the user information database 231 may store information of all users who have registered on the vehicle.
  • the database 230 may also include a real-time information database 232.
  • the real-time information database 232 may be used to store the mapping relationship between the users in the vehicle and the positions in the vehicle.
  • the mapping relationship between the user and the user may be determined based on the facial image collected by the camera.
  • the user may be indicated through the user's voiceprint information.
  • the mapping relationship between the voiceprint features of the user and the location in the vehicle can be obtained according to the mapping relationship between the voiceprint features of the user and the user and the mapping relationship between the user in the vehicle and the location in the vehicle.
  • the real-time information database 232 can be used to store the mapping relationship between the user's voiceprint feature and the position in the vehicle.
  • the positioning module 222 can match the user's voiceprint characteristics in the real-time information database 232 to determine the user's in-vehicle position.
  • the mapping relationship between the voiceprint feature of the user and the position in the vehicle may be updated as the mapping relationship between the user in the vehicle and the position in the vehicle is updated.
  • FIG. 5 shows a method 300 for locating users in a car provided by an embodiment of the present application.
  • the method 300 for locating users in a car can be executed by a device or system capable of locating users in a car, for example, by the in-vehicle system shown in FIG. 4 200 execution.
  • the method 300 includes steps S310 to S350. Steps S310 to S350 will be described in detail below.
  • S310 Acquire a sound signal collected by a microphone in the vehicle.
  • the microphone in the vehicle may be one microphone or multiple microphones.
  • the multiple microphones may be a microphone array.
  • the microphone array can be set as required, for example, the microphone array can be a uniform circular array or a uniform linear array.
  • Fig. 6 shows a schematic structural diagram of a hardware configuration in a vehicle.
  • the microphone array in FIG. 6 includes four microphones 410, which are located in a linear arrangement on the bridge in front of the cockpit.
  • voice recognition may be performed on the voice signal to determine whether the voice signal contains the first voice command.
  • the voice recognition can be performed by the voice recognition module 221 shown in FIG. 4.
  • the first voice command may be a wake-up word
  • the wake-up word may be used to wake up the voice assistant and activate subsequent processing procedures.
  • the manner of determining the first user who issued the first voice command may include multiple manners.
  • the embodiment of the present application does not limit the manner of determining the first user.
  • voiceprint feature matching may be performed on the first voice command to identify the first user who issued the first voice command.
  • the voiceprint recognition of the first voice command may be performed according to the mapping relationship between the voiceprint features of the user and the user, and the first user who issued the first voice command may be determined. In this way, users who need to interact can be identified based on the voiceprint characteristics, and the positioning accuracy is improved.
  • the database 230 stores the mapping relationship between the voiceprint features of the user and the user. Different voiceprint features may correspond to different users, and the voiceprint features may be used to indicate the user.
  • the voiceprint recognition is performed by the voiceprint recognition module 223 of FIG. 4, and the feature data of the voiceprint corresponding to the extracted first voice command is searched and matched with the feature template (voiceprint feature) stored in the database 230, and the matching is obtained.
  • the first voiceprint feature that is, the first user is determined.
  • the first voiceprint feature is voiceprint information of the first user.
  • the voiceprint information may not be recorded by the user, for example, the user may ride the car for the first time. In this case, the user can be reminded to supplement the voiceprint feature and update the information in the database.
  • the first user corresponding to the first voice command may be determined according to the mapping relationship between the user and the voice command.
  • the mapping relationship between the user and the voice command can be stored in the database 230.
  • different voice commands can be set for different users in advance, and the voice commands can be bound to the users, that is, a mapping relationship is established.
  • the mapping relationship is stored in the database 230 as a data pair.
  • user A establishes a mapping relationship with the voice command "turn on A's speaker”
  • user B establishes a mapping relationship with the voice command "turn on B's speaker”. If the first voice command is "turn on A's speaker”, it is determined that the first user who issued the first voice command is user A.
  • S330 Determine the in-vehicle position of the first user according to the mapping relationship between the in-vehicle user and the in-vehicle position.
  • the method for acquiring the mapping relationship between the in-vehicle user and the in-vehicle location may include multiple methods.
  • the mapping relationship between the user in the car and the position in the car may be pre-stored, or it may be obtained from other devices, such as a cloud server, or it may be updated in real time.
  • the embodiment of the present application does not limit the method for obtaining the mapping relationship between the user in the car and the position in the car.
  • the position in the vehicle may be represented by an angle range.
  • the position in the vehicle can also be represented by an angle value.
  • the interior of the vehicle may be divided into multiple areas, and the location of the interior of the vehicle may be represented by the area within the vehicle.
  • the area in the car can be the seat in the car, that is, the position in the car is indicated by the seat in the car. This can reduce the number of options in the vehicle, reduce the amount of calculation, and increase the processing speed.
  • the location in the vehicle may be a specific location in the vehicle. In this way, even if the user is not sitting in a certain seat, for example, the user is sitting between two seats or the user is standing in the aisle in the car, the accurate position of the user can be obtained.
  • mapping relationship between the user in the vehicle and the location in the vehicle may be stored in the database 230.
  • Step S330 may be performed by the aforementioned positioning module 222.
  • the method for determining the mapping relationship between the in-vehicle user and the in-vehicle location may include multiple methods.
  • the mapping relationship between the user in the vehicle and the location in the vehicle may be determined according to the face image collected by the vehicle-mounted camera.
  • the mapping relationship between the user and the inside of the car is determined according to the face image collected by the vehicle camera, without the user's cooperation, which can avoid disturbing the user and improve the user experience.
  • the user corresponding to the face image may be determined according to the mapping relationship between the user information and the face image; the user's in-vehicle position may be determined according to the position and/or angle of the vehicle-mounted camera. That is to establish a mapping relationship between the user and the user's in-vehicle location.
  • the user information acquisition method may include multiple methods.
  • user information may be pre-stored, or obtained from other devices, such as a cloud server, or updated in real time.
  • the embodiment of the application does not limit the method for obtaining user information.
  • the user corresponding to the face image is determined, that is, the face recognition is performed on the face image, and the user corresponding to the face image is determined.
  • mapping relationship between user information and facial images can also be understood as the mapping relationship between the user and the facial features of the user.
  • the database 230 may store the mapping relationship between the facial features of the user and the user. Different facial features may correspond to different users, and the facial features may be used to indicate the user. Face recognition is performed by the face recognition module 241 in FIG. 4, and the feature data of the extracted face image is searched and matched with the feature template (face feature) stored in the database 230, and the matching face feature is obtained. That is, the user corresponding to the face image is determined.
  • the voiceprint feature corresponding to the face image cannot be recognized, that is, the corresponding user cannot be matched in the database
  • the user may not save the facial feature, for example, the user may ride the car for the first time. In this case, the user can be reminded to add facial features and update the information in the database.
  • the user's information is stored in the database, and data sharing can be realized through the database.
  • the database stores the mapping relationship between the voiceprint features of the user and the user and the mapping relationship between the user in the vehicle and the location in the vehicle, so that the mapping relationship between the voiceprint information of the user and the location in the vehicle can be determined.
  • the position in the vehicle may be represented by an angle range.
  • the position in the vehicle can also be represented by an angle value.
  • the interior of the vehicle may be divided into multiple areas, and the location of the interior of the vehicle may be represented by the area within the vehicle.
  • the area in the car can be the seat in the car, that is, the position in the car is indicated by the seat in the car.
  • the user's seat range can be determined according to the layout of the cockpit. This can reduce the number of options in the vehicle, reduce the amount of calculation, and increase the processing speed.
  • the location in the vehicle may be a specific location in the vehicle. In this way, even if the user is not sitting in a certain seat, for example, the user is sitting between two seats or the user is standing in the aisle in the car, the accurate position of the user can be obtained.
  • the vehicle-mounted camera may be an in-vehicle camera or an outside-vehicle camera.
  • the camera outside the vehicle can be installed on the B-pillar of the vehicle.
  • the following is an example of determining the user's position in the car through the in-car camera.
  • Method 1 Determine the location of the user in the vehicle through the mapping relationship between the location of the camera in the vehicle and the location in the vehicle.
  • mapping relationship between the position of the camera and the position in the vehicle can also be understood as the mapping relationship between different cameras and the position in the vehicle.
  • the location of the user captured by different cameras is the location in the vehicle corresponding to the camera.
  • the position in the car can be indicated by the seat in the car.
  • the mapping relationship between the position of the camera and the position in the vehicle may be the mapping relationship between the position of the camera and the seat in the vehicle.
  • the cameras in different positions correspond to the seats in the car one-to-one, and the position of the user captured by the different cameras is the seat in the car corresponding to the camera.
  • the camera 420 may be provided, which are respectively located in front of the four seats.
  • the camera 420 may be set on the driver's platform; for the rear seat, the camera 420 may be set on the back of the front seat.
  • the user captured by the camera in front of the passenger's seat can be considered as the passenger's seat in the vehicle, that is, the shaded area in FIG. 6.
  • the area can be -20° to -45°. It should be understood that the angle can be set according to needs, and the angles corresponding to different cockpit layouts of different vehicle types are also different.
  • Manner 2 Determine the user's in-vehicle position through the mapping relationship between the rotation angle of the in-vehicle camera and the in-vehicle position.
  • the camera is located on the rear view mirror of the car, and the rotation angle of the camera corresponds to the position in the car.
  • the corresponding position in the car is the middle of the car.
  • the position of the user captured by the camera is the middle of the car;
  • the rotation angle of the camera When it is the angle A, the corresponding position in the car is the position A in the car.
  • the position of the user captured by the camera is the position A in the car.
  • the position in the car can be indicated by the seat in the car.
  • the mapping relationship between the rotation angle of the camera and the position in the vehicle may be the mapping relationship between the rotation angle of the camera and the seat in the vehicle.
  • the camera is located on the rear view mirror of the car.
  • the seat in the car corresponds to the rotation angle of the camera.
  • the user captured by cameras located at different rotation angles is located in the seat in the vehicle corresponding to the rotation angle of the camera.
  • the position in the car is represented by the seat in the car, so that the rotation angle can be set according to the seat, which can reduce the amount of calculation during the positioning process and increase the processing speed.
  • determining the user's in-vehicle position through the camera may include: determining by the mapping relationship between the field of view range of the camera and the in-vehicle position The vehicle location of the user captured by this camera.
  • the camera's field of view corresponds to the center of the car, for example, the center line of the field of view is parallel to the car.
  • the user's position captured by the camera is the center of the car.
  • the field of view of the camera corresponds to the position in the car as position A, in this case, the position of the user captured by the camera is position A.
  • the user corresponding to the facial image is determined according to the mapping relationship between the facial features of the user and the user, and the position in the vehicle of the user is determined according to the relative position of the user in the lens of the vehicle-mounted camera.
  • the position in the car can be indicated by the seat in the car, so that the relative position in the lens can be mapped to the seat in the car, which can reduce the amount of calculation during the positioning process and increase the processing speed.
  • the vehicle-mounted camera is located on the rear-view mirror of the vehicle, and all users in the vehicle can be put into the viewfinder in one shot.
  • the user corresponding to the face image and its relative position in the lens are determined through face recognition, and the in-vehicle position corresponding to the user is determined according to the relative position.
  • the vehicle-mounted camera is located on the B-pillar outside the vehicle, and the vehicle-mounted cameras can be set on the B-pillars on both sides.
  • the user corresponding to the face image and the relative position in the lens are determined through face recognition. For example, if the relative position of the user in the camera is on the right side, the user gets in the car from the front door on the left side of the vehicle, and the position in the car corresponding to the user may be on the driver's seat.
  • the number of cameras can be one or more, and the position of the camera or the number of the camera can be set as required.
  • the mapping relationship between the user in the vehicle and the location in the vehicle may be stored in the real-time information database 232.
  • the real-time information database may be a database established by the user after riding in the car.
  • step S310 to step S330 can be executed.
  • mapping relationship between the user and the position in the car can be searched in the real-time information database, avoiding the need to re-determine the mapping relationship between the user in the car and the position in the car every time the user is located.
  • the number of users in the real-time information database is less than or equal to the number of users in the database. Searching for the mapping relationship in the real-time information database can reduce the search range and increase the processing speed.
  • the mapping relationship between the user in the vehicle and the location in the vehicle may be updated in real time, or may also be updated regularly.
  • a specific method of updating the mapping relationship between the user in the vehicle and the position in the vehicle, or the method of updating the real-time information database refer to method 600 in the following text.
  • mapping relationship between the user in the vehicle and the location in the vehicle may be established in real time.
  • step S310 and step S320 are executed to determine the first user.
  • Obtain the facial image taken by the camera and determine the user corresponding to the facial image according to the mapping relationship between the facial features of the user and the user.
  • the in-vehicle position is to establish a mapping relationship between the first user and the in-vehicle position of the first user, and determine the in-vehicle position of the first user.
  • mapping relationship between the in-vehicle user and the in-vehicle location can be established only when the user in the vehicle needs to be located, avoiding unnecessary calculations, and at the same time, the mapping relationship between the first user and its location can be established only for the first user, reducing the calculation quantity.
  • mapping relationship between the in-vehicle user and the in-vehicle location is only an example, and the embodiment of the present application does not limit the manner in which the mapping relationship between the in-vehicle user and the in-vehicle location is determined.
  • fingerprint collectors can be set at different locations in the car.
  • the fingerprint is collected by the fingerprint collector, and the user corresponding to the fingerprint collected by the fingerprint collector is determined according to the mapping relationship between the fingerprint characteristics of the user and the user. Since different fingerprint collectors correspond to different positions in the car, a mapping relationship between users and positions in the car can be established. For example, the fingerprint collector A corresponds to the position A in the car, the fingerprint A is collected by the fingerprint collector A, and the fingerprint A is determined to correspond to the user A through fingerprint identification, and then the user A and the position A in the car are mapped.
  • mapping relationship between the user in the vehicle and the location in the vehicle may be updated regularly.
  • mapping relationship between the users in the car and the position in the car is updated regularly, which can improve the accuracy of user positioning.
  • mapping relationship between the user in the car and the position in the car is updated regularly, including:
  • mapping relationship between the user in the car and the position in the car it is determined that there is no user in the current position in the car, and the current face image collected by the on-board camera is obtained.
  • Current user Establish a mapping relationship between the current user and the current position in the car, and update the mapping relationship between the user in the car and the position in the car.
  • mapping relationship between the user in the car and the position in the car it is determined that the user corresponding to the current position in the car is user A, and the current facial image collected by the on-board camera is obtained.
  • the mapping relationship between the facial features of the user and the user the current The current user corresponding to the face image is user B; when user A is different from user B, a mapping relationship between user B and the current position in the car is established, and the mapping relationship between the user in the car and the position in the car is updated.
  • the method 300 further includes step S340 and step S350.
  • S340 Perform sound source localization according to the sound signal to obtain at least one sound source position.
  • a sound source localization method based on a microphone array can be used to localize the sound source of the sound signal.
  • the sound source localization method may be the MUSIC algorithm.
  • the microphone array can be set according to requirements.
  • the microphone array can be a uniform circular array or a uniform linear array.
  • Fig. 6 shows a microphone array, which includes 4 microphones, which are arranged in a linear arrangement on the bridge in front of the cockpit.
  • the distribution map of the signal energy at different angles can be obtained, and the peak value is the incoming direction of the signal.
  • the position of at least one sound source can be obtained. For example, as shown in Figure 2, when users at 0° and 60° speak at the same time, two peaks will be generated on the spectrum.
  • step S340 it should be understood that this is only for illustration, and the embodiment of the present application does not limit the specific manner of sound source localization, and other methods that can realize sound source localization may also be used in step S340.
  • S350 Correct the in-vehicle position of the first user according to the in-vehicle position of the first user and the at least one sound source position.
  • the in-vehicle position of the first user may be a default value, for example, -90° ⁇ 90°, also That is, the position in the vehicle of the first user is determined directly according to the position of the at least one sound source.
  • one of the at least one sound source positions that is closest to the in-vehicle position of the first user is determined as the corrected in-vehicle position of the first user.
  • the sound source position in the at least one sound source position that falls within the angular range is used as the corrected in-vehicle position of the first user Location.
  • FIG. 7 shows a schematic diagram of correcting the position in the vehicle of the first user.
  • the in-vehicle position of the first user determined by step S330 is in the range of 50° to 70°, the sound source location of the sound signal is performed to obtain an energy map about the sound source position, and the energy map is superimposed with the angle range of the first user, 50° to 70°
  • the middle value of the angular range can be taken as the corrected in-vehicle position of the first user. For example, if the angle range is from 20° to 40°, and there are multiple sound source positions in this range, 30° is used as the corrected in-vehicle position of the first user.
  • the angle range is from 20° to 40°, and there are multiple sound source positions in this range, 30° is used as the corrected in-vehicle position of the first user.
  • this solution can improve the accuracy of positioning the user and reduce the noisy environment, for example, the impact of multiple users talking at the same time on voice interaction.
  • the in-vehicle position of the first user cannot be determined in step S330, for example, the in-vehicle position of the first user is in the range of -90° to 90°, then the sound source position in this range is regarded as the corrected first user position.
  • the sound source position closest to the value among the at least one sound source position is taken as the corrected in-vehicle position of the first user.
  • the sound source localization can further improve the accuracy of the position of the first user in the car, and can accurately locate the user who needs to interact in a noisy environment in the car. In other words, combining the information collected by acoustics and vision can improve the accuracy and scope of sound source localization.
  • the user who needs to interact is determined based on the voice command, and then the position in the car of the user who needs to interact is determined according to the mapping relationship between the user in the car and the position in the car, even when multiple users are talking at the same time. It can locate the location of the user who needs to interact, realize one-to-one interaction, and improve the user experience.
  • the in-vehicle user positioning method in the embodiments of the present application can be applied to a variety of in-vehicle interaction scenarios, specifically, it can be applied to scenarios that need to locate a user.
  • in-vehicle voice assistants are relatively simple, most of which only interact through voice, and most of them are two-dimensional animation images.
  • 3D holographic projection is a 3D technology that uses the principles of interference and diffraction to record and reproduce real three-dimensional images of objects, and the audience can see three-dimensional virtual characters without glasses.
  • the basic principle of 3D holographic projection is that the interference principle is used to record the light wave information of the object during the shooting process, and the diffraction principle is used to reproduce the light wave information of the object during the imaging process, so as to reproduce the real three-dimensional image of the object.
  • 3D holographic projection may also include holographic-like projection technology that uses methods such as Peber's phantom and edge blanking to achieve 3D effects.
  • the laser and air projection system cannot maintain the projection effect for a long time, and most of the projection frames of the holographic projection are relatively fixed, which cannot achieve a better interactive experience.
  • Displaying the visual image of the voice assistant through three-dimensional projection can improve entertainment and significantly improve user experience.
  • the image of the voice assistant can face the user who needs to interact and provide a one-to-one interactive experience.
  • FIG. 8 shows an in-vehicle interaction method 500 provided by an embodiment of the present application.
  • the in-vehicle user positioning method involved in the in-vehicle interaction method can be implemented by the in-vehicle user positioning method 300.
  • the method 500 includes steps S510 to S570.
  • the method 500 may be executed by the system in FIG. 4.
  • the voice assistant component 210 in FIG. 4 may also include a projection component.
  • the projection assembly includes a projection frame and a steering motor 212.
  • the projection frame can be used to display the three-dimensional image of the voice assistant.
  • FIG. 9 shows a schematic structural diagram of a projection frame provided by an embodiment of the present application.
  • the projection frame in FIG. 9 mainly includes four parts: a frame body 910, a display screen 920, a rotating base 930, and a projection structure 940.
  • the frame body 910 is cylindrical tempered glass, and the rear and top of the frame are covered with light-shielding paint, which can prevent the visual effect caused by the diffuse reflection of sunlight from being weakened.
  • the display screen 920 is used for projection.
  • the shape of the display screen 920 can be set as required, for example, it can be a square display screen.
  • the display screen 920 is located on the top of the frame body, for example, it can be embedded inside the top of the cylinder.
  • the rotating base 930 is connected to a rotating motor in the cockpit, and is used to drive the projection structure 940 and the display screen 920 to rotate.
  • the frame body 910 remains stationary during the rotation.
  • the projection structure 940 is fixed on the rotating base 930 while being relatively fixed in position with the display screen 920, that is, the projection structure 940 and the display screen 920 remain relatively stationary during the rotation.
  • the projection structure 940 includes four isosceles trapezoidal tempered glass, and a holographic film is pasted on the tempered glass.
  • the rotating motor can be used to control the rotation of the projection frame.
  • the rotating motor may be a stepping motor.
  • the rotating electric machine can receive the result returned by the positioning module 222, for example, position information, and perform a corresponding steering operation according to the result, so that the three-dimensional image of the voice assistant faces the user who needs to interact.
  • the projection frame can be set on the center console and located on the center line of the cockpit.
  • the in-vehicle microphone 410, the camera 420, and the projection frame 430 may be arranged as shown in FIG. 6. It should be understood that FIG. 6 is only for illustration, and the positions of the microphone, camera, and projection frame in the vehicle can be set as required.
  • the centerline of the cockpit may be set to 0°, and the angular range of the sound source location is between -90° and 90°.
  • S510 Acquire a sound signal collected by a microphone.
  • Step S520 Determine whether the voice signal contains the first voice command. If the voice signal does not contain the first voice command, end the process and continue monitoring; if the voice signal contains the first voice command, perform step S530 and step S550. Step S530 and step S550 may be executed asynchronously.
  • voice recognition may be performed on the voice signal to determine whether the voice signal contains the first voice command.
  • the first voice command may be a wake-up word.
  • the voiceprint recognition is performed on the first voice command according to the mapping relationship between the voiceprint features of the user and the user, and the first user who issued the first voice command is determined.
  • step S320 in the aforementioned method 300.
  • S540 Determine the in-vehicle position of the first user according to the mapping relationship between the in-vehicle user and the in-vehicle position.
  • the real-time information database stores the mapping relationship between the user's voiceprint features and the seats in the vehicle. By comparing the voiceprint characteristics of the first voice command in the real-time information database, the corresponding seat in the car is determined.
  • mapping relationship between the user and the seat in the car is determined according to the facial image collected by the car camera, and then the user’s voiceprint feature is determined according to the mapping relationship between the user’s voiceprint feature and the user’s voiceprint feature.
  • the mapping relationship is stored in the real-time information database. For a detailed description, refer to step S330 in the aforementioned method 300.
  • the real-time information database may be updated in real time, or may also be updated regularly.
  • a detailed update method of the real-time information database please refer to method 600 in the following text.
  • the position in the vehicle may be an angular range.
  • the angle range may be -20° to -45°. It should be understood that the angle range here is only for illustration, and the angle range corresponding to the seats in the car may be different for different car models and different cabin layouts.
  • S550 Perform sound source localization according to the sound signal.
  • the sound source location of the sound signal may be performed by the MUSIC algorithm to obtain at least one sound source position.
  • the MUSIC algorithm may be performed by the MUSIC algorithm to obtain at least one sound source position.
  • the in-vehicle position of the first user is corrected according to the angle range determined in step S540 and the at least one sound source position determined in step S550.
  • the sound source position within the angle range determined in step S540 may be used as the corrected position in the vehicle of the first user.
  • step S350 please refer to step S350 in the aforementioned method 300.
  • S570 Execute a first voice command according to the position in the vehicle of the first user.
  • the position of the projection interface is controlled according to the position in the vehicle of the first user.
  • the projection interface can be used to display the projected image.
  • the position of the projection interface can be controlled by the rotation angle of the projection component.
  • the in-vehicle position information of the first user can be used to control the steering of the projection assembly.
  • the corrected in-vehicle position of the first user is returned to the steering motor 212 in the voice assistant component, and the projection frame is controlled to rotate.
  • step S550 and step S560 are optional steps. If the method 500 does not include step S550 and step S560, then in step S570, the in-vehicle position of the first user obtained in step S540 is returned to the projection component.
  • the first voice command is not necessarily a command for the position of the projection interface, that is, the first voice command can be any voice command.
  • the first voice command may be adjusting the temperature of the air conditioner, and executing the first voice command includes adjusting the temperature of the air conditioner and controlling the position of the projection interface.
  • the method for locating users in a vehicle can be applied to various on-vehicle interaction scenarios, such as music playback, air conditioning control, seat heating, seat adjustment, and other scenarios that require locating users in the vehicle.
  • various on-vehicle interaction scenarios such as music playback, air conditioning control, seat heating, seat adjustment, and other scenarios that require locating users in the vehicle.
  • different voice commands can be used to achieve interaction.
  • the following is an example of the in-vehicle interaction method 500.
  • executing the first voice command according to the in-vehicle position of the first user includes: controlling a speaker corresponding to the in-vehicle position of the first user.
  • Step S570 includes: controlling the speaker corresponding to the position in the vehicle of the first user, for example, turning on the speaker and playing corresponding music.
  • executing the first voice command according to the location in the vehicle of the first user includes: controlling the air conditioner corresponding to the location in the vehicle of the first user.
  • the in-vehicle interaction scene may be air conditioning control.
  • the first voice command is to turn on the air conditioner.
  • Step S570 includes: turning on the air conditioner corresponding to the position in the vehicle of the first user.
  • the user who needs to interact is determined based on the voice command, and then the position in the car of the user who needs to interact is determined according to the mapping relationship between the user in the car and the position in the car, even when multiple users are talking at the same time. Locate the location of the user who needs to interact.
  • the sound source localization algorithm can further improve the accuracy of localization and improve user experience.
  • the projection image is displayed through the projection component, for example, the visual image of the voice assistant is displayed, and the rotation angle of the projection component is adjusted according to the position in the car, so that the projection image can face the user who needs to interact, and provide the user with one-to-one interaction Experience.
  • FIG. 10 shows a schematic flowchart of a method 600 for updating a real-time information database.
  • the in-vehicle location may be represented by the in-vehicle area.
  • the in-vehicle area is divided according to the seats in the vehicle in the method 600.
  • the method of updating the real-time information database can also be understood as a method of updating the seat information in the vehicle.
  • the method 600 includes steps S610 to S670. Steps S610 to S670 will be described in detail below.
  • S610 Acquire a face image collected by a camera.
  • S620 Perform face recognition on the face image, and determine a user corresponding to the face image.
  • the user corresponding to the facial image can be determined according to the mapping relationship between the user and the facial features of the user.
  • the mapping relationship between the user and the facial features of the user may be stored in the database.
  • step S630 Determine whether the voiceprint feature of the user is included in the database. If the voiceprint information of the user is included in the database, step S640 is executed. If the user's voiceprint information is not included in the database, the process can be ended.
  • the method 600 further includes step S660.
  • step S660 is executed.
  • S640 Store the user's voiceprint information and the user's seat as a data pair in the real-time information database.
  • the user's seat can be determined by the camera. For example, Figure 6 shows 4 cameras, corresponding to 4 seats. Camera A corresponds to seat A, and the user corresponding to the face image taken by camera A is located on seat A.
  • the user's seat can be distinguished by the seat number, that is, step S640 can be to store the user's voiceprint information and the user's seat number seat data pair.
  • face recognition is performed again at regular intervals.
  • the mapping relationship between the user and the seat in the car can be re-established, or it can be understood as searching the database for the The user’s voiceprint features, and the mapping relationship between the user’s voiceprint features and the seats in the car is established.
  • step S610 to step S640 can be performed periodically. It should be understood that step S650 is an optional step.
  • S660 Supplementally record the voiceprint characteristics of the user, and perform step S640.
  • step S670 is an optional step.
  • the seats in the vehicle can correspond to different states, and the state of the seats in the vehicle will change accordingly during the execution of the above steps S610 to S670. It should be understood that the status of the seat in the embodiment of the present application refers to the status of the seat in the system.
  • FIG. 11 shows a schematic flow chart of a process of state transition of a seat in a vehicle.
  • Initial (Init) state This state is the initial state. In this state, initialize settings. For example, when the voice assistant is activated or the user on the seat changes, the seat enters the Init state. In this state, the user's face is recognized, the user's identity is confirmed, the voiceprint feature is matched, and the corresponding mapping relationship is stored in the real-time information database for the positioning of the user in the car. The specific initialization process can be seen in Figure 12 below. After completion, the seat enters the Ready state. If it is detected that there is no user in the seat, the seat enters the Idle state.
  • the seat is in an available state, that is, it can be used to implement the method in the embodiment of the present application. If all the seats in the car are in an available state, the system is in an available state. Further, it can be updated regularly. For example, at regular intervals, such as 10s, enter the Check state to perform user verification.
  • Check state In this state, check whether the user on the seat has changed. If there is no change, return to Ready state. If there is a change, it enters the Init state. Or, further, if there is no user on the seat, for example, the user gets off the bus or changes seats, the seat enters the Idle state.
  • the specific inspection process can be seen in Figure 13 below.
  • Idle state There is no user on the seat in this state. Further, it can be updated regularly in this state. For example, every certain period of time, such as 10s, re-determine whether there is a user in the seat, and when a passenger appears in the seat, enter the Init state and restart to update the user's seat information.
  • Stop state Turn off the voice assistant in this state. Further, the data in the real-time information database can be deleted. In any of the aforementioned states, power off or shut down the voice assistant will enter this state.
  • Fig. 12 shows a schematic flow chart of the initialization setting in the process of updating the seat information of the user.
  • the initial setting includes step S710 to step S7120.
  • S710 Acquire a face image collected by a camera.
  • S720 Perform face recognition on the face image.
  • step S730 Determine whether there is a user in the current seat. If there is a user in the current seat, step S740 is executed. If there is no user in the current seat, the seat enters the Idle state.
  • S740 Determine whether the user exists in the database.
  • step S750 is executed. If the user does not exist in the database, that is, the user cannot be identified according to the face recognition result, step S780 is executed. Step S780 is an optional step,
  • S750 Associate the user with the seat in the car. That is, the mapping relationship between users and seats in the car is established.
  • step S760 Determine whether there is a voiceprint feature of the user in the database. If the voiceprint feature of the user is in the database, step S770 is executed. If there is no voiceprint feature of the user in the database, step S7100 is executed. Step S7100 is an optional step.
  • S770 Associate the user's voiceprint feature with the seat in the car. That is, the mapping relationship between the user's voiceprint characteristics and the seats in the car is established. For example, if the user's voiceprint feature and seat number are saved as a data pair in the real-time information database, the seat enters the Ready state.
  • S780 Determine whether to enter user information.
  • step S790 is executed. If the user does not agree to enter the user information, step S7120 is executed.
  • S790 Collect user information, including facial features, and save it in a database. For example, register and log in in the car machine.
  • S7100 Determine whether to enter the voiceprint feature.
  • step S7110 the user may be asked whether to enter the voiceprint information of the user, and if the user agrees to enter the voiceprint information, step S7110 is executed. If the user does not agree to enter the voiceprint information, the seat enters the Ready state.
  • S7110 Associate the user with the voiceprint characteristics of the user. That is, the mapping relationship between the user and the voiceprint characteristics of the user is established. For example, save the mapping relationship as a data pair in the database, and execute S770.
  • S7120 Associate the user's facial features with the seat, and the seat enters the Ready state. That is, the mapping relationship between the user's facial features and the seats in the car is established. For example, the mapping relationship seat data pair is stored in a real-time information database. In this way, the facial features of the user can be saved, so as to determine whether the user on the seat has changed.
  • Fig. 13 shows a schematic flow chart of verification in the process of updating seat information.
  • the verification process includes step S810 to step S860.
  • S810 Acquire a face image collected by a camera.
  • S820 Perform face recognition on the image.
  • the result of face recognition can be compared with the facial features corresponding to the seat to determine whether the user has changed.
  • step S840 is executed. If there is no change, the seat enters the Ready state.
  • step S840 the data is acquired again n times at regular intervals, that is, steps S810 to S830 are executed again at intervals of a period of time to obtain n results.
  • step S850 if the number of changes exceeds the threshold, step S860 is executed, otherwise the seat enters the Ready state.
  • step S850 is only an example. Step S850 may also be that, if the result of n times meets the preset condition, step S860 is executed. For example, if the result of n times is that the user of the seat has changed, step S860 is executed. If there is an unchanged result among the results obtained in step S850, the seat enters the Ready state. In this way, it is possible to "vote" whether the user of the seat has changed, reducing the possibility of misjudgment.
  • step S840 and step S850 are optional steps.
  • Step S830 can also be that, if there is a change, step S860 can be executed.
  • step S860 Determine whether there is a user in the current seat. If there is a user in the seat, the seat enters the Init state, for example, the steps in FIG. 12 are executed. It should be noted that in this case, step S730 may not be executed. If there is no user in the seat, the seat enters the Idle state.
  • step S860 can also be executed before step S830. For example, if there is a user in the seat, step S830 is executed. In this case, in step S850, if the number of changes exceeds the threshold, the seat enters the Init state. If there is no user in the seat, the seat enters the Idle state.
  • FIG. 14 is a schematic block diagram of an in-vehicle system according to an embodiment of the present application.
  • the in-vehicle system 900 shown in FIG. 14 includes an in-vehicle machine 910, a voice assistant component 920, a gateway 930, a bus 940, an in-vehicle data center 950, a camera, and a controller area network (CAN) 970.
  • CAN controller area network
  • the vehicle machine 910 includes multiple computing modules, and communicates with other devices through the bus 940.
  • the vehicle machine 910 includes a voice recognition module 911 and a positioning module 912.
  • the vehicle machine further includes an in-vehicle location update module 913.
  • the vehicle machine further includes a voiceprint recognition module 914.
  • the voice recognition module 911 corresponds to the voice recognition module 221 in FIG. 4, and the positioning module 912 corresponds to the positioning module 222 in FIG. 4.
  • the voiceprint recognition module 914 corresponds to the voice recognition module 223 in FIG. 4.
  • the in-vehicle location update module 913 is used to update the mapping relationship between in-vehicle users and in-vehicle locations. Exemplarily, the in-vehicle position update module 913 may perform the steps in FIG. 10, FIG. 12, or FIG. 13.
  • the voice assistant component 920 includes a microphone 921.
  • the microphone 921 may be a microphone array.
  • the voice assistant component 920 further includes a steering motor 922 and a projection frame 923.
  • the voice assistant component can be connected to the cockpit through the gateway.
  • the voice assistant component 920 corresponds to the voice assistant component 210 in FIG. 4.
  • the vehicle-mounted data center 950 contains a database, for example, the database 230 in FIG. 4.
  • the vehicle-mounted data center 950 may include a user information database and a real-time information database.
  • the vehicle-mounted data center 950 can provide data for the computing modules in the vehicle through the bus.
  • the cameras may include four, which are distributed as the main driving camera 961, the co-driving camera 962, the rear camera 963, and the rear camera 964.
  • the camera can be connected to the gateway 930 via CAN970 like other sensors in the cockpit.
  • the gateway 930 is connected to the bus 940 to provide access to other components and hardware.
  • FIG. 15 is a schematic block diagram of an in-vehicle device according to an embodiment of the present application.
  • the vehicle-mounted device 1000 shown in FIG. 15 includes a storage unit 1010 and a processing unit 1020.
  • the storage unit 1010 and the processing unit 1020 may be used to execute the in-vehicle user positioning method or the in-vehicle interaction method in the embodiments of the present application. Specifically, the processing unit 1020 may execute the above-mentioned method 300 or method 500.
  • the storage unit 1010 is used to store the mapping relationship between the user in the car and the position in the car.
  • the processing unit 1020 is used to obtain the sound signal collected by the microphone in the car; when the first voice command is recognized from the sound signal, determine the first user who issued the first voice command; obtain the mapping relationship between the user in the car and the position in the car;
  • the in-vehicle position of the first user is determined according to the mapping relationship between the in-vehicle user and the in-vehicle position.
  • the processing unit 1020 is specifically configured to perform voiceprint feature matching on the first voice command, and identify the first user who issued the first voice command.
  • the mapping relationship between the user in the vehicle and the location in the vehicle is determined according to the face image collected by the vehicle camera.
  • the mapping relationship between the user in the vehicle and the location in the vehicle is determined according to the position of the camera and the facial image collected by the camera, including: obtaining user information; according to the mapping relationship between the user information and the facial image , Determine the user corresponding to the face image; determine the user's position in the car according to the position and/or angle of the camera.
  • the processing unit 1020 is further configured to: perform sound source localization according to the sound signal to obtain at least one sound source position; and correct the first user's position according to the first user's car position and the at least one sound source position. Location in the car.
  • the processing unit 1020 is specifically configured to: determine a sound source position closest to the in-vehicle position of the first user among the at least one sound source position as the corrected in-vehicle position of the first user .
  • the processing unit 1020 is further configured to execute the first voice command according to the position in the vehicle of the first user.
  • the processing unit 1020 is specifically configured to: control the position of the projection interface according to the position in the vehicle of the first user.
  • in-vehicle device 1000 is embodied in the form of a functional unit.
  • unit herein can be implemented in the form of software and/or hardware, which is not specifically limited.
  • a "unit” may be a software program, a hardware circuit, or a combination of the two that realizes the above-mentioned functions.
  • the hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, and a processor for executing one or more software or firmware programs (such as a shared processor, a dedicated processor, or a group processor). Etc.) and memory, merged logic circuits and/or other suitable components that support the described functions.
  • the units of the examples described in the embodiments of the present application can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
  • FIG. 16 is a schematic diagram of the hardware structure of a vehicle-mounted device provided by an embodiment of the present application.
  • the vehicle-mounted device 1100 shown in FIG. 16 includes a memory 1101, a processor 1102, a communication interface 1103, and a bus 1104.
  • the memory 1101, the processor 1102, and the communication interface 1103 implement communication connections between each other through the bus 1104.
  • the memory 1101 may be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
  • the memory 1101 may store a program.
  • the processor 1102 is used to execute each step of the in-vehicle user positioning method or the in-vehicle interaction method of the embodiment of the present application, for example, execute FIG. 5 or Figure 8 shows the various steps.
  • vehicle-mounted device shown in the embodiment of the present application may be a server, for example, it may be a server in the cloud, or may also be a chip configured in a server in the cloud.
  • the processor 1102 may adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more
  • the integrated circuit is used to execute related programs to implement the in-vehicle user positioning method or the in-vehicle interaction method in the method embodiment of the present application.
  • the processor 1102 may also be an integrated circuit chip with signal processing capabilities.
  • each step of the method of the present application can be completed by an integrated logic circuit of hardware in the processor 1102 or instructions in the form of software.
  • the above-mentioned processor 1102 may also be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, Discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and combines its hardware to complete the functions required by the units included in the vehicle-mounted device shown in FIG. 15 in the implementation of this application, or execute the method implementation of this application Example of the method shown in Figure 5 or Figure 8.
  • the communication interface 1103 uses a transceiving device such as but not limited to a transceiver to implement communication between the vehicle-mounted device 1100 and other devices or a communication network.
  • a transceiving device such as but not limited to a transceiver to implement communication between the vehicle-mounted device 1100 and other devices or a communication network.
  • the bus 1104 may include a path for transferring information between various components of the device 1100 (for example, the memory 1101, the processor 1102, and the communication interface 1103).
  • vehicle-mounted device 1100 only shows a memory, a processor, and a communication interface, in a specific implementation process, those skilled in the art should understand that the vehicle-mounted device 1100 may also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the above-mentioned vehicle-mounted device 1100 may further include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the above-mentioned in-vehicle device 1100 may also only include the components necessary to implement the embodiments of the present application, and not necessarily include all the components shown in FIG. 16.
  • the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.
  • a part of the processor may also include a non-volatile random access memory.
  • the processor may also store device type information.
  • the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not correspond to the embodiments of the present application.
  • the implementation process constitutes any limitation.
  • the disclosed system, device, and method can be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: Universal Serial Bus flash disk (USB flash disk, UFD), UFD can also be referred to as U disk or USB flash drive, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes.
  • USB flash disk Universal Serial Bus flash disk
  • UFD can also be referred to as U disk or USB flash drive
  • mobile hard disk read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes.
  • read-only memory read-only memory
  • RAM random access memory
  • magnetic disks or optical disks and other media that can store program codes.

Abstract

本申请提供了一种车内用户定位方法、车载交互方法、车载装置及车辆，可应用于自动驾驶领域。该车内用户定位方法包括：获取车内麦克风采集的声音信号；当从声音信号中识别到第一语音命令时，确定发出第一语音命令的第一用户；根据车内用户与车内位置的映射关系确定第一用户的车内位置。本申请的方案能够基于语音命令判断需要交互的用户，进而根据车内用户与车内位置的映射关系确定需要交互的用户的车内位置，即使在多个用户同时说话的情况下也能定位至需要交互的用户所在的位置。

Description

车内用户定位方法、车载交互方法、车载装置及车辆 技术领域
本申请实施例涉及人工智能领域，尤其涉及一种车内用户定位方法、车载交互方法、车载装置及车辆。
背景技术
人工智能(artificial intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个分支，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式作出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。人工智能领域的研究包括机器人，自然语言处理，计算机视觉，决策与推理，人机交互，推荐与搜索，AI基础理论等。
智能座舱是人工智能领域的一种主流应用，其中车机智能化是智能座舱研究的一个重要方面。车机在功能上能够实现人与车、车与外界的信息通讯，实现多种交互功能，例如，收发短信、拨打电话、导航、语音控制、播放音乐、在线影视等，大大增强了用户体验。当车内存在多个用户时，为了实现车内不同用户的个性化需求，通常需要定位正在进行交互的用户。例如，车内配置多个显示屏幕(如座椅背面)，各个位置的用户均可以欣赏视频。使用机械按钮或者触摸屏的方式进行交互的方式已经难以满足用户需求。语音交互的方式能够减少用户操作，用户发出语音命令，语音助手定位声源，仅打开声源位置的用户对应的显示屏幕，能够避免干扰他人，同时保护用户的隐私。然而，车内空间较小，用户声音嘈杂，对定位精度的要求较高，若车内同一时刻多个用户在对话时，现有方法无法准确定位发出语音命令的用户，进而无法执行相应的操作，影响交互体验。
因此，如何在车内存在多个声源的情况下实现用户的定位成为一个亟待解决的问题。
发明内容
本申请提供一种车内用户定位方法、车载交互方法、车载装置及车辆，能够在车内存在多个声源的情况下实现用户的定位。
第一方面，提供了一种车内用户定位方法，包括：获取车内麦克风采集的声音信号；当从声音信号中识别到第一语音命令时，确定发出第一语音命令的第一用户；获取车内用户与车内位置的映射关系；根据车内用户与车内位置的映射关系确定第一用户的车内位置。
其中，车内麦克风可以为一个麦克风，也可以为多个麦克风。例如，多个麦克风可以为麦克风阵列。麦克风阵列可以根据需要设置，例如，该麦克风阵列可以为均匀圆阵或均匀线阵。
例如，该第一语音命令可以为唤醒词，该唤醒词可以用于唤醒语音助手，激活后续处理流程。
确定发出第一语音命令的第一用户的方式可以包括多种方式。本申请实施例对确定第一用户的方式不做限定。
例如，可以根据用户和语音命令的映射关系，确定第一语音命令对应的第一用户。
具体地，预先为不同的用户设置不同的语音命令，并将该语音命令与用户绑定，即建立映射关系。例如，用户A与语音命令“打开A的扬声器”建立映射关系，用户B与语音命令“打开B的扬声器”建立映射关系。若第一语音命令为“打开A的扬声器”，则确定发出第一语音命令的第一用户为用户A。
车内用户与车内位置的映射关系可以是车内全部用户在车内的位置映射关系，也可以是车内的一个用户，或者一部分用户在车内的位置映射关系。
车内用户与车内位置的映射关系的获取方法可以包括多种方式。例如，车内用户与车内位置的映射关系可以是预存的，也可以是从其他设备，例如云服务器获取的，还可以是实时更新的。本申请实施例对获取车内用户与车内位置的映射关系的方法不做限定。
车内用户与车内位置的映射关系的确定方式可以包括多种方式。
例如，车内不同的位置处可以设置指纹采集器。通过指纹采集器采集指纹，根据用户与用户的指纹特征的映射关系，确定指纹采集器采集的指纹对应的用户。由于不同指纹采集器即对应不同的车内位置，可以将用户与车内位置建立映射关系。例如，指纹采集器A对应车内位置A，由该指纹采集器A采集到指纹A，通过指纹识别确定指纹A对应用户A，进而将该用户A与车内位置A建立映射关系。本申请实施例对车内用户与车内位置的映射关系的确定方法不做限定。
根据本申请实施例的方案，基于语音命令判断需要交互的用户，进而根据车内用户与车内位置的映射关系确定需要交互的用户的车内位置，即使在多个用户同时说话的情况下也能定位至需要交互的用户所在的位置，有利于实现一对一交互，提高用户体验。
结合第一方面，在第一方面的某些实现方式中，确定发出第一语音命令的第一用户包括：对第一语音命令进行声纹特征匹配，识别发出第一语音命令的第一用户。
根据本申请实施例的方案，基于声纹特征识别需要交互的用户，提高了定位准确性。
结合第一方面，在第一方面的某些实现方式中，车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的。
根据本申请实施例的方案，车载摄像头采集的人脸图像确定用户与车内的映射关系，无需用户配合，能够避免打扰用户，提高用户体验。
结合第一方面，在第一方面的某些实现方式中，车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的，包括：获取用户信息；根据用户信息和人脸图像的映射关系，确定人脸图像对应的用户；根据车载摄像头的位置和/或角度确定用户的车内位置。
该用户信息的获取方法可以包括多种方式。例如，用户信息可以是预存的，也可以是从其他设备，例如云服务器获取的，还可以是实时更新的。本申请实施例对获取用户信息的方法不做限定。根据用户信息和人脸图像的映射关系，确定人脸图像对应的用户，也就是对人脸图像进行人脸识别，确定人脸图像对应的用户。
车载摄像头可以为车内摄像头，也可以为车外摄像头。
可选地，通过车内摄像头的位置与车内位置之间的映射关系确定用户的车内位置。
例如，在车内不同的位置设置多个摄像头，不同位置的摄像头与车内位置一一对应。在该情况下，摄像头的位置与车内位置之间的映射关系也可以理解为不同摄像头与车内位置的映射关系。不同摄像头拍摄的用户所在的位置即为该摄像头对应的车内位置。
可选地，通过车内摄像头的旋转角度与车内位置之间的映射关系确定用户的车内位置。
例如，摄像头位于车内后视镜上，摄像头的旋转角度与车内位置对应。以车身的中线方向为0°，摄像头的旋转角度为0°时，对应车内的位置为车的中部，在该情况下，摄像头拍摄的用户所在的位置即为车的中部；摄像头的旋转角度为角度A时，对应车内的位置为车内位置A，在该情况下，摄像头拍摄的用户所在的位置即为车内位置A。
可选地，可以通过车内座位指示车内位置，这样，可以根据座位设置摄像头的位置/或角度，在定位过程中能够减少计算量，提高处理速度。
结合第一方面，在第一方面的某些实现方式中，车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的，包括：根据用户信息和人脸图像的映射关系，确定人脸图像对应的用户，根据用户在车载摄像头的镜头中的相对位置确定用户的车内位置。
根据用户在车载摄像头的镜头中的相对位置确定用户的车内位置，也就是根据车载摄像头采集的图像信息确定用户的车内位置。
例如，车载摄像头位于车内后视镜上，在一次拍摄中可以将车内所有用户放入取景框中。通过人脸识别确定人脸图像对应的用户及其在镜头中的相对位置，根据该相对位置确定用户对应的车内位置。再如，车载摄像头位于车外的B柱上，两侧B柱可以分别设置车载摄像头。对于左侧B柱上的摄像头采集的人脸图像，通过人脸识别确定人脸图像对应的用户及其在镜头中的相对位置。例如，该用户在镜头中的相对位置靠右侧，则该用户从车辆左侧的前侧车门上车，该用户对应的车内位置可能位于驾驶座上。
可选地，可以通过车内座位指示车内位置，这样，可以将镜头中的相对位置映射到车内座位上，在定位过程中能够减少计算量，提高处理速度。
结合第一方面，在第一方面的某些实现方式中，车内用户与车内位置的映射关系是定时更新的。
由于座舱内的用户可能更换座位或者上/下车，定时更新车内用户与车内位置的映射关系，能够提高用户定位的准确性。
结合第一方面，在第一方面的某些实现方式中，车内用户与车内位置的映射关系是定时更新的，包括：根据车内用户与车内位置的映射关系确定当前车内位置对应的用户，获取车载摄像头采集的当前人脸图像，根据用户信息和人脸图像的映射关系，确定当前人脸图像对应的当前用户；在当前车内位置对应的用户与当前用户不同时，更新车内用户与车内位置的映射关系。
例如，根据车内用户与车内位置的映射关系确定当前车内位置没有用户，获取车载摄像头采集的当前人脸图像，根据用户信息和人脸图像的映射关系，确定当前人脸图像对应的当前用户；将当前用户与当前车内位置建立映射关系，更新车内用户与车内位置的映射关系。
再如，根据车内用户与车内位置的映射关系确定当前车内位置对应的用户为用户A，获取车载摄像头采集的当前人脸图像，根据用户信息和人脸图像的映射关系，确定当前人脸图像对应的当前用户为用户B；当用户A与用户B不同时，将用户B与当前车内位置建立映射关系，更新车内用户与车内位置的映射关系。
结合第一方面，在第一方面的某些实现方式中，该方法还包括：根据声音信号进行声源定位，获得至少一个声源位置；根据第一用户的车内位置和至少一个声源位置修正第一用户的车内位置。
具体地，可以通过基于麦克风阵列的声源定位方法对声音信号进行声源定位。例如，该声源定位方法可以为MUSIC算法。该麦克风阵列可以根据需要设置，例如，该麦克风阵列可以为均匀圆阵或均匀线阵。
根据本申请实施例的方案，通过声源定位能够进一步提高第一用户的车内位置的准确性，在车内嘈杂的环境中也能准确定位需要交互的用户。
结合第一方面，在第一方面的某些实现方式中，根据第一用户的车内位置和至少一个声源位置修正第一用户的车内位置，包括：将至少一个声源位置中与第一用户的车内位置最接近的一个声源位置确定为修正后的第一用户的车内位置。
例如，若修正前的第一用户的车内位置为一个角度范围，则将至少一个声源位置中的落于该角度范围内的声源位置作为修正后的第一用户的车内位置。
再如，若修正前的第一用户的车内位置为一个角度值，则将至少一个声源位置中与该值最接近的声源位置作为修正后的第一用户的车内位置。
第二方面，提供了一种车载交互方法，包括：获取车内麦克风采集的声音信号；当从声音信号中识别到第一语音命令时，确定发出第一语音命令的第一用户；获取车内用户与车内位置的映射关系，根据车内用户与车内位置的映射关系确定第一用户的车内位置；根据第一用户的车内位置执行第一语音命令。
其中，车内麦克风可以为一个麦克风，也可以为多个麦克风。例如，多个麦克风可以为麦克风阵列。麦克风阵列可以根据需要设置，例如，该麦克风阵列可以为均匀圆阵或均匀线阵。
例如，该第一语音命令可以为唤醒词，该唤醒词可以用于唤醒语音助手，激活后续处理流程。
可选地，可以根据用户和语音命令的映射关系，确定第一语音命令对应的第一用户。
具体地，预先为不同的用户设置不同的语音命令，并将该语音命令与用户绑定，即建立映射关系。例如，用户A与语音命令“打开A的扬声器”建立映射关系，用户B与语音命令“打开B的扬声器”建立映射关系。若第一语音命令为“打开A的扬声器”，则确定发出第一语音命令的第一用户为用户A。
根据本申请实施例的方案，基于语音命令判断需要交互的用户，进而根据车内用户与车内位置的映射关系确定需要交互的用户的车内位置，即使在多个用户同时说话的情况下也能定位至需要交互的用户所在的位置，能够根据用户位置执行语音命令，实现一对一交互，提高用户体验。
结合第二方面，在第二方面的某些实现方式中，根据第一用户的车内位置执行第一语音命令，包括：根据第一用户的车内位置控制投影界面的位置。
该投影界面可以用于显示投影形象。
示例性地，该投影界面的位置可以由投影组件的旋转角度控制。投影组件可以包括投影框架和转向电机。该投影框架可以用于投影，例如，用于展示语音助手的三维形象。该三维形象即显示在投影界面上，也就是说可以根据第一用户的车内位置控制该三维形象的位置。
第一语音命令不一定是控制投影界面位置的命令，也就是说第一语音命令可以为任意语音命令。例如，该第一语音命令可以为调节空调温度，那么执行第一语音命令包括，调节空调温度，以及控制投影界面的位置。
根据本申请实施例的方案，能够通过投影界面展示投影形象，并根据车内位置调整投影界面的位置，以使投影形象能够面对需要交互的用户，为用户提供一对一的交互体验。
结合第二方面，在第二方面的某些实现方式中，根据第一用户的车内位置执行第一语音命令，包括：控制第一用户的车内位置对应的扬声器。
例如，第一语音命令可以为播放音乐。控制第一用户的车内位置对应的扬声器，可以包括，打开扬声器，并播放相应音乐等。
结合第二方面，在第二方面的某些实现方式中，根据第一用户的车内位置执行第一语音命令，包括：控制第一用户的车内位置对应的空调。
例如，第一语音命令为打开空调。控制第一用户的车内位置对应的空调，可以包括，打开第一用户的车内位置对应的空调。
以上仅为车载交互方法的应用示例，本申请实施例中的车载交互方法还可以应用于其他需要定位用户的场景。
第三方面，提供了一种车载装置，包括：存储单元，用于存储车内用户与车内位置的映射关系；处理单元，用于：获取车内麦克风采集的声音信号；当从声音信号中识别到第一语音命令时，确定发出第一语音命令的第一用户；获取车内用户与车内位置的映射关系；根据车内用户与车内位置的映射关系确定第一用户的车内位置。
结合第三方面，在第三方面的某些实现方式中，处理单元具体用于：对第一语音命令进行声纹特征匹配，识别发出第一语音命令的第一用户。
结合第三方面，在第三方面的某些实现方式中，车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的。
结合第三方面，在第三方面的某些实现方式中，车内用户与车内位置的映射关系是根据摄像头的位置与摄像头采集的人脸图像确定的，包括：获取用户信息；根据用户信息和人脸图像的映射关系，确定人脸图像对应的用户；根据摄像头的位置和/或角度确定用户的车内位置。
结合第三方面，在第三方面的某些实现方式中，处理单元还用于：根据声音信号进行声源定位，获得至少一个声源位置；根据第一用户的车内位置和至少一个声源位置修正第一用户的车内位置。
结合第三方面，在第三方面的某些实现方式中，处理单元具体用于：将至少一个声源位置中与第一用户的车内位置最接近的一个声源位置确定为修正后的第一用户的车内位置。
结合第三方面，在第三方面的某些实现方式中，处理单元还用于：根据第一用户的车 内位置执行第一语音命令。
结合第三方面，在第三方面的某些实现方式中，处理单元具体用于：根据第一用户的车内位置控制投影界面的位置。
第四方面，提供了一种车辆，包括上述第三方面中的装置。
第五方面，提供一种车载装置，包括输入输出接口、处理器和存储器。该处理器用于控制输入输出接口收发信息，该存储器用于存储计算机程序，该处理器用于从存储器中调用并运行该计算机程序，使得该车载装置执行上述第一方面或第二方面中的方法。
可选地，上述装置可以是终端设备/服务器，也可以是终端设备/服务器内的芯片。
可选地，上述存储器可以位于处理器内部，例如，可以是处理器中的高速缓冲存储器(cache)。上述存储器还可以位于处理器外部，从而独立于处理器，例如，装置的内部存储器(memory)。
应理解，上述第一方面包括第一方面中的任意一种实现方式，第二方面包括第二方面中的任意一种实现方式，第三方面包括第三方面中的任意一种实现方式。
第六方面，提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述各方面中的方法。
需要说明的是，上述计算机程序代码可以全部或者部分存储在第一存储介质上，其中，第一存储介质可以与处理器封装在一起的，也可以与处理器单独封装，本申请实施例对此不作具体限定。
第七方面，提供了一种计算机可读介质，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述各方面中的方法。
第八方面，提供了一种芯片，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，执行上述各方面中的方法。
可选地，作为一种实现方式，所述芯片还可以包括存储器，所述存储器中存储有指令，所述处理器用于执行所述存储器上存储的指令，当所述指令被执行时，所述处理器用于执行上述各方面中的方法。
附图说明
图1是本申请实施例提供的车辆的示意图；
图2是本申请实施例提供的麦克风阵列的结构示意图；
图3是本申请实施例提供的能量谱的示意图；
图4是本申请实施例提供的车载系统的结构示意图；
图5是本申请实施例提供的车内用户定位方法的流程示意图；
图6是本申请实施例提供的一种车内硬件配置的结构示意图；
图7是本申请实施例提供的修正第一用户的车内位置的示意图；
图8是本申请实施例提供的一种车载交互方法的示意性流程图；
图9是本申请实施例提供的投影框架的结构示意图；
图10是本申请实施例提供的一种用户的车内位置更新方法的示意性流程图；
图11是本申请实施例提供的车内座位状态转换的示意性流程图；
图12是本申请实施例提供的车内座位更新的初始化过程的示意性流程图；
图13是本申请实施例提供的车内座位更新的校验过程的示意性流程图；
图14是本申请实施例提供的车载系统的结构示意图；
图15是本申请实施例提供的车载装置的示意性框图；
图16是本申请实施例提供的车载装置的示意性框图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
图1是本发明实施例提供的车辆100的功能框图。
车辆100可包括各种子系统，例如行进系统102、传感器系统104、控制系统106、一个或多个外围设备108以及电源110、计算机系统112和用户接口116。可选地，车辆100可包括更多或更少的子系统，并且每个子系统可包括多个元件。另外，车辆100的每个子系统和元件可以通过有线或者无线互连。
行进系统102可包括为车辆100提供动力运动的组件。在一个实施例中，推进系统102可包括引擎118、能量源119、传动装置120和车轮/轮胎121。
传感器系统104可包括感测关于车辆100周边的环境的信息的若干个传感器。例如，传感器系统104可包括定位系统122(定位系统可以是GPS系统，也可以是北斗系统或者其他定位系统)、惯性测量单元(inertial measurement unit，IMU)124、雷达126、激光测距仪128以及相机130。传感器系统104还可包括被监视车辆100的内部系统的传感器(例如，车内空气质量监测器、燃油量表、机油温度表等)。来自这些传感器中的一个或多个的传感器数据可用于检测对象及其相应特性(位置、形状、方向、速度等)。这种检测和识别是自主车辆100的安全操作的关键功能。
控制系统106为控制车辆100及其组件的操作。控制系统106可包括各种元件，其中包括转向系统132、油门134、制动单元136、传感器融合算法138、计算机视觉系统140、路线控制系统142以及障碍物避免系统144。
计算机视觉系统140可以操作来处理和分析由相机130捕捉的图像以便识别车辆100周边环境中的物体和/或特征。所述物体和/或特征可包括交通信号、道路边界和障碍物。在本申请实施例中，计算机视觉系统140还可以操作来处理和分析由相机130捕捉的图像以便识别车辆100内部环境中的用户和/或特征。计算机视觉系统140可使用物体识别算法、人脸识别算法、运动中恢复结构(structure from motion，SFM)算法、视频跟踪和其他计算机视觉技术。
当然，在一个实例中，控制系统106可以增加或替换地包括除了所示出和描述的那些以外的组件。或者也可以减少一部分上述示出的组件。
车辆100通过外围设备108与外部传感器、其他车辆、其他计算机系统或用户之间进行交互。外围设备108可包括无线通信系统146、车载电脑148、麦克风150和/或扬声器152。
在一些实施例中，外围设备108提供车辆100的用户与用户接口116交互的手段。例如，车载电脑148可向车辆100的用户提供信息。用户接口116还可操作车载电脑148来接收用户的输入。车载电脑148可以通过触摸屏进行操作。在其他情况中，外围设备108可提供用于车辆100与位于车内的其它设备通信的手段。例如，麦克风150可从车 辆100的用户接收音频(例如，语音命令或其他音频输入)。在本申请实施例中，车载电脑148可以通过接收用户的语音命令或其他音频输入进行操作。类似地，扬声器152可向车辆100的用户输出音频。
电源110可向车辆100的各种组件提供电力。车辆100的部分或所有功能受计算机系统112控制。计算机系统112可包括至少一个处理器113，处理器113执行存储在例如数据存储装置114这样的非暂态计算机可读介质中的指令115。计算机系统112还可以是采用分布式方式控制车辆100的个体组件或子系统的多个计算设备。
处理器113可以是任何常规的处理器，诸如商业可获得的CPU。替选地，该处理器可以是诸如ASIC或其它基于硬件的处理器的专用设备。尽管图1功能性地图示了处理器、存储器、和在相同块中的计算机110的其它元件，但是本领域的普通技术人员应该理解该处理器、计算机、或存储器实际上可以包括可以或者可以不存储在相同的物理外壳内的多个处理器、计算机、或存储器。例如，存储器可以是硬盘驱动器或位于不同于计算机110的外壳内的其它存储介质。因此，对处理器或计算机的引用将被理解为包括对可以或者可以不并行操作的处理器或计算机或存储器的集合的引用。不同于使用单一的处理器来执行此处所描述的步骤，诸如转向组件和减速组件的一些组件每个都可以具有其自己的处理器，所述处理器只执行与特定于组件的功能相关的计算。
在此处所描述的各个方面中，处理器可以位于远离该车辆并且与该车辆进行无线通信。在其它方面中，此处所描述的过程中的一些在布置于车辆内的处理器上执行而其它则由远程处理器执行，包括采取执行单一操纵的必要步骤。
在一些实施例中，数据存储装置114可包含指令115(例如，程序逻辑)，指令115可被处理器113执行来执行车辆100的各种功能，包括以上描述的那些功能。数据存储装置114也可包含额外的指令，包括向推进系统102、传感器系统104、控制系统106和外围设备108中的一个或多个发送数据、从其接收数据、与其交互和/或对其进行控制的指令。
除了指令115以外，数据存储装置114还可存储数据，例如道路地图、路线信息，车辆的位置、方向、速度以及其它这样的车辆数据，以及其他信息。这种信息可在车辆100在自主、半自主和/或手动模式中操作期间被车辆100和计算机系统112使用。在本申请实施例中，数据存储装置114还可存储用户的相关信息。这种信息可在人机交互期间被车辆100和计算机系统112使用。
用户接口116，用于向车辆100的用户提供信息或从其接收信息。可选地，用户接口116可包括在外围设备108的集合内的一个或多个输入/输出设备，例如无线通信系统146、车车在电脑148、麦克风150和扬声器152。
计算机系统112可基于从各种子系统(例如，行进系统102、传感器系统104和控制系统106)以及从用户接口116接收的输入来控制车辆100的功能。例如，计算机系统112可利用来自用户接口116的输入以便控制外围设备108实现人机交互。在一些实施例中，计算机系统112可操作来对车辆100及其子系统的许多方面提供控制。
可选地，上述这些组件中的一个或多个可与车辆100分开安装或关联。例如，数据存储装置114可以部分或完全地与车辆100分开存在。上述组件可以按有线和/或无线方式来通信地耦合在一起。
可选地，上述组件只是一个示例，实际应用中，上述各个模块中的组件有可能根据实际需要增添或者删除，图1不应理解为对本发明实施例的限制。
上述车辆100可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车、游乐场车辆、施工设备、电车、高尔夫球车、火车、和手推车等，本发明实施例不做特别的限定。
本申请实施例提供的车内用户定位方法能够应用在所有需要对车内用户进行定位的场景中。具体地，本申请实施例提供的车内用户定位方法能够应用在音乐播放、空调控制、座椅加热、座椅调节等需要对车内用户进行定位的场景。下面分别对两种常用的应用场景进行简单的介绍。
应用场景一：音乐播放
图1中的扬声器152可以分布于车内多个位置。具体地，可以分别控制不同位置的扬声器播放不同的音乐，实现分区音乐播放的功能，以使不同位置的用户能够根据需要打开/关闭扬声器或播放不同的音乐等。例如，当用户需要播放音乐时，可以发出语音命令，语音助手等可以定位发出语音命令的用户，并根据该语音指令对该用户附近的扬声器进行控制，例如，打开扬声器，并播放相应音乐等。
其中，定位发出语音命令的用户可以由本申请实施例中的车内用户定位方法来执行。即使在车内存在多个用户同时说话的情况下，本申请实施例中的车内用户定位方法也能够准确定位发出语音命令的用户，进而根据该语音指令对该用户附近的扬声器进行控制。这样不影响其他用户的正常交流，同时提高车内用户定位的准确性，避免干扰他人，满足不同用户的需求。
应用场景二：空调控制
图1的车辆100中还可以包括空调，空调可以分布于车内多个位置。具体地，可以分别控制不同位置的空调调节不同位置的温度，实现分区空调控制的功能，以使不同位置的用户能够根据需要调节附近的温度。例如，当用户需要调节空调温度时，可以发出语音命令，语音助手等可以定位发出语音命令的用户，并根据该语音指令对控制该用户附近的空调，根据用户的需要调节该空调的温度。
其中，定位发出语音命令的用户可以由本申请实施例中的车内用户定位方法来执行。即使在车内存在多个用户同时说话的情况下，本申请实施例中的车内用户定位方法也能够准确定位发出语音命令的用户，并根据该语音指令对控制该用户附近的空调。这样不影响其他用户的正常交流，同时提高车内用户定位的准确性，避免干扰他人，满足不同用户的需求。
应理解，以上应用场景仅为示意，对于车内其他能够分区实现的功能，均可以采用本申请实施例提供的车内用户定位方法定位至需要使用相应功能的用户，根据需要开启相应功能，提高用户的交互体验。
为了便于理解本申请实施例，下面先对本申请实施例涉及的相关术语的相关概念进行介绍。
(1)多重信号分类(multiple signal classification，MUSIC)算法
MUSIC算法是借助信号子空间和噪音子空间的正交性计算信号在不同角度的能量分布，从而以信号峰值出现的位置来进行定位。多个人同时在不同角度说话时，该算法会产 生多个峰值。
假设有D个声源，即同时有D个人在说话，M个麦克风组成麦克风阵列。图2示出了一种麦克风阵列的结构示意图。在图2中，M为4，即由4个间距为d的麦克风组成麦克风阵列。
麦克风阵列接收的信号可以如公式(1)所示：
X(t)＝AS(t)+N(t)  (1)
其中，X(t)为接收的信号，t为时间，S(t)为信号源在阵元上的信号强度，A为不同麦克风对接收信号的影响，是角度θ的函数，θ为接收的信号与麦克风阵列法线的夹角，即信号的方位角，N(t)为噪声。协方差矩阵由信号子空间和噪声子空间构成，具体可以如公式(2)所示：
R(t)＝AR SA H+σ 2I  (2)
其中，R(t)为协方差矩阵，H表示矩阵共轭转置，σ 2为噪声功率，I为M*M阶的单位矩阵，R S为信号的相关矩阵。
根据样本，可以得到最大似然的估计值，如公式(3)所示：
Figure PCTCN2020088227-appb-000001
其中，R x为最大似然的估计值，N为采样数。
对协方差矩阵进行特征分解，分解后的特征值后按大小排序，前D个为与信号相关的特征值，后M-D个特征值为噪声空间，其特征向量构成了噪声矩阵E n＝[v D+1,v D+2,...v M]，其中，v D+1表示第D+1个特征值，v D+2表示第D+2个特征值，v M表示第M个特征值。能量谱关于角度θ的分布，具体如公式(4)所示：
Figure PCTCN2020088227-appb-000002
其中，a(θ)为麦克风对方位角为θ的信号源的响应，P music(θ)为能量谱，该能量谱中包括一个或多个峰值，通过寻找波峰得到声源的位置。也就是说同一时刻，多个声源同时说话可以显示在能量谱中。图3示出了多人同时说话时的能量谱的示意图。在图3中，两个人同时在相对于麦克风阵列0°和60°的位置说话。
(2)神经网络
神经网络可以是由神经单元组成的，神经单元可以是指以x s和截距1为输入的运算单元，该运算单元的输出可以如公式(5)所示：
Figure PCTCN2020088227-appb-000003
其中，s＝1、2、……n，n为大于1的自然数，W s为x s的权重，b为神经单元的偏置。f为神经单元的激活函数(activation functions)，该激活函数用于对神经网络中的特征进行非线性变换，将神经单元中的输入信号转换为输出信号。该激活函数的输出信号可以作为下一层卷积层的输入。激活函数可以是sigmoid函数。神经网络是将许多个上述单一的神经单元联结在一起形成的网络，即一个神经单元的输出可以是另一个神经单元的输入。每个神经单元的输入可以与前一层的局部接受域相连，来提取局部接受域的特征，局部接受域可以是由若干个神经单元组成的区域。
(3)深度神经网络
深度神经网络(deep neural network，DNN)，也称多层神经网络，可以理解为具有多层隐含层的神经网络。按照不同层的位置对DNN进行划分，DNN内部的神经网络可以分为三类：输入层，隐含层，输出层。一般来说第一层是输入层，最后一层是输出层，中间的层数都是隐含层。层与层之间是全连接的，也就是说，第i层的任意一个神经元一定与第i+1层的任意一个神经元相连。
虽然DNN看起来很复杂，但是就每一层的工作来说，其实并不复杂，简单来说就是如下线性关系表达式：
Figure PCTCN2020088227-appb-000004
其中，
Figure PCTCN2020088227-appb-000005
是输入向量，
Figure PCTCN2020088227-appb-000006
是输出向量，
Figure PCTCN2020088227-appb-000007
是偏移向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量
Figure PCTCN2020088227-appb-000008
经过如此简单的操作得到输出向量
Figure PCTCN2020088227-appb-000009
由于DNN层数多，系数W和偏移向量
Figure PCTCN2020088227-appb-000010
的数量也比较多。这些参数在DNN中的定义如下所述：以系数W为例，假设在一个三层的DNN中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为
Figure PCTCN2020088227-appb-000011
上标3代表系数W所在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。
综上，第L-1层的第k个神经元到第L层的第j个神经元的系数定义为
Figure PCTCN2020088227-appb-000012
需要注意的是，输入层是没有W参数的。在深度神经网络中，更多的隐含层让网络更能够刻画现实世界中的复杂情形。理论上而言，参数越多的模型复杂度越高，“容量”也就越大，也就意味着它能完成更复杂的学习任务。训练深度神经网络的也就是学习权重矩阵的过程，其最终目的是得到训练好的深度神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。
(4)损失函数
在训练深度神经网络的过程中，因为希望深度神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为深度神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断的调整，直到深度神经网络能够预测出真正想要的目标值或与真正想要的目标值非常接近的值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么深度神经网络的训练就变成了尽可能缩小这个loss的过程。
(5)反向传播算法
神经网络可以采用误差反向传播(back propagation，BP)算法在训练过程中修正初始的神经网络模型中参数的数值，使得神经网络模型的重建误差损失越来越小。具体地，前向传递输入信号直至输出会产生误差损失，通过反向传播误差损失信息来更新初始的神经网络模型中参数，从而使误差损失收敛。反向传播算法是以误差损失为主导的反向传播运动，旨在得到最优的神经网络模型的参数，例如权重矩阵。
目前，车内用户的定位通常采用声源定位系统。声源定位系统主要包括基于麦克风阵列的声源定位系统和基于人耳听觉机理的声源定位系统。基于麦克风阵列的声源定位指的是用麦克风拾取声音信号，通过对麦克风阵列的各路输出信号进行分析和处理，得到一个或多个声源的位置信息。然而，当多个用户在车内说话时，上述方案无法分辨是哪个或哪 几个用户发出的声音。当上述方案应用于人机交互时，系统无法判断需要进行交互的方向，无法提供一对一的交互体验。
图4示出了本申请实施例中的一种车载系统的结构示意图。图4中的车载系统200包括语音助手组件210、语音处理单元220和数据库230。
其中，语音助手组件210包括麦克风211。麦克风211可以是指将声音信号转换为电信号的能量转换器件。示例性地，麦克风211可以为麦克风阵列。麦克风阵列指的是由一定距离的多个麦克风组成的阵列，例如，线性阵列或圆形阵列等。麦克风211采集的声音信号可以传输至语音处理单元220进行处理。
在一些实施例中，语音助手组件210还可以包括转向电机212。示例性地，该转向电机212可以为控制旋转的步进电机。例如，该转向电机212可以为用于控制3D投影组件旋转的步进电机。该转向电机212可以接收定位模块222发送的位置信息，并进行相应地转动。
语音处理单元220包括语音识别模块221和定位模块222。
语音识别模块221用于在麦克风采集的声音信号中识别语音命令。
例如，该语音命令可以为唤醒词。唤醒词可以用于激活系统200的处理流程。
示例性地，识别到语音命令后，根据语音命令与用户的映射关系，确定发出该语音命令的用户。也就是说不同的用户可以设置不同的语音命令。
可选地，语音处理单元220还可以包括声纹识别模块223。声纹识别模块223用于根据说话用户的声波特性识别用户。在该情况下，语音识别模块221识别到语音命令后，可以将声音信号发送至声纹识别模块223。声纹识别模块223可以用于根据用户和用户的声纹特征的映射关系对语音命令进行声纹识别，确定发出该语音命令的用户。
示例性地，声纹识别包括：声音信号处理、声纹特征提取、声纹比对以及判别决策等。
例如，声纹特征提取是指提取并选择对说话人的声纹具有可分性强、稳定性高等特性的声学或语言特征。
例如，声纹对比以及判别决策是指声纹模式匹配(声纹模式识别)。具体地，提取的声纹的特征数据与数据库230中存储的特征模板(声纹特征)进行搜索匹配的过程。例如，可以采用神经网络方法、最近邻方法等。
应理解，上述声纹识别模块223可以部署于车辆中，或者，上述声纹识别模块223也可以部署于云端。
还应理解，上述声纹识别模块223仅为举例说明，声纹识别模块223用于通过采集说话用户的声音从而对用户进行识别，声纹识别模块223可以是任何形式，本申请实施例对此不作任何限定。
定位模块222可以根据车内用户与车内位置的映射关系确定发出语音命令的用户的车内位置，即需要交互的用户的位置。其中，车内用户与车内位置的映射关系可以存储于数据库230中。
进一步地，定位模块222可以将该位置信息反馈至语音助手组件210，以使语音助手组件210执行后续操作。例如，控制转向电机212转动。
进一步地，定位模块222还可以用于实现声源定位，获得至少一个声源位置。示例性地，语音识别模块可以将麦克风采集的声音信号传输至定位模块222，以用于实现声源定 位。
声源定位指的是根据声音信号定位一个或多个声源的方向和距离。例如，声源定位可以采用MUSIC算法、到达时间差(time difference of arrival，TDOA)法等方法实现。
示例性地，定位模块222可以根据声纹识别模块223得到的车内位置和声源定位得到的至少一个声源位置修正发出语音命令的用户的位置，得到更准确的需要交互的车内位置θ。
应理解，上述定位模块222可以部署于车辆中，或者，定位模块222也可以部署于云端。
还应理解，上述定位模块222仅为举例说明，定位模块222可以用于通过采集说话用户的声音信号从而对用户的位置进行定位，声源定位方式可以是任何形式，本申请实施例对此不作任何限定。
可选地，系统200还可以包括图像采集单元250和图像处理单元240。
图像采集单元250包括摄像头251。摄像头251可以具有获取视频摄像/传播或者静态图像捕捉等基本功能，它是借由镜头采集图像后，由摄像头内的感光组件电路及控制组件对图像进行处理并转换成数字信号以便进行后续的信号处理。
具体地，图像采集单元250可以将采集的人脸图像发送至图像处理单元240。
图像处理单元240包括人脸识别模块241。
人脸识别模块241用于根据用户的人脸图像识别用户。具体地，人脸识别模块241可以用于根据用户和用户的人脸特征的映射关系确定人脸图像对应的用户。其中，该用户和用户的人脸特征的映射关系可以存储于数据库230中。
示例性地，人脸识别包括：人脸图像预处理、人脸图像特征提取以及匹配与识别。
例如，人脸图像预处理是指基于人脸检测结果，对图像进行处理并最终服务于特征提取的过程。比如，系统获取的原始图像由于受到各种条件的限制和随机干扰，往往不能直接使用，需要在图像处理的早期阶段对它进行灰度校正、噪声过滤等图像预处理。
例如，上述人脸特征提取的方法可以包括：基于知识的表征方法和基于代数特征或统计学习的表征方法。
例如，上述人脸图像匹配与识别是指提取的人脸图像的特征数据与数据库230中存储的特征模板(人脸特征)进行搜索匹配的过程；通过设定一个预设阈值，当相似度超过预设阈值，则把匹配得到的结果输出。人脸识别就是将待识别的人脸特征与已得到的人脸特征模板进行比较，根据相似程度对人脸对应的用户进行判断。
应理解，上述人脸识别模块241可以部署于车辆中，或者，上述人脸识别模块241也可以部署于云端。
还应理解，上述人脸识别模块241仅为举例说明，人脸识别模块241用于通过采集用户的图像从而对用户进行识别，人脸识别模块241可以是任何形式，本申请实施例对此不作任何限定。
可选地，图像处理单元240还包括映射模块242。映射模块242可以将用户的车内位置与用户建立映射关系，并将车内用户与车内位置的映射关系存储至数据库230。其中，用户的车内位置可以是根据摄像头251的角度和/或位置确定的。
可选地，图像处理单元240还可以用于定时更新车内用户与车内位置的映射关系。
示例性地，数据库230可以包括用户信息数据库231。该用户信息数据库231可以用于存储用户与用户的声纹特征的映射关系以及用户与用户的人脸特征的映射关系。示例性地，用户信息数据库231可以存储有所有在车机上注册过的用户的信息。
示例性地，数据库230还可以包括实时信息数据库232。该实时信息数据库232可以用于存储车内用户与车内位置的映射关系。例如，该用户与用户之间的映射关系是可以基于摄像头采集的人脸图像确定的。示例性地，基于用户信息数据库231中的数据，可以通过用户的声纹信息指示用户。具体地，可以根据用户与用户的声纹特征的映射关系以及车内用户与车内位置的映射关系，得到用户的声纹特征与车内位置的映射关系。也就是说，该实时信息数据库232可以用于存储用户的声纹特征与车内位置之间的映射关系。在该情况下，定位模块222可以在实时信息数据库232中匹配用户的声纹特征，进而确定用户的车内位置。用户的声纹特征与车内位置之间的映射关系可以随着车内用户与车内位置的映射关系的更新而更新。
下面结合图5对本申请实施例提供的车内用户定位方法进行详细的描述。
图5示出了本申请实施例提供的车内用户定位方法300，该车内用户定位方法300可以由能够进行车内用户定位的装置或系统执行，例如，可以由图4所示的车载系统200执行。方法300包括步骤S310至步骤S350。下面对步骤S310至步骤S350进行详细说明。
S310，获取车内麦克风采集的声音信号。
其中，车内麦克风可以为一个麦克风，也可以为多个麦克风。例如，多个麦克风可以为麦克风阵列。麦克风阵列可以根据需要设置，例如，该麦克风阵列可以为均匀圆阵或均匀线阵。图6中示出了一种车内硬件配置的结构示意图。图6中的麦克风阵列包括4个麦克风410，位于座舱前方的驾驶台，呈线性排列。
S320，在从该声音信号中识别到第一语音命令时，确定发出该第一语音命令的第一用户。
具体地，可以对该声音信号进行语音识别，判断声音信号中是否包含第一语音命令。在声音信号中包含第一语音命令时确定发出该第一语音命令的第一用户。例如，该语音识别可以由图4所示的语音识别模块221执行。
示例性地，该第一语音命令可以为唤醒词，该唤醒词可以用于唤醒语音助手，激活后续处理流程。
确定发出第一语音命令的第一用户的方式可以包括多种方式。本申请实施例对确定第一用户的方式不做限定。
可选地，可以对第一语音命令进行声纹特征匹配，识别发出第一语音命令的第一用户。
具体地，可以根据用户和用户的声纹特征的映射关系，对第一语音命令进行声纹识别，确定发出第一语音命令的第一用户。这样能够基于声纹特征识别需要交互的用户，提高了定位准确性。
示例性地，数据库230中保存用户与用户的声纹特征的映射关系，不同的声纹特征可以对应不同的用户，声纹特征可以用于指示用户。通过图4的声纹识别模块223进行声纹识别，将提取的第一语音命令对应的声纹的特征数据与数据库230中存储的特征模板(声纹特征)进行搜索匹配，得到与之匹配的第一声纹特征，也即确定第一用户。其中，该第一声纹特征是第一用户的声纹信息。
若无法识别第一语音命令对应的声纹特征，也就是无法在数据库中匹配到对应的用户，则该用户可能未录制声纹信息，例如，该用户可能第一次乘坐该车。在该情况下，可以提醒用户补录声纹特征，更新数据库中的信息。
可替换地，可以根据用户和语音命令的映射关系，确定第一语音命令对应的第一用户。用户与语音命令的映射关系可以存储于数据库230中。
示例性地，可以预先为不同的用户设置不同的语音命令，并将该语音命令与用户绑定，即建立映射关系。例如，将该映射关系作为数据对存储于数据库230中。
例如，用户A与语音命令“打开A的扬声器”建立映射关系，用户B与语音命令“打开B的扬声器”建立映射关系。若第一语音命令为“打开A的扬声器”，则确定发出第一语音命令的第一用户为用户A。
S330，根据车内用户与车内位置的映射关系确定该第一用户的车内位置。
车内用户与车内位置的映射关系的获取方法可以包括多种方式。例如，车内用户与车内位置的映射关系可以是预存的，也可以是从其他设备，例如云服务器获取的，还可以是实时更新的。本申请实施例对获取车内用户与车内位置的映射关系的方法不做限定。
示例性地，车内位置可以通过角度范围表示。或者，车内位置也可以通过角度值表示。
示例性地，车内可以划分为多个区域，车内位置可以由车内区域表示。例如，车内区域可以为车内座位，即通过车内座位指示车内位置。这样能够减少车内位置的可选项，减少计算量，提高处理速度。
或者，车内位置可以为具体的车内位置。这样，即使用户不是坐在某一座位上，例如，用户坐在两个座位之间或者用户站在车内的过道上，也可以得到该用户的准确位置。
示例性地，该车内用户与车内位置的映射关系可以存储于数据库230中。步骤S330可以由前述定位模块222执行。
车内用户与车内位置的映射关系的确定方式可以包括多种方式。
可选地，车内用户与车内位置的映射关系可以是根据车载摄像头采集的人脸图像确定的。根据车载摄像头采集的人脸图像确定用户与车内的映射关系，无需用户配合，能够避免打扰用户，提高用户体验。
可选地，可以根据用户信息和人脸图像的映射关系确定人脸图像对应的用户；根据车载摄像头的位置和/或角度确定用户的车内位置。即为用户与用户的车内位置建立映射关系。
该用户信息的获取方法可以包括多种方式。例如，用户信息可以是预存的，也可以是从其他设备，例如云服务器获取的，还可以是实时更新的。本申请实施例对获取用户信息的方法不做限定。
根据用户信息和人脸图像的映射关系，确定人脸图像对应的用户，也就是对人脸图像进行人脸识别，确定人脸图像对应的用户。
用户信息和人脸图像的映射关系也可以理解为用户和用户的人脸特征的映射关系。
示例性地，数据库230中可以保存用户与用户的人脸特征的映射关系，不同的人脸特征可以对应不同的用户，人脸特征可以用于指示用户。通过图4的人脸识别模块241进行人脸识别，将提取的人脸图像的特征数据与数据库230中存储的特征模板(人脸特征)进行搜索匹配，得到与之匹配的人脸特征，也即确定该人脸图像对应的用户。
若无法识别人脸图像对应的声纹特征，也就是无法在数据库中匹配到对应的用户，则该用户可能未保存面部特征，例如，该用户可能第一次乘坐该车。在该情况下，可以提醒用户补录面部特征，更新数据库中的信息。
将用户的信息均保存于数据库中，能够通过数据库实现数据共享。
例如，数据库中存储有用户与用户的声纹特征的映射关系以及车内用户与车内位置的映射关系，进而可以确定用户的声纹信息与车内位置的映射关系。
示例性地，车内位置可以通过角度范围表示。或者，车内位置也可以通过角度值表示。
示例性地，车内可以划分为多个区域，车内位置可以由车内区域表示。例如，车内区域可以为车内座位，即通过车内座位指示车内位置。根据座舱的布局能够确定用户的座位范围。这样能够减少车内位置的可选项，减少计算量，提高处理速度。
或者，车内位置可以为具体的车内位置。这样，即使用户不是坐在某一座位上，例如，用户坐在两个座位之间或者用户站在车内的过道上，也可以得到该用户的准确位置。
其中，车载摄像头可以为车内摄像头，也可以为车外摄像头。例如，车外摄像头可以安装于车辆B柱上。
下面举例说明通过车内摄像头确定用户的车内位置。
方式1：通过车内摄像头的位置与车内位置之间的映射关系确定用户的车内位置。
例如，在车内不同的位置设置多个摄像头，不同位置的摄像头与车内位置一一对应。在该情况下，摄像头的位置与车内位置之间的映射关系也可以理解为不同摄像头与车内位置的映射关系。不同摄像头拍摄的用户所在的位置即为该摄像头对应的车内位置。
进一步地，如前所述，可以通过车内座位指示车内位置。在该情况下，摄像头的位置与车内位置之间的映射关系可以为摄像头的位置与车内座位的映射关系。不同位置的摄像头与车内座位一一对应，不同摄像头拍摄的用户所在的位置即为该摄像头对应的车内座位。
例如，如图6所示，对于四座汽车，可以设置四个摄像头420，分别位于四个座位的前方。其中，对于前排座位，摄像头420可以设置于驾驶台；对于后排座位，摄像头420可以设置于前排座位的背面。如图6所示，由副驾驶座位前方的摄像头拍摄到的用户，可以认为该用户的车内位置为副驾驶座位，即图6中的阴影区域。以车身的中线方向为0°，该区域可以为-20°至-45°。应理解，该角度可以根据需要设置，不同的车型不同座舱布局对应的角度也不同。
方式2：通过车内摄像头的旋转角度与车内位置之间的映射关系确定用户的车内位置。
例如，摄像头位于车内后视镜上，摄像头的旋转角度与车内位置对应。以车身的中线方向为0°，摄像头的旋转角度为0°时，对应车内的位置为车的中部，在该情况下，摄像头拍摄的用户所在的位置即为车的中部；摄像头的旋转角度为角度A时，对应车内的位置为车内位置A，在该情况下，摄像头拍摄的用户所在的位置即为车内位置A。
进一步地，如前所述，可以通过车内座位指示车内位置。在该情况下，摄像头的旋转角度与车内位置的映射关系可以为摄像头的旋转角度与车内座位的映射关系。
例如，对于四座汽车，摄像头位于车内后视镜上。车内座位与摄像头的旋转角度对应。位于不同旋转角度的摄像头拍摄的用户即位于该摄像头在该旋转角度对应的车内座位。将车内位置用车内座位表示，这样可以根据座位设置旋转角度，在定位过程中能够减少计算 量，提高处理速度。
由于摄像头的视场范围与摄像头的位置和/或角度有关，在一些实施例中，通过摄像头确定用户的车内位置可以包括：通过摄像头的视场范围与车内位置之间的映射关系确定由该摄像头拍摄的用户所在的车载位置。
例如，若摄像头的视场范围对应车内的位置为车的中部，比如，视场角的中线与车身平行，在该情况下，摄像头拍摄到的用户所在的位置为车的中部。若摄像头的视场范围对应车内的位置为位置A，在该情况下，摄像头拍摄到的用户所在的位置为位置A。
可选地，根据用户和用户的人脸特征的映射关系，确定人脸图像对应的用户，根据用户在车载摄像头的镜头中的相对位置确定用户的车内位置。
可选地，可以通过车内座位指示车内位置，这样，可以将镜头中的相对位置映射到车内座位上，在定位过程中能够减少计算量，提高处理速度。
例如，车载摄像头位于车内后视镜上，在一次拍摄中可以将车内所有用户放入取景框中。通过人脸识别确定人脸图像对应的用户及其在镜头中的相对位置，根据该相对位置确定用户对应的车内位置。
再如，车载摄像头位于车外的B柱上，两侧B柱可以分别设置车载摄像头。对于左侧B柱上的摄像头采集的人脸图像，通过人脸识别确定人脸图像对应的用户及其在镜头中的相对位置。例如，该用户在镜头中的相对位置靠右侧，则该用户从车辆左侧的前侧车门上车，该用户对应的车内位置可能位于驾驶座上。
应理解，以上仅为示例，摄像头的数量可以为一个或多个，摄像头的位置或摄像头的数量等可以根据需要设置。
示例性地，该车内用户与车内位置的映射关系可以存储于实时信息数据库232中。该实时信息数据库可以为用户乘车后建立的数据库。
例如，获取摄像头拍摄的人脸图像，根据用户和用户的人脸特征的映射关系，确定人脸图像对应的用户，根据所述车载摄像头的位置和/或角度确定所述用户的车内位置，得到车内用户与车内位置的映射关系，在数据库230中获取该用户的声纹特征，建立用户的声纹特征与车内位置的映射关系，将用户的声纹特征与车内位置的映射关系存储于实时信息数据库232中。然后可以执行步骤S310至步骤S330。这样可以在实时信息数据库中查找用户与车内位置之间的映射关系，避免每次定位用户均需重新确定车内用户与车内位置的映射关系。此外实时信息数据库中的用户数量小于或等于数据库中的用户数量，在该实时信息数据库中查找映射关系，能够减小查找范围，提高处理速度。
示例性地，车内用户与车内位置的映射关系可以是实时更新，或者，也可以是定时更新的。具体的更新车内用户与车内位置的映射关系或者说更新该实时信息数据库的方法详见后文中的方法600。
示例性地，车内用户与车内位置的映射关系可以是实时建立的。
例如，执行步骤S310和步骤S320，确定第一用户。获取摄像头拍摄的人脸图像，根据用户和用户的人脸特征的映射关系，确定人脸图像对应的用户，在该用户为第一用户时，根据车载摄像头的位置和/或角度确定该用户的车内位置，即建立第一用户与第一用户的车内位置之间的映射关系，确定第一用户的车内位置。这样可以仅在需要定位车内用户时才建立车内用户与车内位置的映射关系，避免不必要的计算，同时可以仅针对第一用户建 立第一用户与其位置之间的映射关系，降低计算量。
应理解，以上确定车内用户与车内位置的映射关系的方式仅为示例，本申请实施例对车内用户与车内位置的映射关系的确定方式不做限定。
示例性地，车内不同的位置处可以设置指纹采集器。通过指纹采集器采集指纹，根据用户与用户的指纹特征的映射关系，确定指纹采集器采集的指纹对应的用户。由于不同指纹采集器即对应不同的车内位置，可以将用户与车内位置建立映射关系。例如，指纹采集器A对应车内位置A，由该指纹采集器A采集到指纹A，通过指纹识别确定指纹A对应用户A，进而将该用户A与车内位置A建立映射关系。
可选地，车内用户与车内位置的映射关系可以是定时更新的。
由于座舱内的用户可能更换座位或者上/下车，定时更新车内用户与车内位置的映射关系，能够提高用户定位的准确性。
结合第一方面，在第一方面的某些实现方式中，车内用户与车内位置的映射关系是定时更新的，包括：
根据车内用户与车内位置的映射关系确定当前车内位置对应的用户，获取车载摄像头采集的当前人脸图像，根据用户和用户的人脸特征的映射关系，确定当前人脸图像对应的当前用户；在当前车内位置对应的用户与当前用户不同时，更新车内用户与车内位置的映射关系。
例如，根据车内用户与车内位置的映射关系确定当前车内位置没有用户，获取车载摄像头采集的当前人脸图像，根据用户和用户的人脸特征的映射关系，确定当前人脸图像对应的当前用户；将当前用户与当前车内位置建立映射关系，更新车内用户与车内位置的映射关系。
再如，根据车内用户与车内位置的映射关系确定当前车内位置对应的用户为用户A，获取车载摄像头采集的当前人脸图像，根据用户和用户的人脸特征的映射关系，确定当前人脸图像对应的当前用户为用户B；当用户A与用户B不同时，将用户B与当前车内位置建立映射关系，更新车内用户与车内位置的映射关系。
可选地，方法300还包括步骤S340和步骤S350。
S340，根据声音信号进行声源定位，获得至少一个声源位置。
具体地，可以通过基于麦克风阵列的声源定位方法对声音信号进行声源定位。例如，该声源定位方法可以为MUSIC算法。该麦克风阵列可以根据需要设置，示例性地，该麦克风阵列可以为均匀圆阵或均匀线阵。例如，图6中示出了一种麦克风阵列，该麦克风阵列包括4个麦克风，位于座舱前方的驾驶台，呈线性排列。
通过MUSIC算法可以得到信号能量在不同角度的分布图谱，峰值即为信号的传入方向。根据该图谱可以得到至少一个声源的位置。例如，如图2所示，当0°和60°的用户同时说话时，在图谱上会产生两个峰值。
应理解，此处仅为示意，本申请实施例对声源定位的具体方式不做限定，其他能够实现声源定位的方法也可用于步骤S340。
S350，根据第一用户的车内位置和所述至少一个声源位置修正第一用户的车内位置。
若步骤S330中无法确定第一用户的车内位置，例如，无法识别第一用户，则在该步骤中的第一用户的车内位置可以为默认值，例如，-90°～90°，也就是说，直接根据至少 一个声源位置确定第一用户的车内位置。
可选地，将至少一个声源位置中与第一用户的车内位置最接近的一个声源位置确定为修正后的第一用户的车内位置。
示例性地，若步骤S330得到的第一用户的车内位置为一个角度范围，则将至少一个声源位置中的落于该角度范围内的声源位置作为修正后的第一用户的车内位置。例如，图7中示出了一种修正第一用户的车内位置的示意图。由步骤S330确定的第一用户的车内位置为50°至70°范围，对声音信号进行声源定位得到关于声源位置的能量图谱，将该能量图谱与第一用户的角度范围叠加，50°至70°范围内只有60°位置一个峰值，排除0°位置的峰值，将60°作为最终的第一用户的车内位置。也就是说0°位置和60°位置的用户均在说话，60°位置处的用户为需要进行语音交互的用户，排除0°位置的用户对语音交互的影响。若该角度范围内的声源位置有多个，则可以取该角度范围的中间值作为修正后的第一用户的车内位置。例如，该角度范围为20°至40°范围，该范围内有多个声源位置，则将30°作为修正后的第一用户的车内位置。在出现多个声源位置时，仅通过声源定位无法判断需要交互的用户。相对于仅通过声源定位确定用户的位置，本方案能够提高定位用户的准确性，减小嘈杂的环境，例如，多个用户同时说话，对语音交互的影响。
示例性地，若步骤S330中无法确定第一用户的车内位置，例如，第一用户的车内位置为-90°～90°范围，则将该范围内的声源位置作为修正后的第一用户的车内位置。若该范围内有多个声源位置，可以取0°作为修正后的第一用户的车内位置。
示例性地，若步骤S330得到的第一用户的车内位置为一个角度值，则将至少一个声源位置中与该值最接近的声源位置作为修正后的第一用户的车内位置。
通过声源定位能够进一步提高第一用户的车内位置的准确性，在车内嘈杂的环境中也能准确定位需要交互的用户。或者说，结合声学和视觉采集的信息，能够提高声源定位的精度和使用范围。
根据本申请实施例的方案，基于语音命令判断需要交互的用户，进而根据车内用户与车内位置的映射关系确定需要交互的用户的车内位置，即使在多个用户同时说话的情况下也能定位至需要交互的用户所在的位置，实现一对一交互，提高用户体验。
本申请实施例中的车内用户定位方法可以应用于多种车载交互场景，具体地，可以应用于需要对用户进行定位的场景。
目前，车载语音助手的表现形式和交互模式较为单一，大多数仅通过语音进行交互，且多为二维动画形象。
3D全息投影是一种利用干涉和衍射原理记录并再现物体真实的三维图像，观众无需佩戴眼镜即可看到的立体的虚拟人物的3D技术。3D全息投影的基本原理为，拍摄过程中利用干涉原理记录物体光波信息，成像过程中利用衍射原理再现物体光波信息，从而再现物体真实的三维图像。3D全息投影也可以包括使用佩伯尔幻像、边缘消隐等方法实现3D效果的类全息投影技术。而就目前座舱内的技术而言，激光和空气的投影系统不能长期保持投影效果，类全息投影的投影框大多较为固定，无法实现更好的交互体验。
通过三维投影展示语音助手的视觉形象，能够提高娱乐性，显著提高用户体验。在用户需要交互时，该语音助手的形象能够面对需要交互的用户，提供一对一的交互体验。
图8示出了本申请实施例提供的一种车载交互方法500，该车载交互方法中涉及的车内用户定位的方法可以通过车内用户定位方法300实现，具体实现方式可以参照前述方法300，为了避免不必要的重复，下面在介绍方法500时适当省略重复的描述。方法500包括步骤S510至步骤S570。
方法500可以由图4中的系统执行。图4中的语音助手组件210中还可以包括投影组件。具体地，该投影组件包括投影框架和转向电机212。该投影框架可以用于展示语音助手的三维形象。
图9示出了本申请实施例提供的一种投影框架的结构示意图。图9中的投影框架主要包括四部分：框架主体910、显示屏920、旋转底座930和投影结构940。
其中，框架主体910为圆柱形的钢化玻璃，正后方和顶部覆盖有遮光涂料，这样可以防止阳光漫反射造成的视觉效果减弱。
显示屏920用于投影。显示屏920的形状可以根据需要设置，例如，可以为正方形显示屏。显示屏920位于框架主体的顶部，例如，可以镶嵌于圆柱顶部内侧。
旋转底座930与座舱内的旋转电机相连，用于带动投影结构940和显示屏920旋转。该框架主体910在旋转过程中保持不动。
投影结构940固定于旋转底座930上，同时与显示屏920的位置相对固定，即投影结构940与显示屏920在旋转过程中保持相对静止。该投影结构940包括4个等腰梯形的钢化玻璃，该钢化玻璃上贴有全息膜。
旋转电机可以用于控制投影框架旋转。例如，该旋转电机可以为步进电机。该旋转电机可以接收定位模块222返回的结果，例如，位置信息，并根据该结果执行相应的转向操作，以使语音助手的三维形象面对需要交互的用户。
投影框架可以设置于中控台上，位于座舱内中线上。示例性地，车内麦克风410、摄像头420、投影框架430可以按照图6所示设置。应理解，图6仅为示意，车内麦克风、摄像头和投影框架的位置可以根据需要设置。
为了便于描述方法500，座舱中线可以设置为0°，声源定位的角度范围在-90°和90°之间。
下面对步骤S510至步骤S570进行说明。
S510，获取麦克风采集的声音信号。
S520，判断声音信号中是否包含第一语音命令，若声音信号中不包含第一语音命令，则结束流程，继续监听；若声音信号中包含第一语音命令，则执行步骤S530和步骤S550。步骤S530和步骤S550可以为异步执行。
具体地，可以对声音信号进行语音识别，判断声音信号中是否包含第一语音命令。
例如，该第一语音命令可以为唤醒词。
S530，确定发出第一语音命令的第一用户。
示例性地，根据用户和用户的声纹特征的映射关系，对所述第一语音命令进行声纹识别，确定发出所述第一语音命令的第一用户。具体描述可以参见前述方法300中的步骤S320。
S540，根据车内用户与车内位置的映射关系确定第一用户的车内位置。
示例性地，该实时信息数据库中存储有用户的声纹特征和车内座位的映射关系。通过 在实时信息数据库中比对第一语音命令的声纹特征，确定对应的车内座位。
例如，根据车载摄像头采集的人脸图像确定用户与车内座位的映射关系，进而根据用户与用户的声纹特征的映射关系确定用户的声纹特征，得到用户的声纹信息与车内座位的映射关系，并存储至实时信息数据库中。具体描述参见前述方法300中的步骤S330。
示例性地，该实时信息数据库可以是实时更新的，或者，也可以是定时更新的。实时信息数据库的详细更新方法可以参见后文中的方法600。
示例性地，该车内位置可以为角度范围。例如，当用户座位为副驾驶座位时，该角度范围可以为-20°～-45°。应理解，此处的角度范围仅为示意，对于不同的车型、不同的座舱布局，车内座位对应的角度范围可能不同。
S550，根据声音信号进行声源定位。
示例性地，可以通过MUSIC算法对声音信号进行声源定位，得到至少一个声源位置。具体描述可以参见前述方法300中的步骤S340。
S560，修正第一用户的车内位置。
示例性地，根据步骤S540确定的角度范围和步骤S550确定的至少一个声源位置修正第一用户的车内位置。
例如，可以将步骤S540确定的角度范围内的声源位置作为修正后的第一用户的车内位置。具体描述可以参见前述方法300中的步骤S350。
S570，根据第一用户的车内位置执行第一语音命令。
可选地，根据第一用户的车内位置控制投影界面的位置。
该投影界面可以用于显示投影形象。
示例性地，该投影界面的位置可以由投影组件的旋转角度控制。
具体地，第一用户的车内位置信息可以用于控制投影组件转向。将修正后的第一用户的车内位置返回至语音助手组件中的转向电机212，控制投影框架旋转。
应理解，步骤S550和步骤S560为可选步骤，若方法500不包括步骤S550和步骤S560，则在步骤S570中，将步骤S540得到的第一用户的车内位置返回至投影组件。
需要说明的是，第一语音命令不一定是投影界面的位置的命令，也就是说第一语音命令可以为任意语音命令。例如，该第一语音命令可以为调节空调温度，那么执行第一语音命令包括，调节空调温度，以及控制投影界面的位置。
如前所述，本申请实施例的车内用户定位方法可以应用于多种车载交互场景，例如，音乐播放、空调控制、座椅加热、座椅调节等需要对车内用户进行定位的场景。在不同的交互场景下可以通过不同的语音命令实现交互。下面举例说明车载交互方法500。
可选地，根据第一用户的车内位置执行第一语音命令，包括：控制第一用户的车内位置对应的扬声器。
示例性地，车载交互场景可以为音乐播放。例如，第一语音命令为播放音乐。步骤S570包括：控制第一用户的车内位置对应的扬声器，例如，打开扬声器，并播放相应音乐等。
可选地，根据第一用户的车内位置执行第一语音命令，包括：控制第一用户的车内位置对应的空调。
示例性地，车载交互场景可以为空调控制。例如，第一语音命令为打开空调。步骤 S570包括：打开第一用户的车内位置对应的空调。
应理解，以上仅为车载交互方法的应用示例，本申请实施例中的车载交互方法还可以应用于其他需要定位用户的场景。
在本申请实施例中，基于语音命令判断需要交互的用户，进而根据车内用户与车内位置的映射关系确定需要交互的用户的车内位置，即使在多个用户同时说话的情况下也能定位至需要交互的用户所在的位置。
此外，通过声源定位算法能够进一步提高定位的准确性，提高用户体验。
此外，通过投影组件展示投影形象，例如，展示语音助手的视觉形象，并根据车内位置调整投影组件的旋转角度，以使投影形象能够面对需要交互的用户，为用户提供一对一的交互体验。
图10示出了一种实时信息数据库更新方法600的示意性流程图。如前所述，可以通过车内区域表示车内位置，为了便于理解和说明，在方法600中通过车内区域按照车内座位划分。在该情况下，实时信息数据库更新方法也可以理解为车内座位信息的更新方法。方法600包括步骤S610至步骤S670。下面对步骤S610至步骤S670进行详细说明。
S610，获取摄像头采集的人脸图像。
S620，对该人脸图像进行人脸识别，确定该人脸图像对应的用户。
具体地，可以根据用户和用户的人脸特征的映射关系，确定人脸图像对应的用户。
示例性地，数据库中可以保存用户与用户的人脸特征的映射关系。对人脸图像进行特征提取，并将提取的特征数据与数据库中的用户的人脸特征进行匹配，判断是否被识别为数据库中的用户，若被识别为数据库中的用户，执行步骤S630。若不能被识别为非数据库中的用户，可以结束流程。
S630，判断数据库中是否包含该用户的声纹特征。若数据库中包括该用户的声纹信息，则执行步骤S640。若数据库中不包括该用户的声纹信息，可以结束流程。
可选地，方法600还包括步骤S660。在该情况下，若数据库中不包括该用户的声纹信息，则执行步骤S660。
S640，将该用户的声纹信息与用户的座位作为数据对存储至实时信息数据库中。
该用户的座位可以是通过摄像头确定的。例如，图6中示出了4个摄像头，分别对应4个座位。摄像头A对应座位A，由摄像头A拍摄的人脸图像对应的用户即位于座位A上。用户的座位可以通过座位号区分，即步骤S640可以为，将用户的声纹信息与用户的座位号座位数据对存储。
S650，执行持续更新。
示例性地，定时重新进行人脸识别，在识别到的面部特征与该座位当前对应的面部特征不一致时，可以重新建立用户与车内座位的映射关系，或者可以理解为，从数据库中查找该用户的声纹特征，建立用户的声纹特征与车内座位的映射关系。
例如，可以周期性执行步骤S610至步骤S640。应理解，步骤S650为可选步骤。
S660，补录用户的声纹特征，执行步骤S640。
S670，更新用户与用户的声纹特征的映射关系。
将补录的用户的声纹特征与用户建立映射关系，并将该映射关系存储至数据库中。应理解，步骤S670为可选步骤。
车内座位可以对应不同的状态，在执行上述步骤S610至步骤S670的过程中，车内座位的状态会随之改变。应理解，本申请实施例中的座位的状态指的是该座位在系统中的状态。
图11示出了一种车内座位状态转换过程的示意性流程图。
初始(Init)态：该状态为初始状态，在该状态下，进行初始化设置。例如，当启动语音助手时或座位上的用户发生变化时，该座位进入Init态。在该状态下，对用户进行人脸识别，确认用户身份，匹配声纹特征，并将相应的映射关系存储至实时信息数据库中，以供车内用户定位使用。具体初始化设置过程可以参见后文中的图12。完成后，该座位进入Ready态。若检测到座位上没有用户，则该座位进入Idle态。
准备(Ready)态：该座位处于可用状态，即可以用于实现本申请实施例中的方法。若车上的座位都处于可用状态，则该系统处于可用状态。进一步地，可以定时更新。例如，每隔一段时间，例如10s，进入Check状态，进行用户的校验。
校验(Check)态：在该状态下，检验座位上的用户是否发生改变。若未发生改变，则退回Ready状态。若发生改变，则进入Init态。或者，进一步地，若座位上没有用户，例如，用户下车或者换了座位，则该座位进入Idle态。具体检验过程可以参见后文中的图13。
空闲(Idle)态：该状态下的座位上没有用户。进一步地，该状态下可以定时更新。例如，每隔一段时间，例如10s，重新判断该座位上是否有用户，当该座位上出现乘客时，进入Init态重新开始更新用户的座位信息。
停止(Stop)态：在该状态下关闭语音助手。进一步地，可以删除实时信息数据库中的数据。前述任何状态下，断电关机或关闭语音助手均会进入该状态。
图12示出了一种用户的座位信息更新过程中初始化设置的示意性流程图。初始化设置包括步骤S710至步骤S7120。
S710，获取摄像头采集的人脸图像。
S720，对该人脸图像进行人脸识别。
S730，判断当前座位上是否有用户。若当前座位上有用户，则执行步骤S740。若当前座位上没有用户，则该座位进入Idle态。
S740，判断数据库中是否有该用户。
判断数据库中是否有该用户也可以理解为判断人脸识别是否能够识别出该用户。若数据库中有该用户，即根据该人脸识别结果识别出用户，则执行步骤S750。若数据库中没有该用户，即根据人脸识别结果无法识别出该用户，则执行步骤S780。步骤S780为可选步骤，
S750，将该用户与该车内座位相关联。也就是建立用户与车内座位的映射关系。
S760，判断数据库中是否有该用户的声纹特征。如果数据库中有该用户的声纹特征，则执行步骤S770。如果数据库中没有该用户的声纹特征，则执行步骤S7100。步骤S7100为可选步骤。
S770，将该用户的声纹特征和车内座位相关联。也就是建立用户的声纹特征与车内座位的映射关系。例如，将用户的声纹特征和座位号作为数据对保存至实时信息数据库中，该座位进入Ready态。
S780，判断是否录入用户信息。
具体地，可以询问该用户是否录入用户信息，若用户同意录入用户信息，则执行步骤S790。若用户不同意录入用户信息，则执行步骤S7120。
S790，采集用户信息，包括人脸特征等，并将其保存至数据库中。例如，在车机中注册登录。
S7100，判断是否录入声纹特征。
具体地，可以询问用户是否录入用户的声纹信息，若用户同意录入声纹信息，则执行步骤S7110。若用户不同意录入声纹信息，则该座位进入Ready态。
S7110，将用户与用户的声纹特征相关联。也就是建立用户与用户的声纹特征的映射关系。例如，将该映射关系作为数据对保存至数据库中，执行S770。
S7120，将该用户的人脸特征与该座位相关联，该座位进入Ready态。也就是建立用户的人脸特征与车内座位的映射关系。例如将该映射关系座位数据对保存至实时信息数据库中。这样可以保存该用户的人脸特征，以便之后判断座位上的用户是否改变。
图13示出了一种座位信息更新过程中校验的示意性流程图。该校验过程包括步骤S810至步骤S860。
S810，获取摄像头采集的人脸图像。
S820，对该图像进行人脸识别。
S830，判断用户是否改变。
具体地，可以根据人脸识别的结果与该座位对应的人脸特征进行对比，从而判断用户是否发生改变。
若发生改变，则执行步骤S840。若未发生改变，则该座位进入Ready态。
S840，定时再次获取n次数据，即每间隔一段时间再次执行步骤S810至步骤S830，得到n次结果。
S850，若改变次数超过阈值，则执行步骤S860，否则该座位进入Ready态。
需要说明的是步骤S850仅为示例。步骤S850还可以为，若n次结果满足预设条件，则执行步骤S860。例如，若n次结果均为该座位的用户发生改变，则执行步骤S860。若步骤S850得到的结果中存在未发生改变的结果，则该座位进入Ready态。这样可以“投票”表决该座位的用户是否发生变化，减少误判的可能性。
需要说明的是，步骤S840和步骤S850为可选步骤。步骤S830还可以为，若发生改变，则可以执行步骤S860。
S860，判断当前座位上是否有用户。若该座位上有用户，则该座位进入Init态，例如，执行图12中的步骤。需要说明的是，在该情况下，可以不执行步骤S730。若该座位上没有用户，则该座位进入Idle态。
示例性地，步骤S860也可以在步骤S830之前执行，例如，若该座位上有用户，则执行步骤S830，在该情况下，步骤S850中，若改变次数超过阈值，则该座位进入Init态。若该座位上没有用户，则该座位进入Idle态。
应理解，上述举例说明是为了帮助本领域技术人员理解本申请实施例，而非要将本申请实施例限于所例示的具体数值或具体场景。本领域技术人员根据所给出的上述举例说明，显然可以进行各种等价的修改或变化，这样的修改或变化也落入本申请实施例的范围 内。
下面结合附图对本申请实施例的装置实施例进行详细的描述，应理解，下面描述的装置能够执行前述本申请实施例的方法，为了避免不必要的重复，下面在介绍本申请实施例的装置时适当省略重复的描述。
图14是本申请实施例的一种车载系统的示意性框图。图14所示的车载系统900包括车机910、语音助手组件920、网关930、总线940、车载数据中心950、摄像头和控制器局域网络(controller area network，CAN)970。
其中，车机910中包含多个计算模块，通过总线940与其他设备通信。具体地，车机910包括语音识别模块911和定位模块912。可选地，车机还包括车内位置更新模块913。可选地，车机还包括声纹识别模块914。
语音识别模块911与图4中的语音识别模块221对应，定位模块912与图4中的定位模块222对应。声纹识别模块914与图4中的语音识别模块223对应。
车内位置更新模块913用于更新车内用户与车内位置的映射关系。示例性地，车内位置更新模块913可以执行图10、图12或图13中的步骤。
语音助手组件920包括麦克风921。麦克风921可以为麦克风阵列。可选地，语音助手组件920还包括转向电机922和投影框架923。语音助手组件可以通过网关接入座舱中。
语音助手组件920与图4中的语音助手组件210对应。
车载数据中心950中包含数据库，例如，图4中的数据库230。示例性地，车载数据中心950可以包括用户信息数据库和实时信息数据库。车载数据中心950可以通过总线为车机中的计算模块提供数据。
摄像头可以为多个。示例性地，对于四座汽车，摄像头可以包括4个，分布为主驾摄像头961，副驾摄像头962，后排摄像头963，后排摄像头964。摄像头可以与座舱内其他传感器一样经由CAN970连接网关930。
网关930与总线940相连，提供其他组件和硬件的接入能力。
图15是本申请实施例的车载装置的示意性框图。图15所示的车载装置1000包括存储单元1010和处理单元1020。
存储单元1010和处理单元1020可以用于执行本申请实施例的车内用户定位方法或车载交互方法，具体地，处理单元1020可以执行上述方法300或方法500。
存储单元1010用于存储车内用户与车内位置的映射关系。处理单元1020用于获取车内麦克风采集的声音信号；当从声音信号中识别到第一语音命令时，确定发出第一语音命令的第一用户；获取车内用户与车内位置的映射关系；根据车内用户与车内位置的映射关系确定第一用户的车内位置。
可选地，作为一个实施例，处理单元1020具体用于：对第一语音命令进行声纹特征匹配，识别发出第一语音命令的第一用户。
可选地，作为一个实施例，车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的。
可选地，作为一个实施例，车内用户与车内位置的映射关系是根据摄像头的位置与摄像头采集的人脸图像确定的，包括：获取用户信息；根据用户信息和人脸图像的映射关系，确定人脸图像对应的用户；根据摄像头的位置和/或角度确定用户的车内位置。
可选地，作为一个实施例，处理单元1020还用于：根据声音信号进行声源定位，获得至少一个声源位置；根据第一用户的车内位置和至少一个声源位置修正第一用户的车内位置。
可选地，作为一个实施例，处理单元1020具体用于：将至少一个声源位置中与第一用户的车内位置最接近的一个声源位置确定为修正后的第一用户的车内位置。
可选地，作为一个实施例，处理单元1020还用于：根据第一用户的车内位置执行第一语音命令。
可选地，作为一个实施例，处理单元1020具体用于：根据第一用户的车内位置控制投影界面的位置。
需要说明的是，上述车载装置1000以功能单元的形式体现。这里的术语“单元”可以通过软件和/或硬件形式实现，对此不作具体限定。
例如，“单元”可以是实现上述功能的软件程序、硬件电路或二者结合。所述硬件电路可能包括应用特有集成电路(application specific integrated circuit，ASIC)、电子电路、用于执行一个或多个软件或固件程序的处理器(例如共享处理器、专有处理器或组处理器等)和存储器、合并逻辑电路和/或其它支持所描述的功能的合适组件。
因此，在本申请的实施例中描述的各示例的单元，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
图16是本申请实施例提供的一种车载装置的硬件结构示意图。图16所示的车载装置1100(该车载装置1100具体可以是一种计算机设备)包括存储器1101、处理器1102、通信接口1103以及总线1104。其中，存储器1101、处理器1102、通信接口1103通过总线1104实现彼此之间的通信连接。
存储器1101可以是只读存储器(read only memory，ROM)，静态存储设备，动态存储设备或者随机存取存储器(random access memory，RAM)。存储器1101可以存储程序，当存储器1101中存储的程序被处理器1102执行时，处理器1102用于执行本申请实施例的车内用户定位方法或车载交互方法的各个步骤，例如，执行图5或图8所示的各个步骤。
应理解，本申请实施例所示的车载装置可以是服务器，例如，可以是云端的服务器，或者，也可以是配置于云端的服务器中的芯片。
处理器1102可以采用通用的中央处理器(central processing unit，CPU)，微处理器，应用专用集成电路(application specific integrated circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路，用于执行相关程序，以实现本申请方法实施例的车内用户定位方法或车载交互方法。
处理器1102还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的方法的各个步骤可以通过处理器1102中的硬件的集成逻辑电路或者软件形式的指令完成。
上述处理器1102还可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(ASIC)、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执 行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1101，处理器1102读取存储器1101中的信息，结合其硬件完成本申请实施中图15所示的车载装置中包括的单元所需执行的功能，或者，执行本申请方法实施例的图5或图8所示的方法。
通信接口1103使用例如但不限于收发器一类的收发装置，来实现车载装置1100与其他设备或通信网络之间的通信。
总线1104可包括在装置1100各个部件(例如，存储器1101、处理器1102、通信接口1103)之间传送信息的通路。
应注意，尽管上述车载装置1100仅仅示出了存储器、处理器、通信接口，但是在具体实现过程中，本领域的技术人员应当理解，车载装置1100还可以包括实现正常运行所必须的其他器件。同时，根据具体需要本领域的技术人员应当理解，上述车载装置1100还可包括实现其他附加功能的硬件器件。此外，本领域的技术人员应当理解，上述车载装置1100也可仅仅包括实现本申请实施例所必需的器件，而不必包括图16中所示的全部器件。
还应理解，本申请实施例中，该存储器可以包括只读存储器和随机存取存储器，并向处理器提供指令和数据。处理器的一部分还可以包括非易失性随机存取存储器。例如，处理器还可以存储设备类型的信息。
应理解，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：通用串行总线闪存盘(USB flash disk，UFD)，UFD也可以简称为U盘或者优盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (19)

  1. 一种车内用户定位方法，其特征在于，包括：
    获取车内麦克风采集的声音信号；
    当从所述声音信号中识别到第一语音命令时，确定发出所述第一语音命令的第一用户；
    获取车内用户与车内位置的映射关系；
    根据所述车内用户与车内位置的映射关系确定所述第一用户的车内位置。
  2. 如权利要求1所述的方法，其特征在于，所述确定发出所述第一语音命令的第一用户包括：
    对所述第一语音命令进行声纹特征匹配，识别发出所述第一语音命令的第一用户。
  3. 如权利要求1或2所述的方法，其特征在于，所述车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的。
  4. 如权利要求3所述的方法，其特征在于，所述车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的，包括：
    获取用户信息；
    根据所述用户信息和所述人脸图像的映射关系，确定所述人脸图像对应的用户；
    根据所述车载摄像头的位置和/或角度确定所述用户的车内位置。
  5. 如权利要求1至4中任一项所述的方法，其特征在于，所述方法还包括：
    根据所述声音信号进行声源定位，获得至少一个声源位置；
    根据所述第一用户的车内位置和所述至少一个声源位置修正所述第一用户的车内位置。
  6. 如权利要求5所述的方法，其特征在于，所述根据所述第一用户的车内位置和所述至少一个声源位置修正所述第一用户的车内位置，包括：
    将所述至少一个声源位置中与所述第一用户的车内位置最接近的一个声源位置确定为修正后的第一用户的车内位置。
  7. 一种车载交互方法，其特征在于，包括：
    获取车内麦克风采集的声音信号；
    当从所述声音信号中识别到第一语音命令时，确定发出所述第一语音命令的第一用户；
    获取车内用户与车内位置的映射关系；
    根据所述车内用户与车内位置的映射关系确定所述第一用户的车内位置；
    根据所述第一用户的车内位置执行所述第一语音命令。
  8. 如权利要求7所述的方法，其特征在于，所述根据所述第一用户的车内位置执行所述第一语音命令，包括：
    根据所述第一用户的车内位置控制投影界面的位置。
  9. 一种车载装置，其特征在于，包括：
    存储单元，用于存储车内用户与车内位置的映射关系；
    处理单元，用于：
    获取车内麦克风采集的声音信号；
    当从所述声音信号中识别到第一语音命令时，确定发出所述第一语音命令的第一用户；
    获取车内用户与车内位置的映射关系；
    根据所述车内用户与车内位置的映射关系确定所述第一用户的车内位置。
  10. 如权利要求9所述的装置，其特征在于，所述处理单元具体用于：
    对所述第一语音命令进行声纹特征匹配，识别发出所述第一语音命令的第一用户。
  11. 如权利要求9或10所述的装置，其特征在于，所述车内用户与车内位置的映射关系是根据车载摄像头采集的人脸图像确定的。
  12. 如权利要求11所述的装置，其特征在于，所述车内用户与车内位置的映射关系是根据摄像头的位置与所述摄像头采集的人脸图像确定的，包括：
    获取用户信息；
    根据所述用户信息和所述人脸图像的映射关系，确定所述人脸图像对应的用户；
    根据所述摄像头的位置和/或角度确定所述用户的车内位置。
  13. 如权利要求9至12中任一项所述的装置，其特征在于，所述处理单元还用于：
    根据所述声音信号进行声源定位，获得至少一个声源位置；
    根据所述第一用户的车内位置和所述至少一个声源位置修正所述第一用户的车内位置。
  14. 如权利要求13所述的装置，其特征在于，所述处理单元具体用于：
    将所述至少一个声源位置中与所述第一用户的车内位置最接近的一个声源位置确定为修正后的第一用户的车内位置。
  15. 如权利要求9至14中任一项所述的装置，其特征在于，所述处理单元还用于：
    根据所述第一用户的车内位置执行所述第一语音命令。
  16. 如权利要求15所述的装置，其特征在于，所述处理单元具体用于：
    根据所述第一用户的车内位置控制投影界面的位置。
  17. 一种车辆，其特征在于，包括如权利要求9至16中任一项所述的车载装置。
  18. 一种车载装置，其特征在于，包括至少一个处理器和存储器，所述至少一个处理器与所述存储器耦合，用于读取并执行所述存储器中的指令，以执行如权利要求1至6或7至8中任一项所述的方法。
  19. 一种计算机可读介质，其特征在于，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行如权利要求1至6或7至8中任一项所述的方法。
PCT/CN2020/088227 2020-04-30 2020-04-30 车内用户定位方法、车载交互方法、车载装置及车辆 WO2021217572A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP20933541.3A EP4134949A4 (en) 2020-04-30 2020-04-30 METHOD OF USER POSITIONING IN A VEHICLE, METHOD OF INTERACTION ON BOARD, BOARD DEVICE AND VEHICLE
CN202080004901.1A CN112655000B (zh) 2020-04-30 2020-04-30 车内用户定位方法、车载交互方法、车载装置及车辆
PCT/CN2020/088227 WO2021217572A1 (zh) 2020-04-30 2020-04-30 车内用户定位方法、车载交互方法、车载装置及车辆
US17/976,424 US20230038039A1 (en) 2020-04-30 2022-10-28 In-vehicle user positioning method, in-vehicle interaction method, vehicle-mounted apparatus, and vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/088227 WO2021217572A1 (zh) 2020-04-30 2020-04-30 车内用户定位方法、车载交互方法、车载装置及车辆

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/976,424 Continuation US20230038039A1 (en) 2020-04-30 2022-10-28 In-vehicle user positioning method, in-vehicle interaction method, vehicle-mounted apparatus, and vehicle

Publications (1)

Publication Number Publication Date
WO2021217572A1 true WO2021217572A1 (zh) 2021-11-04

Family

ID=75368408

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/088227 WO2021217572A1 (zh) 2020-04-30 2020-04-30 车内用户定位方法、车载交互方法、车载装置及车辆

Country Status (4)

Country Link
US (1) US20230038039A1 (zh)
EP (1) EP4134949A4 (zh)
CN (1) CN112655000B (zh)
WO (1) WO2021217572A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113486760A (zh) * 2021-06-30 2021-10-08 上海商汤临港智能科技有限公司 对象说话检测方法及装置、电子设备和存储介质
CN113593572A (zh) * 2021-08-03 2021-11-02 深圳地平线机器人科技有限公司 在空间区域内进行音区定位方法和装置、设备和介质
CN113978399A (zh) * 2021-08-12 2022-01-28 华为技术有限公司 设备调节方法、装置及存储介质
CN114125138B (zh) * 2021-10-29 2022-11-01 歌尔科技有限公司 音量调整优化方法、装置、电子设备及可读存储介质
CN114255753A (zh) * 2021-12-21 2022-03-29 北京地平线机器人技术研发有限公司 语音交互指令处理方法、装置及计算机可读存储介质
CN114274902A (zh) * 2021-12-23 2022-04-05 阿波罗智联(北京)科技有限公司 模式控制方法、装置、设备以及存储介质
US20230282210A1 (en) * 2022-03-03 2023-09-07 GM Global Technology Operations LLC System and method for integrating auditory and non-auditory inputs for adaptable speech recognition
CN116299179B (zh) * 2023-05-22 2023-09-12 北京边锋信息技术有限公司 一种声源定位方法、声源定位装置和可读存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017151672A2 (en) * 2016-02-29 2017-09-08 Faraday & Future Inc. Voice assistance system for devices of an ecosystem
CN108621981A (zh) * 2018-03-30 2018-10-09 斑马网络技术有限公司 基于座椅的语音识别系统及其识别方法
CN108790674A (zh) * 2018-04-27 2018-11-13 上海博泰悦臻网络技术服务有限公司 一种车载空调的控制方法及系统
CN110390932A (zh) * 2018-04-20 2019-10-29 比亚迪股份有限公司 基于人脸识别的语音处理方法及其设备
WO2019227774A1 (zh) * 2018-05-31 2019-12-05 上海商汤智能科技有限公司 车辆及车门解锁控制方法、装置和车门解锁系统
CN110718217A (zh) * 2019-09-04 2020-01-21 上海博泰悦臻电子设备制造有限公司 一种控制方法、终端及计算机可读存储介质
CN111045642A (zh) * 2019-12-17 2020-04-21 上海博泰悦臻电子设备制造有限公司 一种音量调节方法、车载终端及计算机存储介质

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104049721B (zh) * 2013-03-11 2019-04-26 联想(北京)有限公司 信息处理方法及电子设备
US9223340B2 (en) * 2013-08-14 2015-12-29 Lenovo (Singapore) Pte. Ltd. Organizing display data on a multiuser display
CN105280183B (zh) * 2015-09-10 2017-06-20 百度在线网络技术（北京）有限公司 语音交互方法和系统
CN109493871A (zh) * 2017-09-11 2019-03-19 上海博泰悦臻网络技术服务有限公司 车载系统的多屏语音交互方法及装置、存储介质和车机
CN109710055A (zh) * 2017-12-15 2019-05-03 蔚来汽车有限公司 车载智能交互系统和车载交互终端的交互控制方法
CN108882202A (zh) * 2018-06-27 2018-11-23 肇庆高新区徒瓦科技有限公司 一种基于智能手机的车载交互方法及装置
CN110874202B (zh) * 2018-08-29 2024-04-19 斑马智行网络(香港)有限公司 交互方法、设备、介质和操作系统
JP2020044879A (ja) * 2018-09-14 2020-03-26 トヨタ自動車株式会社 車両用音入出力装置
KR20200033617A (ko) * 2018-09-20 2020-03-30 현대자동차주식회사 차량용 음성 인식 장치 및 그 제어 방법
CN109545219A (zh) * 2019-01-09 2019-03-29 北京新能源汽车股份有限公司 车载语音交互方法、系统、设备及计算机可读存储介质
CN109941231B (zh) * 2019-02-21 2021-02-02 初速度（苏州）科技有限公司 车载终端设备、车载交互系统和交互方法
CN110082726B (zh) * 2019-04-10 2021-08-10 北京梧桐车联科技有限责任公司 声源定位方法及装置、定位设备及存储介质
CN110223690A (zh) * 2019-06-10 2019-09-10 深圳永顺智信息科技有限公司 基于图像与语音融合的人机交互方法及装置
CN110286771A (zh) * 2019-06-28 2019-09-27 北京金山安全软件有限公司 交互方法、装置、智能机器人、电子设备及存储介质
CN110509931B (zh) * 2019-08-28 2021-02-19 广州小鹏汽车科技有限公司 语音问答的信息展示方法、装置和系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017151672A2 (en) * 2016-02-29 2017-09-08 Faraday & Future Inc. Voice assistance system for devices of an ecosystem
CN108621981A (zh) * 2018-03-30 2018-10-09 斑马网络技术有限公司 基于座椅的语音识别系统及其识别方法
CN110390932A (zh) * 2018-04-20 2019-10-29 比亚迪股份有限公司 基于人脸识别的语音处理方法及其设备
CN108790674A (zh) * 2018-04-27 2018-11-13 上海博泰悦臻网络技术服务有限公司 一种车载空调的控制方法及系统
WO2019227774A1 (zh) * 2018-05-31 2019-12-05 上海商汤智能科技有限公司 车辆及车门解锁控制方法、装置和车门解锁系统
CN110718217A (zh) * 2019-09-04 2020-01-21 上海博泰悦臻电子设备制造有限公司 一种控制方法、终端及计算机可读存储介质
CN111045642A (zh) * 2019-12-17 2020-04-21 上海博泰悦臻电子设备制造有限公司 一种音量调节方法、车载终端及计算机存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4134949A4 *

Also Published As

Publication number Publication date
EP4134949A1 (en) 2023-02-15
US20230038039A1 (en) 2023-02-09
EP4134949A4 (en) 2023-04-05
CN112655000B (zh) 2022-10-25
CN112655000A (zh) 2021-04-13

Similar Documents

Publication Publication Date Title
WO2021217572A1 (zh) 车内用户定位方法、车载交互方法、车载装置及车辆
US11281944B2 (en) System and method for contextualized vehicle operation determination
CN108241371B (zh) 自动驾驶系统
CN108137052B (zh) 驾驶控制装置、驾驶控制方法和计算机可读介质
CN108537326B (zh) 用于自动驾驶车辆的方法、介质和系统
JP6922739B2 (ja) 情報処理装置、および情報処理方法、並びにプログラム
CN105270179B (zh) 车辆驾驶辅助装置及车辆
US20210097408A1 (en) Systems and methods for using artificial intelligence to present geographically relevant user-specific recommendations based on user attentiveness
US20210086779A1 (en) Evaluating driving control systems for elegant driving
US9919611B2 (en) Vehicle charge assistance device and vehicle including the same
WO2022204925A1 (zh) 一种图像的获取方法以及相关设备
US11237241B2 (en) Microphone array for sound source detection and location
US20230095988A1 (en) Systems and methods for performing commands in a vehicle using speech and image recognition
CN112989914A (zh) 具有自适应加权输入的注视确定机器学习系统
US10663312B2 (en) Vehicle and control method thereof
US20200074060A1 (en) User authentication device and method for triggering user-specific target operation
JP2020126166A (ja) エージェントシステム、情報処理装置、情報処理方法、およびプログラム
US11886634B2 (en) Personalized calibration functions for user gaze detection in autonomous driving applications
WO2022141225A1 (en) Methods, apparatus, and systems for operating device based on speech command
WO2021033574A1 (ja) 情報処理装置、および情報処理方法、並びにプログラム
WO2021217575A1 (zh) 用户感兴趣对象的识别方法以及识别装置
US20220277556A1 (en) Information processing device, information processing method, and program
US20230112797A1 (en) Systems and methods for using artificial intelligence to present geographically relevant user-specific recommendations based on user attentiveness
KR102220912B1 (ko) 차량용 디스플레이 장치 및 이를 구비한 차량
CN112584079A (zh) 视频通话人脸呈现方法、视频通话装置及汽车

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20933541

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020933541

Country of ref document: EP

Effective date: 20221109

NENP Non-entry into the national phase

Ref country code: DE