WO2020087747A1 - 一种音频设备定向显示方法、装置和音频设备 - Google Patents

一种音频设备定向显示方法、装置和音频设备 Download PDF

Info

Publication number
WO2020087747A1
WO2020087747A1 PCT/CN2018/125232 CN2018125232W WO2020087747A1 WO 2020087747 A1 WO2020087747 A1 WO 2020087747A1 CN 2018125232 W CN2018125232 W CN 2018125232W WO 2020087747 A1 WO2020087747 A1 WO 2020087747A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
audio device
screen
user
display screen
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/125232
Other languages
English (en)
French (fr)
Inventor
赵吉福
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Goertek Inc
Original Assignee
Goertek Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Goertek Inc filed Critical Goertek Inc
Priority to US17/309,136 priority Critical patent/US11910178B2/en
Publication of WO2020087747A1 publication Critical patent/WO2020087747A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D3/00—Control of position or direction
    • G05D3/12—Control of position or direction using feedback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303—Tracking of listener position or orientation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1601—Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays
    • G06F1/1605—Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
    • H04R1/028—Casings; Cabinets ; Supports therefor; Mountings therein associated with devices performing functions other than acoustics, e.g. electric candles
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0088—Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/15—Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/15—Aspects of sound capture and related signal processing for recording or reproduction

Definitions

  • the invention relates to an audio equipment directional display method, device and audio equipment.
  • Some smart audio devices on the market such as smart speakers, are mostly equipped with a display screen.
  • the display screen can be used not only to display related information, but also to realize human-computer interaction control of the smart speakers through a touch screen.
  • the display screen of a few smart audio devices can track the user for directional display. For example, when the user's position changes, the display screen of the device can automatically track the user and adjust the display direction of the display screen.
  • the existing directional display of the smart audio device can only track a single user. For the situation of multi-person interactive use, the display function of the smart audio device still needs to be strengthened.
  • the invention provides a method, device and audio equipment for audio equipment directional display, to solve the problem that the existing audio equipment cannot be applied to the directional display in the case of multi-person interactive use.
  • the audio device includes a camera module and a display screen distributed circumferentially on the side of the audio device.
  • the display screen can rotate relative to the audio device.
  • the method of the present invention includes: The start signal of the camera module is activated to perform a panoramic shot to detect the user's position; according to the relationship between the user's position and the center position of the audio device, determine whether the display is in the half-screen display mode; when it is determined to be in the half-screen display mode, determine The compensation angle of the display direction of the half-screen display area relative to the user position, and the display direction of the half-screen display area is adjusted so that the compensation angle is zero.
  • the directional display method of the present invention uses a camera module to detect the user's position, and determines whether the display screen is a half-screen display based on the positional relationship between the user's position and the center position of the audio device, and when the half-screen display is determined, adjusts the half-screen display
  • the display direction of the area makes the compensation angle of the display direction of the half-screen display area relative to the user's position zero, ensuring that multiple users are all within the display area of the audio device, enhancing the visual experience of the audio device when interacting with multiple people.
  • the audio equipment includes a camera module and a display screen distributed circumferentially on the side of the audio equipment.
  • the display screen can rotate relative to the audio equipment.
  • the device of the present invention includes: a detection unit , Used to start the camera module for panoramic shooting to detect the user's position in response to the acquired start signal; the judgment unit is used to determine whether the display screen is in the half-screen display mode according to the relationship between the user's position and the center position of the audio device
  • the calculation and adjustment unit is used to determine the compensation angle of the display direction of the half-screen display area relative to the user position when adjusting to the half-screen display mode, and adjust the display direction of the half-screen display area so that the compensation angle is zero.
  • the directional display device of the present invention uses a detection unit to drive a camera module to detect the user's position, and the determination unit determines whether the display screen is a half-screen display based on the positional relationship between the user's position and the center position of the audio equipment, and the calculation and adjustment unit determines During half-screen display, adjust the display direction of the half-screen display area so that the compensation angle of the display direction of the half-screen display area relative to the user position is zero, to ensure that multiple users are in the display area of the audio device Visual experience.
  • an audio device including: a camera module and a display screen distributed circumferentially on the side of the audio device, the display screen can rotate relative to the audio device, and further includes a processor and machine executable instructions stored The machine-readable storage medium, by reading and executing the machine-executable instructions in the machine-readable storage medium, the processor can execute the aforementioned audio device directional display method.
  • the audio device of the present invention uses a camera module to detect the user's position, and determines whether the display screen is a half-screen display based on the positional relationship between the user's position and the center position of the audio device.
  • the display direction of the display makes the compensation angle of the display direction of the half-screen display area relative to the user position zero, ensuring that multiple users are all within the display area of the audio device, improving the visual experience of the audio device when interacting with multiple people.
  • Another aspect of the present invention provides a machine-readable storage medium that stores machine-executable instructions.
  • the machine-executable instructions are executed by a processor, the foregoing audio device directional display method is implemented.
  • FIG. 1 is a flowchart of a method for directional display of an audio device according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a spatial coordinate system established in an audio device according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of an audio device used by three users according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of positions of three users on the circumference of a display screen in a panoramic image shown in an embodiment of the present invention
  • FIG. 5 is a schematic diagram of adjusting a half-screen display area according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of another three-user using an audio device according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of panoramic images at different moments shown in an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of the positions of three users on the circumference of the display screen in the panoramic image shown in an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of adjusting a half-screen display area according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a single user using an audio device according to an embodiment of the present invention.
  • FIG. 11 is a schematic diagram of panoramic images at different moments shown in an embodiment of the present invention.
  • FIG. 12 is a schematic diagram of the position of a single user on the circumference of the display screen in the panoramic image shown in an embodiment of the present invention
  • FIG. 13 is a schematic diagram of adjusting a half-screen display area according to an embodiment of the present invention.
  • FIG. 14 is a schematic structural diagram of an audio device directional display device according to an embodiment of the present invention.
  • FIG. 15 is a structural block diagram of an audio device according to an embodiment of the present invention.
  • 16 is a schematic diagram of a system hardware structure according to an embodiment of the present invention.
  • the technology of the present invention may be implemented in the form of hardware and / or software (including firmware, microcode, etc.).
  • the technology of the present invention may take the form of a computer program product on a computer-readable medium storing instructions, which may be used by or in conjunction with an instruction execution system.
  • a computer-readable medium may be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions.
  • computer-readable media may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, devices, or propagation media.
  • Specific examples of computer-readable media include: magnetic storage devices such as magnetic tapes or hard disks (HDD); optical storage devices such as compact disks (CD-ROM); memories such as random access memory (RAM) or flash memory; and / or wired / Wireless communication link.
  • the present invention first describes the composition structure of the audio device through the following embodiments.
  • the audio device in this embodiment includes smart speakers and other products.
  • the audio device of this embodiment includes a case surrounded by an upper case, a lower case, and a bottom case.
  • the camera module is disposed at the top of the upper case.
  • the camera module includes at least one panoramic camera or at least one group of binocular cameras or multiple cameras.
  • a spherical array of cameras for the camera; the display is set on the upper shell around the upper shell, the upper shell can rotate relative to the lower shell, and the position where the lower shell contacts the upper shell is provided with a PCB board, a micro motor assembly, etc.
  • the PCB board is audio
  • the main control board of the device, the micro motor assembly can drive the upper shell to rotate relative to the lower shell.
  • a ring of meshing teeth is provided at the position where the inner wall of the upper shell contacts the lower shell
  • the gear is set at the position where the inner wall of the lower shell contacts the upper shell.
  • the display screen of the audio device can follow the upper case, so that the display screen of the audio device can be controlled according to the following control mode to achieve display kinetic energy adjustment when multiple people use it interactively.
  • a method of the present invention provides an audio device directional display method.
  • the audio device includes a camera module and a display screen that is circumferentially distributed on the side of the audio device, and the display screen can rotate relative to the audio device.
  • the structure of the audio device in this embodiment reference may be made to the related description above.
  • FIG. 1 is a flowchart of a method for directional display of an audio device according to an embodiment of the present invention. As shown in FIG. 1, the method in this embodiment includes:
  • S110 in response to the acquired start signal, start the camera module to perform panoramic shooting to detect the user's position.
  • S120 Determine whether the display screen is in the half-screen display mode according to the relationship between the user position and the center position of the audio device.
  • the display screen in this embodiment is a ring-shaped display screen composed of two curved display screens. After the audio device is powered on, any curved display screen can be started for half-screen display.
  • the display area of the display screen is the currently working curved display screen, and the compensation angle of the currently working curved display screen relative to the user position determined at this time is used to display the curved display screen
  • the direction is adjusted so that when the audio device is in the half-screen display mode, the current curved display screen can track the user's position for directional display.
  • the camera module is used to detect the user's position, and based on the positional relationship between the user's position and the center position of the audio device, it can be determined whether the display screen is a half-screen display.
  • the display direction makes the compensation angle of the display direction of the half-screen display area relative to the user's position zero, ensuring that multiple users are all within the display area of the audio device, improving the visual experience of the audio device when interacting with multiple people.
  • step S110 is executed, that is, in response to the acquired activation signal, the camera module is activated to perform panoramic shooting to detect the user's position.
  • the activation signal includes an electrical signal generated from a voice signal carrying a preset keyword, and the voice signal can be picked up by a microphone array in the audio device and converted into an electrical signal, and the voice signal carries the preset keyword
  • the detector array is activated according to the preset keywords carried.
  • the preset keyword includes "Xiaozhi”
  • the received voice signal is "hello, Xiaozhi”
  • the voice signal can be obtained by voice recognition carrying the preset keyword "Xiaozhi” which can be generated according to the voice signal
  • the electrical signal activates the camera module of the audio device to take a panoramic shot to detect the user's position.
  • the camera module includes at least one panoramic camera or at least one set of binocular cameras or a spherical array of cameras with multiple cameras to enable panoramic shooting to obtain panoramic images, and to detect the user's position based on the analysis of the panoramic images.
  • the start signal acquired in this embodiment may be a periodic signal.
  • the camera module is periodically started for panoramic shooting to detect the user's position. For example, after the camera module is started for the first panoramic shot, the camera mode is turned off, the captured panoramic image is used to detect the user position, and the relationship between the user position and the center position of the audio device is obtained based on the user position and the center position of the audio device
  • the camera module for panoramic shooting analyze user changes based on panoramic images, based on user changes and user position and the center of the audio device
  • the relationship between the positions controls the display to display in half screen or full screen.
  • the period can be set to 5-20 minutes.
  • the user's position is detected in the following manner: First, the camera module is used to continuously take pictures to obtain a panoramic image centered on the center position of the audio device; then, according to the change of the position of the target relative to the background in the panoramic image at different moments, the panorama The target in the image is the detected user; then, based on the user's position in the panoramic image and the preset relationship between the length of the panoramic image and the circumference of the display screen of the audio device, determine the user's position relative to the circumference of the audio device.
  • the camera module continuously takes photos to compare photos at different moments, and analyzes whether there are users around the audio device and the number of users present.
  • the user is not absolutely stationary at any time, and the high-definition camera module can perfectly capture the user's movement.
  • targets A, B, and C are located in the space at the same time. After the camera module shoots, two panoramic images at time t0 and time t1 are obtained, and the position change of the same target in the two panoramic images can be compared. Determine if the target is a user.
  • target A is located at position Ia0, at time t1, target A is located at position Ia1, it can be determined that there is a displacement difference ⁇ Ia between target A at times t0 and t1, indicating that at this time in the same background, Target A has moved, and target A is the user.
  • ⁇ Ia displacement difference between target A at times t0 and t1
  • target A is the user.
  • ⁇ Ib and ⁇ Ic there is a position difference between the targets B and C relative to the background of the panoramic image, and both targets B and C are detected users.
  • the circumference of the display screen of the audio device is a line segment with a certain length, and the length of the line segment and the length of the panoramic image are both L.
  • step S120 is continued, that is, it is determined whether the display screen is in the half-screen display mode according to the relationship between the position of the user and the center position of the audio device.
  • this embodiment pre-sets the display direction to include the center position of the audio device and the center position of the arc of the half-screen display area.
  • the third connection between the position of the user at the other end of the image and the center position of the audio device then obtain the first angle between the second connection and the third connection, and according to the center position of the audio device and the circumference of the display screen
  • the horizontal distance and the vertical distance determine the first projection angle of the first included angle on the horizontal plane; then determine whether the display screen is in the half-screen display mode according to the first projection angle.
  • the horizontal plane where the center position of the audio device is located and the horizontal plane where the display circumference is located may not be the same horizontal plane. Therefore, the first angle formed by the second connection line and the third connection line may be projected on the horizontal plane For an audio device whose display screen can rotate in the horizontal direction, this embodiment needs to determine that the compensation angle of the half-screen display area relative to the user position is the angle on the horizontal plane.
  • the projection point of the center position of the audio device on the plane where the circumference of the display screen is the coordinate origin O ', the vertical line in the space of the audio device as the Z axis, and the plane where the circumference of the display screen is located The two perpendicular lines on the above establish the spatial rectangular coordinate system for the X 'axis and the Y' axis respectively. Since the length of the circumference of the display screen is a known value, the angle of any point on the circumference of the display screen equivalent to the origin of the coordinate is determined.
  • this embodiment provides that the number 1234 in FIG. 2 is used to represent the four quadrants in the X′O′Y ′ plane, that is, the number 1 represents the first quadrant, the number 2 represents the second quadrant, and the number 3 represents the third. For quadrants, use the number 4 to represent the fourth quadrant.
  • the horizontal distance between the center position O of the audio device and the circumference of the display screen is the radius length of the circumference of the display screen
  • the vertical distance between the center position O of the audio device and the circumference of the display screen is OO ′
  • the first projection angle of the first included angle ⁇ on the horizontal plane is ⁇ '.
  • the first projection angle is ⁇ 'greater than the preset angle, assuming that the preset angle is 180 °, referring to FIG. 3, the positions of users A, C on the left and right ends of the panoramic image and the center position of the audio device on the circumference of the display screen The angle formed by the connection line on the horizontal plane is greater than 180 °. At this time, it is determined that the display screen is in the full-screen display mode. As shown in FIG. 5, in the full-screen display mode, there is no need to adjust the display direction.
  • the preset angle is 180 °, referring to FIGS. 6 and 8, corresponding to the positions of the users A and C at the left and right ends of the panoramic image on the circumference of the display screen.
  • the angle formed by the line of the center position of the audio device on the horizontal plane is less than 180 °, and it is determined that the display screen is in the half-screen display mode.
  • step S130 After determining that the display screen is in the half-screen display mode, proceed to step S130, that is, determine the compensation angle of the half-screen display area relative to the user position, and adjust the display direction of the half-screen display area so that the compensation angle is zero.
  • the first The second projection angle of the two included angles on the horizontal plane is the compensation angle of the display direction of the half-screen display area relative to the user position.
  • the second included angle formed on the plane between the angle bisector of the first included angle ⁇ and the first connecting line is the compensation angle of the half-screen display area relative to the user's position.
  • the angle between the bisector L4 of the first projection angle ⁇ ' and the positive half axis O'X 'of the X' axis is the compensation angle
  • the compensation angle is the horizontal rotation of the display screen
  • the display direction of the adjusted half-screen display area ACDF is the direction of L4 in FIG. 9, so that the center position of the half-screen display area is toward the middle position of users A to C, and balance the user A to C
  • the field of view of the display area ensures the visual experience of all users.
  • control method of the display screen in this embodiment is still applicable to the case of single-user use.
  • FIG. 10-12 when analyzing the panoramic image at time t0 and time t1 to determine a single user A around the audio device, According to the position of user A in the panoramic image at time t1 and the proportional relationship between the length of the panoramic image and the circumference of the display screen, the position of user A relative to the circumference of the display screen is determined.
  • the positive half axis OX of the X axis is the current display direction, that is, OX corresponds to the first line, and the position of the user A in the panoramic image and the center position O of the audio device are obtained on the circumference of the display screen.
  • the third angle a formed by the fifth line L5 and the first line OX determines the third angle a on the horizontal plane according to the horizontal distance and the vertical distance between the center position of the audio device and the circumference of the display screen
  • the projection angle ⁇ is a compensation angle
  • the display screen is horizontally adjusted according to the third projection angle ⁇ .
  • the adjusted display direction of the half-screen display area MNIL is not set, so that the compensation angle ⁇ is zero.
  • a method of the present invention provides a directional display device for audio equipment.
  • the audio device includes a camera module and a display screen that is circumferentially distributed on the side of the audio device.
  • the display screen can rotate relative to the audio device.
  • FIG. 14 is a schematic structural diagram of an audio device directional display device according to an embodiment of the present invention. As shown in FIG. 14, the device in this embodiment includes:
  • the detection unit 141 is configured to start the camera module to perform panoramic shooting in response to the acquired start signal to detect the user's position;
  • the judging unit 142 is used to determine whether the display screen is in the half-screen display mode according to the relationship between the user position and the center position of the audio device;
  • the calculation and adjustment unit 143 is used for determining the compensation angle of the display direction of the half-screen display area relative to the user position when determining that it is in the half-screen display mode, and adjusting the display direction of the half-screen display area so that the compensation angle is zero.
  • the directional display device of this embodiment uses the detection unit to drive the camera module to detect the user's position, and the determination unit determines whether the display screen is a half-screen display based on the positional relationship between the user's position and the center position of the audio device, which is determined in the calculation and adjustment unit For half-screen display, adjust the display direction of the half-screen display area so that the compensation angle of the display direction of the half-screen display area relative to the user's position is zero, ensure that multiple users are in the display area of the audio device, and enhance the multi-person interaction of the audio device Visual experience.
  • the display direction in this embodiment includes the direction of the display position indicated by the first connection between the center position of the audio device and the center position of the half-screen display area; wherein the center position of the audio device is located on the vertical line of the audio device in space.
  • the detection unit 141 is used to continuously take photos with the camera module to obtain a panoramic image centered on the center position of the audio device; according to the change in the position of the foreground relative to the background in the panoramic image at different moments, determine the panoramic image Existing users; determine the user's position relative to the circumference of the display screen based on the user's position in the panoramic image and the preset relationship between the length of the panoramic image and the circumference of the display screen.
  • the judging unit 142 is used to acquire the second connection of the position of the user corresponding to the left end of the panoramic image and the center position of the audio device on the circumference of the display screen and the position of the right end user of the corresponding panoramic image and the center of the audio device on the circumference of the display screen
  • the third connection of the position obtain the first angle between the second connection and the third connection, and determine the first angle according to the horizontal distance and the vertical distance between the center position of the audio device and the circumference of the display screen
  • the first projection angle on the horizontal plane determine whether the display screen is a half-screen display according to the first projection angle.
  • the determining unit 182 specifically determines that the display screen is in the full-screen display mode when the first projection angle is greater than the preset angle; and determines the display screen in the half-screen display mode when the first projection angle is not greater than the preset angle.
  • the calculating and adjusting unit 143 is used to obtain the second included angle between the angle bisector of the first included angle and the first connecting line when determining to be in the half-screen display mode, and according to the center position of the audio device and the circumference of the display screen The horizontal distance and the vertical distance between them determine that the second projection angle of the second included angle on the horizontal plane is the compensation angle of the half-screen display area relative to the user position.
  • the relevant parts can be referred to the description of the method embodiments.
  • the device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located One place, or can be distributed to multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without paying creative labor.
  • Another aspect of the invention provides an audio device.
  • the audio device includes: a camera module and a display screen distributed circumferentially on the side of the audio device, the display screen can rotate relative to the audio device, and further includes processing And a machine-readable storage medium storing machine-executable instructions.
  • the processor can perform the audio device directional display method described above.
  • the compensation angle obtained by driving the upper casing to rotate through the motor drive gear is adjusted to make the adjusted
  • the compensation angle of the display direction of the half-screen display area relative to the user's position is zero, ensuring that all users are in the half-screen display area, improving the listening experience of multiple users of audio equipment.
  • Another aspect of the invention provides a system.
  • the system provided by this application may be implemented by software, or by hardware or a combination of hardware and software.
  • the system provided by the present application may include a processor 1601 and a machine-readable storage medium 1602 storing machine-executable instructions.
  • the processor 1601 and the machine-readable storage medium 1602 can communicate via a system bus 1603. And, by reading and executing the machine-executable instructions corresponding to the audio device directional display logic in the machine-readable storage medium 1602, the processor 1601 can execute the audio device directional display method described above.
  • Another aspect of the invention provides a machine-readable storage medium.
  • a machine-readable storage medium stores machine-executable instructions.
  • the audio device directional display method described above is implemented.
  • readable storage medium of the embodiment of the present invention may be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions.
  • readable storage media may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, devices, or propagation media.
  • Specific examples of readable storage media include: magnetic storage devices such as magnetic tapes or hard disks (HDD); optical storage devices such as compact disks (CD-ROM); memories such as random access memory (RAM) or flash memory; and / or wired / Wireless communication link.
  • the machine-readable storage medium may include a computer program, and the computer program may include code / computer-executable instructions, which when executed by the processor, cause the processor to perform, for example, the audio device directional display method flow described above and any variations thereof.
  • the computer program may be configured to have, for example, computer program code including computer program modules.
  • the code in the computer program may include one or more program modules. It should be noted that the division mode and number of modules are not fixed, and those skilled in the art may use appropriate program modules or program module combinations according to actual conditions.
  • the processor may execute, for example The flow of the audio device directional display method described above and any variations thereof.
  • the words “first” and “second” are used to distinguish the same or similar items with basically the same functions and functions. Personnel can understand that the words “first” and “second” do not limit the number and execution order.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Automation & Control Theory (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)
  • Stereophonic System (AREA)

Abstract

本发明公开一种音频设备定向显示方法、装置和音频设备。音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，显示屏可相对音频设备转动；本发明的方法包括：响应于获取的启动信号，启动摄像头模组进行全景拍摄以检测用户位置；根据用户位置与音频设备的中心位置之间的关系，确定显示屏是否处于半屏显示模式；在确定处于半屏显示模式时，确定半屏显示区域的显示方向相对用户位置的补偿角度，并调整半屏显示区域的显示方向，以使得补偿角度为零。本发明可以基于用户位置与音频设备的中心位置之间的位置关系确定在为半屏显示时，调整显示方向，使显示方向相对用户位置的补偿角度为零，保证多用户都处于音频设备的显示区域内，提升视觉体验。

Description

一种音频设备定向显示方法、装置和音频设备
本申请要求于2018年10月29日提交中国专利局、申请号为201811271455.1、发明名称为“一种音频设备定向显示方法、装置和音频设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及一种音频设备定向显示方法、装置和音频设备。
背景技术
随着生活水平的提高，智能家居越来越多的出现在人们的日常生活中，而智能音频设备作为其中的一类产品广受大众欢迎。市场上的一些智能音频设备，例如智能音箱大多设置有显示屏，显示屏不仅可以用于显示相关信息，还可以通过触控显示屏实现智能音箱的人机交互控制。
目前，只有少数智能音频设备的显示屏的是可以跟踪用户进行定向显示的，例如在用户位置发生变化时，设备的显示屏可以自动跟踪用户，调整显示屏的显示方向。但智能音频设备现有的定向显示仅能够跟踪单个用户，对于多人交互使用的情况，智能音频设备的显示功能仍然有待加强。
发明内容
本发明提供了一种音频设备定向显示方法、装置和音频设备，以解决现有音频设备无法适用于多人交互使用情况下的定向显示的问题。
本发明的一个方面提供了一种音频设备定向显示方法，音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，显示屏可相对音频设备转动，本发明的方法包括：响应于获取的启动信号，启动摄像头模组进行全景拍摄以检测用户位置；根据用户位置与音频设备的中心位置之间的关系，确定显示屏是否处于半屏显示模式；在确定处于半屏显示模式时，确定半屏显示区域的显示方向相对用户位置的补偿角度，并调整半屏显示区域的显示方向，以使得补偿角度为零。
本发明的定向显示方法利用摄像头模组检测用户位置，基于用户位置与音频设备的中心位置之间的位置关系，确定显示屏是否为半屏显示，在确定 为半屏显示时，调整半屏显示区域的显示方向，使半屏显示区域的显示方向相对用户位置的补偿角度为零，保证多用户都处于音频设备的显示区域内，提升音频设备多人交互时的视觉体验。
本发明的另一个方面提供了一种音频设备定向显示装置，音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，显示屏可相对音频设备转动，本发明的装置包括：检测单元，用于响应于获取的启动信号，启动摄像头模组进行全景拍摄以检测用户位置；判断单元，用于根据用户位置与音频设备的中心位置之间的关系，确定显示屏是否处于半屏显示模式；计算与调整单元，用于在确定处于半屏显示模式时，确定半屏显示区域的显示方向相对用户位置的补偿角度，并调整半屏显示区域的显示方向，以使得补偿角度为零。
本发明的定向显示装置利用检测单元驱动摄像头模组检测用户位置，判断单元基于用户位置与音频设备的中心位置之间的位置关系，确定显示屏是否为半屏显示，在计算与调整单元确定为半屏显示时，调整半屏显示区域的显示方向，使半屏显示区域的显示方向相对用户位置的补偿角度为零，保证多用户都处于音频设备的显示区域内，提升音频设备多人交互时的视觉体验。
本发明的另一个方面提供了一种音频设备，包括：包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，显示屏可相对音频设备转动，还包括处理器和存储有机器可执行指令的机器可读存储介质，通过读取并执行机器可读存储介质中的机器可执行指令，处理器可执行前述的音频设备定向显示方法。
本发明的音频设备利用摄像头模组检测用户位置，基于用户位置与音频设备的中心位置之间的位置关系，确定显示屏是否为半屏显示，在确定为半屏显示时，调整半屏显示区域的显示方向，使半屏显示区域的显示方向相对用户位置的补偿角度为零，保证多用户都处于音频设备的显示区域内，提升音频设备多人交互时的视觉体验。
本发明的另一个方面提供了一种机器可读存储介质，存储有机器可执行指令，机器可执行指令被处理器执行时以实现前述音频设备定向显示方法。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实 施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一部分附图，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本发明实施例示出的音频设备定向显示方法的流程图；
图2为本发明实施例示出的在音频设备中建立的空间坐标系示意图；
图3为本发明实施例示出的一种三用户使用音频设备的示意图；
图4为本发明实施例示出的全景图像中三用户在显示屏圆周的位置示意图；
图5为本发明实施例示出的调整半屏显示区域的示意图；
图6为本发明实施例示出的另一种三用户使用音频设备的示意图；
图7为本发明实施例示出的不同时刻的全景图像示意图；
图8为本发明实施例示出的全景图像中三用户在显示屏圆周的位置示意图；
图9为本发明实施例示出的调整半屏显示区域的示意图；
图10为本发明实施例示出的单个用户使用音频设备的示意图；
图11为本发明实施例示出的不同时刻的全景图像示意图；
图12为本发明实施例示出的全景图像中单个用户在显示屏圆周的位置示意图；
图13为本发明实施例示出的调整半屏显示区域的示意图；
图14为本发明实施例示出的音频设备定向显示装置的结构示意图；
图15为本发明实施例示出的音频设备的结构框图；
图16为本发明实施例示出的系统硬件结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
以下，将参照附图来描述本发明的实施例。但是应该理解，这些描述只是示例性的，而并非要限制本发明的范围。此外，在以下说明中，省略了对公知结构和技术的描述，以避免不必要地混淆本发明的概念。
在此使用的术语仅仅是为了描述具体实施例，而并非意在限制本发明。这里使用的词语“一”、“一个(种)”和“该”等也应包括“多个”、“多 种”的意思，除非上下文另外明确指出。此外，在此使用的术语“包括”、“包含”等表明了所述特征、步骤、操作和/或部件的存在，但是并不排除存在或添加一个或多个其他特征、步骤、操作或部件。
在此使用的所有术语(包括技术和科学术语)具有本领域技术人员通常所理解的含义，除非另外定义。应注意，这里使用的术语应解释为具有与本说明书的上下文相一致的含义，而不应以理想化或过于刻板的方式来解释。
附图中示出了一些方框图和/或流程图。应理解，方框图和/或流程图中的一些方框或其组合可以由计算机程序指令来实现。这些计算机程序指令可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器，从而这些指令在由该处理器执行时可以创建用于实现这些方框图和/或流程图中所说明的功能/操作的装置。
因此，本发明的技术可以硬件和/或软件(包括固件、微代码等)的形式来实现。另外，本发明的技术可以采取存储有指令的计算机可读介质上的计算机程序产品的形式，该计算机程序产品可供指令执行系统使用或者结合指令执行系统使用。在本发明的上下文中，计算机可读介质可以是能够包含、存储、传送、传播或传输指令的任意介质。例如，计算机可读介质可以包括但不限于电、磁、光、电磁、红外或半导体系统、装置、器件或传播介质。计算机可读介质的具体示例包括：磁存储装置，如磁带或硬盘(HDD)；光存储装置，如光盘(CD-ROM)；存储器，如随机存取存储器(RAM)或闪存；和/或有线/无线通信链路。
为便于说明本实施例音频设备显示屏的控制方式，本发明首先通过下述实施例说明音频设备的组成结构，本实施例的音频设备包括智能音箱等产品。
本实施例的音频设备包括由上壳、下壳和底壳围成的壳体，摄像头模组设置在上壳的顶端，摄像头模组包括至少一个全景摄像头或至少一组双目摄像头或具有多个摄像头的摄像头球形阵列；显示屏环绕上壳一周设置在上壳上，上壳可以相对下壳转动，下壳与上壳接触的位置设置有PCB板、微型电机组件等结构，PCB板为音频设备的主控板，微型电机组件能够驱动上壳相对下壳转动，例如在上壳内侧壁与下壳接触的位置设置一圈啮合齿，在下壳内侧壁与上壳接触的位置设置齿轮，在上壳与下壳固定连接时，上壳的啮合齿与下壳的齿轮相互啮合，微型电机驱动齿轮转动，以带动啮合齿，从而驱 动上壳转动，实现显示屏的定向显示；本实施例中下壳与底壳围成的腔体内容纳有发声组件，如喇叭、麦克风等。
在本实施例中，音频设备的显示屏可以跟随上壳转动，由此，可以根据下述控制方式对音频设备的显示屏进行控制，实现多人交互使用时的显示动能调整。
本发明的一个方法提供了一种音频设备定向显示方法。
在本实施例中，音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，所述显示屏可相对音频设备转动，本实施例的音频设备的结构可以参考上文的相关描述。
图1为本发明实施例示出的音频设备定向显示方法的流程图，如图1所示，本实施例的方法包括：
S110，响应于获取的启动信号，启动摄像头模组进行全景拍摄以检测用户位置。
S120，根据用户位置与音频设备的中心位置之间的关系，确定显示屏是否处于半屏显示模式。
本实施例中的显示屏为有两块曲面显示屏构成的环形显示屏，在音频设备上电工作后，可以启动任一块曲面显示屏进行半屏显示。
S130，在确定处于半屏显示模式时，确定半屏显示区域相对用户位置的补偿角度，并调整半屏显示区域的显示方向，以使得补偿角度为零。
在确定处于半屏显示模式时，此时显示屏的显示区域为当前工作的曲面显示屏，此时确定的该当前工作的曲面显示屏相对用户位置的补偿角度，以对该曲面显示屏的显示方向进行调整，使得在音频设备处于半屏显示模式时，当前的曲面显示屏可以跟踪用户位置进行定向显示。
本实施例利用摄像头模组检测用户位置，基于用户位置与音频设备的中心位置之间的位置关系，可以确定显示屏是否为半屏显示，在确定为半屏显示时，调整半屏显示区域的显示方向，使半屏显示区域的显示方向相对用户位置的补偿角度为零，保证多用户都处于音频设备的显示区域内，提升音频设备多人交互时的视觉体验。
以下结合图2-17对上述步骤S110-S130进行详细的说明。
首先，执行步骤S110，即响应于获取的启动信号，启动摄像头模组进行 全景拍摄以检测用户位置。
在实施例中，启动信号包括根据携带预设关键词的语音信号生成的电信号，可以通过音频设备中的麦克风阵列拾取语音信号并将语音信号转换为电信号，在语音信号携带预设关键词时，根据所携带的预设关键词启动检测器阵列。例如，预设关键词包括“小智”，所接收的语音信号为“hello，小智”，通过语音识别可以得到语音信号携带预设关键词“小智”，此时可以根据由语音信号生成的电信号启动音频设备摄像头模组进行全景拍摄来检测用户位置。其中，摄像头模组包括至少一个全景摄像头或至少一组双目摄像头或具有多个摄像头的摄像头球形阵列，以能够进行全景拍摄，获得全景图像，基于对全景图像的分析检测用户位置。
本实施例中获取的启动信号可以为周期性信号，按照启动信号的周期特性，周期性启动摄像头模组进行全景拍摄以检测用户位置。例如，在首次启动摄像头模组进行全景拍摄后关闭摄像头模式，利用拍摄得到的全景图像检测用户位置，并获得用户位置与音频设备的中心位置之间的关系，基于用户位置与音频设备的中心位置之间的关系控制显示器进行半屏显示或全屏显示之后，在下个周期节点时，重新启动摄像头模组进行全景拍摄，基于全景图像分析用户变化情况，基于用户变换情况和用户位置与音频设备的中心位置之间的关系控制显示器进行半屏显示或全屏显示。在设置启动信号的周期时，需要考虑摄像头模组耗用的电能以及显示屏定向跟踪用户位置的准确性，例如周期可以设定为5-20分钟。
本实施例通过下述方式检测用户位置：首先利用摄像头模组进行连续拍照，获取以音频设备的中心位置为中心的全景图像；接着根据不同时刻的全景图像中目标相对背景的位置变化，确定全景图像中该目标为检测到的用户；然后根据用户在全景图像中的位置，以及全景图像的长度与音频设备显示屏圆周长之间的预设关系，确定用户相对音频设备圆周的位置。
参考图7、11和15，摄像头模组通过连续拍照，进行不同时刻的照片对比，分析音频设备周围是否存在用户，以及所存在用户的人数。用户在任何时刻都不是绝对静止的，高清晰度的摄像头模组可以完美的捕捉用户的移动。例如，图7中目标A、B、C同时位于空间内，摄像头模组拍摄完成后，获取t0时刻和t1时刻的两张全景图像，通过对比两张全景图像中同一目标的位置 变换情况即可确定目标是否为用户。以目标A为例，在t0时刻，目标A位于Ia0位置，在t1时刻，目标A位于Ia1位置，即可确定目标A在t0与t1时刻存在位移差ΔIa，说明此时在相同的背景中，目标A发生了移动，目标A为用户。同样的，在t0与t1时刻，目标B和C相对全景图像的背景存在位置差ΔIb和ΔIc，目标B和C都是检测到的用户。
在分析不同时刻的全景图像确定全景图像中存在用户后，根据全景图像的长度与显示屏圆周长之间的预设关系，例如根据全景图像的长度与显示屏圆周长之间的长度比例关系，根据用户在全景图像中的位置，即可确定用户相对显示屏圆周的位置。参考图7，音频设备显示屏圆周展开后为具有一定长度的线段，该线段的长度和全景图像的长度均为L。以该线段的一端为基点，假设该线段的基点对应全景图像的右端，确定t1时刻的全景图像中用户A～C在全景图像中与全景图像右端之间距离依次为Ia1，Ib1，Ic1，那么根据公式
Figure PCTCN2018125232-appb-000001
可以分别确定用户A相对音频设备显示屏圆周的位置为角度α，用户B相对音频设备显示屏圆周的位置为角度β，用户C相对音频设备显示屏圆周的位置为角度γ。
在检测用户位置之后，继续执行步骤S120，即根据用户位置与音频设备的中心位置之间的关系，确定显示屏是否处于半屏显示模式。
为便于计算半屏显示区域的显示方向相对用户位置的补偿角度，减少计算量，简化计算步骤，本实施例预先设定显示方向包括音频设备的中心位置与半屏显示区域的圆弧中心位置的第一连线所指示的显示位置的方向；其中，音频设备的中心位置位于音频设备的空间中垂线上。
本实施例通过下述方式确定显示屏是否处于半屏显示模式：首先在显示屏圆周上获取对应全景图像一端用户的位置与音频设备中心位置的第二连线 和在显示屏圆周上获取对应全景图像另一端用户的位置与音频设备中心位置的第三连线；接着获取第二连线与第三连线之间的第一夹角，并根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定第一夹角在水平面上的第一投影角；然后根据第一投影角确定显示屏是否处于半屏显示模式。
由于在本实施例中，音频设备的中心位置所在的水平面与显示屏圆周所在水平面可能不是同一水平面，因此，第二连线与第三连线所形成的第一夹角可能在水平面上存在投影角，针对显示屏可以进行水平方向转动的音频设备，本实施例需要确定半屏显示区域相对用户位置的补偿角度为水平面上的角度。
为便于说明，如图2所示，以音频设备的中心位置在显示屏圆周所在平面上的投影点为坐标原点O’，以音频设备的空间中垂线为Z轴，以显示屏圆周所在平面上的相互垂直的两条直线分别为X’轴和Y’轴建立空间直角坐标系。由于显示屏所在圆周的长度是已知值，那么显示屏圆周上任一点相当于坐标原点的角度是确定的，因此，可以根据显示屏圆周长与全景图像长度的比例关系和全景图像中各用户位置在图像中的位置关系既可以确定用户相对显示屏圆周的位置。参考图2，本实施例规定利用图2中数字1234表示X’O’Y’平面的四个象限，即利用数字1表示第一象限，利用数字2表示第二象限，利用数字3表示第三象限，利用数字4表示第四象限。
参考图4和图8，假设对t0时刻和t1时刻的全景图像分析后，检测出音频设备四周存在三个用户A～C，确定出用户A～C相对显示屏圆周的位置(或角度)，在确定用户A相对显示屏圆周的角度为α，用户B相对显示屏圆周的角度为β，用户C相对显示屏圆周的角度为γ后，在显示屏圆周上获取对应全景图像左端的用户C的位置与音频设备中心位置O的第二连线L2，以及在显示屏圆周上获取对应全景图像右端的用户A的位置与音频设备中心位置O的第三连线L3；获取第二连线L2与第三连线L3之间的第一夹角θ，θ＝γ-α，根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定第一夹角在水平面上的第一投影角。如图5和9所示，音频设备的中心位置O与显示屏圆周之间的水平距离为显示屏圆周的半径长度，音频设备的中心位置O与显示屏圆周之间的垂直距离为OO’，第一夹角θ在水平面上的第一投影 角为θ’。
在第一投影角为θ’大于预设角度时，假设预设角度为180°，参考图3，在显示屏圆周上对应于全景图像中左右两端的用户A、C的位置与音频设备中心位置的连线在水平面上所形成的夹角大于180°，此时确定显示屏处于全屏显示模式，如图5所示，在全屏显示模式时，无需调整显示方向。
在第一投影角为θ’不大于预设角度时，同样假设预设角度为180°，参考图6和8，在显示屏圆周上对应于全景图像中左右两端的用户A、C的位置与音频设备中心位置的连线在水平面所形成的夹角小于180°，此时确定显示屏处于半屏显示模式。
在确定显示屏处于半屏显示模式后，继续执行步骤S130，即确定半屏显示区域相对用户位置的补偿角度，并调整半屏显示区域的显示方向，以使得补偿角度为零。
一个实施例中，通过获取第一夹角的角平分线和第一连线之间的第二夹角，并根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定第二夹角在水平面上的第二投影角为半屏显示区域的显示方向相对用户位置的补偿角度。
参考图8，假设当前的显示方向在X’O’Y’平面上投影为X’轴的正半轴O’X’所在方向，那么依据前文描述的方法，通过在显示屏圆周上对应于全景图像中左两端的用户C的位置与音频设备中心位置O形成的第二连线L2，以及在显示屏圆周上对应于全景图像中右两端的用户A的位置与音频设备中心位置O形成的第三连线L3，基于第二连线L2和第三连线L3形成的第一夹角θ在水平面上的第一投影角θ’小于180°的情况，确定显示屏为半屏显示时，获取第一夹角θ的角平分线和第一连线之间在平面上形成的第二夹角为半屏显示区域相对用户位置的补偿角度。相当于，在X’O’Y’平面上，获取第一投影角θ’的角平分线L4与X’轴的正半轴O’X’之间的夹角为补偿角度，根据该补偿角度，对显示屏进行水平方向的转动调整，调整后的半屏显示区域ACDF的显示方向为图9中L4方向，使得半屏显示区域的中心位置朝向用户A～C的中间位置，均衡用户A～C对显示区域的视野范围，保证全部用户的视觉体验。
当然，本实施例对显示屏的控制方法仍然适用于单用户使用的情况，如 图10-12所示，在通过分析t0时刻与t1时刻的全景图像，确定音频设备周围位置单个用户A时，根据用户A在t1时刻全景图像中的位置，以及全景图像长度与显示屏圆周长的比例关系，确定用户A相对显示屏圆周的位置。
如图13所示，假设X轴的正半轴OX为当前的显示方向，即OX对应为第一连线，在显示屏圆周上获取对应全景图像中用户A的位置与音频设备中心位置O的第五连线L5和第一连线OX形成的第三夹角a，根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定第三夹角a在水平面上的第三投影角σ为补偿角度，根据第三投影角σ对显示屏进行水平方向的转动调整，调整后的半屏显示区域MNIL的显示方向未，使该补偿角度σ为零。
本发明的一个方法提供了一种音频设备定向显示装置。
在本实施例中，音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，显示屏可相对音频设备转动，音频设备的结构可以参见前文对音频设备的相关描述。
图14为本发明实施例示出的音频设备定向显示装置的结构示意图，如图14所示，本实施例的装置包括：
检测单元141，用于响应于获取的启动信号，启动摄像头模组进行全景拍摄以检测用户位置；
判断单元142，用于根据用户位置与音频设备的中心位置之间的关系，确定显示屏是否处于半屏显示模式；
计算与调整单元143，用于在确定处于半屏显示模式时，确定半屏显示区域的显示方向相对用户位置的补偿角度，并调整半屏显示区域的显示方向，以使得补偿角度为零。
本实施例的定向显示装置利用检测单元驱动摄像头模组检测用户位置，判断单元基于用户位置与音频设备的中心位置之间的位置关系，确定显示屏是否为半屏显示，在计算与调整单元确定为半屏显示时，调整半屏显示区域的显示方向，使半屏显示区域的显示方向相对用户位置的补偿角度为零，保证多用户都处于音频设备的显示区域内，提升音频设备多人交互时的视觉体验。
本实施例中显示方向包括音频设备的中心位置与半屏显示区域的中心位置的第一连线所指示的显示位置的方向；其中，音频设备的中心位置位于音 频设备的空间中垂线上。
在一个实施例中，检测单元141用于利用摄像头模组进行连续拍照，获取以音频设备的中心位置为中心的全景图像；根据不同时刻的全景图像中前景相对背景的位置变化，确定全景图像中存在的用户；根据用户在全景图像中的位置，以及全景图像的长度与显示屏圆周长之间的预设关系，确定用户相对显示屏圆周的位置。
相应的，判断单元142，用于在显示屏圆周上获取对应全景图像左端用户的位置与音频设备中心位置的第二连线和在显示屏圆周上获取对应全景图像右端用户的位置与音频设备中心位置的第三连线；获取第二连线与第三连线之间的第一夹角，并根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定第一夹角在水平面上的第一投影角；根据第一投影角确定显示屏是否为半屏显示。判断单元182具体是在第一投影角大于预设角度时，确定显示屏为全屏显示模式；在第一投影角不大于预设角度时，确定显示屏为半屏显示模式。
计算与调整单元143用于在确定处于半屏显示模式时，获取第一夹角的角平分线和第一连线之间的第二夹角，并根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定所述第二夹角在水平面上的第二投影角为半屏显示区域相对用户位置的补偿角度。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
本发明的另一个方面提供了一种音频设备。
图15为本发明实施例示出的音频设备的结构框图，如图15所示音频设备包括：摄像头模组和呈圆周分布在音频设备侧面的显示屏，显示屏可相对音频设备转动，还包括处理器和存储有机器可执行指令的机器可读存储介质，通过读取并执行机器可读存储介质中的机器可执行指令，处理器可执行前文 描述的音频设备定向显示方法。
本实施例中，在确定显示屏为半屏显示模式时，且获得半屏显示区域的显示方向相对用户位置的补偿角度之后，通过电机驱动齿轮带动上壳转动获得的补偿角度，使调整后的半屏显示区域的显示方向相对用户位置的补偿角度为零，保证全部用户都处于半屏显示区域内，提升音频设备多用户的听觉体验。
本发明的另一个方面提供了一种系统。
本申请提供的系统可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。以软件实现为例，参照图16所示，本申请提供的系统可包括处理器1601、存储有机器可执行指令的机器可读存储介质1602。处理器1601与机器可读存储介质1602可经由系统总线1603通信。并且，通过读取并执行机器可读存储介质1602中与音频设备定向显示逻辑对应的机器可执行指令，处理器1601可执行前文描述的音频设备定向显示方法。
本发明的另一个方面提供了一种机器可读存储介质。
本发明实施例的机器可读存储介质，存储有机器可执行指令，机器可执行指令被处理器执行时以实现前文描述的音频设备定向显示方法。
需要说明的是，本发明实施例的可读存储介质，例如可以是能够包含、存储、传送、传播或传输指令的任意介质。例如，可读存储介质可以包括但不限于电、磁、光、电磁、红外或半导体系统、装置、器件或传播介质。可读存储介质的具体示例包括：磁存储装置，如磁带或硬盘(HDD)；光存储装置，如光盘(CD-ROM)；存储器，如随机存取存储器(RAM)或闪存；和/或有线/无线通信链路。
机器可读存储介质可以包括计算机程序，该计算机程序可以包括代码/计算机可执行指令，其在由处理器执行时使得处理器执行例如前文所描述的音频设备定向显示方法流程及其任何变形。
计算机程序可被配置为具有例如包括计算机程序模块的计算机程序代码。例如，在示例实施例中，计算机程序中的代码可以包括一个或多个程序模块。应当注意，模块的划分方式和个数并不是固定的，本领域技术人员可以根据实际情况使用合适的程序模块或程序模块组合，当这些程序模块组合被处理器执行时，使得处理器可以执行例如上文所描述的音频设备定向显示 方法流程及其任何变形。
为了便于清楚描述本发明实施例的技术方案，在发明的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分，本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定。
以上所述，仅为本发明的具体实施方式，在本发明的上述教导下，本领域技术人员可以在上述实施例的基础上进行其他的改进或变形。本领域技术人员应该明白，上述的具体描述只是更好的解释本发明的目的，本发明的保护范围应以权利要求的保护范围为准。

Claims (11)

  1. 一种音频设备定向显示方法，所述音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，所述显示屏可相对音频设备转动，所述方法包括：
    响应于获取的启动信号，启动所述摄像头模组进行全景拍摄以检测用户位置；
    根据用户位置与所述音频设备的中心位置之间的关系，确定所述显示屏是否处于为半屏显示模式；
    在确定处于半屏显示模式时，确定半屏显示区域的显示方向相对所述用户位置的补偿角度，并调整所述半屏显示区域的显示方向，以使得所述补偿角度为零。
  2. 根据权利要求1所述的方法，其特征在于，所述摄像头模组包括至少一个全景摄像头或至少一组双目摄像头或具有多个摄像头的摄像头球形阵列。
  3. 根据权利要求1所述的方法，其特征在于，所述显示方向包括所述音频设备的中心位置与半屏显示区域的圆弧中心位置的第一连线所指示的显示位置的方向；其中，所述音频设备的中心位置位于所述音频设备的空间中垂线上。
  4. 根据权利要求3所述的方法，其特征在于，所述启动所述摄像头模组进行全景拍摄以检测用户位置，包括：
    利用摄像头模组进行连续拍照，获取以所述音频设备的中心位置为中心的全景图像；
    根据不同时刻的全景图像中目标相对背景的位置变化，确定全景图像中所述目标为检测到的用户；
    根据用户在全景图像中的位置，以及全景图像的长度与显示屏圆周长之间的预设关系，确定用户相对显示屏圆周的位置。
  5. 根据权利要求4所述的方法，其特征在于，所述根据用户位置与所述音频设备的中心位置之间的关系，确定所述显示屏是否处于半屏显示模式，包括：
    在显示屏圆周上获取对应全景图像一端用户的位置与音频设备的中心位置的第二连线和在显示屏圆周上获取对应全景图像另一端用户的位置与音频设备中心位置的第三连线；
    获取第二连线与第三连线之间的第一夹角，并根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定所述第一夹角在水平面上的 第一投影角；
    根据所述第一投影角确定所述显示屏是否处于半屏显示模式。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述第一投影角确定所述显示屏是否处于半屏显示模式，包括：
    在所述第一投影角大于预设角度时，确定所述显示屏处于全屏显示模式；
    在所述第一投影角不大于预设角度时，确定所述显示屏处于半屏显示模式。
  7. 根据权利要求5所述的方法，其特征在于，所述在确定处于半屏显示模式时，确定所述半屏显示区域的显示方向相对所述用户位置的补偿角度，包括：
    获取第一夹角的角平分线和所述第一连线之间的第二夹角，并根据音频设备的中心位置与显示屏圆周之间的水平距离和垂直距离，确定所述第二夹角在水平面上的第二投影角为所述半屏显示区域的显示方向相对所述用户位置的补偿角度。
  8. 一种音频设备定向显示装置，所述音频设备包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，所述显示屏可相对音频设备转动，所述装置包括：
    检测单元，用于响应于获取的启动信号，启动所述摄像头模组进行全景拍摄以检测用户位置；
    判断单元，用于根据用户位置与所述音频设备的中心位置之间的关系，确定所述显示屏是否处于半屏显示模式；
    计算与调整单元，用于在确定处于半屏显示模式时，确定半屏显示区域的显示方向相对所述用户位置的补偿角度，并调整所述半屏显示区域的显示方向，以使得所述补偿角度为零。
  9. 一种音频设备，包括摄像头模组和呈圆周分布在音频设备侧面的显示屏，所述显示屏可相对音频设备转动，还包括处理器和存储有机器可执行指令的机器可读存储介质，通过读取并执行机器可读存储介质中的机器可执行指令，处理器可执行如权利要求1-7任一项所述的音频设备定向显示方法。
  10. 根据权利要求9所述的音频设备，其特征在于，可相对音频设备转动的显示屏设置于所述音频设备的上壳，所述音频设备的上壳内部设置有齿轮，所述音频设备的下壳内壁设置有与所述齿轮啮合的齿环，所述齿轮由电机驱动。
  11. 一种机器可读存储介质，存储有机器可执行指令，机器可执行指令被处理器执行时以实现如权利要求1-7任一项所述的音频设备定向显示方法。
PCT/CN2018/125232 2018-10-29 2018-12-29 一种音频设备定向显示方法、装置和音频设备 Ceased WO2020087747A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/309,136 US11910178B2 (en) 2018-10-29 2018-12-29 Orientated display method and apparatus for audio device, and audio device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811271455.1 2018-10-29
CN201811271455.1A CN109460074B (zh) 2018-10-29 2018-10-29 一种音频设备定向显示方法、装置和音频设备

Publications (1)

Publication Number Publication Date
WO2020087747A1 true WO2020087747A1 (zh) 2020-05-07

Family

ID=65608717

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/125232 Ceased WO2020087747A1 (zh) 2018-10-29 2018-12-29 一种音频设备定向显示方法、装置和音频设备

Country Status (3)

Country Link
US (1) US11910178B2 (zh)
CN (1) CN109460074B (zh)
WO (1) WO2020087747A1 (zh)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020213115A1 (ja) * 2019-04-18 2020-10-22 日本電信電話株式会社 発話画像化装置、発話画像化方法及びプログラム
CN110691016B (zh) * 2019-09-29 2021-08-31 歌尔股份有限公司 一种基于音频设备实现的交互方法及音频设备
CN110850973B (zh) * 2019-10-29 2023-10-03 歌尔股份有限公司 音频设备的控制方法、音频设备及存储介质
CN111031414B (zh) 2019-11-15 2021-06-01 歌尔股份有限公司 音频设备的控制方法、音频设备及存储介质
WO2021131862A1 (ja) * 2019-12-27 2021-07-01 ソニーグループ株式会社 情報処理装置、方法及びプログラム
CN113608641B (zh) * 2020-06-18 2024-01-16 深圳市冠旭电子股份有限公司 曲面屏显示位置的调整方法、装置、智能音箱及存储介质
CN111885342B (zh) * 2020-07-27 2022-07-15 北京小米移动软件有限公司 智能设备
CN112217975A (zh) * 2020-09-29 2021-01-12 星络智能科技有限公司 智能音箱控制方法、系统、计算机设备及计算机可读存储介质
CN114842841B (zh) * 2022-03-22 2024-01-05 国家珠宝检测中心(广东)有限责任公司 基于语音识别技术的珠宝终端机交互控制方法及其装置
CN114413141B (zh) * 2022-03-30 2022-06-24 广东中强精英电子科技有限公司 一种显示屏的位姿调整方法、装置、终端设备和存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5818950A (en) * 1994-02-25 1998-10-06 Kabushiki Kaisha Kenwood Speaker system and its support legs
CN102404545A (zh) * 2010-09-28 2012-04-04 微软公司 双向视频会议系统
CN103744474A (zh) * 2013-12-19 2014-04-23 张国训 一种智能终端与智能系统
CN106155200A (zh) * 2016-06-30 2016-11-23 联想(北京)有限公司 一种电子设备及显示方法
CN107643886A (zh) * 2017-09-29 2018-01-30 联想(北京)有限公司 一种多媒体设备及控制方法
CN108105943A (zh) * 2017-11-30 2018-06-01 珠海格力电器股份有限公司 一种空调室内机面板、控制方法、室内机和空调器
CN108614583A (zh) * 2018-04-28 2018-10-02 北京小米移动软件有限公司 智能终端及智能终端的控制方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0432376A (ja) * 1990-05-29 1992-02-04 Nec Corp 画像追尾方式
US20040008423A1 (en) * 2002-01-28 2004-01-15 Driscoll Edward C. Visual teleconferencing apparatus
US20030220971A1 (en) * 2002-05-23 2003-11-27 International Business Machines Corporation Method and apparatus for video conferencing with audio redirection within a 360 degree view
US7515174B1 (en) * 2004-12-06 2009-04-07 Dreamworks Animation L.L.C. Multi-user video conferencing with perspective correct eye-to-eye contact
EP1969193A2 (en) * 2006-01-03 2008-09-17 Csav, Inc. Motorized mount for electronic display
US8572183B2 (en) * 2006-06-26 2013-10-29 Microsoft Corp. Panoramic video in a live meeting client
CN102033549B (zh) * 2009-09-30 2014-02-05 三星电子（中国）研发中心 显示设备的视角调整装置
CN102778892A (zh) * 2011-05-12 2012-11-14 鸿富锦精密工业（深圳）有限公司 显示装置及该显示装置的转动方法
CN206894784U (zh) * 2017-04-19 2018-01-16 东莞市隆锐行电子有限公司 一种视频会议系统的一体化全无线终端
CN108551619A (zh) * 2018-04-13 2018-09-18 深圳市沃特沃德股份有限公司 智能定向音响系统及其交互方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5818950A (en) * 1994-02-25 1998-10-06 Kabushiki Kaisha Kenwood Speaker system and its support legs
CN102404545A (zh) * 2010-09-28 2012-04-04 微软公司 双向视频会议系统
CN103744474A (zh) * 2013-12-19 2014-04-23 张国训 一种智能终端与智能系统
CN106155200A (zh) * 2016-06-30 2016-11-23 联想(北京)有限公司 一种电子设备及显示方法
CN107643886A (zh) * 2017-09-29 2018-01-30 联想(北京)有限公司 一种多媒体设备及控制方法
CN108105943A (zh) * 2017-11-30 2018-06-01 珠海格力电器股份有限公司 一种空调室内机面板、控制方法、室内机和空调器
CN108614583A (zh) * 2018-04-28 2018-10-02 北京小米移动软件有限公司 智能终端及智能终端的控制方法

Also Published As

Publication number Publication date
US11910178B2 (en) 2024-02-20
US20220014867A1 (en) 2022-01-13
CN109460074B (zh) 2021-02-23
CN109460074A (zh) 2019-03-12

Similar Documents

Publication Publication Date Title
WO2020087747A1 (zh) 一种音频设备定向显示方法、装置和音频设备
CN107852485B (zh) 投影单元
CN104240606B (zh) 显示装置及显示装置观看角度的调节方法
US10158798B2 (en) Imaging apparatus and method of controlling the same
CN109460073B (zh) 一种音频设备定向显示方法、装置和音频设备
WO2020082722A1 (zh) 基于混合相机的视频成像方法、系统、设备及存储介质
JP2017518664A (ja) モバイルコンピューティングデバイスの配置及び定位を容易にするマウント
US12108224B2 (en) Method and apparatus for dynamic reduction of camera body acoustic shadowing in wind noise processing
US11062422B2 (en) Image processing apparatus, image communication system, image processing method, and recording medium
WO2021143296A1 (zh) 控制镜头运动的方法、装置、设备及存储介质
KR20170029404A (ko) 단말 및 비디오 이미지를 제어하는 장치 및 방법
CN110839128A (zh) 拍照行为检测方法、装置及存储介质
CN107925724A (zh) 在具有相机的设备中支持摄影的技术及其设备
WO2020087751A1 (zh) 一种音频设备定向显示方法、装置和音频设备
EP3472667B1 (en) System and method for capturing horizontal disparity stereo panorama
WO2022228119A1 (zh) 图像采集方法、装置、电子设备及介质
CN111325790B (zh) 目标追踪方法、设备及系统
CN205305019U (zh) 3d摄像头机构及具有该机构的移动设备
US20210067707A1 (en) Electronic device and control method for same
WO2020056658A1 (zh) 显示控制方法及显示装置
TWI755938B (zh) 影像擷取系統
CN111353934B (zh) 一种视频合成的方法及装置
US11356786B2 (en) Method and apparatus for wind noise detection and beam pattern processing
KR20230126495A (ko) 제어 대상 객체의 지도를 생성하는 방법, 장치, 비일시성의 컴퓨터 판독 가능 기록 매체
CN117641224A (zh) 电子设备管理方法、通信系统、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18938344

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18938344

Country of ref document: EP

Kind code of ref document: A1