CN108595139B - Intelligent multimedia equipment and display method thereof - Google Patents
Intelligent multimedia equipment and display method thereof Download PDFInfo
- Publication number
- CN108595139B CN108595139B CN201810431760.6A CN201810431760A CN108595139B CN 108595139 B CN108595139 B CN 108595139B CN 201810431760 A CN201810431760 A CN 201810431760A CN 108595139 B CN108595139 B CN 108595139B
- Authority
- CN
- China
- Prior art keywords
- microphone
- display area
- determining
- display
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
Abstract
The invention discloses an intelligent multimedia device and a display method thereof. An intelligent multimedia device comprises a plurality of display areas arranged on the side surface of the intelligent multimedia device and a ring-shaped microphone array arranged on the surface of the intelligent multimedia device, and the corresponding method comprises the following steps: determining a microphone receiving a sound signal in the annular microphone array as a first microphone; and determining the first display area for displaying according to the corresponding relation between the microphone and the display area. Another intelligent multimedia device comprises a dynamic display area arranged on the side surface of the intelligent multimedia device and a ring-shaped microphone array arranged on the surface of the intelligent multimedia device, wherein the corresponding method comprises the following steps: determining a microphone receiving a sound signal in the annular microphone array as a first microphone; determining the corresponding relation between the first microphone and the dynamic display area; and controlling the first sub-area of the dynamic display area to display according to the corresponding relation. The intelligent multimedia equipment can automatically adjust the visual display area in real time along with the change of the direction/position of the sound source.
Description
Technical Field
The invention relates to an intelligent multimedia device and a display method thereof.
Background
With the improvement of living standard, smart homes are more and more appeared in daily life of people, and smart multimedia devices (such as smart speakers) are popular among people as one of the products. Most of some intelligent multimedia devices in the market are provided with display screens, and the display screens can be used for displaying related information and can also realize the control of intelligent multimedia through touch display screens.
At present, the display screen of intelligent multimedia equipment is generally fixed in orientation, and can not be adjusted in real time to carry out directional display to aim at people's direction, make the user watch the display screen or when carrying out touch operation, all need remove to the opposite of display screen or manually remove equipment, lead to the human-computer interaction effect poor, influence user experience.
Disclosure of Invention
The invention provides intelligent multimedia equipment and a display method thereof, and aims to solve the problems that the display screen in the prior art is fixed in orientation and poor in human-computer interaction effect.
One aspect of the present invention provides a display method of an intelligent multimedia device, the intelligent multimedia device including a plurality of display areas disposed on a side surface thereof, and a ring-shaped microphone array disposed on a surface thereof, wherein microphones in the ring-shaped microphone array have a correspondence with the display areas, the method including: determining a microphone receiving a sound signal in the annular microphone array as a first microphone; determining a first display area corresponding to the first microphone according to the corresponding relation between the microphones and the display area; and controlling the first display area to display.
Optionally, determining that a microphone receiving the sound signal in the annular microphone array is a first microphone comprises: determining a microphone with the maximum intensity of the received sound signals in the annular microphone array as a first microphone, wherein the intensity of the sound signals is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
Optionally, the correspondence between the microphones and the display area includes a correspondence of a set position between the microphones and the display area, and determining the first display area corresponding to the first microphone includes: determining a first display area according to the corresponding relation of the setting positions and the setting position of the first microphone, wherein when the setting position of the first microphone is on the first display area, the first display area is determined; when the setting position of the first microphone is on the boundary of two adjacent display areas, one of the two adjacent display areas is randomly determined as a first display area; or determining a display area conforming to the set rule as a first display area; alternatively, no response is made.
Optionally, the first microphone includes a plurality of reference microphones, and determining a microphone of the annular microphone array that receives the sound signal as the first microphone includes: determining a plurality of microphones in the annular microphone array, wherein the intensity of the sound signals received by the plurality of microphones exceeds a preset intensity value, as the plurality of reference microphones, and the intensity of the sound signals is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
Optionally, the correspondence between the microphones and the display area includes a correspondence of a set position between the microphones and the display area, and determining the first display area corresponding to the first microphone includes: determining a first display area according to the corresponding relation of the setting positions and the setting positions of the plurality of reference microphones, wherein when the number of the setting positions of the plurality of reference microphones on the display area is different, the display area with the number of the reference microphones larger than that of the reference microphones on other display areas is determined as the first display area; when the number of the plurality of reference microphone setting positions on the display area is the same, randomly determining a first display area from the display area where the setting positions of the plurality of reference microphones are located; or determining a display area conforming to the set rule as a first display area; alternatively, no response is made.
An aspect of the present invention also provides a display method of an intelligent multimedia device, the intelligent multimedia device including a dynamic display area disposed at a side thereof and a ring-shaped microphone array disposed at a surface thereof, including: determining a microphone receiving a sound signal in the annular microphone array as a first microphone; determining a corresponding relation between the first microphone and the display area; and controlling the first sub-area of the display area to perform dynamic display according to the corresponding relation.
Optionally, determining that a microphone receiving the sound signal in the annular microphone array is a first microphone comprises: determining a microphone with the maximum intensity of sound signals received in an annular microphone array as a first microphone, wherein the intensity of the sound signals is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
Optionally, determining a correspondence between the first microphone and the display area includes: and determining a first sub-area of the display area according to the setting position relation between the first microphone and the display area, wherein the setting position of the first microphone is on the central line of the first sub-area, and the first sub-area is a cone bottom surface which is set by taking the central point of the intelligent multimedia device body as a vertex.
Another aspect of the present invention provides an intelligent multimedia device, including a plurality of display areas disposed on a side surface thereof, and a ring-shaped microphone array disposed on a surface thereof, wherein microphones in the ring-shaped microphone array have a corresponding relationship with the display areas, and the intelligent multimedia device further includes: the first signal processing unit is used for determining a microphone receiving a sound signal in the annular microphone array as a first microphone; the first analysis processing unit is used for determining a first display area corresponding to the first microphone according to the corresponding relation between the microphones and the display area; and the first screen display control unit is used for controlling the first display area to display.
Another aspect of the present invention also provides an intelligent multimedia device, including a dynamic display area disposed at a side thereof, and a ring-shaped microphone array disposed at a surface thereof, further including: the second signal processing unit is used for determining a microphone which receives a sound signal in the annular microphone array as a first microphone; the second analysis processing unit is used for determining the corresponding relation between the first microphone and the dynamic display area; and the second screen display control unit is used for controlling the first sub-area of the dynamic display area to display according to the corresponding relation.
Yet another aspect of the present invention also provides an intelligent multimedia system, including: one or more memory units storing executable instructions; and the one or more processing units execute executable instructions for realizing the display method of the intelligent multimedia device.
The invention provides an intelligent multimedia device and a display method thereof, wherein the intelligent multimedia device with a plurality of display areas is preset with a corresponding relation between each display area and a corresponding microphone, when the visual display area of the intelligent multimedia device is adjusted, a first microphone capable of identifying the position of a sound source is determined according to the signal characteristics of sound signals received by each microphone in a microphone array, so that the first display area corresponding to the first microphone is determined according to the corresponding relation between the microphones and the display areas for displaying, the intelligent directional display of the intelligent multimedia device is realized, the visual display area can be automatically adjusted in real time along with the change of the direction/position of the sound source, the human-computer interaction effect is enhanced, and the user experience is improved.
The invention provides an intelligent multimedia device and a display method thereof, wherein for the intelligent multimedia device with a dynamic display area, the dynamic display area can be dynamically divided into a plurality of sub-areas, when the visual display area of the intelligent multimedia is adjusted, the first sub-area corresponding to the first microphone is dynamically determined on the dynamic display area of the intelligent multimedia device by determining the corresponding relation between the first microphone and the first sub-area, and the first sub-area is controlled to be displayed, so that the intelligent directional display of the intelligent multimedia device is realized, the visual display area can be dynamically adjusted in real time along with the change of the direction/position of a sound source, the man-machine interaction effect is enhanced, and the user experience is improved.
Drawings
Fig. 1 is a schematic perspective view of an intelligent multimedia device according to an embodiment of the present invention;
FIG. 2 is an exploded view of an embodiment of the present invention;
fig. 3 is a schematic cross-sectional view of an intelligent multimedia according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a plurality of display areas of a prism-sided smart multimedia device according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating a display method of an intelligent multimedia device according to an embodiment of the present invention;
fig. 6 is a schematic diagram illustrating a determination manner of the first display area according to an embodiment of the present invention;
FIG. 7 is a flow chart illustrating a display method of another intelligent multimedia device according to an embodiment of the present invention;
FIG. 8 is a diagram illustrating mapping of a first sub-area on a dynamic display area according to an embodiment of the invention;
fig. 9 is a schematic diagram illustrating a correspondence relationship between a first microphone and a first sub-area according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating the dynamic variation of the first sub-area with the variation of the first microphone according to an embodiment of the present invention;
fig. 11 is a block diagram illustrating an intelligent multimedia device according to an embodiment of the present invention;
fig. 12 is a block diagram illustrating another structure of an intelligent multimedia device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It is to be understood that such description is merely illustrative and not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The words "a", "an" and "the" and the like as used herein are also intended to include the meanings of "a plurality" unless the context clearly indicates otherwise. Furthermore, the terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Some block diagrams and/or flow diagrams are shown in the figures. It will be understood that some blocks of the block diagrams and/or flowchart illustrations, or combinations thereof, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the instructions, which execute via the processor, create means for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks.
Thus, the techniques of the present invention may be implemented in hardware and/or in software (including firmware, microcode, etc.). Furthermore, the techniques of this disclosure may take the form of a computer program product on a computer-readable medium having instructions stored thereon for use by or in connection with an instruction execution system. In the context of the present invention, a computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the instructions. The computer readable medium can include, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Specific examples of the computer readable medium include: magnetic storage devices, such as magnetic tape or Hard Disk Drives (HDDs); optical storage devices, such as compact disks (CD-ROMs); a memory, such as a Random Access Memory (RAM) or a flash memory; and/or wired/wireless communication links.
In order to improve the human-computer interaction effect and break through the limitation of fixed orientation to the manual operation range, the invention sets a plurality of display areas on the side surface of the intelligent multimedia device, when the side surface of the intelligent multimedia is an annular side surface, the plurality of display areas can be set on the annular side surface, when the side surface of the intelligent multimedia is composed of polyhedral surfaces, a display area can be set on each polyhedral surface, the corresponding relation between a microphone and the display areas is set, a first microphone capable of identifying the position of a sound source is determined by analyzing the characteristics of picked sound signals, the display area corresponding to the first microphone is controlled to display pictures, and other display areas do not display pictures, so that the energy is saved, and the visual position of the display screen is adjustable.
To facilitate the description of the display scheme of the intelligent multimedia in this embodiment, the present invention first describes the structure of the intelligent multimedia device through the following embodiments, where the intelligent multimedia in this embodiment includes, but is not limited to, an intelligent speaker, an intelligent television, and other devices.
Fig. 1 is a schematic perspective view of an intelligent multimedia device according to an embodiment of the present invention, fig. 2 is a schematic exploded view of the intelligent multimedia device according to the embodiment of the present invention, fig. 3 is a schematic cross-sectional view of the intelligent multimedia device according to the embodiment of the present invention, as shown in fig. 1 to 3, the intelligent multimedia device according to the embodiment of the present invention includes: the loudspeaker comprises an upper shell 1, a shell 7, a first PCB 2, a display screen 3, a second PCB 4, a microphone array and a loudspeaker module 6;
the upper shell 1 is buckled on the shell 7 and forms an intelligent multimedia shell together with the shell 7, the first PCB 2 is fixed on the lower side of the upper shell 1, the microphone array is arranged between the first PCB 2 and the upper shell 1, a microphone hole a corresponding to the position of a microphone b in the microphone array is arranged on the upper shell 1, and the microphone b in the microphone array picks up a sound signal of an external sound source through the microphone hole a.
The second PCB 4 of this embodiment may integrate components corresponding to each function, for example, an audio processor, a microprocessor, a screen control module, etc. are disposed on the second PCB 4, the audio processor is used to perform noise reduction, dereverberation, etc. on the sound signals picked up by the microphones, and transmit the processed sound signals to the microprocessor, the microprocessor determines the intensity values of the sound signals picked up by the microphones to determine the sound source direction according to the intensity values, and drives the screen control module to display the display area corresponding to the sound source direction. It can be understood that, in this embodiment, corresponding components may be integrated on the second PCB 4 according to the function of the intelligent multimedia device, and the microprocessor is configured to perform logic control, for example, when the intelligent multimedia device supports a conference working mode and a general working mode, when the microprocessor receives an instruction to enter the conference working mode, the screen control module may be driven to display each display area corresponding to a plurality of sound source directions, and when the microprocessor receives an instruction to enter the general working mode, the screen control module may be driven to control the display screen to perform directional display.
It should be noted that fig. 1 to 3 of the present embodiment only exemplarily show an intelligent multimedia device having a cylindrical side surface, and the display screen includes a circumference of the cylindrical side surface, in this case, the display screen may be divided into a plurality of display areas, and the display is selected to display according to the sound source position. The intelligent multimedia device of this embodiment may also be a prism side surface, and as shown in fig. 4, a display screen may be disposed on each side surface of the prism, where the intelligent multimedia device includes five display regions.
In the embodiment, the setting positions of the microphones in the microphone array correspond to the display area of the intelligent multimedia device. Thus, the present embodiment converts the determination of the display area for human-computer interaction from the sound source position to the determination of the display area for human-computer interaction from the position of the first microphone capable of identifying the sound source position. The present invention specifically describes a display method of an intelligent multimedia device by the following embodiments.
One aspect of the present invention provides a display method of an intelligent multimedia device.
Fig. 5 is a flowchart of a display method of an intelligent multimedia device according to an embodiment of the present invention, where the intelligent multimedia device includes a plurality of display areas disposed on a side surface thereof and an annular microphone array disposed on a surface thereof, and in this embodiment, there is a corresponding relationship between microphones in the annular microphone array and the display areas, for example, a corresponding relationship between arrangement positions of the microphones and the plurality of display areas. As shown in fig. 5, the method of the present embodiment includes:
and S510, determining a microphone receiving the sound signal in the annular microphone array as a first microphone.
In the embodiment, based on the characteristics that the strength of a signal of a sound signal received by a microphone is related to the position/direction of the sound source, and the strength of a sound signal picked up by a microphone which is directly opposite to the sound source and is close to the position of the sound source is high, the sound signal of the sound source picked up by the microphone in an annular microphone array of the intelligent multimedia device is utilized, a first microphone capable of representing the position/direction of the sound source is determined from a plurality of microphones picking up the sound signal, and a display area for man-machine interaction corresponding to the first microphone is determined according to the corresponding relation between the microphone and the plurality of display areas.
S520, determining a first display area corresponding to the first microphone according to the corresponding relation between the microphones and the display area.
In this embodiment, when the intelligent multimedia device is initialized, a corresponding relationship between the microphone and the plurality of display areas is set, for example, a corresponding relationship between a set position of the microphone and a set position of the plurality of display areas. According to the embodiment, when the intelligent multimedia device is initialized, the corresponding relation between the plurality of display areas and the microphones is initialized according to the positions of the display areas of the intelligent multimedia device and the setting positions of the microphones in the annular microphone array, so that after the first microphone is determined, the first display area corresponding to the first microphone can be determined according to the corresponding relation.
And S530, controlling the first display area to display.
In the embodiment, a plurality of display areas set by the intelligent multimedia are preset, each display area is in corresponding relation with the corresponding microphone, when the visual display area of the intelligent multimedia is adjusted, the first microphone capable of identifying the sound source position is determined according to the signal characteristics of the sound signals received by the microphones in the microphone array, the first display area corresponding to the first microphone is determined according to the corresponding relation between the microphones and the display areas to display, intelligent directional display of the intelligent multimedia equipment is realized, the visual display area can be automatically adjusted in real time along with the change of the sound source direction/position, the man-machine interaction effect is enhanced, and the user experience is improved.
According to the embodiment of the invention, the correspondence between the microphones and the display area in the annular microphone array includes the correspondence between the setting positions of the microphones and the display area. In the embodiment, after the sound signals of the sound source are picked up by using each microphone in the annular microphone array, and the microphone with the maximum intensity of the received sound signals in the annular microphone array is determined as the first microphone according to the characteristics of the sound signals, the setting position of the first microphone is acquired, so that the first microphone is determined to correspond to the first display area according to the corresponding relationship of the setting positions between the microphones and the display area and the setting position of the first microphone:
when the set position of the first microphone is on the first display area, the first display area is determined. When the setting position of the first microphone is on the boundary of the two adjacent display areas, one of the two adjacent display areas is randomly determined as the first display area, or the display area conforming to the setting rule is determined as the first display area, or no response is made. The setting rule in the present embodiment includes, but is not limited to, determining the first display area in the clockwise/counterclockwise direction of the reference position, which may be the setting position of the first microphone.
According to the embodiment of the present invention, the first display area is determined from the plurality of display areas of the intelligent multimedia device with the cylindrical side surface, and the determination of the first display area from the plurality of display areas of the intelligent multimedia device with the other shaped side surfaces is the same as the method of the embodiment, and is not repeated here.
Fig. 6 is a schematic diagram illustrating a determination method of a first display area according to an embodiment of the present invention, and referring to fig. 6, it is assumed that an intelligent multimedia device includes four display areas, each display area is provided with 8 microphones, and the microphones are represented by dots in fig. 6. As shown in fig. 6, this embodiment assumes that five microphones a7, B0, B1, B2, and B3 represented by black solid dots in the smart multimedia device can pick up a sound signal to a sound source, and when the intensity of the sound signal picked up by the microphone B1 is the maximum, the display area 2 is determined as the first display area, when the intensity of the sound signal picked up by the microphone a7 is the maximum, the display area 1 is determined as the first display area, and when the intensity of the sound signal picked up by the microphone B0 is the maximum, the display area 1 or the display area 2 is determined as the first display area, and the first display area may be determined according to a set rule, for example, when the first microphone B0 is located at the boundary of an adjacent display area, the display area 2 in the clockwise direction thereof is determined as the first display area, or the sound pickup may be re-entered without response.
According to the embodiment of the invention, the first microphone is determined according to the sound signal picked up by the microphone, and the first display area corresponding to the first microphone is determined according to the corresponding relation between the microphone and the display area, so that the method and the device are simple and rapid, and the visual display area of the intelligent multimedia device can be adjusted in real time conveniently.
The embodiment of the invention can also determine a plurality of reference microphones included in the first microphone according to the sound signals picked up by the microphones, and then determine the first display area according to the distribution conditions of the plurality of reference microphones in the display area.
According to the embodiment of the invention, the correspondence between the microphones and the display area in the annular microphone array includes the correspondence between the setting positions of the microphones and the display area. In the embodiment, after the sound signal of the sound source is picked up by using the microphones in the annular microphone array, the plurality of reference microphones with the intensity exceeding the preset intensity value are determined in the annular microphone array according to the characteristics of the sound signal, the setting positions of the plurality of reference microphones are obtained, and the first display area is determined according to the corresponding relationship between the microphones and the display area and the setting positions of the plurality of reference microphones:
when the number of the setting positions of the plurality of reference microphones on the display area is different, the display area in which the number of the reference microphones is set to be larger than the number of the reference microphones on the other display areas is determined as the first display area. Assuming that when the arrangement positions of the plurality of reference microphones are distributed in three display areas, the number of reference microphones in the three display areas is different, one of the three display areas in which the number of reference microphones is the largest is determined as the first display area.
When the number of the setting positions of the plurality of reference microphones on the display area is the same, randomly determining the first display area from the display area where the setting positions of the plurality of reference microphones are located, or determining the display area conforming to the setting rule as the first display area, or not responding. The setting rule in this embodiment includes, but is not limited to, setting the display area where the reference microphone with the maximum picked-up sound signal intensity is located as the first display area, or determining the first display area in the clockwise/counterclockwise direction of the reference position, where the reference position may be the setting position of the setting microphone.
Assuming that when the setting positions of the plurality of reference microphones are distributed in three display areas, the number of reference microphones in the three display areas is the same, any one of the three display areas is randomly determined as the first display area. When the number of the reference microphones in the three display areas is the same, the first display area may also be determined according to a set rule, for example, the display area where the reference microphone with the largest picked-up sound signal intensity is located is the first display area. The sound pickup may be re-entered without response when the number of reference microphones is the same in the three display areas.
Referring to fig. 6, in the present embodiment, it is assumed that the intensity of the sound signals picked up by the five microphones a7, B0, B1, B2 and B3 represented by black solid dots in the intelligent multimedia device exceeds the preset intensity value, and the five microphones are reference microphones, and as can be seen from fig. 6, only the microphone a7 is located in the display area 1, and four microphones B0, B1, B2 and B3 are located in the display area 2, at this time, the present embodiment determines that the display area 2 is the first display area.
The strength of the sound signal determined by the embodiment is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time, so that the determined first microphone can accurately represent the direction/position of the sound source.
The above embodiment of the present invention provides two schemes for determining the first display area, wherein the first scheme is simple and fast to determine the first display area according to the intensity of the sound signal picked up by the microphone; the second scheme determines a first display area according to the distribution of the reference microphones and the intensity of the sound signals picked up by the reference microphones, and the determined first display area can better represent the sound source position. The two schemes of the embodiment have advantages respectively, and a suitable scheme can be selected according to use requirements.
Fig. 7 is a flowchart illustrating another display method of an intelligent multimedia device according to an embodiment of the present invention, where the intelligent multimedia device includes a dynamic display area disposed on a side surface of the intelligent multimedia device, and the dynamic display area may be dynamically divided into a plurality of sub-areas. As shown in fig. 7, the method of the present embodiment includes:
and S710, determining a microphone receiving the sound signal in the annular microphone array as a first microphone.
In the embodiment, based on the characteristic that the strength of a signal of a sound signal received by a microphone is related to the position/direction of the sound source, and the strength of a sound signal picked up by the microphone which is opposite to the sound source and is close to the position of the sound source is high, the sound signal of the sound source is picked up by the microphone in the annular microphone array of the intelligent multimedia device, and a first microphone capable of representing the position/direction of the sound source is determined from a plurality of microphones picking up the sound signal, so that a first display area for man-machine interaction is determined according to the setting position of the first microphone.
S720, determining the corresponding relation between the first microphone and the dynamic display area.
The corresponding relationship between the first microphone and the dynamic display area in this embodiment includes, but is not limited to, the setting position of the first microphone corresponds to a center line of a first sub-area of the dynamic display area, and a vertex angle enclosed by a boundary line of the first sub-area and a body center point of the intelligent multimedia device has a set angle.
And S730, controlling the first sub-area of the display area to display according to the corresponding relation.
In this embodiment, the first sub-area corresponding to the first microphone may be mapped on the display area according to the corresponding relationship between the first microphone and the dynamic display area, so as to control the first sub-area to perform human-computer interaction display.
According to the method, the first microphone capable of identifying the sound source position is determined according to the signal characteristics of sound signals received by all microphones in the microphone array, the first sub-area corresponding to the first microphone is dynamically determined on the dynamic display area of the intelligent multimedia device by determining the corresponding relation between the first microphone and the first sub-area, the first sub-area is controlled to display, intelligent directional display of the intelligent multimedia device is achieved, the visual display area can be automatically adjusted in real time along with the change of the sound source direction/position, the man-machine interaction effect is enhanced, and user experience is improved.
In the embodiment, the sound signals of a sound source are picked up by using the microphones in the annular microphone array, the microphone with the maximum sound signal intensity received by the annular microphone array is determined as the first microphone, the first sub-area of the dynamic display area is determined according to the setting position relation between the first microphone and the dynamic display area, the setting position of the first microphone is on the central line of the first sub-area, and the first sub-area is the bottom surface of the cone with the central point of the intelligent multimedia device body as the vertex.
According to the embodiment of the present invention, the first sub-region is determined from the dynamic display region of the intelligent multimedia device having the cylindrical side surface, and the determination of the first sub-region from the dynamic display region of the intelligent multimedia device having the other shaped side surfaces is the same as the method of the embodiment, which is not repeated herein.
Fig. 8 is a schematic diagram illustrating mapping of a first sub-area on a dynamic display area according to an embodiment of the present invention, and fig. 9 is a schematic diagram illustrating a corresponding relationship between a first microphone and the first sub-area according to the embodiment of the present invention, and as shown in fig. 8 to 9, in this embodiment, a microphone a is determined as the first microphone according to intensity of a sound signal picked up by the microphone, a straight line of a setting position of the first microphone on the dynamic display area is determined as AF according to a setting position relationship between the first microphone and the dynamic display area, the straight line AF is taken as a center line of the first sub-area, the first sub-area is a bottom surface BCDE of a cone O-BCDE taking a body center point O of the smart multimedia device as a vertex, and an arc degree corresponding to the arc BC on the bottom surface BCDE is α.
In this embodiment, when the first sub-area is determined, if the dynamic display area is a cylindrical surface or a spherical surface, the first area may be set to be an arc surface with a set radian, and if the dynamic display area is a plane, the first area may be set to be a plane with a set length or width.
Fig. 10 is a schematic diagram illustrating that the first sub-area dynamically changes with the change of the first microphone according to an embodiment of the present invention, in this embodiment, the microphone a is determined to be the first microphone according to the intensity of the sound signal picked up by the microphone at the first time, and the first sub-area corresponding to the microphone a can be determined to be the position of the microphone BAC according to the method for determining the first sub-area described above. In this embodiment, the microphone a ' is determined as the first microphone according to the intensity of the sound signal picked up by the microphone at the second moment, and the first sub-region corresponding to the microphone a ' can be determined as the position of the microphone B ' a ' C ' according to the above-described method for determining the first sub-region, so that it can be seen that the first sub-region can be dynamically adjusted based on the determined position of the first microphone in the implementation, so that the visual display region of the intelligent multimedia device can be dynamically adjusted in real time based on the direction of the sound source.
Another aspect of the present invention provides an intelligent multimedia device.
Fig. 11 is a block diagram of an intelligent multimedia device according to an embodiment of the present invention, where the intelligent multimedia device includes a plurality of display areas disposed on a side surface thereof, and a ring-shaped microphone array disposed on a surface thereof, and there is a corresponding relationship between microphones in the ring-shaped microphone array and the display areas. As shown in fig. 11, the intelligent multimedia device of this embodiment further includes:
a first signal processing unit 111 for determining a microphone receiving a sound signal in the annular microphone array as a first microphone;
a first analysis processing unit 112, configured to determine, according to a correspondence between the microphones and the display areas, a first display area corresponding to the first microphone;
the first screen display control unit 113 is configured to control the first display area to display.
In this embodiment, the first signal processing unit 111 is specifically configured to determine that a microphone with the largest intensity of the sound signals received by the annular microphone array is the first microphone, and the intensity of the sound signals is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
In this embodiment, the correspondence between the microphones and the display area includes a correspondence between setting positions of the microphones and the display area, and the first analysis processing unit 112 is configured to determine the first display area according to the correspondence between the setting positions and the setting position of the first microphone; wherein when the setting position of the first microphone is on the first display area, the first display area is determined; when the setting position of the first microphone is on the boundary of two adjacent display areas, one of the two adjacent display areas is randomly determined as a first display area; or determining a display area conforming to the set rule as a first display area; alternatively, no response is made.
The first microphone of the embodiment includes a plurality of reference microphones, and the first signal processing unit 111 is further configured to determine, as the plurality of reference microphones, a plurality of microphones in the annular microphone array that receive sound signals with intensities exceeding a preset intensity value, where the intensities of the sound signals are an average of amplitudes of sound signals picked up by the microphones in the annular microphone array within a preset time.
The correspondence between the microphones and the multiple display areas of the present embodiment includes a correspondence between setting positions of the microphones and the display areas, and the first analysis processing unit 112 is configured to determine the first display area according to the correspondence between the setting positions and the setting positions of the multiple reference microphones; when the number of the reference microphone setting positions on the display area is different, determining that the display area with the number of the reference microphones larger than that of the reference microphones on the other display areas is set as a first display area; when the number of the setting positions of the plurality of reference microphones on the display area is the same, randomly determining a first display area from the display area where the setting positions of the plurality of reference microphones are located; or determining a display area conforming to the set rule as a first display area; alternatively, no response is made.
The specific working modes of the units of the device embodiment of the invention can be seen in the method embodiment of the invention.
Fig. 12 is a block diagram illustrating another intelligent multimedia device according to an embodiment of the present invention, where the intelligent multimedia device includes a dynamic display area disposed on a side surface thereof and a ring-shaped microphone array disposed on a surface thereof. As shown in fig. 12, the intelligent multimedia device of the present embodiment further includes:
a second signal processing unit 121, configured to determine that a microphone receiving a sound signal in the annular microphone array is a first microphone;
a second analysis processing unit 122, configured to determine a correspondence between the first microphone and the dynamic display area;
and the second screen display control unit 123 is configured to control the first sub-area of the dynamic display area to display according to the corresponding relationship.
The second signal processing unit 121 of this embodiment is specifically configured to determine that a microphone in the annular microphone array that receives the largest sound signal intensity is the first microphone, and the sound signal intensity is an average value of sound signal amplitudes picked up by microphones in the annular microphone array within a preset time.
The second analysis processing unit 122 is specifically configured to determine a first sub-region of the dynamic display region according to a setting position relationship between the first microphone and the dynamic display region, where the setting position of the first microphone is on a central line of the first sub-region, and the first sub-region is a bottom surface of a cone that is set with a central point of the intelligent multimedia device body as a vertex.
Another aspect of the invention provides an intelligent multimedia system.
According to an embodiment of the present invention, an intelligent multimedia system according to an embodiment of the present invention includes: one or more memory units storing executable instructions; one or more processing units executing executable instructions for implementing the display method of the intelligent multimedia device as described above.
Another aspect of the invention provides a readable storage medium.
According to the embodiment of the present invention, the readable storage medium of the embodiment of the present invention stores executable instructions, and the executable instructions are executed by the processor to implement the display method of the intelligent multimedia device described above.
It is noted that the readable storage medium of the embodiments of the present invention may be, for example, any medium that can contain, store, communicate, propagate, or transport the instructions. For example, a readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Specific examples of the readable storage medium include: magnetic storage devices, such as magnetic tape or Hard Disk Drives (HDDs); optical storage devices, such as compact disks (CD-ROMs); a memory, such as a Random Access Memory (RAM) or a flash memory; and/or wired/wireless communication links.
The readable storage medium may comprise a computer program that may comprise code/computer executable instructions that, when executed by a processor, cause the processor to perform a method flow such as that described above and any variations thereof.
The computer program may be configured with computer program code, for example comprising computer program modules. For example, in an example embodiment, code in the computer program may include one or more program modules. It should be noted that the division and number of modules are not fixed, and those skilled in the art may use suitable program modules or program module combinations according to actual situations, which when executed by a processor, enable the processor to execute, for example, the above-described method flows and any variants thereof.
For the convenience of clearly describing the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second", and the like are used to distinguish the same items or similar items with basically the same functions and actions, and those skilled in the art can understand that the words "first", "second", and the like do not limit the quantity and execution order.
While the foregoing is directed to embodiments of the present invention, other modifications and variations of the present invention may be devised by those skilled in the art in light of the above teachings. It should be understood by those skilled in the art that the foregoing detailed description is for the purpose of better explaining the present invention, and the scope of the present invention should be determined by the scope of the claims.
Claims (9)
1. A display method of an intelligent multimedia device, the intelligent multimedia device including a plurality of display areas provided at a side surface thereof, and a ring-shaped microphone array provided at a surface thereof, microphones of the ring-shaped microphone array having a corresponding relationship with arrangement positions between the display areas, the method comprising:
determining a microphone in the annular microphone array, which receives a sound signal, as a first microphone; the first microphone comprises a plurality of reference microphones;
acquiring a setting position of a first microphone, and determining a first display area corresponding to the first microphone according to the corresponding relation of the setting positions between the microphone and the display area and the setting position of the first microphone;
controlling the first display area to display;
the determining a first display area corresponding to the first microphone includes:
determining a first display area according to the corresponding relation of the setting positions and the setting positions of the plurality of reference microphones,
wherein the content of the first and second substances,
when the number of the reference microphone setting positions on the display area is different, determining that the display area with the reference microphone number larger than that on other display areas is set as a first display area;
when the number of the plurality of reference microphone setting positions on the display area is the same,
randomly determining a first display area from the display areas where the setting positions of the plurality of reference microphones are located; alternatively, the first and second liquid crystal display panels may be,
determining a display area conforming to the set rule as a first display area; alternatively, the first and second liquid crystal display panels may be,
no response is made.
2. The method of claim 1, wherein determining that a microphone of the annular microphone array that receives a sound signal is a first microphone comprises:
determining a microphone with the maximum intensity of the sound signals received by the annular microphone array as a first microphone, wherein the intensity of the sound signals is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
3. The method of claim 2, wherein determining the first display area corresponding to the first microphone comprises:
determining a first display area according to the corresponding relation of the setting positions and the setting position of the first microphone,
wherein the content of the first and second substances,
determining the first display area when the setting position of the first microphone is on the first display area;
when the first microphone is disposed at the boundary between two adjacent display areas,
randomly determining one of two adjacent display areas as a first display area; alternatively, the first and second electrodes may be,
determining a display area conforming to a set rule as a first display area; alternatively, the first and second electrodes may be,
no response is made.
4. The method of claim 1, wherein determining that a microphone of the annular microphone array that receives a sound signal is a first microphone comprises:
determining a plurality of microphones in the annular microphone array, wherein the intensities of the sound signals received by the plurality of microphones exceed a preset intensity value, and the intensities of the sound signals are an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
5. A display method of an intelligent multimedia device including a dynamic display area disposed at a side thereof and a circular microphone array disposed at a surface thereof, the method comprising:
determining a microphone receiving a sound signal in the annular microphone array as a first microphone; the first microphone comprises a plurality of reference microphones;
determining a set positional relationship between the first microphone and a dynamic display area;
controlling a first sub-area of the dynamic display area to display according to the setting position relation and the setting position of the first microphone;
the determining a first sub-region corresponding to the first microphone comprises:
determining a first sub-area according to the corresponding relation of the setting positions and the setting positions of the plurality of reference microphones,
wherein the content of the first and second substances,
when the number of the reference microphone setting positions on the sub-area is different, determining that the display area with the reference microphone number larger than that on other display areas is set as a first sub-area;
when the number of the plurality of reference microphone setting positions on the display area is the same,
randomly determining a first sub-area from a display area where the setting positions of the plurality of reference microphones are located; alternatively, the first and second electrodes may be,
determining a display area conforming to a set rule as a first sub-area; alternatively, the first and second electrodes may be,
no response is made.
6. The method of claim 5, wherein determining that a microphone of the annular microphone array that receives the sound signal is a first microphone comprises:
determining a microphone with the maximum intensity of the sound signals received by the annular microphone array as a first microphone, wherein the intensity of the sound signals is an average value of the amplitudes of the sound signals picked up by the microphones in the annular microphone array within a preset time.
7. The method of claim 6, wherein determining a set-up position relationship between the first microphone and the dynamic display area comprises:
and determining a first sub-area of the dynamic display area according to the setting position relation between the first microphone and the dynamic display area, wherein the setting position of the first microphone is on the central line of the first sub-area, and the first sub-area is a cone bottom surface which is set with the central point of the intelligent multimedia device body as a vertex.
8. An intelligent multimedia device, the intelligent multimedia device including a plurality of display areas provided on a side surface thereof, and a ring-shaped microphone array provided on a surface thereof, the arrangement positions between the microphones in the ring-shaped microphone array and the display areas having a correspondence, further comprising:
a first signal processing unit, configured to determine a microphone of the annular microphone array that receives a sound signal as a first microphone; the first microphone comprises a plurality of reference microphones;
the first analysis processing unit is used for acquiring the setting position of the first microphone and determining a first display area corresponding to the first microphone according to the corresponding relationship of the setting positions between the microphone and the display area and the setting position of the first microphone;
the first screen display control unit is used for controlling the first display area to display;
the determining a first display area corresponding to the first microphone includes:
determining a first display area according to the corresponding relation of the setting positions and the setting positions of the plurality of reference microphones,
wherein the content of the first and second substances,
when the number of the reference microphone setting positions on the display area is different, determining that the display area with the reference microphone number larger than that on other display areas is set as a first display area;
when the number of the plurality of reference microphone setting positions on the display area is the same,
randomly determining a first display area from the display areas where the setting positions of the plurality of reference microphones are located; alternatively, the first and second liquid crystal display panels may be,
determining a display area conforming to the set rule as a first display area; alternatively, the first and second electrodes may be,
no response is made.
9. An intelligent multimedia device, the intelligent multimedia device including a dynamic display area disposed at a side thereof, and an annular microphone array disposed at a surface thereof, further comprising:
a second signal processing unit for determining a microphone receiving a sound signal in the annular microphone array as a first microphone; the first microphone comprises a plurality of reference microphones;
the second analysis processing unit is used for determining a setting position relation between the first microphone and the dynamic display area;
the second screen display control unit is used for controlling the first sub-area of the dynamic display area to display according to the setting position relation and the setting position of the first microphone;
the determining a first sub-region corresponding to the first microphone comprises:
determining a first sub-area according to the corresponding relation of the setting positions and the setting positions of the plurality of reference microphones,
wherein the content of the first and second substances,
when the number of the reference microphone setting positions on the sub-area is different, determining that a display area with the reference microphone number larger than that on other display areas is set as a first sub-area;
when the number of the plurality of reference microphone setting positions on the display area is the same,
randomly determining a first sub-area from a display area where the setting positions of the plurality of reference microphones are located; alternatively, the first and second electrodes may be,
determining a display area conforming to a set rule as a first sub-area; alternatively, the first and second liquid crystal display panels may be,
no response is made.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810431760.6A CN108595139B (en) | 2018-05-08 | 2018-05-08 | Intelligent multimedia equipment and display method thereof |
| PCT/CN2018/118566 WO2019214216A1 (en) | 2018-05-08 | 2018-11-30 | Smart multimedia device and display method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810431760.6A CN108595139B (en) | 2018-05-08 | 2018-05-08 | Intelligent multimedia equipment and display method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108595139A CN108595139A (en) | 2018-09-28 |
| CN108595139B true CN108595139B (en) | 2022-09-23 |
Family
ID=63636038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810431760.6A Active CN108595139B (en) | 2018-05-08 | 2018-05-08 | Intelligent multimedia equipment and display method thereof |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108595139B (en) |
| WO (1) | WO2019214216A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108595139B (en) * | 2018-05-08 | 2022-09-23 | 歌尔科技有限公司 | Intelligent multimedia equipment and display method thereof |
| CN113168298A (en) * | 2018-12-27 | 2021-07-23 | 深圳市柔宇科技股份有限公司 | Control method and display terminal |
| CN111857619A (en) * | 2019-04-26 | 2020-10-30 | 深圳市冠旭电子股份有限公司 | Display control method and display control device for sound box screen and sound box |
| CN112309380B (en) * | 2019-07-26 | 2024-02-06 | 北京新能源汽车股份有限公司 | Voice control method, system, equipment and automobile |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015082689A (en) * | 2013-10-21 | 2015-04-27 | 株式会社日本自動車部品総合研究所 | Acoustic system and ultrasonic filter |
| CN105229570A (en) * | 2013-03-14 | 2016-01-06 | 三星电子株式会社 | Display control method and device |
| CN106066758A (en) * | 2015-04-21 | 2016-11-02 | 三星电子株式会社 | Electronic equipment and control method thereof for display screen |
| CN107222798A (en) * | 2017-07-19 | 2017-09-29 | 歌尔科技有限公司 | A kind of intelligent sound box display device |
| CN107643886A (en) * | 2017-09-29 | 2018-01-30 | 联想(北京)有限公司 | A kind of multimedia equipment and control method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102820022B (en) * | 2011-06-09 | 2014-12-24 | 宏碁股份有限公司 | Display device and operation method thereof |
| TWI544807B (en) * | 2014-07-18 | 2016-08-01 | 緯創資通股份有限公司 | Displayer device having speaker module |
| CN107636696B (en) * | 2016-06-16 | 2021-04-06 | 深圳市柔宇科技股份有限公司 | Multi-user interaction method and device and accompanying robot |
| CN107589760B (en) * | 2017-08-31 | 2024-02-20 | 重庆海尔制冷电器有限公司 | Rotary control console |
| CN108595139B (en) * | 2018-05-08 | 2022-09-23 | 歌尔科技有限公司 | Intelligent multimedia equipment and display method thereof |
-
2018
- 2018-05-08 CN CN201810431760.6A patent/CN108595139B/en active Active
- 2018-11-30 WO PCT/CN2018/118566 patent/WO2019214216A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105229570A (en) * | 2013-03-14 | 2016-01-06 | 三星电子株式会社 | Display control method and device |
| JP2015082689A (en) * | 2013-10-21 | 2015-04-27 | 株式会社日本自動車部品総合研究所 | Acoustic system and ultrasonic filter |
| CN106066758A (en) * | 2015-04-21 | 2016-11-02 | 三星电子株式会社 | Electronic equipment and control method thereof for display screen |
| CN107222798A (en) * | 2017-07-19 | 2017-09-29 | 歌尔科技有限公司 | A kind of intelligent sound box display device |
| CN107643886A (en) * | 2017-09-29 | 2018-01-30 | 联想(北京)有限公司 | A kind of multimedia equipment and control method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019214216A1 (en) | 2019-11-14 |
| CN108595139A (en) | 2018-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108595139B (en) | Intelligent multimedia equipment and display method thereof | |
| US10827291B2 (en) | Audio apparatus adaptable to user position | |
| WO2021027649A1 (en) | Dark-mode display interface processing method, electronic device, and storage medium | |
| US10694291B2 (en) | Directional propagation method and apparatus for audio signal, a terminal device and a storage medium | |
| JP7221013B2 (en) | Electronic device having a connector capable of connecting an antenna | |
| JP2020205592A (en) | Voice control electronic device | |
| US20140370817A1 (en) | Determining proximity for devices interacting with media devices | |
| US20130177198A1 (en) | Wireless Audio Player and Speaker System | |
| KR102127390B1 (en) | Wireless receiver and method for controlling the same | |
| CN111629300A (en) | Signal processing method and device and electronic equipment | |
| CN110933452B (en) | Method and device for displaying lovely face gift and storage medium | |
| CN107643886A (en) | A kind of multimedia equipment and control method | |
| CN109276895B (en) | Building block system, and method, device and system for identifying topological structure | |
| CN108337926A (en) | Sound collection means | |
| WO2020237617A1 (en) | Screen control method, device and apparatus, and storage medium | |
| US11303979B2 (en) | Electronic device including multi-channel speaker system | |
| CN109151659B (en) | Directional sound production method and device for audio equipment and audio equipment | |
| TWI676130B (en) | Display device | |
| US9942660B2 (en) | Sound output device | |
| CN114384466A (en) | Sound source direction determining method, sound source direction determining device, electronic equipment and storage medium | |
| CN103716540A (en) | Locating method, related equipment and system | |
| CN107479250B (en) | Light-emitting device and electronic equipment | |
| CN110769104A (en) | Electronic device and voice control method | |
| US11907494B2 (en) | Electronic device comprising expandable display, and method for providing content of electronic device | |
| US20230413002A1 (en) | Electronic device for applying directionality to audio signal, and method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |