WO2019214216A1

WO2019214216A1 - Smart multimedia device and display method thereof

Info

Publication number: WO2019214216A1
Application number: PCT/CN2018/118566
Authority: WO
Inventors: 董学会; 刘鲁松
Original assignee: 歌尔股份有限公司
Priority date: 2018-05-08
Filing date: 2018-11-30
Publication date: 2019-11-14
Also published as: CN108595139A; CN108595139B

Abstract

Provided are a smart multimedia device and display method thereof; said smart multimedia device comprises a plurality of display regions arranged on a side face of said device and a ring-shaped microphone array arranged on a surface of said device, the corresponding method comprising: according to a sound signal received by each microphone in the ring-shaped microphone array, determining a first microphone (S510); according to correlation between the plurality of microphones and the plurality of display regions, determining a first display region corresponding to the first microphone (S520); controlling the first display region to perform display (S530). Another type of smart multimedia device comprises a dynamic display region arranged on a side face of said device and a ring-shaped microphone array arranged on a surface of said device, the corresponding method comprising: according to a sound signal received by each microphone in the ring-shaped microphone array, determining a first microphone (S710); determining a correlation between the first microphone and the dynamic display region (S720); according to said correlation, controlling a first sub-region of the dynamic display region to perform display (S730). The smart multimedia device is capable of automatically adjusting the visual display region in real time following changes in the direction/location of the sound source.

Description

Intelligent multimedia device and display method thereof

Technical field

The invention relates to an intelligent multimedia device and a display method thereof.

Background technique

With the improvement of living standards, smart homes are more and more appearing in people's daily life. Especially intelligent multimedia devices (such as smart speakers) are widely popular as a kind of products in smart homes. Some intelligent multimedia devices on the market are mostly equipped with a display screen, and the display screen can not only display related information, but also realize intelligent multimedia control through the touch display screen.

At present, however, the display screen of the intelligent multimedia device is generally fixed, and cannot be adjusted in real time for directional display to align with the direction of the person, so that the user needs to move to the opposite side of the display or manually when viewing the display screen or performing a touch operation. Moving the device results in poor human-computer interaction and affects the user experience.

Summary of the invention

The invention provides an intelligent multimedia device and a display method thereof, so as to solve the problem that the display screen of the prior art has a fixed orientation and causes a poor human-computer interaction effect.

An aspect of the present invention provides a display method of an intelligent multimedia device including a plurality of display areas disposed on a side thereof, and a ring microphone array disposed on a surface thereof, and a plurality of microphones and a plurality of microphones in the ring microphone array Corresponding relationship between the display areas, the method comprising: determining a first microphone according to a sound signal received by each microphone in the ring microphone array; determining and comparing according to a correspondence between the plurality of microphones and the plurality of display areas a first display area corresponding to a microphone; controlling the first display area for display.

An aspect of the present invention also provides a display method of a smart multimedia device, comprising: a dynamic display area disposed on a side thereof, and a ring microphone array disposed on a surface thereof, comprising: receiving according to each microphone in the ring microphone array The sound signal determines a first microphone; determines a correspondence relationship between the first microphone and the display area; and controls the first sub-area of the display area to perform dynamic display according to the correspondence relationship.

Another aspect of the present invention provides an intelligent multimedia device including a plurality of display areas disposed on a side thereof, and a ring microphone array disposed on a surface thereof, between a plurality of microphones and a plurality of display areas in the ring microphone array Corresponding relationship, further comprising: a first signal processing unit, configured to determine a first microphone according to a sound signal received by each microphone in the ring microphone array; and a first analysis processing unit configured to use the plurality of microphones and the plurality of display areas Corresponding relationship between the first display area corresponding to the first microphone; the first on-screen display control unit is configured to control the first display area for display.

Another aspect of the present invention also provides an intelligent multimedia device comprising a dynamic display area disposed on a side thereof, and a ring microphone array disposed on a surface thereof, further comprising: a second signal processing unit, according to each of the ring microphone arrays a sound signal received by the microphone determines a first microphone; a second analysis processing unit is configured to determine a correspondence between the first microphone and the dynamic display area; and a second on-screen display control unit is configured to control the dynamic display area according to the corresponding relationship The first sub-area is displayed.

Still another aspect of the present invention provides an intelligent multimedia system comprising: one or more storage units for storing executable instructions; and one or more processing units for executing executable instructions to implement the above-described intelligent multimedia The display method of the device.

The intelligent multimedia device and the display method thereof are provided. For a smart multimedia device having multiple display areas, a corresponding relationship between each display area and a corresponding microphone is preset, and when the visual display area of the intelligent multimedia is adjusted, according to the microphone array The signal characteristics of the sound signals received by the respective microphones determine the first microphone that can identify the position/direction of the sound source, and then determine the first display area corresponding to the first microphone according to the correspondence between the microphone and the display area, thereby realizing the intelligence of the intelligent multimedia device. The directional display enables it to automatically adjust the visual display area in real time following the change of the direction/position of the sound source, enhance the human-computer interaction effect, and enhance the user experience.

The intelligent multimedia device and the display method thereof provided by the present invention, for a smart multimedia device having a dynamic display area, the dynamic display area can be dynamically divided into a plurality of sub-areas, and when the visual display area of the intelligent multimedia is adjusted, the first Corresponding relationship between a microphone and a dynamic display area, dynamically determining a first sub-area corresponding to the first microphone on the dynamic display area of the smart multimedia device, and controlling the first sub-area for display, thereby implementing intelligent orientation display of the intelligent multimedia device, It enables it to dynamically adjust the visual display area in real time following the change of direction/position of the sound source, enhance the human-computer interaction effect and enhance the user experience.

DRAWINGS

FIG. 1 is a schematic diagram of a three-dimensional structure of an intelligent multimedia according to an embodiment of the present invention; FIG.

2 is an exploded perspective view of an intelligent multimedia according to an embodiment of the present invention;

3 is a schematic cross-sectional view of an intelligent multimedia according to an embodiment of the present invention;

4 is a schematic diagram of multiple display areas of smart multimedia on the side of a prism body according to an embodiment of the present invention;

FIG. 5 is a flowchart of a display method of an intelligent multimedia device according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a determining manner of a first display area according to an embodiment of the present invention; FIG.

FIG. 7 is a flowchart of a display method of another smart multimedia device according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of mapping a first sub-area on a dynamic display area according to an embodiment of the present invention;

FIG. 9 is a schematic diagram showing a correspondence relationship between a first microphone and a first sub-area according to an embodiment of the present invention;

FIG. 10 is a schematic diagram showing a first sub-area dynamically changing with a change of a first microphone according to an embodiment of the present invention; FIG.

FIG. 11 is a structural block diagram of a smart multimedia device according to an embodiment of the present invention;

FIG. 12 is a structural block diagram of another intelligent multimedia device according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, it should be understood that the description is only illustrative, and is not intended to limit the scope of the invention. In addition, descriptions of well-known structures and techniques are omitted in the following description in order to avoid unnecessarily obscuring the inventive concept.

The terminology used herein is for the purpose of describing particular embodiments, The words "a", "an", "the" and "the" In addition, the terms "including", "comprising", etc., are used to indicate the presence of the described features, steps, operations and/or components, but do not exclude the presence or addition of one or more other features, steps, operations or components .

All terms (including technical and scientific terms) used herein have the meaning commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted as having a meaning consistent with the context of the present specification and should not be interpreted in an ideal or too rigid manner.

Some block diagrams and/or flowcharts are shown in the drawings. It will be understood that some blocks or combinations of the block diagrams and/or flowcharts can be implemented by computer program instructions. These computer program instructions may be provided to a general purpose computer, a special purpose computer or a processor of other programmable data processing apparatus such that when executed by the processor, the instructions may be used to implement the functions illustrated in the block diagrams and/or flowcharts. / operating device.

Thus, the techniques of this disclosure may be implemented in the form of hardware and/or software (including firmware, microcode, etc.). Additionally, the techniques of this disclosure may take the form of a computer program product on a computer readable medium storing instructions for use by or in connection with an instruction execution system. In the context of the present invention, a computer readable medium can be any medium that can contain, store, communicate, propagate or transport the instructions. For example, a computer readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Specific examples of the computer readable medium include: a magnetic storage device such as a magnetic tape or a hard disk (HDD); an optical storage device such as a compact disk (CD-ROM); a memory such as a random access memory (RAM) or a flash memory; and/or a wired /Wireless communication link.

In order to improve the human-computer interaction effect and break the limitation on the range of the user's operation, the present invention sets a plurality of display areas on the side of the smart multimedia device. For example, when the side of the smart multimedia is an annular side, the number of the ring side can be set. Display area, when the side of the smart multimedia is composed of the surface of the polyhedron, the display area can be set on each surface of the polyhedron; and the corresponding relationship between the microphone and the display area is set, and the sound source can be identified by analyzing the characteristics of the picked-up sound signal. The first microphone of the position/direction controls the display area of the display area corresponding to the first microphone, and the other display areas do not display the screen to save energy and realize the adjustable visual position of the display screen.

To facilitate the description of the intelligent multimedia display scheme of the embodiment of the present invention, the present invention first describes the composition of the intelligent multimedia device through the following embodiments. The intelligent multimedia of the embodiment of the invention includes, but is not limited to, a smart speaker, a smart TV and the like.

1 is a schematic perspective view of an intelligent multimedia according to an embodiment of the present invention, FIG. 2 is an exploded perspective view of the smart multimedia according to an embodiment of the present invention, and FIG. 3 is a schematic cross-sectional view of the smart multimedia according to an embodiment of the present invention, as shown in FIG. 3, the intelligent multimedia device of the embodiment of the present invention includes: an upper case 1, a housing 7, a first PCB board 2, a display screen 3, a second PCB board 4, a microphone array and a speaker module 6;

The upper casing 1 is fastened on the outer casing 7 and encloses a smart multimedia casing with the outer casing 7. The first PCB board 2 is fixed on the lower side of the upper casing 1, and the microphone array is disposed on the first PCB board 2 and the upper casing 1. Meanwhile, the upper casing 1 is provided with a microphone hole a corresponding to the position of the microphone b in the microphone array, and the microphone b in the microphone array picks up the sound signal of the external sound source through the microphone hole a.

The second PCB board 4 is a main control board of intelligent multimedia, and is fixed in the outer casing 7. The display screen 3 is disposed 360 degrees outside the outer casing 7 around the outer casing, and the speaker module 6 is disposed between the second PCB board 4 and the bottom of the outer casing 7. The horn module 6 transmits sound to the outside through the horn hole c on the outer casing 7, and the outer casing 7 is sleeved with a transparent outer casing 5, and the display screen 3 is disposed on the outer casing 7 and between the transparent outer casing 5 and the outer casing 7. The display screen may be composed of a complete display screen, or may be formed by splicing a plurality of display screens, which is not specifically limited in the embodiment of the present invention.

In some embodiments, components corresponding to each function may be integrated on the second PCB board 4, such as an audio processor, a microprocessor, a screen control module, etc., disposed on the second PCB board 4, and the microphone is picked up by the audio processor. The sound signal is subjected to noise reduction, dereverberation, etc., and the processed sound signal is transmitted to the microprocessor, and the microprocessor determines the intensity value of the sound signal picked up by each microphone, thereby determining the sound source direction and driving the screen. The control module displays the display area corresponding to the direction of the sound source. It can be understood that, in other embodiments, other corresponding components can be integrated on the second PCB 4 according to the function of the intelligent multimedia, and the microprocessor is configured to perform logic control, such as when the intelligent multimedia device supports the conference work. In the mode and the normal working mode, when the microprocessor receives the instruction to enter the conference working mode, the screen control module is driven to display the respective display areas corresponding to the plurality of sound source directions, and the microprocessor enters the general working mode. The screen control module is driven to control the display orientation display.

It should be noted that FIGS. 1 to 3 are only exemplarily showing a smart multimedia device having a cylindrical side, and the display screen includes a cylindrical side of the circumference, in which case the display screen can be divided into a plurality of display areas, according to the sound source. The location selects the corresponding display area for display. It is obvious that the smart multimedia device can also be a prismatic side. As shown in FIG. 4, a display screen can be provided on each side of the prism, and the smart multimedia device includes five display areas. Of course, the smart multimedia device can also be the display side of other shape geometry. The embodiment of the present invention does not limit the shape of the display side of the smart multimedia device.

In the embodiment of the present invention, the setting position of each microphone in the microphone array has a corresponding relationship with the display area of the intelligent multimedia device. Thereby, the display area for human-computer interaction determined by the sound source position is determined to be converted into the display area for human-computer interaction according to the position of the first microphone capable of identifying the sound source position. The present invention specifically describes a display method of an intelligent multimedia device by the following embodiments.

One aspect of the present invention provides a display method of a smart multimedia device.

FIG. 5 is a flowchart of a display method of a smart multimedia device according to an embodiment of the present invention, wherein the smart multimedia device includes a plurality of display areas disposed on a side thereof, and a ring microphone array disposed on a surface thereof, wherein the ring microphone array is The plurality of microphones have a corresponding relationship with the plurality of display areas, such as a correspondence relationship between the plurality of microphones and the plurality of display areas. As shown in FIG. 5, the method of the embodiment of the present invention includes:

S510. Determine a first microphone according to a sound signal received by each microphone in the ring microphone array.

The embodiment of the present invention is based on the fact that the signal strength of the microphone receiving sound signal is related to the sound source position/direction, and the sound signal picked up by the microphone close to the sound source is large, and the ring microphone of the intelligent multimedia device is utilized. Each microphone in the array picks up a sound signal of the sound source, and determines a first microphone capable of indicating a sound source position/direction from among a plurality of microphones picked up to the sound signal, for example, the microphone having the highest received sound signal strength is the first microphone, or The plurality of microphones whose received sound signal strength exceeds the preset intensity value are the first microphones to determine a display area for human-computer interaction corresponding to the first microphone according to the correspondence between the plurality of microphones and the plurality of display areas.

S520. Determine a first display area corresponding to the first microphone according to a correspondence between the plurality of microphones and the plurality of display areas.

In the embodiment of the present invention, when the intelligent multimedia device is initialized, the corresponding relationship between the plurality of display areas and the plurality of microphones is initialized according to the position of each display area of the intelligent multimedia device and the setting position of each microphone in the ring microphone array, for example, one microphone corresponds to one The display area, or multiple microphones corresponding to the same display area. Therefore, after the first microphone is determined, the first display area corresponding to the first microphone may be determined according to the correspondence.

S530. Control the first display area for display.

In the embodiment of the present invention, a plurality of display areas of the intelligent multimedia setting are preset, and each display area has a corresponding relationship with a corresponding microphone. When the visual display area of the intelligent multimedia is adjusted, the sound signals are received according to the respective microphones in the microphone array. The signal feature determines a first microphone that can identify the position/direction of the sound source, and then determines the first display area corresponding to the first microphone according to the correspondence between the microphone and the display area, thereby implementing intelligent orientation display of the intelligent multimedia device, so that the Follow the change of direction/position of the sound source to automatically adjust the visual display area in real time, enhance the human-computer interaction effect and enhance the user experience.

According to an embodiment of the invention, the correspondence between the microphone and the display area in the ring microphone array includes a correspondence relationship between the positions of the plurality of microphones and the plurality of display areas. After the sound signal of the sound source is picked up by each microphone in the ring microphone array, the microphone with the highest intensity of the sound signal received in the ring microphone array is determined as the first microphone according to the characteristics of the sound signal, and the set position of the first microphone is obtained, and then according to the microphone. Determining, by the correspondence relationship between the position and the setting position of the first microphone, the first microphone corresponds to the first display area.

For example, when the set position of the first microphone is on the first display area, the first display area is determined. When the set position of the first microphone is on the boundary of two adjacent display areas, one of the two adjacent display areas is randomly determined as the first display area, or the display area corresponding to the set rule is determined to be the first display Area, or not responding. The setting rule in the embodiment of the present invention includes, but is not limited to, determining the first display area in the forward/counterclockwise direction of the reference position, and the reference position may be the set position of the first microphone.

The above is an example of determining a first display area from a plurality of display areas of a smart multimedia device having a cylindrical side, and it is apparent that the first display area is determined from a plurality of display areas of the smart multimedia device having other shaped sides. The method is similar or identical to the above, and the present invention will not be described again.

FIG. 6 is a schematic diagram of a method for determining a first display area according to an embodiment of the present invention. Referring to FIG. 6, it is assumed that a smart multimedia device includes four display areas, and each of the display areas is provided with eight microphones, and FIG. 6 uses a circle. The dot indicates the microphone. As shown in FIG. 6, it is assumed that the sound signals of the five microphones A7, B0, B1, B2, and B3 indicated by the black solid dots in the smart multimedia device picked up to the sound source exceed the preset intensity value, and the sound signal picked up by the microphone B1. When the intensity of the sound is maximum, the display area 2 is determined to be the first display area. When the intensity of the sound signal picked up by the microphone A7 is maximum, the display area 1 is determined to be the first display area. When the intensity of the sound signal picked up by the microphone B0 is maximum, Determining that the display area 1 or the display area 2 is the first display area, and determining the first display area according to the setting rule, for example, setting the display area 2 of the clockwise direction when the first microphone B0 is located at the boundary of the adjacent display area It is determined as the first display area, and it is also possible to re-enter the sound pickup without responding.

The embodiment of the present invention determines the first microphone according to the microphone with the highest sound signal strength picked up in the microphone array, and determines the first display area corresponding to the first microphone according to the correspondence between the plurality of microphones and the plurality of display areas, which is simple and fast. The advantage is that it is convenient to adjust the visual display area of the intelligent multimedia device in real time.

The embodiment of the present invention may further determine the first microphone according to the plurality of microphones whose sound signal intensity picked up in the microphone array exceeds the preset intensity value, the first microphone includes a plurality of reference microphones, and then according to the distribution of the plurality of reference microphones in the display area The situation determines the first display area.

According to an embodiment of the invention, the correspondence between the plurality of microphones and the plurality of display areas in the ring microphone array includes a correspondence relationship between the positions of the plurality of microphones and the plurality of display areas. In some embodiments, after the sound signal of the sound source is picked up by the microphone in the ring microphone array, and the plurality of reference microphones whose sound signal strength received in the ring microphone array exceeds the preset intensity value is determined according to the characteristics of the sound signal, multiple Referring to the set position of the microphone, the first display area is determined according to the correspondence relationship between the set position between the microphone and the display area and the set position of the plurality of reference microphones.

In a scenario, when the number of setting positions of the plurality of reference microphones is different on the display area, it is determined that the display area in which the number of reference microphones is larger than the number of reference microphones on the other display areas is the first display area.

For example, when the set positions of the plurality of reference microphones are distributed in the three display areas, assuming that the number of reference microphones in the three display areas is different, one display area having the largest number of reference microphones among the three display areas is determined as the first A display area.

In another scenario, when the number of setting positions of the plurality of reference microphones is the same on the display area, the first display area is randomly determined from the display area where the set positions of the plurality of reference microphones are located, or the setting rule is determined. The display area is the first display area or does not respond. The setting rule includes: but is not limited to, the display area where the reference microphone with the highest intensity of the sound signal is picked up is the first display area, or the first display area is determined by the forward/counterclockwise direction of the reference position, and the reference position may be some The setting position of the reference microphones.

For example, when the set positions of the plurality of reference microphones are distributed in the three display areas, assuming that the number of reference microphones in the three display areas is the same, any one of the three display areas is randomly determined as the first Display area. The first display area may also be determined according to a setting rule, for example, the display area where the reference microphone having the highest sound signal intensity is picked up is the first display area. Of course, you can also refuse to respond and re-enter the sound pickup.

Referring to FIG. 6, it is assumed that the intensity of the sound signals picked up by the five microphones A7, B0, B1, B2, and B3 indicated by the black solid dots in the smart multimedia device exceeds the preset intensity value, and the five microphones are reference microphones. As can be seen from Fig. 6, only the microphone A7 is located in the display area 1, and the four microphones B0, B1, B2, and B3 are located in the display area 2, and it can be determined that the display area 2 is the first display area.

The intensity of the sound signal determined by the embodiment is the average value of the amplitude of the sound signal picked up by the corresponding microphone in the ring microphone array in a preset time, so that the determined first microphone can accurately represent the direction of the sound source/ position.

The foregoing embodiment of the present invention provides two schemes for determining a first display area. The first scheme determines the first display area according to the intensity of the sound signal picked up by the microphone, which is simple and fast; the second scheme is based on the distribution of the reference microphone and the reference microphone. The intensity of the picked up sound signal determines the first display area, and the determined first display area is more representative of the sound source position/direction. Both of these options have their own advantages, and you can choose the right solution according to your needs.

FIG. 7 is a flowchart of a display method of another smart multimedia device according to an embodiment of the present invention. The smart multimedia device includes a dynamic display area disposed on a side thereof, and the dynamic display area can be dynamically divided into multiple sub-areas and disposed on the same. A circular microphone array of surfaces. As shown in FIG. 7, the method of the embodiment of the present invention includes:

S710. Determine a first microphone according to a sound signal received by each microphone in the ring microphone array.

The embodiment of the present invention is based on the fact that the signal strength of the microphone receiving sound signal is related to the sound source position/direction, and the sound signal picked up by the microphone close to the sound source is large, and the ring microphone of the intelligent multimedia device is utilized. The microphone in the array picks up the sound signal of the sound source, and determines a first microphone capable of indicating the position/direction of the sound source from the plurality of microphones picked up to the sound signal, and then determines the number for human-computer interaction according to the set position of the first microphone. A display area.

S720. Determine a correspondence between the first microphone and the dynamic display area.

In the embodiment of the present invention, the corresponding relationship between the first microphone and the dynamic display area includes, but is not limited to, the set position of the first microphone corresponds to the center line of the first sub-area of the dynamic display area, and the boundary line of the first sub-area The apex angle enclosed by the body center point of the intelligent multimedia device has a set angle.

S730, the first sub-area of the display area is controlled to be displayed according to the correspondence relationship.

The embodiment of the present invention may map the first sub-region corresponding to the first microphone on the display area according to the correspondence between the first microphone and the dynamic display area, thereby controlling the first sub-area to perform human-computer interaction display.

In the embodiment of the present invention, a first microphone capable of identifying a sound source position is determined according to a signal characteristic of a sound signal received by each microphone in the microphone array, and the dynamic relationship between the first microphone and the first sub-area is dynamically displayed on the dynamic display area of the intelligent multimedia device. Determining the first sub-region corresponding to the first microphone, and controlling the first sub-region to display, realizing intelligent orientation display of the intelligent multimedia device, so that it can automatically adjust the visual display region in real time following the change of the direction/position of the sound source, and enhance Human-computer interaction effects enhance user experience.

In the embodiment of the present invention, the microphone that picks up the sound source in the ring microphone array is used to determine that the microphone with the highest intensity of the received sound signal in the ring microphone array is the first microphone, according to the positional relationship between the first microphone and the dynamic display area. And determining a first sub-area of the dynamic display area, where the first microphone is disposed on a center line of the first sub-area, and the first sub-area is a bottom surface of the cone with the center point of the intelligent multimedia device as a vertex.

The above is an example of determining a first sub-area from a dynamic display area of a smart multimedia device having a cylindrical side, and determining the first sub-area from the dynamic display area of the smart multimedia device having other shape sides is the same as the above method. , will not repeat them here.

FIG. 8 is a schematic diagram of mapping a first sub-area on a dynamic display area according to an embodiment of the present invention. FIG. 9 is a schematic diagram showing a correspondence between a first microphone and a first sub-area according to an embodiment of the present invention, as shown in FIG. 8-9. The microphone A is determined to be the first microphone according to the intensity of the sound signal picked up by the microphone, and according to the set positional relationship between the first microphone and the dynamic display area, determining that the straight line of the setting position of the first microphone on the dynamic display area is AF, The straight line AF is the center line of the first sub-area, the first sub-area is the bottom surface BCDE of the cone O-BCDE with the body center point O of the intelligent multimedia device as the apex, and the arc corresponding to the arc BC of the bottom surface BCDE is α.

It should be noted that, when determining the first sub-area, if the dynamic display area is a cylinder or a spherical surface, the first area may be set as a curved surface having a set arc. If the dynamic display area is a plane, the first area may be set as A plane with a set length or width.

FIG. 10 is a schematic diagram showing a first sub-area dynamically changing with a change of a first microphone according to an embodiment of the present invention. As shown in FIG. 10, at the first moment, the microphone A is determined to be the first microphone according to the sound signal strength picked up by the microphone. According to the method for determining the first sub-area described above, it may be determined that the first sub-region corresponding to the microphone A is the microphone BAC. location. At the second moment, the microphone A′ is determined to be the first microphone according to the sound signal strength picked up by the microphone. According to the method for determining the first sub-region described above, it may be determined that the first sub-region corresponding to the microphone A′ is the microphone B′A′C. The location of the present invention can be seen that the first sub-area can be dynamically adjusted based on the determined position of the first microphone, so that the visual display area of the intelligent multimedia device can be dynamically adjusted in real time based on the position/direction of the sound source. The display method of the embodiment of the invention is particularly applicable to the case where the sound source position is frequently moved.

Yet another aspect of the present invention provides an intelligent multimedia device.

FIG. 11 is a structural block diagram of a smart multimedia device according to an embodiment of the present invention. The intelligent multimedia device of the embodiment of the present invention includes a plurality of display areas disposed on a side thereof, and a ring microphone array disposed on a surface thereof, and the plurality of microphones in the ring microphone array have a corresponding relationship with the plurality of display areas. As shown in FIG. 11, the intelligent multimedia device of the embodiment of the present invention further includes:

The first signal processing unit 111 is configured to determine the first microphone according to the sound signal received by each microphone in the ring microphone array;

a first analysis processing unit 112, configured to determine, according to a correspondence between the plurality of microphones and the plurality of display areas, a first display area corresponding to the first microphone;

The first on-screen display control unit 113 is configured to control the first display area for display.

In some embodiments, the first signal processing unit 111 is specifically configured to determine that the microphone having the highest intensity of the received sound signal in the ring microphone array is the first microphone, wherein the intensity of the sound signal is a corresponding microphone in the ring microphone array within a preset time period. Pick the average of the amplitude of the sound signal.

In some embodiments, the correspondence between the plurality of microphones and the plurality of display areas includes a correspondence between the positions of the plurality of microphones and the plurality of display areas, and the first analysis processing unit 112 is configured to use the plurality of microphones and the plurality of Determining a first display area by determining a positional relationship between the display areas and a set position of the first microphone; wherein, when the set position of the first microphone is on the first display area, determining the first display area; When the setting position of the microphone is on the boundary of two adjacent display areas, one of the two adjacent display areas is randomly determined as the first display area; or, the display area corresponding to the setting rule is determined to be the first display area; Or, no response.

In some embodiments, the first signal processing unit 111 is further configured to determine that the plurality of microphones whose received sound signal strength exceeds a preset intensity value in the circular microphone array is a first microphone, and the first microphone includes a plurality of reference microphones, wherein the sound The intensity of the signal is the average of the amplitudes of the sound signals picked up by the corresponding microphones in the ring microphone array within a preset time period.

In some embodiments, the correspondence between the plurality of microphones and the plurality of display areas includes a correspondence between the positions of the plurality of microphones and the plurality of display areas, and the first analysis processing unit 112 is configured to use the plurality of microphones and the plurality of Determining a first display area by determining a positional relationship between the display areas and a plurality of reference microphones; wherein, when the number of the plurality of reference microphone setting positions on the plurality of display areas is different, determining the number of reference microphones a display area larger than the number of reference microphones on the other display areas is the first display area; when the number of the plurality of reference microphone setting positions on the plurality of display areas is the same, randomly from the display area where the set positions of the plurality of reference microphones are located Determining the first display area; or determining that the display area corresponding to the set rule is the first display area; or, not responding.

For a specific mode of operation of each unit of the device embodiment of the present invention, reference may be made to the method embodiments of the present invention.

FIG. 12 is a structural block diagram of another intelligent multimedia device according to an embodiment of the present invention. The intelligent multimedia device of the embodiment of the invention comprises a dynamic display area disposed on a side thereof, and an annular microphone array disposed on a surface thereof. As shown in FIG. 12, the intelligent multimedia device of the embodiment of the present invention further includes:

a second signal processing unit 121, configured to determine a first microphone according to a sound signal received by each microphone in the ring microphone array;

a second analysis processing unit 122, configured to determine a correspondence between the first microphone and the dynamic display area;

The second on-screen display control unit 123 is configured to control the first sub-area of the dynamic display area to perform display according to the correspondence relationship.

In some embodiments, the second signal processing unit 121 is specifically configured to determine that the microphone with the highest received sound signal strength in the ring microphone array is the first microphone, wherein the intensity of the sound signal is corresponding to the ring microphone array in a preset time. The average of the amplitude of the sound signal picked up by the microphone.

The second analysis processing unit 122 is specifically configured to determine a first sub-area of the dynamic display area according to a set positional relationship between the first microphone and the dynamic display area, where the set position of the first microphone is at a center line of the first sub-area The first sub-area is a bottom surface of the cone set with the center point of the intelligent multimedia device body as a vertex.

Yet another aspect of the present invention provides an intelligent multimedia system.

The intelligent multimedia system of the embodiment of the present invention includes: one or more storage units for storing executable instructions; and one or more processing units for executing executable instructions to implement the display method of the intelligent multimedia device described above.

Yet another aspect of the present invention provides a readable storage medium.

The readable storage medium of the embodiment of the present invention stores executable instructions, and when the executable instructions are executed by the processor, the display method of the intelligent multimedia device described above can be implemented.

It should be noted that the readable storage medium of the embodiments of the present invention may be, for example, any medium capable of containing, storing, transmitting, transmitting, or transmitting instructions. For example, a readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Specific examples of the readable storage medium include: a magnetic storage device such as a magnetic tape or a hard disk (HDD); an optical storage device such as a compact disk (CD-ROM); a memory such as a random access memory (RAM) or a flash memory; and/or a wired /Wireless communication link.

The readable storage medium can include a computer program, which can include code/computer executable instructions that, when executed by a processor, cause the processor to perform, for example, the method flow described above and any variations thereof.

The computer program can be configured to have, for example, computer program code comprising a computer program module. For example, in an example embodiment, the code in a computer program can include one or more program modules. It should be noted that the division manner and number of modules are not fixed, and those skilled in the art may use suitable program modules or program module combinations according to actual situations. When these program module combinations are executed by the processor, the processor may execute, for example, The method flow described above and any variations thereof.

In order to facilitate the clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words “first” and “second” are used to distinguish the same items or similar items whose functions and functions are substantially the same. Personnel can understand that the words “first” and “second” do not limit the quantity and order of execution.

The above is only the embodiment of the present invention, and other improvements or modifications may be made by those skilled in the art based on the above embodiments. It should be understood by those skilled in the art that the foregoing detailed description of the invention is intended to provide a better understanding of the scope of the invention.

Claims

A display method of a smart multimedia device, comprising: a plurality of display areas disposed on a side thereof, and a ring microphone array disposed on a surface thereof, the plurality of microphones in the ring microphone array and the plurality of There is a corresponding relationship between the display areas, and the method includes:

Determining a first microphone according to a sound signal received by each microphone in the ring microphone array;

Determining, according to a correspondence between the plurality of microphones and the plurality of display areas, a first display area corresponding to the first microphone;

The first display area is controlled for display.
The method according to claim 1, wherein the determining the first microphone according to the sound signal received by each microphone in the ring microphone array comprises:

The microphone that determines the intensity of the sound signal received in the ring microphone array is the first microphone, wherein the intensity of the sound signal is the average value of the amplitude of the sound signal picked up by the corresponding microphone in the ring microphone array in a preset time.
The method according to claim 2, wherein the correspondence between the plurality of microphones and the plurality of display areas comprises a correspondence between the positions of the plurality of microphones and the plurality of display areas The determining the first display area corresponding to the first microphone includes:

Determining the first display area according to a correspondence relationship between the plurality of microphones and a position of the plurality of display areas and a setting position of the first microphone, wherein

Determining the first display area when the set position of the first microphone is on the first display area;

When the set position of the first microphone is on a boundary of two adjacent display areas, randomly determining one of the two adjacent display areas as the first display area; or determining a display consistent with the setting rule The area is the first display area; or, no response is given.
The method according to claim 1, wherein the determining the first microphone according to the sound signal received by each microphone in the ring microphone array comprises:

Determining, by the plurality of microphones, that the sound signal strength received in the ring microphone array exceeds a preset intensity value is the first microphone, the first microphone includes a plurality of reference microphones, wherein the intensity of the sound signal is a preset time The mean value of the amplitude of the sound signal picked up by the corresponding microphone in the annular microphone array.
The method according to claim 4, wherein the correspondence between the plurality of microphones and the plurality of display areas comprises a correspondence between the positions of the plurality of microphones and the plurality of display areas The determining the first display area corresponding to the first microphone includes:

Determining the first display area according to a correspondence relationship between the positions of the plurality of microphones and the plurality of display areas and a set position of the plurality of reference microphones, wherein

When the number of the plurality of reference microphone setting positions on the plurality of display areas is different, determining that the display area in which the number of reference microphones is larger than the number of reference microphones on the other display areas is the first display area;

Determining the first display area randomly from a display area where the set positions of the plurality of reference microphones are located when the number of the plurality of reference microphone setting positions is the same on the plurality of display areas; or determining and setting rules The corresponding display area is the first display area; or, no response is given.
A display method of a smart multimedia device, comprising: a dynamic display area disposed on a side thereof, and an annular microphone array disposed on a surface thereof, the method comprising:

Determining a first microphone according to a sound signal received by each microphone in the ring microphone array;

Determining a correspondence between the first microphone and a dynamic display area;

Controlling, by the corresponding relationship, the first sub-area of the dynamic display area for display.
The method according to claim 6, wherein the determining the first microphone according to the sound signal received by each microphone in the ring microphone array comprises:

The microphone that determines the intensity of the sound signal received in the ring microphone array is the first microphone, wherein the intensity of the sound signal is the average value of the amplitude of the sound signal picked up by the corresponding microphone in the ring microphone array in a preset time.
The method according to claim 7, wherein the determining the correspondence between the first microphone and the dynamic display area comprises:

Determining, according to a set positional relationship between the first microphone and the dynamic display area, a first sub-area of the dynamic display area, where the setting position of the first microphone is on a center line of the first sub-area, The first sub-area is a bottom surface of the cone that is set to apex with the center point of the intelligent multimedia device body.
An intelligent multimedia device comprising a plurality of display areas disposed on a side thereof, and a ring microphone array disposed on a surface thereof, a plurality of microphones in the ring microphone array and the plurality of display areas Correspondence between them also includes:

a first signal processing unit, configured to determine a first microphone according to a sound signal received by each microphone in the ring microphone array;

a first analysis processing unit, configured to determine, according to a correspondence between the plurality of microphones and the plurality of display areas, a first display area corresponding to the first microphone;

The first on-screen display control unit is configured to control the first display area for display.
The device according to claim 9, wherein the first signal processing unit is specifically configured to determine that the microphone having the highest intensity of the received sound signal in the circular microphone array is the first microphone, wherein the sound signal is The intensity is the mean value of the amplitude of the sound signal picked up by the corresponding microphone in the ring microphone array within a preset time.
The device according to claim 10, wherein the correspondence between the plurality of microphones and the plurality of display areas comprises a correspondence between the positions of the plurality of microphones and the plurality of display areas ,

The first analysis processing unit is configured to determine the first display area according to a correspondence between a position of the plurality of microphones and the plurality of display areas and a set position of the first microphone, where Determining the first display area when the set position of the first microphone is on the first display area; random when the set position of the first microphone is on a boundary of two adjacent display areas Determining one of the two adjacent display areas as the first display area; or determining that the display area corresponding to the set rule is the first display area; or, not responding.
The device according to claim 9, wherein the first signal processing unit is specifically configured to determine that the plurality of microphones whose sound signal strength received in the ring microphone array exceeds a preset intensity value is the first a microphone, the first microphone includes a plurality of reference microphones, wherein an intensity of the sound signal is an average value of amplitudes of sound signals picked up by corresponding microphones in the ring microphone array within a preset time period.
The device according to claim 12, wherein the correspondence between the plurality of microphones and the plurality of display areas comprises a correspondence between the positions of the plurality of microphones and the plurality of display areas ,

The first analysis processing unit is configured to determine the first display area according to a correspondence between a position of the plurality of microphones and the plurality of display areas and a set position of the plurality of reference microphones. Wherein, when the number of the plurality of reference microphone setting positions on the plurality of display areas is different, determining that the display area in which the number of reference microphones is larger than the number of reference microphones on the other display areas is the first display area; Determining the first display area randomly from the display area where the set positions of the plurality of reference microphones are located when the number of the plurality of reference microphone setting positions is the same on the plurality of display areas; or determining that the setting rule is consistent The display area is the first display area; or, no response is received.
An intelligent multimedia device comprising a dynamic display area disposed on a side thereof, and a ring microphone array disposed on a surface thereof, further comprising:

a second signal processing unit, configured to determine a first microphone according to a sound signal received by each microphone in the ring microphone array;

a second analysis processing unit, configured to determine a correspondence between the first microphone and the dynamic display area;

The second on-screen display control unit is configured to control the first sub-area of the dynamic display area to perform display according to the correspondence relationship.
The device according to claim 14, wherein the second signal processing unit is specifically configured to determine that the microphone having the highest intensity of the received sound signal in the ring microphone array is the first microphone, wherein the sound signal is The intensity is the mean value of the amplitude of the sound signal picked up by the corresponding microphone in the ring microphone array within a preset time.
The device according to claim 15, wherein the second analysis processing unit is configured to determine a first dynamic display area according to a set positional relationship between the first microphone and the dynamic display area. In the sub-area, the first microphone is disposed on a center line of the first sub-area, and the first sub-area is a bottom surface of the cone that is set to be a vertex of the smart multimedia device body center point.
An intelligent multimedia system, characterized in that the intelligent multimedia system comprises:

One or more storage units for storing executable instructions;

One or more processing units for executing executable instructions, implementing any one of claims 1-5 or implementing the display method of the intelligent multimedia device according to any one of claims 6-8.
A readable storage medium, characterized in that the readable storage medium stores executable instructions that, when executed by a processor, can implement any of claims 1-5 or implement any of claims 6-8 The display method of the intelligent multimedia device described in the item.