WO2024037460A1

WO2024037460A1 - Screen display control method, and medium and electronic device

Info

Publication number: WO2024037460A1
Application number: PCT/CN2023/112671
Authority: WO
Inventors: 罗友; 高月莉; 赵志伟; 黎椿键; 金伟; 张恒; 胡成博; 陶婧雅; 李珠莹
Original assignee: 华为技术有限公司
Priority date: 2022-08-16
Filing date: 2023-08-11
Publication date: 2024-02-22
Also published as: CN117631803A

Abstract

The present application relates to the technical field of electronics. Disclosed are a screen display control method, and a medium and an electronic device. The method comprises: transmitting an ultrasonic signal; acquiring a plurality of echo signals, which are received within a first time period and are generated after the ultrasonic signal is reflected by objects; when there is a user who moves relative to an electronic device among the objects that reflect the ultrasonic signal, executing a first screen display mode if the plurality of echo signals meet a first condition for indicating whether the user is present; and when there is not a user who moves relative to the electronic device among the objects that reflect the ultrasonic signal, executing a second screen display mode if the plurality of echo signals do not meet the first condition for indicating whether the user is present. On this basis, an electronic device can correspond to different screen display modes according to whether there is a user in front of the electronic device, such that the user experience is improved, thereby avoiding power consumption waste caused by a normally open screen when there is no user in front of the electronic device.

Description

Screen display control method, medium and electronic device

This application claims priority to the Chinese patent application filed with the China Patent Office on August 16, 2022, with the application number 202210983298.7 and the application name "Screen display control method, medium and electronic device", the entire content of which is incorporated herein by reference. Applying.

Technical field

The present application relates to the field of electronic technology, and in particular to a screen display control method, medium and electronic equipment.

Background technique

With the development of electronic technology, electronic devices with display functions, such as TVs, smart screens, etc., are increasingly used in daily life. For example, television is one of the common electronic devices in daily life at home. In addition to watching TV programs, it can also be used for decoration and information prompts. For example, it can display beautiful wallpapers or animations during standby, integrating the TV and interior decoration to enhance the visual effect of the home; or it can be used to display prompt messages such as clock, weather or text messages. However, keeping the TV screen on has the disadvantages of wasting power and reducing screen life. The commonly used solutions now are for the user to manually set the display time period, or manually click a physical button or wake up with voice and speak instructions to turn on/off the standby display screen. The user experience is not very friendly.

Contents of the invention

The purpose of the embodiments of the present application is to provide a screen display control method, a medium and an electronic device, so that the electronic device can more accurately determine whether the user exists, and can correspond to different screen display modes according to the user's presence in front of the electronic device, improving user It can also better avoid the waste of power consumption caused by the screen being always on when there is no user, thereby extending the service life of electronic devices.

In a first aspect, embodiments of the present application provide a screen display control method, applied to electronic devices, including:

emit ultrasonic signals;

Obtaining a plurality of echo signals generated after the ultrasonic signal reflected by the object received in the first time period;

When there is a user moving relative to the electronic device among the objects that reflect the ultrasonic signal, executing the first screen display mode when multiple echo signals satisfy the first condition indicating whether there is a user;

When there is no user moving relative to the electronic device among the objects that reflect the ultrasonic signal, in the case where the multiple echo signals do not satisfy the first condition indicating whether there is a user, executing the second screen display mode;

Wherein, the first condition includes: among the multiple echo signals, there are multiple motion echo signals whose phase offsets with respect to the transmitted ultrasonic signal change with time.

It can be understood that the screen display control method provided by the embodiment of the present application determines whether there is a user in front of the electronic device based on the phase offset of the echo signal, that is, in the echo signal, if there is a user relative to the transmitted ultrasonic wave Multiple motion echo signals whose phase offsets change over time determine that there is a user moving relative to the electronic device, thereby controlling the screen of the electronic device in two situations: a user who is moving and a user who is not moving. The display mode allows the screen of the electronic device to respond differently according to the presence of the user moving relative to the electronic device. Since the wavelength of the ultrasonic frequency band is short (for example, 20kHz ultrasound has a speed of 340 meters/second in the air and a wavelength of 1.7 cm), the displacement caused by human body movement is often larger than the wavelength (such as 1.7 cm), so the displacement caused by the small movements of the human body is Changes in the echo path of the echo signal can cause changes in the phase offset of the echo signal, so that the movement of the human body can be accurately depicted by extracting the phase offset of the echo signal, thereby improving the accuracy of detecting the presence of the user. higher.

In a possible implementation of the above first aspect, the above motion includes limb motion in which the displacement of the user does not change.

That is to say, when the user does not move, based on the change of the phase offset of the echo signal, the tiny movements of the user's limbs can be detected. Compared with the existing solution through ranging and echo signal amplitude integration, The detection accuracy is higher.

In a possible implementation of the above first aspect, the ultrasonic signal includes a continuous ultrasonic signal.

It can be understood that phase data is easier to extract from frequency domain data of continuous ultrasonic signals, which is beneficial to determining the presence of a user moving relative to the electronic device.

In a possible implementation of the above first aspect, transmitting the ultrasonic signal includes:

Multiple continuous ultrasonic signals are emitted at the same time, and the emission frequencies of each continuous ultrasonic signal are different.

It can be understood that transmitting multiple ultrasonic signals with different transmitting frequencies at the same time can avoid frequency selective fading of ultrasonic signals, that is, avoiding the attenuation of multipath signal superposition of ultrasonic signals with certain transmitting frequencies at certain spatial locations, resulting in the received echo signal The energy after attenuation is too low to be used for phase offset estimation.

In a possible implementation of the above first aspect, the plurality of continuous ultrasonic signals include a first continuous ultrasonic signal and a second continuous ultrasonic signal, and

The difference between the initial transmission frequencies of the first continuous ultrasonic signal and the second continuous ultrasonic signal is greater than the frequency bandwidth of the first continuous ultrasonic signal and the frequency bandwidth of the second continuous ultrasonic signal; or

The difference between the maximum transmission frequencies of the first continuous ultrasonic signal and the second continuous ultrasonic signal is greater than the frequency bandwidth of the first continuous ultrasonic signal and the frequency bandwidth of the second continuous ultrasonic signal.

It can be understood that there is no overlap in the frequency domain between multiple continuous ultrasonic signals, which can ensure that there will be no interference between each carrier frequency, and frequency selective fading will not occur at the same detection position at the same time, so that it can be used where users exist. detection.

The same ultrasonic sounder is used to transmit multiple continuous ultrasonic signals at multiple different transmitting frequencies at the same time.

It can be understood that the frequency selective fading of ultrasonic signals is related to the emission source and location of the ultrasonic signal. Therefore, it is easier to ensure that multiple continuous ultrasonic signals are emitted at multiple different transmit frequencies using the same ultrasonic sounder. There must be an ultrasonic signal that does not undergo frequency-selective fading, which is beneficial to measurement of the presence of users moving relative to the electronic device. It can be understood that in other embodiments of the present application, different ultrasonic generators can also be used to transmit multiple continuous ultrasonic signals.

In a possible implementation of the first aspect above, the first condition also includes:

The equivalent speed of the user's movement relative to the electronic device belongs to the first speed range, and the first speed range does not include 0.

That is, when using multi-carrier frequency continuous ultrasonic signals for user detection, if there are multiple continuous ultrasonic signals without frequency selective fading, calculate the movement speed corresponding to the ultrasonic signal of each transmission frequency, and then find the equation of these speeds. The effective speed represents the user's final movement speed, and only when the user's final movement speed is within the normal human speed range (i.e., the first speed range), the user is considered to be moving relative to the electronic device.

In a possible implementation of the first aspect above, the equivalent speed is obtained in the following way:

Select multiple motion echo signals whose phase offset changes with time from the echo signals corresponding to multiple continuous ultrasonic signals;

Based on the phase offsets of the multiple motion echo signals corresponding to each continuous ultrasonic signal, multiple motion speeds corresponding to each continuous ultrasonic signal are calculated, where the phase offset is related to the echo path of the motion echo signal;

Use a preset algorithm to process multiple motion speeds to obtain the user's equivalent speed.

In a possible implementation of the first aspect above, the preset algorithm includes at least one of the following:

Calculate the average of multiple movement speeds;

Calculate least squares fits for multiple motion velocities.

In a possible implementation of the above first aspect, the first speed range is 0m/s-5m/s.

It can be understood that the normal movement speed of the human body is generally between 0m/s-5m/s. Therefore, in order to confirm that it is a human being moving relative to the electronic device, the first speed range is set to 0m/s-5m/s.

In a possible implementation of the first aspect above, the continuous ultrasonic signal includes any of the following:

Sinusoidal ultrasonic signal, frequency modulated continuous ultrasonic signal.

Humans are present in images collected by electronic devices of objects that reflect ultrasonic signals.

It can be understood that when there is a motion echo signal in the echo signal, it can be determined that there is a reflecting object that moves relative to the electronic device. However, whether the reflecting object is a human, a pet, or an object similar to a sweeping robot can also be confirmed by collecting images. , so the presence of humans in the image of the object that reflects the ultrasonic signal collected by the electronic device is used as part of the first condition, which is conducive to further accurately determining whether the moving object is the user based on the first condition, thereby making it more accurate according to the user's situation. control screen display to enhance user experience.

In a possible implementation of the above first aspect, the first screen display method includes at least one of the following:

Convert the screen of the electronic device from a black screen state to a standby screen;

Convert the screen of the electronic device from the standby screen or black screen state to the bright screen state and display the screen displayed before it last entered the standby screen or black screen state;

Convert the screen of the electronic device from the standby screen or black screen state to the bright screen state, and play the program that the user has paused.

In a possible implementation of the first aspect above, the screen displayed before entering the standby screen or black screen state last time includes at least one of the following:

startup screen;

The pause screen of the program watched by the user;

The application interface that the user browses.

In a possible implementation of the above first aspect, the second screen display method includes:

Keep the screen black or keep the standby screen displayed.

It can be understood that maintaining the black screen state or maintaining the screen display mode of displaying the standby screen can keep the power consumption of the screen in a lower state.

In a possible implementation of the above first aspect, in the case where multiple echo signals do not meet the first condition indicating whether a user is present, executing a second screen display mode includes:

Corresponding to a plurality of echo signals not satisfying the first condition indicating whether there is a user, and none of the historical echo signals received within the first preset historical duration satisfying the first condition, at least one of the following screen displays is performed Way:

Convert the screen of the electronic device from the bright screen state to the standby screen or black screen state,

Convert the screen of the electronic device from standby screen to black screen state,

Pause the program currently playing on the screen of your electronic device.

It is understandable that the program being played on the screen of the electronic device may be an audio program, a video program, a game, etc.

In a possible implementation of the above first aspect, the first screen display method also includes:

Corresponding to the user gradually approaching the electronic device and the electronic device is in a black screen state, gradually light up the screen of the electronic device and display the standby screen or the screen displayed before the last time it entered the standby screen or black screen state; and

Second screen display methods also include:

Corresponding to the user gradually moving away from the electronic device and the electronic device is in a bright screen state or displays a standby screen, the screen of the electronic device is gradually dimmed.

In a possible implementation of the above first aspect, it also includes that the faster the user approaches the electronic device, the faster the screen gradually lights up;

The faster a user moves away from an electronic device, the faster the screen gradually dims.

It can be understood that controlling the speed at which the screen of the electronic device lights up or dims according to the speed at which the user approaches or moves away from the electronic device can improve the user experience.

In a possible implementation of the first aspect above, it is determined whether the user is gradually approaching or moving away from the electronic device in the following manner:

Calculate the equivalent frequency of multiple echo signals acquired successively, where the equivalent frequency is related to the echo path of the echo signal, and the smaller the echo path, the greater the equivalent frequency;

The equivalent frequencies corresponding to the multiple echo signals acquired successively increase sequentially, and it is determined that the user is gradually approaching the electronic device;

The equivalent frequencies corresponding to the multiple echo signals acquired successively decrease in sequence, and it is determined that the user is gradually moving away from the electronic device.

It can be understood that according to the equivalent frequencies of the multiple echo signals acquired successively increasing or decreasing in sequence, it can be more reasonably judged whether the user is gradually approaching or moving away from the electronic device.

Corresponding to the direction of the user relative to the electronic device being in the first direction range, convert the screen of the electronic device from the standby screen or black screen state to the bright screen state and display the screen displayed before entering the standby screen or black screen state last time, or change the electronic device to the first direction range. The device's screen changes from the standby screen or black screen state to the bright screen state, and plays the program that the user has paused;

Corresponding to the direction of the user relative to the electronic device being in the second direction range, the screen of the electronic device is converted from a black screen state to displaying a standby screen.

By determining the relative direction of the user relative to the electronic device, the user's intention to view the electronic device can be determined, and then the picture the user wants to watch is displayed when the user wants to watch the electronic device, and the standby picture is displayed when the user is just passing by the area near the electronic device. .

In a possible implementation of the first aspect above, the phase offset of the echo signal is calculated in the following way:

Perform Fourier transform on the time domain data of the received echo signal to obtain the corresponding frequency domain data;

Based on the frequency domain data of the transmitted ultrasonic signal and the frequency domain data of the echo signal, obtain the first phase of the transmitted ultrasonic signal and the second phase of the echo signal respectively;

The difference between the second phase and the first phase is taken as the phase offset of the echo signal.

In a possible implementation of the first aspect above, it also includes:

Corresponding to a user who does not move relative to the electronic device, ultrasonic signals are emitted at multiple set volumes, and the difference between the echo energy of the echo signal and the reference echo energy is within the first energy difference range. The volume is set as a predetermined emission volume of the electronic device for emitting ultrasonic signals.

In a possible implementation of the above first aspect, the ultrasonic signal is transmitted through a speaker of the electronic device, and the echo signal is received through a microphone of the electronic device.

It is understandable that the speaker that emits the ultrasonic signal and the microphone that receives the echo signal are all built-in with the electronic device, avoiding additional costs.

In a possible implementation of the above first aspect, the electronic device includes a smart TV.

In a possible implementation of the first aspect above, it also includes:

In the case where the plurality of echo signals satisfy the first condition indicating whether a user is present, transmitting the first ultrasonic signal at a first transmission volume;

When the first relative position between the user and the electronic device is in the first position range, the second ultrasonic signal is emitted at the second transmission volume corresponding to the first position range next time,

When the first relative position between the user and the electronic device is in the second position range, the second ultrasonic signal is emitted at a third transmission volume corresponding to the second position range next time.

For example, the first position range is "(d1, d2)" below, and the second emission volume is any emission volume value in the volume range "(Vd1, Vd2)" below" or Vd2' below, The second position range is "(0, d1)" below, and the third emission volume is any emission volume value in the volume range "(0, Vd1)" below or Vd1' below.

It is understandable that if the ultrasonic signal is emitted with a fixed transmission volume, when the relative position between the user and the electronic device changes, the energy of the ultrasonic signal received by the user may be too high, which may cause harm to the user's body or cause harm to the user. The energy of the received ultrasonic signal is weak, and the energy of the echo signal reflected back by the user cannot meet the subsequent user presence detection requirements. Therefore, adjusting the next transmission volume according to the change in the relative position between the user and the electronic device can ensure that the energy of the echo signal reflected back by the user is sufficient to detect the user's movement, while avoiding the user's body from being subjected to strong force. Hazards of energy ultrasonic signals.

In a possible implementation of the above first aspect, the first location range includes a first distance range, and the second location range includes a second distance range; and

The first relative position being in the first position range includes: the first distance between the user and the electronic device corresponding to the first relative position being in the first position range,

The second relative position being in the second position range includes: the second distance between the user and the electronic device corresponding to the second relative position being in the second position range; and

The minimum value of the first distance range is greater than the maximum value of the second distance range, and the second emission volume is greater than the third emission volume.

That is, multiple distance ranges are set for the distance between the user and the electronic device, so that when the distance between the user and the electronic device is large, the ultrasonic emission volume is increased, and when the distance is closer, the ultrasonic emission volume is decreased. This ensures that when the distance between the user and the electronic device changes, the received echo signal can meet the detection requirements and does not cause physical harm to the user in the ultrasonic environment.

For example, the first distance range is "(d1, d2)" below, and the second distance range is "(0, d1)" below. At this time, d1 and d2 increase in sequence.

In a possible implementation of the first aspect above, it also includes:

The first distance of the object relative to the electronic device is represented by the motion component energy of the plurality of echo signals, and a first motion component energy range and a second motion component energy range respectively corresponding to the first distance range and the second distance range are set,

Wherein, the greater the energy of the motion component, the smaller the first distance of the object relative to the electronic device, the maximum value of the first motion component energy range corresponds to the minimum value of the first distance range, and the minimum value of the first motion component energy range corresponds to the first The maximum value of the distance range, the maximum value of the second motion component energy range corresponds to the minimum value of the second distance range, and the minimum value of the second motion component energy range corresponds to the maximum value of the second distance range.

It is understandable that there is a relationship between the motion component energy and the distance between the user and the electronic device. The greater the motion component energy, the smaller the distance between the user and the electronic device. Therefore, the motion component energy range can be used to represent each distance range, which is more convenient according to actual conditions. Adjust the transmit volume according to the situation. There is no need to pre-measure the transmit volume range corresponding to each distance range. In the process of adaptively adjusting the transmit volume, you only need to determine the transmit volume range. Whether the motion component energy of the motion echo signal received after the emission volume is adjusted is within the corresponding volume energy range can determine whether the adjusted emission volume is within the emission volume range corresponding to the distance range.

For example, the first distance range is "(d1, d2)" below, the corresponding first motion component energy range is the distance energy range "(Esd3, Esd2)" below, and the second distance range is "(Esd3, Esd2)" below. (0, d1)", the corresponding energy range of the second motion component is the distance energy range "(Esd2, Esd1)" below, where d1 and d2 increase in sequence, Esd2 and Esd3 decrease in sequence, and the first distance range The minimum value d1 in the corresponding "(d1, d2)" corresponds to the maximum value Esd2 in the first motion component energy "(Esd3, Esd2)", and the maximum value d2 in the first distance range "(d1, d2)" corresponds to The minimum value Esd3 of the first motion component energy; the minimum value 0 in "(0, d1)" corresponding to the second distance range corresponds to the maximum value Esd1 of the second motion component energy "(Esd2, Esd1)", the second distance The maximum value d1 in the range "(0, d1)" corresponds to the minimum value Esd2 of the second motion component energy.

In a possible implementation of the above first aspect, the distance range within which the first distance is located is determined in the following manner:

When the motion component energy is within the first motion component energy range, determine that the first distance is within the first distance range;

When the motion component energy is in the second motion component energy range, it is determined that the first distance is in the second distance range.

In a possible implementation of the above first aspect, the motion component energy of multiple echo signals is calculated in the following manner:

The amplitudes of the motion echo signals among the multiple echo signals are integrated to obtain the motion component energies of the multiple motion echo signals.

It can be understood that by integrating the amplitudes of the motion echo signals among the multiple echo signals, the motion component energy of the multiple motion echo signals can be obtained conveniently and reasonably.

In a possible implementation of the above first aspect, the first position range includes a third direction range, and the second position range includes a fourth direction range; and

The first relative position being in the first position range includes: the user corresponding to the first relative position is located in the third direction range relative to the first direction of the electronic device,

The second relative position being in the second position range includes: the user corresponding to the second relative position is located in the fourth direction range relative to the second direction of the electronic device; and

The minimum value of the third direction range is greater than the maximum value of the fourth direction range, and the second emission volume is greater than the third emission volume.

It is understandable that the propagation of ultrasonic signals has radiation directivity. When playing ultrasonic signals, the energy of the ultrasonic signals propagating directly in front of the electronic device is strong enough to detect the presence of the user, while the energy of the ultrasonic signals propagating to the side of the electronic device is Weak, so that the energy of the echo signal reflected back by the reflecting object is weak and cannot be used to detect the user's presence. Therefore, when the electronic device uses ultrasonic signals to detect the user's presence, the user's direction relative to the electronic device can be measured. The emission volume is adjusted according to the direction of the user relative to the electronic device, so that different directions of the user in front of the electronic device correspond to different emission volumes according to the actual situation, thereby facilitating detection of the user's presence.

For example, the third direction range is the direction range "(0, r1)" below, the second emission volume is the emission volume Vr1 below, the fourth direction range is the direction range "(r1, r2)" below, and the third The emission volume is the emission volume Vr2 below, and Vr2 is greater than Vr1.

In a possible implementation of the first aspect above, it also includes:

When the second emission volume or the third emission volume is greater than the volume threshold, and the historical volume of the ultrasonic signal transmitted in the first historical time period continues to be greater than the volume threshold, the ultrasonic signal is transmitted at the second transmission volume or the third transmission volume next time. When, the duty cycle of the emitted ultrasonic signal is reduced and the detection frame rate is reduced.

It is understandable that the user can be prevented from being in a high-energy ultrasonic environment for a long time by reducing the duty cycle of the transmitted ultrasonic signal and lowering the detection frame rate, that is, reducing the power-on time in one signal cycle of the originally transmitted ultrasonic signal. For example, the power-on time in one signal cycle can be reduced to 50%, thereby ensuring that users are not exposed to high-energy ultrasound environments for long periods of time and improving safety.

In a possible implementation of the first aspect above, it also includes:

Obtaining a captured user image of the user when the plurality of echo signals satisfy a first condition indicating whether a user is present;

When there is a target object in the user image, the fourth transmission volume is used to transmit the ultrasonic signal;

When there is no target object in the user image, the ultrasonic signal is transmitted using a fifth transmission volume, where the fifth transmission volume is greater than the fourth transmission volume.

It can be understood that in the captured user image, different transmission volumes are further set according to the presence or absence of the target image. The transmission volume of the transmitted ultrasonic signal can be adjusted in real time due to different user conditions, further preventing the user from suffering from strong energy ultrasonic signals. harm. For example, the transmission volume of the ultrasonic signal can be adjusted in real time based on the user's age, whether there are pets around the user, etc., so as to ensure that the energy of the echo signal reflected back by the user is sufficient to detect the user's motion, while avoiding the risk of the user, pet body Hazardous by strong energy ultrasonic signals.

In a possible implementation of the above first aspect, the target objects include children and pets.

It is understandable that children and pets are less tolerant of ultrasonic signal energy than adults. If an ultrasonic signal is emitted at a fixed volume during user detection, the volume may be suitable for adults, but it may cause harm to children and pets. Hazards, so take children, pets and pets into consideration and set the emission volume that is different from that of adults, which can prevent the bodies of children and pets from being harmed by strong energy ultrasonic signals.

In the second aspect, embodiments of the present application provide an object motion detection method, which is applied to electronic devices, including:

emit ultrasonic signals;

Obtain multiple echo signals generated after ultrasonic signals reflected by objects received at different times;

Corresponding to the multiple echo signals satisfying the first condition, there is an object moving relative to the electronic device among the objects that reflect the ultrasonic signal;

Corresponding to the multiple echo signals not meeting the first condition, there is no object that moves relative to the electronic device among the objects that reflect the ultrasonic signal;

It can be understood that the object motion detection method provided by the embodiment of the present application determines whether there is a moving object in front of the electronic device based on the phase offset of the echo signal, that is, in the echo signal, if there is a moving object relative to the emission If the phase offset of the ultrasonic signal changes with time for multiple motion echo signals, it is determined that there is a user moving relative to the electronic device; conversely, it is determined that there is no user moving relative to the electronic device. Since the wavelength of the ultrasonic frequency band is short (for example, 20kHz ultrasound has a speed of 340 meters/second in the air and a wavelength of 1.7 cm), the displacement caused by the movement of the object is often greater than the wavelength (such as 1.7 cm). Therefore, when the moving object makes small movements The resulting change in the echo path of the echo signal can lead to a change in the phase offset of the echo signal, so that the movement of the object can be accurately depicted by extracting the phase offset of the echo signal, thereby enabling the detection of motion. Objects exist with higher accuracy.

In a possible implementation of the above second aspect, the motion includes limb motion in which the displacement of the object does not change.

That is, without the object being displaced, the tiny movements of the object's limbs can be detected based on the changes in the phase offset of the echo signal. Compared with the existing solution through ranging and echo signal amplitude integration, The detection accuracy is higher.

In a possible implementation of the above second aspect, the ultrasonic signal includes a continuous ultrasonic signal.

It can be understood that the phase data is easier to extract from the frequency domain data of the continuous ultrasonic signal, which is beneficial to determining the existence of objects moving relative to the electronic device.

In a possible implementation of the above second aspect, transmitting the ultrasonic signal includes:

It can be understood that transmitting multiple ultrasonic signals with different transmit frequencies at the same time can avoid frequency-selective fading of ultrasonic signals, that is, avoid attenuation of multipath signal superposition of ultrasonic signals with certain transmit frequencies at certain spatial locations, resulting in poor reception. The attenuated energy of the received echo signal is too low to be used for phase offset estimation.

In a possible implementation of the above second aspect, the plurality of continuous ultrasonic signals include a first continuous ultrasonic signal and a second continuous ultrasonic signal, and

It can be understood that there is no overlap in the frequency domain between multiple continuous ultrasonic signals, which can ensure that there will be no interference between each carrier frequency, and frequency selective fading will not occur at the same detection position at the same time, so it can be used for the presence of moving objects. detection.

It can be understood that the frequency selective fading of ultrasonic signals is related to the emission source and location of the ultrasonic signal. Therefore, it is easier to ensure that multiple continuous ultrasonic signals are emitted at multiple different transmit frequencies using the same ultrasonic sounder. There must be an ultrasonic signal that does not undergo frequency-selective fading, which is beneficial to the measurement of the presence of objects moving relative to electronic devices. It can be understood that in other embodiments of the present application, different ultrasonic generators can also be used to transmit multiple continuous ultrasonic signals.

In a possible implementation of the second aspect above, the first condition also includes:

The equivalent speed of the object moving relative to the electronic device belongs to the first speed range, and the first speed range does not include 0.

That is, when using multi-carrier frequency continuous ultrasonic signals for user detection, if there are multiple continuous ultrasonic signals without frequency selective fading, calculate the movement speed corresponding to the ultrasonic signal of each transmission frequency, and then find the equation of these speeds. The effective speed represents the final moving speed of the object, and the final moving speed of the object is within the normal speed range of the object relative to the electronic device (i.e., the first speed range). It is considered that the object moving relative to the electronic device exists.

In a possible implementation of the second aspect above, the equivalent speed is obtained in the following way:

Use a preset algorithm to process multiple motion speeds to obtain the equivalent speed of an object moving relative to the electronic device.

In a possible implementation of the second aspect above, the preset algorithm includes at least one of the following:

Calculate the average of multiple movement speeds;

Calculate least squares fits for multiple motion velocities.

In a possible implementation of the above second aspect, the first speed range is 0m/s-5m/s.

It can be understood that the normal movement speed of the human body is generally between 0m/s-5m/s. Therefore, in order to consider the situation where the object moving relative to the electronic device is a human being, the first speed range is set to 0m/s-5m/s. .

In a possible implementation of the second aspect above, the continuous ultrasonic signal includes any of the following:

Sinusoidal ultrasonic signal, frequency modulated continuous ultrasonic signal.

In a possible implementation of the second aspect above, the phase offset of the echo signal is calculated in the following way:

In a possible implementation of the second aspect above, it also includes:

Corresponding to the fact that there is no object moving relative to the electronic device, ultrasonic signals are emitted at multiple set volumes, and the difference between the echo energy of the echo signal and the reference echo energy is selected within the first energy difference range. The volume is set as a predetermined emission volume of the electronic device for emitting ultrasonic signals.

In a possible implementation of the second aspect above, it also includes:

Transmit a first ultrasonic signal at a first transmission volume, and acquire a plurality of first echo signals generated after the first ultrasonic signal is reflected by the object and received within a second time period;

In the case where multiple first echo signals satisfy the second condition, there is an object moving relative to the electronic device;

When the first distance between the moving object and the electronic device is within the first distance range, the second ultrasonic signal is emitted at the second emission volume corresponding to the first distance range next time,

When the first distance between the moving object and the electronic device is within the second distance range, the second ultrasonic signal is emitted at the third emission volume corresponding to the second distance range next time;

Wherein, the minimum value of the first distance range is greater than the maximum value of the second distance range, and the second emission volume is greater than the third emission volume.

It is understandable that if ultrasonic signals are emitted at a fixed emission volume and the relative position between the moving object and the electronic device changes, the energy of the ultrasonic signal received by the moving object may be too high, which may cause harm to the moving object. Or the energy of the ultrasonic signal received by the moving object is weak, and the energy of the echo signal reflected back by the moving object cannot meet the subsequent detection requirements for the presence of the moving object. Therefore, adjusting the next emission volume according to the change in the relative position between the moving object and the electronic device can ensure that the energy of the echo signal reflected back by the moving object is sufficient to detect the movement of the object, while avoiding the moving object from suffering Hazards of strong energy ultrasonic signals.

In a possible implementation of the second aspect above, the second condition includes:

Among the plurality of first echo signals, there are a plurality of first motion echo signals whose phase offsets with respect to the first ultrasonic signal change with time.

In a possible implementation of the second aspect above, it also includes:

The first motion component energy of the plurality of first motion echo signals represents the first distance of the object relative to the electronic device, and a first motion component energy range and a first motion component energy range respectively corresponding to the first distance range and the second distance range are set. The energy range of the second motion component,

Wherein, the greater the energy of the first motion component, the smaller the first distance of the object relative to the electronic device, the maximum value of the first motion component energy range corresponds to the minimum value of the first distance range, and the minimum value of the first motion component energy range corresponds to The maximum value of the first distance range, the maximum value of the second motion component energy range correspond to the minimum value of the second distance range, and the minimum value of the second motion component energy range corresponds to the maximum value of the second distance range.

It is understandable that there is a relationship between the motion component energy and the distance between the user and the electronic device. The greater the motion component energy, the smaller the distance between the user and the electronic device. Therefore, the motion component energy range can be used to represent each distance range, which is more convenient according to actual conditions. To adjust the transmit volume according to the situation, there is no need to pre-measure the transmit volume range corresponding to each distance range. In the process of adaptively adjusting the transmit volume, it is only necessary to determine whether the motion component energy of the motion echo signal received after the transmit volume is adjusted is in the corresponding Volume energy range, you can determine whether the adjusted emission volume is within the emission volume range corresponding to the distance range.

In a possible implementation of the second aspect above, the distance range within which the first distance is located is determined in the following manner:

When the first component energy is within the first motion component energy range, determine that the first distance is within the first distance range;

When the first component energy is within the second motion component energy range, it is determined that the first distance is within the second distance range.

In a possible implementation of the second aspect above, the motion component energies of the plurality of first motion echo signals are calculated in the following manner:

The amplitudes of the plurality of first motion echo signals are integrated to obtain the motion component energy of the plurality of first motion echo signals.

In a possible implementation of the second aspect above, it also includes:

When the second transmission volume is greater than the volume threshold, and the historical volume of the transmitted ultrasonic signal in the first historical time period continues to be greater than the volume threshold, when the ultrasonic signal is transmitted at the second transmission volume next time, the duty of the transmitted ultrasonic signal is reduced. ratio, and reduce the detection frame rate.

In a possible implementation of the above second aspect, the ultrasonic signal is emitted through a speaker of the electronic device, and the echo signal is received through a microphone of the electronic device.

In a possible implementation of the second aspect above, a sports application is installed on the electronic device, and the above method further includes:

Calculate the movement speed of an object moving relative to the electronic device based on the phase offsets of multiple motion echo signals;

Based on the calculated movement speed of the object, the preset operation of the sports application is performed.

In a third aspect, embodiments of the present application provide an object motion detection method, which further includes:

Transmitting an ultrasonic signal at a sixth transmit volume, and acquiring a plurality of echo signals generated after the ultrasonic signal is reflected by the object and received during the second time period;

When multiple echo signals satisfy the first condition, among the objects that reflect the ultrasonic signal, there is an object that moves relative to the electronic device, where the first condition includes: among the multiple echo signals, there is an object that moves relative to the emitted ultrasonic signal. Multiple motion echo signals whose phase offset changes with time;

When the second relative position between the user and the electronic device is in the third position range, the ultrasonic signal is emitted at the sixth transmission volume corresponding to the third position range next time,

When the second relative position between the user and the electronic device is within the fourth position range, the ultrasonic signal is emitted at the seventh transmit volume corresponding to the fourth position range next time.

It can be understood that the object motion detection method provided by the embodiments of the application is not only a solution for detecting the presence of a user in front of an electronic device based on the phase offset of the echo signal relative to the transmitted signal, but also includes other solutions for detecting the user in front of the electronic device based on ultrasonic waves. There are existing solutions, for example, the solution mentioned in the prior art is to use ultrasonic waves to detect the distance between the reflective object and the electronic device, and determine whether there is a user in front of the electronic device based on the distance between the reflective object and the electronic device; or, through The amplitude integration of the Doppler shifted component of the ultrasound determines the presence or absence of a user's program in front of the electronic device.

In these detection methods, if the ultrasonic signal is emitted with a fixed emission volume, when the relative position between the user and the electronic device changes, the energy of the ultrasonic signal received by the user may be too high, which may cause harm to the user's body. Or the energy of the ultrasonic signal received by the user is weak, and the energy of the echo signal reflected back by the user cannot meet the subsequent user presence detection requirements. Therefore, adjusting the next transmission volume according to the change in the relative position between the user and the electronic device can ensure that the energy of the echo signal reflected back by the user is sufficient to detect the user's movement, while avoiding the user's body from being subjected to strong force. Hazards of energy ultrasonic signals.

In a possible implementation of the above third aspect, the third location range includes a third distance range, and the fourth location range includes a fourth distance range; and

The second relative position being in the third position range includes: the second distance between the user and the electronic device corresponding to the second relative position being in the third position range,

The second relative position being in the fourth position range includes: the second distance between the user and the electronic device corresponding to the second relative position being in the fourth position range; and

The minimum value of the third distance range is greater than the maximum value of the fourth distance range, and the sixth emission volume is greater than the seventh emission volume.

In a possible implementation of the third aspect above, it also includes:

The second distance of the object relative to the electronic device is represented by the motion component energy of the plurality of motion echo signals, and a third motion component energy range and a fourth motion component energy range respectively corresponding to the third distance range and the fourth distance range are set. ,

Among them, the greater the energy of the motion component, the smaller the second distance of the object relative to the electronic device, the maximum value of the third motion component energy range corresponds to the minimum value of the third distance range, and the minimum value of the third motion component energy range corresponds to the third The maximum value of the distance range, the maximum value of the fourth motion component energy range corresponds to the minimum value of the fourth distance range, and the minimum value of the fourth motion component energy range corresponds to the maximum value of the fourth distance range.

In a possible implementation of the above third aspect, the distance range within which the second distance is located is determined in the following manner:

When the motion component energy is within the third motion component energy range, determine that the second distance is within the third distance range;

When the motion component energy is in the fourth motion component energy range, it is determined that the second distance is in the fourth distance range.

In a possible implementation of the third aspect above, the motion component energies of multiple echo signals are calculated in the following manner:

In a possible implementation of the above third aspect, the third position range includes a fifth direction range, and the fourth position range includes a sixth direction range; and

The second relative position being in the third position range includes: the user corresponding to the second relative position is located in the fifth direction range relative to the third direction of the electronic device,

The second relative position being in the fourth position range includes: the user corresponding to the second relative position is located in the sixth direction range relative to the fourth direction of the electronic device; and

The minimum value of the fifth direction range is greater than the maximum value of the sixth direction range, and the sixth emission volume is greater than the seventh emission volume.

In a possible implementation of the above third aspect, it also includes:

When the sixth or seventh emission volume is greater than the volume threshold, and the historical volume of the transmitted ultrasonic signal in the first historical time period continues to be greater than the volume threshold, the ultrasonic wave is transmitted at the sixth or seventh transmission volume next time. When the signal is detected, the duty cycle of the emitted ultrasonic signal is reduced and the detection frame rate is reduced.

In a fourth aspect, embodiments of the present application provide an electronic device, including:

one or more processors;

One or more memories; one or more memories store one or more programs. When the one or more programs are executed by one or more processors, the electronic device performs the above-mentioned first aspect and various aspects of the first aspect. Implement any one of the screen display control methods, implement any one of the object motion detection methods of the above second aspect and various implementations of the second aspect, or implement any of the above third aspect and various implementations of the third aspect. An object motion detection method.

In a fifth aspect, embodiments of the present application provide a computer-readable storage medium. Instructions are stored on the storage medium. When the instructions are executed on a computer, they cause the computer to execute any one of the above-mentioned first aspect and various implementations of the first aspect. A screen display control method, performing the second aspect and any one of the object motion detection methods in various implementations of the second aspect, or performing any one of the object motion detection in the above third aspect and various implementations of the third aspect method.

In a sixth aspect, embodiments of the present application provide a computer program product. The computer program product includes instructions that, when executed, cause the computer to perform any screen display control in the above-mentioned first aspect and various implementations of the first aspect. method, execute any one of the object motion detection methods in the above second aspect and various implementations of the second aspect, or execute any one of the object motion detection methods in the above third aspect and various implementations of the third aspect.

Description of drawings

Figure 1A shows a scene diagram of turning on the standby display screen according to some embodiments of the present application;

Figure 1B shows a scene diagram for closing the standby display screen according to some embodiments of the present application;

Figure 2A shows a time domain waveform diagram of a sinusoidal ultrasonic signal according to some embodiments of the present application;

Figure 2B shows a schematic diagram of the change of the transmission frequency f of a sinusoidal ultrasonic signal over time according to some embodiments of the present application;

Figure 2C shows a schematic diagram of the change of the transmission frequency f of a frequency-modulated continuous ultrasonic signal over time according to some embodiments of the present application;

Figure 2D shows a schematic diagram of a rectangular pulse ultrasonic signal according to some embodiments of the present application;

Figure 3 shows a schematic structural diagram of a smart TV 10 according to some embodiments of the present application;

Figure 4 shows a schematic flowchart of a screen display control method according to some embodiments of the present application;

Figure 5A shows a schematic diagram of an interface 501 according to some embodiments of the present application;

Figure 5B shows a schematic diagram of a setting interface 502 according to some embodiments of the present application;

Figure 5C shows a schematic diagram of a standby display interface 503 according to some embodiments of the present application;

Figure 5D shows a schematic diagram of a standby display type interface 504 according to some embodiments of the present application;

Figure 6A shows a graph of the variation of the transmission frequency f over time of a multi-carrier sinusoidal ultrasonic signal according to some embodiments of the present application;

Figure 6B shows a schematic diagram of the change of the transmission frequency f of a multi-carrier frequency modulated continuous ultrasonic signal over time according to some embodiments of the present application;

Figure 6C shows a schematic diagram of a smart TV 10 according to some embodiments of the present application;

Figure 7 shows a schematic flow chart of a method for calibrating the predetermined emission volume of the smart TV 10 according to some embodiments of the present application;

Figure 8 shows a schematic flowchart of adjusting the emission volume by presetting the volume range corresponding to each distance range for the smart TV 10 according to some embodiments of the present application;

Figure 9 shows a schematic flowchart of a smart TV adaptively adjusting the transmission volume according to the distance energy in the echo signal corresponding to the transmission volume according to some embodiments of the present application;

Figure 10 shows a schematic flowchart of adaptively adjusting the transmission volume according to the user's direction relative to the smart TV according to some embodiments of the present application;

Figure 11 shows a schematic flowchart of adaptively adjusting the emission volume when the user moves relative to the smart TV by using the directional energy range to represent the emission volume corresponding to the angular range, according to some embodiments of the present application;

Figure 12 shows a schematic flowchart of adjusting the emission volume according to identifying users as different target objects according to some embodiments of the present application;

Figure 13 shows a schematic software structure diagram of a smart TV 10 according to some embodiments of the present application.

Detailed ways

Illustrative embodiments of the present application include, but are not limited to, a screen display control method, media, and electronic devices.

It should be noted that the technical solution of this application is applicable to various electronic devices with screens, such as smart TVs, smart screens, advertising machines, and portable computers. For convenience of description, the following description takes a smart TV as an example.

Although the description of the present application will be introduced in conjunction with some embodiments, this does not mean that the features of the application are limited to this embodiment. On the contrary, the purpose of introducing the application in conjunction with the embodiments is to cover other options or modifications that may be extended based on the claims of the application. In order to provide an in-depth understanding of the application, the following description contains many specific details. The application may be implemented without these details. Furthermore, some specific details will be omitted from the description in order to avoid confusing or obscuring the focus of the present application. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other.

In the embodiment of this application, "and/or" is just an association relationship describing associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, and A and A exist simultaneously. B, there are three situations of B alone. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.

Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise.

The technical solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

As mentioned before, always-on smart TV screens have the disadvantages of wasting power and reducing screen life. The commonly used solution now is for users to manually set the display time period, or manually click physical buttons and wake up with voice and speak commands. To turn on/off the standby display screen, the user experience is not very friendly. Therefore, a technical solution is needed that can turn on the standby display screen when a user is detected in front of the smart TV, and close the standby display screen when it is detected that the user is not present. For example, in the scene shown in FIG. 1A , the smart TV 10 does not display the standby screen at first. When the user 20 enters the room where the smart TV 10 is located, such as walking from point A to point B, the smart TV 10 can detect that the user 20 is in front of the smart TV 10 There is a user 20, and the standby display screen of the smart TV 10 is turned on. In the scene shown in FIG. 1B , when the user is at point B, the smart TV 10 displays the standby screen. When the user 20 leaves the room, such as walking from point B to point A, the smart TV 10 can detect the user 20 If the user 20 is moving away from the smart TV 10 or there is no presence in front of the smart TV 10, the standby display screen will be turned off.

In some embodiments, the smart TV 10 can determine whether the user 20 exists through the detection results of its own camera 180 or a dedicated sensor, and then control whether the smart TV 10 displays the standby screen. For example, specialized sensors can be ultrasonic transducers, infrared sensors, millimeter wave sensors, etc. However, the method of detecting the presence of a user through the camera 180 not only requires high detection conditions, but also has a high false detection rate; and using a dedicated sensor to detect the presence of a user not only adds extra cost, but also affects the appearance.

In other embodiments, whether there is a user in front of the smart TV 10 may be determined based on ultrasonic ranging technology. For example, the smart TV 10 transmits and receives ultrasonic signals, and calculates the reflection based on the product of the time interval between the transmitted ultrasonic signal (hereinafter referred to as the transmission signal) and the received corresponding ultrasonic signal (hereinafter referred to as the echo signal) and the ultrasonic wave speed. The distance between the ultrasonic reflecting object and the smart TV. If the difference between the distance between the reflecting object and the smart TV calculated based on the echo signals received at the time before and after is greater than the threshold, it is considered that the object exists, and if the presence of the object is detected twice in a row, the smart TV is determined based on the two detection results. Whether there are users before TV 10. However, this technical solution has a large ranging error when the user is far away from the smart TV 10. For example, for a stationary object that is far away from the smart TV 10, such as a sofa, based on the echo signals reflected at all times before and after the stationary object ( The measured distance (hereinafter referred to as the stationary echo signal) may cause a distance difference greater than the threshold due to ranging errors, and may be misjudged as the presence of an object. Therefore, this technical solution has poor accuracy in determining whether the user exists.

In addition, in some other embodiments, the amplitude integration of the Doppler frequency shift component can also be used to detect whether there is a user in front of the smart TV. It can be understood that when the reflecting object of the ultrasonic wave moves relative to the smart TV that emits the ultrasonic wave, according to the Doppler effect, the reflecting object The frequency of the received ultrasonic signal changes with the movement speed of the reflecting object and the angle between the movement direction of the reflecting object and the propagation direction of the ultrasonic signal received by the reflecting object. The solution of using the amplitude integral of the Doppler frequency shift component to detect whether there is a user in front of the smart TV can be:

The time domain data of the ultrasonic signal received by the reflecting object is converted into frequency domain data through Fourier transform. For example, Figure 2A shows the time domain waveform diagram of the sinusoidal ultrasonic signal, and Figure 2B shows the frequency domain after Fourier transformation. Waveform diagram; then integrate the instantaneous amplitude values of the frequency domain data of multiple ultrasonic signals received by the reflecting object to find the peak value of the frequency after the Doppler frequency shift, thereby determining the frequency after the frequency shift occurs, and then Using the calculated frequency magnitude after frequency shift, the moving speed of the reflecting object can be calculated based on the Doppler frequency shift formula. For example, the Doppler shift formula is:

Among them, f is the emission frequency of the ultrasonic signal, f _d is the frequency after frequency shift, θ is the angle between the user's movement direction and the propagation direction of the ultrasonic signal received by the user, v ₀ is the propagation speed of the ultrasonic signal in space , v is the speed of the launching object.

In formula (1), f and v ₀ are known, f _d can be calculated by the above amplitude integration method, and θ can be measured or estimated, so the speed of the reflecting object can be calculated. Then, if the calculated movement speed of the reflecting object is within the normal movement speed range of the human body (such as 0m/s-5m/s), it is considered that there is a user in front of the smart TV 10 .

However, when the user 20 is far away from the smart TV 10, the instantaneous amplitude value of the frequency domain data in the echo signal is weak, and it takes a long time (at least 500 milliseconds) to integrate to find the peak value of the frequency after frequency shift. Obtain more accurate measurement results. Therefore, this method cannot capture relatively brief movements of the user 20 (such as raising the head), and is prone to missed detections.

In order to solve the above problem, embodiments of the present application propose a technical solution for determining whether there is a user in front of the electronic device based on the phase offset of the ultrasonic signal. Specifically, after an electronic device such as a smart TV transmits an ultrasonic signal (i.e., a transmission signal), it receives the echo signal returned after the reflection object reflects the transmission signal, and determines the phase deviation of the echo signal received at different times relative to the transmission signal. Whether the displacement changes, it can be determined whether any of the reflective objects that reflect the ultrasonic signal moves, and then the display mode of the electronic device is determined based on whether there is a moving reflective object. For example, if a reflective object moves, it can be further determined whether the moving reflective object is the user, for example, by combining the images captured by the camera to determine whether it is the user, or by determining whether the speed of the moving reflective object is within the normal range of human movement. Determine whether the moving reflective object is the user. Further, the display mode of the electronic device can be determined based on the determination result of whether the moving reflective object is the user, such as controlling the smart TV to display or turn off the standby screen, controlling the smart TV to switch from a black screen to a bright screen, etc. Since the wavelength of the ultrasonic frequency band is short (for example, 20kHz ultrasound has a speed of 340 meters/second in the air and a wavelength of 1.7 cm), the displacement caused by human body movement is often larger than the wavelength (such as 1.7 cm), so the displacement caused by the small movements of the human body is Changes in the echo path of the echo signal can cause changes in the phase offset of the echo signal, so that the movement of the human body can be accurately depicted by extracting the phase offset of the echo signal.

For example, taking the sinusoidal ultrasonic wave W(t) represented by the following formula (2) as the transmitted signal, the echo signal after being reflected by the reflecting object is W′(t) represented by the formula (3):
W(t)＝A _p sin(2πft-φ ₀ ) (2)

Among them, f is the transmission frequency of sine wave W(t), A _p is the amplitude of sine wave W(t), t is the time variable, φ ₀ is the initial phase, A _P ′ is the echo signal W′(t) Amplitude, c is the speed of sound, s(t) is the echo path, that is, the propagation path of the echo signal.

Among them, the phase offset of the echo signal W'(t) relative to the transmitted signal W(t) can be expressed as formula (4):

It can be seen from formula (4) that the phase offset Δφ(t) is determined by the echo path s(t). When the reflecting object moves, s(t) changes, causing Δφ(t) to change. In other words, if the phase offset Δφ(t) of the echo signal changes, it indicates the presence of a moving reflecting object.

Since the change in the phase offset of the echo signal reflects the speed of the moving object to a certain extent, for example, the speed v(t) of the moving object can be determined based on the above formula (4):

Considering that it is difficult to distinguish between a stationary human body and other stationary objects (such as sofas, coffee tables, etc.) using only distance information, especially at long distances, the ranging error is large and it is impossible to determine whether the reflecting object is a stationary object. However, The amplitude of the human body's small movements is generally greater than the wavelength of the ultrasonic signal (1.7 cm as mentioned above). The changes in the echo path of the echo signal caused by the human body's small movements can lead to changes in the phase offset of the echo signal. , and then the movement speed determined based on the phase offset of the ultrasonic signal can distinguish certain small movements of the stationary human body, such as turning the head and moving the arms of a person sitting on the sofa, so the technical solution of this application is adopted Detecting the presence of a user is more precise. Furthermore, in some embodiments of the present application, the definition of limiting the presence of a user is: if there is at least one movement of the user within a certain period of time, the user is considered to exist. Because under normal circumstances, people will inevitably move within a certain period of time, the aforementioned movements can be relative movements due to changes in the person's position, or slight movements of limbs when the person's position remains unchanged, such as movement. Arms etc. For example, within 10 minutes, the user will not remain motionless, and will make at least one slight movement of the limbs, head, or trunk. Therefore, if the phase offset of the echo signals received by the electronic device at different times within 10 minutes occurs, changes, it can be considered that there is a user in front of the electronic device.

It can be understood that the time domain data of the echo signal generated after the ultrasonic signal is reflected by the reflecting object can be received by a radio equipment (such as the microphone of a smart TV). The time domain data of the received echo signal can be obtained in the frequency domain after Fourier transformation. Data, compared with pulsed ultrasonic signals, it is easier to extract the phase component from the frequency domain data of continuous ultrasonic signals. Therefore, in some embodiments of the present application, continuous ultrasonic signals are used to detect whether there is a user in front of the electronic device.

It can be understood that the continuous ultrasonic wave described in this application is relative to the pulsed ultrasonic signal (as shown in Figure 2D), and refers to an ultrasonic signal with a continuous waveform within the transmission time period, or in one transmission cycle, Ultrasonic signals are continuous, such as sine waves, frequency modulated continuous waves (Frequency Modulated Continuous Wave, FMCW), etc. Figure 2B shows a graph of the variation of the transmission frequency f of a sinusoidal ultrasonic signal with time. As shown in the figure, the transmission frequency f of the sinusoidal ultrasonic signal within the transmission time remains unchanged over time, and the bandwidth is 0. Figure 2C shows the variation of the transmission frequency f of a frequency-modulated continuous ultrasonic signal with time. As shown in the figure, the frequency of the transmission frequency f of the frequency-modulated continuous (FMCW) ultrasonic signal within a transmission period T changes with time. The proportion changes, ranging from fmin to fmax, where fmin is the starting transmission frequency, fmax is the maximum transmission frequency, the bandwidth B is fmax-fmin, and the transmission frequency within a transmission period T is always not zero, that is, during the transmission During the period, the emitted ultrasonic signal is continuous; Figure 2D shows a rectangular pulse ultrasonic signal, in which the pulse width is The pulse amplitude is A and the transmission period is T. It can be seen that within a transmission period T, for example, within the time from 0 to T, from 0 to and Ultrasonic signals are emitted within T, but no ultrasonic signals are emitted during other time periods. That is, the ultrasonic signals emitted within a transmission period T are discontinuous.

It can be understood that the waveform of the continuous ultrasonic signal mentioned in this application is not limited to the waveforms shown in Figures 2A and 2B-2C, as long as it is a continuous ultrasonic signal. For example, the transmission frequency of the FMCW ultrasonic signal can also be a signal whose frequency within a sweep period T decreases proportionally with time. In addition, the changes in the transmission frequency in different frequency sweep periods T can also be different. For example, the transmission frequency in the first frequency sweep period T changes to a signal that decreases proportionally with time, and the transmission frequency in the second frequency sweep period T changes to a signal that decreases proportionally with time. The change is a signal that increases proportionally with time.

As mentioned above, compared with pulsed ultrasonic signals, it is easier to extract phase data from frequency domain data of continuous ultrasonic signals. Therefore, the following will take continuous ultrasonic signals as an example to illustrate the technical solutions of each embodiment of the present application.

Specifically, in some embodiments, the electronic device calculates the phase offset of the echo signal received at different times relative to the transmitted signal by transmitting a continuous ultrasonic signal and extracting the phase component in the received echo signal. Based on the calculated phase offset, it is determined whether there is an echo signal reflected by the moving reflective object (hereinafter referred to as the motion echo signal). In the case where there is a motion echo signal reflected by the moving reflective object, it is determined There is a reflective object that moves relative to the electronic device, and then when the moving reflective object is determined to be the user, the screen display mode of the electronic device is controlled, such as controlling the smart TV to start or close the standby display screen. As mentioned above, the phase offset of the moving echo signal reflected by the moving reflecting object changes with time. In other embodiments, it may be determined based on the calculated phase offset whether there is a motion echo signal reflected by the moving reflective object. In the case where there is a motion echo signal reflected by the moving reflective object, Eliminate the stationary echo signal reflected by the stationary reflecting object from the calculated phase offset, calculate the speed of the reflecting object of the echo signal based on the phase offset of the moving echo signal, and determine the movement in front of the electronic device based on the speed value Whether the reflective object is a user. If it is determined that the movement speed of the reflective object is within the normal range of human movement speed, it is determined that the moving reflective object is a human being, thereby controlling the electronic device to perform a certain screen display method, such as controlling the startup or shutdown of a smart TV. Standby display screen, start or pause the program watched by the user, etc.

Specifically, for example, the smart TV 10 continues to transmit ultrasonic signals when detecting user presence, and at the same time continues to receive echo signals, and performs Fourier transform on the time domain data of the echo signals received every 100 ms to obtain the corresponding frequency. domain data, extract the phase component from the frequency domain data, and then calculate the phase offset of each echo signal relative to the transmitted signal within 100ms. Assume that 500 echo signals are received in the time period from t to t+100ms. The phase offsets of 100 of the echo signals are all Δφ(t ₁ ), and the phase offsets of the 150 echo signals are all Δφ(t ₂ ), then the echo signals corresponding to these two phase offsets are the echo signals reflected by the stationary object, and the phase offsets of the remaining 150 echo signals change with time, then it can These echo signals are considered to be echo signals reflected by moving reflective objects. The velocity of the moving reflecting object can be calculated based on the phase offsets of the remaining 150 echo signals.

It can be understood that in various embodiments of the present application, the bright screen state of the smart TV is different from the display of the standby screen, which refers to the smart TV displaying the startup screen, programs watched by the user, application interfaces opened by the user, etc.

It can be understood that, as mentioned above, the phase offset of the echo signal relative to the transmitted signal is determined by the echo path of the echo signal, and the echo path of the stationary reflecting object that reflects the ultrasonic signal does not change, so the stationary echo The phase offset of the wave signal does not change with time, and the phase offset of the stationary echo signal whose phase offset does not change with time can be eliminated from the calculated echo signal. In addition, since the phase offset of the echo signal relative to the transmitted signal is determined by the echo path of the echo signal, and the position of the stationary reflective object that reflects the ultrasonic signal is fixed, the stationary echo signal reflected by the same stationary reflective object The phase offset of is a value that does not change with time, and the phase offset of the stationary echo signals reflected by different stationary reflecting objects is different. Therefore, different stationary reflecting objects can be determined based on the phase offset. As in the previous example, the reflecting object of 100 echo signals with a phase offset of Δφ(t ₁ ) can be considered to be reflected by the same stationary reflecting object, and the 150 echo signals of a phase offset of Δφ(t ₂ ) can be considered as reflections from the same stationary reflecting object. A reflecting object of a wave signal can be thought of as being reflected by another stationary reflecting object. In the same way, moving reflecting objects can also be determined based on the phase offset.

Specifically, for example, the speaker 171 of the smart TV 10 can play continuous ultrasonic signals with a carrier frequency above 16 kHz (such as sinusoidal ultrasonic signals and FMCW ultrasonic signals, etc.). The continuous ultrasonic signal encounters various reflecting objects in the room to form an echo signal, so that the echo signal received by the microphone 172 of the smart TV 10 includes the phase offset introduced by various reflecting objects relative to the transmitted signal. Obtain the time domain data of the echo signals received at different times, perform Fourier transform on the received time domain data to obtain the frequency domain data, and extract the phase components of the echo signals received at different times from the frequency domain data. Then determine the phase offset of the echo signals received at different times relative to the reflected signal, and eliminate the time-invariant components brought by the stationary reflector from these phase offsets (i.e., the stationary reflector reflected by the stationary reflective object). The phase offset of the echo signal), and then calculate the speed of the user 20 who reflects the echo signal, and when the speed is not zero or is the normal human movement speed, determine that the user 20 exists in front of the smart TV 10; or extract The phase components in the frequency domain data of the echo signals received at different times are calculated, and the phase offsets of the echo signals received at different times relative to the transmitted signals are calculated. Among these phase offsets, there are occurrences that occur over time. In the case of changing phase offsets, it is directly determined that the user 20 exists in front of the smart TV 10, and in the case where the phase offsets of the echo signals returned by each reflecting object do not change over time, it is determined that the smart TV User 20 did not exist before 10. The specific application process will be described in detail below.

It can be understood that the continuous ultrasonic signal can be emitted by the sound-emitting device of the electronic device itself (such as the speaker of a smart TV), or can be emitted by a speaker external to the electronic device. Similarly, it can also be emitted by the sound-receiving device of the electronic device (such as a smart TV). The built-in microphone of the TV) or an external sound collection device (such as an external microphone) receives the echo signal. In some embodiments, the speaker that emits ultrasonic signals and the speaker that emits audible sounds may be the same speaker. That is, the speaker of the electronic device that emits audible audio to the user may also emit ultrasonic waves for detection. In some embodiments, the microphone that receives the ultrasonic signal and the microphone that receives the audible sound may be the same microphone, that is, the microphone of the electronic device that receives the audible sound emitted by the user can also receive the reflected ultrasonic wave. In some embodiments, the speaker that emits the ultrasonic signal and the speaker that emits the audible sound may be different speakers. In some embodiments, the microphone that receives ultrasonic signals and the microphone that receives audible sound may be different microphones.

In addition, it can be understood that in the embodiments of the present application, the technical solution of determining whether there is a moving reflective object in front of the smart TV based on changes in the phase offset of the continuous ultrasonic signal can be applied to other applications in addition to controlling the display of the standby screen. in application scenarios. For example, when the smart TV determines that the user is close to the smart TV, it controls the screen to gradually light up and displays the set content; when it determines that the user is far away from the smart TV, it controls the screen of the smart TV to gradually dim; for another example, the screen will be controlled based on the phase offset. The change in the movement amount determines the movement speed of the user existing in front of the smart TV, which is used for sports application software (Application, APP) of the smart TV, etc. This will be described in detail below.

In addition, it can also be understood that if the sound-emitting device and sound-receiving device of the electronic device itself are used to emit and receive continuous ultrasonic waves to detect the user's presence, there will be no privacy risks involved in using the camera in the previous embodiment, and the detection conditions will not be high. , wide detection range and low cost. In addition, the speed is calculated through the change of the phase offset in the echo signal, unlike the previous embodiment which is based on the echo signal. The integrated value of the amplitude value of the frequency shift signal determines whether the user exists. There is no need to integrate for a long time in time. It can detect short-term human body movements, thereby more accurately determining whether the user exists and improving the user experience. It can better avoid the waste of power consumption caused by the screen being always on when there is no user in the electronic device, and extend the service life of the electronic device.

Figure 3 shows a schematic structural diagram of a smart TV 10 according to some embodiments of the present application. As shown in Figure 3, the smart TV 10 includes a processor 110, a communication module 120, a screen 130, an interface module 140, a memory 150, a power module 160, an audio module 170, a camera 180, and a sensor module 190. The audio module 170 includes a speaker. 171. Microphone 172.

The processor 110 may include one or more processing units, for example, it may include a central processing unit (CPU), an image processor (GPU), a digital signal processor (DSP), or a microprocessor. Processing modules or processing circuits such as processor MCU (Micro-programmed Control Unit), AI (Artificial Intelligence, artificial intelligence) processor or programmable logic device FPGA (Field Programmable Gate Array). Among them, different processing units can be independent devices or integrated in one or more processors. In some embodiments, the processor 110 can detect whether there is a user in front of the smart TV 10 and control the opening or closing of the standby display screen of the smart TV 10 according to the detection results by executing programs related to the screen display control method of the present application.

The communication module 120 may include various wired or wireless communication modules, such as a Bluetooth module (BlueTooth, BT), a wireless local area network (Wireless Local Area Networks, WLAN) module, etc., for providing wireless fidelity (Wireless Fidelity, Wi-Fi) , Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared technology (Infrared, IR), Wired or wireless communication solutions such as Wide Area Network (WAN).

Screen 130 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (Active-matrix Organic Light). -emitting Diode, AMOLED), Flexible Light-emitting Diode (FLED), Mini LED, Micro LED, Micro OLED, Quantum Dot Light-emitting Diodes (QLED), etc. In some embodiments of the present application, the screen 130 can be used to display the user interface of the smart TV 10 to facilitate the user's interaction with the smart TV 10, such as setting the standby display screen in the application.

The interface module 140 may include various forms of input or output interfaces. The smart TV 10 may transmit video and/or audio data to other electronic devices through the output interfaces, and receive video and/or audio data from other electronic devices through the input interfaces. . In some embodiments, the input and output interfaces may include: S/PDIF interface, HDMI interface, LAN (Local Area Networks, LAN) interface, Universal Serial Bus (Universal Serial Bus, USB) interface, AV interface, etc.

The memory 150 can be used to store data, software programs and modules, and can be a volatile memory (Volatile Memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (Non-Volatile Memory), For example, read-only memory (Read-Only Memory, ROM), flash memory (Flash Memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); or a combination of the above types of memory, or It can also be a removable storage medium, such as a Secure Digital (SD) memory card. Specifically, in some embodiments of the present application, the memory may be used to store relevant instructions for executing the screen display control method of the present application.

The power module 160 may include a power button, an IR receiver, etc., for turning on or off the power of the smart TV 10 according to user operations.

The audio module 170 can convert digital audio signals into analog audio signal output, or convert analog audio input into digital audio signals, and can also transmit digital audio signals and/or analog audio signals to other electronic devices through the interface module 140 . In some embodiments, audio module 170 may include speaker 171 and microphone 172 .

Among them, the speaker 171 can be used to emit continuous ultrasonic signals, such as sine waves and/or frequency modulated continuous waves. The built-in speaker 171 of the smart TV 10 may be one or more speakers, and may be one or more of various speaker types such as dynamic coil type and piezoelectric type. In some embodiments, the speaker 171 can also support playback of ultrasonic frequency bands above 16 kHz, which can meet the frequency response characteristics required for ultrasonic detection. When the speaker 171 is a tweeter or a full-range speaker, the playback of audible sound in a higher frequency band can also be achieved. It should be noted that when the speakers 171 are multiple speakers, since the higher the ultrasonic frequency is, the stronger the directivity is, so the speaker 171 on the smart TV 10 can be selected to emit the ultrasonic signal with the sound direction toward the front. In order to ensure that user movement positions in other directions can also be covered by ultrasonic signals, speakers 171 in other sound emitting directions may also be used, for example, speakers 171 located on the left and right sides of the smart TV 10 . It can be understood that multiple speakers can emit ultrasonic signals at the same time. For example, when the smart TV 10 is turned on, the ultrasonic signals are continuously emitted. Detection of user presence, for example, if the power-on time is from 7 to 10 pm, each of the multiple speakers will continuously emit ultrasonic signals from 7 to 10 o'clock. In addition, multiple speakers can also emit ultrasonic signals in different time periods. For example, when the smart TV 10 is turned on, it continuously emits ultrasonic signals to detect the user's presence. For example, if the startup time is still from 7 to 10 pm, multiple speakers can take turns to transmit ultrasonic signals at different times. For example, the speaker 171-1 is at 7 p.m. to 8 o'clock, speaker 171-2 from 8 to 9 o'clock, and speaker 171-3 to emit ultrasonic signals in time intervals from 9 to 10 o'clock. It can be understood that the rotation time of multiple speakers taking turns to emit ultrasonic signals at different time periods can be any time interval actually required, and there is no limit here. For example, each speaker can emit a few seconds, a few minutes, etc.

Since ultrasonic signals can be reflected multiple times by indoor walls, ceilings, floors, etc., in some embodiments of the present application, the smart TV 10 can use the echo signals of all propagation paths for detection, thus further ensuring that the ultrasonic signals in different directions in the room are Reflective objects can be detected.

The microphone 172 may be used to collect and receive continuous ultrasonic signals (ie, echo signals), such as the aforementioned sine waves and/or frequency modulated continuous waves, etc. The built-in microphone 172 of the smart TV 10 is one or more microphones, which can be one or more of various types such as electrets or vector microphones. It supports the reception of ultrasonic signal frequency bands above 16 kHz and has the ability to use ultrasonic signal detection. Whether the user has the required frequency response characteristics and can also receive sounds in the audible frequency range. In the case where there are multiple microphones, the position of the user 20 relative to the smart TV 10 can be determined based on the echo signals collected by the multiple microphones. For example, the phase delay information between the microphones can be determined, and the phase delay information can be used to determine the position of the user 20 relative to the smart TV 10 . The echo signals collected by each microphone are beamformed to obtain the energy value in the preset fixed beam direction. The target beam direction is determined based on the energy value corresponding to each fixed beam direction, and the target beam direction is used as the direction of the user. After the user's direction is determined, the display method of the smart TV can be further controlled based on the relative position of the user and the smart TV. Specific embodiments of determining the display method based on the user's position will be described below. In addition, it can be understood that the method of determining the user's position relative to the smart TV 10 can be any existing technology without limitation. In order to better receive the echo signal, you can avoid selecting the microphone 172 located on the back of the smart TV 10 when receiving the echo signal, and select the microphone 172 located on the front of the smart TV 10 .

Camera 180 is used to obtain still images or videos. The optical image generated by the scene through the lens is projected onto the surface of the image sensor, and then converted into an electrical signal. After analog-to-digital conversion (A/D) conversion, it is converted into a digital image signal and then sent to the digital signal processing chip. Processing. In some embodiments of the present application, the camera 180 may be used for detection of moving objects.

The sensor module 190 may include a magnetic sensor, an acceleration sensor, a temperature sensor, a voice sensor, etc.

It can be understood that the structure of the smart TV 10 shown in Figure 3 is only an example. In other embodiments, the smart TV 10 may also include more or fewer modules, and some modules may also be combined or split. This application The examples are not limiting.

Some embodiments of the present application are described below in conjunction with the hardware structure of the smart TV 10 described in FIG. 3 .

Figure 4 shows a schematic flowchart of a screen display control method according to some embodiments of the present application. This method can be based on the technical solution mentioned above, by determining the change in the phase offset between the echo signal and the transmitted signal received by the smart TV 10 at different times, to determine whether there is a user 20 in front of the smart TV 10, and then control the smart TV 10. Whether the TV 10 displays the standby screen. In some embodiments, the execution subject of this method may be the processor 110 of the smart TV 10 . Specifically, as shown in Figure 4, the method includes:

S401, control the speaker 171 to transmit an ultrasonic signal, and obtain the echo signal received by the microphone 172.

It can be understood that in some embodiments, the smart TV 10 can be controlled to continuously transmit a continuous ultrasonic signal in a standby state, or can transmit a continuous ultrasonic signal at a certain time interval for a period of time, for example, every 1 minute and continue to transmit for 30 minutes. The ultrasonic signal is to transmit a continuous ultrasonic signal for 30 minutes, stop for 1 minute, and then transmit a continuous ultrasonic signal for 30 minutes. The time interval can be determined according to specific circumstances and is not limited here. The transmission frequency can be in the range of 16KHZ-45KHZ, and the type of continuous ultrasonic signal can be the sine wave and/or frequency modulated continuous wave mentioned above.

In addition, in some embodiments, the user needs to enable the smart display function in the smart TV 10 in advance to implement the screen display control method disclosed in the embodiments of the present application. For example, in the above scenarios shown in FIGS. 1A and 1B , the user 20 can enable the smart display function in advance through voice, remote control of the smart TV 10 , etc. For example, FIGS. 5A-5C show a schematic diagram of the process in which the user 20 turns on the smart display function. As shown in the figure, the user 20 can first select the setting application 51 in the interface 501 shown in Figure 5A, enter the setting interface 502 shown in Figure 5B, and select the "standby display" control 52 in the setting interface 502, and enter the setting interface 502 shown in Figure 5B. In the standby display interface 503 shown in 5C, turn on the "standby display" switch control 53 and select the display mode as smart display. In this way, the smart display control function of the smart TV 10 has been turned on, and the smart TV 10 can automatically detect whether the user 20 exists, and confirm whether to display the standby screen based on the detection result.

In addition, as shown in FIG. 5C , the user 20 can also click the "standby display type" control 55 to select the corresponding standby display type. For example, FIG. 5D shows the standby display type interface 504. The user 20 can select types including dial clock, digital clock, artistic style, etc.

S402: Determine whether there is a user in front of the smart TV 10 based on the received echo signal.

It can be understood that, as mentioned above, in some embodiments, the phase offset between the echo signal and the corresponding transmission signal can be calculated by extracting the phase component from the frequency domain data of the echo signal received within a certain period of time. shift, and then determine whether there is a moving object in front of the smart TV 10 based on the phase shift of the echo signals at different times.

For example, assuming that the continuous ultrasonic signal emitted is the sinusoidal ultrasonic signal mentioned above, the sinusoidal ultrasonic signal W(t ₀ ) emitted at time t ₀ is expressed by the following formula (6):
W(t ₀ )=A _p sin(2πft ₀ -φ ₀ ) (6)

According to formula (3), the echo signals W′(t ₁ ) and W′(t ₂ ) corresponding to the transmitted continuous ultrasonic wave W (t ₀ ) received at t ₁ and t ₂ are respectively expressed by the following formulas ( 7) and formula (8) represent:

The phase components φ′(t ₁ ) and φ′(t ₂ ) can be extracted from the frequency domain data of the received echo signals W′(t ₁ ) and W′(t ₂ ), and then the extracted The difference between the phase components φ'(t ₁ ), φ'(t ₂ ) and the phase component φ(t ₀ ) of the continuous ultrasonic wave W(t ₀ ) is used to obtain the echo signals W'(t ₁ ) and W'(t ₂ ) The phase offset amounts are Δφ(t ₁ )=φ′(t ₁ )-φ(t ₀ ) and Δφ(t ₂ )=φ′(t ₂ )-φ(t ₀ ).

Similarly, the phase offsets Δφ( _{t 3} ₎ , Δφ( _{t 4} ₎ , Δφ ₍ t ₅ ),……..

In some embodiments, when determining Δφ(t ₁ ), Δφ(t ₂ ), Δφ(t ₃ ), Δφ(t ₄ ), Δφ(t ₅ ), ... are different or partially different Below, it can be determined that there is user 20 before smart TV 10.

In some other embodiments, the speed of the reflective object moving in front of the smart TV 10 can also be calculated based on the phase offset, and whether the reflective object is the user is determined based on the speed of the reflective object.

For example, the speed of the reflecting object that reflects the echo signals W′(t ₁ ) and W′(t ₂ ) can be obtained according to the following formula (9):

In some embodiments, it can be determined that the user 20 is present in front of the smart TV 10 by determining that the speed of the reflecting object is not 0. In addition, in some other embodiments, since the moving object may be a pet or a sweeping robot or other furniture equipment, when the calculated speed is not 0 and the presence of a moving reflective object is determined, it is also necessary to confirm whether the moving reflective object is a human being. , only when the moving object is determined to be a human being, the user is confirmed to be in front of the smart TV. For example, a camera can be used to determine whether a moving object is a user. For example, in the scenarios shown in Figures 1A and 1B of this application, when the smart TV 10 calculates that the speed of the moving object is not zero based on the phase offset of the echo signals at t ₁ and t ₂ , at this time Turn on the camera 180. If the camera 180 captures a human in the space, it is considered that the user exists. It is understandable that when the smart TV 10 detects a moving object, it turns on the camera 180 to take pictures of the space, which not only reduces the frequency of use of the camera 180 and avoids issues related to user privacy and security, but also makes it easier to determine whether the user is The results are more accurate.

In addition, in some embodiments, when the smart TV 10 calculates the speed based on the phase offset in the echo signal and can determine that there is a moving object, it also needs to determine whether the calculated speed is within the user's normal movement speed range. , when the speed is within the user's normal range of motion, the presence of a moving object is considered to be the presence of the user; when the speed exceeds the user's normal range of motion, it is considered interference. For example, the normal movement speed range of a general user is 0m/s-5m/s. When the movement speed calculated based on the phase offset exceeds the normal movement speed range, it is considered to be interference. For example, it may be interference caused by a pet passing by quickly. wait. For example, in the scene shown in Figure 1A and Figure 1B of this application, when the user 20 is approaching the smart TV 10, the smart TV 10 obtains the speed of the moving object according to the speed calculation formula shown in formula (6) as 3m/ s, the speed is within the user's normal movement speed range of 0m/s-5m/s, then the presence of a moving object is considered to be the presence of the user.

It can be understood that in some embodiments, the echo signals within a certain period of time can be continuously analyzed to determine whether there is movement. Reflective objects. For example, the smart TV 10 continues to emit ultrasonic signals when detecting user presence, and at the same time continues to receive echo signals, and performs Fourier transform on the time domain data of the echo signals received every 100 ms to obtain the corresponding frequency domain data. The phase component is extracted from the frequency domain data, and then the phase offset of each echo signal relative to the transmitted signal within 100ms is calculated. Assume that 500 echo signals are received in the time period from t to t+100ms. The phase offsets of 100 of the echo signals are all Δφ(t ₁ ), and the phase offsets of the 150 echo signals are all Δφ(t ₂ ), if the phase offset of the 150 echo signals changes with time, it can be considered that there is a moving reflective object in front of the TV 10, and then it can be determined whether the reflective object is a user. The fixed time period for analyzing the echo signal can be determined according to the actual situation. For example, it can be any time between 10 ms and 100 ms, or any time between 10 ms and 1 s, without limitation.

It can be understood that although the above example uses a sinusoidal ultrasonic signal as an example, in other embodiments of the present application, the presence of moving objects can also be detected by analyzing the phase offset of other types of continuous ultrasonic signals. For example, for the frequency modulated continuous wave mentioned above, assume that the continuous ultrasonic signal W(t ₀ ) transmitted at time t ₀ is:
W(t ₀ )=cos(2πf _min t ₀ +πBt ₀ ² /T) (10)

The echo signals W'(t ₁₎ and W'(t ₂ ) corresponding to the transmitted continuous ultrasonic wave W(t ₀ ) received after t ₁ and t ₂ have passed are respectively:
W′(t ₁ )=A _P ′cos(2πf _min (t ₀ -t ₁ )+πB(t ₀ -t ₁ ) ² /T) (11)
W′(t ₂ )＝B _P ′cos(2πf _min (t ₀ -t ₂ )+πB(t ₀ -t ₂ ) ² /T) (12)

Among them, A _P ′ and B _P ′ are the echo amplitudes at two moments respectively, f _min represents the starting frequency, B represents the bandwidth, and T represents the frequency sweep period.

Then the echo signal and the transmitted signal are down-mixed and band-pass filtered, and the phase representation of the two moments can be obtained as follows:

The speed of the object that reflects the echo signals W′(t ₁ ) and W′(t ₂ ) can be obtained according to the following formula:

S403: Execute the preset screen display method according to the judgment result.

It can be understood that if it is determined that the user 20 exists in front of the smart TV 10, multiple display modes can be set.

For example, in some embodiments, if it is determined that there is a user 20 in front of the smart TV 10 and the smart TV 10 was previously in a black screen state, the screen 130 of the smart TV 10 can be controlled to display the standby display screen according to the set standby display type and display mode. For example, the standby display type is the dial clock corresponding to the "standby display type" control 55 in Figure 5C, and the standby display mode is the smart display corresponding to the "display mode" control 54 in Figure 5C.

In some other embodiments, if it is determined that there is a user 20 in front of the smart TV 10, and the smart TV 10 previously displayed a standby screen or was in a black screen state, then: the smart TV 10 can be controlled to directly turn on the screen, or display the power-on state while turning on the screen. screen, or while the screen is on, the screen that the user 20 watched before he left (i.e. last entered the standby screen or black screen) is displayed, such as the program that the user 20 paused when he left last time, or while the screen is on, the screen is played while the user 20 last left. The program is paused. It can be understood that in various embodiments of the present application, the programs may be various multimedia data played by smart TVs, such as various types of audio, various types of videos, games, and so on.

In addition, in some other embodiments, when it is determined that the user 20 exists in front of the smart TV 10, the relative direction of the user 20 with respect to the smart TV 10 can also be determined, for example, through the above-mentioned method based on multiple microphones. The direction of the user 20 is determined by beamforming. Then, the screen display mode is determined according to the relative direction of the user 20 relative to the smart TV 10 . For example, when the smart TV 10 previously displayed a standby screen or was in a black screen state, and detects that the user 20 is located directly in front of the TV 10, it directly controls the smart TV 10 to turn on the screen, display the startup screen, or play the program that was paused when the user 20 left last time. ; When the smart TV 10 was previously in a black screen state and it was detected that the user 20 was located on the side of the TV 10, the smart TV 10 was controlled to display the standby screen; when the smart TV 10 was previously displaying the standby screen and it was detected that the user 20 was located on the side of the TV 10 , control the smart TV 10 to turn on the screen and display the startup screen. Among them, users The position 20 directly in front of the TV 10 may be the determined first direction of the user 20 relative to the smart TV 10, and the included angle with the vertical direction of the screen of the smart TV 10 is within the first included angle range, for example, the first included angle Range is 0° to 75°. When the user 20 is located on the side of the TV 10, the angle between the determined first direction of the user 20 relative to the smart TV 10 and the vertical direction of the screen of the smart TV 10 is within the second angle range, such as the second angle range. Angle range is 75° to 90°.

In addition, in some other embodiments, when it is determined that the user 20 exists in front of the smart TV 10 , the movement direction of the user 20 relative to the smart TV 10 can also be determined, that is, whether the user 20 is approaching or moving away from the smart TV 10 . When the screen 130 of the smart TV 10 is in a black screen state and the user 20 is close to the smart TV 10 , the control screen 130 gradually lights up and displays the set content, for example, displays the standby screen, displays the startup screen, and displays the last time the user 20 left The picture previously viewed (such as the pause picture of the program watched by the user 20). When the screen of the smart TV 10 is in a bright screen state or displays a standby screen, and it is determined that the user 20 is away from the smart TV 10 , the control screen 130 gradually dims, and finally enters a black screen state.

In addition, in some embodiments, the speed at which the screen 130 gradually lights up or fades out can also be controlled according to the speed at which the user approaches or moves away from the smart TV 10 . For example, when the user approaches the smart TV 10 at a high speed, the screen 130 is controlled to gradually light up or dim. 130 lights up at a larger speed, and when the user approaches the smart TV 10 at a smaller speed, the control screen 130 lights up at a smaller speed. For example, as mentioned above, the moving speed of the user 20 can be calculated based on the phase offset of the echo signal, and then the speed at which the screen 130 turns on or off can be controlled.

In addition, in some embodiments, after the movement direction of the user 20 relative to the smart TV 10 is determined, the volume of the program played by the smart TV 10 can also be controlled. For example, when the user 20 is determined to be close to the smart TV 10, the volume of the program played by the smart TV 10 can be controlled. The playback volume of the TV 10 is increased, and when it is determined that the user 20 is far away from the smart TV 10, the playback volume of the smart TV 10 is increased.

It can be understood that due to the Doppler shift effect, the frequency of the echo signals received at different times will change. For example, if the frequency of the echo signal continues to increase, it indicates that the user 20 is close to the smart TV 10, and if the frequency of the echo signal continues to decrease, it indicates that the user 20 is far away from the smart TV 10. Therefore, in some embodiments, the echo signal can be determined by Whether the equivalent frequency of the signal continues to increase or decrease is used to determine whether the user 20 is close to the smart TV 10 or far away from the smart TV 10 .

For example, taking the sinusoidal ultrasonic wave W(t) mentioned above as an example, the frequency f'(t) of the echo signal W′(t) can be equivalent to:

Among them, f is the transmission frequency of W(t), s(t) is the echo path, and c is the speed of sound.

It can be seen from the above formula that the frequency f’(t) of the echo signal is determined by the echo path s(t). When the user approaches the smart TV, the echo path s(t) in the echo signal received by the smart TV decreases, so the equivalent frequency of the echo signal increases. When the user moves away from the smart TV, the echo signal received by the smart TV The echo path s(t) in the wave signal increases, so the equivalent frequency of the echo signal decreases.

Therefore, in some embodiments, when it is determined that the user 20 exists in front of the smart TV 10 , changes in the equivalent frequencies of the echo signals at different times can be extracted to determine whether the user 20 is close to or far away from the smart TV 10 .

It can be understood that in other embodiments of the present application, other existing technologies can also be used to determine whether the user 20 is close to or far away from the smart TV 10, and there is no limitation here. For example, the distance between the user 20 and the smart TV 10 can be measured based on the time and sound wave speed of the transmitted and received ultrasonic signals.

Correspondingly, if it is determined that there is no user 20 in front of the smart TV 10, multiple display modes can also be set.

For example, in some embodiments, if it is determined that there is no user 20 in front of the smart TV 10, the screen will remain in a black state or the standby screen.

In some other embodiments, if it is determined that there is no user 20 in front of the smart TV 10 and the smart TV 10 is currently in the standby screen, then based on the historical echo signal or the result of determining whether the user 20 exists based on the historical echo signal, determine Whether the user 20 continues to be absent within a certain period of time (such as more than 30 minutes). If it is determined that the user 20 continues to exist within a certain period of time, the screen 130 of the smart TV 10 is controlled to enter a black screen state.

In addition, in some other embodiments, if it is determined that there is no user 20 in front of the smart TV 10 and the smart TV 10 is currently in a bright screen state, such as playing a program, then there will be no user 20 for a certain period of time (such as more than 30 minutes). 20, the screen 130 of the smart TV 10 is controlled to pause the program playback, or display a standby screen or enter a black screen state. In addition, if it is determined that there is no user 20 in front of the smart TV 10 and the smart TV 10 is currently in a bright-screen state, such as playing a program, you can also control the smart TV 10 to pause the program first, and then determine within a certain period of time (such as 10 minutes). If the user 20 still does not exist, the control screen 130 enters the standby screen, and then if it is determined that the user 20 still does not exist within a certain period of time (such as 10 minutes), the control screen 130 enters a black screen state.

In addition, in some situations, if the user appears and disappears in front of the smart TV 10, it cannot be because the user is absent for a short period of time. The standby display screen is turned off. Therefore, before the judgment, the smart TV 10 continues to display the standby screen for a certain period of time and the user 20 continues not to be present. Then, corresponding to the judgment result that there is no user 20 before the current smart TV 10, control the smart TV 10. Screen 130 closes the standby display screen and enters a black screen state.

It can be understood that the above execution order of S401 to S403 is only an example. In other embodiments, other execution orders may also be adopted, and some steps may also be split or combined, which is not limited here.

In addition, it can be understood that the ultrasonic signal will have a multipath effect on the transmission and reception paths, that is, the ultrasonic signal will be reflected multiple times during its propagation in the room. During the transmission process, the ultrasonic signal will produce frequency-selective fading, that is, a certain Ultrasonic signals with certain transmission frequencies will be attenuated by the superposition of multipath signals at certain spatial locations, resulting in the attenuated energy of the received echo signal being too low to be used for phase offset estimation. Since in a fixed space, the frequency range in which ultrasonic signals emitted by the transmitting source at the same location fade is generally within a certain or certain frequency ranges, so in order to solve this problem, in some embodiments of the present application, multiple carrier frequencies can be used. Ultrasonic signals are used to measure user presence, that is, ultrasonic signals of multiple frequencies are sent at the same time to ensure that ultrasonic signals with certain frequencies will not undergo frequency selective fading at the same time as other frequencies, and can be used for user presence detection. The following will still take the continuous ultrasonic signal as an example for explanation.

In addition, it can be understood that in order to ensure that multiple carrier frequency ultrasonic signals do not interfere with each other, it can be set that in the multi-carrier frequency continuous ultrasonic signal, the frequency difference between adjacent continuous ultrasonic waves in two frequency domains is greater than these two continuous ultrasonic waves. bandwidth. That is, there is no overlap in the frequency domain between ultrasonic waves in multi-carrier frequency continuous ultrasonic waves.

For example, as shown in Figure 6A, the smart TV 10 can transmit sinusoidal ultrasonic signals with frequencies of 20kHz, 21kHz, 22kHz, ... for user presence detection. Since the bandwidth of the sinusoidal ultrasonic signal is close to zero, it transmits sinusoidal signals with a difference of 1kHz. Ultrasonic signals can ensure that there is no interference between each carrier frequency and that frequency selective fading does not occur at the same detection position at the same time, and can be used for user presence detection.

For another example, as shown in Figure 6B, the smart TV 10 can transmit frequency modulated continuous ultrasonic signals with frequencies of 20kHz-21kHz, 22kHz-23kHz, 24kHz-25kHz, ... for user presence detection. The frequency modulated continuous ultrasonic signal in the figure is The bandwidth is 1kHZ, and the difference in transmission frequency between FM continuous ultrasonic signals is 2kHz. For example, the difference between the initial transmission frequency of a continuous ultrasonic signal with a transmission frequency of 20kHz-21kHz and a continuous ultrasonic signal with a transmission frequency of 22kHz-23kHz is 20kHz and 22kHz. 2kHz, the difference between the maximum transmission frequency of 21kHz and 23kHz is 2kHz, both are larger than the bandwidth of the frequency modulated continuous ultrasonic signal 1kHZ, so that there is no overlap in the frequency domain of each frequency modulated continuous ultrasonic signal in the multi-carrier frequency modulated continuous ultrasonic signal, ensuring that each carrier There will be no interference between frequencies, and frequency selective fading will not occur at the same detection location at the same time, so it can be used for user presence detection.

It can be understood that the frequency selective fading of ultrasonic signals is related to the emission source of the ultrasonic signal and the location of the emitter. For example, continuous ultrasonic signals of the same transmission frequency are emitted by emission sources located at different locations. There may be an emission source at a certain location. The transmitted continuous ultrasonic signal has frequency-selective fading at the reflector, while the continuous ultrasonic signal emitted by the transmitting source at another location does not have frequency-selective fading at the reflector.

For example, the following discusses the signal superposition at the reflector. In Figure 6C, the speaker 171A located at P1 on the left side of the smart TV 10 will not emit a 20kHz sinusoidal ultrasonic signal. Frequency selective fading will not occur, but a 21kHz sinusoidal ultrasonic signal will. Frequency selective fading occurs, and the speaker 171B located at P2 on the right side of the smart TV 10 emit a 20kHz sinusoidal ultrasonic signal will undergo frequency selective fading. At this time, if the frequencies of the sinusoidal ultrasonic waves emitted by the multi-carrier frequency sinusoidal ultrasonic waves are set to 20 kHz and 21 kHz respectively, and the sinusoidal ultrasonic waves of both frequencies are emitted by the speaker 171A, then the echo of the sine ultrasonic wave of 20 kHz without frequency selective fading can be used. wave signal for user presence detection. If the speaker 171A is configured to transmit a 21 kHz sinusoidal ultrasonic signal, and the speaker 171B is configured to transmit a 21 kHz sinusoidal ultrasonic signal, the received echo signal cannot be used because both the sinusoidal ultrasonic signals emitted by the two speakers undergo frequency selective fading. Perform user presence detection. Therefore, in some embodiments, when using multi-carrier frequency continuous ultrasonic signals for user presence measurement, you may choose to use a transmitting source at the same location to transmit multi-carrier frequency continuous ultrasonic signals.

However, in order to avoid the above-mentioned problem of frequency-selective fading of continuous ultrasonic signals caused by using multi-carrier frequency continuous ultrasonic signals emitted from different locations, the multi-carrier frequency continuous ultrasonic signal can be set to include multiple frequency bands. Continuous ultrasonic signals, and each transmitting source at different locations can transmit continuous ultrasonic signals in multiple frequency bands, thereby ensuring that the transmitting sources at different locations have continuous ultrasonic signals in frequency bands where frequency selective fading does not occur for users. detection. For example, in the above example, the smart TV 10 can be set to emit sinusoidal ultrasonic signals with frequencies of 20kHz, 21kHz, 22kHz, and 23kHz respectively. The speaker 171A emits the sinusoidal ultrasonic signals of 20kHz and 21kHz, and the speaker 171B emits the sinusoidal ultrasonic waves of 22kHz and 23kHz. signal, in which neither the 20kHz sinusoidal ultrasonic signal emitted by speaker 171A nor the 22kHz sinusoidal ultrasonic signal emitted by speaker 171B has frequency selectivity. Fading, it can ensure that the sinusoidal ultrasonic signals emitted by the two speakers have echo signals that can be used for user presence detection.

If there are multiple frequency bands of continuous ultrasonic signals in the multi-carrier frequency continuous ultrasonic signal without frequency selective fading, the equivalent value of the reflecting object can be calculated based on the phase offset of the echo signals of multiple frequency bands received at the same time. speed, and then determine whether there is a moving object in front of the smart TV based on the equivalent speed. For example, the phase components can be extracted from the echo signals of continuous ultrasonic waves in multiple frequency bands received at the same time, and multiple phase offsets corresponding to the echo signals in each frequency band can be obtained based on the extracted phase components. Calculate the average of the velocities corresponding to multiple phase offsets that change over time, or use algorithms such as the least squares method to obtain the corresponding equivalent velocity.

For example, in the above example, the 20kHz and 21kHz sinusoidal ultrasonic signals emitted by the speaker 171A located at P1 on the left side of the smart TV 10 in Figure 6C, and the 22kHz and 23kHz sine wave signals emitted by the speaker 171B located at P2 on the right side of the smart TV 10. If the ultrasonic signal does not undergo frequency selective fading, then after the smart TV 10 controls the microphone 172 to receive the corresponding echo signal, it can use a band-pass filter, or mix the echo signal and then perform low-pass filtering to obtain The echo signals W 1 ′ corresponding to the 20kHz, 21kHz, 22kHz, and 23kHz sinusoidal ultrasonic signals W ₁ (t), W ₂ (t), W ₃ (t), and W ₄ (t) emitted by the speakers 171A and _171B respectively. (t), W ₂ ′(t), W ₃ ′(t), W ₄ ′(t). Extract the phase components in the frequency domain data of the echo signals W ₁ ′(t), W ₂ ′(t), W ₃ ′(t), and W ₄ ′(t), and obtain the corresponding values of these four frequency bands based on the phase components. Speed, which is more accurate based on the estimation formula, such as the averaging formula or the least squares fitting formula.

For example, taking the corresponding echo signals of the sinusoidal ultrasonic signals of 20kHz, 21kHz, 22kHz, and 23kHz in the four frequency bands transmitted above as an example, a more accurate speed can be estimated by averaging.

Assume that the phase components of the sinusoidal ultrasonic signals W ₁ ( _{t 0} ₎ , W ₂ (t ₀ ), W ₃ (t ₀ ), and W ₄ (t ₀ ) emitted at time t 0 are φ ₁ (t ₀ ) and φ ₂ respectively. (t ₀ ), φ ₃ (t ₀ ), φ ₄ (t ₀ ), the corresponding echo signals in the four frequency bands at time t ₁ are W ₁ ′(t ₁ ), W ₂ ′(t ₁ ), The phase components in W ₃ ′(t ₁ ) and W ₄ ′(t ₁ ) are: φ ₁ ′(t ₁ ), φ ₂ ′(t ₁ ), φ ₃ ′(t ₁ ), φ ₄ ′(t ₁ ), and then calculate the velocities v ₁ ′(t ₁ ), v ₂ ′(t ₁ ), v ₃ ′(t ₁ ), and v ₄ ′(t ₁ ) corresponding to each frequency band based on the phase components.

Then according to the following mean calculation formula (17), the equivalent speed AveΔv(t ₁ ) can be obtained,

Similarly, the equivalent velocity AveΔv(t ₂ ) of the echo signals in the four frequency bands at time t ₂ can be obtained, and then whether AveΔv(t ₁ ) and AveΔv(t ₂ ) are determined.

It can be understood that the equivalent velocity estimated by taking the average value is located between the velocity values corresponding to the four frequency bands, making the estimated velocity more accurate, thereby avoiding the phase difference caused by a certain frequency band being affected by the multipath effect. The offset error is large, resulting in a misjudgment of the presence of the user through the phase offset.

As described above, after turning on the smart display function, the smart TV can transmit continuous ultrasonic signals of a preset type (such as a sinusoidal ultrasonic signal) and a predetermined transmission frequency to detect the presence of the user. In order to make the energy of the received echo signal (hereinafter referred to as echo energy) is suitable for the detection of user presence, and the volume of the emitted continuous ultrasonic wave needs to be preset (hereinafter referred to as predetermined emission volume). Since different electronic equipment hardware (such as speakers, microphones, etc.) have different working performances, the predetermined emission volumes applicable to different electronic equipment are also different.

For example, a smart TV manufacturer can set a predetermined emission volume suitable for each smart TV produced, thereby ensuring that the smart TV shipped from the factory has a predetermined emission volume suitable for itself. However, if it is used for a long time after leaving the factory, the hardware performance of the smart TV will change, and the predetermined emission volume set before leaving the factory will no longer apply to the smart TV. For smart TVs that have shipped from the factory but have not been set with a predetermined emission volume, in some embodiments, a smart TV with better working performance can be used as a reference device, a predetermined emission volume suitable for the reference device can be set, and then the predetermined emission volume of the reference device can be set. The transmit volume is the predetermined transmit volume for these factory-ready smart TVs. However, since the hardware performance of some smart TVs that have been shipped is quite different from that of the benchmark device, the predetermined emission volume applicable to the benchmark device does not apply to these smart TVs that have been shipped.

Therefore, some embodiments of the present application provide a calibration method for predetermined transmission volume. By comparing the energy of the echo signal received by the smart TV with the reference echo energy of the reference device, the calibration method suitable for the smart TV is determined. Predetermined transmit volume. Specifically, FIG. 7 shows a method for calibrating the predetermined transmission volume of the smart TV 10 shown in FIG. 3 . As shown in Figure 7, the method includes:

S701, detect whether the switch of the intelligent display function is turned on.

If it is detected that the smart display function of the smart TV 10 has been turned on, you can enter S702 and proceed in the absence of the user. Schedule the calibration of the transmit volume; otherwise, you can wait for the user to turn on the smart display function switch before calibrating. In other embodiments, the smart display function can be automatically turned on after a period of use to calibrate the predetermined emission volume.

S702, determine whether the user exists.

In some embodiments, the calibration of the predetermined emission volume of the smart TV 10 can be performed in the absence of the user, so as not to affect the user's normal use of the smart TV 10 . For example, the technical solution mentioned above can be used to determine whether the user exists by using the phase offset of the echo signal, or other existing technical solutions can be used to determine whether the user exists. The specific method is not limited.

After the smart TV 10 determines that the user does not exist, it officially starts the calibration of the scheduled transmission volume and enters S703. Otherwise, repeat S702.

It can be understood that in other embodiments, it is also possible to calibrate the predetermined transmission volume directly within a period of time (such as after the smart TV 10 has been used for a certain period of time) without determining whether the user exists.

S703, control the speaker 171 to transmit a continuous ultrasonic signal at the currently set transmission volume V1 of the smart TV 10, and control the microphone 172 to receive the echo signal to obtain the echo energy E1 corresponding to the volume V1.

It can be understood that the echo energy E1 can be obtained by integrating the amplitudes in the frequency domain data of multiple echo signals received by the smart TV within a certain period of time. For example, from the frequency domain data of the echo signals within 30 ms, The amplitude is integrated. For example, taking the previously mentioned sinusoidal ultrasonic wave W(t)=A _p cos(2πft-φ ₀ ) as the transmitted signal, the echo energy E1 can be the corresponding multiple echo signals received from time t ₃ to t ₄ . The integral of the amplitude _AP ′, assuming that the echo signals received at different times in this time period have different _AP ′, then E1 is:

It can be understood that, similarly, the reference echo energy E0 mentioned in the embodiment of the present application can also be calculated in the same way. In addition, the length of the integration time period can be set according to actual needs, for example, it can also be 10ms, 100ms, etc., and is not limited here.

S704: Determine whether the difference between the echo energy E1 and the reference echo energy E0 is within the first energy difference range.

If the difference between E1 and E0 is within the first energy difference range, it means that the volume of the transmitted signal is V1, which can meet the detection of user presence. Enter S706 and the calibration is completed. Otherwise, enter S705 to adjust the size of V1. .

S705, adjust the transmit volume V1.

For example, in some embodiments, when E1 is greater than E0, the transmission volume V1 is turned down, and when E1 is less than E0, the transmission volume V1 is turned up.

S706, calibration completed.

It is understandable that the number of scheduled transmission volume calibrations for smart TVs 10 is generally small. Generally, it is adjusted once after leaving the factory, or it is set to be adjusted once after a fixed period of time. The adjustment process does not require user intervention, and the user experience is relatively low. good.

As mentioned above, ultrasonic signals can be used to detect whether there is a user in front of the smart TV. For example, detect the distance between the reflecting object and the smart TV through ultrasonic waves, and determine whether there is a user in front of the smart TV based on the distance between the reflecting object and the smart TV; or, through the amplitude integration of the Doppler frequency shift component of the ultrasonic wave, or The change in the phase offset of the echo signal relative to the transmitted signal determines whether there is a user in front of the smart TV. In these detection solutions, if the ultrasonic signal is emitted with a fixed emission volume, when the relative position between the user and the smart TV changes, the energy of the ultrasonic signal received by the user may be too high, which may cause harm to the user's body. Or the energy of the ultrasonic signal received by the user is weak, and the energy of the echo signal reflected back by the user cannot meet the subsequent user presence detection requirements.

For example, when a user is detected in front of a smart TV and the distance between the user and the smart TV changes, if the smart TV emits an ultrasonic signal with a fixed transmission volume, the energy of the ultrasonic signal received by the user will increase with the distance. large and reduced. Specifically, when the user is close to the smart TV, the energy of the ultrasonic signal received by the user is relatively large, which may cause harm to the user's body. For example, if the user exceeds the safety threshold (such as the recommended upper limit of the intensity of ultrasonic waves with a frequency of 20 kHz) 75dB) ultrasonic waves can cause physical harm to humans. When the user is far away from the smart TV, the energy of the echo signal reflected back by the user received by the smart TV is small, and the energy of the received echo signal cannot meet the subsequent detection needs of the user's presence.

For another example, the propagation of ultrasonic signals has radiation directivity, that is, the energy of the emitted ultrasonic signal is stronger in some propagation directions and weaker in some propagation directions. For example, when a smart TV plays an ultrasonic signal through a speaker, the energy of the ultrasonic signal propagated directly in front of the smart TV is strong enough to detect the presence of the user, while the energy of the ultrasonic signal propagated to the side of the smart TV is weak, causing reflections from reflective objects. The energy of the returned echo signal is weak and cannot be used for user presence detection.

In addition, children and pets are less tolerant of ultrasonic signal energy than adults. If an ultrasonic signal is emitted at a fixed volume during user detection, the volume may be suitable for adults, but may cause harm to children and pets.

In order to solve the above problems, some embodiments of the present application provide an adaptive adjustment method for ultrasonic detection volume, which can be adjusted in real time according to changes in the relative position between the user and the smart TV, the user's age, whether there are pets around the user, etc. Adjust the emission volume of the transmitted ultrasonic signal to ensure that the energy of the echo signal reflected back by the user is sufficient to detect the user's movement, while preventing the body of the user and pet from being harmed by the high-energy ultrasonic signal.

It can be understood that the adaptive adjustment method of the present application is applicable to various types of ultrasonic signals, such as continuous ultrasonic signals and pulsed ultrasonic signals. The adaptive adjustment method of this application is also applicable to various user presence detection technical solutions. As mentioned above, the distance between the reflective object and the smart TV is detected through ultrasonic waves, and the smart TV is determined based on the distance between the reflective object and the smart TV. Whether there is a user's technical solution in front of the TV; or, whether there is a user's technical solution in front of the smart TV through the amplitude integration of the Doppler frequency shift component of the ultrasonic wave; or based on the change in the phase offset of the echo signal relative to the transmitted signal Determine whether the user's technical solution exists in front of the smart TV.

For example, in some embodiments, when a smart TV uses ultrasonic signals to detect user presence, if a user is detected in front of the smart TV, if the distance d between the user and the smart TV is measured to be small, the ultrasonic wave is reduced. The transmission volume of the signal is to ensure that the energy of the ultrasonic signal received by the user in the ultrasonic environment is below the safety threshold, and when setting the reduced volume, it is considered that the received echo signal can meet the detection needs of the user; if When the distance d between the user and the smart TV is measured to be large, the volume of the transmitted ultrasonic signal is increased to ensure that the energy of the received echo signal is sufficient for detection of the user's presence. After setting the increased transmission volume value When doing so, you need to ensure that the increased volume will not be too high, causing physical harm to users in the ultrasonic environment. The specific solution will be introduced in detail below in conjunction with Figures 8 and 9.

In addition, in some other embodiments, when the smart TV uses ultrasonic signals to detect user presence, the direction of the user relative to the smart TV can be measured, for example, through the beam forming method based on multiple microphones mentioned above. Determine the user's orientation relative to the smart TV. The transmit volume is then adjusted based on the user's orientation relative to the smart TV. For example, when it is measured that the user is located directly in front of the smart TV, a smaller transmitting volume is used to transmit the ultrasonic signal. When it is measured that the user is located to the side of the smart TV, a larger transmitting volume is used to transmit the ultrasonic signal to ensure that it can accurately Detects the movement of the user located on the side of the smart TV. For example, the front direction may be the determined first direction of the user relative to the smart TV, and the angle with the vertical direction of the smart TV screen is within the first included angle range. For example, the first included angle range is 0° to 75°. °. When the user is located on the side of the smart TV, the angle between the determined reflecting object and the vertical direction of the smart TV screen is within the second angle range relative to the smart TV. For example, the second angle range is 75° to 90°. The specific solution will be introduced in detail below in conjunction with Figure 10.

In addition, in some other embodiments, when the smart TV uses ultrasonic signals to detect user presence, when it is detected that there is a user moving relative to the smart TV, a photo of the user can be taken through the camera of the smart TV, and the image can be Recognition technology identifies whether the user in the photo is an adult or a child, and whether there are pets around the user. If it is determined that the user is a child or there is a pet around the user, the ultrasonic signal can be transmitted using a smaller transmission volume than if the user is an adult. The specific solution will be introduced in detail below in conjunction with Figure 12.

The following describes a technical solution for adaptively adjusting the emission volume of ultrasonic waves as the distance d between the reflecting object and the smart TV changes according to the embodiment of the present application.

Specifically, in some embodiments, multiple distance ranges can be set for the distance d between the user and the smart TV, and a corresponding transmission volume range is set in advance for each distance range. The distance d between the user and the smart TV When located in a certain distance range, adjusting the emission volume of the ultrasonic signal emitted by the smart TV within the range of the emission volume corresponding to the distance range can ensure that the received echo signal can meet the detection requirements and is suitable for those in an ultrasonic environment. The user does not cause physical harm, that is, the ultrasound received by the user is below the safety threshold.

It can be understood that in some embodiments of the present application, the transmission volume range or transmission volume corresponding to each distance range can be determined in advance through testing, that is, it is determined in advance which transmission volume or which transmission volume is used when the user is in a certain distance range. It can ensure that the received echo signal can meet the detection requirements without causing physical harm to users in the ultrasonic environment. In some embodiments, when setting the transmission volume range or the transmission volume corresponding to the distance range, the greater the distance d, the greater the transmission volume corresponding to the distance range where the distance d is located. That is, as the distance between the user and the smart TV increases, The increase in distance increases the emission volume of ultrasonic waves.

For example, Table 1 below shows an example of the transmission volume range corresponding to different distance ranges between the user and the smart TV.

Table 1

Among them, d1, d2, d3, and d4 increase in sequence, and Vd1, Vd2, Vd3, and Vd4 increase in sequence. It can be understood that when the emission volume of the ultrasonic signal emitted by the smart TV is adjusted to the emission volume range corresponding to the distance range d, it can ensure that the reflecting object is not harmed by the ultrasonic signal, and the ultrasonic signal reflected by the reflecting object can Meet the needs of users for presence detection. For example, when the distance d between the reflecting object and the smart TV is in the distance range (d1, d2), if the emission volume of the ultrasonic signal emitted by the smart TV is adjusted to be within the emission volume range (Vd1, Vd2), it can ensure that the reflecting object is not affected by the The harm of ultrasonic signals, and the ultrasonic signals reflected back by reflective objects can meet the user's needs for presence detection.

In addition, as mentioned above, a corresponding transmission volume can also be set for different distance ranges. For example, Table 2 below shows an example of the transmission volume corresponding to different distance ranges between the user and the smart TV.

Table 2

Among them, d1, d2, d3, and d4 increase in sequence, and Vd1', Vd2', Vd3', and Vd4' increase in sequence. It can be understood that when the emission volume of the ultrasonic signal emitted by the smart TV is adjusted to the emission volume corresponding to the distance range d, it can ensure that the reflecting object is not harmed by the ultrasonic signal, and the ultrasonic signal reflected by the reflecting object can satisfy Users have detection needs. For example, when the distance d between the reflecting object and the smart TV is within the distance range (d1, d2), if the emission volume of the ultrasonic signal emitted by the smart TV is adjusted to Vd2', it can ensure that the reflecting object is not harmed by the ultrasonic signal, and The ultrasonic signal reflected by the reflective object can meet the user's needs for presence detection.

Specifically, FIG. 8 shows a schematic flowchart of a smart TV adaptively adjusting the emission volume of an ultrasonic signal transmitted by the smart TV according to the distance between the reflecting object and the smart TV according to some embodiments of the present application, in which the execution subject may be The processor 110 in the smart TV 10.

It can be understood that in the embodiment shown in FIG. 8 , in the process of adaptively adjusting the volume, the transmission volume is adjusted by presetting the volume range corresponding to each distance range.

In addition, it can be understood that the adjustment process as shown in Figure 8 below can be performed multiple times as the distance d changes during the user's movement, such as continuously or once every predetermined time interval (such as 100ms).

Specifically, as shown in Figure 8, the method includes:

S801, detect whether the switch of the smart display function of the smart TV is turned on.

If it is detected that the smart display function of the smart TV has been turned on, you can enter S802 to detect whether there is a user; otherwise, you can wait for the user to turn on the smart display function switch before adaptively adjusting the emission volume V2.

S802, determine whether the user exists.

It can be understood that in this embodiment, the presence of the user can be detected through various existing technical means of detecting the user's presence through ultrasonic waves. For example, the distance d between the user and the smart TV is detected through ultrasonic waves mentioned above, and based on The distance d determines whether there is a user in front of the smart TV; or, determines whether there is a user in front of the smart TV through the amplitude integration of the Doppler Shift component of the ultrasonic wave; or, based on the phase shift of the echo signal relative to the transmitted signal The change in quantity determines whether there is a user in front of the smart TV, as shown in the embodiment of Figure 4.

If there is a user in front of the smart TV, enter S803 to adaptively adjust the volume; otherwise, repeat S802.

S803, control the smart TV to transmit ultrasonic signals at the currently set transmission volume Vd, and obtain the echo received after reflection by the user. wave signal.

For example, in the scene shown in FIGS. 1A-1B , the speaker 171 of the smart TV 10 can be controlled to transmit an ultrasonic signal at the currently set transmission volume Vd, and the microphone 172 of the smart TV 10 can be controlled to receive the echo signal. Wherein, when the ultrasonic signal is transmitted for the first time, the currently set transmission volume Vd may be the predetermined transmission volume V1 mentioned above. In the subsequent process, the currently set transmission volume Vd is the volume adjusted after the previous execution of S806.

S804: Determine the distance d between the user and the smart TV based on the received echo signal.

It can be understood that in various embodiments of the present application, various technical means can be used to determine the distance d between the user and the smart TV, which is not limited here. For example, the distance d between the user and the smart TV can be directly measured based on the time and sound wave speed of the transmitted and received ultrasonic signals.

S805: Obtain the emission volume range corresponding to the distance range of distance d.

For example, as shown in Table 1 above, assuming that the calculated distance d between the user and the smart TV belongs to the distance range (d1, d2), the corresponding emission volume range is (Vd1, Vd2).

It can be understood that in other embodiments, in a solution where a single or multiple volumes are set corresponding to the distance range, it is also possible to obtain a single transmission volume or multiple transmission volumes corresponding to the distance range where the distance d is located. size. For example, as shown in Table 2, when the range of the measured distance d between the user and the smart TV is (d1, d2), the corresponding emission volume is Vd2'.

S806: Determine the next emission volume Vd to be a volume value within the volume range corresponding to the distance range d.

For example, distance d belongs to the distance range (d1, d2), and the corresponding transmission volume range is (Vd1, Vd2). If the volume Vd is greater than Vd2, the next time the ultrasonic signal is transmitted, Vd is reduced by the transmission volume range (Vd1 , any value in Vd2), if Vd is less than Vd1, the volume Vd will be increased the next time the ultrasonic signal is transmitted to any value in the transmission volume range (Vd1, Vd2).

In addition, it can be understood that when the distance energy Esd is used to represent the distance d between the user and the smart TV,

It can be understood that since the above processes S801 to S806 are performed multiple times as the distance d changes during the user's movement, the emission volume can be adjusted in real time according to the change in the distance between the user and the smart TV, so as to achieve the desired effect within the distance between the user and the smart TV. When the smart TV is far away, the ultrasonic emission volume is increased. When the user is closer to the smart TV, the ultrasonic emission volume is lowered to achieve adaptive adjustment of the emission volume. This ensures that the energy of the echo signal reflected back by the user is sufficient to detect the user's motion, while preventing the body of the user and pet from being harmed by the high-energy ultrasonic signal.

It can be understood that the above execution sequence of S801 to S806 is only an example. In other embodiments, other execution sequences can also be adopted, and some steps can also be split or combined, which is not limited here.

It can be understood that in the presence of moving users, after the smart TV transmits continuous ultrasonic signals, among the echo signals received by the smart TV within a certain period of time, some echo signals come from the moving user, and some echo signals come from stationary objects. Such as furniture, home appliances, etc. Therefore, the total echo energy E of these echo signals includes the motion component energy introduced by the user's movement and the stationary component energy introduced by the stationary object. Among them, the magnitude of the motion component energy is negatively correlated with the distance d between the user and the smart TV. That is, the smaller the motion component energy, the farther the distance d between the user and the smart TV is. The greater the motion component energy, the greater the distance d between the user and the smart TV. The closer the distance d is to the smart TV. Therefore, in some embodiments of the present application, the distance d between the user and the smart TV can be represented by extracting the motion component energy (hereinafter referred to as the distance energy Esd) in the echo energy E of the echo signal. Correspondingly, it can also be A corresponding motion component energy range (hereinafter referred to as distance energy range) is set for each distance range. In this way, here, there is no need to directly calculate the distance d between the user and the smart TV. It is only necessary to calculate the distance energy Esd representing the distance d based on the echo signal. In addition, it can be understood that the dynamic component energy is not only negatively related to the distance, but also related to the reflection coefficient and reflection cross-sectional area of the reflecting object. In order to eliminate the impact of these factors on distance judgment, you can appropriately increase the margin when setting the distance energy range to include a reasonable user reflection coefficient and cross-sectional area range, or you can start measuring user distance based on motion component energy. function, it guides users to perform personalized calibration measurements to calibrate distance errors introduced by factors such as the reflection coefficient and reflection cross-sectional area of reflective objects.

As mentioned before, the echo energy E of the echo signal can be obtained by integrating the amplitudes in the frequency domain data of multiple echo signals received by the smart TV within a preset time period. Therefore, the echo energy E The motion component energy Es, that is, the distance energy Esd here, can be obtained in the following way: from the multiple echo signals received by the smart TV within a preset time period, filter out the motion echo signals introduced due to user motion, and then The amplitude in the frequency domain data of the motion echo signal is integrated to obtain the motion component energy Es. That is, the motion component energy Es is the echo energy of the motion echo signal.

As mentioned above, the motion echo signals received at different times have different phase offsets relative to the transmitted signal. Therefore, by analyzing the echo By analyzing the phase offset of the signal, the motion echo signals whose phase offset changes with time can be screened out, that is, the echo signals with different phase offsets received at different times and reflected by the same moving object are determined as motion echo signal.

Correspondingly, when the distance energy Esd is used to represent the distance d between the user and the smart TV, the corresponding motion component energy range can be set for each distance range, that is, the distance energy range, and then the distance energy range is established with the aforementioned emission volume range. Corresponding relationship, so that the emission volume range corresponding to the distance energy range where the calculated echo signal Esd is located is regarded as the emission volume range corresponding to the distance range where distance d is located.

For example, Table 3 below shows the corresponding relationship between the set distance range, distance energy range and transmission volume range based on the relationship in Table 1.

table 3

Among them, d1, d2, d3, and d4 increase in sequence, and Esd1, Esd2, Esd3, Esd4, and Esd5 decrease in sequence. The minimum value (such as d1) of the distance range (such as (d1, d2)) corresponds to the maximum value (such as Esd2) of the distance energy range (such as (Esd3, Esd2)), and the maximum value of the distance range (such as (d1, d2)) (such as d2) corresponds to the minimum value of the distance energy range (such as Esd3).

In addition, since the energy of the motion component of the echo signal is positively related to the size of the transmission volume, that is, the greater the transmission volume, the greater the energy of the motion component, and the smaller the volume of the transmission, the smaller the energy of the motion component. Therefore, in some embodiments, it is possible to use The motion component energy range (hereinafter referred to as the volume energy range) represents the emission volume range corresponding to each distance range or distance energy range. In this way, there is no need to pre-measure the transmission volume range corresponding to each distance range. In the process of adaptively adjusting the transmission volume, it is only necessary to determine whether the motion component energy of the motion echo signal received after the transmission volume is adjusted is within the corresponding volume energy range. , you can determine whether the adjusted emission volume is within the emission volume range corresponding to the distance range. It can be understood that in some embodiments, if a corresponding single transmission volume is set for each distance range, the volume energy range can also be set corresponding to a single transmission volume.

It can be understood that although the volume energy range here and the value represented by the distance energy range mentioned above are the magnitude of the motion component energy of the echo signal, the volume energy range is related to the emission volume and is used to use the motion component energy. To replace the emission volume range, the distance energy range is related to the distance d, and is used to replace the distance energy range with the motion component energy. The two are not the same and have no necessary relationship.

In addition, it can be understood that since the distance energy and the volume energy are essentially the motion component energy of the echo signal, the calculation method of the distance energy and the volume energy can refer to the calculation method of the motion component energy mentioned above.

For example, Table 4 below shows the corresponding relationship between the set volume energy range and the transmission volume range based on Table 1 or 3.

Table 4

Among them, Esv1, Esv2, Esv3, and Esv4 increase in sequence.

As described above, in some embodiments, the distance d between the user and the smart TV can be represented by the distance energy Esd. Each distance range can be represented by a corresponding distance energy range. As mentioned above, the transmission volume range can be represented by the volume. Therefore, in some embodiments, only the distance energy Esd of the user during movement relative to the smart TV can be measured, the corresponding volume energy range can be determined based on the distance energy range, and then the emission of the smart TV can be adjusted based on the volume energy range. Volume Vd. For example, Table 5 below shows the correspondence between the distance range, distance energy range, volume energy range, and emission volume range determined based on Table 1, Table 3, and Table 4.

table 5

The following is a detailed introduction in conjunction with Figure 9. The distance energy Esd is used to represent the distance d between the user and the smart TV. The distance energy range is used to represent each distance range. The volume energy range is used to represent the emission volume range. When the user moves relative to the smart TV, A technical solution for adaptively adjusting transmit volume.

Specifically, FIG. 9 shows a schematic flowchart of a smart TV adaptively adjusting the transmission volume Vd according to the distance energy Esd in the echo signal corresponding to the transmission volume, according to some embodiments of the present application, in which the execution subject may be the smart TV. Processor 110 in 10.

In addition, it can be understood that the adjustment process as shown in Figure 9 below can be performed multiple times as the distance d changes during the user's movement, such as continuously or once every predetermined time interval (such as 100ms).

Specifically, as shown in Figure 9, the method includes:

S901 to S903, where S901 to S903 are the same as S801 to S803, and will not be described again.

S904: Based on the received echo signal, calculate the distance energy Esd representing the distance d between the user and the smart TV.

The distance energy Esd can be calculated using the method in S804, which will not be described again here.

S905: Obtain the volume energy range corresponding to the distance energy range where the distance energy Esd is located.

For example, as shown in Table 5 above, assuming that the distance energy Esd calculated based on the echo signal belongs to the volume energy range (Esd3, Esd2), the corresponding volume energy range is (Esv1, Esv2).

S906: Determine whether the current distance energy is within the acquired volume energy range.

Specifically, as mentioned above, the motion component energy of the echo signal, that is, the distance energy Esd, is positively correlated with the size of the emission volume Vd, so the relationship between the currently calculated distance energy Esd and the acquired volume energy range can be judged, for example, When Esd belongs to the distance energy range (Esd3, Esd2) and the corresponding volume energy range is (Esv1, Esv2), step S910 is entered to complete the volume adaptive adjustment; when Esd does not belong to the distance energy range (Esd3, Esd2), When the corresponding volume energy range is (Esv1, Esv2), step S907 is entered, and the transmission volume Vd is adjusted according to whether Esd is smaller than Esv1 or larger than Esv2.

S907, adjust the transmit volume Vd.

If Esd is less than Esv1, and Vd is positively related to Esd, it means that the current emission volume Vd is too low, and the emission volume Vd needs to be increased; if Esd is greater than Esv2, it means that the current emission volume Vd is too high, and the emission needs to be reduced. The size of the volume Vd.

S908: Determine whether the adjusted emission volume Vd is greater than the volume threshold Vt, and whether the duration T for transmitting ultrasonic waves with a transmission volume Vd greater than the volume threshold Vt is greater than the time threshold Tv.

It can be understood that when an error occurs in the distance detection between the user and the smart TV, the detected distance between the user and the smart TV may continue to be large. In this case, in order to keep the motion component energy Es at the corresponding Within the energy range of the motion component within the distance range, the emission volume Vd may be increased or maintained at a high volume for a long time, thereby causing the user to remain in a high-energy ultrasonic environment for a long time, causing harm to the user's body. Therefore, in order to solve this problem, when the emission volume Vd is increased, it is necessary to determine whether the increased volume Vd is greater than the volume threshold Vt, and whether the duration T for transmitting ultrasonic waves at a volume Vd greater than the volume threshold Vt is greater than the time threshold Tv.

For example, the volume threshold Vt is related to the distance d between the user and the smart TV. The greater the distance d between the user and the smart TV, the greater the volume threshold Vt.

It can be understood that in other embodiments, S908 may also be executed before adjusting the transmission volume Vd, and the order is not limited.

If it is determined that the adjusted emission volume Vd is greater than the volume threshold Vt, and the duration T of ultrasonic waves emitted with a transmission volume Vd greater than the volume threshold Vt is greater than the time threshold Tv, it indicates that the user has been in a high-energy ultrasonic environment for a long time and has entered S909, otherwise, return to S903 to control the smart TV to transmit the ultrasonic signal at the currently set transmission volume Vd, and obtain the echo signal received after reflection by the user.

S909, reduce the duty cycle of the emitted ultrasonic signal and reduce the detection frame rate.

After S909, return to S903 to determine whether the adjusted transmission volume Vd can be adjusted to the corresponding volume energy range by adjusting the distance energy Esd calculated from the echo signal after adjusting the transmission volume Vd.

S910, completes adaptive volume adjustment.

It can be understood that the above execution sequence of S901 to S910 is only an example. In other embodiments, other execution sequences can also be adopted, and some steps can also be split or combined, which is not limited here.

In addition, it can be understood that since the above-mentioned processes S901 to S910 are performed multiple times as the distance d changes during the user's movement, the emission volume can be adjusted in real time according to the change in the distance between the user and the smart TV. When the user is far away from the smart TV, the ultrasonic emission volume is increased. When the user is closer to the smart TV, the ultrasonic emission volume is reduced to achieve adaptive adjustment of the emission volume. This ensures that the energy of the echo signal reflected back by the user is sufficient to detect the user's motion, while preventing the body of the user and pet from being harmed by the high-energy ultrasonic signal.

As mentioned above, the propagation of ultrasonic signals has radiation directivity. Therefore, when the smart TV plays the ultrasonic signal through the speaker, the energy of the ultrasonic signal propagated directly in front of the smart TV is strong enough to detect the user's presence, while the energy of the ultrasonic signal propagated directly in front of the smart TV is strong enough to detect the presence of the user, while the energy of the ultrasonic signal propagated in the side of the smart TV is The energy of the propagated ultrasonic signal is weak, so that the energy of the echo signal reflected back by the reflecting object is weak and cannot be used for user presence detection. Therefore, in some embodiments, during the process of user presence detection by the smart TV using ultrasonic signals, the direction of the user relative to the smart TV can be measured. For example, it can be determined through the beamforming method based on multiple microphones mentioned above. The user's orientation relative to the smart TV. The transmit volume is then adjusted based on the user's orientation relative to the smart TV. Specifically, FIG. 10 shows a schematic flowchart of a method for adaptively adjusting the transmission volume according to the user's direction relative to the smart TV, according to some embodiments of the present application.

In addition, it can be understood that the adjustment process as shown in Figure 10 below can be performed multiple times as the distance d changes during the user's movement, such as continuously or once every predetermined time interval (such as 100ms).

Specifically, as shown in Figure 10, it includes:

S1001 and S1002, where S1001 and S1002 are the same as S801 and S802, and will not be described again here.

S1003. Determine the user's direction relative to the smart TV.

It can be understood that various existing technologies can be used to determine the user's direction relative to the smart TV. For example, the user's direction relative to the smart TV can be determined through the beam forming method based on multiple microphones mentioned above.

S1004: Obtain the volume range corresponding to the direction range to which the relative direction R between the user and the smart TV belongs.

Specifically, multiple directional ranges can be preset for different directions between the user and the smart TV, and different applicable emission volumes or emission volume ranges can be set for each directional range. It can be understood that when the user is located in a certain direction range relative to the direction r of the smart TV, adjusting the transmission volume of the ultrasonic signal transmitted by the smart TV to the transmission volume corresponding to the direction range or within the transmission volume range can ensure that the received The echo signal can meet the detection needs and does not cause physical harm to users in the ultrasonic environment.

For example, two direction ranges can be set, one represents the front of the smart TV, that is, the user's first direction r1 relative to the smart TV, and the angle between the vertical direction of the smart TV screen is within the range (0, r1), and the other represents The angle between the side of the smart TV, that is, the user's second direction r2 relative to the smart TV, and the vertical direction of the smart TV screen is within the range of (r1, r2). For example, r1 can be 75° and r2 can be 90°. Table 6 below shows the correspondence between the directional range and the emission volume:

Table 6

Among them, Vr2 is greater than Vr1, that is, the emission volume when the user is located to the side of the smart TV is greater than the emission volume when the user is directly in front of the smart TV, to ensure that the echo signal received by the smart TV can meet the needs of detecting side users, and at the same time , ensuring that the set volume does not cause physical harm to the user.

S1005: Adjust the transmission volume Vd of the next ultrasonic signal transmission to the obtained transmission volume corresponding to the direction range.

For example, if it is detected that the user's direction r relative to the smart TV is located in the direction range (r1, r2), that is, on the side of the smart TV, the transmission volume Vd of the next ultrasonic signal transmission will be adjusted to Vr2.

It can be understood that since the above processes S1001 to S1005 are performed multiple times with changes in the direction r during the movement of the user, the emission volume can be adjusted in real time according to the direction change between the user and the smart TV, so as to achieve the desired result when the user moves. When the user is located directly in front of the smart TV, a smaller emission volume is used to emit ultrasonic signals. When it is measured that the user is located to the side of the smart TV, a larger emission volume is used to emit ultrasonic signals to ensure that the ultrasonic signal located on the smart TV can be accurately detected. The movement of the user on the side of the TV.

In addition, it can be understood that as mentioned above, since the energy of the motion component of the echo signal is positively related to the size of the emission volume, that is, the greater the emission volume, the greater the energy of the motion component, and the smaller the emission volume, the smaller the energy of the motion component. Therefore, in some embodiments, the motion component energy range (hereinafter referred to as the directional energy range) can be used to represent the emission volume or emission volume range corresponding to each angle range. In this way, there is no need to pre-measure the transmission volume or transmission volume range corresponding to each angle range. In the process of adaptively adjusting the transmission volume, it is only necessary to determine whether the motion component energy of the motion echo signal received after the transmission volume is adjusted is located in the corresponding Within the directional energy range, you can determine whether the adjusted reflection volume is within the emission volume range corresponding to the angle range.

In addition, it can be understood that since the angular energy is essentially the motion component energy of the echo signal, the calculation method of the angular energy can refer to the calculation method of the motion component energy mentioned above.

For example, Table 7 below shows the corresponding relationship between the set direction range, the direction energy range, and the emission volume based on Table 6.

Table 7

Among them, Esr1, Esr2, Esr3, and Esr4 increase in sequence. Since the directional energy range is positively related to the emission volume, setting Esr1, Esr2, Esr3, and Esr4 to increase in sequence can ensure that when the user's angular energy is in the range (Esr1, Esr2), the emission volume is smaller than when the user's angular energy is in the range (Esr3, Esr4). The emission volume corresponds to Vr1 being smaller than Vr2, and the user's emission volume on the side of the smart TV is greater than the user's emission volume directly in front of the smart TV.

The following describes in detail a technical solution for adaptively adjusting the emission volume by using the directional energy range to represent the emission volume corresponding to the angular range in conjunction with Figure 11 as the user moves relative to the smart TV.

In some embodiments, the execution subject of FIG. 11 may be the processor 110 in the smart TV 10 .

In addition, it can be understood that the adjustment process as shown in Figure 11 below can be performed multiple times as the distance d changes during the user's movement, such as continuously or once every predetermined time interval (such as 100ms).

Specifically, as shown in Figure 11, the method includes:

S1101 and S1102, where S1101 and S1102 are the same as S801 and S802, and will not be described again here.

S1103. Determine the user's direction r relative to the smart TV, and obtain the directional energy range corresponding to the direction range in which direction r is located.

It can be understood that various existing technologies can be used to determine the user's direction relative to the smart TV. For example, the user's direction relative to the smart TV can be determined through the beam forming method based on multiple microphones mentioned above. Moreover, the calculation method of the angular energy Esr can refer to the calculation method of the motion component energy in S804.

S1104: Determine whether the direction range of the direction r is the same as the direction range of the last measured direction r'.

Since the user may be directly in front of or to the side of the smart TV for a period of time (such as 30 minutes) during movement, it is necessary to first determine whether the direction range of direction r is the same as the previous determination. If they are the same, go to S1105 and keep the current transmit volume Vr; if they are different, go to S1106 and adjust the transmit volume.

S1105, maintain the current transmission volume Vr.

After S1105, enter S1109 to confirm that the volume adaptive adjustment is completed.

S1106, adjust the transmission volume Vr, and transmit the ultrasonic signal with the adjusted transmission volume.

Specifically, if the direction range of the direction r is larger than the direction range of the last measured direction r', it means that the user moves from the front to the side of the smart TV and needs to increase the transmission volume. Vr, if the direction decreases, it means that the user moves from the side to the front of the smart TV and needs to reduce the emission volume Vr.

S1107: Obtain the echo signal and calculate the directional energy Esr based on the echo signal.

S1108: Determine whether the directional energy Esr is within the obtained directional energy range.

If it is determined that the directional energy Esr is within the obtained directional energy range, then enter S1109 to confirm that the volume adaptive adjustment is completed. If it is determined that the directional energy Esr is not within the obtained directional energy range, then return to S1106 to continue adjusting the emission volume Vr. .

S1109: Confirm that the volume adaptive adjustment is completed.

It can be understood that when there is an error in the direction detection between the user and the smart TV, the emission volume Vd may be increased or maintained at a high volume for a long time, causing the user to remain in a high-energy ultrasound environment for a long time, causing harm to the user's body. harm. Therefore, in order to solve this problem, in some embodiments, in the above step S1106, when the transmission volume Vr is increased, it is necessary to determine whether the increased volume Vr is greater than the volume threshold Vt, and transmit with a volume Vr greater than the volume threshold Vt. Whether the duration T of the ultrasound is greater than the time threshold Tv. If the duration T of transmitting ultrasonic waves with a transmit volume Vr greater than the volume threshold Vt is greater than the time threshold Tv, it indicates that the user has been in a high-energy ultrasonic environment for a long time. The ultrasonic signal duty cycle reduces the detection frame rate.

It can be understood that the above execution order of S1101 to S1109 is only an example. In other embodiments, other execution orders can also be adopted, and some steps can also be split or combined, which is not limited here.

It can be understood that since the above processes S1101 to S1109 are performed multiple times with changes in the direction r during the movement of the user, the emission volume can be adjusted in real time according to the direction change between the user and the smart TV, so as to achieve the desired result when the user moves. When the user is located directly in front of the smart TV, a smaller emission volume is used to emit ultrasonic signals. When it is measured that the user is located to the side of the smart TV, a larger emission volume is used to emit ultrasonic signals to ensure that the ultrasonic signal located on the smart TV can be accurately detected. The movement of the user on the side of the TV.

As mentioned before, children and pets are less tolerant to ultrasonic signal energy than adults. If an ultrasonic signal is emitted at a fixed volume during user detection, the volume may be suitable for adults, but it may be harmful to children and pets. cause harm. Therefore, some embodiments of the present application disclose a volume adaptive adjustment method. In the process of user presence detection using ultrasonic signals on a smart TV, when a user moving relative to the smart TV is detected, the user can be photographed by the camera of the smart TV. The user's photos are taken, and image recognition technology is used to identify whether the user in the photo is an adult or a child, and whether there are pets around the user. If it is determined that the user is a child or there is a pet around the user, the ultrasonic signal can be transmitted using a smaller transmission volume than if the user is an adult. Specifically, as shown in Figure 12, it includes:

S1201 and S1102, where S1201 and S1202 are the same as S801 and S802, and will not be described again.

S1203. Determine whether there is a target reflective object among the objects that reflect the ultrasonic signal.

For example, the target reflective object can be a child or a pet. The default ultrasonic emission volume of smart TVs is generally set to be suitable for adults. Therefore, if children or pets are recognized in the photo, it means that the ultrasonic emission volume needs to be lowered and enter S1204. Otherwise, enter S1205 and maintain the current emission volume.

S1204, lower the emission volume of the ultrasonic wave to the emission volume corresponding to the target reflecting object.

It can be understood that the emission volume suitable for children and pets must not only be smaller than the emission volume for adults, but also ensure that the received echo signal can meet the detection requirements. When the ultrasonic emission volume is lowered to a level corresponding to that of a child or pet, enter S1202.

S1205, maintain the current transmit volume.

It can be understood that since the above processes S1201 to S1202 can be performed multiple times at a certain time (such as every 1 minute) during the user's movement, the transmission volume can be adjusted in real time according to changes in the user type to adjust the transmission volume on the smart TV. When children and pets are present, prevent the bodies of children and pets from being harmed by high-energy ultrasonic signals.

S1206, complete adaptive volume adjustment.

In addition, in other embodiments, the smart TV 10 can determine whether there is an ultrasonic signal greater than a safety threshold in the environment by extracting the motion component energy Es of the received ultrasonic signal in the environment, and detect the ultrasonic wave exceeding the safety threshold. After receiving the signal, the user is reminded that there is ultrasonic pollution in the current environment that is harmful to the body by displaying prompt information or voice reminders. For example, the prompt message "There are ultrasonic signals that are harmful to the body in the current environment, please detect the source of ultrasonic pollution" can be displayed on the screen of a smart TV.

In addition, as mentioned above, the technical solution of determining whether there is a moving reflective object in front of a smart TV based on changes in the phase offset of continuous ultrasonic signals can be used in other application scenarios in addition to controlling the display of the standby screen. For example, sports apps for smart TVs. For example, in some embodiments, when a user of a smart TV follows a sports APP (such as a fitness APP, a virtual reality interactive game APP) in front of the smart TV, the continuous ultrasonic waves emitted are returned by the user and received. echo signal The phase offset is used to calculate the user's movement speed, and then the corresponding screen is fed back or exercise suggestions are given to the user based on the user's movement speed.

Furthermore, in order to facilitate a deeper understanding of the technical solutions provided by the embodiments of the present application, Figure 13 shows a schematic software structure diagram of a smart TV 10 according to some embodiments of the present application. As shown in Figure 13, the software structure of the smart TV 10 is a multi-layer structure, which includes an application layer 91, an application framework layer 92, a system library 93 and a kernel layer 94 from top to bottom.

Among them, the application layer 91 may include a series of application packages. As shown in Figure 13, the application layer 91 includes applications such as Bluetooth 910, music 911, gallery 912, standby display 913, and calendar 914. Among them, in some embodiments of the present application, the standby display 913 can provide the user 20 with a switch for setting the screen display control function. The standby display 913 receives the instruction triggered by the user 20 to open the smart display program, turns on the screen display control function, and sends a message to the application. The program framework layer 92 sends a service instruction to call and detect whether the user exists.

The application framework layer 92 provides an application programming interface (API) and programming framework for applications in the application layer 91. The application framework layer includes some predefined functions. The application framework layer 92 includes a notification manager 920, a view system 921, a content provider 922, a presence awareness module 923, a resource manager 924, and so on. Among them, in some embodiments of the present application, the presence sensing module 923 responds to the service instruction sent by the standby display 913 to detect the presence of the user, sends the enabling ultrasonic signal processing library 933 and monitors the user presence detection results to the system library 93 instructions. Among them, the ultrasonic signal processing library 933 can obtain the echo signal recorded by the microphone, and execute the above-mentioned technical solution of determining whether the user exists based on the echo signal.

The system library 93 is the core of the smart TV 10 software system and can provide services to applications in the application layer 91 through the application framework layer 92 . The system service layer 93 media library 930, graphics processor 931, surface management library 932, ultrasonic signal processing library 933, voice wake-up library 934, etc. In some embodiments of the present application, the ultrasonic signal processing library 933 responds to the instruction sent by the presence sensing module 923 to enable the ultrasonic signal processing library 933 and monitor the user presence detection results, and sends a call to the audio recording stream and Play stream command.

The kernel layer 94 includes a display driver 940, a camera driver 941, a network card driver 942, an audio driver 943, a sensor driver 944, etc. Among them, in some embodiments of the present application, the audio driver 943 responds to the calling audio recording stream and playback stream sent by the ultrasonic signal processing library 933, and then drives the microphone to record, such as recording echo signals, and starts the speaker for ultrasonic playback. Such as playing continuous ultrasonic signal.

It can be understood that the software structure of the smart TV 10 shown in FIG. 13 is only an example. In other embodiments, the smart TV 10 can also adopt other forms of software structures, such as Android ^TM , iOS ^TM , HarmonyOS ^TM , Tizen ^TM , etc., are not limited by the embodiments of this application.

Embodiment 1 of the present application discloses a screen display control method, which is applied to electronic devices, including:

emit ultrasonic signals;

Obtaining a plurality of echo signals generated after the ultrasonic signal is reflected by the object and received within a first time period;

When there is a user moving relative to the electronic device among the objects that reflect the ultrasonic signal, and the plurality of echo signals satisfy the first condition indicating whether the user is present, the first screen display mode is executed. ;

When there is no user moving relative to the electronic device among the objects that reflect the ultrasonic signal, in the case where the plurality of echo signals do not satisfy the first condition indicating whether the user is present, a second condition is performed. Two screen display mode;

Wherein, the first condition includes: among the plurality of echo signals, there are a plurality of motion echo signals whose phase offset relative to the transmitted ultrasonic signal changes with time.

Embodiment 2 of the present application is a screen display control method according to Embodiment 1, wherein the movement includes body movement in which the displacement of the user does not change.

The third embodiment of the present application is the screen display control method according to the first embodiment, wherein the ultrasonic signal includes a continuous ultrasonic signal.

Embodiment 4 of the present application is a screen display control method according to Embodiment 3, wherein the emitting ultrasonic signal includes:

Embodiment 5 of the present application is a screen display control method according to Embodiment 4, wherein the plurality of continuous ultrasonic signals include a first continuous ultrasonic signal and a second continuous ultrasonic signal, and

Embodiment 6 of the present application is a screen display control method according to Embodiment 4, wherein the transmitting an ultrasonic signal includes:

The same ultrasonic sounder is used to transmit the plurality of continuous ultrasonic signals at multiple different transmitting frequencies at the same time.

Embodiment 7 of the present application is a screen display control method according to Embodiment 4, wherein the first condition further includes:

The equivalent speed of the user's movement relative to the electronic device belongs to a first speed range, and the first speed range does not include 0.

Embodiment 8 of the present application is a screen display control method according to Embodiment 7, wherein the equivalent speed is obtained in the following manner:

Select a plurality of motion echo signals whose phase offset changes with time from the echo signals respectively corresponding to the plurality of continuous ultrasonic signals;

Based on the phase offsets of multiple motion echo signals corresponding to each continuous ultrasonic signal, multiple motion speeds corresponding to each continuous ultrasonic signal are calculated, wherein the phase offset and the echo of the motion echo signal are calculated. Wave path dependent;

A preset algorithm is used to process the multiple movement speeds to obtain the user's equivalent speed.

Embodiment 9 of the present application is the screen display control method according to Embodiment 8, wherein the preset algorithm includes at least one of the following:

Calculate the average of the plurality of movement speeds;

Calculate least squares fitting values for the plurality of motion velocities.

Embodiment 10 of the present application is the screen display control method according to Embodiment 7, wherein the first speed range is 0m/s-5m/s.

Embodiment 11 of the present application is a screen display control method according to Embodiment 3, wherein the continuous ultrasonic signal includes any of the following:

Sinusoidal ultrasonic signal, frequency modulated continuous ultrasonic signal.

Embodiment 12 of the present application is the screen display control method according to Embodiment 1, wherein the first condition further includes:

There is a human being in the image of the object that reflects the ultrasonic signal collected by the electronic device.

Embodiment 13 of the present application is a screen display control method according to Embodiment 1, wherein the first screen display method includes at least one of the following:

Convert the screen of the electronic device from a black screen state to displaying a standby screen;

Convert the screen of the electronic device from the standby screen or black screen state to the bright screen state and display the screen displayed before the last time it entered the standby screen or black screen state;

Convert the screen of the electronic device from a standby screen or a black screen state to a bright screen state, and play the program that the user has paused.

Embodiment 14 of the present application is a screen display control method according to Embodiment 13, wherein the screen displayed before entering the standby screen or black screen state last time includes at least one of the following:

startup screen;

The pause screen of the program watched by the user;

The application interface that the user browses.

Embodiment 15 of the present application is a screen display control method according to Embodiment 1, wherein the second screen display method includes:

Keep the screen black or keep the standby screen displayed.

Embodiment 16 of the present application is the screen display control method according to Embodiment 1, wherein in the case where the plurality of echo signals do not satisfy the first condition indicating whether the user is present, executing a second Screen display methods, including:

Corresponding to the fact that the plurality of echo signals do not satisfy the first condition indicating whether the user exists, and that none of the historical echo signals received within the first preset historical duration satisfies the first condition, execute At least one of the following screen displays:

Convert the screen of the electronic device from a bright screen state to a standby screen or a black screen state,

Convert the screen of the electronic device from a standby screen to a black screen state,

Pause the program being played on the screen of the electronic device.

Embodiment 17 of the present application is a screen display control method according to Embodiment 1, wherein the first screen display method further includes:

Corresponding to the user gradually approaching the electronic device and the electronic device is in a black screen state, gradually light up the screen of the electronic device and display the standby screen or the screen displayed before entering the standby screen or black screen state last time; and

The second screen display method also includes:

In response to the user gradually moving away from the electronic device and the electronic device being in a bright screen state or displaying a standby screen, the screen of the electronic device is gradually dimmed.

Embodiment 18 of the present application is the screen display control method according to Embodiment 17, wherein the user is close to the electronic device The faster the speed, the faster the screen gradually lights up;

The faster the user moves away from the electronic device, the faster the screen gradually dims.

Embodiment 19 of the present application is a screen display control method according to Embodiment 17, wherein it is determined whether the user is gradually approaching or moving away from the electronic device in the following manner:

Calculate the equivalent frequency of multiple echo signals acquired successively, wherein the equivalent frequency is related to the echo path of the echo signal, and the smaller the echo path, the greater the equivalent frequency;

Corresponding to the equivalent frequencies of multiple echo signals acquired successively, it is determined that the user is gradually approaching the electronic device;

Embodiment 20 of the present application is a screen display control method according to Embodiment 1, wherein the first screen display method includes at least one of the following:

Corresponding to the user's direction relative to the electronic device being in the first direction range, converting the screen of the electronic device from a standby screen or a black screen state to a bright screen state and displaying the screen displayed before entering the standby screen or black screen state last time. screen, or convert the screen of the electronic device from the standby screen or black screen state to the bright screen state, and play the program that the user has paused;

Embodiment 21 of the present application is a screen display control method according to Embodiment 1, wherein the phase offset of the echo signal is calculated in the following manner:

Perform Fourier transform on the time domain data of the received echo signal to obtain corresponding frequency domain data;

The difference between the second phase and the first phase is used as the phase offset of the echo signal.

Embodiment 22 of the present application is a screen display control method according to Embodiment 1, which further includes:

Corresponding to a user who does not move relative to the electronic device, ultrasonic signals are emitted at multiple set volumes successively, and the difference between the echo energy of the echo signal and the reference echo energy is selected to be within the first energy difference range. The set volume within is used as the predetermined transmission volume of the electronic device for transmitting the ultrasonic signal.

Embodiment 23 of the present application is the screen display control method according to Embodiment 1, wherein the ultrasonic signal is emitted through a speaker of the electronic device, and the echo signal is received through a microphone of the electronic device.

Embodiment 24 of the present application is the screen display control method according to Embodiment 1, wherein the electronic device includes a smart TV.

Embodiment 25 of the present application is a screen display control method according to Embodiment 1, which further includes:

If the plurality of echo signals satisfy a first condition indicating whether the user is present, transmitting a first ultrasonic signal at a first transmission volume;

Embodiment 26 of the present application is the screen display control method according to Embodiment 25, wherein the first position range includes a first distance range, and the second position range includes a second distance range; and

The twenty-seventh embodiment of the present application is the screen display control method according to the twenty-sixth embodiment, and further includes:

The first distance of the object relative to the electronic device is represented by the motion component energy of the plurality of echo signals, and a first motion component energy range corresponding to the first distance range and the second distance range is set respectively and the energy range of the second motion component,

Wherein, the greater the energy of the motion component, the smaller the first distance of the object relative to the electronic device, and the first motion component The maximum value of the component energy range corresponds to the minimum value of the first distance range, the minimum value of the first motion component energy range corresponds to the maximum value of the first distance range, and the maximum value of the second motion component energy range corresponds to the The minimum value of the second distance range, the minimum value of the second motion component energy range corresponds to the maximum value of the second distance range.

The twenty-eighth embodiment of the present application is a screen display control method according to the twenty-seventh embodiment, and the distance range of the first distance is determined in the following manner:

When the motion component energy is within the first motion component energy range, determining that the first distance is within the first distance range;

When the motion component energy is within the second motion component energy range, it is determined that the first distance is within the second distance range.

Embodiment 29 of the present application is a screen display control method according to Embodiment 27, which calculates the motion component energy of the multiple echo signals in the following manner:

The amplitudes of the motion echo signals among the multiple echo signals are integrated to obtain the motion component energy of the multiple motion echo signals.

Embodiment 30 of the present application is the screen display control method according to Embodiment 25, the first position range includes a third direction range, the second position range includes a fourth direction range; and

The first relative position being in the first position range includes: the first direction of the user corresponding to the first relative position relative to the electronic device is located in the third direction range,

The second relative position being in the second position range includes: the second direction of the user corresponding to the second relative position relative to the electronic device is located in the fourth direction range; and

Embodiment 31 of the present application is the screen display control method according to Embodiment 25, and further includes:

In the case where the second transmitting volume or the third transmitting volume is greater than the volume threshold, and the historical volume of the transmitted ultrasonic signal in the first historical time period continues to be greater than the volume threshold, the second transmitting volume or the third transmitting volume will be used next time. When transmitting ultrasonic signals at three transmit volumes, reduce the duty cycle of the emitted ultrasonic signals and reduce the detection frame rate.

Embodiment 32 of the present application is a screen display control method according to Embodiment 1, which further includes:

In the case where the plurality of echo signals satisfy a first condition indicating whether the user is present, obtaining a captured user image of the user;

If there is a target object in the user image, transmit the ultrasonic signal using a fourth transmission volume;

Embodiment 33 of the present application is the screen display control method according to Embodiment 1, wherein the target objects include children and pets.

Embodiment 34 of the present application discloses an object motion detection method, which is applied to electronic equipment, including:

emit ultrasonic signals;

Obtaining multiple echo signals generated after the ultrasonic signal is reflected by the object and received at different times;

Corresponding to the plurality of echo signals satisfying the first condition, among the objects that reflect the ultrasonic signal, there is an object that moves relative to the electronic device;

Corresponding to the plurality of echo signals not satisfying the first condition, there is no object that moves relative to the electronic device among the objects that reflect the ultrasonic signal;

Embodiment 35 of the present application is the object motion detection method according to Embodiment 34, wherein the motion includes limb motion in which the displacement of the object does not change.

Embodiment 36 of the present application is the object motion detection method according to Embodiment 34, wherein the ultrasonic signal includes a continuous ultrasonic signal.

Embodiment 37 of the present application is the object motion detection method according to Embodiment 36, wherein the emitting ultrasonic signal includes:

Embodiment 38 of the present application is the object motion detection method according to Embodiment 37, wherein the plurality of continuous ultrasonic signals include a first continuous ultrasonic signal and a second continuous ultrasonic signal, and

Embodiment 39 of the present application is the object motion detection method according to Embodiment 37, wherein the emitting ultrasonic signal includes:

Embodiment 40 of the present application is the object motion detection method according to Embodiment 37, wherein the first condition further includes:

The equivalent speed of the object moving relative to the electronic device belongs to a first speed range, and the first speed range does not include 0.

Embodiment 41 of the present application is the object motion detection method according to Embodiment 40, wherein the equivalent speed is obtained in the following manner:

The plurality of movement speeds are processed using a preset algorithm to obtain the equivalent speed of the object moving relative to the electronic device.

Embodiment 42 of the present application is the object motion detection method according to Embodiment 41, wherein the preset algorithm includes at least one of the following:

Calculate the average of the plurality of movement speeds;

Calculate least squares fitting values for the plurality of motion velocities.

Embodiment 43 of the present application is the object motion detection method according to Embodiment 40, wherein the characteristic is that the first speed range is 0m/s-5m/s.

Embodiment 44 of the present application is the object motion detection method according to Embodiment 36, wherein the continuous ultrasonic signal includes any of the following:

Sinusoidal ultrasonic signal, frequency modulated continuous ultrasonic signal.

Embodiment 45 of the present application is the object motion detection method according to Embodiment 34, wherein the phase offset of the echo signal is calculated in the following manner:

Embodiment 46 of the present application is the object motion detection method according to Embodiment 34, which further includes:

Corresponding to the fact that there is no object moving relative to the electronic device, ultrasonic signals are successively emitted at multiple set volumes, and the difference between the echo energy of the echo signal and the reference echo energy is selected to be within the first energy difference range. The set volume within is used as the predetermined transmission volume of the electronic device for transmitting the ultrasonic signal.

Embodiment 47 of the present application is the object motion detection method according to Embodiment 34, which further includes:

Transmitting a first ultrasonic signal at a first transmission volume, and acquiring a plurality of first echo signals generated after the first ultrasonic signal is reflected by an object and received within a second time period;

In the case where the plurality of first echo signals satisfy the second condition, there is an object moving relative to the electronic device;

When the first distance between the moving object and the electronic device is within the second distance range, transmit the second ultrasonic signal at the third transmission volume corresponding to the second distance range next time;

Embodiment 48 of the present application is the object motion detection method according to Embodiment 47, wherein the second condition includes:

Among the plurality of first echo signals, there are a plurality of first motion echo signals whose phase offsets relative to the first ultrasonic signal change with time.

Embodiment 50 of the present application is the object motion detection method according to Embodiment 49, which further includes:

The first motion component energy of the plurality of first motion echo signals represents the first distance of the object relative to the electronic device, and a first distance range corresponding to the first distance range and the second distance range is set. a first motion component energy range and a second motion component energy range,

Wherein, the greater the energy of the first motion component, the smaller the first distance of the object relative to the electronic device, the maximum value of the energy range of the first motion component corresponds to the minimum value of the first distance range, and the The minimum value of the first motion component energy range corresponds to the maximum value of the first distance range, the maximum value of the second motion component energy range corresponds to the minimum value of the second distance range, and the minimum value of the second motion component energy range corresponds to the minimum value of the second motion component energy range. The value corresponds to the maximum value of the second distance range.

Embodiment 50 of the present application is the object motion detection method according to Embodiment 49, wherein the distance range of the first distance is determined in the following manner:

When the first component energy is within the first motion component energy range, determining that the first distance is within the first distance range;

Embodiment 51 of the present application is the object motion detection method according to Embodiment 49, wherein the motion component energy of the plurality of first motion echo signals is calculated in the following manner:

Embodiment 52 of the present application is the object motion detection method according to Embodiment 49, which further includes:

In the case that the second emission is greater than the volume threshold, and the historical volume of the ultrasonic signal transmitted in the first historical time period continues to be greater than the volume threshold, when the ultrasonic signal is transmitted at the second transmission volume next time, the emitted ultrasonic signal is reduced. The duty cycle of the ultrasonic signal and reduces the detection frame rate.

Embodiment 53 of the present application is the object motion detection method according to Embodiment 34, wherein the ultrasonic signal is emitted through a speaker of the electronic device, and the echo signal is received through a microphone of the electronic device.

Embodiment 54 of the present application is an object motion detection method according to Embodiment 34, wherein a sports application is installed on the electronic device, and the method further includes:

Calculate the movement speed of the object moving relative to the electronic device based on the phase offsets of the plurality of motion echo signals;

According to the calculated movement speed of the object, a preset operation of the sports application is executed.

Embodiment 55 of the present application discloses an object motion detection method, which is applied to electronic equipment and includes: an object motion detection method, which is characterized by including:

Transmit an ultrasonic signal at a sixth transmission volume, and obtain the ultrasonic signal received during a second time period and reflected by the object

Multiple echo signals generated later;

In the case where the multiple echo signals satisfy a first condition, among the objects that reflect the ultrasonic signal, there is an object that moves relative to the electronic device, wherein the first condition includes: the multiple echo signals In the signal, there are multiple motion echo signals whose phase offset changes with time relative to the transmitted ultrasonic signal;

When the second relative position between the user and the electronic device is in the third position range, the ultrasonic signal is emitted at the sixth transmit volume corresponding to the third position range next time,

When the second relative position between the user and the electronic device is within the fourth position range, the ultrasonic signal is emitted at the seventh transmission volume corresponding to the fourth position range next time.

Embodiment 56 of the present application is the object motion detection method according to Embodiment 55, wherein the third position range includes a third distance range, and the fourth position range includes a fourth distance range; and

Embodiment 57 of the present application is the object motion detection method according to Embodiment 56, which further includes:

The motion component energy of the plurality of motion echo signals represents the second distance of the object relative to the electronic device, and a third motion component energy respectively corresponding to the third distance range and the fourth distance range is set. range and fourth motion component energy range,

Wherein, the greater the energy of the motion component, the smaller the second distance of the object relative to the electronic device, the maximum value of the third motion component energy range corresponds to the minimum value of the third distance range, and the third The minimum value of the motion component energy range corresponds to the maximum value of the third distance range, the maximum value of the fourth motion component energy range corresponds to the minimum value of the fourth distance range, and the minimum value of the fourth motion component energy range corresponds to The maximum value of the fourth distance range.

Embodiment 58 of the present application is the object motion detection method according to Embodiment 57, wherein the distance range of the second distance is determined in the following manner:

When the motion component energy is within the third motion component energy range, determining that the second distance is within the third distance range;

When the motion component energy is within the fourth motion component energy range, it is determined that the second distance is within the fourth distance range.

Embodiment 59 of the present application is the object motion detection method according to Embodiment 56, wherein the motion component energy of the multiple echo signals is calculated in the following manner:

Embodiment 60 of the present application is the object motion detection method according to Embodiment 55, wherein the third position range includes a fifth direction range, and the fourth position range includes a sixth direction range; and

The second relative position being in the third position range includes: the third direction of the user corresponding to the second relative position relative to the electronic device is located in the fifth direction range,

The second relative position being in the fourth position range includes: the user corresponding to the second relative position being in the sixth direction range relative to the fourth direction of the electronic device; and

Embodiment 61 of the present application is the object motion detection method according to Embodiment 55, which further includes:

In the case where the sixth or seventh transmission volume is greater than the volume threshold, and the historical volume of the transmitted ultrasonic signal in the first historical time period continues to be greater than the volume threshold, the sixth or seventh transmission volume will be used next time. When transmitting an ultrasonic signal, reduce the duty cycle of the emitted ultrasonic signal and reduce the detection frame rate.

Embodiment 62 of the present application discloses an electronic device, including:

one or more processors;

One or more memories; the one or more memories store one or more programs. When the one or more programs are executed by the one or more processors, the electronic device executes Embodiments 1 to 1. The screen display control method described in any one of Embodiment 33, the object motion detection method of any one of Embodiment 34 to Embodiment 54, or the method of executing Embodiment 55 to 61 Any object motion detection method.

Embodiment 63 of the present application discloses a computer-readable storage medium. Instructions are stored on the storage medium. When the instructions are executed on a computer, they cause the computer to execute any one of Embodiments 1 to 33. The screen display control method described in the item, performs any one of the object motion detection methods in Embodiments 34 to 54, or performs any one of the object motion detection methods in Embodiments 55 to 61 .

Embodiment 64 of the present application discloses a computer program product. The computer program product includes instructions. When executed, the instructions cause the computer to execute the screen display control method described in any one of Embodiments 1 to 33. Perform any object motion detection method in Embodiment 34 to Embodiment 54, or perform any object motion detection method in Embodiment 55 to Embodiment 61.

Various embodiments of the mechanisms disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of the present application may be implemented as a computer program or program code executing on a programmable system including at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements) , at least one input device and at least one output device.

Program code may be applied to input instructions to perform the functions described herein and to generate output information. Output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

Program code may be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system. When necessary, assembly language or machine language can also be used to implement program code. In fact, the mechanisms described in this application are not limited to the scope of any particular programming language. In either case, the language may be a compiled or interpreted language.

In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried on or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be operated by one or more processors Read and execute. For example, instructions may be distributed over a network or through other computer-readable media. Thus, machine-readable media may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magnetic Optical disk, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical card, flash memory, or Tangible machine-readable storage used to transmit information (e.g., carrier waves, infrared signals, digital signals, etc.) using electrical, optical, acoustic, or other forms of propagated signals over the Internet. Thus, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, computer).

In the drawings, some structural or methodological features may be shown in specific arrangements and/or orders. However, it should be understood that such specific arrangement and/or ordering may not be required. Rather, in some embodiments, the features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments these features may not be included or may be combined with other features.

It should be noted that each unit/module mentioned in each device embodiment of this application is a logical unit/module. Physically, a logical unit/module can be a physical unit/module, or it can be a physical unit/module. Part of the module can also be implemented as a combination of multiple physical units/modules. The physical implementation of these logical units/modules is not the most important. The combination of functions implemented by these logical units/modules is what solves the problem of this application. Key technical issues raised. In addition, in order to highlight the innovative part of this application, the above-mentioned equipment embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems raised by this application. This does not mean that the above-mentioned equipment embodiments do not exist. Other units/modules.

As used in the above embodiments, depending on the context, the terms "when" or "after" may be interpreted to mean "if..." or "after" or "in response to determining..." or "in response to detecting …”. Similarly, depending on the context, the phrase "when determining..." or "if (stated condition or event) is detected" may be interpreted to mean "if it is determined..." or "in response to determining..." or "on detecting (stated condition or event)” or “in response to detecting (stated condition or event)”. In addition, in the above embodiments, relational terms such as first and second are used to distinguish one entity from another entity, without limiting any actual relationship and order between these entities.

It should be noted that in the examples and descriptions of this patent, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply There is no such actual relationship or sequence between these entities or operations. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a" does not exclude the presence of additional identical elements in a process, method, article, or device that includes the stated element.

Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes may be made in form and detail without departing from the present invention. The spirit and scope of the application.

Claims

A screen display control method, applied to electronic equipment, is characterized by including:

emit ultrasonic signals;

Obtaining a plurality of echo signals generated after the ultrasonic signal is reflected by the object and received within a first time period;

When there is a user moving relative to the electronic device among the objects that reflect the ultrasonic signal, and the plurality of echo signals satisfy the first condition indicating whether the user is present, the first screen display mode is executed. ;

When there is no user moving relative to the electronic device among the objects that reflect the ultrasonic signal, in the case where the plurality of echo signals do not satisfy the first condition indicating whether the user is present, a second condition is performed. Two screen display mode;

Wherein, the first condition includes: among the plurality of echo signals, there are a plurality of motion echo signals whose phase offset relative to the transmitted ultrasonic signal changes with time.
The method of claim 1, wherein the movement includes limb movement in which the user's displacement does not change.
The method of claim 1, wherein the ultrasonic signal includes a continuous ultrasonic signal.
The method according to claim 3, characterized in that said transmitting ultrasonic signals includes:

Multiple continuous ultrasonic signals are emitted at the same time, and the emission frequencies of each continuous ultrasonic signal are different.
The method of claim 4, wherein the plurality of continuous ultrasonic signals include a first continuous ultrasonic signal and a second continuous ultrasonic signal, and

The difference between the initial transmission frequencies of the first continuous ultrasonic signal and the second continuous ultrasonic signal is greater than the frequency bandwidth of the first continuous ultrasonic signal and the frequency bandwidth of the second continuous ultrasonic signal; or

The difference between the maximum transmission frequencies of the first continuous ultrasonic signal and the second continuous ultrasonic signal is greater than the frequency bandwidth of the first continuous ultrasonic signal and the frequency bandwidth of the second continuous ultrasonic signal.
The method according to claim 4, characterized in that said transmitting ultrasonic signals includes:

The same ultrasonic sounder is used to transmit the plurality of continuous ultrasonic signals at multiple different transmitting frequencies at the same time.
The method according to claim 4, characterized in that the first condition further includes:

The equivalent speed of the user's movement relative to the electronic device belongs to a first speed range, and the first speed range does not include 0.
The method according to claim 7, characterized in that the equivalent speed is obtained in the following manner:

Select a plurality of motion echo signals whose phase offset changes with time from the echo signals respectively corresponding to the plurality of continuous ultrasonic signals;

Based on the phase offsets of multiple motion echo signals corresponding to each continuous ultrasonic signal, multiple motion speeds corresponding to each continuous ultrasonic signal are calculated, wherein the phase offset and the echo of the motion echo signal are calculated. Wave path dependent;

A preset algorithm is used to process the multiple movement speeds to obtain the user's equivalent speed.
The method according to claim 7, characterized in that the first speed range is 0m/s-5m/s.
The method according to claim 1, wherein the first condition further includes:

There is a human being in the image of the object that reflects the ultrasonic signal collected by the electronic device.
The method according to claim 1, characterized in that the first screen display mode includes at least one of the following:

Convert the screen of the electronic device from a black screen state to displaying a standby screen;

Convert the screen of the electronic device from the standby screen or black screen state to the bright screen state and display the screen displayed before the last time it entered the standby screen or black screen state;

Convert the screen of the electronic device from a standby screen or a black screen state to a bright screen state, and play the program that the user has paused.
The method according to claim 11, characterized in that the screen displayed before entering the standby screen or black screen state last time includes at least one of the following:

startup screen;

The pause screen of the program watched by the user;

The application interface that the user browses.
The method according to claim 1, characterized in that the second screen display method includes:

Keep the screen black or keep the standby screen displayed.
The method according to claim 1, characterized in that the failure of the plurality of echo signals to indicate whether there is the In the case of the user's first condition, the second screen display method is executed, including:

Corresponding to the fact that the plurality of echo signals do not satisfy the first condition indicating whether the user exists, and that none of the historical echo signals received within the first preset historical duration satisfies the first condition, execute At least one of the following screen displays:

Convert the screen of the electronic device from a bright screen state to a standby screen or a black screen state,

Convert the screen of the electronic device from a standby screen to a black screen state,

Pause the program being played on the screen of the electronic device.
The method according to claim 1, characterized in that the first screen display method further includes:

Corresponding to the user gradually approaching the electronic device and the electronic device is in a black screen state, gradually light up the screen of the electronic device and display the standby screen or the screen displayed before entering the standby screen or black screen state last time; and

The second screen display method also includes:

In response to the user gradually moving away from the electronic device and the electronic device being in a bright screen state or displaying a standby screen, the screen of the electronic device is gradually dimmed.
The method of claim 15, wherein the faster the user approaches the electronic device, the faster the screen gradually lights up;

The faster the user moves away from the electronic device, the faster the screen gradually dims.
The method according to claim 1, characterized in that the first screen display mode includes at least one of the following:

Corresponding to the user's direction relative to the electronic device being in the first direction range, converting the screen of the electronic device from a standby screen or a black screen state to a bright screen state and displaying the screen displayed before entering the standby screen or black screen state last time. screen, or convert the screen of the electronic device from the standby screen or black screen state to the bright screen state, and play the program that the user has paused;

Corresponding to the direction of the user relative to the electronic device being in the second direction range, the screen of the electronic device is converted from a black screen state to displaying a standby screen.
The method according to claim 1, further comprising:

Corresponding to a user who does not move relative to the electronic device, ultrasonic signals are emitted at multiple set volumes successively, and the difference between the echo energy of the echo signal and the reference echo energy is selected to be within the first energy difference range. The set volume within is used as the predetermined transmission volume of the electronic device for transmitting the ultrasonic signal.
The method of claim 1, wherein the electronic device includes a smart TV; and/or

The ultrasonic signal is transmitted through a speaker of the electronic device, and the echo signal is received through a microphone of the electronic device.
The method according to claim 1, further comprising:

If the plurality of echo signals satisfy a first condition indicating whether the user is present, transmitting a first ultrasonic signal at a first transmission volume;

When the first relative position between the user and the electronic device is in the first position range, the second ultrasonic signal is emitted at the second transmission volume corresponding to the first position range next time,

When the first relative position between the user and the electronic device is in the second position range, the second ultrasonic signal is emitted at a third transmission volume corresponding to the second position range next time.
The method of claim 20, wherein the first location range includes a first distance range, and the second location range includes a second distance range; and

The first relative position being in the first position range includes: the first distance between the user and the electronic device corresponding to the first relative position being in the first position range,

The second relative position being in the second position range includes: the second distance between the user and the electronic device corresponding to the second relative position being in the second position range; and

The minimum value of the first distance range is greater than the maximum value of the second distance range, and the second emission volume is greater than the third emission volume.
The method of claim 21, further comprising:

The first distance of the object relative to the electronic device is represented by the motion component energy of the plurality of echo signals, and a first motion component energy range corresponding to the first distance range and the second distance range is set respectively and the energy range of the second motion component,

Wherein, the greater the energy of the motion component, the smaller the first distance of the object relative to the electronic device, and the first motion component The maximum value of the component energy range corresponds to the minimum value of the first distance range, the minimum value of the first motion component energy range corresponds to the maximum value of the first distance range, and the maximum value of the second motion component energy range corresponds to the The minimum value of the second distance range, the minimum value of the second motion component energy range corresponds to the maximum value of the second distance range.
The method of claim 20, wherein the first position range includes a third direction range, and the second position range includes a fourth direction range; and

The first relative position being in the first position range includes: the first direction of the user corresponding to the first relative position relative to the electronic device is located in the third direction range,

The second relative position being in the second position range includes: the second direction of the user corresponding to the second relative position relative to the electronic device is located in the fourth direction range; and

The minimum value of the third direction range is greater than the maximum value of the fourth direction range, and the second emission volume is greater than the third emission volume.
The method according to claim 1, further comprising:

In the case where the plurality of echo signals satisfy a first condition indicating whether the user is present, obtaining a captured user image of the user;

When there is a target object in the user image, the fourth transmission volume is used to transmit the ultrasonic signal, where,

The target objects include children and pets;

When there is no target object in the user image, the ultrasonic signal is transmitted using a fifth transmission volume, where the fifth transmission volume is greater than the fourth transmission volume.
An object motion detection method, applied to electronic equipment, is characterized by including:

emit ultrasonic signals;

Obtaining multiple echo signals generated after the ultrasonic signal is reflected by the object and received at different times;

Corresponding to the plurality of echo signals satisfying the first condition, among the objects that reflect the ultrasonic signal, there is an object that moves relative to the electronic device;

Corresponding to the plurality of echo signals not satisfying the first condition, there is no object that moves relative to the electronic device among the objects that reflect the ultrasonic signal;

Wherein, the first condition includes: among the plurality of echo signals, there are a plurality of motion echo signals whose phase offset relative to the transmitted ultrasonic signal changes with time.
The method of claim 25, wherein the movement includes limb movement in which the displacement of the object does not change.
The method of claim 25, wherein the ultrasonic signal includes a continuous ultrasonic signal; and the transmitting the ultrasonic signal includes:

Multiple continuous ultrasonic signals are emitted at the same time, and the emission frequencies of each continuous ultrasonic signal are different.
The method of claim 27, wherein the plurality of continuous ultrasonic signals include a first continuous ultrasonic signal and a second continuous ultrasonic signal, and

The difference between the initial transmission frequencies of the first continuous ultrasonic signal and the second continuous ultrasonic signal is greater than the frequency bandwidth of the first continuous ultrasonic signal and the frequency bandwidth of the second continuous ultrasonic signal; or

The difference between the maximum transmission frequencies of the first continuous ultrasonic signal and the second continuous ultrasonic signal is greater than the frequency bandwidth of the first continuous ultrasonic signal and the frequency bandwidth of the second continuous ultrasonic signal.
The method of claim 25, wherein a sports application is installed on the electronic device, and the method further includes:

Calculate the movement speed of the object moving relative to the electronic device based on the phase offsets of the plurality of motion echo signals;

According to the calculated movement speed of the object, a preset operation of the sports application is executed.
An electronic device, characterized by including:

one or more processors;

One or more memories; the one or more memories store one or more programs, which when the one or more programs are executed by the one or more processors, cause the electronic device to execute claim 1 - The screen display control method in any one of claims 24 or the object motion detection method in any one of claims 25-29.
A computer-readable storage medium, characterized in that instructions are stored on the storage medium, and the instructions are executed on a computer. When executed, the computer is caused to execute the screen display control method in any one of claims 1-24 or the object motion detection method in any one of claims 25-29.
A computer program product, characterized in that the computer program product includes instructions, which when executed, cause the computer to execute the screen display control method of any one of claims 1-24 or to execute any one of claims 25-29. An object motion detection method.