Method and device for detecting intelligent equipment placed on display screen
Technical Field
The invention relates to the field of multi-screen interaction or cross-screen interaction, in particular to a method and a device for detecting that intelligent equipment is placed on a display screen.
Background
The multi-screen interaction means that a series of operations such as transmission, analysis, display, control and the like of multimedia (audio, video and picture) contents can be performed on different multimedia terminal devices (such as a common mobile phone and a common television) through wireless network connection, the same contents can be displayed on different terminal devices, and the content intercommunication among all terminals is realized.
The prior art WO2016066079a1 discloses a multi-screen interaction method and system, including acquiring the position of a fixed terminal, and monitoring the position of a mobile terminal; judging whether the position of the mobile terminal is within a set range from the position of the fixed terminal, if so, automatically butting the mobile terminal with the fixed terminal, and performing multi-screen interaction.
However, in the field of multi-screen interaction or cross-screen interaction, in the prior art, a single sensor is used for device discovery or detection, so that connection or docking is established, or manual selection is added for docking based on sensor detection. The prior art scheme is not intelligent enough a bit, needs to add manual selection a bit, and can not solve a plurality of mobile terminal's in the fixed terminal settlement distance range problem based on single sensor.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method and a device for detecting that intelligent equipment is placed on a display screen, wherein the method and the device have the advantages of high intelligent degree, extremely high detection precision and low implementation cost.
The purpose of the invention can be realized by the following technical scheme:
according to one aspect of the invention, a method for detecting that a smart device is placed on a display screen is provided, which comprises the following steps:
step 1), acquiring a continuous analog quantity value through a continuous analog quantity sensor on intelligent equipment;
step 2), detecting whether the last N times of continuous analog quantity values meet one of the set conditions, if so, turning to the step 3, otherwise, turning to the step 1, wherein N is more than 1;
step 3), calculating a numerical value D representing the distance between the intelligent equipment and the display screen through the data acquired by the distance sensor, if the numerical value D is smaller than a set threshold value D, turning to the step 4, and if not, turning to the step 1;
and 4), giving out a detection result that the intelligent equipment is placed on the display screen.
As a preferable technical solution, the setting conditions in step 2) specifically include:
a) the absolute value of the difference between the N times of analog quantity values and the calibration zero value is smaller than a set threshold, or the absolute value of the difference between the average value of the N times of analog quantity values and the calibration zero value is smaller than the set threshold;
b) the absolute value of the difference between the extreme value M and the calibrated zero value is greater than the set tolerance value M, and after the extreme value M, the absolute value of the difference between the analog magnitude value from (N-N) th to Nth times and the calibrated zero value is less than the set threshold value, or the absolute value of the difference between the average value of the analog magnitude values from (N-N) th to Nth times and the calibrated zero value is less than the set threshold value, wherein N < N;
c) of the N analog quantities, the absolute value of the difference between the analog quantities appearing from the N1 th to the (N1+ a) th times and the calibrated zero value is less than the set threshold or the absolute value of the difference between the mean of the analog quantities appearing from the N1 th to the (N1+ a) th times and the calibrated zero value is less than the set threshold, where N1+ a < N-1, and after (N1+ a) times, an extreme value M2 occurs, the absolute value of the difference between M2 and the calibrated zero value is greater than the set tolerance value M2, and after an extreme value M2, the absolute value of the difference between the analog quantities appearing from the (N-N2) th to the N th times and the calibrated zero value is less than the set threshold or the absolute value of the difference between the mean of the analog quantities appearing from the (N-N2) th to the N th times and the calibrated zero value is less than the set threshold, where N1+ a < N2< N.
As a preferred technical solution, the calibration zero value is an analog value of the sensor when the smart device is placed on the display screen or is ready to be placed on the display screen but not yet in contact with the display screen.
As a preferred technical solution, the average of the N analog quantities is calculated as:
a) a weighted average of analog quantities, equivalent to an arithmetic average when the weight is an average;
or b) removing the weighted average of the highest and lowest values;
or c) traversing the analog quantities X, XiRepresents the ith analog quantity value, if Abs (X)i-Xi-1)>e, then XiDoes not participate in the mean calculation, otherwise XiParticipating in mean value calculation; abs (X)i-Xi-1) The absolute value of the difference value between the ith analog quantity value and the (i-1) th analog quantity value is represented, and e represents the set tolerance value.
As a preferred technical solution, the continuous analog sensor is one of the following sensors, or a combination of multiple sensors:
a) a sound sensor;
b) an acceleration sensor;
c) a gyroscope sensor;
d) a gravity sensor.
As a preferred technical solution, the distance sensor is a single one of the following sensors, or a combination of multiple sensors:
a) a sound sensor;
b) bluetooth iBeacon;
c) a camera on the smart device;
d) a radar sensor;
e) an infrared sensor.
According to another aspect of the present invention, there is provided an apparatus for detecting placement of a smart device on a display screen, including:
the acquisition analog quantity module is used for acquiring a continuous analog quantity value through a continuous analog quantity sensor on the intelligent equipment;
the continuous analog quantity value judging module is connected with the analog quantity value collecting module and is used for detecting whether the last N times of continuous analog quantity values meet one of the set conditions;
the distance calculation and judgment module is connected with the continuous analog quantity value judgment module and used for calculating a numerical value D representing the distance between the intelligent equipment and the display screen and judging whether the numerical value D is smaller than a set threshold value D or not;
and the result output module is connected with the distance calculation and judgment module and used for outputting the information of the intelligent equipment placed on the display screen.
As a preferable technical solution, the setting conditions specifically include:
a) the absolute value of the difference between the N times of analog quantity values and the calibration zero value is smaller than a set threshold, or the absolute value of the difference between the average value of the N times of analog quantity values and the calibration zero value is smaller than the set threshold;
b) the absolute value of the difference between the extreme value M and the calibrated zero value is greater than the set tolerance value M, and after the extreme value M, the absolute value of the difference between the analog magnitude value from (N-N) th to Nth times and the calibrated zero value is less than the set threshold value, or the absolute value of the difference between the average value of the analog magnitude values from (N-N) th to Nth times and the calibrated zero value is less than the set threshold value, wherein N < N;
c) of the N analog quantities, the absolute value of the difference between the analog quantities appearing from the N1 th to the (N1+ a) th times and the calibrated zero value is less than the set threshold or the absolute value of the difference between the mean of the analog quantities appearing from the N1 th to the (N1+ a) th times and the calibrated zero value is less than the set threshold, where N1+ a < N-1, and after (N1+ a) times, an extreme value M2 occurs, the absolute value of the difference between M2 and the calibrated zero value is greater than the set tolerance value M2, and after an extreme value M2, the absolute value of the difference between the analog quantities appearing from the (N-N2) th to the N th times and the calibrated zero value is less than the set threshold or the absolute value of the difference between the mean of the analog quantities appearing from the (N-N2) th to the N th times and the calibrated zero value is less than the set threshold, where N1+ a < N2< N;
the calibration zero value is an analog value of the sensor when the intelligent device is placed on the display screen or is ready to be placed on the display screen but is not in contact with the display screen;
wherein the mean of the N times of analog values is calculated as:
a) a weighted average of analog quantities, equivalent to an arithmetic average when the weight is an average;
or b) removing the weighted average of the highest and lowest values;
or c) traversing the analog quantities X, XiRepresents the ith analog quantity value, if Abs (X)i-Xi-1)>e, then XiDoes not participate in the mean calculation, otherwise XiParticipating in mean value calculation; abs (X)i-Xi-1) The absolute value of the difference value between the ith analog quantity value and the (i-1) th analog quantity value is represented, and e represents the set tolerance value.
As a preferred technical solution, the continuous analog sensor is one of the following sensors, or a combination of multiple sensors:
a) a sound sensor;
b) an acceleration sensor;
c) a gyroscope sensor;
d) a gravity sensor.
As a preferred technical solution, the distance sensor is a single one of the following sensors, or a combination of multiple sensors:
a) a sound sensor;
b) bluetooth iBeacon;
c) a camera on the smart device;
d) a radar sensor;
e) an infrared sensor.
Compared with the prior art, the invention has the following advantages:
1) the intelligent mobile terminal has high intelligent degree, and the accuracy of a detection result is greatly improved and the problem of selection of a plurality of mobile terminals is solved by adopting a combination mode of a continuous analog quantity sensor and a distance sensor of the intelligent terminal;
2) the detection precision is extremely high, and the detection precision is ensured from the source by adopting the matching of N times of analog values and a calibration zero value;
3) the realization cost is low, after the continuous analog quantity sensor of the intelligent terminal is adopted for detection, only a low-precision distance sensor is needed for further detection, and compared with the prior art that a single high-precision distance sensor is adopted, the cost is reduced a lot;
4) the realization mode is diversified, and continuous analog quantity sensor can adopt single sensor also can adopt the combination sensor, and the distance inductor also can adopt single sensor also can adopt the combination sensor simultaneously, therefore the realization mode is fairly changeable.
Drawings
FIG. 1 is a schematic view of a scenario in which a mobile phone according to an embodiment of the present invention is placed on a touch display screen;
FIG. 2 is an enlarged view of portion A of FIG. 1;
FIG. 3 is a graph showing an acceleration value of the mobile phone stopping for 2 seconds after shaking the mobile phone handle;
FIG. 4 is a graph of the mean square error of the acceleration values of FIG. 3 and 30 acceleration values thereafter;
FIG. 5 is a graph of acceleration values for a cell phone resting on a table from a hand for 2 seconds;
FIG. 6 is a graph of the mean square error of the acceleration values of FIG. 5 and 30 acceleration values thereafter;
FIG. 7 is a graph showing the comparison of the maximum acceleration mean square deviation values between the state where the mobile phone is placed on the hand and the state where the mobile phone is placed on the table.
Wherein 1 is the display screen, 2 is the smart machine, and 3 is the infrared Led lamp of deployment in the touch table four sides.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
The smart device in this embodiment is a mobile phone, the display screen is a touch table equipped with bluetooth iBeacon, and the mobile phone and the touch table establish communication through a network. The method for detecting the placement of the mobile phone on the display screen of the touch table specifically comprises the following steps:
1. the touch table is provided with Bluetooth iBeacon, and the iBeacon broadcasts Bluetooth BLE service with UUID as UUID at a time interval of 200 ms;
2. app starts bluetooth scanning on the cell-phone, scans UUID for UUID's bluetooth signal, detects the cell-phone and has been close to the desk:
a) the RSSI of the Bluetooth signal strength value is initialized to be infinite, if the RSSI is not scanned, the RSSI is infinite, and if the RSSI is scanned, a specific value is taken;
b) converting the scanned RSSI value into a distance value by the following formula:
d=10^((abs(RSSI)-A)/(10*n))
wherein: d represents a distance; abs (rssi) represents the absolute value of the received bluetooth signal strength; a represents the RSSI signal strength absolute value when the transmitting end and the receiving end are separated by 1 meter; n is a preset empirical constant; 10 x represents the power x of 10;
c) taking the distance d obtained by the latest M times of calculation, removing the maximum and minimum values, and taking the average value as the currently scanned distance value Dcur;
d) if Dcur is less than 1.5 meters, the mobile phone is close to the touch table;
3. the touch module on the touch table scans periodically, and when an object meeting the following conditions is detected, the mobile phone App is notified that a suspected mobile phone object is detected:
a) the absolute value of the physical width of the object minus the physical width of the mobile phone is within 15 centimeters;
b) the absolute value of the physical height of the object minus the physical height of the mobile phone is within 16 cm;
4. after the mobile phone App receives a message from the touch table at time t1, wherein the message is that a suspected mobile phone object is detected, the display screen on the touch table is in a lighting working state, the camera is opened by the mobile phone App, and whether the mobile phone is placed on the lighting display screen of the touch table is detected:
a) acquiring n images after t1 by using a camera;
b) for each image, traversing pixels on the image, and calculating the overall average brightness of the image;
c) if the overall average brightness of the image is larger than a certain threshold value, the mobile phone App is possibly placed on a display screen of the touch table;
5. the mobile phone App receives a message from the touch table at time t1 that a suspected mobile phone object is detected, and detects whether the mobile phone is placed on the table according to the following steps:
a) sampling acceleration sensor values after t1 at a time interval of 50ms, wherein Acc (X, n) represents acceleration values sampled at the nth time in the X direction, similarly, Acc (Y, n), Acc (Z, n) respectively represents acceleration values sampled at the nth time in the Y direction and the Z direction, and n is in a value range [1,90 ]; the first sampling time n after time t1 is equal to 1;
b) calculating the mean square error STDEV (x, n1,30) of 30 times of acceleration values Acc (x, n1) to Acc (x, n1+29) after the n1 time sampling, similarly calculating the mean square error STDEV (y, n1,30) in the Y, Z direction, STDEV (z, n1,30), n1 value range [0,60], n1> -n;
c) attempt to find a quiescent state: if the mean square error of the acceleration of the sample at the n2 th time (n2> ═ n, namely the sample after the time t1, n2 takes a value range [0,60]), and the maximum value of the mean square error of the three directions is less than 0.1, namely Max (STDEV (x, n2,30), STDEV (y, n2,30), STDEV (z, n2,30)) < 0.1), then the static state is found, and the sample at the n2 th time corresponds to the starting time t2 of the static state;
d) if a static state is found, the static starting time is set as t2, namely the n2 th sampling, the state of the violent jump change is searched forward: calculating the Range (x, n3) of the polar difference between the n3 th sample and the acceleration value Acc (x, n3) before 20 samples to Acc (x, n3-19), namely Max (Acc (x, n3) to Acc (x, n3-19)) -Min (Acc (x, n3) to Acc (x, n3-19)), and if the maximum value of the polar difference in the three directions meets Max (Range (n3)) >20, indicating that a severe jump change state is found, and marking the minimum n3 as the end time t3 of the severe jump change state; n3 has a value range of-90, 0 representing the sample of the last before the time t1, and-90 representing the sample of the last 90 before the time t 1;
e) if the acceleration value meets the condition of finding a static state of 5.c) and finding a violent jump change state of 5.d), the mobile phone is placed on the display screen of the touch table with a high probability;
6. when the above conditions are met, the mobile phone is placed on the touch table display screen in the working state at a high probability, and the mobile phone App sets the state value in the step 5 as follows: state (t2, Max (STDEV (n2)), t3, Max (Range (n3))) is sent to the table end; if the table terminal finds that a plurality of mobile phone apps send back state values meeting the conditions, a most reasonable state value is selected as a matched mobile phone, and the method comprises the following steps: it is most reasonable to take the Max (STDEV (n2)) smallest and t2-t1 smallest.
Fig. 3 to 7 show the characteristics of the change in the acceleration value sampled during the process of placing the mobile phone on the display screen, and the change in the acceleration value sampled during the process of simulating the action of placing the mobile phone on the display screen when the mobile phone is held by hand.
The horizontal axis shown in fig. 3 corresponds to sampling time, the time interval is 50ms, and the vertical axis corresponds to acceleration values sampled by the mobile phone, and the acceleration values respectively have sampling values of the acceleration sensors in the X direction, the Y direction and the Z direction; fig. 3 shows the handset held in hand, and the handset has undergone two simulated motion sequences: the horizontal hand-held mobile phone position in short time descends and ascends again, the horizontal axis in fig. 3 represents corresponding sampling time (time interval 50ms), the processes of two times of short-time descent and ascent of the mobile phone position in short time correspond to the 37 th sampling time and the 115 th sampling time, and it can be obviously found that severe jump change of acceleration occurs in the process, and the processes of horizontal hand-held mobile phone position and immovable keeping for a period of time correspond to the periods of time after the 49 th sampling time and the 127 th sampling time.
Fig. 4 shows a process of calculating the mean square error of the values of fig. 3 of nearly 30 times, for example, the vertical axis value corresponding to the horizontal axis 1 is the mean square error of the vertical axis value corresponding to the horizontal axis interval [1,30] in fig. 3, and it can be found that the mean square error value is significantly large in the horizontal axis intervals [6,38] and [91,121], which actually corresponds to the process of two times of short-time falling and rising of the mobile phone bang position.
The horizontal axis shown in fig. 5 corresponds to sampling time, the time scale is 50ms, and the vertical axis corresponds to acceleration values sampled by the mobile phone, and the acceleration values respectively have sampling values of the acceleration sensors in the X, Y and Z directions; the handset shown in fig. 5 undergoes a sequence of two actions: the mobile phone is held horizontally, the mobile phone touches the display screen at the first time in the process that the mobile phone is placed on the display screen in a rigid mode (the top of the mobile phone touches the display screen), the mobile phone is placed on the display screen, extreme values (maximum values or minimum values) of two jump changes in the Y and Z directions appear in intervals [60,75] and [169,180] of the horizontal axis, a section of continuous small-amplitude change value appears in the intervals [75,115] and [190,239], and the values in the X, Y and Z directions are the same and small in amplitude change.
FIG. 6 shows the calculation of the mean square error of the values of FIG. 5 for nearly 30 times, for example, the mean square error of the values of the vertical axis in the interval [25,55] of the horizontal axis in FIG. 5 is associated with the value of the vertical axis in the interval [79,91] and the interval [187,211] of FIG. 6 is close to 0.
FIG. 7 illustrates the maximum of the three vertical axis values in FIGS. 4 and 6, such as at horizontal axis 43, where the vertical axis value for the light color line is the maximum of the three vertical axis values at horizontal axis 43 in FIG. 4, and the vertical axis value for the dark color line is the maximum of the three vertical axis values at horizontal axis 43 in FIG. 6; fig. 7 clearly shows that dark lines close to 0 appear in the intervals [79,91] and [187,211], actually corresponding to the static state of the mobile phone during the period of standing on the display screen, whereas before the two intervals, the interval [65,75] in fig. 5 shows the range of 12, and the interval [169,180] shows the range of 18, actually corresponding to the process of the rigid touch of the mobile phone on the display screen; the light line corresponds to the simulated operation of the handheld mobile phone, and the static state with the maximum value close to 0 is not found.
The intelligent device in the above embodiment is a mobile phone, and the mobile phone can be replaced by a pad or other handheld terminals with corresponding functions, which also belongs to the protection scope of the present invention;
the display screen in the above embodiment is a touch table equipped with bluetooth iBeacon, and the touch table may be replaced with a display screen device equipped with one or more combinations of a sound sensor, a camera, a radar sensor and an infrared sensor, and also belongs to the protection scope of the present invention.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.