CN116795231A - Writing material identification method based on elastic wave and interaction panel - Google Patents
Writing material identification method based on elastic wave and interaction panel Download PDFInfo
- Publication number
- CN116795231A CN116795231A CN202210266653.9A CN202210266653A CN116795231A CN 116795231 A CN116795231 A CN 116795231A CN 202210266653 A CN202210266653 A CN 202210266653A CN 116795231 A CN116795231 A CN 116795231A
- Authority
- CN
- China
- Prior art keywords
- touch
- elastic wave
- detection device
- wave signal
- medium type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 239000000463 material Substances 0.000 title claims abstract description 51
- 230000003993 interaction Effects 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 claims abstract description 174
- 238000012545 processing Methods 0.000 claims abstract description 129
- 230000002452 interceptive effect Effects 0.000 claims abstract description 75
- 238000013441 quality evaluation Methods 0.000 claims abstract description 60
- 230000004044 response Effects 0.000 claims abstract description 41
- 239000000284 extract Substances 0.000 claims description 2
- 239000002609 medium Substances 0.000 description 139
- 230000008569 process Effects 0.000 description 47
- 238000010586 diagram Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 13
- 238000004891 communication Methods 0.000 description 9
- 239000000872 buffer Substances 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 5
- 230000000903 blocking effect Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000007499 fusion processing Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000001303 quality assessment method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241001422033 Thestylus Species 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000007789 gas Substances 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 239000012120 mounting media Substances 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Abstract
The embodiment of the disclosure discloses a writing material identification method based on elastic waves and an interaction panel. According to the technical scheme, the touch sensor detects touch data generated when the touch object touches the touch detection area of the interactive panel, the elastic wave detection device detects an elastic wave signal generated when the touch object touches the display screen, quality evaluation is conducted on the elastic wave signal according to time information in the touch data, the medium type is determined according to the quality evaluation result and the elastic wave signal, the recognition accuracy of the touch object is improved by combining the elastic wave signal and the quality evaluation result, and medium type information of the medium type containing the touch object is sent to the processing equipment, so that the processing equipment can execute different response operations according to different medium types of the touch object, operation is simple, and efficiency is high.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of electronic equipment control, in particular to a writing material identification method based on elastic waves and an interaction panel.
Background
Touch operation is performed on the display screen by using a touch sensor (such as an infrared touch frame, a capacitive touch screen, an electromagnetic touch screen and the like), and the touch operation is one of the common touch modes of the interactive flat panel, namely, the touch operation performed on the display screen is identified by the touch sensor, so that corresponding touch data is obtained and sent to a processor of the interactive flat panel. The interactive panel executes corresponding operations according to the touch data, for example, corresponding writing tracks are displayed on a display screen of the interactive panel according to the movement track of the touch object.
In the use process of the interactive flat panel, the response to the touch operation is basically only based on specific input judgment of the touch track, for example, track input or simple instruction receiving based on the touch track, when the input effect of the touch track is to be changed or a more complex control instruction is to be realized, the target control can be realized according to the combination of a plurality of touch inputs, for example, in the writing process, the setting of the display attribute of the writing track or the switching of the writing and erasing operation modes can be realized by the more complex combination of the touch operation, the operation is more complicated, and the efficiency is lower.
Disclosure of Invention
The embodiment of the disclosure provides a writing material identification method based on elastic waves and an interactive flat plate, which are used for solving the technical problems that in the prior art, the interactive flat plate needs more complicated touch operation combination to realize a setting target, and the operation is complex and the efficiency is low.
In a first aspect, an embodiment of the present disclosure provides a writing material identification method based on elastic waves, which is applied to an interactive tablet, where the interactive tablet includes a display screen, a processing device, and a touch sensor, and the interactive tablet further includes an elastic wave detection device, and the method further includes:
the touch data are generated when a touch object performs touch operation in a touch detection area of the interaction panel; the touch data includes touch time information;
the processing device receives the touch data sent by the touch sensitive sensor;
the elastic wave detection device receives the touch data, wherein the touch data is sent to the elastic wave detection device by the touch sensor or the processing equipment;
the elastic wave detection device detects an elastic wave signal; the elastic wave signal is generated when the touch object contacts the display screen of the interaction panel;
The elastic wave detection device carries out signal quality evaluation according to the elastic wave signals in different time periods;
the elastic wave detection device determines the medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result;
the elastic wave detection device sends medium type information to the processing equipment, wherein the medium type information comprises a medium type corresponding to the touch object.
In a second aspect, an embodiment of the present disclosure provides an interactive tablet, including a display screen, a processing device, a touch-sensitive sensor, and an elastic wave detection device, where the display screen, the touch-sensitive sensor, and the elastic wave detection device are all connected to the processing device;
the touch sensor is used for detecting touch data, and the touch data are generated when a touch object performs touch operation in a touch detection area of the interaction panel; the touch data includes touch time information;
the elastic wave detection device is used for detecting an elastic wave signal, wherein the elastic wave signal is generated when the touch object contacts the display screen of the interaction panel; and
after receiving the touch data sent by the touch sensor or the processing device, performing signal quality evaluation according to the elastic wave signals of different time periods; determining a medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result; and
Transmitting medium type information to the processing equipment, wherein the medium type information comprises a medium type corresponding to the touch object;
the processing device is used for receiving the touch data sent by the touch-sensitive sensor.
According to the embodiment of the disclosure, the touch sensor detects touch data generated when a touch object touches a touch detection area of the interactive panel, the elastic wave detection device detects an elastic wave signal generated when the touch object touches the display screen, quality evaluation is performed on the elastic wave signal according to time information in the touch data, a medium type is determined according to a quality evaluation result and the elastic wave signal, the recognition accuracy of the touch object is improved by combining the elastic wave signal and the quality evaluation result, and medium type information of the medium type containing the touch object is sent to the processing equipment, so that the processing equipment can execute different response operations according to different medium types of the touch object, and the method is simple to operate and high in efficiency.
Drawings
FIG. 1 is a schematic structural diagram of an interactive tablet provided by an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of a method for identifying writing materials based on elastic waves according to an embodiment of the disclosure;
Fig. 3 is a schematic diagram of a detection principle of an infrared touch frame according to an embodiment of the disclosure;
FIG. 4 is a schematic flow chart of signal quality evaluation on an elastic wave signal according to an embodiment of the disclosure;
FIG. 5 is a schematic flow chart of quality evaluation of an elastic wave signal according to second characteristic information and first characteristic information according to an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a media type determination process provided by an embodiment of the present disclosure;
FIG. 7 is a schematic diagram of an elastic wave signal during writing provided by an embodiment of the present disclosure;
FIG. 8 is a logic diagram of touch data processing provided by an embodiment of the present disclosure;
fig. 9 is an operation schematic diagram of a touch operation provided in an embodiment of the disclosure;
FIG. 10 is a schematic illustration of a display of a writing trace provided by an embodiment of the present disclosure;
fig. 11 is a block diagram of an interactive tablet provided by an embodiment of the present disclosure.
Detailed Description
For the purposes of promoting an understanding of the principles and technical aspects of the disclosure, reference will now be made in detail to specific embodiments thereof with reference to the drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the present disclosure and not limiting thereof. For ease of description, only some, but not all, of the matters relevant to the present disclosure are shown in the accompanying drawings. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
It should be noted that in this disclosure, relational terms such as first and second are used solely to distinguish one entity or action or object from another entity or action or object without necessarily requiring or implying any actual such relationship or order between such entities or actions or objects. For example, "first" and "second" of a first media type and a second media type are used to distinguish between two different media types, such as a writing tip and a pen tail. The specific meaning of the terms in this disclosure will be understood by those of ordinary skill in the art as the case may be. Furthermore, in the description of the present disclosure, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
For easy understanding, in the embodiment, the interactive flat board is taken as an actual carrier, and detailed processes of detection of touch operation, judgment of touch operation and response display of touch operation when the writing material identification based on elastic waves is implemented are exemplarily described. The interactive flat panel can be integrated equipment for controlling the content displayed on the display flat panel and realizing man-machine interaction operation through a touch technology, and one or more functions of a projector, an electronic whiteboard, a curtain, a sound, a television, a video conference terminal and the like are integrated.
Fig. 1 is a schematic structural diagram of an interactive tablet provided by the present disclosure, as shown in fig. 1, in this embodiment, an infrared touch frame 300 is taken as an example of a touch sensor, corresponding to the touch data generated by an infrared touch block is infrared touch data, the interactive tablet 100 includes at least one display screen 500, and in addition, the interactive tablet 100 further includes a processing device 200, the infrared touch frame 300 and an elastic wave detection device 400, where the infrared touch frame 300 is disposed around the interactive tablet 100 to form a touch detection area of the interactive tablet 100, and four sides of the interactive tablet 100 refer to four sides of the interactive tablet 100. The touch detection area is an area for detecting a touch operation occurring on the interactive pad 100, and is an area in a physical sense. If the border of the interactive tablet 100 is relatively narrow, and the display screen 500 is near the border of the interactive tablet 100, the infrared touch frame 300 may be considered to be disposed around the display screen 500, and in a specific application, the position of the infrared touch frame 300 may be set according to the area actually required to be detected, or may be disposed only around the display screen 500, or may be disposed around the entire interactive tablet 100. The infrared touch frame 300 can detect infrared touch data generated by touch operation of a touch object at any position on a touch detection area, the infrared touch frame 300 and the elastic wave detection device 400 communicate with the processing device 200 based on a USB protocol, that is, the infrared touch frame 300 and the elastic wave detection device 400 are connected with the processing device 200 through USB interfaces, in fig. 1, when only one USB interface of the processing device 200 needs to be connected with the infrared touch frame 300 and the elastic wave detection device 400 at the same time, a port transceiver (HUB) is set to expand a connection port, so that communication connection with the infrared touch frame 300 and the elastic wave detection device 400 is realized. The infrared touch frame 300 and the elastic wave detection device 400 can also directly perform data interaction through an IO port, a USB protocol or a serial port.
The user can use an object such as a finger or a writing pen as a touch object, and touch operations are realized by clicking or moving the touch object on a display screen of the interactive flat panel, accordingly, the infrared touch frame 300 detects the touch operations of the touch object in the touch detection area, corresponding infrared touch data are generated and sent to the interactive flat panel, and after the interactive flat panel receives the infrared touch data, the interactive flat panel responds according to the touch position represented by the infrared touch data, so that a touch function is realized. For example, if a control is located at the touch position represented by the infrared touch data, the interaction panel responds to the control, or if the touch position represented by the infrared touch data is a track, the interaction panel generates a corresponding writing track according to the infrared touch data. Here, the control refers to an element of a graphical interface of the application software or the operating system, such as a page turning control, a minimizing control, and the like, and by touching the control, the application software or the operating system of the interactive tablet can execute a predefined function, for example, after clicking the page turning control, the application software of the interactive tablet correspondingly performs page turning processing. Typically, the interactive tablet is installed with at least one operating system, wherein the operating system includes, but is not limited to, an android system, a Linux system, and a Windows system. As an embodiment, the interactive tablet may install at least one application program having a writing function. The application program can be an application program of an operating system, and meanwhile, the application program downloaded from a third party device or a server is also installed. Optionally, the application program has other editing functions, such as form insertion, picture insertion, form drawing, figure drawing, and the like, in addition to the writing function based on the touch operation.
The graphical interface displayed in the display screen of the interactive tablet corresponds to a projected area of the touch detection area on the display screen for detecting user operation (i.e., the touch detection area is in front of the graphical interface). When the touch detection area is in touch operation, for example, a writing pen or a finger is contacted with a display screen of the interactive panel, an infrared touch frame of the display screen can sense the change of infrared light, a touch signal containing coordinates of a touch position and the triggering time of the touch signal are obtained, infrared touch data reflecting a writing track input in the process from each time the writing pen or the finger of a user falls into the lifting writing process to stopping writing can be obtained according to the coordinates of the touch position and the triggering time of the touch signal, and the writing track input by the user is displayed on a graphical interface of the display screen in real time according to the infrared touch data. Of course, the writing process is not limited to writing, and any entry process for displaying an operation trace implemented on the display screen of the interactive tablet may be regarded as the writing process. Further, the touch operation may be responsive to a click, a drag, or the like operation by the user according to a display element of the occurrence position of the touch operation. These different response patterns are identical at the underlying coordinate processing pattern and ultimately visually result in a change in the displayed picture on the display screen. Based on the hardware realization of basic functions such as touch and display, various interactive display designs of the interactive tablet can be realized.
The touch operation performed in the graphical interface presented during the use of the whiteboard application, i.e., the interface of the whiteboard application, is explained as an example. As before, the whiteboard application refers to an application for a user to perform writing, displaying, and other operations, and may be used to generate writing handwriting according to writing operations (touch operations) of the user on an interface of the whiteboard application, and may also be used to insert and display multimedia elements on the interface of the whiteboard application, where the multimedia elements may include graphics, images, tables, documents, audio files, and/or video files. In the interface of the whiteboard application, the user can realize writing, drawing, erasing and other operations similar to the physical blackboard, and further has better digital functions of moving, saving, scaling, inserting pictures, adjusting colors, setting stroke weights and the like. In practical applications, the whiteboard application may also be named as a writing application, an electronic whiteboard application, a collaborative whiteboard application, or the like, regardless of the change in the names, as long as the application for realizing the above functions is equivalent to the whiteboard application of the present disclosure.
As an implementation mode, the elastic wave detection device provided by the scheme can be arranged on a display screen and used for detecting an elastic wave signal generated when a touch object touches the display screen of the interactive flat panel. For example, the elastic wave detection device comprises an elastic wave acquisition plate and at least one elastic wave sensor, wherein the piezoelectric elastic wave sensor is arranged at the frame of the display screen or at the inner side of the cover plate, and when a touch object touches the display screen, an elastic wave signal with characteristics is generated, and the elastic wave signal propagates from a contact point, along the display screen to the periphery or propagates to the inner side of the display screen. The piezoelectric elastic wave sensor positioned at the frame of the display screen or at the inner side of the cover plate can convert an elastic wave signal into a voltage signal, the voltage signal is sent to the elastic wave acquisition plate, the voltage signal is transmitted to the IC chip with the temperature compensation circuit through the elastic wave acquisition plate for amplification treatment, and the voltage signal is converted into a digital elastic wave signal through the analog-to-digital conversion circuit. It should be noted that, the position set by the elastic wave detection device is not limited specifically in this scheme, so long as the elastic wave signal generated by the touch object touching the display screen of the interactive panel can be detected.
For mechanical vibrations below the audio frequency, sounds in the audio frequency range, which are ultrasonic waves exceeding the audio frequency, are wave phenomena of a medium such as gas, liquid, solid, etc., which wave phenomena are called elastic waves with respect to light and electromagnetic waves. For the detection of the elastic wave signal, the detection is completed through the elastic wave sensor, and the elastic wave sensor is specifically arranged at a position capable of transmitting vibration generated by the display screen, so that an event that a touch object on the display screen touches the display screen is realized, and the touch object is not necessarily arranged at the position where the vibration occurs. For example, the elastic wave sensors may be arranged at 4 corners of the display screen, or other arrangement manners may be provided, for example, the elastic wave sensors may be arranged at the middle points of each side of the rectangular frame of the display screen, and the number of the elastic wave sensors may be other numbers, for example, 2 or 5, so long as the elastic wave sensors can detect vibration when a touch object contacts the display screen in the touch operation process, the number of specific arrangements may be set in a targeted manner according to the size and detection precision of the display screen, and in general, the larger the size of the display screen, the higher the detection precision requirement is, and the more the number of the elastic wave sensors is set. The elastic wave sensor can be directly arranged on the surface of the display screen, for example, directly arranged on the upper surface of the display screen or the lower surface of the display screen, so that vibration transmitted by the display screen is received, and the accuracy of touch detection is improved. The elastic wave sensor can also be arranged in the frame of the display screen, so that the influence on an internal mechanism is reduced, and meanwhile, the common mode noise interference from the display screen is reduced. Of course, the elastic wave sensor may be mounted on another member in contact with the display screen, and the vibration generated on the display screen may be received by transmission of the other member.
From the source of elastic wave generation, the device comprises various noise-generating elements (such as a motor, a loudspeaker, a fan and the like) in the interactive flat panel and noise-generating devices (such as indoor air conditioners, televisions and the like) outside the interactive flat panel, and the elements can cause the interactive flat panel to generate elastic waves which are not controlled by users in the use process. By means of the type of the element, it can be confirmed that the element generates noise, for example, the noise generated by the loudspeaker mainly comes from elastic waves generated in the whole medium of the interactive flat panel caused by the sound generated by the loudspeaker, and various elastic wave signals can be detected at any time for the elastic wave sensor, but many of the elastic wave signals are not from the touch operation itself.
Fig. 2 is a schematic flow chart of a writing material identification method based on elastic waves according to an embodiment of the present disclosure, where the writing material identification method based on elastic waves provided by the embodiment of the present disclosure is applied to an interaction panel, and the interaction panel may be formed by two or more physical entities or one physical entity. For example, the interactive tablet may be a conference tablet, a teaching tablet, an industrial control computer, etc. for enabling content display and interactive response to human-machine interaction.
Referring to fig. 2, the method for recognizing writing materials based on elastic waves includes:
s101: the touch sensor detects touch data, and the touch data is generated when a touch object performs touch operation in a touch detection area of the interaction panel.
According to the scheme, aiming at the situation that the generation of touch data is prior to the generation of elastic wave signals in the touch scheme, the quality evaluation is carried out on the elastic wave signals according to time information in the touch data, the medium type is determined according to the quality evaluation result and the elastic wave signals, and the recognition accuracy of a touch object is improved by combining the elastic wave signals and the quality evaluation result to determine the medium type. The touch sensor provided by the disclosure can be an infrared touch frame, a capacitive touch screen, an electromagnetic touch screen and the like. In this embodiment, an infrared touch frame is taken as an example of a touch sensor, and touch data generated by the infrared touch block is corresponding to infrared touch data.
The touch data includes touch time information, touch position information and/or touch parameter information, and the touch parameter information at least reflects a touch area of a touch point of a touch object when the touch object contacts the display screen, and the touch area can be represented by a width and a height corresponding to the touch area, and can also be represented by a radius or a diameter corresponding to the touch area.
For example, in order to realize the touch function of the display screen, an infrared touch frame is arranged around the surface of the display screen, and when a touch object touches the display screen, the infrared touch frame scans the touch object through an optical signal so as to detect infrared touch data generated when the touch object performs touch operation on the display screen. The infrared touch frame comprises an infrared emitter and an infrared receiver, wherein the infrared emitter is used for emitting infrared light, the infrared receiver is used for receiving the infrared light, and a light beam grid is formed by utilizing the infrared light densely distributed in different directions to position a touch point. When a user touches the display screen through a touch object, an infrared light transmission path between an infrared emitter and an infrared receiver arranged on the display screen is blocked by the touch object, the infrared touch frame outputs a corresponding infrared blocking signal, the contact area of the touch object and the display screen, the moving speed of the touch object on the display screen and the touch position when the touch object is in contact with the display screen can be measured according to the infrared blocking signal, and corresponding infrared touch data are obtained.
Fig. 3 is a schematic diagram of a detection principle of an infrared touch frame, as shown in fig. 3, taking an infrared touch frame as a touch sensor as an example, in one touch operation, a touch object may go through states 131-137 in fig. 3, where a process from starting a touch operation to starting to contact with a display screen 500 is described from state 131, state 132 to state 133, when the touch object just enters a touch detection area between state 131 and state 132, the infrared touch frame detects first frame touch data, and at this time, a touch state of the touch object is a touch falling state, in state 131, the touch object is above the touch detection area and does not generate infrared touch data, in state 132, the touch object is in the touch detection area, but does not touch a surface of a display screen 500, infrared touch data (first frame infrared touch data generated in the process is first frame touch data), in state 133, the touch object is in the touch detection area, has touched to the display screen 500, and infrared touch data is generated.
The process in which the touching object contacts the display screen 500 and moves on the display screen 500 is described from the state 133, the state 134 to the state 135, in which the touching state of the touching object is always a touch movement state in which the touching object continuously touches the display screen 500 while infrared touch data is generated.
The process of lifting the touch object from the display screen 500 to leave the touch detection area is described from the state 135, the state 136 to the state 137, when the touch object just leaves the touch detection area between the state 136 and the state 137, the infrared touch frame detects the last frame of touch data, and the touch state of the touch object is the touch lifting state at this time, in the state 136, the touch object is in the touch detection area and does not touch the display screen 500, infrared touch data can be generated, and in the state 137, the touch object is above the touch detection area and does not generate infrared touch data. It will be appreciated that in one touch operation, the infrared touch data generated during the states 132-136 is continuous infrared touch data.
When the touch object performs touch operation in the touch detection area of the interaction panel, the infrared touch frame generates infrared touch data, so that calculation is performed according to the infrared touch data, and the infrared touch data is sent to the processing equipment after the infrared touch data is correspondingly generated. Here, the infrared touch data generated by the infrared touch frame is one continuous data, and is embodied as data including a plurality of frames.
S102: the processing device receives touch data transmitted by the touch sensitive sensor.
When the processing device receives touch data transmitted by the touch sensitive sensor (e.g., infrared touch data transmitted by an infrared touch frame), it is necessary to determine when to respond to the received infrared touch data. By response, in embodiments of the present invention, it is meant whether the processing device processes infrared touch data as valid data. In one embodiment, the processing device processes the continuously received infrared touch data as valid data, requiring a pre-condition that: the processing device confirms that the touch object triggering the infrared touch data is in contact with the display screen of the interactive flat panel. In an exemplary embodiment of the present invention, an elastic wave detection device is disposed in the interactive flat panel, and a vibration signal (i.e., an elastic wave signal) generated on a display screen of the interactive flat panel is detected by the elastic wave detection device, and communication hardware and a communication protocol of the elastic wave detection device and a processing device of the interactive flat panel are disposed, so that the elastic wave detection device can send information (such as medium type information) confirming that a touch object triggering infrared touch data is in contact with the display screen of the interactive flat panel to the processing device, and help the processing device obtain the judging capability.
In one touch operation, when the touch object is approaching but not touching the display screen, in the process, the infrared emitter and the infrared receiver at corresponding positions detect the blocking of the infrared rays by the touch object, the touch sensor starts to detect touch data (for example, the infrared touch frame starts to detect infrared touch data), and the elastic wave detection device does not detect an elastic wave signal. Because the touch object does not contact the display screen at this time, the touch data generated in the process does not need to be processed, the touch data can be discarded, and the corresponding writing track does not need to be displayed on the display screen.
In one embodiment, when the processing device receives the touch data sent by the touch sensor but does not receive the medium type information sent by the elastic wave detection device, the processing device may not respond to the received touch data.
S103: the elastic wave detection device receives touch data, wherein the touch data is sent to the elastic wave detection device by a touch-sensitive sensor or a processing device.
In the embodiment of the invention, in order to diversify the response of the interaction panel to the touch operation of the user, different medium types based on the touch object can be designed, and the response or implementation mode corresponding to the medium types can be realized. By taking an infrared touch frame as an example of a touch sensor, the embodiment of the invention designs a new communication hardware circuit and a communication protocol of the infrared touch frame and the elastic wave detection device, so that the infrared touch frame can send infrared touch data to the elastic wave detection device when detecting infrared touch data, or the processing equipment sends the infrared touch data to the elastic wave detection device, and in the subsequent step, the elastic wave detection device can identify a noise signal (a first elastic wave signal) and a touch object signal (a second elastic wave signal) by combining touch time information in the infrared touch data.
Wherein the touch data is sent by the touch sensitive sensor or processing device to the elastic wave detection means, e.g. the infrared touch data is sent by the infrared touch frame or processing device to the elastic wave detection means. When the infrared touch frame is in communication connection with the elastic wave detection device, the infrared touch frame directly sends the infrared touch data to the elastic wave detection device, namely the infrared touch frame directly sends the infrared touch data to the elastic wave detection device when receiving the continuous infrared touch data, and simultaneously sends the continuous infrared touch data to the processing equipment. When the infrared touch frame is not in communication connection with the elastic wave detection device, the infrared touch frame transmits continuous infrared touch data to the processing equipment when detecting the continuous infrared touch data, and the processing equipment transmits the infrared touch data to the elastic wave detection device.
S104: the elastic wave detection device detects an elastic wave signal; the elastic wave signal is generated when the touch object contacts the display screen of the interactive flat panel.
When the touch object contacts the display screen of the interactive flat panel, an elastic wave signal with characteristics is generated, and the elastic wave signal starts from the contact point of the touch object and the display screen, propagates around the display screen or propagates to the inner side of the display screen, so that the elastic wave signal is detected by the elastic wave detection device. After the elastic wave signal is detected, the elastic wave signal is buffered by the elastic wave detection device.
S105: the elastic wave detection device performs signal quality evaluation according to the elastic wave signals in different time periods.
It can be understood that, in the process of executing the touch operation, when the infrared touch frame starts to detect the blocking of the touch object at the time t1, the touch area detected by the infrared touch frame does not reach the set touch area threshold, and at this time, the touch object does not touch the display screen. As the touch object approaches the display screen, when the infrared touch frame detects at time t2 that the touch area detected by the infrared touch frame reaches the set touch area threshold, the touch object is considered to contact the display screen, and corresponding touch data is sent to the elastic wave detection device. The elastic wave data collected by the elastic wave detection device before the time t2 can be understood as noise signals, the elastic wave data collected after the time t2 is touch object signals generated by touching the display screen by a touch object, the elastic wave signals in different time periods (time sequences) can be determined according to time information corresponding to the touch data of which the touch area reaches a set touch area threshold value, the touch object signals and the noise signals are distinguished from the elastic wave data in different time periods, signal quality evaluation is carried out on the elastic wave signals according to the comparison condition of the touch object signals and the noise signals, and quality evaluation results are obtained.
In one embodiment, the noise signal and the touch object signal may be distinguished according to time information corresponding to when the touch area reacted in the touch data reaches a set touch area threshold, and the signal quality of the elastic wave signal may be evaluated according to the noise signal and the touch object signal. Based on this, as shown in a flow chart of signal quality evaluation on an elastic wave signal provided in fig. 4, the elastic wave detection device provided in this embodiment specifically includes:
s1051: the elastic wave detection device determines a first elastic wave signal of the elastic wave signal before a touch object contacts with the display screen of the interaction panel and a second elastic wave signal of the elastic wave signal after the touch object contacts with the display screen of the interaction panel based on time information of the touch data.
The elastic wave detection device determines, based on time information of the infrared touch data, a second time corresponding to when a touch area (a shielding area of a touch object) detected by the infrared touch frame reaches a set touch area threshold. The elastic wave signal detected after the second moment is understood to be generated by touching the display screen by the touch object, the elastic wave signal generated before the second moment is determined to be the first elastic wave signal of the touch object before the touch object touches the display screen of the interactive flat panel, and the elastic wave signal generated after the second moment is determined to be the second elastic wave signal of the touch object after the touch object touches the display screen of the interactive flat panel.
In one embodiment, according to the time information of the touch data, a first time corresponding to when the touch sensor just starts to detect the blocking of the touch object is determined, and an elastic wave signal generated between the first time and the second time is determined as a first elastic wave signal of the touch object before the touch object contacts the display screen of the interactive panel.
Taking the example of generating infrared touch data by the infrared touch frame, the infrared touch data is sent to the elastic wave detection device by the infrared touch frame or the processing equipment. When a touch object performs touch operation in a touch detection area of the interaction panel, firstly, an infrared touch frame detects that infrared rays are blocked, then corresponding conversion calculation is performed, and specific touch position information, touch area and other touch parameter information are calculated according to the blocked condition. In this process, the elastic wave detection device detects the elastic wave signal, but may not receive the infrared touch data sent by the infrared touch frame, so it is necessary to wait for receiving the infrared touch data, and then combine the infrared touch data and the elastic wave signal to determine the first elastic wave signal and the second elastic wave signal.
S1052: the elastic wave detection device evaluates the signal quality of the elastic wave signal according to the first elastic wave signal and the second elastic wave signal.
The signal quality evaluation of the elastic wave signal refers to the signal quality evaluation of the elastic wave signal including the first elastic wave signal and the second elastic wave signal. The first elastic wave signal is an elastic wave signal generated before the touch object touches the display screen, so that the noise condition can be well reflected, the second elastic wave is an elastic wave signal generated when the touch object touches the display screen, the characteristics of the touch object can be accurately reflected, signal quality evaluation can be performed according to the comparison result of the second elastic wave signal and the first elastic wave, and whether the signal quality of the second elastic wave signal meets the set requirement is judged.
As shown in a schematic flow chart of quality evaluation of an elastic wave signal according to second characteristic information and first characteristic information provided in fig. 5, when the elastic wave detection device provided in this embodiment evaluates signal quality of an elastic wave signal according to a first elastic wave signal and a second elastic wave signal, the device specifically includes:
s10521: the elastic wave detection device extracts first characteristic information of the first elastic wave signal and second characteristic information of the second elastic wave signal.
S10522: the elastic wave detection device takes the ratio of the difference between the second characteristic information and the first characteristic information to the first characteristic information as the quality evaluation result of the elastic wave signal.
The elastic wave detection module respectively performs characteristic lifting on the first elastic wave signal and the second elastic wave signal after determining the first elastic wave signal and the second elastic wave signal, so as to obtain first characteristic information of the first elastic wave signal and second characteristic information of the second elastic wave signal. Features extracted from the elastic wave include, but are not limited to, time domain features (e.g., amplitude, morphology, etc.), frequency domain features (e.g., frequency response, energy, etc.).
Further, signal quality evaluation is performed on the elastic wave signal according to the first characteristic information and the second characteristic information. Illustratively, the difference between the second characteristic information and the first characteristic information is calculated, and the ratio of the difference between the second characteristic information and the first characteristic information to the first characteristic information is calculated, and the ratio is used as the quality evaluation result of the elastic wave signal.
Exemplary, the first elastic wave signal S is extracted noise First characteristic information F of (1) noise And a second elastic wave signal S object Second characteristic information F of (2) object The quality evaluation result of the elastic wave signal is:
in one embodiment, after the elastic wave detection device detects the elastic wave signal, the elastic wave signal is buffered until touch data is received, and then signal quality evaluation is performed according to the elastic wave signals in different time periods. For example, the elastic wave detection device buffers the detected elastic wave signal within a set time period in real time, so as to perform signal quality evaluation according to the infrared touch data according to the elastic wave detection device after receiving the infrared touch data in a subsequent step. Meanwhile, the elastic wave detection device clears away the elastic wave signals with the buffer time exceeding the set time length, so that the influence of interference signals can be filtered, unnecessary processing steps of the elastic wave detection device are avoided, and the processing efficiency is improved.
S106: and the elastic wave detection device determines the medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result.
The elastic wave detection device further determines whether the signal quality of the elastic wave signal reaches a set requirement according to the quality evaluation result after determining the quality evaluation result of the elastic wave signal, and identifies the medium type corresponding to the touch object according to the elastic wave signal or directly outputs a default medium type when the signal quality of the elastic wave signal does not reach the set requirement.
In one embodiment, whether the signal quality of the elastic wave signal meets the set requirement may be determined according to a comparison result between the quality evaluation result and the set threshold value. Based on this, as shown in a schematic diagram of a medium type determining flow provided in fig. 6, when determining a medium type corresponding to a touch object according to an elastic wave signal and a quality evaluation result, the elastic wave detecting device provided in this embodiment specifically includes:
s1061: and when the quality evaluation result of the elastic wave signal reaches a set threshold value, the elastic wave detection device determines the medium type corresponding to the touch object according to the elastic wave signal.
S1062: and when the quality evaluation result of the elastic wave signal is lower than a set threshold value, the elastic wave detection device determines that the medium type corresponding to the touch object is a default medium type.
It can be understood that when the quality evaluation result of the elastic wave signal is that the set threshold value is reached, the signal quality of the elastic wave signal reaches the set requirement at this time, and the current elastic wave signal is considered to have higher quality at this time, so that the characteristics of the touch object can be accurately and completely expressed, and the type of the medium corresponding to the touch object can be determined according to the elastic wave signal.
When the quality evaluation result of the elastic wave signal is that the set threshold value is not reached, the signal quality of the elastic wave signal does not reach the set requirement at the moment, and the current elastic wave signal is considered to be low in quality at the moment, the characteristics of the touch object cannot be accurately and completely expressed, and then the type of the medium corresponding to the touch object is directly determined to determine the default type of the medium.
For example, when determining the medium type corresponding to the touch object according to the elastic wave signal, the information of the time domain and the frequency domain of the elastic wave signal (the information includes the time point exceeding the set signal-to-noise ratio threshold in the time domain) may be combined first to obtain the absolute value and the relative value of the energy at the characteristic frequency point in the small-band time-frequency analysis, and the vibration starting point of the elastic wave signal may be determined according to the absolute value and the relative value, and then the material identification may be performed according to the signal after the vibration starting point. The method for judging the vibration starting point can be a threshold method (judging whether the signal-to-noise ratio or the signal quantity exceeds a preset threshold), a characteristic matching method (judging whether the correlation coefficient of the signal characteristic of the elastic wave signal and the set touch characteristic exceeds the set correlation threshold) or a neural network characteristic point detection method (realizing the segmentation of the elastic wave signal at the vibration starting point through a trained characteristic point detection model), or can be performed through a zero crossing rate (judging whether the signal is a non-random vibration signal).
When the elastic wave detection device detects the elastic wave signal by using a plurality of elastic wave sensors, the elastic wave detection device can judge or calculate to obtain an effective elastic wave signal (a signal for medium identification) according to touch position information and/or touch parameter information of touch data, so as to judge the starting point of the effective elastic wave signal. For example, an elastic wave sensor nearest to a touch point may be determined according to touch position information in touch data, and the elastic wave signal generated by the elastic wave sensor is used as an effective elastic wave signal because the signal strength generated by the elastic wave sensor is highest and the signal to noise ratio of the generated signal is highest. In addition, the elastic wave signals of the elastic wave sensors may be weighted according to the touch position information of the touch data, and the result obtained by the weighting is taken as an effective elastic wave signal, and the weight corresponding to each elastic wave sensor may be a preset value or a function inversely related to the distance of the touch position, such as the weight being the inverse of the distance or the square of the inverse.
After the vibration starting point is determined, analyzing to obtain a signal before vibration starting, taking the signal before vibration starting as a noise estimation value, identifying the signal after the vibration starting point according to the noise estimation value, and taking the signal after the vibration starting point as an identification signal for identifying the elastic wave material. In addition, the vibration starting point may be determined by a time corresponding to when the touch area reacted in the touch data reaches the set touch area threshold value, that is, the second elastic wave signal may be used as an identification signal for identifying the elastic wave material. As an implementation mode, signal processing is carried out on the identification signal, the signal processing comprises time-frequency conversion, windowing smoothing, multichannel fusion, noise reduction and other algorithms, a fused elastic wave frequency domain signal is obtained, the elastic wave frequency domain signal is put into a set algorithm (including model algorithms such as statistical learning, machine learning and deep learning or a characteristic value comparison algorithm), an ID of a material is obtained, and medium type information carrying the material ID is reported to processing equipment.
When the scheme is specifically implemented, the characteristic recognition model can be utilized to determine the medium type, namely, the recognition signal is input into the trained characteristic recognition model, and the medium type corresponding to the touch object is obtained. The characteristic recognition model can determine the touch position information and the medium type corresponding to the elastic wave signal in a neural network mode. For example, a feature recognition model may be built based on a neural network and trained based on elastic wave signals (or recognition signals) as inputs and corresponding media types as outputs.
In one embodiment, the number of the elastic wave signals is multiple, and when the elastic wave detection device determines the type of the medium corresponding to the touch object according to the elastic wave signals, the elastic wave detection device may read touch position information in the touch data, and perform fusion processing on the multiple elastic wave signals to obtain the preprocessed signal. And inputting the touch position information and the preprocessing signals into the trained feature recognition model to obtain the medium type corresponding to the touch object. One embodiment of the fusion process is as follows: based on the preset position information of the plurality of elastic wave sensors, the distances S1, S2 and … Sn (n is a natural number) between the touch points represented by the touch position information and different elastic wave sensors are calculated by combining the touch position information, and then fusion processing is carried out on elastic wave signals acquired by the plurality of elastic wave sensors based on the calculated distances, so that a preprocessing result is obtained. For example, one possible implementation is to weight the signal of each sensor by 1/(Sn)/(2), and then take the weighted result as the preprocessing result.
When the scheme is specifically implemented, the characteristic database can be used for determining the medium type, and the corresponding elastic wave detection device can also be used for extracting the characteristics of the elastic wave signal and comparing the extracted characteristics with the preset characteristic database when determining the medium type corresponding to the touch object according to the elastic wave signal, so as to obtain the medium type corresponding to the touch object. The elastic wave detection device is provided with a characteristic database in advance, and the characteristic database is recorded with a corresponding relation between preset characteristics and preset medium types. The corresponding relation between the preset features and the preset media types can be recorded in a key value pair mode.
S107: the elastic wave detection device sends medium type information to the processing device, wherein the medium type information comprises a medium type corresponding to the touch object.
The medium type information refers to information, which is sent to the processing device by the elastic wave detection device and contains a medium type corresponding to the touch object, and may be information which is generated independently and only contains the medium type, or may be other information which is used for data interaction between the elastic wave detection device and the processing device, and the medium type is integrated in the other information and is sent to the processing device.
After determining the medium type corresponding to the touch object, the elastic wave detection device generates corresponding medium type information based on the medium type and sends the medium type information to the processing equipment. According to the method and the device, the medium type corresponding to the touch object is determined through the time information of the touch data and the elastic wave signal, the corresponding medium type information is fed back to the processing equipment, when a plurality of touch objects exist, the medium types corresponding to different touch positions can be accurately judged, different responses can be carried out on the touch operation of the corresponding touch positions according to the different medium types, and the interaction experience of the interaction panel is enriched.
In one possible embodiment, the writing material identification method based on elastic waves provided by the present embodiment further includes:
s108: after receiving the medium type information, the processing device starts a response to the received touch data.
For example, when the processing device receives touch data sent by the touch sensitive sensor, a determination is needed as to when to respond to the received touch data. The response means, in the embodiment of the present invention, whether the processing device processes the touch data as valid data. In the embodiment of the present invention, a condition is pre-set by the processing device to process the continuously received touch data as valid data, namely: the processing device confirms that the touch object triggering the touch data is in contact with the display screen of the interactive tablet. In an exemplary embodiment of the present invention, an elastic wave detection device is disposed in the interactive flat panel, and a vibration signal generated on a display screen of the interactive flat panel is detected by the elastic wave detection device, and communication hardware and a communication protocol of the elastic wave detection device and a processing device of the interactive flat panel are disposed, so that the elastic wave detection device can send information (for example, the foregoing medium type information) for confirming that a touch object triggering touch data is in contact with the display screen of the interactive flat panel to the processing device, and help the processing device obtain the judging capability.
In one embodiment, when the processing device receives the touch data (for example, the infrared touch data sent by the infrared touch frame) sent by the touch sensor, but does not receive the information sent by the elastic wave detection device and used for confirming that the touch object triggering the touch data is in contact with the display screen of the interaction panel, the processing device may not respond to the received touch data.
The processing device determines a medium type corresponding to the touch object according to the medium type information after receiving the medium type information returned by the elastic wave device, and processes and responds subsequently received touch data according to the determined medium type. The processing and responding to the subsequently received touch data may be updating the subsequently received touch data and responding accordingly (for example, displaying a writing track) according to the media type, or may be performing touch responding to the subsequently received touch data according to the media type (for example, performing a clicking operation, a long-press operation, etc. according to the media type). The processing equipment processes and responds to the subsequently received touch data after receiving the medium type, ignores the touch data generated before the touch object contacts the display screen, only processes the touch data generated after the touch object contacts the display screen, and keeps consistent time for processing and responding the touch data with the position and time of the touch object contacting the display screen, so that the touch operation of a user can be fed back more accurately, for example, the position and time for displaying a writing track are consistent with the position and time of the touch object contacting the display screen, and zero height of writing is realized.
In one possible embodiment, step S108 is specifically: after receiving the medium type information, the processing device starts a response to the received touch data and outputs a response result associated with the medium type of the touch object.
Wherein the response to the touch data and the output of the associated corresponding result may be based on the media type of the touch object. In one embodiment, in response to the received touch data, a response result associated with the medium type of the touching object is output, which may be elastic wave data generated in response to a touch operation performed by a pen head of the writing pen, the medium type is determined to be the pen head, a writing track is determined according to the touch data, and a corresponding writing track is displayed on the display screen. Illustratively, after receiving the media type information, the processing device initiates updating of the received touch data according to the media type. Wherein, the starting of updating the received touch data means updating the touch data received after the moment of receiving the medium type information.
Further, the processing device displays a corresponding writing track on the display screen according to the updated touch data. After receiving the medium type information, the processing equipment starts to buffer the touch data sent by the received touch sensor; and sequentially reading the cached touch data, updating the received touch data according to the type of the medium, and displaying the corresponding writing track on the display screen according to the updated touch data. Illustratively, the processing device begins to buffer touch data sent by the touch sensor after receiving the media type reported by the elastic wave detection device. Optionally, the processing device notifies the elastic wave detection device that the medium type has been successfully received after determining that the medium type reported by the elastic wave detection device is received. The processing device starts to buffer the touch data after receiving the medium type, ignores the touch data generated before the touch object contacts the display screen (when the touch data are generated, the touch object only enters the touch detection area and does not contact the display screen), only processes the touch data generated after the touch object contacts the display screen, and the position and the time of displaying the writing track are consistent with the position and the time of the touch object contacting the display screen, so that the zero height of writing is realized. And the touch data generated before the touch object contacts the display screen is not cached, so that the processing resource of the processing equipment is saved. In the updating process of the step, the processing equipment sequentially reads the cached touch data according to the first-in first-out sequence, and updates the read touch data according to the medium type. In this step, the corresponding writing track is drawn according to the updated touch data, and displayed on the display screen, for example, the corresponding writing track is drawn according to the display attribute (thickness, color, line type, etc.) represented by the updated infrared touch data, and displayed on the display screen, so as to achieve the purpose of displaying different writing tracks according to the touch objects of different media types.
In one embodiment, in response to the received touch data, a response result associated with the media type of the touch object is output, or the media type is determined to be the pen tail in response to elastic wave data generated by a touch operation performed by using the pen tail of the writing pen, and the writing track displayed on the display screen is correspondingly erased according to the touch data. When the pen tail of the writing pen with the medium type is identified according to the elastic wave data, determining that the corresponding position needs to be erased in the current touch operation, determining a first moving range corresponding to the touch data, and erasing writing tracks corresponding to the first moving range and displayed on a display screen.
Specifically, the pen head, the pen body and the pen tail of the writing pen are respectively made of materials corresponding to different medium types. Based on different medium types of different parts, the operation of different positions of one writing pen can directly complete corresponding touch operation under certain conditions, and the control operation needing to be triggered can be hit only by operating the functional control for a plurality of times in a touch detection area. The method is suitable for the operation habit of the user during paper writing, in the scheme, the medium types of the pen point and the pen tail are correspondingly used as touch input operation, the medium type of the pen body is correspondingly used as touch control operation, and for example, the horizontal sliding of the pen body of the writing pen is used as a triggering mode of a page turning instruction. More specifically, the type of medium of the pen point corresponds to a writing operation of synchronous input of a writing track, and the pen tail is an operation (such as erasing) of correcting the existing writing track. Of course, the pen point and the pen tail can also be correspondingly touch control operations through touch control operations of a specific track or touch control operations of a specific area. Based on comprehensive judgment of the medium type and the movement track, richer touch control functions can be realized through simpler interaction. For example, currently, a user performs a touch operation on a display screen by using a writing pen, and at this time, according to the infrared touch data and the elastic wave data, the current touch operation can be confirmed as follows: the medium type is the writing operation of the writing pen, so that a writing track is drawn according to the touch data, and the display attribute of the writing track is determined according to the corresponding display attribute of the current touch medium. Similarly, if the pen tail of the writing pen slides in the display screen, the current touch operation can be confirmed according to the touch data and the elastic wave data, and the current touch operation is as follows: and (3) an erasing operation of the existing writing track, responding to the operation, displaying the eraser mark at the contact position in the erasing process, and erasing the writing track displayed in the area where the eraser mark passes.
In conjunction with the schematic diagram of the elastic wave signal in the writing process provided in fig. 7, it should be explained that in the touch operation process of writing, the elastic wave generated by touching the display screen by the touch object can be divided into two processes, namely, the elastic wave generated at the beginning of the touch event of touching the display screen by the touch object (S0 in the drawing) and the elastic wave generated at the subsequent writing process (S1 in the drawing), it can be understood that the touch process of touching the touch object and the display screen is only performed once each time, i.e. only one elastic wave S0 is generated in one touch operation, and a plurality of elastic waves S1 are generated in the subsequent writing sliding process. Since the elastic wave S0 has a relatively high collision energy, and accordingly, the signal-to-noise and signal quality of the corresponding elastic wave signal are relatively high, and the type of the touch object material is more easily identified, when the material identification is performed, the elastic wave signal corresponding to the elastic wave S0 is preferentially used as the material type identification signal (the second elastic wave signal), and the elastic wave signal corresponding to the elastic wave S0 can be obtained based on the elastic wave signal in the set time range after the second time. The elastic wave S1 has low vibration energy generated by sliding, low signal-to-noise ratio, and is easy to be interfered by external vibration (such as sound, vibration of a panel casing, etc.), if the elastic wave signal corresponding to the elastic wave S1 is utilized to perform material identification, the situation of false identification is easy to occur, the accuracy of material identification in the process of writing sliding is low, the subsequent writing sliding process or continuous triggering of the elastic wave is performed, if each elastic wave signal is performed with material identification, the response delay of writing is increased, the material identification effect is also poor, and the scheme performs material identification by performing material identification on the elastic wave signal with high signal-to-noise ratio acquired at the beginning of a touch event, and the subsequent writing sliding process performs writing response based on the previously identified material, so that the interaction efficiency and interaction quality of the interactive panel are improved.
In one embodiment, when receiving the media type information corresponding to the default media type, the processing device may perform media correction on the media type information according to the touch data, and then initiate a response to the received touch data according to the corrected media type information. For example, the processing device records infrared data features (such as touch area and position range) corresponding to different media types, and compares the received infrared touch data with the infrared data features to find a matched infrared data feature, or the elastic wave detection device or the processing device inputs the elastic wave signal and the infrared touch data into a trained feature recognition model together to obtain the media type matched with the touch object. And taking the medium type corresponding to the matched infrared data characteristics as the medium type according to which the infrared touch data is responded. If no matching infrared data characteristics are found, continuing to initiate a response to the received infrared touch data based on the default media type. By classifying the signal quality of the elastic wave signals, the non-elastic wave information such as infrared area, position and the like can be increased for the elastic wave signals with low quality, the information quantity input by the recognition model can be enhanced, for example, the touch position information of touch data and the elastic wave signals can be input into the trained characteristic recognition model for the elastic wave signals with low quality, and the characteristic recognition model analyzes and processes the touch position information and the elastic wave signals to determine the medium type corresponding to the touch object. The characteristic recognition model can determine the touch position information and the medium type corresponding to the elastic wave signal in a neural network mode. For example, a feature recognition model can be built based on a neural network, and training can be performed based on elastic wave signals and touch position information as inputs and corresponding media types as outputs. In one embodiment, the touch parameter information, the touch position information and the elastic wave signal may also be input into a trained feature recognition model to obtain a medium type corresponding to the touch object. The touch position information and/or touch parameter information is used as a feature to be added into the feature recognition model for training, the feature can be the most original input feature of the feature recognition model, or can be an additional feature of a certain layer in a network of the feature recognition model, and the feature recognition model directly outputs a recognition result. By using the touch data as one identification parameter of the characteristic identification model of the medium type, the information quantity input by the identification model is enhanced, the material identification performance is improved and solved, the problem of false identification caused by the fact that a single elastic wave signal is easy to be interfered due to low signal-to-noise ratio of a low-quality signal is solved, and the identification accuracy of the medium type is improved.
In one possible embodiment, after the processing device receives the medium type information and initiates a response to the received touch data, the method for identifying writing materials based on elastic waves provided in the present embodiment further includes: after the processing equipment receives the medium type information, if the touch time difference between the received touch data and the touch data generated by the last touch operation is within a set time threshold value and the touch distance is within a set distance threshold value, stopping sending the touch data to the elastic wave detection device, and starting response to the received touch data according to the medium type information corresponding to the last touch operation.
Wherein the touch data is sent by the processing device to the elastic wave detection means. After the processing device starts a response to the touch data according to the medium type information, if the touch data generated by a new touch operation is continuously received, the processing device judges a touch time difference and a touch distance of the touch data relative to the touch data generated by a previous touch operation (last frame touch data, for example, last frame infrared touch data generated by an infrared touch frame), if the touch time difference is within a set time threshold and the touch distance is within the set distance threshold, the two touch operations are considered to be triggered continuously in a short time, the medium type information corresponding to the current touch operation is consistent with the medium type information determined in the previous touch operation, the medium type does not need to be determined again by sending the touch data to the elastic wave detection device, and the response to the received touch data can be started according to the medium type information corresponding to the previous touch operation. For example, the touch position of the current infrared touch data is within 1cm relative to the touch distance between the touch positions of the last frame of infrared touch data corresponding to the last touch operation, and the touch time of the current infrared touch data is within 150ms relative to the touch time of the last frame of infrared touch data corresponding to the last touch operation, at this time, the two touch operations are considered to be triggered continuously in a short time, and the time interval of 150ms is less than the time consumption of switching the writing material in the normal writing behavior, so that the touch material is considered to be unchanged, and the response of the received infrared touch data can be started by continuously using the last medium type information.
Fig. 8 is a logic diagram for processing touch data, as shown in fig. 8, where, in the example where the processing device sends first frame touch data to the elastic wave detection device, T is a time span of touch operation, and time T1 is a time when the infrared touch frame starts to detect shielding of a touch object, and a time from time T1 to time T2 is a time when the touch object approaches to the display screen. After the infrared touch block detects the infrared touch data at the time T1, the infrared touch data are sent to the processing equipment, and meanwhile, the infrared touch frame or the processing equipment sends the infrared touch data to the elastic wave detection device, and the elastic wave detection device starts to detect and buffer elastic wave signals.
In the figure, TOUCH represents a time point when a TOUCH object starts to perform TOUCH operation but does not enter a TOUCH detection area, DOWN is a time point when an infrared TOUCH frame reports infrared TOUCH data after detecting the infrared TOUCH data after the TOUCH object starts to enter the TOUCH detection area, the reported infrared TOUCH data is first frame TOUCH data, meanwhile, the infrared TOUCH frame reports the TOUCH data to a processing device at a time T2, and the TOUCH data is sent to an elastic wave detection device or sent to the elastic wave detection device by the processing device. In the figure, the MOVE (i.e. time T2 to time T5) is the movement process of the touch operation on the display screen, and the infrared touch frame will continuously send infrared touch data to the processing device. In actual processing, a plurality of frames of time intervals exist between the infrared touch frame detecting the infrared touch data and reporting the infrared touch data, the infrared touch frame does not report the first frame of the infrared touch data immediately (noise points such as dust are avoided) after calculation, but determines that real touch operation exists after more than two frames of infrared touch data are continuously calculated, and reports the infrared touch data. The first frame of infrared touch data reported to the processing device is referred to as first frame of infrared touch data, that is, the first frame of infrared touch data may be second frame, third frame or fourth frame of infrared touch data obtained by calculating an infrared touch frame, and specifically, which frame is reported is determined by the working characteristics of the infrared touch frame. The infrared touch data sent by the infrared touch frame or the processing device to the elastic wave detection device can be continuous infrared touch data, or can be corresponding infrared touch data when the first frame of infrared touch data and the touch area reach a set touch area threshold value.
The infrared touch frame or the processing device sends infrared touch data corresponding to the touch area reaching the set touch area threshold value to the elastic wave detection device at the time T3. After receiving the first frame of infrared touch data or the corresponding infrared touch data when the touch area reaches a set touch area threshold value, the elastic wave detection device determines a first elastic wave signal and a second elastic wave signal from elastic wave signals according to the infrared touch data, performs signal quality assessment according to the first elastic wave signal and the second elastic wave signal, determines a medium type corresponding to the touch object according to the elastic wave signal and a quality assessment result (from time T3 to time T4 in the figure), and uploads the medium type to processing equipment (from time T4 in the figure). After receiving the medium type, the processing device may feed back to the elastic wave detection device to confirm that the medium type is received. Meanwhile, the elastic wave detection device pauses operation after transmitting the medium type to the processing device.
When the elastic wave detection device performs signal quality evaluation according to the first elastic wave signal and the second elastic wave signal and determines the medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result, the elastic wave detection device may determine the first time t1 according to time information corresponding to the detected effective infrared touch data and determine the second time t2 according to time information corresponding to the touch area reaching the set touch area threshold. And determining the elastic wave signals acquired from the first time t1 to the second time t2 as first elastic wave signals of the touch object before the touch object contacts the display screen of the interactive panel, and determining the elastic wave signals after the second time t2 (set time length) as second elastic wave signals of the touch object after the touch object contacts the display screen of the interactive panel. The signal-to-noise ratio level (signal quality evaluation) of the detected elastic wave signal is estimated according to the first elastic wave signal between the first time t1 and the second time t2, and the elastic wave signal is divided into a low-quality elastic wave signal or a high-quality elastic wave signal, wherein the low-quality elastic wave signal is an elastic wave signal with a signal-to-noise ratio lower than a signal-to-noise ratio threshold and incapable of being subjected to material identification through the elastic wave, and the high-quality elastic wave signal is an elastic wave signal with a signal-to-noise ratio higher than the signal-to-noise ratio threshold and capable of being subjected to material identification through the elastic wave. Optionally, when the elastic wave signal is a low-quality elastic wave signal, a default medium type may be returned to the processing device, and when the elastic wave signal is a high-quality elastic wave signal, the elastic wave signal may be noise-reduced, and then a characteristic recognition model is used to perform material recognition to determine the medium type of the touch object, and the corresponding medium type is returned to the processing device. The feature recognition model can utilize time domain and frequency domain information of the elastic wave signals to judge materials in a machine learning mode, a deep learning mode or a feature matching mode.
After receiving the media type, the processing device starts to buffer the infrared touch data sent by the received infrared touch frame (infrared touch data after time T4 in the figure). Meanwhile, the processing device sequentially reads the cached infrared touch data, updates the infrared touch data according to the medium type, and displays a corresponding writing track (from the moment T4 in the figure) on the display screen according to the updated infrared touch data. Finally, after receiving the last frame touch data reported by the infrared touch frame (at time T5 in the figure), the processing device sends the last frame touch data to the elastic wave detection device, and the elastic wave detection device clears the buffered elastic wave signal in response to the last frame touch data and continues to work.
Fig. 9 is a schematic operation diagram of a touch operation provided by the present disclosure, and fig. 10 is a schematic display diagram of a writing track provided by the present disclosure. As shown in fig. 9, it is assumed that a stylus is used as a touch object to perform a touch operation on a display screen, and the arrow direction in fig. 9 is the moving direction of the stylus. In the writing operation, a pen falling event, a writing event and a pen lifting event can occur in sequence, and the touch of the corresponding writing pen on the display screen falls into a state, a touch moving state and a touch lifting state. In fig. 10, a track a is an operation track of the writing pen, and the operation track is divided into A1, A2 and A3 according to the movement direction of the writing pen, and corresponds to a touch falling state (corresponding to a DOWN point in fig. 8), a touch movement state (corresponding to a DOWN point to an UP point in fig. 8) and a touch lifting state (corresponding to an UP point in fig. 8) of the writing pen on the display screen, respectively. In the process of pen-down, the writing pen does not contact the display screen, the elastic wave detection device cannot detect touch operation of the writing pen on the display screen, and when the writing pen enters a writing stage, the writing pen contacts the display screen, and the elastic wave detection device detects an elastic wave signal. In the whole process of pen-down, movement and pen-up, the infrared touch frame can detect infrared touch data. The elastic wave detection device caches the elastic wave signal in the A1 stage, evaluates the signal quality of the elastic wave signal according to the time information of the infrared touch data sent by the infrared touch frame or the processing equipment, determines the medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result, reports the medium type to the processing equipment, and stops working after reporting the medium type. The processing device displays a writing track to the display screen according to the medium type and the infrared touch data in the A2 stage. And the elastic wave detection device clears the buffered elastic wave signal and continues to work after receiving the touch data of the last frame in the A3 stage. The writing trace finally displayed on the display screen is shown as trace B in fig. 10, wherein trace B is divided into B2 and B3, corresponding to A2 and A3, respectively.
According to the method, the touch data generated when the touch object touches the touch detection area of the interactive panel is detected through the touch sensor, the elastic wave signal generated when the touch object touches the display screen is detected through the elastic wave detection device, the quality evaluation is carried out on the elastic wave signal according to the time information in the touch data, the medium type is determined according to the quality evaluation result and the elastic wave signal, the recognition accuracy of the touch object is improved by combining the elastic wave signal and the quality evaluation result to determine the medium type, and the medium type information containing the medium type of the touch object is sent to the processing equipment, so that the processing equipment can execute different response operations according to different medium types of the touch object, and the method is simple to operate and high in efficiency. Meanwhile, the writing track is displayed when the touch object is determined to be in contact with the display screen, rather than displaying the writing track before the touch object is actually in contact with the display screen (although the touch object enters the touch detection area and touch data are generated at the moment), the display time of the writing track is closer to the time of the touch object in contact with the display screen, the display of the writing track is more accurate, the zero writing height of writing operation is realized, the display attribute of the writing track can be rapidly determined through the medium type, the modification of the display attribute can be rapidly realized through the touch object of different medium types, the differentiated display of writing tracks of different display attributes is realized, the operation process is concise and efficient, and the interaction efficiency of the interaction panel is improved.
Fig. 11 shows a block diagram of an interactive tablet provided by the present disclosure. Referring to fig. 11, the interactive tablet may be used to perform the elastic wave-based writing material recognition method in the embodiment of fig. 2, and includes a display screen, a processing device, a touch-sensitive sensor, and an elastic wave detection device, where the display screen, the touch-sensitive sensor, and the elastic wave detection device are all connected to the processing device;
the touch sensor is used for detecting touch data, and the touch data is generated when a touch object performs touch operation in a touch detection area of the interaction panel; the touch data includes touch time information;
the elastic wave detection device is used for detecting an elastic wave signal, wherein the elastic wave signal is generated when a touch object contacts with the display screen of the interaction panel; and
after receiving touch data sent by a touch sensor or processing equipment, evaluating signal quality according to elastic wave signals of different time periods; determining a medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result; and
transmitting medium type information to the processing equipment, wherein the medium type information comprises a medium type corresponding to the touch object;
and the processing device is used for receiving the touch data sent by the touch-sensitive sensor.
In one embodiment, after receiving the media type information, the processing device initiates a response to the received touch data.
According to the method, the touch data generated when the touch object touches the touch detection area of the interactive panel is detected through the touch sensor, the elastic wave signal generated when the touch object touches the display screen is detected through the elastic wave detection device, the quality evaluation is carried out on the elastic wave signal according to the time information in the touch data, the medium type is determined according to the quality evaluation result and the elastic wave signal, the recognition accuracy of the touch object is improved by combining the elastic wave signal and the quality evaluation result to determine the medium type, and the medium type information containing the medium type of the touch object is sent to the processing equipment, so that the processing equipment can execute different response operations according to different medium types of the touch object, and the method is simple to operate and high in efficiency. The specific technical details and technical effects of the interactive tablet provided in this embodiment may refer to the foregoing embodiments (for example, the method for identifying writing materials based on elastic waves in the embodiment of fig. 2), which are not repeated in this disclosure.
The present disclosure also provides a readable storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform elastic wave based writing material identification as provided by the above embodiments.
Storage media-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, lanbas (Rambus) RAM, etc.; nonvolatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a second, different computer system connected to the first computer system through a network such as the internet. The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer systems connected by a network). The storage medium may store program instructions (e.g., embodied as a computer program) executable by one or more processors.
Of course, the disclosure provides a readable storage medium containing computer-executable instructions, where the computer-executable instructions are not limited to the above-mentioned elastic wave-based writing material identification, but may also perform related operations in the elastic wave-based writing material identification provided by any embodiment of the disclosure.
The interactive tablet and the storage medium provided in the above embodiments may perform the elastic wave-based writing material identification provided in any embodiment of the present disclosure, and technical details not described in detail in the above embodiments may be referred to the elastic wave-based writing material identification provided in any embodiment of the present disclosure.
The foregoing description is only of the preferred embodiments of the present disclosure and the technical principles employed. The present disclosure is not limited to the specific embodiments described herein, but is capable of numerous obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the disclosure. Thus, while the present disclosure has been described in greater detail in the foregoing embodiments, the present disclosure is not limited to the above embodiments, but may include many other equivalent embodiments without departing from the spirit of the present disclosure, the scope of which is determined by the scope of the claims.
Claims (10)
1. The writing material identification method based on the elastic wave is applied to an interaction panel, and is characterized in that the interaction panel comprises a display screen, processing equipment and a touch sensor, the interaction panel further comprises an elastic wave detection device, and the method further comprises the following steps:
the touch data are generated when a touch object performs touch operation in a touch detection area of the interaction panel; the touch data includes touch time information;
the processing device receives the touch data sent by the touch sensitive sensor;
the elastic wave detection device receives the touch data, wherein the touch data is sent to the elastic wave detection device by the touch sensor or the processing equipment;
the elastic wave detection device detects an elastic wave signal; the elastic wave signal is generated when the touch object contacts the display screen of the interaction panel;
the elastic wave detection device carries out signal quality evaluation according to the elastic wave signals in different time periods;
the elastic wave detection device determines the medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result;
The elastic wave detection device sends medium type information to the processing equipment, wherein the medium type information comprises a medium type corresponding to the touch object.
2. The method of elastic wave-based writing material recognition according to claim 1, further comprising:
and after receiving the medium type information, the processing equipment starts a response to the received touch data.
3. The method for identifying writing materials based on elastic waves according to claim 2, wherein after receiving the medium type information, the processing device starts a response to the received touch data, which specifically comprises:
and after receiving the medium type information, the processing equipment starts a response to the received touch data and outputs a response result associated with the medium type of the touch object.
4. The method for identifying writing materials based on elastic waves according to claim 1, wherein after the elastic wave detection device detects the elastic wave signals, the elastic wave signals are buffered until the touch data is received, and then signal quality evaluation is performed according to the elastic wave signals in different time periods.
5. The elastic wave-based writing material recognition method according to claim 1, wherein the touch data is transmitted to the elastic wave detection device by the processing device, the method further comprising:
after the processing equipment receives the medium type information, if the touch time difference between the received touch data and the touch data generated by the last touch operation is within a set time threshold value and the touch distance is within a set distance threshold value, stopping sending the touch data to the elastic wave detection device, and starting response to the received touch data according to the medium type information corresponding to the last touch operation.
6. The method for identifying writing materials based on elastic waves according to claim 1, wherein the elastic wave detection device performs signal quality evaluation according to the elastic wave signals of different time periods, specifically:
the elastic wave detection device determines a first elastic wave signal of the elastic wave signal before the touch object contacts the display screen of the interaction panel and a second elastic wave signal of the elastic wave signal after the touch object contacts the display screen of the interaction panel based on the time information of the touch data;
The elastic wave detection device evaluates signal quality of the elastic wave signals according to the first elastic wave signal and the second elastic wave signal.
7. The method for identifying writing materials based on elastic waves according to claim 6, wherein the elastic wave detection device performs signal quality evaluation on the elastic wave signals according to the first elastic wave signal and the second elastic wave signal, specifically:
the elastic wave detection device extracts first characteristic information of the first elastic wave signal and second characteristic information of the second elastic wave signal;
the elastic wave detection device takes the ratio of the difference between the second characteristic information and the first characteristic information to the first characteristic information as the quality evaluation result of the elastic wave signal.
8. The method for identifying writing materials based on elastic waves according to claim 1, wherein the elastic wave detection device determines a medium type corresponding to the touching object according to the elastic wave signal and the quality evaluation result, specifically:
when the quality evaluation result of the elastic wave signal reaches a set threshold value, the elastic wave detection device determines the medium type corresponding to the touch object according to the elastic wave signal;
And when the quality evaluation result of the elastic wave signal is lower than a set threshold value, the elastic wave detection device determines that the medium type corresponding to the touch object is a default medium type.
9. The interactive flat panel is characterized by comprising a display screen, processing equipment, a touch-sensitive sensor and an elastic wave detection device, wherein the display screen, the touch-sensitive sensor and the elastic wave detection device are all connected with the processing equipment;
the touch sensor is used for detecting touch data, and the touch data are generated when a touch object performs touch operation in a touch detection area of the interaction panel; the touch data includes touch time information;
the elastic wave detection device is used for detecting an elastic wave signal, wherein the elastic wave signal is generated when the touch object contacts the display screen of the interaction panel; and
after receiving the touch data sent by the touch sensor or the processing device, performing signal quality evaluation according to the elastic wave signals of different time periods; determining a medium type corresponding to the touch object according to the elastic wave signal and the quality evaluation result; and
transmitting medium type information to the processing equipment, wherein the medium type information comprises a medium type corresponding to the touch object;
The processing device is used for receiving the touch data sent by the touch-sensitive sensor.
10. The interactive tablet of claim 9, wherein the processing device, upon receiving the media type information, initiates a response to the received touch data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210266653.9A CN116795231A (en) | 2022-03-16 | 2022-03-16 | Writing material identification method based on elastic wave and interaction panel |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210266653.9A CN116795231A (en) | 2022-03-16 | 2022-03-16 | Writing material identification method based on elastic wave and interaction panel |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116795231A true CN116795231A (en) | 2023-09-22 |
Family
ID=88048570
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210266653.9A Pending CN116795231A (en) | 2022-03-16 | 2022-03-16 | Writing material identification method based on elastic wave and interaction panel |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116795231A (en) |
-
2022
- 2022-03-16 CN CN202210266653.9A patent/CN116795231A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102144208B (en) | Multi-touch touchscreen incorporating pen tracking | |
| US8619063B2 (en) | Method for determining a touch event and touch sensitive device | |
| TWI581171B (en) | Method and device for identifying multipoint gestures | |
| CN111208917B (en) | Touch data processing method and device, intelligent equipment and storage medium | |
| CN114690853B (en) | Interaction method and interaction panel | |
| WO2018018624A1 (en) | Gesture input method for wearable device, and wearable device | |
| CN113791700A (en) | Elastic wave-based interaction method and device, interaction panel and storage medium | |
| US20230057020A1 (en) | Meeting interaction system | |
| CN114690930A (en) | Handwriting processing method and device, interactive panel and storage medium | |
| Wen et al. | UbiTouch: ubiquitous smartphone touchpads using built-in proximity and ambient light sensors | |
| JP5486977B2 (en) | Coordinate input device and program | |
| CN113608661A (en) | Man-machine interaction method, interaction panel and storage medium thereof | |
| CN106527828B (en) | Notebook hand-written recognition method and its hand-written input system | |
| CN116795231A (en) | Writing material identification method based on elastic wave and interaction panel | |
| WO2022222384A1 (en) | Touch data processing method and interaction tablet | |
| CN113608662A (en) | Touch response method and device, terminal equipment and storage medium | |
| CN110989844A (en) | Input method, watch, system and storage medium based on ultrasonic waves | |
| WO2023039797A1 (en) | Method and apparatus for identifying touch operation | |
| WO2022222982A1 (en) | Touch-control signal verification method, man-machine interaction method and handwriting display method, and related apparatuses | |
| CN114690931A (en) | False touch prevention method and device based on contact tracking, interactive flat plate and storage medium | |
| WO2023016430A1 (en) | Writing effect processing method and apparatus, device, and storage medium | |
| CN114690977B (en) | Interactive calling method and device based on elastic waves | |
| WO2023231731A1 (en) | Writing mode switching method and device, interactive tablet, and storage medium | |
| CN113805725B (en) | Display control method and device for interactive panel | |
| CN114690933A (en) | Virtual pen application control method and device, interactive panel and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |