CN115291726B - Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device - Google Patents
Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device Download PDFInfo
- Publication number
- CN115291726B CN115291726B CN202210952607.4A CN202210952607A CN115291726B CN 115291726 B CN115291726 B CN 115291726B CN 202210952607 A CN202210952607 A CN 202210952607A CN 115291726 B CN115291726 B CN 115291726B
- Authority
- CN
- China
- Prior art keywords
- data
- haptic
- tactile
- media
- simulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 98
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000008569 process Effects 0.000 claims description 16
- 230000035807 sensation Effects 0.000 claims description 9
- 238000012986 modification Methods 0.000 claims description 8
- 230000004048 modification Effects 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims 3
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 11
- 230000008447 perception Effects 0.000 description 9
- 238000013480 data collection Methods 0.000 description 7
- 238000000605 extraction Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000001149 cognitive effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
Abstract
The present disclosure provides a haptic simulation method, haptic simulation apparatus, haptic simulation system, storage medium, and electronic device, and relates to the field of computer technology. The haptic simulation method includes: acquiring first tactile data and media data corresponding to the first tactile data; correcting the first tactile data according to the media data to obtain second tactile data; haptic simulation is performed based on the second haptic data. The present disclosure may improve the accuracy of haptic simulation.
Description
Technical Field
The present disclosure relates to the field of computer technology, and in particular, to a haptic simulation method, a haptic simulation apparatus, a haptic simulation system, a storage medium, and an electronic device.
Background
With the rapid development of digital signal processing technology and communication technology, the metauniverse concept represented by virtual reality technology is brought into the field of view of the public, and the way of mutual blending of virtual world and real world is one direction of technological development and research in the future. From audio-visual interaction to multi-perception interaction, how to realize the immersion of the multimedia interaction experience becomes an important research content.
Haptic sensation is an important perception mode, and signal reconstruction is one of core technologies for realizing deep immersion and interactive experience. However, the haptic simulation scheme generally has the problem of poor accuracy of haptic feedback, which affects user experience.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure is directed to a haptic simulation method, a haptic simulation apparatus, a haptic simulation system, a storage medium, and an electronic device, and thus overcomes, at least to some extent, the problem of poor haptic feedback accuracy in haptic simulation schemes.
According to a first aspect of the present disclosure, there is provided a haptic simulation method, comprising: acquiring first tactile data and media data corresponding to the first tactile data; correcting the first tactile data according to the media data to obtain second tactile data; haptic simulation is performed based on the second haptic data.
Optionally, acquiring the first haptic data and the media data corresponding to the first haptic data includes: acquiring first tactile data and determining generation time of the first tactile data; and acquiring the media data acquired at the generation time.
Optionally, modifying the first haptic data according to the media data to obtain the second haptic data includes: extracting semantic tag information of media data; and ordering the data units in the first tactile data by using the semantic tag information and/or performing data compensation on the first tactile data by using the semantic tag information to obtain second tactile data.
Optionally, modifying the first haptic data according to the media data to obtain the second haptic data includes: extracting media characteristics of the media data; extracting haptic features of the first haptic data; the media features are fused with the haptic features to determine second haptic data.
Optionally, performing haptic simulation based on the second haptic data includes: filtering out data with the haptic sensation degree smaller than a degree threshold value from the second haptic data to obtain third haptic data; haptic simulation is performed using the third haptic data.
Optionally, performing haptic simulation using the third haptic data includes: determining a haptic module corresponding to the third haptic data; the haptic module is controlled to perform haptic feedback.
According to a second aspect of the present disclosure, there is provided a haptic simulation apparatus comprising: the data acquisition module is used for acquiring the first tactile data and the media data corresponding to the first tactile data; the data correction module is used for correcting the first tactile data according to the media data so as to obtain second tactile data; and the simulation module is used for carrying out haptic simulation based on the second haptic data.
Alternatively, the data acquisition module may be configured to perform: acquiring first tactile data and determining generation time of the first tactile data; and acquiring the media data acquired at the generation time.
Alternatively, the data modification module may be configured to perform: extracting semantic tag information of media data; and ordering the data units in the first tactile data by using the semantic tag information and/or performing data compensation on the first tactile data by using the semantic tag information to obtain second tactile data.
Alternatively, the data modification module may be configured to perform: extracting media characteristics of the media data; extracting haptic features of the first haptic data; the media features are fused with the haptic features to determine second haptic data.
Alternatively, the simulation module may be configured to perform: filtering out data with the haptic sensation degree smaller than a degree threshold value from the second haptic data to obtain third haptic data; haptic simulation is performed using the third haptic data.
Alternatively, the process of haptic simulation by the simulation module using the third haptic data may be configured to perform: determining a haptic module corresponding to the third haptic data; the haptic module is controlled to perform haptic feedback.
According to a third aspect of the present disclosure, there is provided a haptic simulation system comprising: the data acquisition end is used for acquiring the first tactile data and the media data corresponding to the first tactile data; the haptic simulation end is used for acquiring the first haptic data and the media data corresponding to the first haptic data sent by the data acquisition end, correcting the first haptic data according to the media data to obtain second haptic data, and performing haptic simulation based on the second haptic data.
According to a fourth aspect of the present disclosure, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements any of the haptic simulation methods described above.
According to a fifth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory for storing executable instructions of the processor; the processor is configured to implement any of the haptic simulation methods described above via execution of the executable instructions.
In some embodiments of the present disclosure, the first haptic data is modified by using the media data to obtain second haptic data, and then haptic simulation is performed based on the second haptic data. According to the method and the device for correcting the first tactile data, the first tactile data is corrected by using the media data, the problem of possible information errors of the first tactile data can be eliminated or at least reduced, the tactile simulation is carried out on the basis, the accuracy of tactile feedback can be improved, and the user experience is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:
FIG. 1 schematically illustrates a block diagram of a haptic simulation system according to an exemplary embodiment of the present disclosure;
FIG. 2 illustrates a schematic diagram of an implementation of a haptic simulation scheme in accordance with an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart of a haptic simulation method according to an exemplary embodiment of the present disclosure;
FIG. 4 illustrates a schematic diagram of determining first haptic data and corresponding media data according to an embodiment of the present disclosure;
FIG. 5 illustrates a schematic diagram of a process of determining haptic data in accordance with an embodiment of the present disclosure;
FIG. 6 schematically illustrates a flow chart of a haptic simulation process of an embodiment of the present disclosure;
FIG. 7 schematically illustrates a block diagram of a haptic simulation apparatus according to an exemplary embodiment of the present disclosure;
fig. 8 schematically illustrates a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. One skilled in the relevant art will recognize, however, that the aspects of the disclosure may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only and not necessarily all steps are included. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations. In addition, all of the following terms "first," "second," "third," etc. are used for distinguishing purposes only and should not be taken as a limitation of the present disclosure.
The problems of timing disorder, information loss and the like of the haptic data in the transmission process can occur, so that the problem of poor haptic feedback accuracy during haptic simulation is caused, and the user experience is influenced.
In view of this, the present disclosure provides a scheme of combining media data with haptic data to modify the haptic data to thereby improve haptic feedback accuracy.
FIG. 1 schematically illustrates a block diagram of a haptic simulation system according to an exemplary embodiment of the present disclosure. Referring to FIG. 1, haptic simulation system 1 may include a data acquisition end 11 and a haptic simulation end 12.
In some embodiments of the present disclosure, the data acquisition end 11 and the haptic emulation end 12 may be different electronic devices. Taking a game as an example, the data collection end 11 and the haptic simulation end 12 may be game devices of different players equipped with the same game application.
In other embodiments of the present disclosure, the data acquisition end 11 and the haptic emulation end 12 may correspond to the same electronic device, that is, the data acquisition end 11 and the haptic emulation end 12 may be different processing units in the same electronic device. Still taking a game as an example, the data collection end 11 and the haptic emulation end 12 may be configured in the same gaming device, such as a joystick.
Specifically, the data collection terminal 11 may be configured to collect the first tactile data and media data corresponding to the first tactile data. The haptic simulation end 12 may be used to obtain the first haptic data transmitted from the data collection end 11 and the media data corresponding to the first haptic data, correct the first haptic data according to the media data to obtain the second haptic data, and perform haptic simulation based on the second haptic data.
In addition, the process of haptic simulation by the haptic simulation end 12 based on the second haptic data may be configured to perform: and filtering out the data with the tactile sensation degree smaller than a degree threshold value from the second tactile data to obtain third tactile data. Then, haptic simulation is performed using the third haptic data.
FIG. 2 illustrates a schematic diagram of an implementation of a haptic simulation scheme in accordance with an embodiment of the present disclosure.
The data acquisition end can comprise a touch data acquisition module, a media data acquisition module, a coding unit and a data transmission module.
Wherein the haptic data collection module may be used to collect haptic data, in embodiments of the present disclosure, the haptic data collection module is used to collect first haptic data, e.g., collect various force related data such as friction, vibration, pressure, etc.; the media data collection module may be used to collect media data, which in embodiments of the present disclosure is used to collect media data such as audio data, video data, image data, etc.; the coding unit is used for coding the acquired data, wherein the coded object can be the acquired data or the data obtained by deep learning feature extraction operation of the acquired data, and the data is compressed and then transmitted; the data transmitting module is used for transmitting the encoded data.
The haptic simulation end may include a data receiving module, a decoding unit, a data compensation module, a perceptual feature extraction module, a fusion simulation module, a control module, and a controlled module.
The data receiving module can be used for receiving the coded data transmitted by the communication network and sent by the data acquisition end; the decoding unit may be used for decoding the encoded data; the data compensation module can be used for aligning the first tactile data with the media data and taking the media data as a compensation basis of the first tactile data; the perception feature extraction module can be used for removing signals which are trapped into a user perception blind area due to smaller stimulus in a short time by utilizing the Weber-Fisher cognitive deviation law, so as to realize feature extraction of user perception interval data; the fusion simulation module can be used for fusing the received data and simulating the tactile characteristics; the control module can be used for responding to the simulation touch characteristics to send a control instruction to the controlled module, and the controlled module executes simulation operation of the touch feedback.
In addition, the processing procedures of the data compensation module, the perception feature extraction module and the fusion simulation module can be realized by means of a neural network. For the neural network, there are an input layer, a hidden layer, and an output layer, the number of neurons of the input layer and the output layer being correspondingly selected by the dimensions of the tactile data input and output. The present disclosure does not limit the specific processing of the neural network structure, training, and reasoning.
The haptic simulation method of the embodiment of the present disclosure, which will be described below, may be performed by an electronic device in which a corresponding haptic simulation apparatus may be configured. The electronic device may include, for example, a smart phone, a gaming device (e.g., a gamepad), etc.
Fig. 3 schematically illustrates a flow chart of a haptic simulation method of an exemplary embodiment of the present disclosure. Referring to fig. 3, the haptic simulation method may include the steps of:
s32, acquiring the first tactile data and media data corresponding to the first tactile data.
In an exemplary embodiment of the present disclosure, the haptic data may be data generated in response to a haptic operation of a user, and may also be referred to as contact force data, or the like. In particular, the haptic data may include frictional force data, pressure data, vibration data, etc., sensed by the electronic device due to an active haptic operation of the user.
For example, the force data may be sensed by a pressure sensor provided by the electronic device. For another example, vibration data may be sensed by inertial sensors (e.g., gyroscopes) provided with the electronic device.
The media data may include one or more of audio data, video data, image data, text data. For example, the image data may be acquired by a camera equipped with the electronic device. For another example, the audio data may be acquired by a microphone provided with the electronic device.
Additionally, the first haptic data and/or media data may also be collected by other devices and sent to an electronic device that performs the haptic simulation method of the present disclosure.
The electronic device may obtain first haptic data and media data corresponding to the first haptic data.
Referring to fig. 4, first, the electronic device may acquire first haptic data and determine a generation time of the first haptic data, which is generally a time of the electronic device in response to a user's haptic operation.
Next, the electronic device may acquire the media data collected at the generation time.
For example, in the context of VR (Virtual Reality) games, on the one hand, VR devices are equipped with tactile sensors that can collect tactile data; on the other hand, VR devices are equipped with cameras that can capture game visuals on a display screen. The VR device can also align the haptic data with the game play surface via a timestamp.
S34, correcting the first tactile data according to the media data to obtain second tactile data.
The media data collected by the method has the characteristic of semantic consistency, such as consistency of audio and video on the semantic level. The electronic device may modify the first haptic data based on the media data that is consistent in time of acquisition.
According to some embodiments of the present disclosure, the electronic device may extract semantic tag information of the media data, and sort data units in the first haptic data using the semantic tag information, that is, adjust an order of the data units in the first haptic data, so as to solve a problem of timing disorder that may exist in the first haptic data.
According to further embodiments of the present disclosure, the electronic device may further utilize the semantic tag information to data compensate for the first haptic data. Specifically, first, the missing data in the first tactile data may be determined using the semantic tag information, where the missing data may be the data lost due to the acquisition process or the transmission process. The missing data may then be reconstructed to complement the first haptic data. For example, data reconstruction may be implemented by means of data interpolation or model prediction.
In addition, it is understood that the electronic device may also perform the sorting of the data in the first haptic data and the data compensation operation for the first haptic data together.
In other embodiments of the present disclosure, the modification of the first haptic data may also be accomplished using a deep learning process.
In one aspect, an electronic device may extract media characteristics of media data; in another aspect, the electronic device can extract haptic features of the first haptic data.
Next, the electronic device can fuse the media data with the haptic data to effect correction of the first haptic data.
The deep learning network can be used for extracting and fusing the features, and the network structure and the training process of the deep learning network are not limited.
In an exemplary embodiment of the present disclosure, the corrected data of the first haptic data is recorded as the second haptic data.
S36, performing haptic simulation based on the second haptic data.
Haptic simulation relies on haptic modules equipped on electronic devices in a manner that includes one or more of temperature feedback, vibration feedback, pressure feedback. Wherein the temperature feedback includes high temperature feedback and low temperature feedback; the vibration feedback includes feedback of different vibration frequencies; the pressure feedback includes feedback of different pressure levels and/or successive pressures with pressure trigger intervals.
The electronic device may be equipped with different haptic modules for generating the above-described feedback, from which haptic simulation may be performed after determining the corresponding haptic data.
According to some embodiments of the present disclosure, the electronic device may determine a corresponding haptic module according to the second haptic data and control the haptic module to perform haptic feedback.
According to other embodiments of the present disclosure, the electronic device may filter out data having a haptic sensation level less than a level threshold from the second haptic data to obtain third haptic data, and perform haptic simulation using the third haptic data. Specifically, the electronic device may determine a haptic module corresponding to the third haptic data and control the haptic module to perform haptic feedback.
In these embodiments, the weber-fishena law of cognitive deviation may be utilized to filter out data with a haptic sensation degree smaller than a degree threshold, i.e., remove signals trapped in human perception dead zones due to smaller stimuli, and implement feature extraction of human perception interval signals. The elimination of the data of the perception dead zone improves the accuracy of the representation of the touch data on one hand; on the other hand, the data transmission amount and the calculation amount are reduced, and the system performance is optimized.
Fig. 5 shows a schematic diagram of a process of determining haptic data in accordance with an embodiment of the present disclosure.
Referring to fig. 5, the electronic device may sense first tactile data by way of tactile sensing; correcting the first tactile data by using the media data acquired corresponding to the first tactile data, and determining the corrected result as second tactile data; the second haptic data is filtered in combination with a user perception threshold (i.e., the degree threshold described above) determined by weber-fisher's law of cognitive deviation, and the filtered result is determined as the third haptic data.
FIG. 6 schematically illustrates a flow chart of a haptic simulation process of an embodiment of the present disclosure.
In step S602, the electronic device may acquire first haptic data and media data corresponding to the first haptic data.
In step S604, the electronic device may correct the first haptic data with the media data to obtain the second haptic data.
In step S606, the electronic device may filter out data with a haptic sensation level less than the level threshold from the second haptic data to obtain third haptic data.
In step S608, the electronic device may determine a haptic module using the third haptic data and control the haptic module for haptic feedback.
It should be noted that although the steps of the methods in the present disclosure are depicted in the accompanying drawings in a particular order, this does not require or imply that the steps must be performed in that particular order, or that all illustrated steps be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.
Further, in this example embodiment, a haptic simulation apparatus is also provided.
Fig. 7 schematically illustrates a block diagram of a haptic simulation apparatus of an exemplary embodiment of the present disclosure. Referring to fig. 7, a haptic simulation apparatus 7 according to an exemplary embodiment of the present disclosure may include a data acquisition module 71, a data correction module 73, and a simulation module 75.
Specifically, the data obtaining module 71 may be configured to obtain the first haptic data and the media data corresponding to the first haptic data; the data correction module 73 may be configured to correct the first haptic data according to the media data to obtain the second haptic data; the simulation module 75 may be used to perform haptic simulation based on the second haptic data.
According to an example embodiment of the present disclosure, the data acquisition module 71 may be configured to perform: acquiring first tactile data and determining generation time of the first tactile data; and acquiring the media data acquired at the generation time.
According to an example embodiment of the present disclosure, the data modification module 73 may be configured to perform: extracting semantic tag information of media data; and ordering the data units in the first tactile data by using the semantic tag information and/or performing data compensation on the first tactile data by using the semantic tag information to obtain second tactile data.
According to an example embodiment of the present disclosure, the data modification module 73 may be configured to perform: extracting media characteristics of the media data; extracting haptic features of the first haptic data; the media features are fused with the haptic features to determine second haptic data.
According to an example embodiment of the present disclosure, the simulation module 75 may be configured to perform: filtering out data with the haptic sensation degree smaller than a degree threshold value from the second haptic data to obtain third haptic data; haptic simulation is performed using the third haptic data.
According to an example embodiment of the present disclosure, the process of the simulation module 75 performing haptic simulation using the third haptic data may be configured to perform: determining a haptic module corresponding to the third haptic data; the haptic module is controlled to perform haptic feedback.
Since each functional module of the haptic simulation apparatus according to the embodiment of the present disclosure is the same as that of the above-described method embodiment, a detailed description thereof will be omitted.
In an exemplary embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, when the program product is run on the terminal device.
The program product for implementing the above-described method according to the embodiments of the present disclosure may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical disk, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.
An electronic device 800 according to such an embodiment of the present disclosure is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is merely an example and should not be construed to limit the functionality and scope of use of embodiments of the present disclosure in any way.
As shown in fig. 8, the electronic device 800 is embodied in the form of a general purpose computing device. Components of electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting the different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.
Wherein the storage unit stores program code that is executable by the processing unit 810 such that the processing unit 810 performs steps according to various exemplary embodiments of the present disclosure described in the above section of the present specification. For example, the processing unit 810 may perform the various steps of the haptic simulation method of an embodiment of the present disclosure.
The storage unit 820 may include readable media in the form of volatile storage units, such as Random Access Memory (RAM) 8201 and/or cache memory 8202, and may further include Read Only Memory (ROM) 8203.
Storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 830 may be one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 800, and/or any device (e.g., router, modem, etc.) that enables the electronic device 800 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 850. Also, electronic device 800 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 over bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 800, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.
Furthermore, the above-described figures are only schematic illustrations of processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (7)
1. A haptic simulation method, comprising:
acquiring first tactile data, determining generation time of the first tactile data, and acquiring media data acquired at the generation time;
correcting the first tactile data according to the media data to obtain second tactile data;
performing haptic simulation based on the second haptic data;
wherein modifying the first haptic data according to the media data to obtain second haptic data comprises:
extracting semantic tag information of the media data, and sequencing data units in the first tactile data by utilizing the semantic tag information and/or performing data compensation on the first tactile data by utilizing the semantic tag information so as to obtain second tactile data; or alternatively
Extracting media characteristics of the media data, extracting tactile characteristics of the first tactile data, and fusing the media characteristics with the tactile characteristics to determine second tactile data.
2. A haptic simulation method as recited in claim 1 wherein performing a haptic simulation based on said second haptic data includes:
filtering out data with the haptic sensation degree smaller than a degree threshold value from the second haptic data to obtain third haptic data;
and carrying out haptic simulation by using the third haptic data.
3. The haptic simulation method of claim 2, wherein performing haptic simulation using the third haptic data comprises:
determining a haptic module corresponding to the third haptic data;
and controlling the haptic module to perform haptic feedback.
4. A haptic simulation device, comprising:
the data acquisition module is used for acquiring first tactile data, determining the generation time of the first tactile data and acquiring media data acquired at the generation time;
the data correction module is used for correcting the first tactile data according to the media data so as to obtain second tactile data;
the simulation module is used for carrying out haptic simulation based on the second haptic data;
wherein the data modification module is configured to perform: extracting semantic tag information of the media data, and sequencing data units in the first tactile data by utilizing the semantic tag information and/or performing data compensation on the first tactile data by utilizing the semantic tag information so as to obtain second tactile data; or alternatively
The data modification module is configured to perform: extracting media characteristics of the media data, extracting tactile characteristics of the first tactile data, and fusing the media characteristics with the tactile characteristics to determine second tactile data.
5. A haptic simulation system, comprising:
the data acquisition end is used for acquiring first tactile data, determining the generation time of the first tactile data and acquiring media data acquired at the generation time;
the haptic simulation end is used for acquiring the first haptic data sent by the data acquisition end and the media data corresponding to the first haptic data, correcting the first haptic data according to the media data to obtain second haptic data, and performing haptic simulation based on the second haptic data;
wherein the process of modifying the first haptic data to obtain second haptic data by the haptic emulation end in accordance with the media data is configured to perform: extracting semantic tag information of the media data, and sequencing data units in the first tactile data by utilizing the semantic tag information and/or performing data compensation on the first tactile data by utilizing the semantic tag information so as to obtain second tactile data; or alternatively
The haptic emulation terminal is configured to perform a process of modifying the first haptic data to obtain second haptic data according to the media data: extracting media characteristics of the media data, extracting tactile characteristics of the first tactile data, and fusing the media characteristics with the tactile characteristics to determine second tactile data.
6. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the haptic simulation method of any of claims 1 to 3.
7. An electronic device, comprising:
a processor; and
a memory for storing executable instructions of the processor;
wherein the processor is configured to implement the haptic simulation method of any one of claims 1 to 3 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210952607.4A CN115291726B (en) | 2022-08-09 | 2022-08-09 | Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210952607.4A CN115291726B (en) | 2022-08-09 | 2022-08-09 | Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115291726A CN115291726A (en) | 2022-11-04 |
| CN115291726B true CN115291726B (en) | 2023-11-07 |
Family
ID=83828939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210952607.4A Active CN115291726B (en) | 2022-08-09 | 2022-08-09 | Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115291726B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107106906A (en) * | 2014-12-23 | 2017-08-29 | 意美森公司 | The tactile of media-driven |
| CN111919381A (en) * | 2018-04-04 | 2020-11-10 | 思睿逻辑国际半导体有限公司 | Method and apparatus for outputting haptic signals to a haptic transducer |
| CN113205070A (en) * | 2021-05-27 | 2021-08-03 | 三一专用汽车有限责任公司 | Visual perception algorithm optimization method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9656163B2 (en) * | 2012-06-29 | 2017-05-23 | Sony Interactive Entertainment Inc. | Haptic enhancements for emulated video game not originally designed with haptic capabilities |
| US9274603B2 (en) * | 2013-05-24 | 2016-03-01 | Immersion Corporation | Method and apparatus to provide haptic feedback based on media content and one or more external parameters |
| TWI557596B (en) * | 2013-08-19 | 2016-11-11 | 瑞昱半導體股份有限公司 | Audio device and audioutilization method having haptic compensation function |
-
2022
- 2022-08-09 CN CN202210952607.4A patent/CN115291726B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107106906A (en) * | 2014-12-23 | 2017-08-29 | 意美森公司 | The tactile of media-driven |
| CN111919381A (en) * | 2018-04-04 | 2020-11-10 | 思睿逻辑国际半导体有限公司 | Method and apparatus for outputting haptic signals to a haptic transducer |
| CN113205070A (en) * | 2021-05-27 | 2021-08-03 | 三一专用汽车有限责任公司 | Visual perception algorithm optimization method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115291726A (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10143924B2 (en) | Enhancing user experience by presenting past application usage | |
| KR102225802B1 (en) | Method and program for making reactive video | |
| KR102488530B1 (en) | Method and apparatus for generating video | |
| CN112135160A (en) | Virtual object control method and device in live broadcast, storage medium and electronic equipment | |
| CN113946211A (en) | Method for interacting multiple objects based on metauniverse and related equipment | |
| CN110174942B (en) | Eye movement synthesis method and device | |
| CN104007826A (en) | Video control method and system based on face movement identification technology | |
| CN111862280A (en) | Virtual role control method, system, medium, and electronic device | |
| KR101023249B1 (en) | Apparatus and method for generating application program of cognitive training using brainwaves, and recording medium thereof | |
| CN111760272B (en) | Game information display method and device, computer storage medium and electronic equipment | |
| CN105975072A (en) | Method, device and system for identifying gesture movement | |
| CN111481923B (en) | Rocker display method and device, computer storage medium and electronic equipment | |
| CN115103175A (en) | Image transmission method, device, equipment and medium | |
| CN115291726B (en) | Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device | |
| CN112734631A (en) | Video image face changing method, device, equipment and medium based on fine adjustment model | |
| CN115578494B (en) | Method, device and equipment for generating intermediate frame and storage medium | |
| CN115223244B (en) | Haptic motion simulation method, device, apparatus and storage medium | |
| CN112016077A (en) | Page information acquisition method and device based on sliding track simulation and electronic equipment | |
| CN116309969A (en) | Method and device for generating scenario animation in game, storage medium and electronic equipment | |
| CN110604918B (en) | Interface element adjustment method and device, storage medium and electronic equipment | |
| CN112449249A (en) | Video stream processing method and device, electronic equipment and storage medium | |
| CN114339197A (en) | Video playing test method, device and equipment | |
| CN115758107B (en) | Haptic signal transmission method and device, storage medium and electronic equipment | |
| CN112911373A (en) | Method, device and equipment for generating video subtitles and storage medium | |
| CN112752146A (en) | Video quality evaluation method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |