CN109343716B - Image display method and device and computer readable storage medium - Google Patents
Image display method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN109343716B CN109343716B CN201811369581.0A CN201811369581A CN109343716B CN 109343716 B CN109343716 B CN 109343716B CN 201811369581 A CN201811369581 A CN 201811369581A CN 109343716 B CN109343716 B CN 109343716B
- Authority
- CN
- China
- Prior art keywords
- image
- displayed
- touch
- display screen
- tactile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
Abstract
The embodiment of the invention discloses an image display method, which is applied to electronic equipment with a touch display screen and comprises the following steps: acquiring an image to be displayed and depth information of each pixel region in the image to be displayed; determining a haptic simulation parameter corresponding to the image to be displayed at least based on the depth information of each pixel region in the image to be displayed; the touch simulation parameters are used for indicating the surface texture characteristics of the target object corresponding to the image to be displayed; and displaying the image to be displayed in an image display screen, and controlling the touch display screen to perform touch output on the image to be displayed based on the touch simulation parameter corresponding to each pixel region in the image to be displayed. The embodiment of the invention also discloses an image display device and a computer readable storage medium.
Description
Technical Field
The present invention relates to the field of electronic devices, and in particular, to an image display method and apparatus, and a computer-readable storage medium.
Background
With the rapid popularization of intelligent terminals such as smart phones and tablet computers, the intelligent terminals have become the most extensive human-computer interaction media, and human-computer interaction through a touch screen has become the most common interaction mode at present.
At present, when images are browsed through a touch screen, two-dimensional images are generally browsed, and the interaction mode is more rigid and rigid; in the related art, three-dimensional modeling of an image may be implemented by an Augmented Reality (AR) technology, so that a user can visually acquire a stereoscopic model of the image. However, in general, humans can acquire real information of objects by combining various sense organs; therefore, the prior art method of obtaining information about an object represented by an image by vision only is very one-sided.
The sense of touch, one of the important senses connecting human consciousness and the external objective world, plays a crucial role in the process of identifying objects; therefore, how to combine three-dimensional stereoscopic vision with human tactile sensation to provide a more realistic experience for a user when browsing images is a problem to be solved urgently.
Disclosure of Invention
In order to solve the above technical problem, embodiments of the present invention provide an image display method and apparatus, and a computer-readable storage medium.
The embodiment of the invention provides an image display method, which is applied to electronic equipment with a touch display screen and comprises the following steps:
acquiring an image to be displayed and depth information of each pixel region in the image to be displayed;
determining a haptic simulation parameter corresponding to the image to be displayed at least based on the depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed;
and displaying the image to be displayed in an image display screen, and controlling the touch display screen to perform touch output on the image to be displayed based on the touch simulation parameter corresponding to each pixel region in the image to be displayed.
An embodiment of the present invention further provides an image display device, including:
the device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring an image to be displayed and depth information of each pixel area in the image to be displayed;
the determining unit is used for determining the corresponding tactile simulation parameters of the image to be displayed at least based on the depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed;
and the processing unit is used for displaying the image to be displayed in the image display screen, outputting the touch simulation parameters corresponding to each pixel region in the image to be displayed on the basis of the display screen, and controlling the touch display screen to enable a user to sense the surface texture characteristics of the target object corresponding to the image to be displayed through touching the display screen by fingers to perform touch output.
There is provided an image display device, the device including: a processor and a memory configured to store a computer program capable of running on the processor,
wherein the processor is configured to execute the steps of the image display method when running the computer program.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of any one of the above methods.
According to the image display method and device and the computer readable storage medium provided by the embodiment of the invention, firstly, an image to be displayed and depth information of each pixel area in the image to be displayed are obtained; determining a touch simulation parameter corresponding to the image to be displayed based on the depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed; displaying the image to be displayed in an image display screen, and controlling a touch display screen to perform touch output on the image to be displayed based on touch simulation parameters corresponding to each pixel region in the image to be displayed; therefore, through the scheme, the image to be displayed can be displayed in the image display screen, the depth information of the image to be displayed can be converted into the tactile simulation parameters, and the image to be displayed is output in the tactile display screen in a tactile manner, so that a user can sense the surface texture characteristics of the shot target object by touching the tactile display screen through fingers; therefore, the touch information can be added on the basis of the existing visual reproduction, the visual experience is combined with the human touch experience, and the more real interactive experience is provided for the user.
Drawings
Fig. 1 is a schematic flowchart of an image display method according to an embodiment of the present invention;
fig. 2 a is a first schematic structural diagram of an electronic device according to an embodiment of the present invention;
fig. 2 b is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating another image display method according to an embodiment of the present invention;
fig. 4 is a schematic view of an operation interface of an electronic device according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating another image displaying method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an image display device according to an embodiment of the present invention;
fig. 7 is a schematic diagram illustrating a hardware structure of an image display device according to an embodiment of the present invention.
Detailed Description
So that the manner in which the features and elements of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings.
Fig. 1 is a schematic flow chart of an image display method according to an embodiment of the present invention, and as shown in fig. 1, the image display method includes the following steps:
In other embodiments of the present invention, a storage device of an electronic device may store a plurality of images to be displayed and depth information corresponding to each pixel region in each image to be displayed; therefore, when the user needs to preview the image, the electronic device can directly select the image to be displayed and the depth information of the image to be displayed from the storage device, and display the image and the depth information on the image display screen and the touch display screen of the electronic device. The depth information refers to distance information between the surface of the target object and the shooting device when the target object is shot; the depth information of each pixel region in the image to be displayed refers to a distance between each part of the surface of the target object and the shooting device, and it can be understood that each pixel region in the image to be displayed represents a different part of the surface of the target object.
In other embodiments of the present invention, the electronic device may further acquire image information and depth information of the target object through the depth camera, and acquire the image to be displayed and the depth information of each pixel region in the image to be displayed.
Specifically, the depth camera comprises an image acquisition device and a depth information acquisition device; the electronic equipment acquires an image of a target object through an image acquisition device to obtain an image to be displayed; the image acquisition device can acquire two-dimensional image information of a target object to obtain position information of each pixel point in an image to be displayed in a two-dimensional space coordinate; the image capturing device is preferably a Red-Green-Blue (RGB) camera. The depth information acquisition device is preferably a Time-of-Flight (TOF) camera; the depth information acquisition device can emit light pulses to a target object, receive the light pulses and receive reflected light of the light pulses reflected by the target object; and acquiring the transmission time from the emission of the light pulse to the reception of the reflected light, and determining the depth information corresponding to each pixel region in the image to be displayed based on the transmission time. It is understood that the depth information acquiring means can acquire the spatial information of the third dimension of the target object.
And 102, determining a touch simulation parameter corresponding to the image to be displayed at least based on the depth information of each pixel region in the image to be displayed.
The tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed.
In other embodiments of the present invention, the step 102 of determining the haptic simulation parameters corresponding to the image to be displayed based on at least the depth information of each pixel region in the image to be displayed may be implemented by an electronic device.
In other embodiments of the present invention, the electronic device may give a certain electrical stimulus to the contour and texture of the target object in the image to be displayed in the tactile display screen through an electrical stimulus haptic rendering principle, so that the user generates different haptic feelings when touching. In practical applications, a transparent touch display screen may be added to a display screen of an electronic device for outputting an electrical stimulation signal, where a hardware frame of the electronic device includes a processor, a camera module (TOF camera, RGB camera, driver chip, power supply chip, etc.), an electrical stimulation module (touch display screen), a processor (driver chip, power supply chip, etc.), and an image display module (image display screen, driver chip, power supply chip, etc.). As for the electronic device 20 shown in fig. 2, a in fig. 2 is a front surface of the electronic device 20, and a tactile display screen 22 is disposed above an image display screen 21 of the electronic device 20; shown in b of fig. 2 is the back of the electronic device 20, on which a TOF camera 22 and a normal RGB camera 23 are arranged.
In this embodiment, different electrical stimulation can be realized through the tactile simulation parameters, that is, by outputting the tactile simulation parameters, the fingers of the user are stimulated differently when contacting the tactile display screen, so that the user feels the rugged feeling. It is understood that the haptic simulation parameters are capable of characterizing surface texture features of the target object corresponding to the image to be displayed.
In particular, at least one haptic electrode may be provided in a haptic display screen of the electronic device for outputting haptic simulation parameters, different haptic simulation parameters being capable of generating different electrical stimulation signals. In order to be able to reproduce the outline and texture features of the image to be displayed more realistically, the tactile electrodes in the electronic device are preferably distributed in a dot matrix throughout the tactile display screen; wherein, the more tactile electrodes are arranged in the tactile display screen, the smaller the pixel area represented by the tactile electrodes is, and the more accurate the tactile representation is. Preferably, one tactile electrode may be disposed at a position corresponding to each pixel in the tactile display screen; therefore, when the image display screen displays the image, the tactile electrode of the tactile display screen at the position corresponding to the pixel can output the corresponding tactile simulation parameter according to the depth information of the corresponding pixel area in the image to be displayed.
In the above solution, the haptic simulation parameter includes at least one of the following parameters:
the current of each tactile electrode;
a voltage of each tactile electrode;
the pulse frequency of each haptic click.
In other embodiments of the present invention, before step 102, the electronic device may further preset a correspondence table between depth information values of the pixels and the haptic simulation parameters output by the haptic electrodes; it is understood that the relationship between the value of the depth information of the pixel region in the image to be displayed and the value of the current, voltage or pulse frequency output by the electrode is set in advance.
Illustratively, the correspondence table between the depth information value of the pixel and the tactile simulation parameter output by the tactile electrode is shown in table 1, when the depth information value of the pixel region is between 0 m and 1m, the output current of the tactile electrode may be 10A, the output current may be 32V, and the output pulse frequency may be 50 kHz.
TABLE 1
In general, the reproduction of the sense of touch can be achieved by current stimulation or charge coulomb force stimulation. Specifically, the current stimulation is to arrange a tactile electrode on the tactile display screen at a position in contact with a finger; when the touch simulation is carried out, the electronic equipment can apply different currents or pulse frequencies to the touch electrodes, when the fingers of a user touch the touch electrodes, the currents can stimulate the nerves of the fingers through the fingers, so that nerve fiber action potentials are triggered and conducted to the sensory nerve center of the brain, the user generates contact consciousness, and therefore touch reproduction is achieved. In this embodiment, the depth information value of each pixel region in the image to be displayed corresponds to a certain current magnitude or pulse frequency.
In addition, the touch screen can be stimulated by charge coulomb force, an insulating medium layer can be added on the outermost side of the touch screen, and the touch electrode is arranged on the insulating medium layer and close to one side of the touch screen. When a finger of a user touches the outermost surface of the insulating medium layer, a voltage difference exists between the bottom electrode and the finger, charges are respectively accumulated, and the finger nerve can be subjected to friction force feeling according to the interaction principle of coulomb force. In this embodiment, the voltage or pulse frequency of the tactile electrode can be controlled to cause different electric charges to be accumulated, so as to generate different coulomb forces, thereby enabling the operator to generate different feelings. Wherein the magnitude of the voltage output by the tactile electrode or the magnitude of the pulse frequency corresponds to the depth information of each pixel region in the image to be displayed.
Step 103 is to display the image to be displayed in the image display screen, and control the tactile display screen to perform tactile output on the image to be displayed based on the tactile simulation parameter corresponding to each pixel region in the image to be displayed, which may be implemented by an electronic device.
Here, the tactile electrode in the tactile display screen has a corresponding relationship with each pixel region in an image displayed by the tactile electrode; the electronic equipment outputs all the tactile simulation parameters to the tactile electrodes in the tactile display screen while the image display screen displays the image to be displayed; in this way, the user can not only visually see the image to be displayed, but also feel the outline and texture characteristics of the image to be displayed through the stimulation effect of the tactile electrode when the user touches the image to be displayed with a finger.
The image display method provided by the embodiment of the invention comprises the steps of firstly, acquiring an image to be displayed and depth information of each pixel area in the image to be displayed; determining a touch simulation parameter corresponding to the image to be displayed at least based on the depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed; displaying the image to be displayed in an image display screen, and controlling a touch display screen to perform touch output on the image to be displayed based on touch simulation parameters corresponding to each pixel region in the image to be displayed; therefore, by the scheme, the depth information of the image to be displayed can be converted into the tactile simulation parameters, so that a user can sense the surface texture characteristics of the shot target object by touching the display screen with fingers; therefore, the touch information can be added on the basis of the existing visual reproduction, and more real interactive experience is provided for the user.
In the above embodiment, the electronic device determines the haptic simulation parameters of the image to be displayed only according to the depth information of the pixel region, and when the TOF camera photographs the same object at different distances from the object, the obtained depth information is different, which may cause the same object to have different haptic feelings. Therefore, in view of the above problems, an embodiment of the present invention provides an image display method, as shown in fig. 3, including the steps of:
The image to be displayed can be understood as a set of plane two-dimensional points, and the depth information of each pixel region in the image to be displayed can be understood as a set of third-dimensional space points; and the electronic equipment constructs and obtains a three-dimensional space data model of the target object in the image to be displayed according to the two-dimensional space information of the target object represented by the image to be displayed and the third-dimensional space information of the target object represented by the depth information of each pixel region in the image to be displayed. Therefore, the electronic equipment can obtain the spatial relative position between any two pixel points in the image to be displayed.
In the above scheme, the electronic device can obtain the spatial relative position relationship between each pixel point in the image to be displayed by constructing the three-dimensional space model. The touch simulation parameters corresponding to the image to be displayed are determined through the obtained three-dimensional space model, and the influence of different distances between an object and a camera on the touch simulation parameters during shooting can be eliminated.
In other embodiments of the present invention, the electronic device may determine the corresponding haptic simulation parameter according to the relative position relationship of the pixel points in the three-dimensional space.
And step 304, the electronic equipment displays the image to be displayed in an image display screen, and controls a touch display screen to perform touch output on the image to be displayed based on the touch simulation parameters corresponding to each pixel region in the image to be displayed.
In other embodiments of the present invention, as shown in fig. 4, the electronic device may display a three-dimensional space model of an image to be displayed in the image display screen, so that a user can visually see a stereoscopic image. Meanwhile, the electronic equipment outputs the touch simulation parameters to the touch electrodes of the touch display screen, so that a user can feel the surface texture characteristics of the target object corresponding to the image to be displayed when touching.
It should be noted that, for the explanation of the same or related steps in this embodiment as in other embodiments, reference may be made to the description in other embodiments, and details are not described here again.
The image display method provided by the embodiment of the invention comprises the steps of firstly, acquiring an image to be displayed and depth information of each pixel area in the image to be displayed; determining a touch simulation parameter corresponding to the image to be displayed at least based on the depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed; displaying the image to be displayed in an image display screen, and controlling a touch display screen to perform touch output on the image to be displayed based on touch simulation parameters corresponding to each pixel region in the image to be displayed; therefore, through the scheme, the image to be displayed can be displayed in the image display screen, the depth information of the image to be displayed can be converted into the tactile simulation parameters, and the image to be displayed is output in the tactile display screen in a tactile manner, so that a user can sense the surface texture characteristics of the shot target object by touching the tactile display screen through fingers; therefore, the touch information can be added on the basis of the existing visual reproduction, the three-dimensional stereoscopic vision feeling and the human touch feeling are combined, and the more real interactive experience is provided for the user.
Based on the foregoing embodiments, an embodiment of the present invention provides an image display method, as shown in fig. 5, the method including the following steps:
The tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed.
As another exemplary embodiment, in order to reduce the power consumption loss caused by the tactile electrodes when operating as much as possible, especially when the number of tactile electrodes in the tactile display screen is numerous, the position of the user touch operation may be determined by acquiring the user touch operation; based on the touch location, haptic simulation parameters for the haptic electrodes are output to reduce the number and time of active haptic electrodes.
In other embodiments of the present invention, the electronic device may obtain a touch position when the user touches according to a pressure sensor or a touch screen in the tactile display screen; wherein the touch position can be represented by two-dimensional coordinates. It should be noted that the touch position may include a plurality of two-dimensional coordinates, that is, the touch position is not a single coordinate point but a continuous area.
And 505, the electronic equipment determines a touch area of the image to be displayed corresponding to the touch position based on the touch position.
In other embodiments of the present invention, the electronic device obtains the coordinates corresponding to the image to be displayed according to the obtained coordinates of the touch position, and determines that the user touches the corresponding area of the image to be displayed.
And step 507, the electronic device controls the touch display screen to perform touch output on the image to be displayed at the touch position based on the touch simulation parameter of each pixel area in the touch area.
Specifically, the electronic device starts the touch electrode in the touch area only when the user touches the touch display screen, and the touch electrode outside the touch area is in the off state, so that power consumption loss possibly caused by the touch electrode in operation can be reduced, and the power supply of the electronic device is saved.
It should be noted that, for the explanation of the same or related steps in this embodiment as in other embodiments, reference may be made to the description in other embodiments, and details are not described here again.
The image display method provided by the embodiment of the invention comprises the steps of firstly, acquiring an image to be displayed and depth information of each pixel area in the image to be displayed; determining a touch simulation parameter corresponding to the image to be displayed at least based on the depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed; displaying the image to be displayed in an image display screen, and controlling the touch display screen to perform touch output on the image to be displayed based on the touch simulation parameter corresponding to each pixel region in the image to be displayed; therefore, through the scheme, the image to be displayed can be displayed in the image display screen, the depth information of the image to be displayed can be converted into the tactile simulation parameters, and the image to be displayed is output in the tactile display screen in a tactile manner, so that a user can sense the surface texture characteristics of the shot target object by touching the tactile display screen through fingers; therefore, the touch information can be added on the basis of the existing visual reproduction, the visual experience is combined with the human touch experience, and the more real interactive experience is provided for the user.
In order to implement the method according to the embodiment of the present invention, an embodiment of the present invention provides an image display apparatus, which may be applied to the electronic device according to the embodiment of the method described above; as shown in fig. 6, the apparatus includes:
the acquiring unit 61 is configured to acquire an image to be displayed and depth information of each pixel region in the image to be displayed;
a determining unit 62, configured to determine a haptic simulation parameter corresponding to the image to be displayed based on at least depth information of each pixel region in the image to be displayed; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed;
and the processing unit 63 is configured to display the image to be displayed in an image display screen, and control a tactile display screen to perform tactile output on the image to be displayed based on the tactile simulation parameter corresponding to each pixel region in the image to be displayed.
In other embodiments of the present invention, the obtaining unit 61 is specifically configured to perform image acquisition on the target object through an image acquisition device to obtain the image to be displayed; emitting a light pulse toward the target object and receiving reflected light of the light pulse reflected by the target object; and acquiring the transmission time from the emission of the light pulse to the reception of the reflected light, and determining the depth information corresponding to each pixel region in the image to be displayed based on the transmission time.
In other embodiments of the present invention, at least one tactile electrode is disposed on the tactile display screen; accordingly, the haptic simulation parameters include at least one of:
the current of each tactile electrode;
a voltage of each tactile electrode;
the pulse frequency of each haptic click.
In other embodiments of the present invention, the determining unit 62 is specifically configured to construct a three-dimensional spatial data model of a target object corresponding to an image to be displayed, based on depth information of each pixel region in the image to be displayed and image information of the image to be displayed; and determining the tactile simulation parameters corresponding to the image to be displayed based on the three-dimensional space data model.
In other embodiments of the present invention, the processing unit 53 is further configured to obtain a touch position of a user touch operation; determining a touch area of an image to be displayed corresponding to the touch position based on the touch position; acquiring a touch simulation parameter of each pixel area in the touch area based on the touch area; and controlling the touch display screen to perform touch output on the image to be displayed at the touch position based on the touch simulation parameters of each pixel area in the touch area.
Based on the hardware implementation of each unit in the multimedia data processing apparatus, in order to implement the image display method provided in the embodiment of the present invention, an embodiment of the present invention further provides an image display apparatus, as shown in fig. 7, where the apparatus 70 includes: a processor 71 and a memory 72 configured to store a computer program capable of running on the processor,
wherein the processor 71 is configured to perform the method steps in the preceding embodiments when running the computer program.
In practice, of course, the various components of the device 70 are coupled together by a bus system 73, as shown in FIG. 7. It will be appreciated that the bus system 73 is used to enable communications among the components. The bus system 73 includes a power bus, a control bus, and a status signal bus in addition to the data bus. For clarity of illustration, however, the various buses are labeled as bus system 73 in fig. 7.
In an exemplary embodiment, the present invention further provides a computer readable storage medium, such as a memory 72, comprising a computer program, which is executable by a processor 71 of an image display apparatus 70 to perform the steps of the aforementioned method. The computer-readable storage medium may be a Memory such as a magnetic random access Memory (FRAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical Disc, or a Compact Disc Read-Only Memory (CD-ROM).
The technical schemes described in the embodiments of the present invention can be combined arbitrarily without conflict.
In the embodiments provided in the present invention, it should be understood that the disclosed method and intelligent device may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present invention may be integrated into one second processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention.
Claims (8)
1. An image display method applied to an electronic device having a tactile display screen, the method comprising:
acquiring an image to be displayed and depth information of each pixel region in the image to be displayed;
constructing a three-dimensional space data model of a target object corresponding to an image to be displayed based on the depth information of each pixel region in the image to be displayed and the image information of the image to be displayed;
acquiring a spatial relative position relation between each pixel point in the image to be displayed based on the three-dimensional spatial data model;
determining a touch simulation parameter corresponding to the image to be displayed based on the spatial relative position relationship between the pixel points; the tactile simulation parameters are used for indicating the surface texture features of the target object corresponding to the image to be displayed;
and displaying the image to be displayed in an image display screen, and controlling the touch display screen to perform touch output on the image to be displayed based on the touch simulation parameter corresponding to each pixel region in the image to be displayed.
2. The method according to claim 1, wherein the obtaining of the image to be displayed and the depth information of each pixel region in the image to be displayed comprises:
acquiring an image of the target object through an image acquisition device to obtain the image to be displayed;
emitting a light pulse toward the target object and receiving reflected light of the light pulse reflected by the target object;
and acquiring the transmission time from the emission of the light pulse to the reception of the reflected light, and determining the depth information corresponding to each pixel region in the image to be displayed based on the transmission time.
3. The method of claim 1, wherein at least one tactile electrode is disposed on the tactile display screen;
accordingly, the haptic simulation parameters include at least one of:
the current of each tactile electrode;
a voltage of each tactile electrode;
the pulse frequency of each haptic click.
4. The method according to claim 1, wherein the controlling the tactile display screen to perform tactile display on the image to be displayed based on the tactile simulation parameters in the image to be displayed comprises:
acquiring a touch position of a user touch operation;
determining a touch area of an image to be displayed corresponding to the touch position based on the touch position;
acquiring a touch simulation parameter of each pixel area in the touch area based on the touch area;
and controlling the touch display screen to perform touch output on the image to be displayed at the touch position based on the touch simulation parameters of each pixel area in the touch area.
5. An image display apparatus, the apparatus comprising:
the device comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring an image to be displayed and depth information of each pixel area in the image to be displayed;
the device comprises a determining unit, a calculating unit and a display unit, wherein the determining unit is used for constructing a three-dimensional space data model of a target object corresponding to an image to be displayed based on the depth information of each pixel region in the image to be displayed and the image information of the image to be displayed; acquiring a spatial relative position relation between each pixel point in the image to be displayed based on the three-dimensional spatial data model; determining a touch simulation parameter corresponding to the image to be displayed based on the spatial relative position relationship between the pixel points;
and the processing unit is used for displaying the image to be displayed in the image display screen and controlling the touch display screen to perform touch output on the image to be displayed based on the touch simulation parameters corresponding to the image to be displayed.
6. The apparatus of claim 5, wherein:
the acquisition unit is specifically used for acquiring an image of the target object through an image acquisition device to obtain the image to be displayed; emitting a light pulse toward the target object and receiving reflected light of the light pulse reflected by the target object; and acquiring the transmission time from the emission of the light pulse to the reception of the reflected light, and determining the depth information corresponding to each pixel region in the image to be displayed based on the transmission time.
7. An image display apparatus, the apparatus comprising: a processor and a memory configured to store a computer program capable of running on the processor,
wherein the processor is configured to execute the steps of the image display method according to any one of claims 1 to 4 when running the computer program.
8. A computer-readable storage medium, on which a computer program is stored which is executed by a processor for implementing the steps of the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811369581.0A CN109343716B (en) | 2018-11-16 | 2018-11-16 | Image display method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811369581.0A CN109343716B (en) | 2018-11-16 | 2018-11-16 | Image display method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109343716A CN109343716A (en) | 2019-02-15 |
| CN109343716B true CN109343716B (en) | 2022-05-10 |
Family
ID=65315724
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811369581.0A Active CN109343716B (en) | 2018-11-16 | 2018-11-16 | Image display method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109343716B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110231865A (en) * | 2019-05-17 | 2019-09-13 | 瑞声科技(新加坡)有限公司 | Tactile feedback method, electronic equipment and storage medium |
| CN111090482B (en) * | 2019-05-22 | 2024-04-19 | 广东小天才科技有限公司 | Content output method for simulating paper material display and electronic equipment |
| CN110659012B (en) * | 2019-09-19 | 2023-04-07 | 闻泰通讯股份有限公司 | Display device and display method of electronic equipment |
| CN112817447A (en) * | 2021-01-25 | 2021-05-18 | 暗物智能科技(广州)有限公司 | AR content display method and system |
| CN113323515B (en) * | 2021-06-07 | 2022-07-22 | 北京京东方技术开发有限公司 | Password output device based on tactile feedback, identity authentication device and method thereof |
| CN113674593A (en) * | 2021-08-25 | 2021-11-19 | 王季源 | Head-wearing forehead machine system for touch display |
| WO2024026638A1 (en) * | 2022-08-01 | 2024-02-08 | Huawei Technologies Co., Ltd. | Haptic interaction with 3d object on naked eye 3d display |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102331838A (en) * | 2011-11-18 | 2012-01-25 | 东南大学 | Method for expressing vibration tactility of image based on depth contour |
| CN102609094A (en) * | 2012-02-24 | 2012-07-25 | 东南大学 | Electric vibration double-mode multidimensional tactile stimulator |
| CN104331155A (en) * | 2014-10-27 | 2015-02-04 | 北京理工大学 | Device using electric stimulation to realize touch reappearing |
| CN104583912A (en) * | 2012-08-20 | 2015-04-29 | 三星电子株式会社 | System and method for perceiving images with multimodal feedback |
| EP3327549A1 (en) * | 2016-11-29 | 2018-05-30 | LG Display Co., Ltd. | Touch sensitive element and display device including the same |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107393017A (en) * | 2017-08-11 | 2017-11-24 | 北京铂石空间科技有限公司 | Image processing method, device, electronic equipment and storage medium |
-
2018
- 2018-11-16 CN CN201811369581.0A patent/CN109343716B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102331838A (en) * | 2011-11-18 | 2012-01-25 | 东南大学 | Method for expressing vibration tactility of image based on depth contour |
| CN102609094A (en) * | 2012-02-24 | 2012-07-25 | 东南大学 | Electric vibration double-mode multidimensional tactile stimulator |
| CN104583912A (en) * | 2012-08-20 | 2015-04-29 | 三星电子株式会社 | System and method for perceiving images with multimodal feedback |
| CN104331155A (en) * | 2014-10-27 | 2015-02-04 | 北京理工大学 | Device using electric stimulation to realize touch reappearing |
| EP3327549A1 (en) * | 2016-11-29 | 2018-05-30 | LG Display Co., Ltd. | Touch sensitive element and display device including the same |
Non-Patent Citations (1)
| Title |
|---|
| 基于ROS和点云库的室内三维物体识别与姿态估计;鄢武;《中国优秀硕士学位论文全文数据库信息科技辑》;20161115;第8-11页，图2-9 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109343716A (en) | 2019-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109343716B (en) | Image display method and device and computer readable storage medium | |
| CN109101111B (en) | Touch sense reproduction method and device integrating electrostatic force, air squeeze film and mechanical vibration | |
| JP6840702B2 (en) | Friction Modulation for 3D Relief in Tactile Devices | |
| CN106570822B (en) | Face mapping method and device | |
| US8330590B2 (en) | User interface feedback apparatus, user interface feedback method, and program | |
| JP4989383B2 (en) | Information processing apparatus and information processing method | |
| CN104657068B (en) | Display device and its driving method | |
| CN102426486B (en) | Stereo interaction method and operated apparatus | |
| KR20150046105A (en) | System and method for perceiving images with multimodal feedback | |
| JP5496032B2 (en) | Tactile sensation presentation apparatus and control method for tactile sensation presentation apparatus | |
| US20180246572A1 (en) | Systems and Methods for Virtual Affective Touch | |
| CN108563327B (en) | Augmented reality method, device, storage medium and electronic equipment | |
| US10678333B2 (en) | Methods and systems for presenting haptically perceptible virtual objects | |
| CN108604130A (en) | Information processing equipment, information processing method and non-transitory computer-readable medium | |
| CN111399645A (en) | Wearable device, tactile feedback method and device, and storage medium | |
| Giannopoulos et al. | Touching the void: exploring virtual objects through a vibrotactile glove | |
| CN105975206A (en) | Virtual reality terminal as well as control method and device thereof | |
| JP6218221B2 (en) | Hairy and flexible user interface | |
| CN110968248B (en) | Generating a 3D model of a fingertip for visual touch detection | |
| CN107748617A (en) | A kind of virtual scene method synchronous with body-sensing, sense of touch | |
| CN102508563B (en) | Stereo interactive method and operated device | |
| CN102880288A (en) | Three-dimensional (3D) display human-machine interaction method, device and equipment | |
| CN108205429B (en) | Control apparatus, system and method for simulating display screen and supporting virtual display apparatus | |
| CN111966222A (en) | High-safety VR virtual reality device, system and method | |
| WO2021060019A1 (en) | Information processing device, information processing method, server device, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |