JP2021522602A - Changing the tactile sensitivity of dialogue with the aerial interface - Google Patents

Abstract

The reflection of the very low frequency signal reflected by the contact surface is captured. The contact surface is in contact with the simulated surface of the object projected from the aerial interface (MAI) device. The difference between the very low frequency signal and the reflection is converted to a measured value of the flow rate at the contact surface. When the measured value is within the measured value, a change in the temperature of the mass of the medium occurs and the simulated surface is projected into the mass of the medium, where the change in temperature is the flow rate of the contact surface. Brings a second change.
[Selection diagram] Fig. 4

Description

The present invention relates to methods, systems, and computer program products for varying the tactile sensitivity of interaction with an aerial interface.

The midair interface (MAI) is a simulation of a solid three-dimensional physical object in a medium such as air by projecting a shape into the medium. When touched, the simulated object feels like a 3D physical object in some respects. The projection in the medium is a holographic projection (hologram), where the tactile attributes are achieved by forming an image in the medium using ultrasound. The hologram formed by projecting sound into the medium may or may not show a visual representation of the simulated object, but the touch sensation when interacting with the simulated object, ie Tactile feedback can be provided.

Holograms can also be produced using light or laser projection into the medium. Light-based holograms may provide better visual representation of simulated objects, but generally lack the ability to provide tactile feedback when interacting with simulated objects.

For purposes of exemplary embodiments, a hologram or simulation capable of providing tactile feedback is the MAI used and described herein. In other words, the MAI, which is considered to be within the scope of the exemplary embodiment, is formed using sound waves to control pressure within the medium or using pressure controlled columns of the medium. Similar to holographic projection.

For example, a MAI-simulated keyboard using an air medium presents the physical keys of a physical keyboard in the air, which allows tactile interaction with the simulated keys to occur in the human user's brain (or humanoid). Make the cognitive system cognitively perceive the physical keys in the air. The keys simulated by MAI are pressed down in a manner similar to the physical keys on a physical keyboard by applying and releasing finger pressure on one or more images that form an air medium, such as physical keys. It can be released.

The simulated key in this example is different from the physical key in the physical keyboard example where the finger pierces the physical key and normally cannot pass through it, but the finger keeps pressing the simulated key. You can finally get through the simulated keys. Other simulated objects generally depict the physical shape, size, and feel of the corresponding physics object, but unlike the corresponding physics object, they form a simulated object rather than a solid one. Due to the non-solid nature of the medium used for this, holes can be drilled as well.

For the sake of clarity and without any limitation to the description, it is assumed that air is the medium when describing various operations and embodiments. Other media, such as water or different gases or fluids, can be used as well in the methods of the described operation or embodiment without departing from the scope of the exemplary embodiment. In addition, for clarity of explanation and without any limitation to exemplary embodiments, it is assumed that a human user is interacting with MAI. This disclosure allows one of ordinary skill in the art to adapt embodiments to work similarly with humanoids or machines acting as users of MAI, such adaptations within the scope of exemplary embodiments. It is thought that it is in.

Devices for projecting holograms into the air are currently available to form MAI simulated objects. For example, a device consisting of a grid of projection nozzles to form a simulated object on the device by adjusting the jet of air emitted from the device towards a mass of air on the device. Currently available.

A mass of air formed, formed, or pressure regulated using sound or other methods to form all or part of a simulated object for MAI is described herein. Then it is called "air foam". Objects or parts of objects simulated using one or more air forms are herein simulated objects, simulated in the air, unless clearly distinguished when used. It is called the shaped shape, or a variant of these phrases.

Aerial dialogue with MAI is the manipulation of simulated objects in MAI using tactile movements. For example, by applying a physical force to a simulated object, the simulated key can be touched, pressed down, and released, and the simulated ball It can be grabbed, rotated, or compressed, and the simulated graph can be touched, rubbed, or pushed. Aerial dialogue provokes cognitive perception of touch. When humans are interacting with MAI, interacting with MAI cognitively induces tactile sensation in the human brain. The human brain is a kind of cognitive receptor. When a humanoid or machine is interacting with MAI, interacting with MAI cognitively induces tactile sensation within the appropriate cognitive receptor unit associated with the humanoid or machine. In the following, the reference to cognitive evoked is a reference to tactile sensation that arises from a physical dialogue with a simulated object produced by MAI and can be received and processed by the appropriate receptor. be.

A first aspect of the present invention comprises a method of capturing the reflection of an ultra-low frequency signal in which the ultra-low frequency signal is reflected at the contact surface and the contact surface is projected from an aerial interface (MAI) device. It is in contact with the simulated surface. In the embodiment, the difference between the very low frequency signal and the reflection is converted into a measured value of the flow rate at the contact surface. The embodiment results in a change in the temperature of the mass of medium in response to the measured value being within the range of the measured value, the simulated surface being projected into the mass of medium and the change in temperature. , Brings a second change in the flow rate of the contact surface.

Another aspect includes a computer-enabled program product. A computer-enabled program product includes a computer-readable storage device, in which program instructions are stored in the storage device and can be executed to perform the method of the first aspect.

Another aspect includes a computer system. The computer system includes a processor, a computer-readable memory, and a computer-readable storage device, in which program instructions are stored by the memory in the storage device for execution by the processor to perform the method of the first aspect.

Certain novel features that are believed to be features of the present invention are set forth in the appended claims. However, the invention itself, as well as the preferred modes of use, its further objectives and advantages, will be best understood when read with the accompanying drawings by reference to the following detailed description of the exemplary embodiments. Let's go.

FIG. 6 is a block diagram of a network of data processing systems in which exemplary embodiments can be implemented. It is a block diagram of a data processing system in which an exemplary embodiment can be executed. FIG. 3 is a block diagram of some examples of MAI configured and manipulated by exemplary embodiments. It is a figure which describes the example of the structure for changing the tactile sensitivity of the dialogue with an aerial interface by an exemplary embodiment. It is a figure which describes the change of the resolution by an exemplary embodiment. It is a figure which describes the contouring of heat by an exemplary embodiment. It is a flow chart of an example of the process for changing the tactile sensitivity of the dialogue with an aerial interface by an exemplary embodiment. It is a flow chart of another example of the process for changing the tactile sensitivity of the dialogue with the aerial interface according to the exemplary embodiment.

Human-machine interface is a well-recognized area of technical effort. Computers and other machines request input from the user and output to the user through such a human-machine interface. Computer keyboards, computer mice or other pointing devices, digital styluses or similar writing instruments, touch screens, and the like are common examples of human-machine interfaces for input. Some interfaces, such as touch screens or pointing devices, provide tactile feedback through vibrations, texture changes, required force changes, pressure changes, and the like.

Projecting an aerial interface as an alternative human-machine interface, and detecting interaction with an aerial interface are also areas of well-recognized technological effort. The current state of technology in this effort-focused area has certain drawbacks and limitations. The operation of the exemplary embodiment provides additional or new capabilities to improve existing technology in the effort-focused areas of human-machine interface technology, especially in the area of aerial interfaces.

Illustrative embodiments recognize that the aerial interface suffers from certain drawbacks. As an example, in human users of MAI, the fingers, arms, and other limbs of the human body that are engaged when interacting with MAI are paralyzed or progressive in their sensory perception during the duration of the dialogue. We found that the decline tended to occur. For example, it was observed that an arm floating in the air on a MAI device would begin to become paralyzed during the floating period. The reduced sensitivity of the fingertips to the tactile sensation is also reduced as a result.

It has also been found that repeated touching, pressing, rubbing, or other tactile manipulations of the surface also tend to cause a gradual decrease in the tactile sensitivity of the limbs used. For example, it is known that when a user repeatedly touches or rubs a textured surface, the user's perception of the texture diminishes after a period of such tactile activity. This observation applies whether the surface is a real physical surface or a simulated surface of MAI.

The surfaces of the user's limbs that come into contact with the MAI to perform tactile movements are referred to herein as "contact surfaces" unless clearly distinguished. The surface of a simulated object touched by a contact surface is referred to herein as an "object surface" unless clearly distinguished.

These and other reductions in tactile sensation on the contact surface are disadvantages of the use of MAI currently available. The current state of effort-focused areas of human-machine interface technology through MAI has not currently addressed or mitigated the tactile decline. It is necessary to detect a decrease in tactile sensation while using MAI. It is necessary to prevent or mitigate such a decrease. If it decreases, it is necessary to restore the tactile sensitivity.

In exemplary embodiments, it is recognized that currently available tools or solutions do not address these needs / problems or provide reasonable solutions to these needs / problems. There is. Illustrative embodiments used to illustrate the present invention generally address the above problems and other related problems by varying the tactile sensitivity of the interaction with the aerial interface.

A device capable of projecting a MAI consisting of simulated objects formed using air foam is envisioned. Such devices are referred to herein as MAI devices. The MAI device of the prior art can be modified using one embodiment such as forming a modified MAI device. To form a modified MAI device, one embodiment can be performed as a combination of certain hardware components, such as a prior art MAI device, with a software application. An embodiment of the embodiment, or one or more components thereof, can be configured as a variant of an existing MAI device, and companion software applications can include (i) the MAI device itself, (ii). It is performed by several combinations of a data processing system that communicates with a MAI device over a short-range radio or local area network (LAN), and a data processing system that communicates with a MAI device over a (iii) wide area network (WAN).

Discharge nozzles are arranged in a two-dimensional array in a prior art MAI device. The discharge nozzle projects the air foam inside a mass of air, which forms a simulated object. An air form can be thought of as a single pixel or group of pixels (or equivalents thereof) that depict a portion of the object face of a simulated object. One or more simulated objects together form a MAI.

To form one embodiment of the modified MAI device, the embodiment adds additional components to the array of discharge nozzles. In one embodiment, for each emission nozzle, the embodiment places an extremely low frequency transducer (SST) in or near the emission nozzle.

The SST is configured to emit an acoustic output (ultra-low frequency signal), where the frequency of sound is below the human audible frequency range (hence the "ultra-low frequency"). The SST is further configured to receive the reflection of the emitted very low frequency signal as an input. SST can be an additional very low frequency component or a modification of an existing acoustic component to operate in the very low frequency frequency range, as described herein.

In addition, the SST is configured to project very low frequency signals into the air foam or onto substantially the same pixels or groups of pixels represented by the air foam. As a result of such a configuration, when a contact surface, such as a human user's fingertip, manipulates a portion of an object surface formed using air foam, the emitted very low frequency signal bounces off the contact surface. Received by SST.

In one embodiment, the transmitter of the very low frequency signal and the receiver of the reflection can be separate devices at different physical locations with respect to the corresponding emission nozzles on the modified MAI device. In another embodiment, the transmitter of the very low frequency signal and the receiver of the reflection are in the same location at substantially the same physical location with respect to the corresponding emission nozzle on the modified MAI device.

Infrasound is currently used to measure the flow rate of liquids. There are currently measurement techniques and devices for calculating the difference between the transmitted very low frequency signal and the reflection of this signal from the surface, and for converting the difference into a flow measurement. In other words, the difference between the transmitted very low frequency signal and the reflected very low frequency signal has a correspondence with the amount of liquid flow on or near the surface on which the signal is reflected. In the case of one example, the difference may be the difference between the transmission frequency and the reception frequency. In another example, the difference may be the difference in signal strength between the transmitted signal and the received signal. In another example, the difference may be the phase difference between the transmitted signal and the received signal.

Using such techniques, embodiments measure the amount of blood flowing through capillaries on or near the contact surface. In the embodiment, for a flow rate range that is acceptable to a given user and a particular contact surface, the volume of flow can be considered normal, low, or high. Is determined from the measured value of the flow rate.

In the embodiment, the flow rate is considered normal when the measured value of the flow rate is within the range defined by the two thresholds and includes the two thresholds. In the embodiment, when the measured value of the flow rate is less than the smaller of the two threshold values, the flow rate is considered to be low. In the embodiment, when the measured value of the flow rate exceeds the larger of the two threshold values, the flow rate is considered to be high.

Embodiments further consider low flow rates to indicate reduced tactile sensation, such as those caused by paralysis of the extremities, or reduced sensitivity of the contact surface to tactile input. The embodiment also considers high flow rates to exhibit reduced tactile sensation, such as those caused by inflammation of the extremities due to repeated tactile movements by the contact surface.

The embodiment adds a thermal element connected to a discharge nozzle as part of forming a modified MAI device. In one embodiment, the thermal element is associated with a single emission nozzle and is located with the nozzle in substantially the same position as the nozzle. In another embodiment, the thermal element is associated with a plurality of discharge nozzles and is in close proximity to a group of discharge nozzles (substantially the same region on the MAI device). In another embodiment, the thermal element is located at a different position than the emission nozzle, such as around a variable MAI device.

In one embodiment, the thermal element is configured to raise or lower the temperature of the air from which the nozzle emits air foam. In another embodiment, the thermal element is configured to only raise the temperature of the air that the nozzle emits air foam into.

When the embodiment detects low blood flow through the very low frequency measurements described herein, the embodiment activates a thermal element to raise the temperature of the air. In particular, embodiments activate these thermal devices configured such that one or more selected discharge nozzles warm this mass of air ejecting air foam, and as a result, these. The air foam forms the object surface where contact with the contact surface is occurring. Such activation is selective and does not have to activate the thermal elements of other emission nozzles that are projecting other parts of the simulated object that are not manipulated by the contact surface.

As the temperature of the air on or near the object surface rises, so does the temperature of the contact surface during the tactile operation of the object surface. As the temperature of the contact surface increases, blood flow to the contact surface increases, reducing paralysis of the contact surface, the limbs associated with the contact surface, or both. When the paralysis is reduced, the tactile sensitivity of the contact surface is improved.

When the embodiment detects high blood flow through the infrasound measurements described herein, and when the thermal element is also configured to cool the air, the embodiment is thermal. Activates the element and lowers the temperature of the air. In particular, embodiments activate these thermal elements in which one or more selected discharge nozzles are configured to cool this mass of air ejecting air foam, and as a result, these. The air foam forms the object surface where contact with the contact surface is occurring. Such activation is selective and does not have to activate the thermal elements of other emission nozzles that are projecting other parts of the simulated object that are not manipulated by the contact surface.

As the temperature of the air on the object surface decreases, so does the temperature of the contact surface during the tactile operation. When the temperature of the contact surface decreases, the blood flow on the contact surface decreases. Decreased blood flow reduces inflammation of the contact surface, the limbs associated with the contact surface, or both. When inflammation is reduced, it improves and / or restores the level of tactile sensitivity of the contact surface.

Another embodiment configures the discharge nozzle to increase or decrease the resolution of the air foam. In an exemplary embodiment, when a large area of contact is used for tactile dialogue with a simulated object, the large area of nerve endings may all be activated to detect touch sensations. , Recognized. Thus, when the contact surface becomes paralyzed or inflamed, all of the nerve endings in the entire region are adversely affected together, i.e., they all encounter a decrease in tactile sensitivity.

When the embodiment detects an indicator that the tactile sensitivity of the contact surface is reduced, the embodiment compares to the size of the area of the object surface that is projected when the reduced tactile sensitivity is detected. Causes the discharge nozzle to reshape the air foam so that it projects an area of a small object surface. The smaller the region, the fewer nerve endings are engaged in a given tactile sensation, giving the unengaged nerve endings time to recover.

The change in resolution by changing the shape of the air foam can be performed by various methods. For example, one embodiment shrinks or enlarges the nozzle opening to change the size of the air foam and, necessarily, the resolution of the features projected by the air foam. Another embodiment blocks or does not block the air foam without altering the nozzle characteristics. Blocking or not blocking the path of air foam through the air changes the magnitude of the air flow and changes the resolution. Another embodiment raises or lowers the discharge nozzle with respect to the contact surface to change the distance between the nozzle opening and the contact surface. The longer the air flow path, the lower the feature resolution (the resolution or grain size becomes coarser). Conversely, shorter airflow paths increase feature resolution (decrease resolution or particle size). Therefore, changing the distance of the air flow path changes the resolution.

Features such as the texture or contours of an object surface appear on a relatively large surface, which can be difficult for the user to perceive. For example, a 0.1 inch (0.254 cm) height difference in a feature may not be perceptible with reduced tactile sensitivity if the feature is projected over an area of 1 square inch (6.4516 square cm). be. However, the same height difference of 0.1 inches (0.254 cm) in features can be perceived even with reduced tactile sensitivity when the features are projected in an area of 0.1 square inches (0.64516 square cm). there is a possibility.

Therefore, varying the resolution, i.e., allowing the embodiment to show features projected in a somewhat smaller area compared to previous projections of the same features, improves the tactile sensation of the features. In addition, increasing the resolution means that even if the projected features appear in a relatively small area depending on the embodiment, the features that will be in the same area will be finer than in the lower resolution projection. Will be possible. The size of the area and the resolution of the feature are described in comparison with each other and are not relative to any absolute size or reference.

In an exemplary embodiment, it is recognized that a paralyzed arm or leg can become paralyzed when in the same position for some time and regain its tactile sensitivity when moved to a different position.

Another embodiment causes the user to move the limbs associated with the contact surface to the user to alter blood flow (and consequent paralysis) on the contact surface. The embodiment induces movement by creating a temperature contour on the simulated object.

For example, suppose the surface of a simulated object contains areas A, B, and C, where area A is next to area B and area B is next to area C. Further, it is assumed that the user keeps the contact surface stationary or repeatedly places the contact surface in the region A forming the current object surface. Further, it is assumed that moving the contact surface from region A to region B or C does not adversely affect the underlying command or action being performed by tactile manipulation.

The embodiment determines that the user should be induced to relocate the contact surface from region A to region C on the simulated object. The embodiment indicates that the user dislikes hot surfaces, i.e. indicates that the surface is above the upper temperature limit, and prefers cold surfaces, i.e. indicates that the surface is below another lower temperature limit. Further decisions, from the user's previous MAI dialogue, or from other methods, such as through machine learning.

Therefore, the embodiment allows the mass of air in the air foam from the nozzle projecting region A to warm to a temperature above the upper limit. The embodiment ensures that the air flow from the nozzle projecting region B is at a temperature between the upper and lower limits. In the embodiment, the air flow from the nozzle projecting the region C is set to a temperature lower than the lower limit. By adjusting the temperature of the air foam forming regions A, B, and C, the embodiment outlines the heat or temperature that gradually descends from region A to region C.

The aerial contour of the projected surface of the simulated object remains unchanged, but the thermal contour of this surface changes in this way. Therefore, the user who prefers the colder temperature is induced to rearrange the contact surfaces from region A to region B and from region B to region C. Such movement results in movement of the limbs. The movement of the limbs facilitates the restoration of tactile sensitivity of the limbs and / or contact surfaces.

In exemplary embodiments, it is also recognized that tetany contractions or tetany spasticity can cause rapid and repetitive movements of limbs such as fingers. Illustrative embodiments recognize that tetany movements can cause the user to repeatedly touch the simulated surface.

The embodiment detects repeated contact of the user's limbs with the simulated surface. The embodiment classifies the speed of user contact with a simulated surface according to a predetermined set of tetany classifications. Some tetany classifications (ie, some of the speeds of rapidly repeated contacts) are acceptable and some are not. When the embodiment determines that the abruptly repeated contact falls into the unacceptable tetany classification, the embodiment induces limb movement. Embodiments use either the method of temperature contouring or the method of changing resolution to induce movement as described herein.

The techniques described herein for varying the tactile sensitivity of dialogue with an aerial interface are not available in currently available methods in the field of effort-focused technology for human-machine interfaces, especially in the field of MAI. The methods of the embodiments described herein, when performed to perform on a device or data processing system, of this device or data processing system in improving tactile sensitivity and projection of features in MAI. Includes substantial progress in functionality.

Illustrative embodiments include certain types of MAI devices, media, capacitances, media formats, users, simulated objects, contact surfaces, fluid flows, temperatures, feature projections, repetitions, tactile sensations, thermal contours. , Object planes, algorithms, equations, configurations, locations of materialization, additional data, devices, data processing systems, environments, components, and applications are described as examples. Specifying any of these and other similar artifacts is not intended to limit the invention. Appropriate manifestation of any of these and other similar artifacts can be selected within the scope of exemplary embodiments.

Further, exemplary embodiments may be implemented with respect to access to any type of data, data source, or data source in a data network. Any type of data storage device can provide data to embodiments of the invention, within the scope of the invention, locally in a data processing system, or in a data network. When an embodiment is described using a mobile device, any type of data storage device suitable for use with the mobile device is local to the mobile device, within the scope of the exemplary embodiment. Data can be provided to such embodiments, or in a data network.

Illustrative embodiments are described as mere examples using specific code, design, architecture, protocols, layouts, diagrams, and tools, but are not limited to exemplary embodiments. In addition, exemplary embodiments are described in some cases as merely examples, using specific software, tools, and data processing environments for the sake of clarity. Illustrative embodiments may be used in conjunction with other similar or similar purpose structures, systems, applications, or architectures. For example, other similar mobile devices, structures, systems, applications, or architectures thereof may be used within the scope of the invention in conjunction with such embodiments of the invention. Illustrative embodiments may be implemented in hardware, software, or a combination thereof.

The examples in the present disclosure are used only for the sake of clarity and are not limited to exemplary embodiments. Additional data, actions, actions, tasks, activities, and operations can be imagined from this disclosure, and the same is believed to be within the scope of exemplary embodiments.

None of the advantages mentioned herein are merely examples and are not intended to be limited to exemplary embodiments. Additional or other advantages may be realized by certain exemplary embodiments. Moreover, certain exemplary embodiments may or may not include some or all of the advantages listed above.

With reference to the figures, and particularly with reference to FIGS. 1 and 2, these figures are examples of figures of a data processing environment in which exemplary embodiments can be implemented. 1 and 2 are examples only and are not intended to assert or imply any limitation on the environment in which different embodiments may be implemented. The particular implementation may make many modifications to the environment depicted based on the following description.

FIG. 1 depicts a block diagram of a network of data processing systems in which exemplary embodiments can be implemented. The data processing environment 100 is a network of computers on which exemplary embodiments can be implemented. The data processing environment 100 includes a network 102. The network 102 is a medium used to provide communication links between various devices and computers connected together within the data processing environment 100. The network 102 may include connections such as wires, wireless communication links, or fiber optic cables.

The client or server is merely an example of the role of certain data processing systems connected to network 102 and is not intended to exclude other configurations or roles of these data processing systems. The server 104 and the server 106 are connected to the network 102 together with the storage unit 108. The software application can be run on any computer in the data processing environment 100. Clients 110, 112, and 114 are also connected to network 102. A data processing system such as a server 104 or 106, or a client 110, 112, or 114 can contain and have software applications or software tools running on it.

By way of example and not by any limitation to such an architecture, FIG. 1 depicts certain components that can be used in the examples of embodiments of the embodiments. For example, servers 104 and 106, and clients 110, 112, 114 are portrayed as servers and clients by way of example only and do not imply a limitation on a client-server architecture. As another example, an embodiment can be distributed across several data processing systems and data networks, as shown, while another embodiment is within the scope of an exemplary embodiment. , Can be run on a single data processing system. Data processing systems 104, 106, 110, 112, and 114 also represent examples of nodes in clusters, partitions, and other configurations suitable for performing embodiments.

Device 132 is an example of a device described herein. For example, device 132 can be in the form of a smartphone, tablet computer, laptop computer, stationary or portable client 110, wearable computing device, or any other suitable device. .. Any software application described as running on another data processing system of FIG. 1 can be configured to run on device 132 in a similar fashion. Any data or information stored or produced in another data processing system of FIG. 1 can be configured to be stored or produced in device 132 in a similar fashion.

Application 115 implements the embodiments described herein. The MAI device 116 is a modified MAI device in which hardware modifications are made according to one embodiment and the application 115 is used to operate a software execution mode of the embodiment as described herein.

Servers 104 and 106, storage units 108, and clients 110, 112, and 114, and device 132 can be connected to network 102 using wired connections, wireless communication protocols, or other suitable data connections. .. Clients 110, 112, and 114 may be, for example, personal computers or network computers.

In the illustrated example, the server 104 can provide data such as boot files, operating system images, and applications to clients 110, 112, and 114. Clients 110, 112, and 114 may be clients to server 104 in this example. Clients 110, 112, 114, or some combination thereof, may include proprietary data, boot files, operating system images, and applications. The data processing environment 100 may include additional servers, clients, and other devices (not shown).

In the illustrated example, the data processing environment 100 may be the Internet. The network 102 can represent a collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) and other protocols for communicating with each other. At the heart of the Internet is the backbone of data communication links between major nodes or host computers, including thousands of commercial, governmental, educational, and other computer systems that route data and messages. Of course, the data processing environment 100 may also be implemented as several different types of networks, such as an intranet, a local area network (LAN), or a wide area network (WAN). FIG. 1 is intended to be an example and is not intended to be a structural limitation to another exemplary embodiment.

Among other uses, the data processing environment 100 may be used to run a client-server environment in which exemplary embodiments can be performed. A client-server environment allows software applications and data to be distributed across a network so that applications can function by using the bidirectionality between the client data processing system and the server data processing system. To. The data processing environment 100 can also use a service-oriented architecture in which interoperable software components distributed throughout the network can be packaged together as a coherent business application. The data processing environment 100 may also be in the form of a cloud of configurable computing resources that can be quickly provided and exposed with minimal management effort or interaction with the service provider. Service-delivered cloud computing to enable convenient and on-demand network access to shared pools (eg, networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services). A wing model may be used.

With reference to FIG. 2, this figure depicts a block diagram of a data processing system in which exemplary embodiments can be implemented. The data processing system 200 may be populated with servers 104 and 106 of FIG. 1, or clients 110, 112, and 114, or computer-enabled program code, or instructions that perform processing for exemplary embodiments. An example of a computer, such as another type of device.

The data processing system 200 also represents a data processing system or configuration thereof, such as the data processing system 132 of FIG. 1 where computer-enabled program code or instructions to perform processing of exemplary embodiments may be located. The data processing system 200 is not limited to these, but is described as a computer as a mere example. Implementations in the form of other devices, such as the device 132 of FIG. 1, can modify the data processing system 200 by adding a touch interface or the like, and the operations and functions of the data processing system 200 described herein. It is even possible to remove certain delineated components from the data processing system 200 without departing from the overall description of.

In the illustrated example, the data processing system 200 has a north bridge and memory controller hub (NB / MCH) 202, as well as a south bridge and input / output (I / O) controller hub (SB / ICH) 204. Use the including hub architecture. The processing unit 206, main memory 208, and graphics processor 210 are coupled to the north bridge and memory controller hub (NB / MCH) 202. The processing unit 206 can accommodate one or more processors and may be implemented using one or more heterogeneous processor systems. The processing unit 206 may be a multi-core processor. The graphics processor 210 may be coupled to the NB / MCH 202 through an Accelerated Graphics Port (AGP) in certain implementations.

In the illustrated example, the local area network (LAN) adapter 212 is connected to the south bridge and the I / O controller hub (SB / ICH) 204. Audio adapter 216, keyboard and mouse adapter 220, modem 222, read-only memory (ROM) 224, universal serial bus (USB) and other ports 232, and PCI / PCIe device 234 are south bridges. And connected to the I / O controller hub 204 via bus 238. The hard disk drive (HDD) or solid state drive (SSD) 226, and CD-ROM230 are connected to the south bridge and I / O controller hub 204 via bus 240. The PCI / PCIe device 234 may include, for example, an Ethernet (R) adapter, an add-in card, and a PC card for a notebook computer. PCI uses a card bus controller, but does not use PCIe. The ROM 224 may be, for example, a flash binary input / output system (BIOS). Hard disk drives 226 and CD-ROM230 are, for example, integrated drive electronics (IDE), Serial Advanced Technology Attachment (SATA) interfaces, or variants such as External SATA (eSATA) and Micro SATA (mSATA). You can use things. The Super I / O (SIO) device 236 may be connected to the south bridge and I / O controller hub (SB / ICH) 204 via bus 238.

Memory such as main memory 208, ROM 224, or flash memory (not shown) is some example of a computer-enabled storage device. Hard disk drives or solid state drives 226, CD-ROM230s, and other similarly usable devices are some examples of computer-enabled storage devices, including computer-enabled storage media.

The operating system runs on processing unit 206. The operating system coordinates and executes control of various components within the data processing system 200 of FIG. The operating system may be a commercially available operating system for any type of computing platform, including but not limited to server systems, personal computers, and mobile devices. Object-oriented or other types of programming systems can work with the operating system to provide calls to the operating system from programs or applications running on the data processing system 200.

Instructions for an operating system, object-oriented programming system, and application or program, such as application 115 in FIG. 1, are placed on a storage device, such as code 226A on hard disk drive 226, and a processing unit. It may be loaded into at least one of one or more memories, such as main memory 208, for execution by 206. The processing of the exemplary embodiment may be performed by the processing unit 206 using computer execution instructions, such as in memory such as main memory 208, read-only memory 224. Alternatively, it may be located on one or more peripheral devices.

Further, in some cases, code 226A may be downloaded from remote system 201B through network 201A, where similar code 201C is stored in storage device 201D. In another case, code 226A may be downloaded to remote system 201B through network 201A, where the downloaded code 201C is stored in storage device 201D.

The hardware of FIGS. 1 and 2 may be changed depending on the mounting form. Other internal hardware or peripheral devices such as flash memory, equivalent non-volatile memory, or optical disk drives, and the like, in addition to, or instead of, the hardware depicted in FIGS. May be used. Further, the processing of the exemplary embodiment may be applied to a multiprocessor data processing system.

In some exemplary examples, the data processing system 200 may be a personal digital assistant (PDA), providing non-volatile memory for storing operating system files and / or user-generated data. Generally configured by flash memory to provide. The bus system can include one or more buses such as a system bus, an I / O bus, and a PCI bus. Of course, the bus system may be implemented using any type of communication fabric or architecture that provides data transfer between different components or devices attached to the fabric or architecture.

The communication unit may include one or more devices used to send and receive data, such as modems or network adapters. The memory may be, for example, main memory 208 or cache, such as the cache found in the north bridge and memory controller hub 202. The processing unit may include one or more processors or CPUs.

The examples depicted in FIGS. 1 and 2 and the examples described above are not intended to imply structural limitations. For example, the data processing system 200 may also be a tablet computer, a laptop computer, or a telephone device that is not just in the form of a mobile or wearable device.

When a computer or data processing system is described as a virtual machine, virtual device, or virtual component, the virtual machine, virtual device, or virtual component virtualizes some or all of the components depicted in the data processing system 200. It operates in the style of a data processing system 200 that uses what appears to be. For example, in a virtual machine, virtual device, or virtual component, the processing unit 206 appears as all or some virtualization instances of the hardware processing unit 206 available in the host data processing system, and the main memory 208 is Appearing as virtualized instances of all or some parts of main memory 208 that may be available in the host data processing system, disk 226 is all or some of the disks 226 that may be available in the host data processing system. Appears as a virtualized instance of that part. The host data processing system in such a case is represented by the data processing system 200.

With reference to FIG. 3, this diagram depicts a block diagram of some examples of MAI configured and manipulated by exemplary embodiments. MAI device 302 is an example of MAI device 116 and includes an array of MAI elements 303. Each MAI element 303 includes a discharge nozzle 303A as shown. The MAI device 302 further includes one or more SSTs (invisible), one or more thermal elements (invisible), or a combination thereof, along with one emission nozzle 303A. In one embodiment, the SST, thermal element, or combination thereof is associated with a single MAI element 303. In another embodiment, the SST, thermal element, or combination thereof is associated with a plurality of MAI elements 303.

Application 115 provides computational features for operating the MAI device 302. For example, application 115 calculates the flow velocity of a fluid from very low frequency measurements, commands the SST to send or receive very low frequency signals, commands the thermal element to activate, and the MAI device 302. The motion changes the resolution of the air foam, calculates the position of the region related to the heat contour, determines the user's likes and dislikes of heat, and calculates and determines the need to guide the movement along the heat contour. Also, perform other computational operations as described herein.

These examples of computational behavior in all or some of the software implementations of the embodiment are not intended to be limiting. From this disclosure, one of ordinary skill in the art will be able to determine many other computational operations that can be performed in the software to provide the features of the embodiments described herein. It is believed to be within the scope of exemplary embodiments.

As an example, a subset of the emission nozzles of the MAI device 302 can be used to project a simulated keyboard key 304. The keys in the simulated key 304 can be touched, tapped, or used when typing in the same manner as the physical keys on a physical keyboard.

As another example, a subset of the emission nozzles of the MAI device 302 can be used to project a simulated ball 306. The ball 306 can be touched, grasped, grabbed, or thrown, or force can be applied to the ball 306 in the same manner as a physical ball.

As an example, a subset of the emission nozzles of the MAI device 302 can be used to project a simulated piano key 308. The keys in the simulated piano key 308 can be touched, pressed down, and released in the same manner as the physical keys of a physical piano.

With reference to FIG. 4, this figure illustrates an example of a configuration for varying the tactile sensitivity of dialogue with an aerial interface, according to an exemplary embodiment. MAI element 402 is an example of MAI element 303 in FIG. The discharge nozzle 404 is an example of the discharge nozzle 303A of FIG.

The discharge nozzle 404 discharges air foam 418 to draw an object surface 405. The SST406 sends an extremely low frequency signal 408 and receives the reflected extremely low frequency signal 410. The signal 408 is directed to the object surface 405. The reflected signal 410 is a reflection of the signal 408 from a contact surface that operates the object surface 405. For example, the contact surface may be part of a finger 412 that is in tactile contact with the object surface 405.

According to one embodiment, the SST406 is arranged with the discharge nozzle 404 of the MAI element 402. According to another embodiment, the SST406 is located outside the MAI element 402. The SST406 is depicted next to the discharge nozzle 404 in the MAI element 402 as a mere non-limiting example. In another embodiment, the SST406 can be configured within the discharge nozzle 404. In yet another embodiment, the SST 406 and the emission nozzle 404 can be exactly the same, where prior art emission nozzles, such as an ultra-low frequency emitting device, similarly deliver an extremely low frequency signal. Reconfigured to send and receive.

The thermal element 414 is associated with the MAI element 402. According to one embodiment, the thermal element 414 is arranged with the discharge nozzle 404 of the MAI element 402. The thermal element 414 is depicted in the emission nozzle 404 as a mere non-limiting example. In another embodiment, the thermal element 414 can be configured next to the discharge nozzle 404. In yet another embodiment, the thermal element 414 is located outside the MAI element 402, but can be associated with air foam 418 emitted from the emission nozzle 404.

With reference to FIG. 5, this figure illustrates the change in resolution according to an exemplary embodiment. Surface 502 is a portion of the surface of the simulated object. The finger 504 contacts the surface 502, as a non-limiting example. Region 506 is the region where the finger 504 comes into contact with the surface 502. The region 506 of the finger 504 forms a contact surface, and the region 506 of the surface 502 forms an object surface such as the object surface 405 of FIG.

The number of nerve endings or other tactile sensors within the contact surface region 506 on the finger 504 is proportional to the size of the region 506. As described herein, embodiments increase the resolution of region 504 of surface 502. In other words, if the finger 504 was previously in contact with a region of the size of region 506, the embodiment allows the surface features of region 506 to appear in the relatively small region 508. In one embodiment, all features that occupy region 506 are included in region 508. In another embodiment, all features occupying region 506 have increased within feature portions and one or more smaller regions, such as by using region 508 and one or more other regions 510. Or it is split into separate parts that will appear with noticeable details.

With reference to FIG. 6, this figure depicts the contouring of heat according to an exemplary embodiment. Surface 602 is an example of surface 502 in FIG. Finger 604 is assumed to be in position 1 (indicated as circle 1). At position 1, finger 604 is in contact with region 606 of surface 602.

The embodiment identifies regions 608 and 610 to form a thermal contour 612. Assuming that the user dislikes hot surfaces and prefers cold surfaces, the embodiment changes the temperature of region 606 to a first temperature, such as in application 115, where the first temperature is above the upper temperature limit. be. Similarly, an embodiment changes the temperature of region 608 to a second temperature, where the second temperature is between the upper and lower temperature limits. In the embodiment, the temperature of the region 610 is changed to a third temperature, where the third temperature is not less than or equal to the lower limit temperature.

In this way, the embodiment draws a temperature contour 612 on the surface 602. The temperature contour 612 moves the finger 604 from position 1 to position 2 (depicted as circle 2) and from position 2 to position 3 (depicted as circle 3). The movement of the finger 604 is guided by the temperature contour from the relatively hot region 606 through the intermediate temperature region 608 and finally to the preferred cold temperature region 610.

Examples of temperature contours that include three regions of gradually changing temperature are not intended to be limiting. From this disclosure, one of ordinary skill in the art will appreciate temperature contours as well as other types of movement induction in a region of an approximate number of different temperatures, different shapes, different textures, different tactile features, different sound wave features, and similar ones. It has become possible to adapt the embodiments to form the contours of the, and such adaptations are considered to be within the scope of the exemplary embodiments.

With reference to FIG. 7, this figure illustrates a flow chart of an example of a process for varying the tactile sensitivity of a dialogue with an aerial interface, according to an exemplary embodiment. Process 700 can be executed by application 115 of FIG.

The application presents MAI using a modified MAI device (block 702). The application detects a tactile dialogue with MAI (block 704). The application sends an extremely low frequency signal to the contact surface (block 706). The application receives an extremely low frequency signal reflected from the contact surface (block 708).

Using the difference between the transmitted very low frequency signal and the received very low frequency signal, the application calculates a reference value for the flow rate (block 710). The reference value of the flow rate indicates the amount of blood flow on the contact surface.

If the flow reference value indicates a high flow rate (the "high" path of block 712), the application lowers the temperature of the simulated surface at the contact point or contact area (block 714). The application then finishes processing 700.

If the reference value of the flow rate indicates a low flow rate (the "low" path of block 712), the application raises the temperature of the simulated surface of the contact point or contact area (block 716). The application then finishes processing 700.

If the flow reference value indicates a normal flow rate (the "normal" path of block 712), the application leaves the temperature of the simulated surface at the contact point or contact area unchanged. The application then finishes processing 700.

With reference to FIG. 8, this figure illustrates a flow chart of another example of processing for varying the tactile sensitivity of a dialogue with an aerial interface, according to an exemplary embodiment. Process 800 can be executed by application 115 of FIG.

The application presents MAI (block 802). The application detects the occurrence of a dialogue with MAI (block 804). The application of the dialogue, such as by classifying the dialogue according to the speed of the touch, the speed of change of the touch pressure, the speed of the touch repetition, the repetition of the touch movement, or some combination of these. Perform tetany classification (block 806).

If the tetany classification is acceptable (the "no" path in block 808), the application does not move the user's limbs and then exits process 800. If the tetany classification is unacceptable (the "yes" path of block 808), the application forms a simulated surface temperature contour (block 810) and changes the resolution of the object surface (block 812). ), Or both, to move the user's limbs. The application then finishes processing 800.

In this way, a computer execution method, system or device, and computer program product is an exemplary embodiment for varying the tactile sensitivity of interaction with an aerial interface, and other related features, functions, or behaviors. Provided at. When an embodiment or part thereof is described for a type of device, the computer execution method, system or device, computer program product, or part thereof is used with appropriate and equivalent manifestation of that type of device. Fitted or configured for.

When an embodiment is described as being executed by an application, the delivery of the application in the software as a service (Software as a Service) model is considered to be within the scope of the exemplary embodiment. In the SaaS model, the ability of an application to execute an embodiment is provided to the user by running the application on a cloud infrastructure. Users can access applications using a variety of client devices through thin client interfaces such as web browsers (eg, web-based email) or other lightweight client applications. Users do not manage or control the underlying cloud infrastructure, including storage for networks, servers, operating systems, or cloud infrastructure. In some cases, the user does not even have to manage or control the capabilities of the SaaS application. In some other cases, the SaaS implementations of the present application can tolerate possible exclusions of limited user-specific application configuration settings.

The present invention may be a system, method, computer program product, or a combination thereof in any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or a plurality of media) having computer-readable program instructions for causing the processor to perform aspects of the present invention.

The computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by the instruction executing device. The computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination described above, but is limited thereto. Not done. An incomplete list of more specific examples of computer-readable storage media includes portable computer disksets, hard disks, random access memory (RAM), read-only memory (ROM), and erasable programmable. Read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory Includes mechanically encoded devices such as sticks, floppy disks, punch cards on which instructions are recorded or grooved ridge structures, and any suitable combination described above. Computer-readable storage media, including but not limited to computer-readable storage devices as used herein, include radio waves or other freely propagating electromagnetic waves, waveguides or other transmitting media (eg,). , Optical pulses through an optical fiber cable), or electrical signals transmitted through wires, should not be construed as essentially transient signals.

The computer-readable program instructions described herein are from computer-readable storage media to individual computing / processing devices, or, for example, the Internet, local area networks, wide area networks, or wireless networks, or a combination thereof. It can be downloaded to an external computer or external storage device via a network such as. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. Each computing / processing device's network adapter card or network interface receives computer-readable program instructions from the network and stores them on computer-readable storage media within individual computing / processing devices. To transfer.

Computer-readable program instructions for performing the operations of the present invention include assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcodes, firmware instructions, state setting data, and integrated circuit equipment. For configuration data, or for one or more programming languages, including object-oriented programming languages such as Smalltalk, C ++, or the like, and procedural programming languages such as the "C" programming language or similar programming languages. It may be source code or object code written in any combination. Computer-readable program instructions, as Stand-Aron software packages, can be executed entirely on the user's computer, partially on the user's computer, and partially on the user's computer and partially remote. It can be run on a computer or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection is (eg, eg). It may be done to an external computer (through the internet using an internet service provider). In some embodiments, electronic circuit devices, including, for example, programmable logic circuit devices, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are used to perform aspects of the invention. , Computer-readable program instructions can be executed by individualizing electronic circuit devices by using the state information of computer-readable program instructions.

Aspects of the invention are described herein with reference to the methods, devices (systems), and flow charts and / or block diagrams of computer program products according to embodiments of the invention. It will be appreciated that each block of the flow chart and / or block diagram, and the combination of blocks in the flow chart and / or block diagram, can be executed by computer-readable program instructions.

These computer-readable program instructions are for instructions executed by the processor of a computer or other programmable data processor to perform a function / action specified in one or more blocks of a flow diagram and / or block diagram. It may be provided to a general purpose computer, a dedicated computer, or the processor of another programmable data processing device that produces a machine. These computer-readable program instructions that can be directed to a computer, programmable data processor, or other device, or a combination thereof, to function in a particular manner are also provided by the computer-readable storage medium in which the instructions are stored. , Flow diagrams and / or block diagrams, may be stored on a computer-readable storage medium to include a product containing instructions that perform a mode of function / action specified in one or more blocks.

Computer-readable program instructions also indicate that an instruction executed on a computer, other programmable device, or other device performs a function / action specified in one or more blocks of a flow diagram and / or block diagram. Thus, it may be loaded into a computer, other programmable data processor, or other device to perform a series of operation steps on the computer, other programmable device, or other device that produces the computer execution process.

Flow charts and block diagrams in the drawings show the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flow chart or block diagram can represent a module, segment, or portion of an instruction that contains one or more executable instructions to perform a given logical function. In some alternative implementations, the functions described in the blocks may occur in a different order than shown in the figure. For example, two blocks shown in succession may actually be executed substantially in parallel, or the blocks may sometimes be executed in reverse order, depending on the functions involved. Each block of the block diagram and / or flow chart, and a combination of blocks of the block diagram and / or flow chart, performs a specified function or action, or a combination of special-purpose hardware and computer instructions. Also note that it can be run by a special purpose hardware-based system.

Claims (18)

Capturing the reflection of an ultra-low frequency signal, the ultra-low frequency signal is reflected by a contact surface, which is projected onto the simulated surface of the object projected from the aerial interface "MAI" device. In contact with the above-mentioned capture and
Converting the difference between the very low frequency signal and the reflection into a measured value of the flow rate on the contact surface,
In response to the measured value being within the measured value, the temperature change of the mass of the medium is brought about, the simulated surface being projected into the mass of the medium and said. A method comprising the effect of causing a change in temperature, wherein the change in temperature results in a second change in the flow rate of the contact surface. Detecting a decrease in the tactile sensitivity of the contact surface from the measured values,
Determining the location of a first region and the first region of the simulated surface where the contact surface is in contact with the simulated surface, the simulated feature of which is the contact. To receive, the above judgment and
Increasing the resolution of the simulated feature to the first resolution by simulating the simulated feature in a second region, wherein the second region occupies the position. The method of claim 1, wherein the first resolution further comprises increasing the tactile sensitivity to compensate for the reduction. Detecting repeated contact and
Changing the temperature of the first region of the simulated surface where the contact surface is in contact with the simulated surface to an undesired temperature.
By changing the temperature of the second region of the simulated surface to a desired temperature, the first region of the undesired temperature and the second region of the desired temperature are the simulated. To form the temperature contour of the surface, and to change to the desired temperature,
The method of claim 1, further comprising using the temperature contour to induce the movement of the contact surface from the first region to the second region. By connecting the thermal element to the MAI device so as to cause the change in the temperature of the mass of the medium, the thermal element is configured to raise the temperature of the air foam being emitted. The method of claim 1, further comprising connecting, wherein the air foam projects a portion of the simulated surface in which the contact surface is in contact with the simulated surface. .. 4. The aspect of claim 4, further comprising configuring the thermal element within an element of the MAI device, wherein the element projects said portion of the simulated surface. Method. Further modifying the ejection nozzles of the elements of the MAI device to cause the changes in temperature, wherein the ejection nozzles project said portion of the simulated surface. The method according to claim 4, which includes. The method according to claim 1, wherein the flow rate on the contact surface is the velocity of blood flow in the capillaries on the contact surface. The first value of the measured value of the flow rate corresponds to the velocity of the blood flow below the first threshold value, and the first value indicates a decrease in tactile sensitivity due to paralysis of the contact surface. , The method according to claim 7. As part of changing the temperature, the temperature is raised to a first temperature, wherein the first temperature restores the tactile sensitivity of the contact surface by suppressing the paralysis. The method of claim 8, further comprising raising. A second value of the measured value of the flow rate corresponds to the velocity of the blood flow above the second threshold, and the second value indicates a reduction in tactile sensitivity due to inflammation of the contact surface. The method according to claim 7. As part of changing the temperature, the temperature is lowered to a second temperature, wherein the second temperature restores the tactile sensitivity of the contact surface by suppressing the inflammation. The method of claim 10, further comprising lowering. By connecting the MAI device and the very low frequency transmitter, the very low frequency transmitter directs the very low frequency signal to the contact surface, and connecting the very low frequency transmitter.
By connecting the MAI device and the very low frequency receiver, the very low frequency receiver connects the very low frequency receiver which receives the reflection of the very low frequency signal from the contact surface. The method according to claim 1, further comprising the above. The ultra-low frequency transmitter and the very low frequency receiver are configured within the elements of the MAI device, the elements projecting a portion of the simulated surface, and the contact surfaces. 12. The method of claim 12, further comprising contacting the portion of the configuration. Modifying the emission nozzle of an element of the MAI device to transmit the very low frequency signal and receive the reflection, the emission nozzle projecting a portion of the simulated surface. 12. The method of claim 12, wherein the contact surface is in contact with the portion, further comprising the modification. The method of claim 1, wherein the simulated surface of the object is projected using ultrasonic waves. The simulated surface of the object is projected using an air column, the first column has a first controllable pressure and the second column has a second controllable pressure. The method according to claim 1. A computer-enabled program product comprising a computer-readable storage device, wherein the program instructions are stored in the storage device, and the program instructions can be executed to perform the method according to any one of claims 1 to 16. .. A processor, a computer-readable memory, and a computer-readable storage device are provided, program instructions are stored in the storage device via the memory for execution by the processor, and the program instructions are contained in claims 1 to 16. A computer system that can be performed to perform the method described in any of the above.

Images