KR20240036491A - System transfering alternative sensory - Google Patents

Abstract

The present invention relates to an alternative sense delivery system, comprising: a motion detection unit that detects the movement of a dancer; A conversion unit that converts information detected from the motion detection unit into tactile information; and a haptic unit that implements the tactile information converted by the conversion unit into a tactile signal to the recipient. An alternative sense delivery system is provided, including a.

Description

Alternative sense transmission system{SYSTEM TRANSFERING ALTERNATIVE SENSORY}

The present invention relates to an alternative sensory delivery system, and more specifically, to an alternative sensory delivery system that reduces the gap in dance acquisition and appreciation of the hearing and visually impaired compared to the general public by converting audiovisual information into tactile information and delivering it to the user.

In general, it is common to enjoy cultural performances or acquire information through audio and visual as a means of learning in arts and physical education. Therefore, if your audiovisual function is impaired or you have a disability, it may be difficult to enjoy cultural performances or learn the movements necessary for arts and physical education. In most cases, audio commentary is focused on the general audience, so it is difficult to convey it to audiences with hearing and visual impairments. In the past, to overcome this problem, information was delivered through sign language interpretation, etc., but there was no means of receiving information more intuitively, so it was impossible to convey the dancer's movements and the texture of the dance well, for example. In addition, the processing of information through motion capture was limited to the use of post-processing technology and complex software, and there were difficulties in creating a special environment. Moreover, it had an interface that was difficult for people with hearing and visual impairments to use, making it difficult to use.

Republic of Korea Patent Publication No. 10-2014-0141649 (December 10, 2014)

One embodiment of the present invention aims to reduce the gap in dance acquisition and appreciation of the hearing and visually impaired compared to the general public by converting audiovisual information into tactile information and delivering it to users.

One embodiment of the present invention aims to secure a wider creative space than the motion capture method through a strain sensor.

The present invention relates to an alternative sense delivery system, comprising: a motion detection unit that detects the movement of a dancer; A conversion unit that converts information detected from the motion detection unit into tactile information; and a haptic unit that implements the tactile information converted by the conversion unit into a tactile signal to the recipient. An alternative sense delivery system is provided, including a.

Additionally, the motion sensing device may include a strain sensor that is worn on the dancer's body and applied to a plurality of predetermined positions on the dancer's body.

Additionally, the conversion unit may convert the speed and displacement of the motion detected by the strain sensor into one or more of the haptic intensity, duration, and response pattern.

In addition, the haptic unit has a first mode in which the haptic unit transmits a tactile signal when the receiver takes a motion detected by the motion detection unit; and a second mode in which a sound source converted into a tactile signal is delivered to the user.

In addition, it may further include a storage unit that stores the tactile information converted into the dancer's dance pattern by the conversion unit as a non-step.

In addition, it further includes a motion recognition unit that recognizes the dancer's movements, and the genre of dance can be classified and stored in the storage unit through the motion recognition information.

Additionally, the haptic unit can transmit signals to at least the same body part as the motion detection unit.

Additionally, the strain sensor can be electrically connected to a conductive electrode woven with fiber yarn.

In addition, the strain sensor is placed at a position corresponding to the dancer's joint, and the tension of a plurality of strain sensors adjacent to each other at the same point is machine learned to infer the dancer's posture.

Additionally, mubo can be created by the approaching user generating one or more different factors among the sound source, body part, dancer's movement line, size and speed of movement.

Additionally, the motion sensing device may include one or more of a magnetic moment sensor (magnetometer) and an optical sensor that are worn on the dancer's body and applied to a plurality of predetermined positions on the dancer's body.

According to one embodiment of the present invention, it is possible to provide an alternative sensory delivery system that reduces the gap in dance acquisition and appreciation of the hearing and visually impaired compared to the general public by converting audiovisual information into tactile information and delivering it to the user.

According to one embodiment of the present invention, an alternative sensory delivery system that secures a wider creative space than the motion capture method can be provided through a strain sensor.

1 is a schematic diagram showing an alternative sensory delivery system according to an embodiment of the present invention.
Figure 2 is a diagram showing a path through which data is transmitted in an alternative sensory transmission system according to an embodiment of the present invention.
Figure 3 is a diagram showing the configuration of a haptic unit according to an embodiment of the present invention.
Figure 4 is a schematic diagram showing a first mode of accessing the storage unit according to an embodiment of the present invention and uploading a new creative work based on existing uninformation, and Figure 5 is a schematic diagram showing the uninformation stored in the storage unit according to an embodiment of the present invention. This is a schematic diagram showing the second mode of delivering to the receiver along with the sound source.
Figure 6 is an example of a graph showing information sensed from a strain sensor according to an embodiment of the present invention.
Figure 7 is a photograph of an actual configuration in which a motion detection unit is attached to a vest and arm to detect motions such as waves and bounces according to an embodiment of the present invention.
FIG. 8 is a diagram illustrating the step of recognizing the operation of FIG. 7.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, this is only an example and the present invention is not limited thereto.

In describing the present invention, if it is determined that a detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification.

The technical idea of the present invention is determined by the claims, and the following examples are only a means to efficiently explain the technical idea of the present invention to those skilled in the art in the technical field to which the present invention pertains.

1 is a schematic diagram showing an alternative sensory delivery system according to an embodiment of the present invention.

Referring to Figure 1, the alternative sensory transmission system of the present invention includes a motion detection unit 100 that detects the movement of the dancer 10, and a conversion unit 200 that converts the information sensed from the motion detection unit 100 into tactile information. ) and a haptic unit 300 that implements the tactile information converted by the conversion unit 200 as a tactile signal (ST) to the receiver 20.

The motion detection unit 100 may employ one or more of a strain sensor 110, a magnetic moment sensor (magnetometer), and an optical sensor, and a plurality of sensors are located on a specific body of the dancer 10 to measure the distance between sensors or By detecting the amount of change in tension or compression energy, the movement of the dancer 10 can be recorded as data. In one embodiment of the present invention, the description below will focus on the strain sensor 110 applied to the motion detection unit 100. Of course, it is also possible to apply the magnetic moment sensor (magnetometer) and optical sensor that detect the amount of energy or the amount of spaced distance (displacement), and for this purpose, a specific part (for example, joint) of the body of the dancer 10 is possible. It is desirable to be able to arrange it adjacent to .

The motion detection unit 100 is worn on the body of the dancer 10 and may include a strain sensor 110 applied to a plurality of predetermined positions on the body of the dancer 10.

The strain sensor 110 generates tension according to the movement of the dancer 10 and can transmit the tension detected accordingly to the conversion unit 200. Furthermore, it may further include a motion recognition unit 170 capable of camera-based motion recognition (motion capture) of the dancer 10. The motion recognition unit 170 may recognize the movements of the dancer 10 using a motion capture method and transmit the motion to the conversion unit 200 so that the genre of dance can be distinguished.

The movement of the dancer 10 during dance can be detected through the strain sensor 110, and the calculation unit 220 included in the conversion unit 200 can convert the detected tension into tactile information. The converted tactile information can be delivered to the receiver 20. At this time, the haptic unit 300 in contact with the receiver 20 can transmit a signal to at least the same body part as the motion detection unit 100. Preferably, the motion detection unit 100 and the haptic unit 300 have a wearable form and can be maintained in contact with the body by wearing them. Accordingly, the degree of tension detected for each body part through the strain sensor 110 can be transmitted as tactile information to the receiver 20 through the conversion unit 200.

In other words, the tension detected at a predetermined body position causes the haptic executing unit 320 located at the corresponding position to respond, making it possible to immediately transmit tactile information through signal conversion. This eliminates the need for post-processing, such as forming feature points for each body part, performing modeling work on a computer, and then calculating movement, and the calculation unit 220 is used to convert the tension detected by the strain sensor 110 into tactile information. intervenes. Of course, since this is tactile information that is different from the information delivered after modeling work, the present invention has a clear difference in mechanism from the prior art.

Meanwhile, a position sensor 150 that detects location information may be provided so that the dancer 10 can determine the movement line and location of the dancer 10 when the dance is performed within the arbitrary area (TA). The location information of the dancer 10 may also be transmitted to the conversion unit 200 and selectively transmitted to the receiver 20.

Figure 2 is a diagram showing a path through which data is transmitted in an alternative sensory transmission system according to an embodiment of the present invention.

Referring to FIG. 2, it may further include a storage unit 225 that stores the tactile information in which the dance pattern of the dancer 10 is converted by the conversion unit 200 in the form of no information. The information detected by the motion recognition unit 170, the position sensor 150, and the strain sensor 110 of the motion detector may be formed based on at least the information sensed by the strain sensor 110.

As described above, information detected from the motion recognition unit 170, the position sensor 150, and the strain sensor 110 of the motion detector may be transmitted to the conversion unit 200. Specifically, the sensed information may be transmitted to the receiving unit 210 of the converting unit 200. Information received by the receiving unit 210 may be calculated by the calculating unit 220. The calculation unit 220 can generate information and store it in the storage unit 225, while simultaneously converting it into tactile information. The no information can be stored in the storage unit 225 and stored like a database, and the converted tactile information can be transmitted to the haptic unit 300 through the transmitter 230.

The haptic unit 300 can receive the transmitted tactile information through the communication unit 310. The received tactile information can be transmitted to the haptic execution unit 320 and implemented through a predetermined haptic response. The haptic response can be implemented through response maintenance time, response strength, and predetermined patterns, thereby distinguishing information about movement.

Meanwhile, the motion recognition unit 170 can store the motion recognition information in the storage unit 225 as a factor for distinguishing genres of dance, and can be exposed to be sorted by genre through the user's access (A1). You can. This will be explained in detail through Figure 5 below.

The signal transmitted to the communication unit 310 may transmit a tactile signal (ST) to the haptic performance unit 320, and each tactile signal (ST) may include location information. For example, the tactile information processed from the information detected by the strain sensor 110 located on the left shoulder of the dancer 10 in the motion detection unit 100 includes the location information and is sent to the receiver ( 20), the tactile signal (ST) can also be transmitted to the left shoulder.

The communication unit 310 may be electrically connected to transmit received information to a plurality of haptic performance units 320. This will be explained in detail through Figure 3 below.

Figure 3 is a diagram showing the configuration of the haptic unit 300 according to an embodiment of the present invention.

Referring to FIG. 3, the haptic performing unit 320 may be electrically connected to a conductive electrode woven with fiber yarn 302. Specifically, as described above, the haptic portion 300, which is formed to be wearable by the receiver 20, may be made of fabric. It is composed of fiber yarn 302, and electrode yarn 301, which is a thin wire of a conductor to be used as an electrode. It can be woven including. Accordingly, the haptic unit 300 may respond to a signal having matching location information among the tactile signals (ST) received by the communication unit 310. Location information can be like a kind of code. Therefore, it is preferable that the strain lines of the motion detection unit 100 and the haptic performance unit 320 of the haptic unit 300 are arranged in the same number and at the same location.

Even if it is a microcurrent, the fiber 302 may be added to the surface that comes into contact with the human body without being directly exposed. Of course, the fiber yarn 302 formed on the surface where the haptic part 300 is exposed, that is, the surface facing the human body, may be formed by weaving an insulating material into thin wires.

Figure 4 is a schematic diagram showing the first mode of accessing the storage unit 225 (A1) according to an embodiment of the present invention and uploading a new creation based on existing information, and Figure 5 is a schematic diagram showing the first mode of accessing the storage unit 225 (A1) according to an embodiment of the present invention. This is a schematic diagram showing the second mode in which the silent information stored in the storage unit 225 is delivered to the receiver 20 together with the sound source (S).

Referring to FIG. 4, in the first mode, the haptic unit 300 transmits a tactile signal (ST) when the receiver 20 takes a motion detected by the motion detection unit 100. That is, when the tactile signal (ST) sequentially passes through the communication unit 310 and the haptic execution unit 320 and is delivered to the receiver 20 according to the information previously stored through the first upload (U1) to the storage unit 225, the receiver (20) You can learn dance by following the guide without guidance, but you can also make movements that deviate from the guide based on the guidance without guidance. Through these movements, a new dance can be formed that includes the created dance. The new information can be newly stored in the storage unit 225 through the second upload (U2). In this way, the user can freely access (A1) and learn the previously stored information or create it through the user's own actions. Through these activities, the storage unit 225 can accumulate a database.

Meanwhile, referring to FIG. 5, a second mode may be included in which the sound source (S) converted into a tactile signal (ST) is transmitted to the user. In the second mode, you can enjoy the information containing the sound source (S) information and the tactile signal (ST) stored in the storage unit 225. At this time, the calculation unit 220 separately generates tactile information according to the sound source (S) so that the sound source (S) and the previously stored tactile signal (ST) can be felt in a complex manner.

Figure 6 is an example of a graph showing information sensed from the strain sensor 110 according to an embodiment of the present invention.

Referring to FIG. 6, the conversion unit 200 may convert the speed and displacement of the motion detected by the strain sensor 110 into one or more of the haptic intensity, duration, and response pattern. In order to recognize the various movements of the dancer 10, the receiver 20 can arrange the haptic performing unit 320 at the same position as the strain sensor 110 placed on the body part of the dancer 10 as described above. there is. Furthermore, haptic responses may be implemented differently using information based on the degree of tension detected by the strain sensor 110 to detect the degree and speed of movement of a specific body part.

As shown, the vertical axis A may be the angle (displacement) of the joint, the vertical axis D may be the tensile distance (tension), and the horizontal axis T may be time. Through the displacement and tension occurring over time, the calculation unit 220 can analyze which motion occurred at which point and form the same or different tactile signals (ST).

As an example for performing such sensing, the strain sensor 110 may be placed at a position corresponding to the joint of the dancer 10. The tension of a plurality of strain sensors 110 adjacent to each other at the same point is machine learned to infer the posture of the dancer 10. For example, while playing the sound source (S), the information on the rotation of the wrist, elbow, and shoulder at a specific point in time is compared and cumulatively learned through a number of dancers (10), and the movement corresponding to the sound source (S) is learned. , Through this, no information can be formed.

As explained in the second mode, the dance formed in this way allows the user (A1) who accesses the storage unit (225) to use one or more of the sound source (S), body parts, movement line of the dancer (10), size and speed of movement. It can also be created by causing factors.

Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. . Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims described later but also by equivalents to the claims.

Figure 7 is a photograph of an actual configuration in which a motion detection unit is attached to a vest and arm to detect motions such as waves and bounces according to an embodiment of the present invention.

Referring to Figure 7, the arm wave is generated by the motion sensor attached to the arm (top of Figure 7), and the bounce is generated by the motion sensor of the chest vest (middle of Figure 7). And it can be seen that the swing (sum and head, bottom of FIG. 7) motion can be recognized by the movement of the motion detection unit of the arm and vest.

FIG. 8 is a diagram illustrating the step of recognizing the operation of FIG. 7.

Referring to Figure 8, sensor-based motion is first detected. Afterwards, machine learning is performed based on dance pattern recognition and converted into data. Afterwards, a symbolic haptic pattern is created, and the information detected from the motion detection unit is converted into tactile information.

10: Dancer
20: Recipient
100: motion detection unit
110: strain sensor
120: transmission unit
150: Position sensor
170: Motion recognition unit
200: conversion unit
210: receiving unit
220: calculation unit
225: storage unit
230: Transmitter
300: Haptic unit
301: Electrode thread
302: fiber yarn
310: Department of Communications
320: Haptic execution unit
TA: Random area
U1: 1st upload
U2: 2nd upload
A1: Approach
S: sound source
ST: Tactile signal

Claims (12)

A motion detection unit that detects the dancer's motion;
a conversion unit that converts information detected from the motion detection unit into tactile information; and
An alternative sense delivery system comprising a haptic unit that implements the tactile information converted by the conversion unit into a tactile signal to the recipient.
In claim 1,
The motion detection device is
An alternative sense delivery system comprising a strain sensor worn on the dancer's body and applied to a plurality of predetermined positions on the dancer's body.
In claim 2,
The conversion unit,
An alternative sensory delivery system that converts the speed and displacement of motion detected by the strain sensor into one or more of haptic intensity, duration, and response patterns.
In claim 1,
The haptic part is,
A first mode in which the haptic unit transmits the tactile signal when the recipient makes a motion detected by the motion detection unit; And a second mode for delivering the sound source converted into the tactile signal to the user.
In claim 1,
An alternative sense delivery system further comprising a storage unit that stores the tactile information converted into the dancer's dance pattern by the conversion unit as a non-informational method.
In claim 5,
Further comprising a motion recognition unit that recognizes the dancer's motion,
An alternative sense delivery system that classifies and stores genres of dance in the storage unit through the motion recognition information.
In claim 1,
An alternative sense delivery system in which the haptic unit transmits signals to at least the same body part as the motion detection unit.
In claim 2,
An alternative sense delivery system in which the strain sensor is electrically connected to a conductive electrode made of fiber yarn.
In claim 2,
The strain sensor is disposed at a position corresponding to the dancer's joint, and the tensile degree of the plurality of strain sensors adjacent to each other at the same point is machine learned to infer the dancer's posture.
In claim 5,
When an approaching user generates one or more different factors among the sound source, body part, movement line of the dancer, size and speed of movement, an alternative sensory delivery system in which a created dance number can be generated and stored in the storage unit.
In claim 1,
The motion detection unit,
An alternative sense delivery system that is worn on the dancer's body and includes at least one of a magnetic moment sensor (magnetometer) and an optical sensor applied to a plurality of predetermined positions on the dancer's body.
According to clause 11,
The motion detection unit,
An alternative sense delivery system, characterized in that it is attached to a band attached to a vest worn on the dancer's body, arms or legs.