KR102347098B1 - Apparatus and method for generating haptic data, apparatus and method for providing haptic effect - Google Patents

Abstract

Disclosed are an apparatus and method for generating haptic data, and an apparatus and method for providing a haptic effect. The haptic data generating apparatus includes a controller that generates haptic data for an event occurring during game play based on haptic data setting information, and a communication unit that transmits the generated haptic data to the haptic effect providing apparatus. The haptic effect providing apparatus includes a communication unit receiving haptic data from the haptic data generating device, a haptic effect reproducing unit generating a haptic effect under the control of the controller, and a controller controlling generation of the haptic effect based on the haptic data.

Description

Apparatus and method for generating haptic data, apparatus and method for providing haptic effects

The following embodiments relate to a technology for generating haptic data and a technology for providing a haptic effect.

Game developers use visual and audible effects to deliver immersive gameplay. For example, when a user plays a game on a smartphone, a visual effect for a game event is provided through a screen or an auditory effect for a game event is provided through a speaker or earphone. Also, recently, a vibration effect is sometimes used to increase a sense of immersion in a game event. As a prior art, in the 'manipulation device and game device for a game machine' of Korean Patent Publication No. 1998-0032560, a game including a response member operated to give feedback to a user in the form of vibration, sound, light, or a combination thereof A device for operating a device is disclosed.

An object of the present invention is to provide a haptic effect and/or lighting effect linked to a game program through a wearable device, a game controller, or a mobile device accessory device when a user plays a game.

Another object of the present invention is to provide a haptic effect and/or a lighting effect to multiple users who are playing the same game at the same time.

An apparatus for generating haptic data in association with a game program according to an embodiment includes: a controller generating haptic data for an event occurring during game play based on haptic data setting information; and a communication unit for transmitting the generated haptic data to an apparatus for providing a haptic effect.

The event may include at least one of a game content event occurring on the game content of the game program and a user input event for a user input input for playing the game.

The haptic data setting information may include setting information regarding at least one of an application range of a haptic effect, a haptic pattern, and a haptic intensity.

When one or more game content events selected by the user occur, the communication unit may transmit haptic data corresponding to the generated game content events.

The controller performs fast Fourier transform processing on the audio data corresponding to the event, selects a target frequency band of the audio data based on a result of the fast Fourier transform process, and uses a bandpass filter to Filtering is performed on the selected target frequency band, performing inverse fast Fourier transform processing on audio data of the target frequency band on which the filtering is performed, and responding to the event based on a threshold in the result of performing the inverse fast Fourier transform processing It is possible to determine a haptic pattern to

The controller performs fast Fourier transform processing on the audio data corresponding to the event, extracts a frequency having a largest value as a result of performing fast Fourier transform processing, and based on a haptic intensity corresponding to the extracted frequency The haptic data may be generated.

The haptic data may include: a first element indicating a type of message; a second element indicating the size of the message value; and a third element including message information.

The controller may generate haptic data to be provided to a plurality of users who play a game together.

In the haptic data generating apparatus according to an embodiment, when the haptic data generating apparatus is connected to the haptic effect providing apparatus by wire through a cable, power and the haptic data are transmitted to the haptic effect providing apparatus through the cable It may further include a USB host.

An apparatus for providing a haptic effect according to an embodiment includes: a communication unit configured to receive haptic data from an apparatus for generating haptic data; a haptic effect reproducing unit that generates a haptic effect under the control of a controller; and the controller for controlling generation of the haptic effect based on the received haptic data, which is associated with an event occurring during game play, and may be generated based on haptic data setting information.

The controller may generate haptic pattern data by using a haptic pattern index and information about a haptic intensity included in the haptic data, and control generation of the haptic effect based on the haptic pattern data.

The controller may generate the haptic pattern data based on a haptic pattern duration corresponding to the haptic pattern index.

The apparatus for providing a haptic effect according to an embodiment may further include a lighting effect reproducing unit that generates a lighting effect under the control of the controller, and the controller may control generation of the lighting effect based on the haptic data.

The controller may extract information about a haptic intensity from the haptic data, and determine at least one of an illumination color and an illumination intensity based on the information about the haptic intensity.

According to an embodiment, a method for generating haptic data in association with a game program includes: generating haptic data for an event occurring during game play based on haptic data setting information; and transmitting the generated haptic data to a haptic effect providing device.

A haptic effect providing method according to an embodiment includes: receiving haptic data associated with an event occurring during game play from a haptic data generating device; generating haptic pattern data based on the haptic data; and generating a haptic effect based on the haptic pattern data, wherein the haptic data may be generated based on haptic data setting information.

According to an embodiment, when a user plays a game, it is possible to provide a haptic effect and/or a lighting effect linked to a game program through a wearable device, a game controller, or a mobile device accessory device.

According to one embodiment, it becomes possible to provide a haptic effect and/or a lighting effect to multiple users playing the same game at the same time.

1 is a diagram illustrating a system for providing a haptic effect according to an embodiment.
2 is a diagram illustrating a system for providing a haptic effect according to another embodiment.
3 is a flowchart illustrating an example of setting haptic data according to an embodiment.
4 is a flowchart illustrating an example of determining a haptic pattern based on audio data of a game program according to an exemplary embodiment.
5 is a diagram illustrating elements of haptic data according to an embodiment.
6 is a flowchart illustrating an operation of a method for generating haptic data according to an embodiment.
7 is a flowchart illustrating an operation of a method for providing a haptic effect according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

The terms used in the examples are used for the purpose of description only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or a combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

1 is a diagram illustrating a system for providing a haptic effect according to an embodiment.

The haptic effect providing system 100 provides a haptic effect and/or a lighting effect to the user when the user plays the game program in conjunction with the game program. Through this, the user can feel a higher level of immersion and fun. For example, the haptic effect providing system 100 may provide a haptic effect to the user according to the timing at which the game character shoots a gun or strikes the game character in game play, and through this, the user actually shoots or You may feel like you are being hit. As another example, the haptic effect providing system 100 analyzes audio data output from the game to generate haptic pattern data corresponding to a sound pattern of the audio data, and provides a haptic effect to the user based on the generated haptic pattern data. can

Referring to FIG. 1 , a haptic effect providing system 100 according to an embodiment includes a haptic data generating device 110 and a haptic effect providing device 140 . The haptic data generating apparatus 110 is an apparatus for generating haptic data in association with a game program, and may correspond to, for example, a personal computer or a mobile device such as a smart phone. The haptic data generating apparatus 110 may generate haptic data corresponding to the game content by analyzing the game content, and the haptic data may include haptic pattern data defining a pattern of a haptic effect to be provided to the user. The haptic data generating device 110 may be operated by being included in a device in which a game program is driven.

The haptic effect providing device 140 is a device that provides a haptic effect (or haptic stimulus) and/or a lighting effect to a user based on the haptic data received from the haptic data generating device 110 . The haptic effect providing device 140 may also be referred to as a 'haptic device'. The haptic effect providing apparatus 140 may reproduce haptic data to express haptic feedback such as vibration or tapping, and may provide a haptic stimulus of a specific pattern to the user. The haptic effect providing apparatus 140 may have, for example, any one of a wearable device, a mobile device accessory device, a game console controller, a game pad, a keyboard, a mouse, and a joystick, but the shape is limited to these no. The haptic effect providing device 140 encompasses a device linked to a device in which a game program is driven.

In an embodiment, when an event occurs during game play, the haptic data generating device 110 may generate haptic data corresponding to the game content, and transmit the generated haptic data to the haptic effect providing device 140 . . The haptic effect providing device 140 may reproduce the haptic data received from the haptic data generating device 110 to provide a haptic effect to the user, and additionally provide the user with lighting effects and sound effects related to game content. The haptic effect providing apparatus 140 may comprehensively provide a haptic effect for an event, a lighting effect, and a sound effect to the user. A user may be provided with a haptic effect and/or a lighting effect through a wearable device, a keyboard, or a mouse while playing a game through a personal computer or a smart phone. In this case, the user is provided with various patterns of haptic effects linked to game content, not simple patterned haptic effects. According to an embodiment, the haptic effect providing system 100 may provide a haptic effect and/or a lighting effect to multiple users who are playing the same game at the same time.

In one example, when a user plays a game alone or when several users play a game together, it is assumed that the user who plays the game wears a wearable device in the form of a watch or a band. When an event such as a gunshot or kick in the game program occurs, haptic pattern data corresponding to the event is reproduced in the wearable device to generate vibration, and a lighting effect reproducing unit (eg, LED) provided in the wearable device can be driven. Through this, the user's game play immersion and fun can be increased. In addition, since the user can feel the haptic effect while listening to the audio related to the game content, the effect of the game can be maximized. In addition, it is possible to effectively deliver game effects on game contents to users who cannot hear audio, such as hearing impaired, through haptic effects or lighting effects.

As shown in FIG. 1 , the haptic data generating device 110 and the haptic effect providing device 140 may be connected wirelessly through a network 170 , or may be connected by wire through a cable or the like as shown in FIG. 2 . have. The haptic effect providing system 200 shown in FIG. 2 will be described below.

The haptic data generating apparatus 110 includes a controller 125 and a communication unit 130 . According to an embodiment, the haptic data generating apparatus 110 may further include an input interface 115 and an output interface 120 . Input interface 115 receives user input. For example, the input interface 115 may receive user's touch data through a touch screen display or the like, and transmit the received touch data to the controller 125 . The output interface 120 may reproduce audio data related to game content and output audio through a speaker or earphone.

The controller 125 controls the operation of the haptic data generating device 110 and may include one or more processors. According to an embodiment, the controller 125 may control the game program. In this case, the controller 125 controls the start and end of the game program, communication linkage with the haptic effect providing device 140 , event processing, and the like.

The controller 125 generates haptic data for an event that occurs during game play. Here, the event includes at least one of a game content event occurring on the game content of the game program and a user input event for a user input input for game play. The game content event includes, for example, when a game character receives damage or a specific game object appears on the game content. The user input event includes, for example, when a user applies a touch input on a touch screen display for manipulation of a game character, and the like.

The game content event may be an event generated by a user input event. For example, when a game character is manipulated by a user input event, such as a user's touch input, and thereby a change in game content (eg, a specific game object is blown up) occurs, such change in game content is reflected in the game content event. corresponds to According to an embodiment, the game content event may also include an event such as an alarm for a specific situation in the game content, such as an alarm for a dangerous moment of a game character. The haptic effect applied to such various events helps the user to play the game better, and accordingly, the user can further improve the game play ability.

In an embodiment, the controller 125 may generate haptic data based on haptic data setting information. The haptic data setting information may include setting information regarding at least one of an application range of a haptic effect that may be determined by a user input, a haptic pattern, and a haptic strength. The user may directly set the application range of the haptic effect, the haptic pattern, and/or the haptic intensity on the game program, or the haptic data may be set according to setting information specified by the developer of the game program. The user may select one or more game content events to which a haptic effect is applied, and may also set a haptic intensity to be applied to each game content event or user input event. In this case, the controller 125 may generate haptic data based on the haptic intensity set by the user according to an embodiment.

In an embodiment, the controller 125 may generate haptic data based on audio data corresponding to each event. The controller 125 processes the haptic pattern for each period set when the audio data is transmitted. As a more specific example, the controller 125 may perform fast Fourier transform (FFT) processing on the audio data of the game program, and select a target frequency band of the audio data based on a result of performing the FFT processing. . For example, the controller 125 may select, as a target frequency band, a frequency band including a frequency having the largest value on which FFT processing is performed. Thereafter, the controller 125 performs filtering on the target frequency band using a bandpass filter, and performs an inverse fast Fourier transform (IFFT) on the audio data of the target frequency band on which the filtering is performed. processing can be performed. The controller 125 may determine a haptic pattern based on a threshold value from a result of performing IFFT processing. For example, the controller 125 converts the filtered data into the time domain through IFFT processing, extracts data values greater than or equal to a threshold value from the time domain converted data, and then the extracted data value in a specific time interval unit. can be summed up The controller 125 may determine a haptic pattern corresponding to the event by determining that a haptic effect is generated when a result of summing data values satisfies a specific condition. Also, the controller 125 may extract a frequency having the largest value as a result of performing FFT processing on audio data corresponding to an event of a game program, and generate haptic data based on a haptic intensity corresponding to the extracted frequency. . A haptic intensity corresponding to the magnitude of each frequency may be defined in advance.

The communication unit 130 transmits the haptic data to the haptic effect providing device 140 through the network 170 . For example, when receiving haptic data from the controller 125 , the communication unit 130 uses haptic data using Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, Long Term Evolution (LTE), etc. data can be transmitted. For example, when a game content event or a user input event occurs, the communication unit 130 may transmit haptic data corresponding to each event to the haptic effect providing device 140 . Alternatively, when one or more game content events selected by the user in the haptic data setting information occur, the communication unit 130 may transmit haptic data corresponding to the generated game content event to the haptic effect providing device 140 .

The haptic effect providing apparatus 140 includes a communication unit 145 , a controller 150 , and a haptic effect reproduction unit 155 . According to an embodiment, the haptic effect providing apparatus 140 may further include a lighting effect reproducing unit 160 .

The communication unit 145 receives haptic data from the haptic data generating device 110 through the network 170 . The haptic data may be related to an event occurring during game play and generated based on haptic data setting information. The haptic data may include data related to a haptic pattern.

The controller 150 controls the operation of the haptic effect providing device 140 and may include one or more processors. The controller 150 controls generation of a haptic effect based on the received haptic data. The controller 150 may generate haptic pattern data by using the haptic pattern index and haptic intensity information included in the haptic data, and control generation of a haptic effect based on the haptic pattern data. The controller 150 may generate haptic pattern data based on a haptic pattern duration corresponding to the haptic pattern index.

The haptic effect reproducing unit 155 reproduces haptic pattern data under the control of the controller 150 and generates a haptic effect. The haptic effect reproducing unit 155 may include one or more actuators that generate a haptic stimulus.

The lighting effect reproducing unit 160 generates a lighting effect under the control of the controller 150 . For example, the lighting effect reproducing unit 160 may include a lighting device such as an LED. The controller 150 may control generation of the lighting effect based on the haptic data. For example, the controller 150 may extract information about a haptic intensity from haptic data, and determine at least one of an illumination color and an illumination intensity based on the extracted haptic intensity information. For example, the controller 150 may display the red light more brightly as the haptic intensity is stronger, and display the blue light more brightly as the haptic intensity is weaker.

2 is a diagram illustrating a system for providing a haptic effect according to another embodiment.

Referring to FIG. 2 , a haptic effect providing system 200 according to another embodiment includes a haptic data generating device 210 and a haptic effect providing device 240 . Unlike the haptic effect providing system 100 illustrated in FIG. 1 , the haptic data generating apparatus 210 and the haptic effect providing apparatus 240 may be connected to each other by wire through a cable or the like. The haptic data generating apparatus 210 may further include a USB (Universal Serial Bus) host 225 and/or a serial communication unit 230 connected to the communication unit 130 , and the haptic effect providing apparatus 240 includes the communication unit 145 . ) and may further include a connected USB client 255 and/or a serial communication unit 260 . For example, the haptic data generating device 210 and the haptic effect providing device 240 are connected to each other through a connection between the USB host 225 and the USB client 255 or between the serial communication unit 230 and the serial communication unit 260 . can communicate The serial communication unit 230 and the serial communication unit 260 may be connected to each other through a serial interface such as RS (Recommended Standard)-232, for example. The pattern reproducing unit 270 of the haptic effect providing apparatus 240 may extract a haptic pattern index and haptic intensity from the received haptic data, and may generate haptic pattern data using the extracted haptic pattern index and haptic intensity. The pattern reproducing unit 270 may reproduce the haptic pattern by driving an actuator based on the generated haptic pattern data. The pattern reproducing unit 270 may include functions of the haptic effect reproducing unit 155 of FIG. 1 .

When the haptic data generating device 210 is connected to the haptic effect providing device 240 by wire through a cable, the USB host 225 may transmit power and haptic data to the haptic effect providing device 240 through the corresponding cable. have. In this case, the haptic effect providing device 240 may receive power that can be used to drive the actuator together with the haptic data from the haptic data generating device 210 at the same time. In an embodiment, the haptic data generating device 210 may be included in a smart phone and driven, and in this case, the haptic effect providing device 240 may be a haptic patch for a game that can be attached to the smart phone. In this case, the USB host 225 may be, for example, a charging jack interface. When power is supplied to the effect providing device 240 , haptic data may be transmitted.

3 is a flowchart illustrating an example of setting haptic data according to an embodiment.

Referring to FIG. 3 , in one embodiment, the user may set haptic data through a setting interface provided in a game program (eg, a game menu) ( 305 ). For example, the user may set an application range of a haptic effect, a haptic pattern, and a haptic intensity through the setting interface. When the haptic effect is set for each event, the haptic effect may be set for each game content event according to a situation of an internal game program or a user input event according to a user input. In one embodiment, audio data of game content may be present at every event. For example, audio data (eg, sound effect) corresponding to each of a game content event and a user input event may exist.

The user may determine ( 310 ) whether to set the application range of the haptic effect through the setting interface. The application range of the haptic effect indicates the haptic effect processing range for the event. When the application range is not set (ie, not set), the application range of the haptic effect is set ( 315 ) to a range designated by a developer (a game program developer). Alternatively, when the user sets the application range, the application range of the haptic effect is set to the range set by the user ( 320 ). In an embodiment, the application range of the haptic effect may be set as a basic setting on the game program to a range designated by a developer. The user may determine whether to generate a haptic effect for each event through a setting interface on the game program. The game program may generate a haptic effect with an application range determined by the user.

Also, the user may determine ( 325 ) whether to set the haptic pattern through the setting interface. If the user does not set the haptic pattern, the haptic pattern designated by the developer is set ( 330 ). The user may select a haptic pattern for each event, and in this case, a haptic pattern corresponding to the event selected by the user is set ( 335 ). For an event not selected by the user, a haptic pattern provided by the developer may be set.

When the user does not set a haptic pattern for a specific event, a haptic pattern designated by a developer may be determined based on audio data corresponding to the corresponding event. For example, a haptic pattern may be determined through a series of processing processes such as Fast Fourier Transformation (FFT) processing on audio data, target frequency band selection, bandpass filtering processing, and haptic pattern extraction using a threshold value. This will be described in more detail with reference to FIG. 4 .

Also, the user may determine ( 340 ) whether to set the haptic intensity for each event through the setting interface. If the user does not set the haptic intensity, the haptic intensity specified by the developer for each haptic effect is set. The user may select the haptic intensity of the haptic effect for each event, and in this case, the haptic intensity corresponding to the event selected by the user is set ( 350 ). An event not selected by the user may be set to a predefined haptic intensity.

When the user does not set the haptic intensity for a specific event, the haptic data generating apparatus may perform FFT processing on audio data (eg, sound effect) corresponding to the corresponding event ( 345 ). The haptic data generating apparatus may extract ( 355 ) the maximum frequency (eg, the frequency with the greatest dB) from the FFT processing result. The haptic data generating apparatus may determine (360) a haptic intensity corresponding to the event based on the extracted maximum frequency. The haptic data generating apparatus may determine the haptic intensity corresponding to the maximum frequency as the haptic intensity corresponding to the corresponding event. The haptic intensity corresponding to each frequency may be predefined, for example, as shown in Table 1 below.

frequency range Haptic Intensity 1 to 50 Hz One 51 to 150 Hz 2 151 to 200 Hz 3 201 to 300 Hz 4 301 to 400 Hz 5 401 to 500 Hz 6 501 to 800 Hz 7 801 to 1000 Hz 8 1001 to 2000 Hz 9 2001 Hz to 40000 Hz 10

For example, if it is assumed that, as a result of FFT processing of audio data corresponding to event A, the maximum frequency with the largest FFT conversion value is 240 Hz, the haptic intensity corresponding to the corresponding event A is a haptic in a frequency range including a frequency of 240 Hz. It can be determined as the intensity '4'. When the user selects the haptic intensity, taking a fighting game as an example, the haptic intensity may be determined for each event as shown in Table 2 below. The duration of the haptic pattern is determined by the duration specified by the developer for each event.

No event Haptic Intensity Haptic pattern duration (ms) One The fist/kick hit of the game character manipulated by the user is hit (hitting sound) 3 500 2 The last hit/bat hit of the game character's fist combo hits (heavy hit sound) 5 1000 3 Fires the game character's pistol/dagger (sharp metal sound) 4 1000 4 The game character is hit by the enemy (weak hitting sound) 7 500 5 Game character is down / killed by enemy attack
(Large and strong vibration) 3 2000

According to the above process, when at least one of the application range of the haptic effect, the haptic pattern, and the haptic intensity is set, the set content may be stored in the haptic data setting information ( 370 ). After the haptic data setting information is stored ( 370 ), after game play is started, haptic data is generated based on the haptic data setting information. According to the above embodiment, it is possible to provide a haptic effect by processing audio data corresponding to an event without separately setting the haptic effect for each event. It is a flowchart for describing an example of determining a haptic pattern by doing so. Referring to FIG. 4 , in step 410 , the haptic data generating apparatus may perform fast Fourier transform (FFT) processing on audio data of a game program. In an embodiment, in order to perform FFT processing on the audio data, the haptic data generating apparatus may first divide the audio data into an arbitrary number of data streams. For example, the haptic data generating apparatus may divide the audio data into 256 or 1024 data streams in order to smoothly perform FFT processing on the audio data. Here, the number of data columns is not limited to the above example. Audio data may be transformed from the time domain to the frequency domain by FFT processing.

In operation 420 , the haptic data generating apparatus may select a target frequency band of audio data based on a result of performing the FFT processing in operation 410 . In an embodiment, the haptic data generating apparatus may divide the audio data converted into the frequency domain into frequency bands, and select a frequency band including a frequency having the largest value on which FFT processing is performed as the target frequency band. In operation 430, the haptic data generating apparatus may perform filtering on the target frequency band using a bandpass filter.

In operation 440 , the haptic data generating apparatus may perform inverse fast Fourier transform (IFFT) processing on the audio data of the target frequency band on which the filtering in operation 430 is performed. IFFT processing allows data to be transformed from the frequency domain to the time domain.

In operation 450 , the haptic data generating apparatus may perform threshold and summation processing on the result of performing the IFFT processing in operation 440 . The haptic data generating apparatus may extract data values greater than or equal to a threshold value from data converted into the time domain, and then sum the extracted data values in units of a specific time period. In operation 460 , the haptic data generating apparatus may determine a haptic pattern based on the threshold value and the summation result of operation 450 . The haptic data generating apparatus may determine the haptic pattern by determining that the haptic effect is generated when the result of summing the data values determined in step 450 satisfies a specific condition.

As described above, it is possible to provide a haptic effect by processing audio data for each event, without the need to separately set the haptic effect for each event.

5 is a diagram illustrating elements of haptic data according to an embodiment.

According to an embodiment, the haptic data may include information about a haptic pattern index and haptic intensity, and may include various information as shown in Table 3 below.

No element Explanation One Tag Indication of message start, 0x47 2 Length Side ~ Strength Length 3 Side Actuator channel information 4 Pattern Index Haptic Pattern Index 5 Strength Haptic Intensity

Meanwhile, the haptic data may be configured in a message format as shown in FIG. 5 and transmitted to the haptic effect providing apparatus. The embodiment of FIG. 5 shows an example in which haptic data is configured in a BER (Basic Encoding Rules) scheme (LTV notation), which is an encoding rule of Abstract Syntax Notation Number One (ASN.1). Referring to FIG. 5 , haptic data may be transmitted in a message format including elements of a first element, a tag, a second element, a length, and a third element, a value 'tag'. The element of indicates the type of message. In an embodiment, when 0x47 (ASCII code: 'G') is specified in the 'tag', it may be determined that information about the haptic pattern is included in the message. The element of 'length' may indicate the size of the message value in bytes. When the size of the message value exceeds 255 (0xFF) bytes, a value of 0x81 may be added in front according to the BER method. For example, in the case of 260 bytes, the 'length' may be designated as '0x81 0x01 0x04'. The element of 'value' contains information of the message. Information on actual haptic data may be included in the 'value' element. For example, the 'value' element may include information about a haptic pattern index and haptic intensity. FIG. 6 is a flowchart illustrating an operation of a method for generating haptic data according to an embodiment. The haptic data generating method may be performed by the haptic data generating apparatus described herein. Referring to FIG. 6 , a game program is executed in step 610 . The device executing the game program (hereinafter, 'execution device') may detect the presence of the haptic effect providing device in step 620 , and determine whether the haptic effect providing device is connectable in step 630 . If the haptic effect providing device can be connected, the execution device connects with the haptic effect providing device in step 640 . The execution device may be connected to the haptic effect providing device through wireless communication using Bluetooth, BLE, Wi-Fi, LTE, etc. or wired communication through a cable. The execution device may include a haptic data generating device, and according to an embodiment, the haptic effect providing device may be connected to the haptic data generating device.

In operation 640 , the haptic data generating apparatus may generate haptic data for an event occurring during game play based on haptic data setting information. In an embodiment, the haptic data generating apparatus may generate haptic data based on haptic data setting information including setting information regarding at least one of an application range of a haptic effect, a haptic pattern, and a haptic intensity. Such haptic data setting information may be determined by a user input, and the user may select an application range of a haptic effect, a haptic pattern, and a haptic intensity through a setting interface in the game program.

Even if not set by the user, the haptic data generating apparatus performs fast Fourier transform processing on audio data corresponding to each event, selects a target frequency band of audio data based on a result of performing fast Fourier transform processing, and selects a band Filtering is performed on a target frequency band using a pass filter, inverse fast Fourier transform processing is performed on audio data of the target frequency band on which the filtering is performed, and a haptic pattern based on a threshold value from the result of performing inverse fast Fourier transform processing can be decided Also, the haptic data generating apparatus may extract a frequency having the largest value as a result of performing fast Fourier transform processing on the corresponding audio data, and generate haptic data corresponding to the event based on the haptic intensity corresponding to the extracted frequency. . The description of the haptic data generating apparatus generating the haptic data may be applied to the contents described with reference to FIGS. 1 to 5 , and a description of overlapping contents will be omitted.

In operation 650, the haptic data generating apparatus may transmit the haptic data generated in operation 640 to the haptic effect providing apparatus.

7 is a flowchart illustrating an operation of a method for providing a haptic effect according to an embodiment. The haptic effect providing method may be performed by the haptic effect providing apparatus described herein.

Referring to FIG. 7 , in step 710 , the device for providing a haptic effect is connected to a device for generating haptic data (or a device in which a game program is executed). In operation 720, the haptic effect providing device may receive haptic data associated with an event occurring during game play from the haptic data generating device. In operation 730, the haptic effect providing apparatus may generate haptic pattern data by processing the received haptic data. For example, the haptic effect providing apparatus may generate haptic pattern data defining a haptic effect by using a haptic pattern index and information about a haptic intensity included in the haptic data.

In step 740, the haptic effect providing apparatus provides a haptic effect to the user by generating a haptic effect based on the haptic pattern data. The haptic effect providing apparatus may drive an actuator based on the haptic pattern data, and the time at which each haptic pattern is provided may be processed according to a duration for each event corresponding to the haptic intensity. That is, the duration may be determined according to the haptic pattern index (the same as the index for the event).

According to an embodiment, in step 750 , the haptic effect providing apparatus may provide a lighting effect to the user by generating a lighting effect based on the haptic pattern data. The haptic effect providing apparatus may generate a lighting effect based on information about the haptic intensity extracted from the haptic pattern data. For example, the haptic effect providing apparatus may determine the color or brightness of an LED light using the haptic pattern index and haptic intensity information included in the haptic pattern data, and generate a lighting effect while the haptic effect is provided. .

The description of the haptic effect providing apparatus providing the haptic effect and/or the lighting effect may be applied to the contents described with reference to FIGS. 1 and 2 , and a description of overlapping contents will be omitted.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (24)

In the haptic effect providing system linked with a game program,
The haptic effect providing system includes a data generating device and a haptic effect providing device,
The data generating device comprises:
a first controller for generating data about an event that occurs during game play; and
a first communication unit for transmitting the generated data to a haptic effect providing device;
includes,
The haptic effect providing device,
a second communication unit for receiving the data from the data generating device;
a second controller for controlling generation of a haptic effect based on the received data and haptic data setting information; and
a haptic effect reproducing unit generating the haptic effect under the control of the second controller;
including,
At least one of an application range of the haptic effect for the event, a haptic pattern, and a haptic intensity among the haptic data setting information is determined by a user input,
When the haptic data setting information by the user input is not set, the data is generated based on a preset application range, a haptic pattern, and a haptic strength;
The haptic effect providing device is a haptic patch attachable to the data generating device, does not include a means for supplying power by itself, and receives power used to drive an actuator of the haptic effect reproducing unit from the data generating device, A system for providing haptic effects. According to claim 1,
The event is
A haptic effect providing system comprising at least one of a game content event occurring on the game content of the game program and a user input event for a user input input for the game play. 3. The method of claim 2,
The game content event includes an event generated by the user input event, a haptic effect providing system. According to claim 1,
The haptic effect providing device is attached to the data generating device, and directly transmits the haptic effect to the data generating device. According to claim 1,
When the data generating device is connected to the haptic effect providing device by wire through a cable,
A system for providing a haptic effect, which simultaneously receives data and power through the cable. delete According to claim 1,
The data is
a first element indicating a type of message;
a second element indicating the size of the message value; and
A third element including message information
Including, haptic effect providing system. 8. The method of claim 7,
The third element is
A system for providing a haptic effect, comprising information about a haptic pattern index and haptic intensity. 9. The method of claim 8,
The second controller,
A haptic effect providing system that generates haptic pattern data by using the haptic pattern index and information about the haptic intensity included in the data, and controls generation of the haptic effect based on the haptic pattern data. 10. The method of claim 9,
The second controller,
and generating the haptic pattern data based on a haptic pattern duration corresponding to the haptic pattern index. According to claim 1,
The haptic effect providing device further comprises a lighting effect reproducing unit for generating a lighting effect under the control of the second controller,
The second controller controls the generation of the lighting effect based on the data, a haptic effect providing system. 12. The method of claim 11,
The second controller,
A haptic effect providing system that extracts information on haptic intensity from the data, and controls at least one of a lighting color and lighting intensity of the lighting effect displayed by the lighting effect reproducing unit based on the extracted haptic intensity information . delete A method for generating and providing haptic data linked to a game program, the method comprising:
generating haptic data for an event occurring during game play based on haptic data setting information in a data generating device;
transmitting the generated haptic data to a haptic effect providing device;
generating haptic pattern data based on the haptic data in the haptic effect providing device;
generating a haptic effect based on the haptic pattern data;
including,
At least one of an application range of the haptic effect for the event, a haptic pattern, and a haptic intensity among the haptic data setting information is determined by a user input,
When the haptic data setting information by the user input is not set, the haptic data is generated based on a preset application range, a haptic pattern, and a haptic strength;
The haptic effect providing device is a haptic patch attachable to the data generating device, does not include a means for supplying power by itself, and receives power used to drive an actuator from the data generating device, generating and providing haptic data Way. In the device for providing a haptic effect linked with a game program,
a communication unit for receiving data from the data generating device;
a controller for controlling generation of a haptic effect based on the received data and haptic data setting information; and
a haptic effect reproducing unit that generates the haptic effect under the control of the controller;
including,
Among the haptic data setting information, at least one of an application range of the haptic effect for an event occurring during game play, a haptic pattern, and a haptic intensity is determined by a user input,
When the haptic data setting information by the user input is not set, the data is generated based on a preset application range, a haptic pattern, and a haptic strength;
The haptic effect providing device is a haptic patch attachable to the data generating device, and does not include a means for supplying power by itself, and receives power used to drive an actuator from the data generating device. 16. The method of claim 15,
The haptic effect providing device is attached to the data generating device, and the actuator of the haptic effect reproducing unit directly transmits the haptic effect to the data generating device. 16. The method of claim 15,
The device for providing a haptic effect is connected to the data generating device by wire through a cable, and is simultaneously provided with data and power through the cable. delete 16. The method of claim 15,
The haptic effect providing device further comprises a lighting effect reproduction unit for generating a lighting effect under the control of the controller,
The controller controls the generation of the lighting effect based on the data, a haptic effect providing device. 20. The method of claim 19,
The controller is
An apparatus for providing haptic effects by extracting information on haptic intensity from the data, and controlling at least one of a lighting color and lighting intensity of the lighting effect displayed by the lighting effect reproducer based on the extracted haptic intensity information . delete delete delete delete

Images