TW202042051A - Multimedia system - Google Patents

Abstract

A multimedia system is provided, which includes a computer cockpit, a vibration module, a host and a processing circuit. The vibration module is disposed on the computer cockpit. The host is utilized for outputting an audio signal. The processing circuit is utilized for obtaining the audio signal from the host and generating a control signal to control the vibration of the vibration module.

Description

Multimedia system

The present invention relates to a multimedia system, in particular to a multimedia system that generates control signals to control somatosensory equipment by capturing and analyzing audio signals or image frame signals.

With the development of technology and the improvement of people's lives, in addition to meeting people's work needs, computer systems have recently been increasingly applied to the game industry. For example, e-sports equipment developed specifically for video games is one of them. In order to increase the fun of video games, various somatosensory devices are often combined with video games to allow interaction between video games and users. For example, according to the plot of the game, vibration or seat rotation is used to feedback the user, so as to increase the fun of the video game.

However, the current somatosensory function is achieved by combining a special control program pre-designed by game software developers with somatosensory equipment, so the game software and hardware must be matched. In other words, when developing games, game developers need to design specific custom software programs to control somatosensory devices to operate in specific game scenarios. For example, take a robot battle game as an example. When designing and developing a game, when the left side of the viewing angle in the game is hit by a missile, the specific program of the game software simultaneously issues the command "vibrate the left vibrator" to the gaming seat connected to the game console Chair, activate the vibrator on the left to produce a somatosensory effect. Therefore, when playing video games, you can only follow the script designed by the game developer when developing the game, and the biggest limitation of the above approach is that the e-sports equipment (such as e-sports seats) can only support a special set of Game software is often not cost-effective for developers of gaming equipment. If the e-sports device does not support the specific custom software program developed by the game software, it will not be able to realize the somatosensory operation, resulting in insufficient scalability. Therefore, how to design e-sports equipment that can be flexibly applied to various motion sensing devices in response to various different application software is one of the main issues of the current e-sports industry.

Therefore, the main purpose of the present invention is to provide a multimedia system for controlling related somatosensory equipment by capturing and analyzing audio signals or image frame signals to generate control signals to solve the above problems.

The present invention provides a multimedia system, including: a computer cockpit; a vibration module arranged in the computer cockpit; a host for outputting an audio signal; and a processing circuit for obtaining the audio signal from the host, and A control signal is generated according to the audio signal to control the vibration of the vibration module.

The present invention also provides a multimedia system, which includes: a computer cockpit; a rotating module for carrying and controlling the rotation of the computer cockpit; a host for outputting an image signal; and a processing circuit from the host Obtain the image frame signal, and generate a control signal according to the image frame signal to control the rotation of the rotating module.

The advantages and spirit of the present invention can be further understood from the following detailed description of the invention and the accompanying drawings.

In the specification and subsequent patent applications, certain words are used to refer to specific elements. Those with ordinary knowledge in the technical field should understand that manufacturers may use different terms to refer to the same components. The scope of this specification and subsequent patent applications does not use differences in names as a way of distinguishing elements, but uses differences in functions as a basis for distinction. The "include" or "include" mentioned in the entire specification and subsequent patent applications are open-ended terms and should be interpreted as "including but not limited to". In addition, the term "coupling" here includes any direct and indirect electrical connection means. Therefore, if it is described that a first device is coupled to a second device, it means that the first device can be directly electrically connected to the second device, or indirectly electrically connected to the second device through other devices or connection means.

Referring to Figure 1, Figure 1 is a schematic diagram of a multimedia system 1 according to an embodiment of the present invention. The multimedia system 1 includes a computer cockpit 10, a host 20, a processing circuit 30, a vibration module 40, a rotation module 50, an audio output device 60, and a display device 70. The computer cockpit 10 can accommodate users to ride in for related operations. The vibration module 40 is installed on the computer cockpit 10. The vibration module 40 may include at least one vibrator. The rotating module 50 can be used to carry the computer cockpit 10 and drive the computer cockpit 10 to rotate. The rotating module 50 may include at least one rotating motor. The host 20 can be a desktop computer, a server computer, a notebook computer, or a mobile communication device, but is not limited to this. The host 20 includes an application program 200 and a storage device 202. The application program 200 can be stored in the storage device 202 and executed by a processor (not shown in the figure) of the host 20. For example, the application program 200 may be game program software, but is not limited to this. The host 20 can output an audio signal and an image frame signal to the processing circuit 30, where the audio signal can be an audio signal in an audio stream processed by the application 200, and the image frame signal can be processed by the application 200 The video signal in the video stream. In addition, the host 20 and the processing circuit 30 are separate devices, which are independent and separate. Alternatively, the processing circuit 30 can also be integrated on the host 20 or the computer cockpit 10.

The processing circuit 30 can be coupled to the host 20 in a wired or wireless manner. The processing circuit 30 can obtain an audio signal from the host 20 and generate a control signal according to the audio signal to control the vibration module 40 to vibrate. At the same time, the processing circuit 30 can transmit the audio signal to the audio output device 60 for playback. The audio output device 60 can be used to play the audio signal output by the host 20. The audio output device 60 can be a headset or a speaker, but is not limited to this. In addition, the audio output device 60 can be connected to the host 20 through wired communication or wireless communication. The host 20 can output audio signals to the processing circuit 30, and then the processing circuit 30 provides the audio signals to the audio output device 60 for playback. The host 20 can also directly output audio signals to the audio output device 60 for playback.

In addition, the processing circuit 30 can obtain an image frame signal from the host 20 and generate a control signal according to the image frame signal to control the rotation module 50 to rotate, so as to drive the computer cockpit 10 to rotate. At the same time, the processing circuit 30 can also transmit the image frame signal to the display device 70 for display. The display device 70 is used to display the image signal output by the host 20. The display device 70 may be a liquid crystal display (Liquid-crystal Display, LCD), an organic light-emitting diode display (Organic Light-Emitting Diode Display, OLED), or a micro-light emitting diode display (micro LED), but not limited to this . In addition, the display device 70 can be connected to the host 20 through wired communication or wireless communication. The host 20 can output the image frame signal to the processing circuit 30, and then the processing circuit 30 provides the image frame signal to the display device 70 for display. The host 20 can also directly output the video frame signal to the display device 70 for playback and display.

Further, the processing circuit 30 can determine whether to control the vibration module 40 to vibrate according to the volume of the audio signal. For example, the audio output device 60 is a multi-channel audio output device, and the audio signal is a multi-channel audio signal, that is, the audio signal includes a plurality of channel audio signals. For example, stereo audio signal, Dolby 5.1, 7.1 channel audio signal, or WXYZ audio signal developed by Ambisonic, but not limited to this. Each channel of audio signal corresponds to one of the plurality of vibrators. In this case, for each channel of audio signal, the processing circuit 30 determines the volume of each channel of audio signal and compares the volume of each channel of audio signal with a threshold to generate a corresponding control signal. For each channel of audio signal, when the volume of each channel of audio signal is greater than the threshold, the processing circuit 30 accordingly generates a corresponding control signal to control the vibrator corresponding to each channel of audio signal to vibrate . In other words, the processing circuit 30 can control each vibrator to vibrate according to the volume of the audio signal of each channel.

For example, the threshold may be a moving average volume value or other volume values of each channel of audio signal. The rolling average volume value of each channel of audio signal can be the average volume of each channel of audio signal in a specific period of time. For example, the processing circuit 30 may use the rolling average calculation method to obtain the volume average value of the previous period of time (for example, 5000 milliseconds (ms) time period) for each channel audio signal to obtain the rolling average volume value. Taking a time segment of 5000 milliseconds as an example, the processing circuit 30 can calculate the average volume of the audio signal of this channel from 5000 milliseconds ago to the current volume, and the calculated average is the rolling average volume value, and use this as The aforementioned threshold. In this case, when the current volume of the channel audio signal is greater than the threshold, it means that the channel audio signal has a glitch that is different from the previous state, and it can be determined that an event has occurred in the direction of this channel ( event), the processing circuit 30 generates a corresponding control signal to control the vibrator corresponding to the audio signal of the channel to vibrate. In this way, it will be able to bring users more real, more realistic and more immediate somatosensory effects.

For example, please refer to Figure 2. Figure 2 is a schematic diagram of an embodiment of the vibration module 40 and the audio output device 60 in Figure 1. As shown in FIG. 2, the computer cockpit 10 includes a base 100 and a seat 102. The seat 102 is arranged on the base 100. Assuming that the user uses the multimedia system 1 of the present invention to execute a real-time online game, the application 200 executed by the host 20 is a game program software. For example, one of the sound effects type audio signals currently processed by the application 200 of the host 20 is a Dolby 5.1 channel audio signal, which includes a left front channel audio signal, a right front channel audio signal, and a center channel audio signal. Channel audio signal, a left-rear channel audio signal, and a right-rear channel audio signal. As shown in Figure 2, the audio output device 60 is a Dolby 5.1 channel speaker module, which includes a left speaker 60_LF, a right speaker 60_RF, a center speaker 60_C, a left surround speaker 60_LB, a Right surround speaker 60_RB. The vibration module 40 includes vibrators 40_LF, 40_RF, 40_C, 40_LB, and 40_RB. As shown in FIG. 2, the speakers of the audio output device 60 and the vibrator of the vibration module 40 are arranged on the seat 102, but not limited to this, and may be arranged on any other part of the computer cockpit 10. The left front channel audio signal corresponds to the left speaker 60_LF and vibrator 40_LF. The right front channel audio signal corresponds to the right speaker 60_RF and the vibrator 40_RF. The center channel audio signal corresponds to the center speaker 60_C and the vibrator 40_C. The left rear channel audio signal corresponds to the left surround speaker 60_LB and vibrator 40_LB. The right rear audio signal corresponds to the left surround speaker 60_RB and vibrator 40_RB.

During operation, the host 20 provides the audio signal of the audio type processed by the application 200 to the processing circuit 30. The processing circuit 30 can provide audio signals to the speakers of the audio output device 60 for playback. For example, the left speaker 60_LF plays the left front channel audio signal. The right speaker 60_RF plays the right front channel audio signal. The left speaker 60_C plays the center channel audio signal. The left surround speaker 60_LB plays the left rear channel audio signal. The left surround speaker 60_RB plays the right rear channel audio signal. At the same time, the processing circuit 30 can determine whether to control the vibration module 40 to vibrate according to the volume of the audio signal. For example, the processing circuit 30 may analyze the volume of each channel audio signal in the audio signal, and then determine whether to control the vibrator corresponding to each channel audio signal to vibrate.

Take the left front channel audio signal as an example, assuming that the rolling average calculation period is 5000 milliseconds. The processing circuit 30 can calculate the average volume of the left front channel audio signal from 5000 milliseconds ago to the current volume. For example, the calculated volume average is 20 decibels, and the threshold is set to 20 decibels. The processing circuit 30 can compare the current signal volume of the left front channel audio signal with the threshold value. Because Dolby 5.1 channel has directivity, if the volume of a certain direction (channel) exceeds 20 decibels, it can be regarded as an event for the direction in the game. When the current signal volume of the left front channel audio signal is greater than 20 decibels, it indicates that the sound of the left front channel has drastically changed. At this time, the processing circuit 30 determines that an event has occurred in the direction of this channel and generates a corresponding Control signal to drive vibrator 40_LF to produce vibration. In this case, the user can not only hear the left front channel audio signal through the left speaker 60_LF, but also feel the shock effect of the vibrator 40_LF. For example, taking a real-time war game as an example, when the front left is hit by a cannonball, a larger explosion sound effect will be generated for the front left (that is, the front left audio signal) compared to other positions, and the processing circuit 30 analyzes all After the sound channel, if this situation is found, it can correspond to this sound effect (at this time, the left speaker 60_LF plays the left front channel audio signal), and the vibrator 40_LF is activated to provide the user's left vibration body feeling. In this case, use The user will feel not only the explosion sound, but the seat will also have a vibrating sensation in that position, making the user feel more like being hit by a cannonball on the left front side with an immersive shock effect. , Will be able to bring users more real, more realistic, more immediate somatosensory effects, and can interact with the game in a more intuitive way.

On the other hand, regarding the control of the vibration frequency and amplitude of the vibrator of the vibration module 40, the processing circuit 30 can also perform related adjustment control based on the current volume and threshold of the audio channel signal. For example, when the current volume of the audio channel signal is greater than a first threshold and less than or equal to the second threshold, the processing circuit 30 generates a corresponding control signal to control the vibrator corresponding to the audio signal of the channel to vibrate. The vibration frequency of the vibrator is the first vibration frequency and/or the vibration amplitude of the vibrator is the first vibration amplitude. When the current volume of the channel audio signal is greater than the first threshold and the second threshold, the processing circuit 30 generates a corresponding control signal to control the vibrator corresponding to the channel audio signal to vibrate, wherein the vibrator is vibrated The frequency is the second vibration frequency and/or the vibration amplitude of the vibrator is the second vibration amplitude, the first vibration frequency is less than the second vibration frequency, and the first vibration amplitude is less than the amplitude is the second vibration amplitude. In this way, the user will be able to feel different vibration effects.

In addition, when there is no corresponding audio output device at the location of the vibrator of the vibration module 40, the corresponding volume value can also be calculated by the channel audio signals of other audio output devices around, and then the calculated volume value can be used Come to compare with the threshold. For example, if a vibrator is set on the right rear channel of the left surround speaker 60_LB and the right surround speaker 60_LB, the volume value of the vibrator can be calculated according to formula (1):

f(x) = Cos(LB) + Cos(RB) (1)

Where f(x) is the volume value of the vibrator, LB is the volume value of the left rear channel audio signal corresponding to the left surround speaker 60_LB, and RB is the volume value of the right rear channel audio signal corresponding to the right surround speaker 60_LB.

In addition, the rotating module 50 can be used to carry the computer cockpit 10 and drive the computer cockpit 10 to rotate. The rotating module 50 includes at least one rotating motor. The rotating module 50 can have multi-axis rotation functions, such as four-axis, six-axis, eight-axis, etc., to drive the computer cockpit 10 to perform up and down, left and right, front and rear, tilt, and pitch angle changes and different motion types. . For example, please refer to FIG. 3, which is a schematic side view of the computer cockpit 10 and the rotating module 50 according to the embodiment of the present invention. As shown in FIG. 3, the computer cockpit 10 includes a base 100 and a seat 102. The seat 102 is disposed on the rotating module 50, and the rotating module 50 is fixed on the base 100. When the rotating module 50 rotates, the seat 102 can be driven to perform corresponding movements.

Further, the processing circuit 30 can generate a control signal according to the image frame signal output by the host 20 to control the rotation module 50 to rotate, so as to drive the computer cockpit 10 to rotate. For example, the processing circuit 30 can determine a horizon of the image frame according to the image frame signal output by the host 20 and compare the difference between the horizon of the image frame and the real horizon, and then generate corresponding control signals to the rotating module 50 accordingly. Accordingly, the rotating module 50 rotates according to the corresponding control signal to drive the computer cockpit 10 to rotate. For example, suppose a user uses the multimedia system 1 of the present invention to execute a real-time online game, and the application program 200 executed by the host 20 is a game program software. The host 20 can provide the image frame signal processed by the application 200 to the processing circuit 30. The processing circuit 30 then transmits the image frame signal to the display device 70 for display to the user for viewing. At the same time, the processing circuit 30 can use the object detection method of Artificial Intelligence (AI) to calculate the image signal output by the host 20 to detect the change of the horizon, for example, the horizon tilts to the left. The representative should turn the seat 102 of the computer cockpit 10 to the left. When the horizon jumps up and down violently, it means that it is bumping, and the seat should make a corresponding shaking up and down, and so on.

For example, the image frame signal at a first time point is as shown in Figure 4, assuming that the real horizon is AH and the horizon of the image frame is FH. The processing circuit 30 can first determine the shape of the "tree" in the image frame by using the AI object recognition algorithm, and then analyze that the "tree" in the image frame is in a normal upright state to determine the horizon FH of the image frame, thereby determining The current perspective should be a horizontal horizon. In other words, the real horizon is AH, which is the same as the horizon of the image frame, and the horizon of the image frame does not appear at an inclination angle. The processing circuit 30 generates a control signal accordingly, so that the rotating module 50 maintains the seat 102 of the computer cockpit 10 in a horizontal position. Then, the video signal at a second time point is as shown in Figure 5. At the second time point, the state of the "tree" in the screen becomes slanted, representing that the horizon FH of the video screen is tilted to the left, such as the horizon of the video screen The angle difference between FH and the real horizon AH is 15 degrees. At this time, the processing circuit 30 will generate a control signal to make the rotation module 50 control the seat 102 of the computer cockpit 10 to tilt 15 degrees to the left to make a corresponding posture change. In this way, the present invention will be able to bring users a more realistic, more realistic, and more immediate somatosensory effect, and can interact with the game in a more intuitive way.

To sum up, in view of the fact that traditional e-sports equipment must rely on game developers to have embedded control signals in the software program to generate somatosensory effects when developing game software. The present invention can generate related control signals to control related somatosensory equipment by capturing and analyzing audio signal or image signal, so as to provide an e-sports seat with vibration or multi-axis rotation function to achieve the somatosensory during game Experience. In other words, the multimedia system of the present invention can be applied to any game software, and is not limited to specific custom software programs used by game developers to control somatosensory devices when developing game designs. The foregoing descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made in accordance with the scope of the patent application of the present invention shall fall within the scope of the present invention.

1: Multimedia system 10: Computer cockpit 100: Pedestal 102: Seat 20: host 200: Application 202: storage device 30: Processing circuit 40: Vibration module 40_C, 40_LB, 40_LF, 40_RB, 40_RF: vibrator 50: Rotating module 60: Audio output device 60_C: Center speaker 60_LB: Left surround speaker 60_LF: Left speaker 60_RB: Right surround speaker 60_RF: Right speaker 70: display device AH: Real Horizon FH: Horizon

Figure 1 is a schematic diagram of a multimedia system according to an embodiment of the present invention. Figure 2 is a schematic diagram of an embodiment of the vibration module and the audio output device in Figure 1. Figure 3 is a schematic side view of a computer cockpit and a rotating module according to an embodiment of the present invention. Figures 4 and 5 are respectively schematic diagrams of the display screens used in real-time online games according to an embodiment of the present invention.

1: Multimedia system

10: Computer cockpit

20: host

200: Application

202: storage device

30: Processing circuit

40: Vibration module

50: Rotating module

60: Audio output device

70: display device

Claims (10)

A multimedia system including: A computer cockpit; A vibration module set in the computer cockpit; A host for outputting an audio signal; and A processing circuit obtains the audio signal from the host, and generates a control signal according to the audio signal to control the vibration of the vibration module. Such as the multimedia system described in item 1 of the scope of patent application, wherein the vibration module includes a plurality of vibrators, the audio signal includes a plurality of channel audio signals, and each channel audio signal corresponds to one of the plurality of vibrators For one of them, the processing circuit determines the volume of the audio signal of each channel and compares the volume of the audio signal of each channel with a threshold value to generate the control signal. For example, the multimedia system described in item 2 of the scope of patent application, wherein when the volume of each channel audio signal is greater than the threshold value, the processing circuit generates the control signal accordingly to control the corresponding to each channel The vibrator of the audio signal vibrates. For the multimedia system described in item 2 of the scope of patent application, the threshold value is a rolling average volume value of each channel of audio signal. For the multimedia system described in item 1 of the scope of patent application, the audio signal is related to an audio signal in an audio stream processed by an application program executed by the host. The multimedia system described in item 1 of the scope of patent application also includes: An audio output device is used to play the audio signal. A multimedia system including: A computer cockpit; A rotating module for carrying and controlling the rotation of the computer cockpit; A host for outputting an image signal; and A processing circuit obtains the image frame signal from the host, and generates a control signal according to the image frame signal to control the rotation of the rotating module. Such as the multimedia system described in item 7 of the scope of patent application, wherein the processing circuit determines a horizon of the image frame signal according to the image frame signal and compares the horizon of the image frame signal with a real horizon to generate the control signal . In the multimedia system described in item 7 of the scope of patent application, the image frame signal is related to the image frame signal in the image stream processed by an application program executed by the host. The multimedia system described in item 7 of the scope of patent application, which also includes: A display device is used to display the image frame signal.

Images