KR102665099B1 - A speaker system equipped with a stretchable display and a driving method - Google Patents

Abstract

The present invention relates to a speaker system equipped with a stretchable display device and a method of driving the same, which includes a touch input unit that recognizes a user's touch and generates touch data, and a content output unit that outputs transformation information and an image on a screen. By manipulating the speaker unit equipped with the stretchable display device using touch or voice, the user can receive famous paintings in three dimensions while listening to music, and when the user exercises through the waveform output three-dimensionally on the stretchable display device. There is an advantage in that you can maintain your exercise pace.

Description

Speaker system equipped with a stretchable display device and a driving method thereof {A speaker system equipped with a stretchable display and a driving method}

The present invention relates to a speaker system equipped with a stretchable display device and a method of driving the same. More specifically, the present invention relates to a speaker system equipped with a stretchable display device and a method of driving the same. More specifically, the present invention relates to a three-dimensional output of content according to music flowing from a speaker using a stretchable display device to visually enjoy music. It relates to a speaker system equipped with a stretchable display device that allows users to experience the same and a method of driving the same.

Recently, research on techniques for performing interaction between humans and terminal devices using various sensory means has become active. These interactions may appear in the form of a user inputting information using voice, keyboard, pen, etc., and outputting the information using voice, graphics, music, and multimedia. Interactions used in conventional virtual experiments, etc. are mostly in the form of watching an output video, selecting an area provided in the video, and playing a video prepared in advance.

Additionally, with the development of display-related technology, flexible displays that can be folded or rolled into a roll shape are being used as displays used for interaction.

A flexible display is a display that is made using a thin, flexible substrate like paper and delivers the same image quality even when bent or folded. OLED (Organic Light Emitting Diode) emits light on its own, so it is thin and does not require a backlight. Additionally, because it is made of organic materials, it is suitable for flexible displays.

The biggest difference between flexible OLED and regular OLED is that the material used for the substrate is polymers such as PI (Polyimide) and polyimide, rather than glass. The reason is that it has good flexibility, can withstand high heat, and is resistant to impact, so it does not break even when bent or bent.

Recently, among flexible displays, stretchable displays, whose screens stretch elastically, are being actively studied as next-generation displays.

A bendable or foldable screen can only be deformed in one direction, but a stretchable display can be deformed in more than two directions and has the elasticity to bounce back even when stretched with force. Due to these characteristics, repeated deformation and restoration to the original state are possible within a given deformation range.

As an example, Patent Document 1 discloses a stretchable substrate 110 including a first electrode 1100, a second electrode 1101, a void substrate 1102, and a lower substrate 1103.

Recently, referring to Figure 1a, the first electrode 1100 and the second electrode 1101 are either a transparent flexible electrode using graphene or a transparent flexible electrode using nanowire. Alternatively, it can be implemented as a transparent flexible electrode in a hybrid form in which graphene is combined.

Accordingly, colors displayed on the OLED display to which the stretchable substrate 100 is combined can be displayed to the outside without interference from the stretchable substrate 100. The stretchable substrate 110 can provide a three-dimensional stretchable display with a protruding shape by adjusting the void spacing and height of the void substrate 1102 according to the shape, pattern, and material desired by the user.

In this regard, the first electrode 1100 may be located on the upper surface of the void substrate 1102. Here, the electrode formed on the first electrode 1100 may be patterned on the void substrate 1102. The void substrate 1102 according to this is a polymer film having a void pattern formed at regular intervals, and may be prepared as a substrate having an elastomer.

The void substrate 1102, which has a void pattern formed at regular intervals, is patterned according to the position of each void formed in the void substrate 1102 to generate a partial surface step, and the height can be controlled according to electrical input. there is. That is, in order to modify part or all of the stretchable display according to the shape, pattern, and material desired by the user, the surface morphology of the void substrate 1102 is changed to the first electrode 1100 and the second electrode 1101. By controlling the intensity of the voltage applied to the device, a shape considering the user's desired shape, pattern, and material can be projected three-dimensionally.

Meanwhile, the second electrode 1101 may be located on the upper surface of the lower substrate 1103. Here, the electrode formed on the second electrode 1101 may be patterned on the lower substrate 1103. Accordingly, the lower substrate 1103 may be prepared as any one of an elastomer substrate, a glass substrate, or a PET (Polyethylene Terephthalate) substrate.

In this regard, one voltage or multiple voltages may be applied to the second electrode 1101. Here, when one voltage is applied to the second electrode 1101, the lower substrate 1103 can be controlled to the same height depending on the intensity of the voltage applied to the void substrate 1102. On the other hand, the second electrode ( When multiple voltages are applied to 1101), each void pattern can be controlled to have a different height.

Next, referring to FIG. 1B, a + electrode is patterned on the first electrode 1100, and a - electrode is patterned on the second electrode 1101, so that a voltage can be applied, respectively. Here, the first electrode 1100 and the second electrode 1101 are patterned with a - electrode. The voltage applied to the two electrodes 1101 can be controlled by a control unit connected to each of the first electrode and the second electrode.

That is, the strength of the voltage applied to the first electrode 1100 and the second electrode 1101 can be controlled using the control unit, and the control unit can control the voltage so that different voltages are applied according to the shape, pattern, and material desired by the user. You can set the intensity. Accordingly, a voltage set by the control unit is applied to the first electrode 1100 and the second electrode 1101 according to the shape, pattern, and material desired by the user, or no voltage is applied so that the stretchable substrate 110 is used by the user. Desired shapes, patterns, and materials can be made to protrude in a three-dimensional form.

Next, when the voltage set by the control unit is applied to the first electrode 1100 and the second electrode 1101, Maxwell stress occurs between the first electrode 1100 and the second electrode 1101. Maxwell stress occurs in the direction of the height of the void, and based on this, the height of the void can be controlled. The Maxwell stress occurring between the first electrode 1100 and the second electrode is a known matter, and detailed description will be omitted.

In other words, when a voltage is applied to the first electrode 1100 and the second electrode 1101 to generate Maxwell stress, that is, electrical attraction, the first electrode 1100 and the second electrode 1101 are adjacent, and each electrode The surface morphology of the void substrate 1102 may be modified by the pressing force. For this purpose, the first electrode 1100 and the second electrode 1101 are prepared as transparent flexible electrodes, and the first electrode 1100 and the second electrode 1101 also change according to the surface morphology deformation of the void substrate 1102. The stretchable substrate 110 deformed and provided with the first electrode 1100, the second electrode 1101, the void substrate 1102, and the lower substrate 1103 may be deformed under the control of the controller.

Accordingly, the control unit sets the intensity of the voltage applied to the first electrode 1100 and the second electrode 1101 in order to output the shape, pattern, and material desired by the user to the display, and applies the set voltage to the first electrode. By applying it to 1100 and the second electrode 1101, part or all of the stretchable substrate 110 can be controlled to three-dimensionally protrude into the shape, pattern, and material desired by the user.

Through this, an attractive force is generated between the first electrode 1100 and the second electrode 1101, and the void height of the void substrate 1102 is changed according to the generated attractive force, thereby forming the first electrode 1100 and the second electrode 1101. ), the void substrate 1102 and the lower substrate 1103 are deformed so that the shape, pattern, and material desired by the user can be projected in a three-dimensional form on the display to which the stretchable substrate 110 is combined.

As such, Patent Document 1 discloses a stretchable substrate 110 that can be transformed into a three-dimensional shape through the above-described configuration.

As another example, Patent Document 2 discloses a stretchable substrate 110 including a first connection electrode 1105, a second connection electrode 1106, and an electroactive polymer substrate 1104.

Recently, referring to FIG. 2A, the stretchable substrate 110 is provided with a first connection electrode 1105 and a second connection electrode 1106 in multiple layers to form an electro active polymer (EAP) substrate 1104. ) It is provided in the form of being joined on both sides.

The electroactive polymer substrate 1104 is a substrate whose thickness is adjusted by being deformed toward the electrode when voltage is applied. When voltage is applied, it is compressed in the direction of the applied electrode to become thinner, and when the voltage is removed, it has its own elasticity. It is a substrate that has the property of being restored to its original shape.

Accordingly, referring to FIG. 2b, the thickness of the electroactive polymer substrate 1104 can be changed according to the control of the control unit connected to the first connection electrode 1105 and the second connection electrode 1106, and the control unit is controlled by the user. The thickness of the electroactive polymer substrate 1104 can be changed by setting the intensity of the voltage applied to the first connection electrode 1105 and the second connection electrode 1106 according to the desired shape, pattern, and material.

Meanwhile, when the stretchable substrate 110 is provided with a first connection electrode 1105, a second connection electrode 1106, and an electroactive polymer substrate 1104, the stretchable substrate 110 is a rectangular electroactive polymer formed with a certain length and width. A plurality of substrates 1104 may be connected to form one substrate on which each of the first connection electrodes 1105 and the second connection electrodes 1106 are stacked.

In addition, the electroactive polymer substrate 1104 is prepared in a rectangular shape with a fine diameter in order to protrude the shape, pattern, and material desired by the user in a three-dimensional form, and the electroactive polymer substrate 1104 provided with a fine diameter is horizontal. And it can be provided in a plurality of connected forms in the vertical direction.

At this time, the control unit sets the voltage strengths of the first connection electrode 1105 and the second connection electrode 1106 of the plurality of electroactive polymer substrates 1104 with fine diameters to different values, and applies the differently set voltages to each of the first connection electrodes 1105 and 1106. By applying it to the first connection electrode 1105 and the second connection electrode 1106, the material desired by the user can be controlled to protrude in a three-dimensional form on the stretchable substrate 110.

Through this, the stretchable substrate 110 can control the electroactive polymer substrate 1104 with a fine diameter under the control of the control unit, so that the shape, pattern, and material desired by the user can be projected in a three-dimensional shape.

As such, Patent Document 2 discloses a stretchable substrate 110 that can be deformed into a three-dimensional shape through the above-described configuration.

Meanwhile, in modern people's lives, stress not only causes various diseases but has also become a factor that greatly affects the quality of life. Office workers suffering from heavy work, students suffering from excessive academic stress, housewives complaining of stress and depression due to difficult housework, elderly people feeling stress due to separation from family and alienation from society, etc. Stress is caused by time, place and It comes regardless of the target. For modern people who are constantly exposed to stress, the question of how to manage this stress has become a very important topic.

Various entertainment activities, such as exercising and watching movies, have been suggested as ways to relieve this stress, but there is still no suitable way to relieve stress for people who do not like extroverted activities such as exercising or watching movies.

Accordingly, Korean Patent Publication No. 10-1987124, 'Artificial intelligence-based meditation assistance system with a work of art displayed in the background' uses a speaker that outputs the voice and sound of meditation content and a display unit that outputs an image of the meditation content. As a result, the art works of current artists were displayed in the background of the meditation content, creating a visual healing effect, but the problem remains that the display part is flat and cannot display abstract three-dimensional elements.

Therefore, there is a need for the development of a speaker system equipped with a stretchable display device that can display the screen three-dimensionally, generate physical buttons according to situations and functions, and display three-dimensional elements that change according to the rhythm, and a method of driving the same. am.

Korean Patent Publication No. 10-2018-0110475 Korean Patent Publication No. 10-2129215

In order to solve the above problem, the present invention equips the speaker with a stretchable display device to display famous paintings and explanations according to the music or waveforms according to the beat and melody, thereby maintaining exercise pace with abstract three-dimensional elements. The purpose is to provide a speaker system equipped with a helpful stretchable display device and a method of driving the same.

In order to solve the above problem, a speaker system equipped with a stretchable display device according to the first embodiment of the present invention has a touch input unit that recognizes the user's touch and generates touch data, and transformation information and images are output on the screen. A speaker system equipped with a stretchable display device including a content output unit, the stretchable display device, a voice input unit for receiving a user's voice, a music output unit for outputting music, and the touch data or the user's voice. Accordingly, it includes a speaker unit including a control unit that controls the content output unit and the music output unit, and a music platform server that is linked with the speaker unit and provides music to the speaker unit, and the modified information is provided on a stretchable display. The device may include a protruding pattern and an image output according to the pattern.

Here, the stretchable display device further includes a haptic sensor unit provided to cause vibration in the speaker unit, and has one or more buttons indicating volume control, play, pause, or music mode using the modified information. output, and the touch input unit may vibrate by the haptic sensor unit when the touch data is input at the position of the button.

In addition, the speaker unit may include a storage unit that stores the modified information and the image, and stores one or more music keywords corresponding to the image and the music; It further includes a proximity detection unit that detects the proximity of the speaker unit and generates proximity data, and a communication unit that requests music or receives music from the music platform server through a wired or wireless network, and the image is configured to output the music. The poem may include a famous painting displayed on the screen and one or more famous painting data formed by a description of the famous painting.

In addition, the control unit may include a voice recognition unit that recognizes one or more words in the user's voice, comprehensively combines the words, and selects the music keyword from the storage unit; Music that transmits the selected music keyword or the music mode to the music platform server through the communication unit, searches the music platform server for the music keyword or the music mode, receives the selected music, and transmits it to the music output unit. selection part; A music conversion unit extracts features according to beat and melody from the selected music and converts them into music data, selects one of the masterpiece data according to the music keyword, and extracts the transformation information in the form of a waveform according to the music data. The modified information in the shape of the volume control, play, pause, and music mode buttons is extracted according to the proximity data or the touch data, and the extracted modified information and the selected masterpiece data are selectively selected to the content output unit. It may include a content control unit that outputs .

When the speaker unit of the speaker system equipped with the stretchable display device according to the second embodiment of the present invention recognizes that the user's hand touches the side using the proximity detection unit, the touch input unit recognizes the user's touch. It further includes a touch blocking unit that blocks, wherein the storage unit stores braille data representing the famous picture data in the form of a convex dot, and the control unit extracts the braille data corresponding to the selected famous picture data or modifies the braille data. It may further include a three-dimensional generating unit that generates the degree to which the stretchable display device protrudes from the information and, when the touch blocking unit operates, transmits the extracted Braille data and the degree of protrusion to the content control unit.

Here, the three-dimensional generator, based on deep learning, applies grayscale to one masterpiece among the masterpiece data selected according to the music, and the stretchable display device is selected from the transformation information according to the brightness level of the masterpiece. A picture conversion unit that extracts the degree of protrusion and transmits the extracted modified information to the content control unit, extracts the braille data that matches the description from the selected masterpiece data, and then extracts the modified information corresponding to the braille data. Thus, it may include a character conversion unit that transmits the extracted modification information to the content control unit.

The speaker unit of the speaker system equipped with the stretchable display device according to the third embodiment of the present invention is installed at the bottom and rotates so that the stretchable display device faces the direction in which the user is located or the position of the user's face. It further includes a position tracking unit, wherein the position tracking unit includes: a photographing unit formed on a lower side of the stretchable display device and collecting image data by photographing the front of the speaker unit; When the image data is input based on deep learning, a facial recognition unit that generates facial data by extracting the position or shape of the user's face from the image data, and the direction in which the user is located using the facial data and the proximity data Alternatively, it may include a rotation driver that rotates the stretchable display device to the position of the user's face.

A method of driving a speaker equipped with a stretchable display device according to an embodiment of the present invention includes a touch input unit that recognizes the user's touch and generates touch data, and a content output unit that outputs transformation information and an image on the screen. A method of driving a speaker equipped with a display device, comprising: receiving touch data or a user's voice as input and selecting a music keyword using the touch data or the user's voice; Music that transmits the music keyword to the music platform server through the communication unit of the speaker unit, searches for music related to the music keyword from the music platform server, receives selected music, and then outputs the selected music from the speaker unit. It may include an output step and a display output step of outputting the modified information and the image to the content output device when music is output to the speaker device.

Here, before the keyword extraction step, a recognition step of detecting the proximity of the speaker unit or the user's voice; When the user's voice is input to the speaker unit, a voice recognition step of interpreting the user's voice and when proximity to the speaker unit is detected, the modified information is applied to one or more music mode buttons to the stretchable display device. It is configured to be output to , and may further include a mode selection step of receiving one of the music mode buttons as input through the touch data.

In addition, in the display output step, when it is detected that the proximity to the speaker unit is the same, the modified information of the volume control button, play button, and pause button is extracted and output from the content output unit to change the volume from the user. In the step of forming a control button to receive input for adjustment, play or pause, and when detecting that the proximity to the speaker unit has increased, the image is output on the screen and the modified information in the form of a waveform is extracted according to the selected music. Thus, it may include a content output step of outputting to the content output unit.

In addition, in the mode selection step or the control button forming step, when the proximity distance to the speaker unit is detected to be less than the first proximity value, the user's touch input is blocked, the modified information is extracted in the form of Braille, and the extracted The modified information is output to the stretchable display device, and when the proximity to the speaker unit is detected to be greater than the first proximity value, input blocking is canceled and the music mode button, volume control button, and play button are activated through the touch data. Alternatively, one of the pause buttons is input, and in the content output step, when the proximity distance to the speaker unit is detected to be less than the first proximity value, grayscale is applied to the masterpiece selected by the speaker unit to determine the brightness level of the masterpiece. Accordingly, the extent to which the stretchable display device protrudes can be adjusted, or the transformation information according to Braille data matching the description of the famous painting can be output to the stretchable display device.

In addition, after the display output step, a photographing step in which a photographing unit formed on a lower side of the stretchable display device photographs the front of the speaker unit to generate image data; When the image data is recognized based on deep learning, a location determination step of analyzing the image data and extracting the position or shape of the user's face to generate facial data and the distance between the facial data and the speaker unit It may further include a rotational driving step of rotating the stretchable display device in the direction of the user's position or the user's face.

A speaker system equipped with a stretchable display device and its driving method according to an embodiment of the present invention outputs music and simultaneously forms a three-dimensional waveform according to the beat and melody of the famous painting and explanation or music on the screen, thereby displaying the famous painting in three dimensions. It can be output and has the effect of allowing the user to maintain their exercise pace when exercising through the waveform.

Figure 1a is a perspective view showing a stretchable display device implemented to enable a three-dimensional shape.
FIG. 1B is an exemplary diagram showing the operating principle of a modification of the stretchable display device of FIG. 1A.
Figure 2a is a cross-sectional view of a stretchable display device implemented to enable a three-dimensional shape.
FIG. 2B is an example diagram illustrating a modified operation of the stretchable display device of FIG. 2A.
Figure 3 is a schematic diagram of a speaker equipped with a stretchable display device according to an embodiment of the present invention.
Figure 4 is a block diagram of a speaker system equipped with a stretchable display device according to the first embodiment of the present invention.
FIG. 5 is a block diagram showing the configuration of the stretchable display device of FIG. 4.
FIG. 6 is an example diagram showing modified information being output in music mode in the stretchable display device of FIG. 5.
Figure 7 is an example diagram showing how transformation information is output to the play, pause, and volume control buttons in the stretchable display device of Figure 5.
FIG. 8 is an example diagram showing transformation information being output in the form of a waveform in accordance with music output from the stretchable display device of FIG. 5.
Figure 9 is an example diagram showing famous picture data being output from the stretchable display device of Figure 5.
FIG. 10 is a block diagram showing the configuration of the control unit of FIG. 4.
Figure 11 is a block diagram of a speaker system equipped with a stretchable display device according to a second embodiment of the present invention.
FIG. 12 is a block diagram showing the configuration of the control unit of FIG. 11.
Figure 13 is a block diagram showing the configuration of the three-dimensional conversion unit of Figure 12.
Figure 14 is a block diagram of a speaker system equipped with a stretchable display device according to a third embodiment of the present invention.
Figure 15 is a block diagram showing the configuration of the location tracking unit of Figure 14.
Figure 16 is a flow chart of a method of driving a speaker equipped with a stretchable display device according to an embodiment of the present invention.
FIG. 17 is a flowchart showing the specific configuration of the display output step of FIG. 16.
FIG. 18 is a flowchart showing the configuration after the display output step of FIG. 16.

Hereinafter, the description of the present invention with reference to the drawings is not limited to specific embodiments, and various changes may be made and various embodiments may be possible. In addition, the content described below should be understood to include all conversions and substitutions included in the spirit and technical scope of the present invention.

In the following description, the terms first, second, etc. are terms used to describe various components, and their meaning is not limited, and is used only for the purpose of distinguishing one component from other components.

Like reference numerals used throughout this specification refer to like elements.

As used herein, singular expressions include plural expressions, unless the context clearly dictates otherwise. In addition, terms such as “comprise,” “provide,” or “have” used below are intended to designate the presence of features, numbers, steps, operations, components, parts, or a combination thereof described in the specification. It should be construed and understood as not precluding the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Hereinafter, a speaker system equipped with a stretchable display device according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 18.

Figure 3 is a schematic diagram of a speaker equipped with a stretchable display device according to an embodiment of the present invention.

Referring to FIG. 3, the speaker unit 10 of the speaker system 1 equipped with a stretchable display device according to the first embodiment of the present invention allows the user to select music using his or her hands or through voice recognition. , it can be arranged to listen to music.

In addition, the speaker unit 10 can output a screen from the stretchable display device 11 provided at the top, so that the user can listen to music while looking at the screen. A detailed description will be made in detail below with reference to the drawings.

FIG. 4 is a block diagram of a speaker system equipped with a stretchable display device according to the first embodiment of the present invention, FIG. 5 is a block diagram showing the configuration of the stretchable display device of FIG. 4, and FIG. 6 is FIG. This is an example diagram showing how modified information is output in music mode from the stretchable display device of Figure 5, and Figure 7 shows modified information output from the stretchable display device of Fig. 5 to the play, pause, and volume control buttons. This is an example diagram showing, Figure 8 is an example diagram showing the transformation information being output in the form of a waveform in accordance with the music output from the stretchable display device of Figure 5, and Figure 9 is an example diagram showing the stretchable display device of Figure 5. This is an example diagram showing how masterpiece data is output, and FIG. 10 is a block diagram showing the configuration of the control unit of FIG. 4.

Referring to FIG. 4, a speaker system 1 equipped with a stretchable display device according to the first embodiment of the present invention may include a speaker unit 10 and a music platform server 20.

First, to explain the music platform server 20, the music platform server 20 is linked with the speaker unit 10 and can provide music requested by the speaker unit 10.

The music platform server 20 may include all service platforms that stream music.

Additionally, the music platform server 20 may provide music starting from the top among the searched music upon a request from the speaker unit 10, but is not limited to this.

The speaker unit 10 includes a stretchable display device 11, a voice input unit 12, a music output unit 13, a control unit 14, a storage unit 15, a proximity detection unit 16, and a communication unit 17. may include.

The stretchable display device 11 may be formed in a shape inclined toward the user from the top of the speaker unit 10 to facilitate viewing by the user, but this is an example and may be formed on the speaker unit 10. It may include all shapes and positions, but is not limited thereto.

In addition, the stretchable display device 11 can be deformed in two or more directions by outputting deformation information, and has elasticity that returns even when stretched by applying force, allowing repeated deformation and restoration to the original state within a given deformation range. .

Accordingly, the stretchable display 11 can provide buttons or patterns to the user so that the user can more easily manipulate the speaker unit 10. This will be explained in detail below with reference to the drawings.

Referring to FIG. 5, the stretchable display device 11 may include a touch input unit 111, a content output unit 112, and a haptic sensor unit 113.

Meanwhile, the stretchable display device 11 may have software (application) installed and executed to provide buttons and patterns to the user using a stretchable display, and includes a touch input unit 111 and a content output unit. (112) and the haptic sensor unit 113 may be controlled by software to provide buttons or patterns to the user using a stretchable display device running on the stretchable display device 11.

Additionally, the stretchable display device 11 may be in the form of a server or engine, and may be used as a device, apparatus, terminal, user equipment (UE), or mobile device (MS). It may be called by other terms such as station, wireless device, or handheld device.

Additionally, the stretchable display device 11 can execute or produce various software based on an operating system (OS), that is, a system. The operating system is a system program that allows software to use the hardware of the device, and includes mobile computer operating systems such as Android OS, iOS, Windows Mobile OS, Bada OS, Symbian OS, Blackberry OS, Windows series, Linux series, Unix series, etc. It can include all computer operating systems such as MAC, AIX, and HP-UX.

The touch input unit 111 can recognize the user's touch and generate touch data.

Specifically, the touch input unit 111 can recognize the touch when the user touches the screen and convert the recognized touch into touch data.

Touch data may include the location where the user pressed the screen, the direction in which the screen was dragged, the time the screen was pressed, and the intensity of pressure on the screen.

Thereafter, the touch input unit 111 may be provided so that the user can manipulate the modified information or image output to the content output unit 112.

The content output unit 112 can output modified information and images on the screen, allowing the user to manipulate the stretchable display device 11 or feel the music in three dimensions while listening to music. The content output unit 112 may be formed with a stretchable substrate 110 to output deformation information, and an OLED display may be formed at the bottom of the stretchable substrate 110.

Here, the modified information may include a pattern in which the stretchable display device 11 protrudes and an image output according to the pattern.

The image may include letters, icons, symbols, pictograms, or pictures output to the stretchable display device 11, but this is an example and is output to the stretchable display device 11 according to the protruding pattern. It can include everything, so it is not limited to this.

Specifically, the transformation information can be used when the stretchable display device 11 protrudes in the form of buttons such as music mode, volume control, play or pause, and in the form of a waveform according to the beat or melody of music. This is an example and may include information about all three-dimensional shapes that the user can see or touch while listening to music, but is not limited to this.

Additionally, when the modified information protrudes in the form of a button on the stretchable display device 11, text and images to inform the purpose of each button may be output together.

For example, when the modified information is output in music mode to the stretchable display device 11 as shown in FIG. 6, the modified information may be changed to 'random play', 'exercise music', 'stress relief', 'meditation', or 'dance'. Words or pictures representing the playlist, such as 'time', can be output at the same time.

Thereafter, when the modified information is output using the volume control button as shown in FIG. 7, a number matching the current volume can be simultaneously output on the content output unit 112, and the user can display the protruding stretchable display device 11 ) can be arranged to output different numbers depending on whether you drag or touch it. Additionally, the content output unit 112 can output modification information for the play and pause buttons simultaneously with the volume control button, but is not limited to this.

Referring to FIG. 8, the deformation information may be simultaneously output as a circular image in accordance with the protruding portion of the stretchable display device 11 in a waveform shape. However, this is an example, and the stretchable display device 11 may be It may include any photo or video that is output when protruding, but is not limited to this.

Meanwhile, a video may include one or more masterpiece data.

Referring to FIG. 9, famous picture data may be formed of a famous picture output on the screen when music is output to the speaker unit 10 and an explanation for interpreting the famous picture. Famous painting data may include one famous painting and a description of the famous painting, and may be provided in the form of one data for each famous painting.

The description of a famous painting can be a sentence describing the famous painting, such as the artist, location where it was displayed, country of possession, year of production, size, technique, criticism, etc., and can include all explanations necessary to describe the famous painting. It is not limited to this.

Additionally, the video may include any photo or video that can be input or manipulated according to the user's touch data, but is not limited thereto.

The haptic sensor unit 113 may be provided to cause vibration in the speaker unit 10.

Specifically, the haptic sensor unit 113 may cause vibration in the speaker unit 10 in accordance with the touch data when a touch is input to the stretchable display device 11 through the touch input unit 111.

In addition, the haptic sensor unit 113 can cause vibration to occur at the position touched by the user, causing vibration to occur at the position of the button protruding on the stretchable display device 11, allowing the user to touch the actual button. It has the advantage of allowing you to feel the sensation of pressing.

Meanwhile, the voice input unit 12 can receive the user's voice as input.

Specifically, the voice input unit 12 may be formed inside the speaker unit 10 in the form of a microphone and can convert the user's voice input into an electrical signal. In addition, the voice input unit 12 may be provided on the side of the speaker unit 10 or the lower side of the stretchable display device 11 to better receive the user's voice, but is not limited to this.

In addition, the voice input unit 12 may display a recording function by storing the user's voice converted into an electric signal in the storage unit 15 so that the user can listen to it again later, but is not limited to this.

The music output unit 13 can output music stored in the speaker unit 10 or provided from the music platform server 20. The music output unit 13 can be controlled by the control unit 14 to output music selected by the user's touch or voice.

Additionally, the music output unit 13 can output sound effects or recorded user voices, but is not limited to this. That is, the music output unit 13 can output all sounds stored or provided in the speaker unit 10.

The control unit 14 can recognize the user's touch and voice, and output music or control screen output according to the recognized touch and voice.

The control unit 14 can control the content output unit 112 and the music output unit 13, so that the content output unit 112 and the music output unit 13 can respond to user operations input to the speaker unit 10. Content can be output accordingly. It will be described in more detail below with reference to FIG. 10.

The storage unit 15 can store transformation information and images to be output to the stretchable display device 11.

Additionally, the storage unit 15 may store music keywords corresponding to images, music modes, or sentences recognized by the voice recognition unit 140. Here, the music keyword may be a word representing the genre or atmosphere of music.

Specifically, the storage unit 15 can store famous paintings or descriptions according to music in advance, and can store famous painting data that matches the genre or mood of the music with music keywords set, respectively.

For example, the storage unit 15 may store famous picture data corresponding to the famous picture 'Kiss' with the music keywords 'brunch' and 'jazz' set, but this is an example and can be stored in one famous picture data. It includes all methods for setting multiple music keywords, so it is not limited to this.

In addition, the storage unit 15 can also store sound effects output when the stretchable display device 11 is touched and vibration patterns to be output from the haptic sensor unit 113, and the three-dimensional conversion unit (to be described below) Braille data used in 144) can also be stored, so it is not limited to this.

The proximity detection unit 16 can generate proximity data by detecting the proximity to the speaker unit 10.

Specifically, the proximity detection unit 16 can detect that a user is near the speaker unit 10 and transmit proximity data about the distance between the user and the speaker unit 10 to the control unit 14.

Afterwards, the screen of the stretchable display device 11 can be controlled by the control unit 14 so that the image output is different depending on the proximity data, and this will be explained in more detail in the control unit 14 below. do.

In addition, the proximity detection unit 16 may include a magnetic proximity sensor, an eddy current proximity sensor, a Hall effect proximity sensor, an optical proximity sensor, an ultrasonic proximity sensor, an inductive proximity sensor, and a capacitive proximity sensor, but this is an example. It may include any proximity sensor that can determine whether the user is close to the speaker unit 10 before touching it, but is not limited to this.

The communication unit 17 may be configured to enable the control unit 14 to select music by communicating with the music platform server 20. In other words, the communication unit 17 is connected to the music platform server 20 through a wired or wireless network and can transmit and receive data with the music platform server 20.

Specifically, the communication unit 17 transmits the music keyword selected from the control unit 14 to the music platform server 20, searches for the music keyword in the music platform server 20, and receives one of the searched music. It can be transmitted to the control unit 14. Details regarding music keyword selection will be explained in detail below.

That is, the communication unit 17 may be provided to connect the speaker unit 10 and the music platform server 20 so that the speaker unit 10 can receive music. Additionally, the communication unit 17 can connect the speaker unit 10 with other platforms as well as the music platform server 20, but is not limited to this.

Referring to FIG. 10, the control unit 14 may include a voice recognition unit 140, a music selection unit 141, a music conversion unit 142, and a content control unit 143.

The voice recognition unit 140 can recognize one or more words in the user's voice and select a music keyword by comprehensively combining the words.

For example, if the sentence 'Play jazz music' is input from the voice input unit 12, the voice recognition unit 140 inputs each of the following words, such as 'jazz', 'music', 'play', and 'give'. Words can be recognized separately, and among them, words corresponding to the genre or atmosphere of music, that is, music keywords suitable for jazz, can be selected in the storage unit 15.

Here, a voice recognition device or algorithm generally used in a terminal can be used to recognize the user's voice, so a detailed description will be omitted.

Accordingly, the voice recognition unit 140 can receive one of the jazz genre music from the music selection unit 141 below.

Additionally, the voice recognition unit 140 can transmit the recognized word to the content output unit 112 so that it can be output on the screen. Accordingly, the voice recognition unit 140 provides the user with a state in which the user's voice input is recognized by the speaker unit 10, so that the user can know which word the speaker unit 10 recognizes and outputs music. It can be arranged so that

The music selection unit 141 can recommend music through the voice recognition unit 140 or music keywords selected in the music mode.

Specifically, the music selection unit 141 transmits the selected music keywords to the music platform server 20 through the communication unit 17, and searches the music platform server for the music keywords to select music.

At this time, the music selection unit 141 can select the music that appears at the top of the music searched through the music keyword by the music platform server 20, as described above, and receive the corresponding music. Additionally, the music selection unit 141 may select music according to the user's taste based on records searched by the user so far, but is not limited to this.

Afterwards, the music selection unit 141 can transmit the selected music to the music output unit 13, allowing the user to listen to the selected music. In addition, the music selection unit 141 can select sound effects stored in the storage unit 15 according to the user's touch and transmit them to the music output unit 13, but is not limited to this.

The music conversion unit 142 can extract features according to beat and melody from the selected music and convert them into music data, and can select one of the famous picture data according to the music keyword.

Here, the music data may be information extracted from the transformation information to represent the stretchable display device 11 in the form of a waveform. In other words, music data is information that prepares a pattern in a circular, convex form and can indicate the height of the pattern and the interval at which the circular shape is formed.

Additionally, the music conversion unit 142 may select one of the famous picture data according to the selected music keyword.

Specifically, the music conversion unit 142 can extract famous picture data corresponding to the selected music keyword from the storage unit 15. When a plurality of famous picture data are extracted, the music conversion unit 142 can select the famous picture data that is most similar to the selected music keyword.

The music conversion unit 142 may select famous picture data corresponding to a plurality of music keywords, but this is an example. The content control unit 143 selects one or more famous picture data matching the selected music keywords in the order of most similarity. ) can be transmitted.

That is, the music conversion unit 142 can select the image to be output to the stretchable display device 11 when music is output, and the content control unit 143 can selectively select the image to be output to the stretchable display device 11. It can be arranged to be output in (11).

The content control unit 143 can extract transformation information using volume control, play, pause, and music mode buttons according to proximity data or touch data.

Specifically, when the content control unit 143 detects that the user is close enough to operate the speaker unit 10 according to the proximity data, the stretchable display device 11 according to the extracted deformation information. You can control the music mode button to be output.

Thereafter, when the content control unit 143 detects that the user has selected one of the music mode buttons through touch data, the play, pause, and volume control buttons are output to the stretchable display device 11 according to the transformation information. The content output unit 112 can be controlled.

The content control unit 143 may control the buttons for each music mode to be formed into buttons of the same size and with different letters or images, or to form the volume control button to be the largest, but this is an example and is a stretchable display device. In (11), the transformation information can be controlled to selectively output the size, text, or image of the button, so it is not limited to this.

Additionally, the content control unit 143 can output famous picture data selected by the music conversion unit 142 on the screen when music is output. The content control unit 143 can change the size and position of the famous painting and output it when the user zooms in with two fingers.

Additionally, if the description of a famous painting is long, the content control unit 143 may receive touch data from the touch input unit 111 and allow the screen to advance to the next page, but is not limited to this.

Afterwards, the content control unit 143 can control the stretchable display device 11 to output modified information in the form of a waveform according to the music data.

For example, if the content control unit 143 determines that the user is not reading the famous picture data based on the user's touch or the time at which the famous picture data is output, it removes the famous picture from the screen and the content output unit 111 provides modified information in the form of a waveform. can be controlled to output.

The content control unit 143 can control the content output unit 112 so that the waveform shape transformation information clearly repeats its rise and fall in accordance with the beat included in the music data.

In other words, when the content control unit 143 moves away to a distance where it is determined that the famous picture data cannot be read according to the proximity data, the content output unit does not output the famous picture data but outputs information on the waveform shape according to the music data. (112) can be controlled.

That is, the content control unit 143 can control the content output unit 112 so that modified information and screens are output differently on the stretchable display device 11 according to the user's approach or request.

Accordingly, the content control unit 143 can allow the user to see a regular waveform when listening to music or exercising, which has the effect of helping to maintain a stable pace of exercise. .

Figure 11 is a block diagram of a speaker system equipped with a stretchable display device according to a second embodiment of the present invention. FIG. 12 is a block diagram showing the configuration of the control unit of FIG. 11. Figure 13 is a block diagram showing the configuration of the three-dimensional conversion unit of Figure 12.

11 to 13, the speaker unit 10 of the speaker system 1 equipped with the stretchable display device according to the second embodiment of the present invention includes a touch blocking unit 18 and a three-dimensional conversion unit 144. ) may further be included.

Here, except for the touch blocking unit 18 and the three-dimensional conversion unit 144, the speaker system 1 equipped with the stretchable display device according to the second embodiment of the present invention is the same as the first embodiment of the present invention described above. It may be substantially the same as the speaker system 1 equipped with the stretchable display device according to the embodiment.

Therefore, only the touch blocking unit 18 and the three-dimensional conversion unit 144 will be described.

Referring to FIG. 11, the speaker unit 10 may further include a touch blocking unit 18.

When the touch blocking unit 18 recognizes that the user's hand touches the side using the proximity detection unit 16, it can prevent the user's touch from being recognized at the touch input unit 111.

Specifically, the touch blocking unit 18 may receive information from the proximity detection unit 16 that recognizes that the user's hand has touched the side of the speaker unit 10.

The touch blocking unit 18 can determine that the user is not manipulating the screen according to the recognized information, and can block the touch input unit 111 so that it does not recognize the user's touch and ignores it.

Accordingly, the touch blocking unit 18 can block the touch so that the screen is not changed even if the user points with his hand when reading the screen output on the stretchable display device 11.

At this time, the storage unit 15 can store braille data that can represent famous picture data in the form of convex dots.

Here, the Braille data may be a form in which the description of the masterpiece data is translated into Braille. In addition, the Braille data may be data in which the purpose of the button formed on the speaker unit 10 is translated into Braille through transformation information, but is not limited thereto.

In other words, the braille data may be data obtained by translating all letters output to the content output unit 112 into braille.

Referring to FIG. 12, the control unit 14 may further include a three-dimensional conversion unit 144.

The three-dimensional conversion unit 144 can extract explanations from the famous picture data selected by the music conversion unit 142 from Braille data. The three-dimensional conversion unit 144 can extract the degree to which the stretchable display device 11 protrudes in order to form a famous painting in three dimensions from the famous painting data.

In addition, the three-dimensional conversion unit 144 controls the content control unit to measure the extracted Braille data and the degree of protrusion when the touch blocking unit 18 operates so that people with poor eyesight, such as the visually impaired, can read while holding the speaker unit 10. It can be delivered to (143).

More specifically, the three-dimensional conversion unit 144 may include a picture conversion unit 1441 and a character conversion unit 1442, as shown in FIG. 13.

The picture conversion unit 1441 can apply grayscale to the masterpieces selected in the music conversion unit 142 based on deep learning, and extract the degree of protrusion according to the brightness of the masterpieces.

Specifically, the picture conversion unit 1441 applies grayscale to a famous painting, and can convert the pixel value of the famous painting to represent one of 0 to 255. The picture conversion unit 1441 may extract the degree to which the stretchable display device 11 protrudes from the transformation information according to each pixel value.

For example, if the pixel value of the famous painting is 0, the picture conversion unit 1441 extracts the extent to which the stretchable display device 11 protrudes to the lowest level, and if the pixel value of the famous painting is 255, the stretchable display It can be extracted to the extent that the device 11 protrudes at its highest level.

That is, the picture conversion unit 1441 can transmit the degree of protrusion of the famous painting to the content control unit 143 so that it can be displayed three-dimensionally on the stretchable display device 11 through the pixel value of the famous painting set between 0 and 255. .

The character conversion unit 1442 can extract Braille data that matches the description from icon or famous picture data. When the user's hand touches the side of the speaker unit 10, the letter conversion unit 1442 can recognize letters being output on the stretchable display device 11 and extract the recognized letters from the braille data. You can.

Afterwards, the character conversion unit 1442 can extract transformation information corresponding to the extracted Braille data and transmit it to the content control unit 143, and when the touch blocking unit 18 operates, the stretchable display device 11 Transformation information can be output in Braille format.

That is, the three-dimensional conversion unit 144 has the same function as the music conversion unit 142 and can transmit information so that the content control unit 143 can control the pattern output to the stretchable display device 11. Thereafter, the content control unit 143 can control the content output unit 112 so that the pattern displays famous paintings and letters in three dimensions according to the proximity data.

In addition, when the character conversion unit 1442 extracts an amount of transformation information that does not fit on the screen of the stretchable display device 11 at one time, the content control unit 143 detects the touch input blocked by the touch blocking unit 18. When this is temporarily released, the transformation information described below can be controlled to be output to the stretchable display device 11 through a single touch input.

Accordingly, the speaker unit 10 has the advantage of allowing users with poor eyesight, such as the visually impaired, to use the stretchable display device 11 through the touch blocking unit 18 and the three-dimensional conversion unit 144. there is.

Figure 14 is a block diagram of a speaker system equipped with a stretchable display device according to a third embodiment of the present invention, and Figure 15 is a block diagram showing the configuration of the position tracking unit of Figure 14.

Referring to FIGS. 14 and 15, the speaker unit 10 of the speaker system 1 equipped with the stretchable display device according to the third embodiment of the present invention may further include a position tracking unit 19. .

Here, except for the position tracking unit 19, the speaker system 1 equipped with the stretchable display device according to the third embodiment of the present invention is the stretchable display device according to the first embodiment of the present invention described above. It may be substantially the same as the speaker system 1 equipped with a display device.

Therefore, only the location tracking unit 19 will be described.

The position tracking unit 19 is installed at the bottom of the speaker unit 10 and can rotate the stretchable display device 11 to face the direction in which the user is located or the position of the user's face.

Here, the stretchable display device 11 may be formed in a shape inclined to one side from the top of the speaker unit 10. The front of the speaker unit 10 may refer to the direction in which the screen of the inclined stretchable display device 11 is output.

Referring to FIG. 15, the location tracking unit 19 may include a photographing unit 190, a facial recognition unit 191, and a rotation driving unit 192.

First, the photographing unit 190 is formed on the lower side of the stretchable display device 11 and can collect image data by photographing the front of the speaker unit 10.

Specifically, the photographing unit 190 may be formed at the bottom of the inclined stretchable display device 11 to determine whether the user is present in the direction in which the screen of the stretchable display device 11 is output. Image data can be collected so that

Additionally, the photographing unit 190 may include any camera that can recognize the user's front or side face, such as a digital camera and a thermal imaging camera, but is not limited thereto.

Accordingly, as the photographing unit 190 is provided at the bottom of the speaker unit 10, it captures the front of the speaker unit 10 without interfering with the image provided to the user by the stretchable display device 11. It can be transmitted to the facial recognition unit 191.

The facial recognition unit 191 can extract the location or shape of the user's face through the image data received from the photographing unit 190.

Specifically, when image data is input based on deep learning, the facial recognition unit 191 can generate facial data by extracting the location or shape of the user's face from the image data.

Here, facial data can be formed by the head position of the user's face relative to the user's height and the positions of the user's eyes, nose, and mouth.

Accordingly, the face recognition unit 191 can recognize not only the front but also the side of the user's face, so it can recognize the user's face even if the user places the speaker unit 10 on the side.

The rotation driver 192 can rotate the stretchable display device 11 in the direction where the user is located or the position of the user's face using facial data and proximity data.

Specifically, the rotation drive unit 192 can use proximity data to recognize the direction in which the user is located among the front, rear, left, and right directions and rotate the speaker unit 10 significantly. Additionally, the rotation drive unit 192 can use facial data to finely rotate the speaker unit 10 in the direction where the user's face is located when the user is lying down or sitting.

Accordingly, the position tracking unit 19 has the advantage of making the screen easier for the user to view by automatically turning the screen toward the user's face when the user exercises on the mat or listens to music while lying down.

Next, a method of driving a speaker equipped with a stretchable display device according to an embodiment of the present invention will be described. Additionally, unlike the speaker system equipped with a stretchable display device, the method of driving a speaker equipped with a stretchable display device of the present invention will be described without dividing the embodiments.

Figure 16 is a flowchart of a method of driving a speaker equipped with a stretchable display device according to an embodiment of the present invention, Figure 17 is a flowchart showing the specific configuration of the display output step of Figure 16, and Figure 18 is the display output of Figure 16. This is a flowchart showing the composition after the steps.

Referring to Figure 16, the method of driving a speaker equipped with a stretchable display device according to an embodiment of the present invention includes a recognition step (S10), a proximity or voice judgment step (S11), a voice recognition step (S12), and a mode selection step. (S13), keyword extraction step (S20), music output step (S30), and display output step (S40).

Here, the speaker unit 10 may be equipped with a stretchable display device 11. The stretchable display device 11 may include a touch input unit 111 that recognizes the user's touch and generates touch data, and a content output unit 112 that outputs transformation information and an image on the screen, description of which has been explained in detail above, so it will be omitted.

First, the recognition step (S10) may be a step of detecting the proximity to the speaker unit 10 and the user's voice.

In the recognition step (S10), the voice input unit 12 formed inside the speaker unit 10 converts the voice input into an electrical signal, or the proximity detection unit 16 detects that a user is near the speaker unit 10. This may be a step in generating proximity data.

In the recognition step (S10), it can be determined in the proximity or voice judgment step (S11) whether the user's voice input or the user's location has been detected.

The proximity or voice determination step (S11) may be a step of determining whether the value input to the speaker unit 10 in the recognition step (S10) is the user's voice or proximity data.

If proximity data is not detected in the proximity or voice judgment step (S11), but voice input is detected, the process can proceed to the voice recognition step (S12). In other words, regardless of whether there is proximity data, if a voice input is detected in the proximity or voice judgment step (S11), the process can proceed to the voice recognition step (S12).

The voice recognition step (S12) may be a step of interpreting the user's voice.

Specifically, the voice recognition step (S12) may be a step of recognizing one or more words from the user's voice and comprehensively combining the words.

For example, if the sentence ‘Play jazz music’ is input, in the voice recognition step (S12), the sentence is recognized by dividing each word into ‘jazz’, ‘music’, ‘play’, and ‘give’. It can be.

Additionally, when proximity data is detected but no voice is detected, the proximity or voice judgment step (S11) can be progressed to the mode selection step (S13).

The mode selection step (S13) may be a step in which transformation information is output to one or more music mode buttons on the stretchable display device 11.

Specifically, in the mode selection step (S13), the music mode button is transformed into a word or picture shape representing a playlist, such as 'random play', 'exercise music', 'stress relief', 'meditation', or 'dance time'. Information can be extracted.

The modified information extracted in the mode selection step (S13) can be output to the stretchable display device 11, and one of the music mode buttons can be selected through the user's touch input, and touch data for the selected music mode can be created.

In addition, if neither proximity data nor voice is detected in the proximity or voice judgment step (S11), the user does not proceed to the voice recognition step (S12) or mode selection step (S13) and returns to the recognition step (S10). It can detect proximity or voice input.

Thereafter, the keyword extraction step (S20) may be a step of selecting a music keyword using the recognized user's voice input and touch data.

Specifically, in the keyword extraction step (S20), music keywords for selecting music can be extracted from the storage unit 15 using the user's voice input and touch data. Since the contents of music keywords have been explained above, they will be omitted.

The music output step (S30) may be a step of receiving music and outputting the music using the extracted music keywords.

Specifically, the music keyword extracted in the music output step (S30) is transmitted to the music platform server 20 through the communication unit 17, so that the music keyword can be searched for in the music platform server 20.

Afterwards, in the music output step (S30), music most similar to the music keyword can be selected and provided among the music searched for in the music platform server 20. In the music output step (S30), the provided music can be output from the speaker unit (10).

The display output step (S40) may be a step of outputting transformation information to the stretchable display device 11 to play, pause, or adjust the volume of music when music is output.

Additionally, the display output step (S40) may be a step in which an image is selected according to a music keyword. The image selected in the display output step (S40) can be output to the stretchable display device 11.

Referring to FIG. 17, the display output step (S40) may include a proximity determination step (S41), a control button forming step (S42), and a content output step (S43).

The proximity determination step (S41) may be a step of determining the user's proximity in a situation where music is output from the speaker unit 10.

If it is determined that the distance between the user and the speaker unit 10 is close in the proximity determination step (S41), the process proceeds to the control button forming step (S42) and the transformation information is stored in a stretchable form so that the user can control volume, playback, and pause. It can be output to the display device 11.

In addition, if it is determined that the distance between the user and the speaker unit 10 is long in the proximity determination step (S41), the content output step (S43) can be progressed to provide the user with content output from the stretchable display device (11). there is.

The control button forming step (S42) may be a step of extracting transformation information from the volume control button, play button, and pause button, and outputting the extracted transformation information to the stretchable display device 11.

In the control button formation step (S42), the user can input one of the volume control button, play button, and pause button through touch data to play or pause music and adjust the volume.

Thereafter, in the control button forming step (S42), it may be determined that the output of music has stopped due to the user's manipulation.

Specifically, if it is determined in the control button formation step (S42) that the output of music has stopped after receiving input through the pause or stop button, it can proceed to the recognition step (S10) and receive voice input or touch data from the user again. This is an example, and the music can be played as is by receiving input from the play button after pausing, but is not limited to this.

The content output step (S43) may be a step in which modified information in the form of a wave formed according to an image or selected music selected according to a music keyword is output to the stretchable display device 11 and provided to the user. In other words, the content output step (S43) may be a step of outputting famous paintings, descriptions, and modification information that match music to the stretchable display device 11.

In addition, in the content output step (S43), if it is determined that the user has read the entire explanation according to the user's touch or the time the explanation was output, the transformation information in the form of a wave can be extracted according to the beat or melody of the music, and extracted. The deformed information can be output to the stretchable display device 11.

Additionally, in the mode selection step (S13) and the control button forming step (S42), if the proximity data is detected to be less than the first proximity value, the user's touch input may be blocked.

Here, the first proximity value may be referred to as the distance at which the user's hand touches the side of the speaker unit 10. That is, when the user's hand touches the side of the speaker unit 10 in the mode selection step (S13) and the control button forming step (S42), the user's touch input is not made to the stretchable display device 11 and is blocked. It can be.

Braille data of the icon formed on the button can be extracted in the mode selection step (S13) and the control button formation step (S42). When the user's hand touches the side of the speaker unit 10 in the mode selection step (S13) and the control button forming step (S42), the characters being displayed on the stretchable display device 11 may be recognized. Braille data corresponding to the letters can be extracted.

Thereafter, the stretchable display device 11 may be controlled to output deformation information in the form of Braille through the braille data extracted in the mode selection step (S13) and the control button forming step (S42).

If the proximity data is detected to be higher than the first proximity value, the blocking of the touch input can be released, and the music mode is selected or the music mode is selected by receiving the user's touch input in the mode selection step (S13) and the control button forming step (S42). This can be adjusted

That is, in the mode selection step (S13) and the control button forming step (S42), a Braille forming step (not shown) is added that allows even people with poor eyesight to select a music mode or control music by touching the speaker unit 10. You can.

Additionally, even in the content output step (S43), if the proximity data is detected to be less than the first proximity value, the user's touch input may be blocked.

Thereafter, grayscale can be applied to the famous painting based on deep learning from the famous painting data selected in the content output step (S43), and the stretchable display device 11 protrudes from the transformation information according to the brightness level of the famous painting. The extent to which it is possible can be extracted.

By applying grayscale to the famous painting in the content output step (S43), the pixel value of the famous painting can be converted to represent one of 0 to 255, and the stretchable display device 11 according to each pixel value. The degree of protrusion may vary.

Additionally, in the content output step (S43), Braille data matching the description can be extracted from the famous painting data. In the content output step (S43), transformation information in the form of Braille corresponding to Braille data can be extracted, and the extracted transformation information can be output to the stretchable display device 11.

Accordingly, there is an effect of providing a more three-dimensional music experience through famous picture data output to the user in the content output step (S43).

Referring to FIG. 18, after the content output step (S43), it may include a photographing step (S50), a position determination step (S51), an image data recognition step (S52), and a rotation drive step (S53).

The photographing step (S50) may be a step of photographing the front of the speaker unit 10 by the photographing unit 190 formed on the lower side of the stretchable display device 11.

In the photographing step (S50), it is possible to determine whether a user is present in the direction in which the screen of the stretchable display device 11 is output through the photographing unit 190 formed at the bottom of the inclined stretchable display device 11. Image data can be collected so that

The location determination step (S51) may be a step of extracting the location or shape of the user's face using the collected image data.

Specifically, when image data is input based on deep learning in the position determination step (S51), facial data can be generated by extracting features such as the position or shape of the user's face from the image data.

In addition, in the position determination step (S51), not only the front but also the side of the user's face can be recognized, so the user's face can be recognized even if the user places the speaker unit 10 on the side, so the direction in which the user's face is located can be determined. It can be judged clearly.

Additionally, the direction in which the user is located may be determined using proximity data in the location determination step (S51), but the method is not limited to this.

The image data recognition step (S52) may be a step of recognizing the location of the user's face through the image data analyzed in the position determination step (S51). If the position of the user's face or the direction in which the user is located is not recognized in the image data recognition step (S52), the user may return to the photographing step (S50) and photograph the front of the speaker unit 10 again.

If the position of the user's face is recognized in the image data recognition step (S52), the speaker unit 10 can be rotated according to the position of the user's face in the rotation driving step (S53).

The rotation driving step (S53) may be a step of rotating the speaker unit 10 based on the face data recognized in the image data recognition step (S52) and the user's proximity data to the speaker unit 10.

The rotation driving step (S53) can cause the screen of the stretchable display device 11 to be output in the direction where the user is located, which has the effect of allowing the user to more comfortably enjoy music in three dimensions.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing the technical idea or essential features of the present invention. You will be able to understand it. Therefore, the embodiments described above are illustrative in all respects and are not restrictive.

1: Speaker system with stretchable display device
10: speaker unit
11: Stretchable display device
110: Stretchable substrate
1100: first electrode
1101: second electrode
1102: void substrate
1103: lower substrate
1104: Electroactive polymer substrate
1105: first connection electrode
1106: second connection electrode
111: Touch input unit
112: Content output unit
113: Haptic sensor unit
12: Voice input unit
13: Music output unit
14: control unit
140: Voice recognition unit
141: Music selection section
142: Music conversion unit
143: Content control unit
144: three-dimensional conversion unit
1441: Picture conversion unit
1442: Character conversion unit
15: storage unit
16: Proximity detection unit
17: Department of Communications
18: Touch blocking unit
19: Location tracking unit
190: Filming Department
191: Face recognition unit
192: Rotation drive unit
20: Music platform server

Claims (12)

In a speaker system equipped with a stretchable display device including a touch input unit that recognizes the user's touch and generates touch data, and a content output unit that outputs transformation information and an image on the screen,
A speaker unit including the stretchable display device, a voice input unit that receives the user's voice, a music output unit that outputs music, and a control unit that controls the content output unit and the music output unit according to the touch data or the user's voice. and
It includes a music platform server that is linked to the speaker unit and provides music to the speaker unit,
The transformation information is,
A stretchable display device includes a protruding pattern and an image output according to the pattern,
The speaker unit,
Receiving the touch data or the user's voice as input, and selecting a music keyword using the touch data or the user's voice,
Passing the music keyword to the music platform server, searching for music related to the music keyword from the music platform server, receiving selected music, and outputting the selected music,
When the music is output, the transformation information and the video are output,
The speaker unit,
Before selecting the music keyword, detect the proximity of the speaker unit or the user's voice,
When the user's voice is input, the user's voice is interpreted,
When proximity to the speaker unit is detected, the modified information is configured to output one or more music mode buttons to the stretchable display device, and one of the music mode buttons is input through the touch data,
The speaker unit,
When it is detected that the proximity to the speaker unit remains the same,
Extract the modified information of the volume control button, play button, and pause button and output it from the content output unit to receive input for volume control, play, or pause from the user,
When it is detected that the proximity to the speaker unit has increased,
A speaker system equipped with a stretchable display device, characterized in that it outputs the image on the screen, extracts the modified information in the form of a wave formed according to the selected music, and outputs it to the content output unit.
According to paragraph 1,
The stretchable display device,
It further includes a haptic sensor unit provided to cause vibration in the speaker unit,
Using the modified information, one or more buttons representing volume control, play, pause, or music mode are output,
The touch input unit,
A speaker system equipped with a stretchable display device, wherein vibration occurs by the haptic sensor unit when the touch data is input at the position of the button.
According to paragraph 2,
The speaker unit,
a storage unit that stores the modified information and the video, and stores one or more music keywords corresponding to the video and the music;
A proximity detection unit that detects the proximity to the speaker unit and generates proximity data, and
It further includes a communication unit that requests music or receives music from the music platform server through a wired or wireless network,
The video above is,
A speaker system equipped with a stretchable display device including a famous painting displayed on the screen when the music is output and one or more famous painting data formed by a description of the famous painting
According to paragraph 3,
The control unit,
a voice recognition unit that recognizes one or more words in the user's voice, comprehensively combines the words, and selects the music keyword from the storage unit;
A music selection method that transmits the selected music keyword or the music mode to the music platform server through the communication unit, searches the music platform server for the music keyword or the music mode, provides selected music, and transmits it to the music output unit. wealth;
A music conversion unit that extracts features according to beat and melody from the selected music, converts them into music data, and selects one of the masterpiece data according to the music keyword;
The modification information in the form of a waveform is extracted according to the music data, the modification information in the shape of the volume control, play, pause, and music mode buttons is extracted according to the proximity data or the touch data, and the extracted modification information and a content control unit that selectively outputs the selected masterpiece data to the content output unit. A speaker system equipped with a stretchable display device.
According to clause 4,
The speaker unit,
When it is recognized that the user's hand touches the side using the proximity detection unit, it further includes a touch blocking unit that blocks recognition of the user's touch of the touch input unit,
The storage unit,
Stores Braille data representing the famous painting data in the form of convex dots,
The control unit,
The braille data corresponding to the selected masterpiece data is extracted or the extent to which the stretchable display device protrudes is generated from the transformation information, and when the touch blocking unit operates, the extracted braille data and the protrusion extent are generated. A speaker system equipped with a stretchable display device further comprising a three-dimensional generating unit that transmits to the content control unit.
According to clause 5,
The three-dimensional generating unit,
Based on deep learning, grayscale is applied to one of the masterpiece data selected according to the music, and the degree to which the stretchable display device protrudes is extracted from the transformation information according to the brightness of the masterpiece, A picture conversion unit that transmits the extracted transformation information to the content control unit, and
A stretchable display device including a character conversion unit that extracts the braille data that matches the description from the selected masterpiece data, extracts the transformation information corresponding to the braille data, and transmits the extracted transformation information to the content control unit. speaker system equipped with
According to paragraph 3,
The speaker unit,
Installed at the bottom, the stretchable display device further includes a position tracking unit that rotates to face the direction in which the user is located or the position of the user's face,
The location tracking unit,
a photographing unit formed on the lower side of the stretchable display device to collect image data by photographing the front of the speaker unit;
When the image data is input based on deep learning, a facial recognition unit that generates facial data by extracting the position or shape of the user's face from the image data;
A speaker system equipped with a stretchable display device including a rotation drive unit that rotates the stretchable display device in the direction in which the user is located or the position of the user's face using the facial data and the proximity data.
In a method of driving a speaker equipped with a stretchable display device including a touch input unit that recognizes a user's touch and generates touch data, and a content output unit that outputs transformation information and an image on a screen,
A keyword extraction step of receiving the touch data or the user's voice as input and selecting a music keyword using the touch data or the user's voice;
Music that transmits the music keyword to the music platform server through the communication unit of the speaker unit, searches for music related to the music keyword from the music platform server, receives selected music, and then outputs the selected music from the speaker unit. output stage and
A display output step of outputting the modified information and the image to the content output unit when music is output to the speaker unit,
The method of driving a speaker equipped with the stretchable display device includes:
Before the keyword extraction step,
A recognition step of detecting the proximity of the speaker unit or the user's voice;
When the user's voice is input to the speaker unit, a voice recognition step of interpreting the user's voice and
When proximity to the speaker unit is detected, the modified information is configured to output one or more music mode buttons to the stretchable display device, and a mode selection step of receiving one of the music mode buttons through the touch data. Contains more,
The display output step is,
When it is detected that the proximity to the speaker unit remains the same, the modified information of the volume control button, play button, and pause button is extracted and output from the content output unit, and the user inputs volume control, play, or pause. Receiving control button formation stage and
When detecting that the proximity to the speaker unit has increased, it includes a content output step of outputting the image on the screen, extracting the modified information in the form of a waveform formed according to the selected music, and outputting it to the content output unit. Method of driving a speaker equipped with a stretchable display device
delete delete According to clause 8,
In the mode selection step or the control button forming step,
When the proximity distance to the speaker unit is detected to be less than the first proximity value, the user's touch input is blocked, the modified information is extracted in the form of Braille, and the extracted modified information is output to the stretchable display device, If the proximity to the speaker unit is detected to be greater than the first proximity value, the input blocking is canceled and one of the music mode button, volume control button, play button, or pause button is input through the touch data,
In the content output step,
When the proximity distance to the speaker unit is detected to be less than the first proximity value, grayscale is applied to the masterpiece selected by the speaker unit to adjust the degree to which the stretchable display device protrudes according to the brightness of the masterpiece, or A method of driving a speaker equipped with a stretchable display device, characterized in that the transformation information according to Braille data matching the description of the famous painting is output to the stretchable display device.
According to clause 8,
After the display output step,
A photographing step in which a photographing unit formed on a lower side of the stretchable display device photographs the front of the speaker unit to generate image data;
When the image data is recognized based on deep learning, a location determination step of analyzing the image data and extracting the position or shape of the user's face to generate facial data;
A method of driving a speaker equipped with a stretchable display device further comprising a rotational driving step of rotating the stretchable display device toward the user's position or the user's face through the facial data and the proximity of the speaker unit.