KR100413162B1 - Method and system for displaying panoramic videos using server-client configuration - Google Patents

Abstract

The present invention relates to a method and system for displaying a continuous image on a wide-angle screen by an image projector connected to each client computer according to the sound and configuration of the server by connecting the server and a plurality of client computers by a network. To this end, the server computer extracts the characteristics from the sound source, combines the information about the appropriate image image after the projection angle range, background image, and environment setting for each client, and transmits them to each client. According to the transmission of such detailed information, each client displays an image file, mainly a video file, on the screen through an image projector according to a request of a server from the image file storage of the client. Therefore, each client is responsible for the screen of the projection angle assigned to them to project only the corresponding area of the image file on the screen, and the projection range of the client is completed to display the entire screen as a continuous panoramic image.

Description

Method and system for displaying panoramic videos using server-client configuration}

The present invention relates to a method and a system for implementing a panoramic image using a server and a client. More particularly, when a sound is inputted by a sound input unit of the server, the server extracts characteristics of the sound and changes the image accordingly. It calculates and combines it with the image information and sends it to the client, and the client displays the video image through the image projector in the corresponding area corresponding to the predetermined projection angle range allocated to each networked client by the command of the server. And to provide a system.

In the conventional technology, an image projector (such as a projector) is connected to a computer to project an image by projecting only one surface onto a screen. That is, in the real world, the space is composed of three dimensions and thus can be mapped to a three-dimensional coordinate system. However, due to the structure of the human body, the eye is only visible about 120 degrees in front of the human body.

Therefore, the remaining 240 degrees are not recognizable at all, so objects outside the eye's angle of view can only be seen by turning or moving the eye or body, and existing graphics and images are constructed only within the range of this angle of view.

In order to display 360-degree full image projectors beyond these limitations, a computer must be connected to an image projector constituting different screens, and thus, multiple computers and corresponding image projectors had to be provided.

In this case, it was necessary to manually adjust each computer for a certain amount of different video data, projection angle, vertical angle and horizontal angle. Was difficult.

Therefore, it is not efficient to continuously display the screen 360 degrees in front of the above method even if configured as another system. This is because a large amount of trial and error occurs in the calculation because it is necessary to calculate the exact time in each system or computer and each computer management is difficult.

The present invention devised to solve the above-described problem is to configure all the images or graphics to the area outside the range of view, so that it can be used for purposes such as indoor / outdoor 360-degree front display in response to music / sound and It is an object to provide a system.

It is also an object of the present invention to provide a method and system for controlling each projection angle and each client in a server through a network.

1 is a block diagram showing a configuration of a server according to a user's sound input and environment setting.

2 is a block diagram showing a configuration of a client connected to a server.

3 is a network diagram illustrating a server, a client, and an image projector according to the present invention.

4 is a flowchart illustrating a process of processing a server as a user inputs a sound.

5 is a flowchart illustrating a data transmission process and a data processing process between a server and a client according to the present invention.

6 is a view showing an embodiment in which the present invention is implemented in real space.

In order to achieve the above-described object, the present invention is connected to the server and a plurality of client computers in a network, according to the sound and configuration of the server, a continuous projection on the front screen of the 360-degree image projector connected to each client computer A method and system for displaying an image are provided.

To this end, the server computer extracts the characteristics from the sound source, combines the information about the appropriate image image after the projection angle range, background image, and environment setting for each client, and transmits them to each client. According to the transmission of such detailed information, each client displays an image file, mainly a video file, on the screen through an image projector according to a request of a server from the image file storage of the client.

That is, each client is responsible for only the screen of the projection angle assigned to it among the 360 degrees to project only the corresponding area of the image file on the screen, and the projection range of the client is configured to display the entire screen as a continuous panoramic image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a configuration of a server according to a user's sound input and environment setting.

The server is largely composed of six parts, which will be described in detail. The audio input unit 110, the audio signal processor 120, the user interface 130, the controller 140, the image file storage unit 150, and the network interface 150 are described. It consists of.

The sound input unit 110 is an input unit for directly receiving an output of a microphone, an audio device, a sound card, etc., and serves to transmit the same to the sound processor 120 of the server. Analog signals from microphones and sound equipment include long and short pitches, pitches, pitches, energy, beats, and the like.

The sound signal processing unit 120 of the server converts an analog signal, which is a sound source received from the sound input unit 110, into a digital signal, and extracts long and short, high, pitch, energy, and beat of the sound from the sound source. To transmit to the control unit 140.

Alternatively, the digital signal related to sound may be directly input to the sound input unit 110.

In addition to the sound input unit, the server is provided with a user interface 130. Through the user interface, the user selects and selects a wallpaper suitable for a sound source, and also sets preferences such as display areas of each client. Is transmitted to the controller 140. Of course, a keyboard, mouse, and touch screen can be used without having to define a specific object as a user interface.

Accordingly, the controller 140 processes the digitally changed information through the sound processor 120 and the user interface 130 and transmits the information to the client computer through the network interface 160.

The image file storage unit 150 stores various image files, that is, video files, on the hard disk.

2 is a block diagram showing a configuration of a client connected to a server.

1 illustrates a client computer processing information transmitted by the server system of FIG. 1, which includes four parts, that is, a network interface unit 200, a controller 210, an image file storage unit 220, and an image output. It consists of a portion 230 and the like.

The network interface unit 200 receives control and data information from a server connected with a network cable and transmits it to the control unit 210. The controller 210 exchanges information with the server in real time and accordingly transmits an image file, that is, video information, from the image file storage unit 220 to the image output unit 230.

Here, the image file storage unit 220 stores all image information on the hard disk drive, not image information allocated to each client, and exchanges input / output control information with the controller from time to time. In addition, since the same image file as the server-side image file storage unit 150 must be maintained, the latest image file is received from the server and stored in synchronization.

Lastly, the image output unit 230 is a device for displaying a combination of image information completed by the controller 210, for example, a specific image on a screen for a predetermined time, for example, an image projector such as an LCD projector or a PDP projector. These devices connect to client computers via USB or RS-232-C, and process the data into frames.

3 is a network diagram illustrating a server, a client, and an image projector according to the present invention.

3 is a diagram illustrating a server 100, clients 300a to 300n, and image projectors 350a to 350n of FIG. 1 and FIG. 2, in which a server controls each client through a network. Therefore, the server can exchange information with each client from time to time, so that the number of currently running clients and which clients are disconnected can be controlled centrally.

4 is a flowchart illustrating a process of processing a server as a user inputs a sound.

When the user inputs a sound source using a microphone or an acoustic device (step S400), the sound signal processor of the server receives the signal and performs signal processing (step S410). According to the signal processing, some features of the signal, for example, the long and short, the high and low, the pitch, the energy, the beat, etc. of the sound are extracted and made into sound signal information (step S420).

More details on how to extract the feature from the acoustic signal are described in "Method of Real-Time Visualization of Music" (Patent Application 2000-0013428).

5 is a flowchart illustrating a data transmission process and a data processing process between a server and a client according to the present invention.

First, in order to implement the present invention on a network, all image files used are stored in the image file storage unit 150 of the server computer. The same image file is also stored in the image file storage unit 220 of each client computer.

Here, synchronization can be used to identify image files on the client and server, or users can copy and save image files directly from the server to the client.

In this way, if the user has stored a new image file on the server (step S500), synchronization is performed to transfer the image file to each client (step S501). Accordingly, each client compares the image file of the server with the stored image file (step S502). If there is a difference in the image file (step S503), the client requests the server to transmit the image file (step S504).

In this case, the synchronization may be periodically performed according to a program, or the user may perform the synchronization forcibly.

The server receiving the transfer request transmits the image file to the client (step S505), and the client receiving the image file stores the image file on the hard disk (step S506).

After all the image files are identified in the server and the client computer, the user sets the background image through the user interface 130 (step S510), and sets the environment for the display area of each client computer (step S511). .

In other words, if you have three client computers, A, B, and C, and you want to represent 360 degrees, let A represent 0 to 120 degrees, B to 120 to 240 degrees, and C to 240 to 360 degrees. If it is 60-60 degree, B will allocate a display area so that it may be 60-180 degree, and C may be 180-300 degree. Then, the information of the image file to be performed by each client corresponding thereto and the characteristics of the sound source generated in FIG. 4 are synthesized (step S512).

In other words, when a beat part is played, it means that a client performs a certain time on a specific image file among the image files to generate information about the change of the image when the beat part is played.

The server transmits detailed information about the image to be processed by each client to the client (step S513). Accordingly, each client computer receives the data.

However, during the transmission of the data, the arrival time of the data is delayed in some cases due to the different lengths of physical cables connecting the server and each client. Therefore, when the time difference accumulates, synchronization between the server and the client is lost.

In this case, for example, a periodic synchronization frame of about 10 seconds is inserted so as to ignore previously received data and to execute the synchronization frame firstly (step S521).

That is, the server sends a synchronization frame every predetermined time, and when the client receives the synchronization frame, the server ignores the previously received data and processes the data included in the synchronization frame. For example, if the position of the fish is in the coordinate "a" according to previously received data and the position of the fish is in the coordinate "b" according to the data included in the synchronization frame, the fish is drawn in the "b" coordinate. Will be.

In this way, for example, when the fish on the screen B enters the screen C as shown in FIG. 6, the fish image on the screen B and the fish image on the screen C are unnatural due to the delay in the client 610. The connection can be prevented.

Here, the transmission protocol for transmitting data uses the existing protocols such as UDP (User data protocol) and TCP (Transmission control protocol) so that the server transmits the same data to all clients at once, so that all clients can receive the data. Multicast IP is used.

Each client computer extracts information of an angle of view to be displayed, that is, a horizontal angle and a vertical angle, from the received data (step S530). Then, the background image and the image according to the sound source characteristics are synthesized. In this case, according to the information of the horizontal angle and the vertical angle, images outside the area to be displayed are clipped to reduce the operation (step 531).

Accordingly, the client computer transmits the synthesized image to the image output unit to which the image projector is connected (step 532).

The data rules exchanged between the server and the client are as follows and the main roles are annotated in the right column.

<Data specification between server and client>

// basic protocol of transmission / reception frame

unsigned char sendcheck Signals that it should be sent in a transmit / receive loop unsigned char recvcheck Indicated that the transmission loop is received unsigned char readalready Information to prevent data loss in double buffer when receiving

// client control

unsigned char DiplayMode Determination of the Client's Display Screen to Full or Windowsed Screen unsigned char IsBeat Whether there is music beat in the current frame unsigned char BeatMode Automatically change foreground or background when a bit is detected unsigned char avimode Whether to use video as a texture in the background unsigned char FpsOn Shows the number of frames displayed per second in debug mode unsigned char ClientAddr [13] IP address of the client unsigned char HorDisp The horizontal angle of the screen to be displayed (the horizontal angle of view of the observer) unsigned char VerDisp Vertical angle to the screen to be displayed (viewer's vertical angle)

// energy used in the background or foreground

int energy 1 frame (256 samples / frame, 11㎑ / sample) energy

// foreground information

unsigned char selFrontAnim Animation to use in the foreground unsigned short FSelTex Type of texture to use for foreground animation short Tab_Spectrum [TAB_S] Spectrum size (currently 8 bands) for each frequency band used for logo (a kind of foreground) animation short gxT X-axis moving coordinate of origin short gyT Y-axis moving coordinate of origin short gzT Z-axis moving coordinate of origin short gxR X-axis rotation coordinate of origin short gyR Y-axis rotation coordinate of origin short gzR Z-axis rotation coordinate of origin short rgblndex Specify color when not using texture during animation

// background information

unsigned short BselTex Type of texture to use for the background effect unsigned char selBackEffect Type of effect to use for the background short Theta Initial rotation angle

// specific animation related data

short Cepstrum [45] Use in keyboard extraction of piano animations short gxRdj Piano x rotation of piano animation short gyRdj Piano y rotation of piano animation short gzRdj Piano z rotation of piano animation short gHandAng Hands animation short SphereMode Determine Sphere Mode (Texture Fixed) short SideSphereMode Determining the Spectral Sphere Mode

The above table is an example of the case of the piano animation which is an animation in which the piano keys are pressed according to the extracted music characteristics. In addition, various animation effects can be provided, and accordingly, an appropriate data regulation can be prepared.

6 is a view showing an embodiment in which the present invention is implemented in real space.

The screen shows an example of configuring a 180-degree screen with three image projectors and three corresponding clients. Screens A, B, and C show different moving images.

Of course, as described above, the image file containing the video or background image of the screen A, B, C is not divided into three parts and stored in each of the three clients, but the entire image including A, B, C is stored. Therefore, according to the environment set in the server, each client is configured to extract different parts of the entire image from the entire image.

That is, the first client connected to the first image projector 600 projects the image on the screen by setting the screen A to a specific vertical angle and horizontal angle, in this case, a horizontal angle of 60 degrees. It is possible to configure stereoscopically while the second client connected to the second image projector 610 displays the screen B and the third client connected to the third image projector 620 displays the screen C. FIG.

Accordingly, it is possible to express the fish swimming on the A-C screen in accordance with the music.

As mentioned above, although this invention was demonstrated with drawing and Example, this invention is not limited to a specific Example, The person of ordinary skill in the art, many corrections and a change without departing from the scope of this invention. I will understand this possible.

In addition, the protection scope of the present invention will be defined by the appended claims.

According to the present invention, by arranging a plurality of clients to control the image projector according to the angle of view, it is inexpensive and easy to display an image responsive to sound on a wide-angle screen.

In addition, by configuring the network through the server and the client has the effect that the central control in the server can be easily.

Claims (10)

A first step in which the user stores the video image file on the server computer and the client computer, and sets environment settings such as a background image, an animation effect type, and an area to be in charge of each client computer; A second step of generating detailed information by calculating the change of the image according to the animation effect according to the characteristics of the sound source inputted by the server and transmitting data related thereto to all client computers, and transmitting a synchronization frame every predetermined time; According to the data received from the server, each client synthesizes the image according to the image image file and the detailed information from the server, and transmits the image corresponding to the set area of the client to the image projector, the received data In the case of a synchronization frame, the third step of disregarding previously received data and processing data included in the synchronization frame A method of implementing a panoramic image using a server and a client comprising a. The method of claim 1, A method of implementing a panoramic image using a server and a client, characterized in that one image projector is connected to each client computer. The method of claim 1, And storing the image image file in the server computer in the first step, and transmitting the image image file to the client computer. The method of claim 1, The configuration of the area to be in charge of each client computer in the first step is to allocate a display area having a horizontal angle and a vertical angle of view to be displayed for each client. Way. The method of claim 1, And the sound source of the second step is a sound input through a microphone. delete A sound input unit for receiving sound; An audio signal processing unit for extracting a feature of an audio signal from the sound input unit; A user interface unit for receiving input from a user, A controller for extracting detailed information by calculating a change in an image according to an animation effect according to the sound signal extracted by the sound signal processor, and generating a synchronization frame at a predetermined time; Image file storage for storing images, A server comprising a network interface unit for transmitting the detailed information and a synchronization frame to a client; A control unit for synthesizing an image using the detailed information received from the server and outputting the synthesized image to an image output unit, in case a synchronization frame is received, ignoring previously received data and processing data included in the synchronization frame; Image file storage for storing images, A network interface unit for exchanging information with a server connected to a network, One or more clients comprising an image output unit for transmitting an image, One or more image projectors connected to the image output unit of the client to project the image image on the screen. A system for implementing a panoramic image using a server and a client comprising a. The method of claim 7, wherein A system for implementing a panoramic image using a server and a client, characterized in that one image projector is connected to each client computer. The method of claim 7, wherein And the sound input unit comprises any one or more of a microphone for directly receiving sound or an input means for directly receiving an electric signal from an acoustic device. The method of claim 7, wherein And the user interface includes any one or more of a keyboard, a mouse, and a touch screen.