CN110471772B

CN110471772B - Distributed system, rendering method thereof and client

Info

Publication number: CN110471772B
Application number: CN201910765782.0A
Authority: CN
Inventors: 王珏; 周峰
Original assignee: Shanghai Yunshen Intelligent Technology Co ltd
Current assignee: Shanghai Hongxing Cloud Computing Technology Co ltd
Priority date: 2019-08-19
Filing date: 2019-08-19
Publication date: 2022-03-15
Anticipated expiration: 2039-08-19
Also published as: CN110471772A

Abstract

The invention discloses a distributed system and a rendering method thereof, and a client, relating to the field of holographic projection, wherein the distributed system comprises: at least two hosts; the host computer which acts as a server realizes information synchronization among the host computers which act as clients through a network, and at least one projection device is connected to the host computer which acts as a client. The framework of the distributed system of the invention distributes the rendering pressure of each client, can realize the rendering of the immersive display environment with higher requirements on the quality of the picture, and has better stability and fluency in the rendering effect. The number of the hosts serving as clients in the distributed system can be flexibly increased and decreased according to actual requirements, and the distributed system is convenient and quick to construct and wide in application.

Description

Distributed system, rendering method thereof and client

Technical Field

The invention relates to the field of holographic projection, in particular to a distributed system, a rendering method thereof and a client.

Background

The existing immersive display environment, whether single-screen surrounding type or multi-screen surrounding type, adopts a host, on which a program is run, and the program creates a corresponding number of rendering windows to render the content projected on each screen according to the number of projection screens.

The system architecture adopted by the traditional immersive display environment concentrates all pressure on one host, one host runs a single program, and the single program starts a plurality of rendering windows, but the plurality of windows render pictures with a smooth frame number very difficultly, so that the traditional system architecture can only meet the rendering with low picture quality requirements, particularly the real-time requirement of ray tracing is not met. For example: the rendering frame number of the multiple screens can be stabilized at 30 frames/second.

With the improvement of aesthetic appearance of people, the requirements on the picture quality are higher and higher, and the system architecture adopted by the traditional immersive display environment can not meet the actual requirements any more.

Disclosure of Invention

The invention aims to provide a distributed system, a rendering method thereof and clients, which share the rendering pressure of each client, can realize the rendering of an immersive display environment with higher requirements on picture quality, and greatly improve the use experience of users.

The technical scheme provided by the invention is as follows:

a distributed system, comprising: at least two hosts; the host computer which acts as a server realizes information synchronization among the host computers which act as clients through a network, and at least one projection device is connected to the host computer which acts as a client.

In the technical scheme, the distribution of rendering pressure is realized through the connection architecture of the plurality of clients and the server, so that the rendering of the immersive display environment with higher requirements on the quality of the picture is realized.

Further, one host acts as both the server and any of the clients.

In the above technical solution, one host can simultaneously assume the functions of a server and one client, and different hosts (or terminal devices) can be set as servers according to actual application conditions, thereby improving the flexibility of setting a distributed system.

Further, the server is configured to receive projection device information of each projection device connected to the client, which is sent by each client; receiving configuration information corresponding to each piece of projection equipment information; and sending each of the configuration information to a corresponding client; and the client is used for starting a rendering program to render the corresponding projection picture according to the configuration information when receiving the configuration information sent by the server.

In the technical scheme, the configuration information of each projection device can be distributed by the server and managed uniformly, so that the reasonable distribution of the rendering pressure of each projection device is ensured.

Further, the server is further used for storing and displaying information of each projection device; the receiving, by the server, configuration information corresponding to each piece of projection device information specifically includes: and the server receives corresponding configuration information set by a user according to the displayed information of each projection device.

In the technical scheme, the configuration information can be set by the user, the requirements of different actual conditions are met, the adaptability is good, and the use experience of the user is improved.

Further, the step of, by the client, starting a rendering program to render the corresponding projection screen according to the configuration information includes: the client acquires the resolution of the projection equipment corresponding to the configuration information; acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information; and starting a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate.

In the technical scheme, the accurate rendering of the projection picture can be ensured by acquiring the window coordinates and the resolution.

Further, when the system display mode is a non-expansion mode (i.e. only one display device), the window coordinates are initial coordinates; when the system display mode is an extended mode and the projection equipment corresponding to the configuration information is main display equipment, the window coordinate of the projection equipment corresponding to the configuration information is an initial coordinate;

when the system display mode is an extended mode and the projection equipment corresponding to the configuration information is non-main display equipment, determining the window coordinates of the projection equipment corresponding to the configuration information according to the position relationship between the non-main display equipment and the resolution of each projection equipment.

In the technical scheme, different modes of acquiring the window coordinates by the projection equipment connected in the non-expansion mode and the expansion mode are disclosed, so that the accuracy of rendering pictures by each projection equipment is ensured.

Further, the server is further configured to execute an operation instruction when receiving the operation instruction sent by the client, and obtain an operation result corresponding to the operation instruction; according to the rendering requirement, sending the corresponding operation result to the corresponding client for synchronization; and the client renders the projection picture according to the synchronized operation result when receiving the synchronized operation result in the rendering process.

In the technical scheme, the server synchronizes the operation results of the clients according to the rendering requirements, so that the consistency of the projection pictures rendered by the projection equipment on the clients is better.

Further, the server is further configured to obtain location information of a user, and send the location information to each client for synchronization; and in the rendering process, when the position information is received, the client carries out interpolation by a linear interpolation method according to the position information acquired last time and the position information received at present to obtain a plurality of interpolation position information, and renders the projection picture corresponding to each interpolation position information and the position information received at present respectively.

In the technical scheme, the position information of the user is considered during the projection picture rendering, the content watched by the user can change along with the change of the position of the user, the transition is smooth during the projection picture changing, and the watching experience of the user is greatly improved.

The present invention also provides a client, which is applied to any one of the above-mentioned distributed systems, and includes: the sending module is used for sending the information of the projection equipment of each connected projection equipment to the server after the communication connection with the server is established; the receiving module is used for receiving the configuration information sent by the server according to the information of the projection equipment; and the rendering module is used for starting a corresponding rendering program to render the projection picture according to the configuration information.

Further, the receiving module is further configured to obtain a resolution of the projection device corresponding to the configuration information; acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information; the rendering module is configured to start a rendering program to render the corresponding projection picture according to the configuration information, and specifically includes: and the rendering module starts a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate.

Further, when the system display mode is a non-expansion mode, the window coordinate is an initial coordinate; when the system display mode is an extended mode and the projection equipment corresponding to the configuration information is main display equipment, the window coordinate of the projection equipment corresponding to the configuration information is an initial coordinate; when the system display mode is an extended mode and the projection equipment corresponding to the configuration information is non-main display equipment, determining the window coordinates of the projection equipment corresponding to the configuration information according to the position relationship between the non-main display equipment and the resolution of each projection equipment.

Further, the sending module is further configured to send the operation instruction to the server when the operation instruction is generated.

Further, still include: the receiving module is further configured to receive an operation result of the server synchronization; and the rendering module is further used for rendering the projection picture according to the synchronous operation result in the rendering process.

Further, the rendering module is further configured to, when the position information sent by the server is received, perform interpolation by an interpolation method according to the position information obtained last time and the currently received position information in the rendering process to obtain a plurality of interpolation position information, and respectively render the projection picture corresponding to each interpolation position information and the currently received position information.

The invention also provides a rendering method of the distributed system, which comprises the following steps: after communication connection with a server is established, sending the information of the projection equipment of each connected projection equipment to the server; receiving configuration information sent by the server according to the projection equipment information; and starting a rendering program to render the corresponding projection picture according to the configuration information.

Further, the method also comprises the following steps: acquiring the resolution of the projection equipment corresponding to the configuration information; acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information; the step of starting a rendering program to render the corresponding projection picture according to the configuration information comprises the following steps: and starting a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate.

Further, the method also comprises the following steps: when an operation instruction is generated, the operation instruction is sent to the server.

Further, the method also comprises the following steps: and when the operation result synchronized by the server is received, rendering the projection picture according to the synchronized operation result in the rendering process.

Further, the method also comprises the following steps: when receiving the position information which is sent by the server and is synchronized, in the process of rendering, carrying out linear interpolation according to the position information which is obtained last time and the position information which is received at present through an interpolation method to obtain a plurality of interpolation position information, and rendering the projection picture corresponding to each interpolation position information and the position information which is received at present respectively.

Compared with the prior art, the distributed system, the rendering method thereof and the client have the advantages that:

the framework of the distributed system of the invention distributes the rendering pressure of each client, can realize the rendering of the immersive display environment with higher requirements on the quality of the picture, and has better stability and fluency in the rendering effect. The number of the hosts serving as clients in the distributed system can be flexibly increased and decreased according to actual requirements, and the distributed system is convenient and quick to construct and wide in application.

Drawings

The above features, technical features, advantages and implementations of a distributed system and its rendering method, a client, and a computer program product will be further described in the following description of preferred embodiments in a clearly understandable manner with reference to the accompanying drawings.

FIG. 1 is a schematic block diagram of one embodiment of a distributed system of the present invention;

FIG. 2 is a schematic diagram of the architecture of one embodiment of the client of the present invention;

FIG. 3 is a flow chart of one embodiment of a rendering method for a distributed system of the present invention;

FIG. 4 is a diagram of a UI for displaying one embodiment of projection device information for a client in accordance with the present invention;

FIG. 5 is a diagram of the projection effect of one embodiment of the present invention on a front side of a projection screen;

FIG. 6 is a diagram of the left projection effect on the same projection screen as in FIG. 5;

fig. 7 is a diagram of the effect of the projection on the same projection screen as in fig. 5, which is located on the right.

The reference numbers illustrate:

1. the system comprises a server, 2, a client, 21, a sending module, 22, a receiving module, 23, a rendering module and 3, a projection device.

Detailed Description

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.

For the sake of simplicity, the drawings only schematically show the parts relevant to the present invention, and they do not represent the actual structure as a product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically illustrated or only labeled. In this document, "one" means not only "only one" but also a case of "more than one".

The immersive display environment of the invention is mainly applied to home decoration display, and a corresponding number of projection screens can be adopted as required. For example: the decoration environment of show kitchen can adopt five projection screens, is respectively: up, down, left, right, front; the decoration effect of a long decoration wall on one side of the living room can be displayed, and only one projection screen can be adopted.

The number of projection devices corresponding to each projection screen can be determined according to the requirements of image quality, for example: each projection screen corresponds to one projection device, or three projection screens correspond to one projection device respectively, and the fourth projection screen corresponds to two projection devices.

The logic services of the distributed system described below can be implemented using existing engines, including but not limited to the ghost engine 4.

A host, which is a terminal device supporting various applications, for example: mobile phones, tablet computers, dedicated terminals, etc.

In one embodiment of the invention, a distributed system comprises: at least two hosts; the hosts serving as the servers 1 synchronize information between the hosts serving as the clients 2 via a network, and at least one projection device 3 is connected to the host serving as the client 2.

Fig. 1 shows an embodiment of a distributed system, which has a server 1 and two clients 2, wherein both clients 2 and the server are connected to the server 1 through TCP/IP communication, and each client 2 has two projection devices 3 connected thereto.

Specifically, the client and the server are connected in the same network, so that communication connection between the client and the server is realized, information is forwarded through the server, and information synchronization between the clients is realized.

The projection device corresponds to a display device, which can be connected via a graphics card interface to a client, for example: the projection device is connected with a client through an HDMI/VGA interface. When the immersive environment is implemented based on a 4K projection device, it is connected with the client through an interface above HDMI 2.1. If a client is connected with a plurality of projection devices, the client needs to have a corresponding number of display card interfaces for connection with each projection device.

For example: one client is connected with three projection devices, at least three HDMI interfaces are arranged on the client, one projection device corresponds to one HDMI interface, and the three projection devices are connected to the client.

Each client bears the rendering pressure of the respective projection device to which it is connected. In the above example, three projection devices are connected to the client, and the client will start three rendering programs, each rendering program being responsible for rendering the image of the corresponding projection device.

If the budget is limited, several projection devices may share one client. Preferably, each projection device is connected to a client, which can achieve good dispersion of rendering pressure.

If there are four projection screens, each projection screen corresponds to a projection device, and the application scenario is used to perform a comparison example between the system of the present invention and the conventional system:

in a system architecture adopted in a conventional immersive display environment, the four projection devices are all connected to a same host (which is equivalent to a client of the present invention), and the host runs a rendering program to open four rendering windows (each of which corresponds to one projection device) to perform rendering of a picture, that is, all rendering pressures are concentrated on the host.

In the distributed system of the present invention, a plurality of clients (equivalent to a plurality of hosts) may be adopted, the four projection devices are connected to the clients, and each client is only responsible for rendering the picture corresponding to the projection device connected thereto, that is, the pressure of rendering the pictures corresponding to the four projection devices is distributed to each client. If budget is enough, four clients can be directly adopted, each client is connected with one projection device, and the connection mode ensures that each client only needs to be responsible for the rendering pressure of one projection device. Of course, if the budget is limited, three clients may be used, wherein one client is connected to two projection devices, and the other two clients are respectively connected to one projection device; or two clients, each client is connected with two projection devices, and compared with the traditional system architecture, the rendering pressure of the clients is reduced.

As an embodiment, a Dedicated Server (dedicate Server) may be used to implement the network communication connection with each client.

As another embodiment, if the application is applied to home decoration exhibition, it does not need a large amount of data interaction like a game, and it may not use a dedicated Server, and the cost performance is higher by using a Listen Server (Listen Server), i.e. adding corresponding parameters to specify whether the host starts a Server client or a client of a client.

For example: the virtual engine 4 is adopted to appoint each host (such as a computer, a mobile phone, a tablet computer and other terminal equipment) to start the client in a Listen mode, and if the client with client parameters is set on the host, the client is equivalent to the client after being started; if the client of the server parameter is set on the host, the client is equivalent to a server after being started; if the client set on the same host contains both client parameters and server parameters, the client is started to execute both the server function and the client function.

It should be noted that the aforementioned rendering program is a functional program for executing the business logic related to rendering on the client with the client parameter.

Examples are as follows:

if there are three projection devices A, B, C and three hosts a ', B', and C ', the camera device a is connected to the host a', the camera device B is connected to the host B ', the camera device C is connected to the host C', clients of client parameters are set on the host B 'and the host C', both of them execute the service logic that the client should execute after being started, a client of a list parameter (both client parameters and server parameters) is set on the host a ', and the host a' executes the client, which is equivalent to both a server and a client.

In practical applications, human-computer interaction may be provided for users to better view an immersive presentation environment. For example: the user can open the displayed cabinet, replace a certain ornament and the like through interaction. Specific implementations are various, for example: the client is enabled to have the function of interacting with the user, or the server is enabled to realize the function of man-machine interaction and the like.

For example: the projection screen is two sides, the front side and the left side are respectively provided, three projection devices are adopted, wherein the first projection device and the second projection device are responsible for front projection, the third projection device is responsible for left projection, the first projection device and the second projection device are connected to a first client, the third projection device is connected to a second client, and a mobile phone of a user is used as a server and is in communication connection with the first client and the second client. The user can operate on the mobile phone, and the mobile phone synchronizes the operation result corresponding to the operation instruction to the first client and the second client at a proper time according to the rendering requirement. Of course, if the first client and the second client also have an interactive function, the client generating the operation instruction may also send the operation instruction to the mobile phone serving as the server, and after the operation instruction is executed by the mobile phone, the operation result corresponding to the operation instruction is synchronized to the other client at an appropriate time according to the rendering requirement of each client.

In the embodiment, the architecture of the distributed system distributes the rendering pressure of each client, so that the projection of one scene can be synchronously operated by a plurality of screens, the distributed system can be applied to the rendering of an immersive display environment with high requirements on the quality of the screen, and the rendering effect has high stability and fluency. The number of the clients in the distributed system can be flexibly increased and decreased according to actual requirements, and the distributed system is convenient and quick to construct and wide in application.

Based on being applied to the field of home decoration display, the data is simpler compared with games, a Listen Server mode can be adopted to establish a distributed system, the cost is low, different terminal devices can be flexibly set as servers according to requirements, and the method has high adaptability.

The improvement based on the above distributed system embodiment, in addition to the same as above, further comprises: the system comprises a server 1, a client and a server, wherein the server 1 is used for receiving projection equipment information of projection equipment connected to the client and sent by the client; receiving configuration information corresponding to each piece of projection equipment information; and sending each of the configuration information to a corresponding client.

Specifically, the server 1 and the client 2 may perform information interaction after establishing a long connection. The two can adopt protocols such as TCP connection in a Socket protocol, Web-Socket protocol and the like to carry out long connection. Once the client 2 is successfully connected to the server 1, the client 2 automatically pushes projection device information of each projection device connected thereto to the server 1.

For example: when the two clients establish long connection with the server, the first client sends the projection device information of the projection device A to the server, and the second client sends the projection device information of the projection device B and the projection device C to the server, so that the server receives the projection device information of the three projection devices.

Projection device information includes, but is not limited to: the resolution of the projection device, the system display mode of the projection device and the client, the name of the projection device, the number of the projection device, and the like.

In one embodiment, after receiving the projection device information of each projection device, the server may obtain the configuration information corresponding to each projection device information through default settings.

Specifically, the configuration information includes: a projection screen responsible for projection.

For example: the server receives the two pieces of projection equipment information, the default setting is that the configuration information of the first piece of projection equipment information is the front projection surface, and the configuration information of the second piece of projection equipment information is the right projection surface.

If a projection screen is responsible for several projection devices, examples are as follows: the front projection screen is responsible for two projection devices, the configuration information of the first projection device information is projected to the left side of the front, and the configuration information of the second projection device information is projected to the right side of the front.

As another embodiment, the server 1 is further configured to store and display information of each projection device; the step of receiving, by the server 1, configuration information corresponding to each piece of projection device information specifically includes: and the server 1 receives corresponding configuration information set by a user according to the displayed information of each projection device.

Specifically, after receiving the projection device information fed back by each client, the server may store the projection device information, and feed back the stored projection device information to an operation User through a User Interface (UI), where the User may set configuration information for the projection device corresponding to each projection device information, so that each client starts a corresponding rendering program to render a projection picture.

As shown in fig. 4, the configuration information of two projection devices connected to one client is received by the server and sent by the client, and is displayed through the UI, and the user sets the configuration information corresponding to the first projection device information as the front projection surface, and sets the configuration information corresponding to the second projection device as the left projection surface.

When each piece of projection equipment information has the corresponding configuration information, each piece of configuration information is sent to the corresponding client to be executed with the subsequent operation.

For example: the configuration information corresponding to the first piece of projection equipment information is a front projection surface, the configuration information corresponding to the second piece of projection equipment information is a left projection surface, the projection equipment corresponding to the two pieces of projection equipment information are connected to the same client, and the server sends the two pieces of configuration information to the client, so that the client knows that the projection equipment corresponding to the first piece of projection equipment information is the front projection surface, and the projection equipment corresponding to the second piece of projection equipment information is the left projection surface.

And the client 2 is used for starting a rendering program to render the corresponding projection picture according to the configuration information when receiving the configuration information sent by the server.

Specifically, each client sets corresponding parameters to run a rendering program according to the configuration information sent by the server, and undertakes rendering of the corresponding projection picture.

For example: if one client is connected with two projection devices (respectively No. 1 and No. 2) and respectively projects the Front and the left, when a server transmits an opening instruction (the form of the opening instruction is not limited, the opening instruction can be transmitted to the client along with configuration information, or the configuration information can be directly mixed in the opening instruction) to the client, data (namely the configuration information, the Front parameter of the No. 1 camera) is respectively attached to the ID of the corresponding projection device, and when the Front picture needs to be started in the No. 1 projector, a rendering program can obtain the Front parameter during starting, so that the forward virtual camera is activated to render the Front picture; similarly, the client responsible for projecting the Left side will activate the corresponding Left virtual camera via the Left parameter. Similarly, if another client connects to a projector and needs to cast Right, the server will transmit the configuration information with Right parameter to the client when informing the client to start the rendering program.

It should be noted that, in actual use, after receiving configuration information of a projection device, a client runs an installed client, sets a startup parameter (e.g., Front parameter, window coordinate, etc.) corresponding to the configuration information to invoke a rendering program, and activates a virtual camera at a corresponding position, thereby implementing image rendering at the position. Therefore, although the same rendering program is called due to different configuration information, the set starting parameters are different, and the rendering of pictures at different positions can be realized. When a plurality of projection devices are arranged on one client, the respective configuration information is adopted to call a rendering program to render different projection pictures, which is equivalent to the game client having more accounts.

Optionally, the starting, by the client 2, of the rendering program to render the corresponding projection screen according to the configuration information includes:

the client 2 is used for acquiring the resolution of the projection equipment corresponding to the configuration information; acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information; and starting a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate.

The window coordinates are obtained to determine frame information of a projection picture rendered by the projection device, that is, Left (Left), Right (Right), Top (Top), and Bottom (Bottom) information of the frame information, so as to ensure that the picture projected after rendering is correct in position, has no offset, and is correct in size.

For example: if the resolution of the projection apparatus is 2560x1440 and the coordinate of the upper Left point is known to be (Left is 0 and Top is 0), then the coordinate of the lower Left point is estimated to be (0, 1440), the coordinate of the lower right point is estimated to be (2560, 1440), and the coordinate of the upper right point is estimated to be (2560, 0).

Therefore, the window coordinates may directly obtain the coordinates of any corner point, and for convenience of description, the window coordinates in this embodiment are selected as the coordinates of the upper left corner.

Optionally, when the system display mode is a non-extended mode (the non-extended mode is understood to mean that only one projection device is connected to one client, and the system display mode of the projection device and the client is the non-extended mode), the window coordinates are initial coordinates;

when the system display mode is an extended mode and the projection equipment corresponding to the configuration information is main display equipment, the window coordinate of the projection equipment corresponding to the configuration information is an initial coordinate;

Specifically, the window coordinates are initial coordinates, and for convenience of calculation, the initial coordinates may be set to (0, 0).

If a plurality of projection devices are connected to one client, the projection devices are all connected to the client in an extended mode, wherein only one projection device is a main display device of the client, and the other projection devices are all non-main display devices of the client, and in an operating system of the client, each non-main display device has a positional relationship with the main display device, for example: a non-primary display device is located to the left, above, or to the right of the primary display device (orientation of the software layer), etc. The window coordinates of the projection apparatuses connected in the extended mode need to be determined according to the positional relationship between them and the projection apparatus as the main display apparatus and the resolution of each projection apparatus.

For example: two projection devices are connected to the same client, the system display mode of the first projection device (with a resolution of 1920x1080) and the client is an extended mode, the first projection device is a main display device and is configured to project on the right side of the front, the system display mode of the second projection device and the client is an extended mode, the second projection device is a non-main display device, the second projection device is located on the left side of the main display device in the system setting of the client, and is configured to project on the left side of the front, the window coordinate corresponding to the first projection device is (0, 0), and the window coordinate adaptability corresponding to the second projection device is (-1920, 0).

For another example: two projection devices are connected to the same client, the system display mode of the first projection device (with a resolution of 1920x1080) and the client is an extended mode, the configuration information is a front projection surface which is a main display device, the system display mode of the second projection device and the client is an extended mode, the configuration information is a right projection surface which is positioned on the right of the main display device, the window coordinate corresponding to the first projection device is (0, 0), and the window coordinate adaptability corresponding to the second projection device is (1920, 0).

The window coordinates of each projection device are determined according to the system display mode, so that the correct rendering of the projection pictures corresponding to each projection device is ensured, the projected pictures can be harmoniously spliced, and users can obtain better watching effect in an immersive display environment.

For example: as shown in fig. 5-7, three projection devices project a projection screen on the same side, where the resolutions of the three projection devices are 2560x1440, and if the three projection devices are all connected to the same client, the projection device a corresponding to fig. 5 is a main display device, the projection device B corresponding to fig. 6 and the projection device C corresponding to fig. 7 are both non-main display devices, the position information corresponding to the projection device B is located on the left side of the main display device in the operating system, the position information corresponding to the projection device B is located on the right side of the main display device in the operating system, the window coordinate corresponding to the projection device a is (0, 0), the window coordinate corresponding to the projection device B is (-2560, 0), and the window coordinate corresponding to the projection device C is (2560, 0).

Of course, the position relationship of each projection device in the operating system may also be: the projection device B is used as a main display device, the projection device A is positioned at the right side of the projection device B, and the projection device C is positioned at the right side of the projection device A; the window coordinate corresponding to the projection device B is (0, 0), the window coordinate corresponding to the projection device a is (2560, 0), and the window coordinate corresponding to the projection device C is (5120, 0).

Optionally, in the distributed system of this embodiment, the server 1 is further configured to execute an operation instruction sent by the client when receiving the operation instruction, so as to obtain an operation result corresponding to the operation instruction; according to the rendering requirement, sending the corresponding operation result to the corresponding client for synchronization; and the client 2 renders the projection picture according to each synchronous operation instruction when receiving the synchronous operation instruction in the rendering process.

Specifically, when an operation instruction of a user is generated on any one of the clients, the operation instruction is sent to the server, the server processes the operation instruction to obtain a corresponding operation result, and then the operation result is sent to other clients at a proper time for synchronization according to the rendering requirements of each client, and each client performs rendering of a projection image according to the synchronized operation result during rendering. For example: the server is a tablet computer held by a user, the client A is connected with one projection device A 'and is responsible for projecting the front face, the client B is connected with the other projection device B' and is responsible for projecting the left face, when the user opens the door of the leftmost cabinet on the front projection screen, the operation instruction is an operation instruction, the client A generates a corresponding operation instruction according to the operation of the user and sends the operation instruction to the server, the server executes the operation instruction and stores a corresponding operation result, when the operation result is found to be related to the picture to be rendered by the client B, the operation result is sent to the client B for synchronization, the client B also knows that the door of the leftmost cabinet on the front face is opened, and the rendered projection picture is rendered into an opened state when the rendered projection picture relates to the cabinet.

It should be noted that the synchronization operation results more essentially from the synchronization between the rendering programs called for different projection screens.

For example: the same client is provided with two projection devices, wherein the projection picture corresponding to the projection device No. 1 is a left projection picture which needs to be projected by a rendering program called by using configuration information as a left projection plane, window coordinates (0, 0) and a resolution of 2560x 1440; the projection screen corresponding to the projection device No. 2 is a front projection screen which needs to be projected by a rendering program called by taking the configuration information as the projection front, window coordinates (2560, 0) and own resolution of 2560x 1440. When a left cabinet is opened, the client sends the operation instruction to the server, the server sends the corresponding operation result, namely the left cabinet is opened and sent to each client at a proper time, and the rendering program for rendering the front projection picture also synchronizes the operation result to ensure the rendering synchronization of the pictures.

The operation instruction can be obtained by the prior art, and the specific manner is not limited. For example: the user inputs an operation instruction on the client a. For another example, the operation command is automatically recognized by detecting the user's motion through a sensor.

In another embodiment, the user may input an operation instruction on the server, and the server synchronizes operation results corresponding to the operation instruction to all the clients.

For example: if the server is arranged at a place which can be contacted by the user, the user can input an operation instruction through an interactive interface on the server, and after the operation instruction is executed, the corresponding operation result is synchronized to all the clients.

The operation results corresponding to the operation instructions are synchronized on all the clients, so that the states of the same ornaments of the projection pictures rendered on the clients at all angles can be kept consistent, and users can have better observation effects. For example: a cabinet is open, with the front projected in an open position, and the sides relating to the cabinet are also open.

In one embodiment, when the client renders the projection pictures, the relative position of the user station in the immersive exhibition environment is also considered, and the position information of the user can be understood to ensure that each rendered projection picture conforms to the picture seen by the user station at the position in the actual scene.

For example: in an actual scene, if a user stands right in front of a television, the user sees the front of the television, and the user can hardly see the side; if the user station is on the left side of the television, he sees the left side, a portion of the front side, of the television.

Similarly, in the process of rendering the projection picture, the position information of the user is considered, so that the rendered and projected picture can better accord with the visual angle of the user, and the watching effect is better.

Optionally, the server 1 is further configured to obtain location information of a user, and send the location information to each client for synchronization;

in the rendering process, when receiving the position information, the client 2 performs interpolation (preferably, linear interpolation is performed, and the picture transition effect is better) by an interpolation method according to the position information acquired last time and the position information received at present, so as to obtain a plurality of interpolation position information, and renders the projection picture corresponding to each interpolation position information and the position information received at present.

Specifically, the server can acquire the position information of the user through a certain frequency, and the acquired position information is sent to each client for synchronization, so that the position information of the user is considered in the rendering process, and the projected picture is more in line with the actual situation.

In consideration of the transmission speed of data in the network, in order to ensure the smoothness of the projection picture, each client needs to perform necessary interpolation between the acquired two successive position information. It is to be noted that the position information of the user is a two-dimensional coordinate representation established from the actual immersive presentation environment, i.e. the position where the user stands in the actual immersive presentation environment.

For example: if the synchronous speed of the position information is 2 times/second, the rendering speed of each client is 60 frames/second, if the coordinate of the position information synchronized for the first time in one second is (0, 0), the coordinate of the position information synchronized for the second time is (20, 30), if the client does not perform interpolation processing in one second, only the projection pictures of each position seen by the two positions, namely the projection picture seen when the user is located at the coordinate (0, 0) and the projection picture seen when the user is located at the coordinate (20, 30), are only rendered in one second, and the two projection pictures are directly jumped, so that the conversion is hard, and the observation effect is not good.

If necessary interpolation is carried out between two pieces of position information acquired in sequence, a plurality of pieces of interpolation position information are obtained, and rendering of projection pictures is carried out one by one, so that transition of the pictures in projection is smoother and smoother, and the viewing effect of a user is improved.

For example: if the coordinates of the first synchronized position information are (0, 0) and the coordinates of the second synchronized position information are (20, 30) within one second, interpolation is performed between the two to obtain nineteen pieces of interpolated position information (1, 1.5) and (2, 3) … …, and if the projection screen corresponding to the position information (0, 0) is rendered, the projection screens corresponding to the coordinates (1, 1.5) and (2, 3) … … (20, 30) are rendered in sequence and projected one by one.

It should be noted that the specific interpolation process is determined according to the actually selected interpolation method, and is not limited herein.

In the embodiment, each client can synchronize information through the server, so that the rendered pictures of each client are relatively good in consistency, and a good watching effect is guaranteed.

In addition, the projection pictures can be rendered according to different positions where the user stands, so that the user can see the projection pictures at different angles at different positions; and when the position is changed, interpolation processing can be carried out, so that the projected picture transition is smoother and smoother, and the watching experience of a user is further improved.

Fig. 2 shows an embodiment of a client 2 of the invention, comprising: and the sending module 21 is configured to send the projection device information of each connected projection device to the server after the communication connection with the server is established.

Specifically, after a long connection can be established between the server and the client 2, information interaction is performed. The two can adopt protocols such as TCP connection in a Socket protocol, Web-Socket protocol and the like to carry out long connection. Once the client 2 is successfully connected to the server, the client 2 automatically pushes the projection device information of each projection device connected thereto to the server.

A receiving module 22, configured to receive configuration information sent by the server according to the information of the projection device.

And the rendering module 23 is configured to start a rendering program to render the corresponding projection picture according to the configuration information.

Specifically, the configuration information includes: a projection screen responsible for projection. And each client runs the rendering program according to the configuration information sent by the server and different parameters to undertake rendering of the corresponding projection picture.

Optionally, the receiving module 22 is further configured to obtain a resolution of the projection device corresponding to the configuration information; acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information; and the number of the first and second groups,

the rendering module 23, which starts a rendering program to render the corresponding projection picture according to the configuration information, includes: and the rendering module 23 starts a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate.

Specifically, the resolution of the projection device corresponding to the configuration information is obtained to determine the number of points specifically rendered in one projection screen.

For a specific example, reference may be made to the second embodiment of the distributed system, which is not described herein again.

If a plurality of projection devices are connected to one client, the projection devices are all connected to the client in an extended mode, wherein only one projection device is a main display device of the client, and the other projection devices are all non-main display devices of the client, and in an operating system of the client, each non-main display device has a positional relationship with the main display device, for example: a non-primary display device is located to the left, above, or to the right of the primary display device, etc. The window coordinates of the projection apparatuses connected in the extended mode need to be determined according to the positional relationship between them and the projection apparatus as the main display apparatus and the resolution of each projection apparatus. For a specific example, reference may be made to the second embodiment of the distributed system, which is not described herein again.

Optionally, the sending module 21 is further configured to send the operation instruction to the server when the operation instruction is generated.

A receiving module 22, further configured to receive an operation result of the server synchronization; the rendering module 23 is further configured to render the projection picture according to the synchronized operation result in the rendering process.

Specifically, when an operation instruction of a user is generated on any one of the clients, the operation instruction is sent to the server, the server processes the operation instruction to obtain a corresponding operation result, and then the operation result is sent to other clients at a proper time for synchronization according to the rendering requirements of each client, and each client performs rendering of a projection picture according to the synchronized operation result during rendering.

For example: the server is a tablet computer held by a user, the client A is connected with one projection device A 'and is responsible for projecting the front face, the client B is connected with the other projection device B' and is responsible for projecting the left face, when the user opens the door of the leftmost cabinet on the front projection screen, the operation instruction is an operation instruction, the client A generates a corresponding operation instruction according to the operation of the user and sends the operation instruction to the server, the server executes the operation instruction and stores a corresponding operation result, when the operation result is found to be related to the picture to be rendered by the client B, the operation result is sent to the client B for synchronization, the client B also knows that the door of the leftmost cabinet on the front face is opened, and the rendered projection picture is rendered into an opened state when the rendered projection picture relates to the cabinet.

The operation results corresponding to the operation instructions are synchronized on all the clients, so that the states of the same ornaments of the projection pictures rendered on the clients at all angles can be kept consistent, and users can have better and consistent observation effects. For example: a cabinet is open, with the front projected in an open position, and the sides relating to the cabinet are also open.

In another embodiment, in addition to the same as the present embodiment, the client may also consider the relative position of the user station in the immersive exhibition environment when rendering the projection images, which may also be understood as the position information of the user, and ensure that each rendered projection image conforms to the image seen by the user station at the position in the actual scene.

Optionally, the rendering module 23 is further configured to, when receiving the position information sent by the server, perform interpolation (preferably, perform linear interpolation, and the picture transition effect is better) by an interpolation method according to the position information obtained last time and the position information received now in the rendering process to obtain a plurality of interpolation position information, and respectively render the projection picture corresponding to each of the interpolation position information and the position information received now.

It should be noted that the specific interpolation process is determined according to the actually selected interpolation method, and is not limited herein. In addition, please refer to the second embodiment of the distributed system for a specific example, which is not described in detail in this embodiment.

Fig. 3 shows a flow chart of a rendering method of a distributed system of the present invention, applied to a client, comprising the steps of:

s301, after the communication connection with the server is established, the information of the projection devices of the connected projection devices is sent to the server.

S302, receiving configuration information sent by the server according to the projection equipment information;

s303, according to the configuration information, a rendering program is started to render the corresponding projection picture.

Specifically, the configuration information includes: a projection screen responsible for projection. And each client undertakes rendering of the corresponding projection picture by the rendering programs which run with different parameters according to the configuration information sent by the server.

Optionally, the rendering method of the distributed system further includes the following steps:

s304, acquiring the resolution of the projection equipment corresponding to the configuration information;

s305, acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information;

in step S303, according to the configuration information, starting a corresponding rendering program to render the projection image includes the following steps:

s313, according to the configuration information, the resolution and the window coordinates, a rendering program is started to render the corresponding projection picture.

The window coordinates are obtained to determine frame information of a projection picture rendered by the projection device, that is, Left (Left), Right (Right), Top (Top), and Bottom (Bottom) information of the frame information, so as to ensure that the picture projected after rendering is correct, has no offset, and has a correct size.

when an operation instruction is generated, the operation instruction is sent to the server.

And when the operation result synchronized by the server is received, rendering the projection picture according to the synchronized operation result in the rendering process.

when receiving the position information sent by the server for synchronization, in the rendering process, interpolating (preferably, performing linear interpolation, and the picture transition effect is better) by an interpolation method according to the position information obtained last time and the position information received at present to obtain a plurality of interpolation position information, and rendering the projection picture corresponding to each interpolation position information and the position information received at present respectively.

It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A distributed system, comprising:

at least two hosts;

the host computer which acts as a server realizes information synchronization among the host computers which act as clients through a network, and at least one projection device is connected to the host computer which acts as a client;

the server is used for receiving the projection equipment information of each projection equipment connected to the client, which is sent by each client; and the number of the first and second groups,

receiving configuration information corresponding to each piece of projection equipment information; and the number of the first and second groups,

sending each of the configuration information to a corresponding client;

the client is used for starting a rendering program to render the corresponding projection picture according to the configuration information when the configuration information sent by the server is received;

the client starts a rendering program to render the corresponding projection picture according to the configuration information, and the rendering process comprises the following steps:

the client acquires the resolution of the projection equipment corresponding to the configuration information; and the number of the first and second groups,

acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information; and the number of the first and second groups,

starting a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate;

when the system display mode is a non-expansion mode, the window coordinate is an initial coordinate;

2. The distributed system of claim 1, wherein: one host acts as both the server and the client.

3. The distributed system of claim 1, wherein:

the server is further used for storing and displaying information of each projection device;

the receiving, by the server, configuration information corresponding to each piece of projection device information specifically includes:

and the server receives corresponding configuration information set by a user according to the displayed information of each projection device.

4. The distributed system of claim 1, wherein:

the server is further used for executing the operation instruction when receiving an operation instruction sent by the client to obtain an operation result corresponding to the operation instruction; according to the rendering requirement, sending the corresponding operation result to the corresponding client for synchronization;

and the client renders the projection picture according to the synchronized operation result when receiving the synchronized operation result in the rendering process.

5. The distributed system of claim 1, wherein:

the server is further used for acquiring the position information of the user and sending the position information to each client for synchronization;

and in the rendering process, when the position information is received, the client carries out interpolation by a linear interpolation method according to the position information acquired last time and the position information received at present to obtain a plurality of interpolation position information, and renders the projection picture corresponding to each interpolation position information and the position information received at present respectively.

6. A client for use in a distributed system according to any one of claims 1 to 5, comprising:

the sending module is used for sending the information of the projection equipment of each connected projection equipment to the server after the communication connection with the server is established;

the receiving module is used for receiving the configuration information sent by the server according to the information of the projection equipment;

the rendering module is used for starting a rendering program to render the corresponding projection picture according to the configuration information;

the receiving module is further configured to obtain a resolution of the projection device corresponding to the configuration information; acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information;

the rendering module is configured to start a rendering program to render the corresponding projection picture according to the configuration information, and specifically includes:

the rendering module starts a rendering program to render the corresponding projection picture according to the configuration information, the resolution and the window coordinate;

7. The client of claim 6, wherein:

the sending module is further used for sending the operation instruction to the server when the operation instruction is generated.

8. The client of claim 7, further comprising:

the receiving module is further configured to receive an operation result of the server synchronization;

and the rendering module is further used for rendering the projection picture according to the synchronous operation result in the rendering process.

9. The client of claim 6, wherein:

the rendering module is further configured to, when receiving the position information sent by the server, perform interpolation by a linear interpolation method according to the position information acquired last time and the position information received at present in the rendering process to obtain a plurality of interpolation position information, and render a projection picture corresponding to each interpolation position information and the position information received at present respectively.

10. A rendering method for a distributed system, comprising the steps of:

after communication connection with a server is established, sending the information of the projection equipment of each connected projection equipment to the server;

receiving configuration information sent by the server according to the projection equipment information;

according to the configuration information, starting a rendering program to render the corresponding projection picture;

further comprising the steps of:

acquiring the resolution of the projection equipment corresponding to the configuration information;

acquiring window coordinates according to the system display modes of the projection equipment and the client corresponding to the configuration information;

the step of starting a rendering program to render the corresponding projection picture according to the configuration information comprises the following steps:

11. The rendering method for a distributed system according to claim 10, further comprising the steps of:

12. The rendering method for a distributed system according to claim 11, further comprising the steps of:

when the operation result of the server synchronization is received,

and rendering the projection picture according to the synchronous operation result in the rendering process.

13. The rendering method for a distributed system according to claim 10, further comprising the steps of:

when receiving the position information which is sent by the server and is synchronized, in the process of rendering, carrying out linear interpolation according to the position information which is obtained last time and the position information which is received at present through an interpolation method to obtain a plurality of interpolation position information, and rendering the projection picture corresponding to each interpolation position information and the position information which is received at present respectively.