WO2009110435A1

WO2009110435A1 - Server device, projector equipped therewith, and display system

Info

Publication number: WO2009110435A1
Application number: PCT/JP2009/053890
Authority: WO
Inventors: 内田桂; 安江範巳
Original assignee: ブラザー工業株式会社
Priority date: 2008-03-03
Filing date: 2009-03-02
Publication date: 2009-09-11
Also published as: JP5029429B2; JP2009212657A

Abstract

Provided are a server device, a projector equipped therewith, and a display system which allow a conference etc. with participants in a distance and furthermore reduces a load on a communication path when participants gather in the same room and hold a conference etc., thereby sharing of information and smooth cooperative work by a plurality of persons are allowed. The server device (1) provided with an image acquisition part which acquires image data relating to images included in each predetermined display area of a plurality of terminal devices (2) through the communication path and an image processing part which generates an image for display by synthesizing a plurality of pieces of image data acquired by the image acquisition part is provided with a transmission processing part comprising a first mode of transmitting the data relating to the image for display to the projector (3) and a second mode of transmitting the data relating to the image for display to the terminal devices (2).

Description

Server apparatus and projector and display system including the same

The present invention relates to a server device and a projector and display system including the server device, and more particularly to a server device capable of joint work and information sharing by a plurality of people, and a projector and display system including the server device.

Conventionally, there has been known a projector that inputs data of an image displayed on a terminal device such as a computer device and enlarges and projects the image on a screen. This type of projector is widely used in conferences where a plurality of people participate, and by using such a projector, discussions can be promoted while a plurality of participants see the enlarged image at the same time. Can do.

However, when the image input from the terminal device is enlarged and projected by the projector as described above, each participant must gather in the same room. Therefore, it is not possible to hold a conference or the like while each participant is remote.

Therefore, there is a conference support system that efficiently shares information and collaborates in a conference by connecting a plurality of terminal devices through a communication path such as a network.

For example, in Patent Document 1, as shown in FIG. 17, in a conference support system including a server device and a plurality of terminal devices, the server device 510 transmits each image transmitted from the plurality of terminal devices 520a to 520c. By combining the data 530a to 530c, transmitting the combined image data 540 to the plurality of terminal devices 520a to 520c, and displaying an image based on the image data 540 on the screen of each terminal device 520a to 520c, A conference support system 500 is disclosed in which each participant can discuss while looking at the same image.

In this conference support system, a conference can be held in a state where the participants are remote from each other. In addition, when each participant is in the same room, a projector is connected to the terminal device that constitutes the conference support system, so that a plurality of participants can view the enlarged projected image at the same time and if necessary. Each participant can proceed with the discussion while editing the image of his terminal device.
JP 2006-31359 A

会議 Meetings held in the same room are relatively easy to hold and communicate, so the number of participants often increases compared to meetings between remote locations.

However, in the conference support system described in Patent Document 1, the server device transmits image information synthesized for each of the plurality of terminal devices even when the projector is connected to the terminal devices constituting the system. As the number of participants increases and the number of terminal devices increases, a load is imposed on a communication path such as a network. When a load is applied to the communication path in this way, transmission and reception of image data may be delayed depending on the communication path, and there is a possibility that information sharing and joint work cannot be performed smoothly.

Therefore, the present invention reduces the load on the communication path when a participant gathers in the same room and conducts a conference while allowing the participant to be in a remote state, so that information by multiple people can be reduced. It is an object of the present invention to provide a server device that can smoothly perform sharing and collaborative work, and a projector and a display system including the server device.

In order to achieve the above object, according to one aspect of the present invention, an image acquisition unit that acquires image data of an image in each predetermined display area of a plurality of terminal devices via a communication path, and the image acquisition unit An image processing unit configured to generate a display image by combining the plurality of image data acquired in the first mode, wherein the display image data is transmitted to a predetermined display device; A transmission processing unit having a second mode for transmitting the display image data to the plurality of terminal devices is provided.

By configuring in this way, when the participant is in a remote state, the terminal device is used when the participant gathers in the same room to hold the conference while transmitting the display image to the terminal device and enabling the conference. The display image is displayed on a display device such as a display or projector without sending the display image to the network, reducing the load on the communication path and sharing information and collaborative work by multiple people Can be performed smoothly.

The server device further includes an operation determination unit that determines whether the display device is in an operating state, and the transmission processing unit displays the display in the first mode when the display device is in an operating state. The image data may be transmitted, and the display image data may be transmitted in the second mode when the display device is not in an operating state.

With this configuration, while the display device installed in the room where the participants gather is not operating, data is not transmitted to the terminal device, so the load on the communication path can be reduced, while the display device is not in the operating state. By doing so, a remote conference is possible, and the screen can be shared without troublesome operations and settings.

In the server device, the operation determining unit determines whether a specific terminal device is in an operating state among the plurality of terminal devices, and the transmission processing unit is configured to display the display in the first mode. In addition to the device, the plurality of terminal devices including the specific terminal device in the second mode, wherein the display image data is transmitted to the specific terminal device in the active state. The display image data may be transmitted to all of them.

With this configuration, a plurality of participants can gather in one room and view the image on the display device, and further remote participants can view the display image. Therefore, even when all of them cannot gather in a conference room or the like, information sharing and collaborative work can be performed efficiently.

In the server device, the communication path is a route through an IP network, and a terminal device other than the terminal device in the same network as the own device is set as the specific terminal device among the plurality of terminal devices. Alternatively, a setting unit may be provided.

With this configuration, it is possible to automatically detect a remote terminal device that is not in the same network as its own device. Therefore, each time a participant's location changes, a specific terminal device is assigned to the server device. There is no need to perform the setting operation, and the troublesome operations and settings can be avoided.

Further, the server device may be provided with an input unit for setting and inputting information on the specific terminal device.

With this configuration, it is possible to efficiently share information and collaborate by setting a specific terminal device according to the participation status of the participants.

In the server device, the image processing unit generates a plurality of display image data having different resolutions, and the transmission processing unit transmits the display image data having a resolution corresponding to a transmission destination. It may be.

With this configuration, it is possible to perform appropriate display on each terminal device, and to share information and collaborate more efficiently.

Further, in the server device, when an operation command for operating a predetermined image among the display images is acquired via the communication path, a terminal that has transmitted the image data of the predetermined image that is a target of the operation command You may make it the structure provided with the operation command process part which transmits the said operation command to the apparatus via the said communication path.

With this configuration, it is possible to edit an image from a device other than the terminal device that generates the image, and it is possible to smoothly perform joint work by a plurality of people.

Further, the projector may include the server device and the display device.

With this configuration, it is not necessary to install the display device and the server device separately, and the installation space can be easily secured.

In order to achieve the above object, according to another aspect of the present invention, there is provided image acquisition for acquiring a plurality of terminal devices and image data of images in each predetermined display area of the plurality of terminal devices via a communication path. A display device comprising: a server device, and an image processing unit that generates a display image by combining a plurality of pieces of image data acquired by the image acquisition unit; The display system includes a transmission processing unit having a first mode for transmitting the display image data to a predetermined display device and a second mode for transmitting the display image data to the plurality of terminal devices. It was.

According to the present invention, in a server device that generates a display image by acquiring and combining image data of images in each predetermined display area of a plurality of terminal devices via a communication path, the display image data is obtained. Since a transmission processing unit having a first mode for transmitting to a predetermined display device and a second mode for transmitting display image data to the plurality of terminal devices is provided, display is performed when the participant is remote. When a participant gathers in the same room to hold a conference, etc., while the image is sent to the terminal device, the display image is not transmitted to the terminal device, and a display such as a display or projector is displayed. Since the display image is displayed on the apparatus, the load on the communication path can be reduced, and information can be shared and collaboratively performed by a plurality of people.

It is a figure which shows the whole system configuration | structure of the display system in this embodiment. It is a schematic block diagram of the server apparatus used for the display system in this embodiment. It is a schematic block diagram of the terminal device used for the display system in this embodiment. It is a figure which shows the outline | summary of the transmission processing operation | movement of the display system in this embodiment. It is a figure which shows the outline | summary of the transmission processing operation | movement of the display system in this embodiment. It is a figure which shows the outline | summary of the transmission processing operation | movement of the display system in this embodiment. It is a block diagram of the server apparatus in this embodiment. It is a figure which shows an example of the image attribute information management table memorize | stored in the server apparatus in this embodiment. It is a figure for demonstrating the coordinate of the image for a display synthesize | combined by the server apparatus in this embodiment. It is an example of the connection information management table memorize | stored in the server apparatus in this embodiment. It is an example of the pointer information management table memorize | stored in the server apparatus in this embodiment. It is a figure which shows an example of the specific terminal management table memorize | stored in the server apparatus in this embodiment. It is a block diagram of the terminal device in this embodiment. It is a main process flowchart of the server apparatus in this embodiment. It is an image delivery process flowchart of the server apparatus in this embodiment. It is a main process flowchart of the terminal device in this embodiment. It is a reception data processing flowchart of the server apparatus in this embodiment. It is explanatory drawing of a prior art example.

Explanation of symbols

S display system 1 server device 2 terminal device 3 projector 4 communication path

[1. Outline of display system S]
Hereinafter, a display system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an overall system configuration of a display system in the present embodiment, FIG. 2 is a schematic configuration diagram of a server device used in the display system in the present embodiment, and FIG. 3 is a display in the present embodiment. It is a schematic block diagram of the terminal device used for a system.

As shown in FIG. 1, the display system S in the present embodiment includes a server device 1, a plurality of terminal devices (here,

terminal devices

2 a and 2 b) connected to the server device 1 via a communication path 4, Of the images generated by an application program (hereinafter also simply referred to as “application”) that is configured by the projector 3 and executed by the plurality of

terminal devices

2a and 2b, each terminal device 2a. , 2b, the image data of the image in the predetermined display area (here, the

virtual display areas

45a, 45b) is synthesized as one display image data, and the display image is displayed on the projector 3 or the like under predetermined conditions. This is a system for displaying on a display device.

Further, the display system S transmits an operation command for operating a predetermined image among the display images generated by the server device 1 from any terminal device to another terminal device via the server device 1. Thus, based on this operation command, not only the image generated by the application executed on the terminal device but also the image generated by the application executed on the other terminal device among the display images. Editable.

In the present embodiment, the case where two terminal devices are connected to the server device 1 will be described, but the number of terminal devices connected to the server device 1 is not limited to this. In the following, when any one of the

terminal devices

2a and 2b is expressed, it may be expressed as the terminal device 2.

Here, in the display system S according to the present embodiment, the server device 1 includes an image acquisition unit 10, an image processing unit 11, an operation determination unit 12, a transmission processing unit 13, and an operation command as illustrated in FIG. A processing unit 14, a setting unit 15, an input unit 16, and a storage unit 17 are provided.

As shown in FIG. 3, the terminal device 2 includes a storage unit 20, an image generation unit 21, a predetermined area image transmission unit 22, an editing unit 23, a display unit 24, and a virtual display region setting unit 25. , An operation unit 26, an operation command transmission unit 27, and an instruction unit 28.

An overall outline of the display system S configured as described above will be described with reference to FIGS. 4A, 4B and 5 are diagrams showing an outline of the transmission processing operation of the display system S in the present embodiment.

The operation of the display system S in the present embodiment is mainly a connection processing operation for connecting the server device 1 and the terminal device 2, and the server device 1 combines a plurality of image data acquired from each terminal device 2. An image processing operation for generating display image data, and when the projector 3 is in an operating state, the display device data is transmitted from the server device 1 to the projector 3 to enlarge the display image from the projector 3 while the projector 3 When the terminal 3 is not in an operating state, the display image data is transmitted to the terminal device 2 to display the display image on the terminal device 2, and an arbitrary data file is edited in accordance with an operation command from the terminal device 2. There are operation command processing operations. Hereinafter, these processing operations will be described in the order of a connection processing operation, an image processing operation, a transmission processing operation, and an operation command processing operation.

(About connection processing operation)
Connection between the server device 1 and the terminal device 2 is performed by a user of the terminal device 2 performing an activation operation on the terminal device 2 using the operation unit 26. When the activation operation is performed, the terminal device 2 transmits a connection request signal from the operation command transmission unit 27 to the server device 1. When the server apparatus 1 receives the connection request signal, the transmission processing unit transmits information on the resolution set in the server apparatus 1 to the terminal apparatus 2 in order to unify the resolution of the image data acquired from the plurality of terminal apparatuses 2. 13 to transmit.

Next, the terminal device 2 provides a virtual display area as a predetermined display area based on the received resolution information. As shown in FIG. 1, in the

terminal devices

2a and 2b, the virtual display areas are

virtual display areas

45a and 45b adjacent to the

display areas

40a and 40b of the respective display units 24. The virtual display area setting unit 25 is set. Hereinafter, when any one of the

display areas

40a and 40b is represented, the display area 40 may be represented, and when any one of the

virtual display areas

45a and 45b is represented, the virtual display area 45 may be represented. . In the present embodiment, the virtual display area 45 is set as the predetermined display area as described above. This predetermined display area is, for example, a part of the display area of the display unit 24 provided in the terminal device 2. May be provided.

After providing the virtual display area 45 in this way, the terminal device 2 notifies the server device 1 that the startup work has been completed from the operation command transmission unit 27. And the server apparatus 1 which received the said notification notifies that the connection was completed with respect to the terminal device 2 from the operation command process part 14, and the connection process of the server apparatus 1 and the terminal device 2 is completed.

(Image processing operation)
After the server device 1 and the terminal device 2 are connected, when an operation for executing an application program of the terminal device 2 is performed by a user's operation using the operation unit 26, the terminal device 2 becomes an operation target. The application program is executed, and image data corresponding to the data file of this application is generated.

Here, the data file is a saved data file of an application executed by various application programs such as spreadsheet software and graphic creation software possessed by the terminal device 2, and is stored in the storage unit 20 possessed by each terminal device 2. Has been.

The application program functions as the image generation unit 21 and the editing unit 23 in the terminal device 2. In other words, when the user of the terminal device 2 performs an operation of executing a data file using the operation unit 26, the application program performs an image based on the data file stored in the storage unit 20 as the image generation unit 21 (for example, FIG. 1, data of images 30 a to 30 c generated by the terminal device 2 a and

images

30 d and 30 e generated by the terminal device 2 b are generated. Further, when the user of the terminal device 2 performs a predetermined operation on the image data generated by the image generation unit 21 using the operation unit 26 or the instruction unit 28, the application program becomes the editing unit 23. Edit the data file based on the operation.

As described above, when the image generation unit 21 generates image data based on the data file, the terminal device 2 displays an image based on the image data in the window format on the display unit 24 (hereinafter, based on the image data as described above). An image displayed in a window format may be referred to as a “window image”). At this time, if there is an input operation for moving the window image to the virtual display area 45 by the user's operation on the operation unit 26, the window displayed on the display unit 24 by the editing unit 23 and the image generation unit 21. The image is moved and positioned in the virtual display area 45. The terminal device 2 detects the window image in the virtual display area 45 and updates the image attribute information based on the detection result. Thereafter, the terminal device 2 transmits the updated image attribute information to the server device 1. The image attribute information is information related to the window image in the virtual display area 45. In addition to identification information (window ID) for identifying each window image in the virtual display area 45, the position and size of the image, etc. This information is described later in detail.

When the image acquisition unit 10 of the server device 1 receives the image attribute information from each terminal device 2, the image acquisition unit 10 updates the information in the image attribute information management table. The image attribute information management table is a table for managing image attribute information related to a plurality of image data acquired from a plurality of terminal devices 2, and is stored in a predetermined area of the storage unit 17 provided in the server device 1. The

Next, the server device 1 transmits image data request information to the

terminal devices

2a and 2b based on the information in the image attribute information management table. In response to this image data request information, the predetermined area image transmission unit 22 of each of the

terminal apparatuses

2a and 2b sends the image data of the

window images

30c and 30e in the

virtual display areas

45a and 45b via the communication path 4 to the server apparatus 1. Send to.

Upon receiving the image data of the

window images

30c and 30e from the

terminal devices

2a and 2b, the server device 1 uses the image processing unit 11 to obtain the image data acquired from the

terminal devices

2a and 2b based on the information in the image attribute information management table. Data of the synthesized image (hereinafter also referred to as “display image”) is generated.

(About transmission processing operation)
When the server device 1 generates the display image data as described above, the operation determination unit 12 determines whether or not the projector 3 is in the operating state, and the transmission processing unit 13 displays based on the determination. It is determined whether the image data transmission is performed in the first mode or the second mode, and the display image data is transmitted.

Here, the first mode is a processing mode for transmitting display image data to the projector 3 as a display device, and the second mode is a processing mode for transmitting display image data to the plurality of terminal devices 2a and 2b. is there.

That is, when the projector 3 is in the operating state, the server apparatus 1 detects the operation of the projector 3 by the operation determining unit 12 and, as shown in FIG. 1st mode processing to transmit to 3 is performed. Then, the projector 3 that has received the display image data enlarges and projects the display image 50 onto a screen or the like.

Here, the “operating state” refers to a state in which the projector 3 can project an image and is connected to the communication path 4 so as to be communicable with the server device 1. In other words, the operation determination unit 12 is in the operating state when the projector 3 receives the display image data via the communication path 4 and is capable of enlarging and projecting the display image. Is determined. Therefore, the operation determination unit 12 of the server device 1 determines that the projector 3 is not in the operating state, for example, when the power of the projector 3 is in the OFF state or when the projector 3 is not connected to the communication path 4. To do.

As described above, when the projector 3 is in the operating state, a display image based on the image data synthesized by the image processing unit 11 of the server device 1 is transmitted to the projector 3 via the communication path 4, and the projector 3 performs a screen or the like. Since the image data for display is not transmitted to each terminal device 2, the load on the communication path can be reduced. As a result, even when many terminal devices 2 are connected to the server device 1, information sharing and collaborative work by a plurality of people can be performed smoothly. Note that the device that transmits the display image as the first mode may be an LCD display or a plasma display instead of the projector 3.

On the other hand, when the projector 3 is not in the operating state, the server apparatus 1 detects the non-operation of the projector 3 by the operation determining unit 12 and, as shown in FIG. A second mode process for transmitting to the

terminal devices

2a and 2b is performed. The

terminal devices

2a and 2b that have received the display image data display the display image on the display unit 24, respectively. When the transmission processing unit 13 can communicate with the terminal device 2, the transmission processing unit 13 determines that the terminal device 2 is in an operating state and transmits a display image.

Each of the

terminal devices

2a and 2b determines whether the image displayed on the display unit 24 is a display image transmitted in the second mode processing by the user operating the operation unit 26, or It is possible to select whether to display an image generated by an application program, and work can be performed while switching the screen as appropriate according to the progress of the conference.

Thus, even when the projector 3 is not in operation, the screen can be shared without troublesome operations and settings. Further, when the

terminal devices

2a and 2b are respectively arranged at locations remote from the room in which the projector 3 is installed, the conference using the

terminal devices

2a and 2b can be performed by setting the projector 3 to a non-operating state. Can be done.

By the way, for example, as shown in FIG. 5, when a participant has a meeting using the

terminal devices

2a and 2b in the room 5 in which the projector 3 in an operating state is installed, the terminal device located at a remote place is used. There is a case where it is desired to participate in this conference using 2c.

In such a case, the operation determination unit 12 of the server device 1 determines whether or not the terminal device 2 is in an operating state with the terminal device 2c as a specific terminal device, and the transmission processing unit 13 of the server device 1 In the first mode, display image data can be transmitted to the terminal device 2c in an operating state in addition to the projector 3. Therefore, even when everyone cannot go to the conference room, the remote terminal device 2 can display a display image, so that information can be shared and collaborated efficiently. it can.

Here, in the display system S in the present embodiment, the input unit 16 of the server device 1 is operated so that information on a specific terminal device can be set and input. Information on the specific terminal device is stored in the storage unit 17. In the first mode, the transmission processing unit 13 of the server device 1 determines a specific terminal device based on the information stored in the storage unit 17, and transmits a display image to the specific terminal device.

Further, when the communication path 4 is a path through the IP network, the setting unit 15 of the server apparatus 1 uses a terminal apparatus other than the terminal apparatus in the same network as the own server apparatus 1 among the plurality of terminal apparatuses 2a to 2c. And this terminal device is set as a specific terminal device. Here, the “same network” means a network of the same router, but a network in the same subnet or the same wireless LAN network can also be defined as the “same network”. Further, the range accessible by the IP address of the local network may be defined as “same network”.

As shown in FIG. 5, when a plurality of participants gather in the same room 5 for a conference, the same router device (not shown) or this router device is used for each participant's

terminal device

2a, 2b and server device 1. A LAN (Local Area Network) cable is connected from a hub (HUB) connected to. Therefore, the IP addresses of the LAN devices formed by the router devices are assigned to the IP addresses of the

terminal devices

2a and 2b and the server device 1.

For example, when the IP address of the LAN network formed by the router device is “192.168.0.1” to “192.168.0.254”, the

terminal devices

2a and 2b and the server device 1 are assigned one of these IP addresses. It will be. On the other hand, since the terminal device 2 c at a remote location is not connected to the same router device as the server device 1, another IP address is assigned when viewed from the server device 1. Therefore, the setting unit 15 of the server device 1 determines the terminal device 2 c as a specific terminal device and stores the determination result in the storage unit 17. Here, a terminal connected to the same router device is a terminal device 2 that transmits a limited broadcast of an IP packet (for example, echo request) of ICMP (Internet Control Message Protocol) from the server device 1 and responds to the IP packet (for example, echo response). It is determined that the device is a device, and the determination result is stored in the storage unit 17.

The setting unit 15 of the server device 1 transmits an ICMP IP packet requesting subnet information from the server device 1 to the router device, and the terminal device based on the subnet information transmitted from the router device accordingly. It may be determined whether or not 2 is connected to the same router device. At this time, the setting unit 15 of the server device 1 stores the IP address range (for example, “192.168.0.1” to “192.168.0.254”) of the network to which the server device is connected in the storage unit 17. A terminal device to which an IP address not in the address range is assigned is specified as a specific terminal device. The storage in the storage unit 17 can also be performed by input from the input unit 16.

The setting unit 15 of the server device 1 may set a terminal device other than a terminal device that can be accessed with an IP address of the local network, that is, a terminal device 2 that can be accessed only with a global IP address, as a specific terminal device. .

As described above, by providing the setting unit 15 that sets a terminal device other than the terminal device having the same IP address among the plurality of terminal devices 2 as a specific terminal device, the remote terminal device 2 is automatically set. Therefore, it is not necessary to perform the operation of setting a specific terminal device in the server device 1 every time the participant changes, and the troublesome operation and setting can be avoided. Furthermore, since the input unit 16 for setting and inputting information of a specific terminal device is provided, the information can be efficiently shared and collaborated by setting the specific terminal device according to the participation status of the participants. Can do.

On the other hand, when the projector 3 is not in the operating state, the transmission processing unit 13 displays on all the

terminal devices

2a, 2b, and 2c including the terminal device 2c that is the specific terminal device in the operating state as the processing in the second mode. The image data is transmitted.

By the way, the image processing unit 11 of the server device 1 generates data of a plurality of display images having different resolutions. Then, when transmitting the display image data to the

terminal devices

2a, 2b, and 2c, the transmission processing unit 13 selects a display image corresponding to the resolution of the display unit 24 of each terminal device 2, and each terminal device 2 to send.

As described above, the server device 1 generates a plurality of display images having different resolutions, and transmits display image data having a resolution corresponding to the transmission destination to each terminal device 2, so that each terminal device 2 displays an appropriate display. It is possible to share information and collaborate more efficiently.

(Operation command processing operation)
In the display system S in the present embodiment, an arbitrary data file can be edited in response to a request from the terminal device 2. That is, in this embodiment, when the server device 1 acquires an operation command for operating any window image of the display images from the terminal device 2, the operation command processing unit 14 performs an operation according to the operation command. This operation command is transmitted to the terminal device 2 that has transmitted the image data of the target window image. When the terminal device 2 receives the operation command, the editing unit 23 edits the data file according to the operation command. It is said.

For example, when the user of the terminal device 2a operates the operation unit 26 or the instruction unit 28, predetermined operation input (for example, window image) is performed on the window image in the virtual display area 45b of the terminal device 2b among the display images. Terminal device 2a acquires the information of the operation input.

Next, in order to prevent the operation performed by the user from being performed on the terminal device 2a by the operation command transmission unit 27, the terminal device 2a discards the operation input information and prevents editing by the editing unit 23, An operation command corresponding to the operation input is transmitted to the server device 1 via the communication path 4.

When the server device 1 receives an operation command from the terminal device 2a, the operation command processing unit 14 performs a process of distributing the operation command to the terminal device 2 to be transmitted. Here, when the user of the terminal device 2a operates the window image of the terminal device 2b in the display image, and the operation command corresponding to the operation input is transmitted from the terminal device 2a, the operation command is transmitted to the terminal device 2b. To be sent to.

Then, the terminal device 2b that has received the operation command from the server device 1 reproduces the operation input based on the operation command, and the application edits the data file being executed as the editing unit 23 according to the reproduced operation input. I do.

As described above, in the display system S according to the present embodiment, when the server apparatus 1 acquires an operation command for operating a predetermined window image of the display images, the window image to be operated by the operation command is displayed. When the operation command is transmitted to the terminal device 2 that has transmitted the image data via the communication path 4, the terminal device 2 receives the operation command from the server device 1 via the communication path 4. Since the data file is edited, it is possible to edit not only the window image that the terminal device 2 has, but also the window image in which the image data is transmitted from the other terminal device 2 to the server device 1, It is possible to smoothly collaborate with multiple people.

In the display system S according to the present embodiment, the terminal device 2 includes an instruction unit 28 for instructing a display image, and the operation command transmission unit 27 operates when the instruction unit 28 is operated in a predetermined manner. When the image processing unit 11 of the server device 1 receives the image instruction command from the terminal device 2, the image processing command corresponding to the image command is sent to the server device 1. The image data of the window image in the virtual display area 45 of the terminal device 2 is combined with the image data.

That is, for example, as shown in FIG. 1, as a display image 50 projected by the projector 3, the server device 1 is based on an image instruction command in addition to a composite image based on image data acquired from each terminal device 2. Pointer images P1 and P2 are generated. With this configuration, the user of the terminal device 2 can grasp where the user is pointing in the display image area generated by the server device 1.

Here, in the above description, the server apparatus 1 and the projector 3 have been described as separate units. However, the projector can be a projector that incorporates the functions of the server apparatus 1 as they are. As described above, since the function of the server device 1 is built in the projector 3, it is not necessary to separately install the server device 1 and the projector 3, and it is easy to secure an installation space.

In this case, the projector 3 transmits display image data to the projection unit in the first mode when the projection unit of the own apparatus is in an operating state, and the display image is transferred from the projection unit to a screen or the like. When the projection unit of the own apparatus is not in operation, the display image is transmitted to the plurality of terminal apparatuses 2 in the second mode. Whether or not the projection unit is in an operating state can be determined, for example, based on whether or not a projection unit operation switch for setting the projection unit in an operating state is in an ON state or an OFF state.

[2. Specific configuration of display system]
Hereinafter, a specific example of the display system S described above will be described.

(About the configuration of the server device 1)
First, the configuration of the server device 1 will be described. 6 is a configuration diagram of the server device 1 in the present embodiment, FIG. 7 is a diagram illustrating an example of an image attribute information management table stored in the server device 1 in the present embodiment, and FIG. 8 is a diagram in the present embodiment. FIG. 9 is a diagram for explaining coordinates of a display image synthesized by the server apparatus 1. FIG. 9 is an example of a connection information management table stored in the server apparatus 1 in the present embodiment, and FIG. FIG. 11 is a diagram showing an example of a specific terminal management table stored in the server device 1 according to the present embodiment.

As shown in FIG. 6, the server device 1 communicates with an operation panel 101 for performing various operations, a storage unit 102 that is a nonvolatile memory for holding setting information, data files, and the like, and a terminal device 2. Interface 103, image processing circuit 104 that performs predetermined processing on the image data acquired via interface 103 to generate display image data, and display image data generated by image processing circuit 104 Is output in the form of an image signal, and a control unit 106 that controls the entire server device 1 is provided.

In the operation panel 101, operation buttons and the like operated by the user in the server device 1 are arranged, and the control unit 106 executes control according to the operation content of the operation buttons and the like.

The storage unit 102 is configured by a flash memory, a hard disk, and the like, and includes resolution information set in the server device 1, pointer image image data, and a window image in the virtual display area 45 of each terminal device 2. In addition to storing image data, an image attribute information storage area 110 for storing an image attribute information management table for managing position information, size information, etc. of the window image, and the connection status of the terminal device 2 connected to the server device 1 A connection information storage area 111 for storing a connection information management table to be managed, a pointer information storage area 112 for storing a pointer information management table for storing pointer position information and the like, and a terminal device for transmitting a display image in the first mode Specific terminal information storage for storing a specific terminal management table for managing information on (specific terminal device) A region 113 and the like are included.

The interface 103 is a LAN interface capable of communicating with an external device via an IP network. When acquiring image data, an operation command, or the like from the terminal device 2 or communicating with an external computer device Used by the control unit 106. Note that either a wireless LAN interface or a wired LAN interface may be used as the LAN interface.

The image processing circuit 104 synthesizes a plurality of image data acquired via the interface 103 under the control of the control unit 106 to generate display image data. The display image data is output as an image signal from the image signal output circuit 105 or transmitted to the terminal device 2 via the interface 103.

The image signal output circuit 105 receives display image data from the image processing circuit 104 and converts it into data in the form of an image signal such as a composite image signal (eg, NTSC video signal) or component image signal (eg, RGB signal). Output.

The control unit 106 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and the CPU reads and executes a program stored in advance in the ROM, so that the server The apparatus 1 is caused to function as an image acquisition unit, an image processing unit, an operation determination unit, a transmission processing unit, an operation command processing unit, a setting unit, and the like. Note that the ROM of the control unit 106 may be a nonvolatile memory such as the flash memory described above.

As the image acquisition unit, the control unit 106 acquires the image data of the window image in the virtual display area 45 of the terminal device 2 from the terminal device 2 via the interface 103. Further, the control unit 106 functions as an image processing unit together with the image processing circuit 104, thereby generating image data for display by synthesizing image data of window images in the virtual display area 45 of the plurality of terminal devices 2. . The composition of the image data by the image processing unit is executed based on the image attribute information management table and the connection information management table stored in the storage unit 102.

Here, the image attribute information management table will be described with reference to FIG.

As shown in FIG. 7, in the image attribute information management table, the name of the terminal device 2 (owner PC) that transmitted the window image in the virtual display area 45 of each terminal device 2, the position to display the window image, and The information regarding the size, the information regarding the file name of the image data, and the information regarding the order (layer level) in which the image data are superimposed when the image processing circuit 104 synthesizes a plurality of image data identify the window image. Is stored in association with the information (window ID).

Information regarding the position and size of the window image in the virtual display area 45 of each terminal device 2 is represented by coordinates defined in the display image 50 generated by the server device 1. That is, as shown in FIG. 8, the server device 1 in this embodiment defines the vertical direction of the display image 50 as the X direction, the horizontal direction as the Y direction, and the top left vertex as the origin. The position information of the window image is information on the coordinates of the upper left vertex of each window image in the virtual display area 45, and the size information of the image is the width (W) and height (H) of the window image. Can be determined by which position and size of the window image in the virtual display area 45 are to be combined with the position information and the size information. For example, in FIG. 8, the window image 30 c has an upper left vertex at the position of coordinates (40, 200), and the size of the image is width (W) 250 and height (H) is 280.

Also, the name of the terminal device 2 (owner PC) that transmitted the image data is associated with the connection information management table shown in FIG. In this connection information management table, as shown in FIG. 9, the name of the terminal device 2 connected to the server device 1 is the connection status (whether it is connected to the server device 1 or not connected) and each terminal. It is stored in association with the IP address assigned to the device 2.

Further, as an image processing unit, the control unit 106 synthesizes a pointer image at a position based on the image instruction command transmitted from the terminal device 2 together with image data of window images in the virtual display areas 45 of the plurality of terminal devices 2. To do. Here, the synthesis of the pointer images is executed based on the information in the pointer information management table.

As shown in FIG. 10, the pointer information management table includes an image file name of a pointer image (cursor image) assigned to the terminal device 2 connected to the server device 1 and a display image 50 synthesized by the server device 1. Is stored in association with the name of the terminal device 2. In FIG. 10, the position information of the pointer image of the terminal device 2 c is “none”, indicating that the pointer position is not in the virtual display area 45.

In the present embodiment, a different hierarchical level is assigned to each image data of the window image in the virtual display area 45 of each terminal device 2, and the image data is synthesized by superimposing the image data in the hierarchical level order. It is said. That is, when combining the image data of the window image in the virtual display area 45 of each terminal device 2, the control unit 106 refers to the information on the hierarchy level stored in the image attribute information management table, and this hierarchy level is Image data of an image that is formed by superimposing images in descending order is generated.

As shown in FIG. 11, the specific terminal information management table is a table in which the name of the terminal device 2 connected to the server device 1 is associated with the specific terminal flag. This indicates that the terminal device 2 whose specific terminal flag is “1” is a specific terminal device. This specific terminal information management table is generated by input from the input unit 16 described above or by setting by the setting unit 15.

Further, when the control unit 106 functions as an operation command processing unit together with the interface 103, when the operation command for operating any window image among the display images generated by the image processing circuit 104 is acquired, This operation command is transmitted to the terminal device 2 that has transmitted the image data of the window image to be operated by the operation command.

Here, in the present embodiment, when an operation command is transmitted to the terminal device 2, the image data of the window image to be operated by the operation command and the terminal device 2 to which the operation command is to be transmitted are managed in the image attribute information management. The determination is based on the image attribute information of the table.

By adopting such a configuration, it is not necessary to separately request each terminal apparatus 2 for information for determining the image data of the window image to be operated, and the burden on the communication path 4 can be reduced. Can do.

(Regarding the configuration of the terminal device 2)
Next, an example of a specific configuration of the terminal device 2 will be described. FIG. 12 is a configuration diagram of the terminal device 2 in the present embodiment.

As the terminal device 2 in the present embodiment, a general personal computer or a dedicated terminal can be applied. As shown in FIG. 12, the LCD 200 as a display unit, a display controller 201 for driving the LCD 200, various data, etc. , An interface 203 for communicating with the server device 1 via the communication path 4, a keyboard 206 and a mouse 207 (instruction unit) capable of inputting predetermined information by a user's operation. And a control unit 208 that controls the entire terminal device 2, and these various elements are connected to each other via a first bus 209 and a second bus 210. In the figure, reference numeral 211 denotes a bus controller for transferring data between the first bus 209 and the second bus 210.

The hard disk device 202 functions as a storage unit. The hard disk device 202 includes an operating system (OS) program, document creation software, presentation software, spreadsheet software, and graphic creation software for the terminal device 2. In addition to various application programs such as the above, data files created by this application program are stored. In addition to the application program, an image sharing program is stored as a processing program for the display system S in the present embodiment.

The image sharing program may be downloaded to the hard disk device 202 by the control unit 208 via the interface 203 from, for example, a server (not shown) connected to the communication path 4 or a CD. -After being recorded on a recording medium such as a ROM, it may be read into the hard disk device 202 by the control unit 208 via a recording medium drive (not shown).

The interface 203 is a LAN interface that can communicate with an external device. When transmitting / receiving image data, an operation command, and the like to / from the server device 1, the control unit 208 is used when communicating with an external computer device. Used by. Note that a wireless LAN interface or a wired LAN interface can be used as the interface 203.

The control unit 208 includes a CPU, a ROM, a RAM, and the like, and the CPU reads and executes an image sharing program stored in the hard disk device 202, whereby a predetermined area image transmission unit, a virtual display area setting unit, and an operation It functions as a command transmission unit.

The control unit 208 functions as an image generation unit and an editing unit by reading and executing various application programs stored in the hard disk device 202 by the CPU.

The CPU of the control unit 208 executes various application programs stored in the hard disk device 202, functions as an image generation unit, and generates image data based on the data file. The generated image data is displayed on the LCD 200 by the LCD 200 as a display unit and the display controller 201 as an image based on the image data (window image).

The control unit 208 functions as a predetermined area image transmission unit together with the interface 203, so that when the position of the window image based on the image data generated by the image generation unit is in the virtual display area 45, a part of the window image or All the image data is transmitted to the server device 1 via the communication path 4. Here, the predetermined display area in the present embodiment is a virtual display area 45 set so as to be adjacent to the display area 40 of the display unit. This virtual display area 45 is stored in the hard disk device 202 by the CPU of the control unit 208. It is set by executing the stored image sharing program and functioning as a virtual display area setting unit.

Here, in the display system S in the present embodiment, every time image data in the virtual display area 45 is generated based on the edited data file by the image generation unit, image data of the window image in the virtual display area 45 is generated. Is transmitted to the server device 1. With this configuration, the display image generated by the server device 1 can be updated in real time, so that information can be shared and collaborated smoothly by a plurality of people. When this image data is transmitted, BMP (Bit Map) data obtained by capturing a window image in the virtual display area 45 is converted into JPEG (Joint Photographic Experts Group) data. Thus, by compressing the image data to be transmitted to the server device 1, the load on the communication path 4 can be reduced.

Further, the control unit 208 functions as an editing unit, so that when an operation command is received from the server apparatus 1 via the communication path 4, the control unit 208 reproduces the operation input from the operation command, and a data file according to the operation input. Edit. The operation command is received by the interface 203.

Further, the control unit 208 functions as an operation command transmission unit together with the interface 203, so that when a predetermined operation / instruction is performed by the keyboard 206 or the mouse 207 when the pointer is positioned in the virtual display area 45, An operation command corresponding to the operation / instruction is transmitted to the server device 1. At this time, in order to prevent the operation / instruction from being executed by the terminal device 2, the control unit 208 transmits the operation command to the server device 1 after discarding the operation input information or the instruction information. Here, “operation” is a command for editing a data file being executed by an application of the own device or another terminal device, and “instruction” is a display image generated by the server device 1. This is a command for displaying or moving a pointer image that points to.

As described above, by using the keyboard 206 and the mouse 207 as the operation unit, the window image in the virtual display area 45 of the other terminal device 2 can be operated and edited, and by using the mouse 207 as the instruction unit. Furthermore, it is possible to grasp which region of the display image generated by the server device 1 is designated by the user of the terminal device 2. Further, since the operation and instruction of the window image are performed by transmitting an operation command to the terminal device that has transmitted the image data of the window image to be operated, the window image can be operated without imposing a heavy load on the communication path 4. It can be performed.

(Specific operation of server device 1)
Next, an example of a specific operation of the server device 1 will be described with reference to FIGS. FIG. 13 is a main process flowchart of the server apparatus 1 in the present embodiment.

As shown in FIG. 13, when a power switch (not shown) of the server apparatus 1 is pressed, the control unit 106 of the server apparatus 1 performs an initialization process (step S100). In this initialization process, the CPU of the control unit 106 executes an initial setting operation such as securing the work area of the internal RAM, sets the program stored in the internal ROM as an execution state, and sets the function as the control unit 106 as an operation state. Is done by.

Next, the control unit 106 determines whether or not there is a connection request from the terminal device 2 (step S101). Whether there is a connection request from the terminal device 2 is determined by whether the control unit 106 has received a connection request signal from the terminal device 2 via the interface 103.

In this process, when it is determined that there is a connection request (step S101: YES), the control unit 106 performs a connection process with the terminal device 2 that has made the connection request (step S102). This connection process is a process for the server device 1 to connect to the terminal device 2 via the communication path 4. Specifically, when the control unit 106 of the server device 1 receives the connection request signal transmitted from the terminal device 2, the control unit 106 first updates the connection information management table. That is, the control unit 106 stores, in the connection information management table, information related to the IP address and connection state of the terminal device 2 newly connected to the server device 1 in association with the name of the terminal device 2. Next, the control unit 106 updates the pointer information management table. That is, the control unit 106 associates, in the pointer information management table, the image file name of the pointer image and the position information of the pointer image assigned to the terminal device 2 newly connected to the server device 1 with the name of the terminal device 2. Remember. And the control part 106 notifies the terminal device 2 that the connection process was completed, and complete | finishes a connection process.

When the connection process of step S102 is completed or when it is determined that there is no connection request from the terminal device 2 (step S101: NO), the control unit 106 determines whether image attribute information has been received (step S103). ). This process is performed depending on whether image attribute information is transmitted from the terminal device 2 via the interface 103.

If it is determined in step S103 that image attribute information has been received (step S103: YES), the control unit 106 performs an image attribute information update process (step S104).

Here, the image attribute information update process will be described. When the image attribute information update process is started, the control unit 106 first determines whether an image attribute information addition flag or a deletion flag is set in the image attribute information transmitted from the terminal device 2. When the addition flag is set, the control unit 106 determines that new addition of image attribute information has been performed, and sets the image attribute information of the window image newly positioned in the virtual display area 45 as image attribute information. A new area is added to the management table, and a storage area for the image data of the window image is secured. Then, the control unit 106 sets the hierarchy level of the window image newly added to the virtual display area 45 to “1”, and the other window images are moved to a hierarchy one level lower than the currently assigned hierarchy level. Update it by lowering sequentially. When the deletion flag is set, the control unit 106 deletes the image attribute information of the window image to be deleted from the image attribute information management table and updates the hierarchical level of other window images. On the other hand, when the addition flag or the deletion flag is not set, the control unit 106 updates the image attribute information (for example, updates the position and size of the image) based on the image attribute information received from the terminal device 2. Then, the hierarchical level of the window image based on the updated image attribute information is set to “1”, and the hierarchical levels of the other window images are sequentially lowered.

Thereafter, the control unit 106 sends an image transmission request to the terminal device 2 that manages the image attribute information in the image attribute information management table to transmit the image data managed in the image attribute information management table. Send a signal.

When the processing in step S104 is completed or when it is determined in step S103 that image attribute information has not been received (step S103: NO), the control unit 106 performs image distribution processing (step S105). This image distribution processing generates display image data based on image data acquired from a plurality of terminal devices 2, determines where to display the display image data, and responds to the determination result. This is processing for transmitting display image data to a transmission destination. This process is a process of steps S200 to S215 in FIG. 14, and will be described later.

Next, the control unit 106 determines whether or not an operation command has been received from the terminal device 2 (step S106). In this process, when it is determined that an operation command has been received from the terminal device 2 (step S106: YES), the control unit 106 determines the operation command (step S107). Specifically, the control unit 106 refers to the pointer information management table and acquires the position information of the pointer of the terminal device 2 that has transmitted the operation command. Next, the control unit 106 specifies the terminal device 2 (owner PC) that transmitted the image data by specifying the image data including the acquired position coordinates of the pointer with reference to the image attribute information management table.

When the determination of the operation command is completed in step S107, the control unit 106 transmits the operation command (step S108). That is, the received operation command is transmitted via the interface 103 to the terminal device 2 (owner PC) identified in step S107.

When the process of step S108 is completed or when an operation command is not received from the terminal device 2 in step S106 (step S106: NO), the control unit 106 determines whether or not the power is turned off (step S106). S109) When the power is not turned off (step S109: NO), the process proceeds to step S101. On the other hand, if it determines with the power supply having been turned off (step S109: YES), the control part 106 will complete | finish a main process.

Next, the image distribution processing in step S105 will be specifically described with reference to FIG. FIG. 14 is an image distribution processing flowchart of the server device 1 in the present embodiment.

As shown in FIG. 14, when the image distribution process is started, the control unit 106 of the server device 1 transmits the image data transmitted from the terminal device 2 via the interface 103 in response to the image transmission request signal transmitted in step S104. It is determined whether or not it has been received (step S200). The image data is image data of a window image in the virtual display area 45 of the terminal device 2 that has transmitted the image transmission request signal.

In this process, when it is determined that the image data transmitted from the terminal device 2 has been received (step S200: YES), the control unit 106 stores the image data in a storage area secured in the storage unit 102. Then, based on the image attribute information management table, the control unit 106 causes the image processing circuit 104 to superimpose the window images according to the hierarchical level of the window images, and further superimposes the pointer images to each terminal device. The window image in the second virtual display area 45 and the image data of the pointer are combined to generate display image data (step S201). At this time, the control unit 106 generates display image data having different resolutions. In this way, the control unit 106 combines the image acquisition unit that acquires the image data of the window image in each predetermined display area of the plurality of terminal devices via the communication path, and the plurality of image data acquired by the image acquisition unit. Thus, it functions as an image processing unit that generates a display image. Note that the control unit 106 may perform processing by software by executing a program stored in advance in the storage unit 102 or the like without using the image processing circuit 104.

When the generation of the display image is completed, the control unit 106 detects the operation of the projector 3 (step S202). In this process, the video input circuit of the projector 3 is connected to the image signal output circuit 105 via a cable or the like, and the projector 3 can project and display an image based on the image signal output from the image signal output circuit 105. Sometimes it is determined that the projector 3 is in operation. If the projector 3 is a LAN network compatible projector, the IP address of the projector 3 is stored in advance in the storage unit 102 by operating the operation panel 101, and the IP address is stored in this way. When the projector 3 can communicate with the projector 3 via the interface 103 and the communication path 4, it can be determined that the projector 3 is in an operating state. Thus, the control unit 106 functions as an operation determination unit that determines whether or not the projector 3 as a display device is in an operating state.

In the process of step S202, when the operation of the projector 3 is detected (step S202: YES), the control unit 106 determines whether or not a plurality of projectors 3 are detected (step S203). For example, when the image signal output circuit 105 can supply image signals to a plurality of projectors 3 and a plurality of projectors 3 in an operating state are connected to the image signal output circuit 105, a plurality of projectors 3 are detected. It is determined that When the projector 3 is a LAN network compatible projector, the projector 3 can communicate with each projector 3 having a plurality of IP addresses stored in the storage unit 102 in advance via the interface 103 and the communication path 4. Is determined to be detected.

When a plurality of projectors 3 are detected (step S203: YES), the control unit 106 determines a projector (PJ) 3 that transmits display image data according to the priority (step S204). For example, when the image signal output circuit 105 has first to third image signal transmission interfaces, the priority of these interfaces decreases in the order of first, second, and third. Set to. When the projector 3 is a LAN network compatible projector, the priority order is stored in advance in the storage unit 102 together with its IP address. Then, according to the priority order of the interfaces or the priority order stored in the storage unit 102, the projector 3 that transmits the display image data among the plurality of detected projectors 3 is determined.

When the projector 3 that transmits display image data is determined as described above, or when a plurality of projectors 3 are not detected in step S203 (step S203: NO), the control unit 106 transmits the display image to the projector. The image is transmitted (output) via the image signal output circuit 105 (step S205), and the display image is enlarged and projected on the screen or the like from the projector (PJ) 3. In this way, the control unit 106 performs a transmission process for executing the first mode process of transmitting display image data to the projector 3 via the image signal output circuit 105 when the projector 3 as a display device is in an operating state. It functions as a part. At this time, the transmission processing unit transmits display image data having a resolution corresponding to the transmission destination.

In step S202, when the operation of the projector 3 cannot be detected (step S202: NO), the control unit 106 performs a process of detecting the terminal device 2 connected to the same router device (step S206). That is, among the terminal devices 2 stored in the connection information management table of the storage unit 102, the control unit 106 connected the terminal device 2 that responded to the limited broadcast of the ICMP IP packet and connected to the same router device. The terminal device 2 is determined. In addition, the other method mentioned above may be used for the detection of the terminal device 2 connected to the same router device (same network).

In this process, the terminal device 2 connected to the same router device is detected (step S206: YES).

The control unit 106 transmits a display image obtained by combining the window image and the pointer to the terminal device 2 that has not output the pointer / window via the interface 103 (step S208). Here, the control unit 106 transmits a display image having an image size corresponding to the resolution of the display unit (LCD 200) of the terminal device 2 that has not output the pointer window. Thereby, the suitable image suitable for the display size of the display part of the terminal device 2 can be displayed, and information can be shared more appropriately.

And the control part 106 performs the process of step S208 with respect to all the terminal devices 2 determined to be connected to the same router apparatus (step S209: NO). In this manner, the control unit 106 functions as a transmission processing unit that executes the second mode in which display image data is transmitted to the plurality of terminal devices 2 when the projector 3 that is a display device is not in an operating state.

When this process ends (step S209: YES), when the terminal device 2 connected to the same router apparatus is not detected in step S206 (step S206: NO), or when the process of step S205 ends, control is performed. The unit 106 performs processing for detecting the terminal device 2 connected to a different router device (step S213). That is, among the terminal devices 2 stored in the connection information management table of the storage unit 102, the control unit 106 connected the terminal device 2 that does not respond to the limited IP broadcast of the ICMP IP packet to a different router device. The terminal device 2 is determined. Note that the above-described other method may be used as a method for detecting whether or not the router is connected to the same router device (same network).

In this process, when the terminal device 2 connected to a different router device is detected (step S213: YES), the control unit 106 connects the window image and the display image synthesized with the pointer to the terminal device 2 connected to the different router device. Is transmitted via the interface 103 (step S214). Here, the control unit 106 transmits a display image having an image size corresponding to the resolution of the display unit (LCD 200) of the terminal device 2 connected to a different router device.

When this processing is finished, the control unit 106 determines whether or not the transmission (distribution) of the display image has been finished for all the terminal devices 2 connected to different router devices (step S215), and the different router devices. Until the transmission of the display image to all of the terminal devices 2 connected to is completed (step S215: NO), the process of step S214 is repeated.

When the transmission of the display image is completed for all the terminal devices 2 connected to different router devices (step S215: YES), the terminal device 2 connected to the different router device is not detected in step S213. Is determined (step S213), or when it is determined that the image data transmitted from the terminal device 2 is not received in response to the image transmission request signal transmitted in step S200 (step S200: NO), the control unit 106 Ends the image distribution process.

As described above, when the control unit 106 determines whether or not a specific terminal device is in an operating state among the plurality of terminal devices 2, and determines that the specific terminal device is in an operating state, It functions as a transmission processing unit that transmits display image data to a specific terminal device in an operating state. Further, the control unit 106 functions as a setting unit that sets a terminal device 2 other than the terminal device 2 in the same network as the own device (for example, in the network of the same router) as a specific terminal device. In addition, the control unit 106 determines a specific terminal device based on information on the specific terminal device set and input from the operation panel 101 as an input unit. In addition, the control part 106 transmits the data of the image for a display to all the several terminal devices 2 including a specific terminal device in a 2nd mode as a transmission process part.

(Specific operation of terminal device)
Next, an example of a specific operation of the terminal device 2 will be described with reference to FIGS. 15 and 16. FIG. 15 is a main process flowchart of the terminal device 2 in the present embodiment.

As shown in FIG. 15, when a power switch (not shown) of the terminal device 2 is pressed, the CPU of the control unit 208 reads out and executes the image sharing program stored in the hard disk device 202. Set the function to the operating state. Then, a connection request is made to the server device 1 (step S300). That is, the control unit 208 transmits a connection request signal to the server device 1 via the interface 203 and the communication path 4. Note that the resolution information transmitted from the server device 1 in response to this connection request signal is received and stored in the hard disk device 202. Further, the control unit 208 functions as the virtual display region setting unit 25 and creates the virtual display region 45 as a predetermined display region based on the received resolution information. When the virtual display area 45 is created in this manner, the control unit 208 functions as the operation command transmission unit 27 and notifies the server device 1 that the startup work has been completed.

Next, the control unit 208 determines whether or not the information in the virtual display area 45 has been updated (step S301). For example, when the window image in the display area of the LCD 200 is moved to the virtual display area 45 by the user's operation of the mouse 207 or the like, the control unit 208 determines that the information in the virtual display area 45 has been updated. Further, when the window image in the virtual display area 45 is moved out of the virtual display area 45 or deleted, the control unit 208 detects the change in the position and size of the window image, thereby It is determined that the information in the display area 45 has been updated.

If it is determined that the information in the virtual display area 45 has been updated (step S301: YES), the control unit 208 creates image attribute information according to the processing result and transmits the image attribute information to the server device 1 via the communication path 4. (Step S302). That is, when a new window image is added to the virtual display area 45, among the image attribute information of the added window image, the window ID of the window image, the position information, the size information, and the file name information of the image data And image attribute information notification data including additional flag data of the image attribute information is generated and transmitted to the server device 1. When the window image in the virtual display area 45 is moved out of the virtual display area 45 or deleted, the window ID of the moved or deleted window image and the deletion flag data of the image attribute information are displayed. The image attribute information notification data including it is created and transmitted to the server device 1. When the image attribute information is updated, image attribute information notification data including the image attribute information of the updated window image is generated and transmitted to the server device 1.

When the process of step S302 is completed or when it is determined in step S301 that the information in the virtual display area 45 has not been updated (step S301: NO), the control unit 208 has an operation input in the virtual display area 45. It is determined whether or not (step S303). That is, the control unit 208 determines whether or not an operation input has been performed on the virtual display area 45 with the keyboard 206 or the mouse 207.

In this process, when it is determined that there is an operation input in the virtual display area 45 (step S303: YES), the control unit 208 creates an operation command (step S304). That is, the control unit 208 acquires operation input information based on this operation input, and generates an operation command including the position information of the pointer in the virtual display area 45 and the operation input information. Note that the operation input information is discarded in order to prevent an operation based on the operation input from being executed by the terminal device 2 of its own.

Next, the control unit 208 transmits the operation command generated in step S304 to the server device 1 via the interface 203 and the communication path 4 (step S305).

When the process of step S305 is completed, or when it is determined in step S303 that there is no operation input in the virtual display area 45 (step S303: NO), the control unit 208 moves from the server device 1 to the virtual display area 45. It is determined whether there is a transmission request for image data of a certain window image (step S306). This request for image data is image data request information that the server device 1 transmits to the terminal device 2 based on information in the image attribute information management table.

In this process, when it is determined that there is a request for transmission of image data from the server apparatus 1 (step S306: YES), the control unit 208 generates image data corresponding to the transmission request, and sets the interface 203 and the communication path 4 to each other. Then, the image data is transmitted to the server device 1 (step S307).

When the transmission of the image data in step S307 ends, or when it is determined in step S306 that there is no image data transmission request from the server apparatus 1 (step S306: NO), the control unit 208 starts reception data processing (step S306: NO). Step S308). The received data processing is processing for receiving operation commands and display image data transmitted from the server device 1. This process is a process from steps S400 to S404 in FIG. 16, and will be described later.

When this received data processing is completed, the control unit 208 determines whether or not an operation for ending the image sharing program has been performed by the user's operation (step S309). When it is determined that the end operation of the image sharing program has not been performed (step S309: NO), the control unit 208 proceeds to step S301. On the other hand, if it is determined that the image sharing program termination operation has been performed (step S309: YES), the control unit 208 terminates the process.

Next, the reception data processing in step S309 will be specifically described with reference to FIG. FIG. 16 is a reception data processing flowchart of the server device 1 in the present embodiment.

As shown in FIG. 16, when the reception data processing is started, the control unit 208 determines whether or not an operation command has been received from the server device 1 via the interface 203 (step S400).

In this process, if it is determined that an operation command has been received (step S400: YES), the control unit 208 reproduces an operation input according to the received operation command, so that an application corresponding to the data file to be operated is obtained. The editing operation based on the operation command is executed (step S401).

When the process of step S401 is completed or when it is determined in step S400 that an operation command has not been received (step S400: NO), the control unit 208 determines whether or not display image data has been received. Determination is made (step S402).

If it is determined in step S402 that display image data has been received (step S402: YES), the control unit 208 displays an image based on the display image data on the LCD 200 (step S403). As described above, each terminal device 2 generates an image to be displayed on the LCD 200 using the display image transmitted from the server device 1 and the application program of its own device by the user's operation on the mouse 207 or the like. Any one of the selected images can be selected and displayed.

When it is determined that the display image data is not received when the process of step S403 is completed or in step S402 (step S402: NO), the control unit 208 ends the received data process.

Although several embodiments of the present invention have been described in detail with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that are variously modified and improved based on the knowledge of those skilled in the art. It is possible to implement.

For example, in the above description, when the server device 1 and the terminal device 2 are connected, the terminal device 2 is connected to the server device 1, but the terminal device 2 that is connected in advance to the storage unit 102 of the server device 1. The location information such as the IP address of the server device 1 is stored, and the control unit 106 of the server device 1 requests connection to the terminal device 2 based on the information stored in the storage unit 102. May be connected.

Claims

An image acquisition unit that acquires image data of images in each predetermined display area of a plurality of terminal devices via a communication path, and an image that generates a display image by combining the plurality of image data acquired by the image acquisition unit A server device comprising: a processing unit;
A server apparatus comprising a transmission processing unit having a first mode for transmitting the display image data to a predetermined display device and a second mode for transmitting the display image data to the plurality of terminal devices.
An operation determining unit for determining whether or not the display device is in an operating state;
The transmission processing unit transmits data of the display image in the first mode when the display device is in an operating state, and the display image in the second mode when the display device is not in an operating state. The server apparatus according to claim 1, wherein the data is transmitted.
The operation determining unit determines whether or not a specific terminal device is in an operating state among the plurality of terminal devices,
In the first mode, in addition to the display device, the transmission processing unit further transmits the display image data to the specific terminal device that is in an operating state. In the second mode, the transmission processing unit The server device according to claim 2, wherein the display image data is transmitted to all of the plurality of terminal devices including the specific terminal device.
The communication path is a path through an IP network,
The server apparatus according to claim 3, further comprising a setting unit configured to set a terminal apparatus other than the terminal apparatus in the same network as the own apparatus among the plurality of terminal apparatuses as the specific terminal apparatus.
The server device according to claim 3, further comprising an input unit configured to set and input information of the specific terminal device.
The image processing unit generates a plurality of display image data having different resolutions,
6. The server apparatus according to claim 1, wherein the transmission processing unit transmits the display image data having a resolution corresponding to a transmission destination.
When an operation command for operating a predetermined image of the display image is acquired via the communication path, the operation command is transmitted to the terminal device that has transmitted the image data of the predetermined image that is the target of the operation command. The server device according to any one of claims 1 to 6, further comprising an operation command processing unit that transmits the data via the communication path.
A projector comprising the server device according to any one of claims 1 to 7 and the display device.
A plurality of terminal devices;
An image acquisition unit that acquires image data of images in each predetermined display area of the plurality of terminal devices via a communication path and a plurality of image data acquired by the image acquisition unit are combined to generate a display image. A server device having an image processing unit;
A display system comprising: a display device;
The server device includes a transmission processing unit having a first mode for transmitting the display image data to a predetermined display device and a second mode for transmitting the display image data to the plurality of terminal devices. Display system.