JP2016053681A - Image display system, information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display system configured to transfer an image between a plurality of information processing apparatuses, while reducing computational complexity in image transfer processing by replacing a page with a capture image during transfer.SOLUTION: In an image display system configured by connecting a plurality of information processing apparatuses 1 for displaying an image on a touch panel display, in transferring the image to another information processing apparatus, a capture image is generated by capturing a page displayed on the touch panel display, the capture image is slide-displayed, pixel data of the slid image having reached a boundary of a display area is extracted, and the pixel data is transferred to the other information processing apparatus for display.SELECTED DRAWING: Figure 9

Description

  The present invention relates to an image display system, an information processing apparatus, an information processing method, and a program capable of displaying an image by combining a plurality of independently operable information processing apparatuses and transferring the image between the information processing apparatuses. .

  In recent years, the demand for introducing an electronic blackboard system in school classrooms has been increasing. At the same time, a display (display device) such as a touch panel type liquid crystal display used for an electronic blackboard is also increasing in size, but is generally smaller than a blackboard used in a conventional school. Therefore, in order not to lose convenience when the conventional blackboard is replaced with the electronic blackboard system, it may be required to configure a multi-display system in which a plurality of displays are combined. Thereby, the area of the display area comparable to or more than that of the conventional blackboard can be ensured.

  When a display system is expanded by configuring a system that combines multiple displays, the information entered on one display must be transferred to another display and displayed in order to be used as a whole system. It is desirable that this is possible. Thereby, since the user can use a wide entry space, new information can be entered while displaying the entered information. Moreover, it becomes possible to mutually use information entered by a plurality of users.

JP 2012-185297 A

  Conventionally known multi-display systems include, for example, a configuration in which a plurality of displays are connected to one information processing apparatus. In such a multi-display system, the number of displays that can be connected is limited by the number of terminals of the information processing apparatus. In addition, since one information processing apparatus performs a process of displaying an image on all connected displays, there is a possibility that an excessive load is applied to the information processing apparatus. Furthermore, in order to change the display connection configuration, the user needs to replace the cable.

  On the other hand, Patent Document 1 is composed of a plurality of independent touch panel display terminal devices, can transfer object data as a unit of display processing between adjacent display terminal devices, and can easily change the configuration. A distributed processing multi-display system is disclosed.

  However, in the multi-display system disclosed in Patent Document 1, object data is directly transferred between display terminal devices to display objects, and therefore when the number of objects displayed on the screen increases, image transfer is related. There is a risk that the amount of calculation processing increases.

  Further, the multi-display system disclosed in Patent Document 1 uses a proximity wireless communication interface as a data transfer means. This is because it is assumed that the system of Patent Document 1 is configured by a portable display terminal device. However, the display used in the electronic blackboard system is generally large and often does not include a proximity wireless communication interface. In addition, the amount of data of an image displayed on a large display may increase, and thus there is a possibility that the bandwidth that can be transmitted and received by the close proximity wireless communication interface is exceeded. Therefore, the system of Patent Document 1 may be unsuitable for application to a system using a large display.

  The present invention has been made in view of such circumstances, and an object of the present invention is to reduce the amount of calculation processing by generating and transferring a captured image when transferring an image between information processing apparatuses. Another object of the present invention is to provide an image display system in which the configuration can be easily changed even for a system using a large display, and an information processing apparatus, an information processing method, and a program used in the image display system.

  An image display system according to the present invention is an image display system in which a plurality of information processing devices having a function of displaying an image is connected to each of the plurality of display devices. A communication unit that performs data communication with another information processing device, an image processing unit that performs image processing including generation of a captured image in units of pages, and pixel data of an image displayed on the display device A pixel data extraction unit; and a storage unit for storing pixel data and object data relating to an object displayed on the display device, wherein the image processing unit of the one information processing device is an object displayed on the display device One capture image generated by capturing one page including the image is slurried in the direction related to the operation according to the operation including the direction input to the input unit. The pixel data extraction unit of the one information processing device is configured to display a pixel of a portion of the one captured image that has reached the boundary of the display area of the display device by sliding. Data is sequentially extracted, and the extracted pixel data is sequentially transmitted to another information processing apparatus via the communication unit. The image processing unit of the other information processing apparatus receives the pixel data received by the communication unit. Is displayed as an image on a display device.

  In the image display system according to the present invention, when the communication unit of the other information processing apparatus receives the pixel data from the one information processing apparatus, the image processing unit is displayed on the display device. An image is displayed so as to slide in the direction related to the operation, and the received pixel data is displayed as an image in an area vacated by sliding of the other image in the display area of the display device. It is characterized by being.

  In the image display system according to the present invention, the pixel data extraction unit of the other information processing apparatus sequentially extracts pixel data of the part of the other image that reaches the boundary of the display area by sliding, and extracts the extracted pixels The data is stored in the storage unit.

  In the image display system according to the present invention, when the communication unit of the one information processing apparatus transmits the pixel data to the other information processing apparatus, the object data of the object is also transmitted. The image processing unit of the information processing apparatus displays the object on the display device based on the object data received by the communication unit when the communication unit receives pixel data of all parts of the captured image. It is characterized by the above.

  In the image display system according to the present invention, when the communication unit of the other information processing apparatus receives the pixel data from the one information processing apparatus, the image processing unit displays the object displayed on the display device. Other captured images generated by capturing other pages including the image are displayed on the display device so as to slide in the direction related to the operation, and further, by sliding the other captured images in the display area of the display device The received pixel data is displayed as an image in a vacant area.

  In the image display system according to the present invention, the pixel data extraction unit of the other information processing apparatus extracts the pixel data of the portion of the other captured image that reaches the boundary of the display area by the slide, and extracts the extracted pixels Data and object data of the object are stored in a storage unit.

  In the image display system according to the present invention, when the other information processing apparatus displays a part or all of the one captured image on the display apparatus, the operation input to the one information processing apparatus The image processing unit slides the captured image in the opposite direction according to an operation including a direction opposite to the direction related to the display area, and the pixel data extracting unit slides the display area by sliding the captured image. The pixel data of the part that has reached the boundary is sequentially extracted, and the extracted pixel data is sequentially transmitted to the one information processing device via the communication unit, and the image processing unit of the one information processing device includes: When the communication unit receives the pixel data, the portion of the captured image that is already displayed on the display device is slid in the opposite direction, The received pixel data is displayed as an image in the display area of the display device, and when the entire captured image is displayed on the display device by receiving the pixel data, the image processing unit The object included in the one page is displayed on the display device based on the object data stored in the storage unit.

  In the image display system according to the present invention, when the other information processing apparatus displays part or all of the one captured image on the display apparatus, the operation including the opposite direction input to the input unit In response, the image processing unit slides the one captured image in the opposite direction, and the pixel data extraction unit is configured to slide a boundary of a display area of the display device by sliding among the one captured image. Are sequentially extracted, and the extracted pixel data is sequentially transmitted to the one information processing device via the communication unit, and a part of the other image is displayed on the display device. And when the pixel data relating to the remaining portion is stored in the storage unit, the image processing unit displays the other image in the display area vacated by the slide of the one captured image. Is slid in the opposite direction, and the pixel data stored in the storage unit is displayed as an image in the display area vacated by the slide of the other image, or on the display device. When the other image is not displayed and the pixel data of the other image is stored in the storage unit, the image processing unit displays the display area vacated by the slide of the one captured image. The pixel data stored in the storage unit is displayed as an image, and the image processing unit of the one information processing apparatus receives the pixel data related to the one captured image when the communication unit receives the pixel data related to the one captured image. The portion of the one captured image that is already displayed on the display device is slid in the opposite direction, and is further received in the display area of the display device that is vacated by the slide. The pixel data is displayed as an image, and when the entire captured image is displayed on the display device by receiving the pixel data, the image processing unit is stored in the storage unit. The object included in the one page is displayed on the display device based on the object data.

  In the image display system according to the present invention, when the other information processing device displays the one page on the display device, the image display system performs the image display according to an operation including the opposite direction input to the input unit. The processing unit displays a captured image generated by capturing the one page on the display device so as to slide in the opposite direction, and the pixel data extracting unit includes the one of the one captured images. The pixel data of the part that has reached the boundary of the display area of the display device by the slide is sequentially extracted, and the extracted pixel data is sequentially transmitted to the one information processing device via the communication unit. When the part of the other captured image is displayed and the pixel data relating to the remaining part is stored in the storage unit, the image processing unit The other image is slid in the opposite direction to the display area vacated by the slide, and the pixel data stored in the storage unit is further imaged in the display area vacated by the slide of the other captured image. When the other capture image is not displayed on the display device and the pixel data of the other capture image is stored in the storage unit, the image processing unit Displays the pixel data stored in the storage unit as an image in the display area vacated by the slide of the one captured image, and all of the other captured images are displayed in the one capture image. When the image is displayed in the display area vacated by the slide of the image, the image processing unit displays the other page stored in the storage unit. The object is displayed on the display device based on the object data, and the image processing unit of the one information processing device receives the pixel data related to the one captured image when the communication unit receives the pixel data related to the one captured image. The part already displayed on the display device of one captured image is slid in the opposite direction, and the received pixel data is displayed as an image in the display area of the display device vacated by the slide, When the entire captured image is displayed on the display device by receiving the pixel data, the image processing unit is included in the one page based on object data stored in a storage unit. The displayed object is displayed on the display device.

  In the image display system according to the present invention, when the communication unit of the other information processing apparatus receives the pixel data from the one information processing apparatus, the image processing unit uses the received pixel data as its own information processing. It is characterized by being overwritten with another page displayed on the display device of the apparatus and displayed as an image.

  In the image display system according to the present invention, the pixel data extraction unit of the other information processing apparatus extracts the pixel data of the overwritten portion of the other page, and stores the extracted pixel data in the storage unit. It is characterized by that.

  In the image display system according to the present invention, when the object included in the page is a moving image object, the image processing unit pauses the reproduction of the moving image object in response to an operation including the direction input to the input unit. The captured image is generated based on the stopped state, and when the entire captured image is transmitted from one information processing apparatus to another information processing apparatus and displayed on the display apparatus, the image The processing unit is configured to reconstruct and display the moving image object on the page, and to start playback from the time when the processing is paused.

  In the image display system according to the present invention, when a page is transmitted from one information processing apparatus to another information processing apparatus by a slide, the image processing unit of the information processing apparatus as a transmission destination has the resolution of the display apparatus. In addition, the page can be enlarged or reduced and displayed on the display device.

  In the image display system according to the present invention, the display device is a touch panel display, and the input unit is a touch panel.

  An information processing device according to the present invention has a function of displaying an image on a display device, and an information processing device that can configure an image display system by connecting a plurality of information processing devices; A communication unit that performs data communication with other information processing devices, an image processing unit that performs image processing including generation of a captured image in units of pages, and pixel data that extracts pixel data of an image displayed on the display device An extraction unit; and a storage unit that stores pixel data and object data relating to an object displayed on the display device, wherein the image processing unit displays a page including the object displayed on the display device. The captured image generated by the capture is displayed on the display device so as to slide in the direction related to the operation according to the operation including the direction input to the input unit. The pixel data extraction unit sequentially extracts pixel data of a portion of the captured image that has reached the boundary of the display area of the display device by sliding, and the extracted pixel data is transmitted via the communication unit. The image processing unit is further configured to display the received pixel data as an image on the display device when the communication unit receives the pixel data. It is characterized by that.

  An information processing method according to the present invention is an information processing method of an image display system formed by connecting a plurality of information processing devices having a function of displaying an image on a display device, and each of the information processing devices receives an operation input Input unit, a communication unit that communicates data with other information processing devices, an image processing unit that performs image processing including generation of a captured image in units of pages, and pixel data of an image displayed on the display device A pixel data extraction unit that extracts the pixel data and a storage unit that stores object data relating to the object displayed on the display device, and the image processing unit of the one information processing device is displayed on the display device. The captured image generated by capturing one page including the object that is present is slid in the direction related to the operation according to the operation including the direction input to the input unit. The step of displaying on the display device, and the pixel data extraction unit of the one information processing device sequentially extracts the pixel data of the portion of the captured image that has reached the boundary of the display region of the display device by sliding, A step of sequentially transmitting the extracted pixel data to another information processing apparatus via a communication unit, and an image processing unit of the other information processing apparatus displays the pixel data received by the communication unit as an image on a display device And a step.

  A program according to the present invention is a program that includes an input unit and a communication unit, and causes an information processing device to display an image on a display device to process information, and according to an operation including a direction received by the input unit, Capturing a page including an object displayed on the display device to generate a captured image; displaying the generated captured image on the display device so as to slide in a direction related to the operation; and the capture A step of extracting pixel data of a portion of the image that reaches the boundary of the display area of the display device by sliding and an information processing device are executed.

  According to the present invention, the amount of calculation processing can be reduced, and the configuration can be easily changed even for a system using a large display.

1 is a schematic view showing an entire image display system according to the present invention. It is a block diagram which shows the hardware constitutions of the information processing apparatus which concerns on this invention. It is a block diagram showing the information processor concerning the present invention by functional part composition. It is a figure which shows the setting method for displaying an image in cooperation with several information processing apparatus. It is a figure which shows the setting method for displaying an image in cooperation with several information processing apparatus. 4 is a flowchart showing an operation procedure on each of a host side and a guest side in image transfer processing from the host side to the guest side in the multi-display system according to the first embodiment. 4 is a flowchart showing an operation procedure on each of a host side and a guest side in image transfer processing from the host side to the guest side in the multi-display system according to the first embodiment. 6 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to Embodiment 1. FIG. It is a flowchart which shows the operation | movement procedure of the slide process P1 which is a subroutine in the flowchart of FIG. It is a flowchart which shows the operation | movement procedure of the pixel data extraction process P2 which is a subroutine in the flowchart of FIG. It is a flowchart which shows the operation | movement procedure of the pixel data extraction process P2 which is a subroutine in the flowchart of FIG. It is a flowchart which shows the operation | movement procedure of the pixel data extraction process P2 which is a subroutine in the flowchart of FIG. It is a flowchart which shows the operation | movement procedure of the pixel data extraction process P2 which is a subroutine in the flowchart of FIG. It is a flowchart which shows the operation | movement procedure of the pixel data extraction process P2 which is a subroutine in the flowchart of FIG. It is a conceptual diagram which shows the process image of the pixel data extraction process P2. It is a conceptual diagram which shows the process image of the pixel data extraction process P2. It is a conceptual diagram which shows the process image of the pixel data extraction process P2. 6 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to Embodiment 1. FIG. 6 is a flowchart showing an operation procedure on each of the host side and the guest side in the reverse image transfer process from the guest side to the host side in the multi-display system according to the first embodiment. 6 is a conceptual diagram illustrating an operation image of image reverse transfer processing from the guest side to the host side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image reverse transfer processing from the guest side to the host side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image reverse transfer processing from the guest side to the host side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image reverse transfer processing from the guest side to the host side in the multi-display system according to Embodiment 1. FIG. 6 is a conceptual diagram illustrating an operation image of image reverse transfer processing from the guest side to the host side in the multi-display system according to Embodiment 1. FIG. 10 is a flowchart showing an operation procedure on each of a host side and a guest side in image transfer processing from the host side to the guest side in the multi-display system according to the second embodiment. 10 is a flowchart showing an operation procedure on each of a host side and a guest side in image transfer processing from the host side to the guest side in the multi-display system according to the second embodiment. FIG. 10 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to the second embodiment. FIG. 10 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to the second embodiment. 10 is a flowchart illustrating an operation procedure on each of the host side and the guest side in the reverse image transfer process from the guest side to the host side in the multi-display system according to the second embodiment. FIG. 10 is a conceptual diagram illustrating an operation image on each of a host side and a guest side in an image reverse transfer process from the guest side to the host side in the multi-display system according to the second embodiment. 15 is a flowchart illustrating an operation procedure on the guest side in image transfer processing from the host side to the guest side in the multi-display system according to the third embodiment. FIG. 10 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to the third embodiment. It is a flowchart which shows the operation | movement procedure of the pixel data extraction process P3 which is a subroutine in the flowchart of FIG. 16 is a flowchart illustrating an operation procedure on the guest side in image transfer processing from the host side to the guest side in the multi-display system according to the fourth embodiment. FIG. 16 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system according to the fourth embodiment. 16 is a flowchart showing an operation procedure on each of a host side and a guest side in image transfer processing from the host side to the guest side in the multi-display system according to the fifth embodiment. 16 is a flowchart showing an operation procedure on each of a host side and a guest side in image transfer processing from the host side to the guest side in the multi-display system according to the fifth embodiment.

  Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.

(Embodiment 1)
FIG. 1 is a schematic diagram showing the entire image display system according to the present invention. In the following description of the embodiments, the image display system is referred to as a multi-display system. The multi-display system 100 is configured by connecting a plurality of information processing apparatuses 1 that are integrated with a touch panel display, for example, via a wired LAN network via a router, for example. Here, the connection method of the information processing apparatus 1 is not limited to the wired LAN, and may be a wireless LAN or another network. In addition, the information processing apparatus 1 is not limited to the touch panel display integrated type illustrated in FIG. 1, and may be connected to the touch panel display by a video output cable or a wireless video transfer system. In the present embodiment, the display resolution of the touch panel display of each information processing apparatus 1 is, for example, the same.

  FIG. 2 is a configuration diagram showing a hardware configuration of the information processing apparatus 1 according to the present invention. In FIG. 2, 2 is a control device, and 3 is a touch panel display.

  The control device 2 includes a CPU (Central Processing Unit) 20, a memory 21, a storage 22, a communication IF (interface) 23, an image output IF 24, and a control I / O (input / output) 25. The memory 21, storage 22, communication IF 23, image output IF 24, and control I / O 25 are connected to the CPU 20. The control device 2 performs various information processing based on various operations input to the information processing device 1 by the user and signals received from other devices via the network, and the processed information is displayed on the touch panel display 3 or the network. Output to.

  The memory 21 includes a RAM such as a DRAM or an SRAM, and temporarily stores data processed by the information processing apparatus 1 and data generated by the processing. The storage 22 is a nonvolatile storage means such as an EEPROM (Electrically Erasable and Programmable ROM), a flash memory, an HDD (Hard Disk Drive) or the like. The storage 22 stores a program for operating the information processing apparatus 1 and necessary data, data created by a user using the information processing apparatus 1, data acquired from the outside, and the like.

  The CPU 20 reads out a program stored in the storage 22 to the memory 21 and performs various processes by reading and executing such a program. The CPU 20 controls the operation of each unit of the information processing apparatus 1 according to the instructions of the program, processes input data by the user, and displays an image on the touch panel display 3.

  The communication IF 23 transmits / receives data to / from external apparatuses including other information processing apparatuses via the network. For example, when the network is a wired LAN, the communication IF 23 includes a LAN port and a network controller, and when the network is a wireless LAN, the communication IF 23 includes an antenna and a wireless controller.

  The image output IF 24 is an IC for outputting image data generated by the CPU 20 in order to display an image on the touch panel display 3, for example.

  The control I / O 25 is an IC for inputting / outputting data related to the operation of the information processing apparatus 1 with the touch panel display 3, for example.

  The touch panel display 3 is configured by combining a display 30 and a touch sensor 33. In addition, the touch panel display 3 includes a display controller 31, an image input IF 32, and a touch panel I / O 34. The display controller 31 is connected to the display 30 and the image input IF 32.

  The display 30 is a liquid crystal display device, for example, and displays the image data generated by the CPU 20 as an image. The touch sensor 33 is a resistive film type, capacitance type, infrared type touch sensor or the like, and detects that the user touches the touch panel display 3 with a finger or a stylus pen and outputs the touch signal.

  The display controller 31 is, for example, a processor that controls the display 30. Image data is input from the image output IF 24 to the image input IF 32. The image input IF 32 outputs the input image data to the display controller 31. The display controller 31 controls the display 30 so that an image based on the input image data is displayed on the display 30.

  A touch panel I / O 34 is connected to the touch sensor 33. The touch sensor 33 receives an instruction signal from the control device 2 that is input via the control I / O 25 and the touch panel I / O 34. The touch sensor 33 operates according to an instruction signal input from the control device 2. The touch sensor 33 outputs the detected touch signal to the CPU 20 of the control device 2 via the touch panel I / O 34 and the control I / O 25. Based on the touch signal detected by the touch sensor 33, the CPU 20 detects the position on the touch panel display 3 touched by the user.

  FIG. 3 is a block diagram showing the information processing apparatus 1 according to the present invention in terms of functional unit configuration. The information processing apparatus 1 includes a control unit 40, a storage unit 41, an image processing unit 42, a pixel data extraction unit 43, a communication unit 44, a display unit 45, and an input unit 46 as functional units.

  The control unit 40, the image processing unit 42, and the pixel data extraction unit 43 are functional units realized by the CPU 20 executing a program. The storage unit 41 includes the memory 21, the communication unit 44 includes the communication IF 23, the display unit 45 includes the display 30 and the display controller 31, and the input unit 46 includes the touch sensor 33.

  The control unit 40 performs operation control of the information processing apparatus 1 including control of each functional unit and hardware. The storage unit 41 stores pixel data of an image displayed on the display unit 45 and object data of an object. Here, an object is a unit for creating and editing an image. Specifically, the object is a character string, a figure, a photograph, or the like. The object data is data necessary for displaying the object on the display unit 45. When a new object is created or when an object is changed (that is, when the object is edited), the object data of the object is stored in the storage unit 41.

  The image processing unit 42 performs image processing of an image displayed on the display unit 45. In particular, the image processing unit 42 performs image processing in units of pages. Here, the page is a unit when the information processing apparatus 1 displays an image. For example, the size of one page can be approximately equal to the size of the display area of the display unit 45. The image processing performed by the image processing unit 42 includes processing for capturing a page and generating the captured image. Further, the image processing unit 42 creates a new object on the page and changes the created object in accordance with a user operation.

  The pixel data extraction unit 43 extracts pixel data of a specific portion of the image displayed on the display unit 45 from the storage unit 41 in which the pixel data is stored. The communication unit 44 transmits / receives data including pixel data and object data to / from another information processing apparatus 1 via a network. An operation by the user is input to the input unit 46.

  With the above configuration, the information processing apparatus 1 according to the present invention forms an image based on an input by a user's touch operation on the touch panel display 3 and displays the formed image on the touch panel display 3. Further, the information processing apparatus 1 performs image processing based on a user's touch operation on the displayed image. Thereby, the user can create and edit an image intuitively by touch input.

  Furthermore, a plurality of information processing apparatuses 1 are connected via a network to form a multi-display system 100, and images can be displayed and edited by transferring and displaying images between the information processing apparatuses. It is possible to enlarge. Thereby, the area in which an image can be displayed and edited is not limited by the size of a single display, and the convenience for the user is improved. Each information processing device 1 that constitutes the multi-display system 100 displays images in units of pages and transfers them to other information processing devices 1, thereby making it possible to perform image transfer processing as described later. A processing method capable of reducing the amount of calculation can be realized.

  Next, an operation performed by the user to configure the multi-display system 100 in cooperation with a plurality of information processing apparatuses 1 will be described with reference to FIGS. 4 and 5.

  4 and 5 are diagrams illustrating a setting method for displaying an image in cooperation with a plurality of information processing apparatuses 1. Here, the two information processing apparatuses 1 shown in FIGS. 4 and 5 are connected to each other. Of the two information processing apparatuses 1 shown in FIGS. 4 and 5, for example, the right side is the host and the left side is the guest.

  First, as shown in FIG. 4, the user displays a setting screen for setting a plurality of units on the host side. When the cooperation destination setting is started on the host side, a list of IP addresses of other information processing apparatuses 1 (guests) connected to the same network is displayed on the touch panel display 3. The user selects a guest to be linked with the host from the displayed list of information processing apparatuses 1 and touches and specifies the touch panel display 3. Note that the user can arbitrarily determine which information processing apparatus 1 on the network is the host. For example, the information processing apparatus 1 that first opens the setting screen may automatically be a host.

  After that, as shown in FIG. 5, the selected cooperation destination guests are arranged side by side on the screen on the host side. On this screen, the user specifies the destination guest by touch operation, and drags and rearranges to specify that image transfer processing is performed between the information processing devices according to the actual host and guest arrangement. Can do. With the above operation, the setting for configuring the multi-display system 100 is completed.

  The arrangement setting of the plurality of information processing apparatuses 1 is not limited to the horizontal direction as shown in FIGS. 4 and 5 and may be a vertical direction or an oblique direction. Further, the arrangement of the information processing apparatus 1 on the setting in the multi-display system 100 and the actual arrangement are not necessarily the same.

  In the multi-display system 100 according to the present embodiment, the guest is used only for viewing images, and is set so that the user cannot create or edit an object on the guest side.

  Hereinafter, the operation of the multi-display system 100 according to the present embodiment will be described with reference to the drawings.

  6 and 7 are flowcharts showing operation procedures on the host side and the guest side in the image transfer processing from the host side to the guest side in the multi-display system 100 according to the first embodiment. FIGS. 8, 9, and 19 to 21 are conceptual diagrams showing an operation image of image transfer processing from the host side to the guest side in the multi-display system 100 according to the first embodiment. FIG. 10 is a flowchart showing an operation procedure of slide processing P1, which is a subroutine in the flowchart of FIG. FIGS. 11 to 15 are flowcharts showing the operation procedure of the pixel data extraction process P2 which is a subroutine in the flowchart of FIG. FIGS. 16 to 18 are conceptual diagrams showing processing images of the pixel data extraction processing P2.

  In the following, an example will be described in which the right one is set as a host and the left one is set as a guest among the adjacent information processing apparatuses 1 shown in the drawing in the above-described cooperation setting. Further, for the sake of simplicity, an example will be described in which the slide display of the page from the host side to the guest side is in the horizontal direction from right to left. However, the arrangement positions of the host and the guest are not limited, and the slide display of the page can be performed in the horizontal direction from the left to the right, the vertical direction, or the diagonal direction.

  Furthermore, each pixel of the image displayed on the display unit 45 has an image coordinate (x, y) which is a position on the image and a position in the display area of the display unit 45 to be displayed, that is, a pixel position of the display 30. Display coordinates (X, Y). In the following description, coordinate variables indicated by lowercase letters relate to image coordinates, and coordinate variables indicated by uppercase letters relate to display coordinates.

  As a first case, as shown in FIG. 8, an operation when a page including an object is displayed on the host of the multi-display system 100 and no page is displayed on the guest will be described. First, the procedure of image transfer processing on the host side will be described. In the state illustrated in FIG. 8, the control unit 40 of the information processing apparatus 1 on the host side determines whether an operation of tracing the touch panel display 3 such as flick or swipe is input to the input unit 46 by the user (Step S <b> 1). .

  When it is determined that the flick or swipe operation by the user has not been input to the input unit 46 (S1: NO), the host-side control unit 40 repeats step S1 until it is determined that it has been input. When the control unit 40 determines that a flick or swipe operation has been input to the input unit 46 (S1: YES), the image processing unit 42 displays a page including the object displayed on the display unit 45 as the image processing unit 42. Replaces the captured image generated by capturing the page and displays it on the display unit 45 (step S2). Here, the captured image is a still image compressed by a compression method such as JPEG or PNG. Further, when replacing with a captured image, the object data of the object displayed on the original page is stored in the storage unit 41.

  The timing at which the image processing unit 42 captures the page including the object and generates the captured image can be set, for example, when the user newly creates or edits the object. In this case, the image processing unit 42 captures a page and saves the captured image in the storage unit 41 every time a user newly creates or edits an object. The image processing unit 42 replaces the page displayed on the display unit 45 with the latest captured image stored in the storage unit 41 in response to a flick or swipe operation by the user. Alternatively, the image processing unit 42 may be configured to generate a captured image in response to a flick or swipe operation performed by the user input to the input unit 46.

  FIG. 9 shows the state of the guest and the host that are executing the image transfer process between the information processing apparatuses 1. After replacing the page with the captured image in step S2, the image processing unit 42 determines the number of pixels per frame determined according to the operation speed, for example, in the direction in which the user inputs the captured image by flicking or swiping operation. The captured image slide processing P1 to be slid on the display unit 45 is performed (step S3).

  With reference to FIG. 10, the operation procedure of the slide process P1, which is a subroutine executed by the image processing unit 42, will be described. In the following description, as an example, the coordinates of the display area of the display unit 45, that is, the origin of the display coordinates is set at the upper left of the display area, the right direction is the positive direction of the horizontal X axis, and the downward direction is vertical. It is assumed that the positive direction of the Y axis is set. Similarly, the origin of the image coordinates is also set to the upper left, and it is assumed that the right direction is the positive direction of the x axis and the downward direction is the positive direction of the y axis.

The image processing unit 42 acquires the touch position (X _n , Y _n ) by the user on the display area at the start of the frame detected by the input unit 46 (step S200). Here, the frame start time is, for example, a timing synchronized with a vertical synchronization signal of the display 30. Next, the image processing unit 42 determines whether or not the touch position (X _n−1 , Y _n−1 ) acquired at the start of the previous frame is stored in the storage unit 41 (step S201).

If it is determined that (X _n−1 , _Yn−1 ) is stored (S201: YES), the image processing unit 42 reads (X _n−1 , Y _n−1 ) from the storage unit 41 (step S202). ). Subsequently, the image processing unit 42 slides the image ( _{S s} , Y _s ) to display the image from the difference between the acquired (X _n , Y _n ) and the read (X _n−1 , Y _n−1 ). ) Is calculated (step S203). More specifically, the image processing unit 42 calculates the _{X s} by subtracting the _{X n-1} from the _{X n,} and calculates the _{Y s} by subtracting the _{Y n-1} from _{Y n.}

On the other hand, when it is determined that (X _n−1 , Y _n−1 ) is not stored (S201: NO), the image processing unit 42 stores (X _s , Y _s ) calculated by the processing in the previous frame. It is determined whether or not it is stored in the unit 41 (step S204). _{(X s, Y} s) if it is determined that the stored (S204: YES), the image processing unit 42 of the previous frame stored _{(X s, Y} s) new from _{(X s, Y} s ) Is calculated (step S205). Specifically, if you have configured an image by flick operation so as to inertial scrolling, for example a (X _{s, Y s)} of the previous frame obtained by subtracting a certain percentage of the new (X _s, Y s) And On the other hand, when it is configured not to cause inertial scrolling, both X _s and Y _s are set to 0.

After calculating (X _s , Y _s ) in step S203 or S205, the image processing unit 42 adds (X _s , Y _s ) to each display coordinate value of each pixel of the image to be slid (step S206). Thus, a new display coordinate value is calculated. The image processing unit 42 stores the display coordinate value after such addition in the storage unit 41 (step S207). Subsequently, the image processing unit 42 displays an image on the display unit 45 based on the display coordinate value after the addition (step S208).

After displaying the image, the image processing unit 42 stores (X _s , Y _s ) calculated in step S205 in the storage unit 41 (step S209). After storing (X _s , Y _s ), or when it is determined in step S204 that (X _s , Y _s ) of the previous frame is not stored (S204: NO), the image processing unit 42 returns to step S200. The acquired (X _n , Y _n ) is stored in the storage unit 41 (step S210). Thereafter, the image processing unit 42 ends the slide process P1 and returns the process to the main routine.

  After the slide processing P1 is executed in step S3 to display the slide image, the pixel data extraction unit 43 is a subroutine for extracting pixel data of pixels that have reached the boundary of the display area due to the slide display of the image. A certain pixel data extraction process P2 is executed (step S4).

The operation procedure of the pixel data extraction process P2 executed by the pixel data extraction unit 43 will be described with reference to the flowcharts of FIGS. 11 to 15 and the conceptual diagrams of FIGS. Direction in which the image is caused to slide, by comparing the X _s and Y _s and 0, can be classified horizontal, vertical, and diagonal directions. The guest to which the data is transferred differs depending on the direction in which the image slides. Therefore, three address areas can be set in the storage unit 41 that stores the extracted pixel data corresponding to the three types of classifications in the sliding direction.

Pixel data extracting section 43, the image processing unit 42 compares the values and 0 _{X s} calculated in slide processing P1 (step S300). Pixel data extracting unit 43 in step S300, either _{X s} is less than 0, it determines whether 0 is equal to a larger or 0.

If X _s is determined smaller than 0 (S300: <0), that is, when the horizontal component of the image of the slide is the left direction, the pixel data extracting section 43, the pixel data extraction processing in the previous frame in the storage unit 41 It is determined whether or not the variable _Lorder in P2 is stored (step S301). Here, L _border is a variable representing the position of the left boundary of the display area in the slid image by the image x coordinate. When it is determined that L _border is stored (S301: YES), the pixel data extraction unit 43 substitutes L _border for the counter c_x related to the image x coordinate (step S302). That is, c_x is initialized. This is because a loop process is performed on the image x coordinate. The fact that L _border is stored in the storage unit 41 means that the pixel data extraction process P2 is executed in the previous frame, and the pixel data of the pixel that has reached the left boundary of the display area is extracted and stored. In such a case, c_x is initialized with the value of L _{border in} order to extract pixel data of the difference area that has newly reached the left boundary in the current frame. On the other hand, when it is determined that L _border is not stored in the storage unit 41 (S301: NO), the pixel data extraction unit 43 substitutes x _min for c_x (step S303) and initializes c_x. Here, x _min is the minimum value of the image x coordinate, and is 0, for example.

After initializing the c_x, pixel data extracting section 43, the image processing unit 42 compares the _{Y s} and 0 calculated in slide processing P1 (step S304). That is, as in step S300, it is determined whether Y _s is less than 0, greater than 0, or equal to 0.

If Y _s is determined smaller than 0 (S304: <0), the sliding direction of the image is the upper left direction of the display unit 45 (FIG. 16). In such a case, the pixel data extraction unit 43 determines whether or not T _border is stored in the storage unit 41 as in step S301 (step S305). Here, T _border is a variable representing the position of the upper boundary of the display area in the slid image by the image y coordinate. When it is determined that T _border is stored (S305: YES), the pixel data extraction unit 43 substitutes T _border for the counter c_y related to the image y coordinate (step S306), and initializes c_y. The fact that T _border is stored in the storage unit 41 means that the pixel data extraction process P2 is executed in the previous frame, and the pixel data of the pixel that has reached the upper boundary of the display area is extracted and stored. On the other hand, when it is determined that T _border is not stored in the storage unit 41 (S305: NO), the pixel data extraction unit 43 substitutes y _min for c_y (step S307). Here, y _min is the minimum value of the image y coordinate, and is 0, for example.

  As shown in FIG. 16, in the pixel data extraction process P2, the pixel data of the difference area between the boundary position of the display area in the image in the previous frame and the boundary position of the display area in the image in the current frame is extracted. To do. When the slide direction of the image is the upper left direction, the pixel data extraction unit 43 extracts the pixel data in order from the upper left of the image.

After the substitution to c_y, the pixel data extraction unit 43 determines whether or not the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is smaller than X _MIN (step S308). Here, X _MIN is the minimum value of the X coordinate of the display area of the display unit 45, that is, the display X coordinate of the left boundary of the display area. By comparing the display X coordinate of the pixel of the image with X _MIN , the pixel data extraction unit 43 can determine whether or not the pixel has reached the left boundary of the display area.

If it is determined that the display X coordinate of the pixel is smaller than X _MIN (S308: YES), the pixel data extraction unit 43 further determines whether the display Y coordinate of the pixel whose image coordinates are (c_x, c_y) is smaller than Y _MIN . It is determined whether or not (step S309). Here, Y _MIN is the minimum value of the Y coordinate of the display area of the display unit 45, that is, the display Y coordinate of the upper boundary of the display area. By comparing the display Y coordinate of the pixel of the image with Y _MIN , the pixel data extraction unit 43 can determine whether or not the pixel has reached the upper boundary of the display area.

When it is determined that the display Y coordinate of the pixel is smaller than Y _MIN (step S309: YES), that is, when the pixel reaches both the left and upper boundaries of the display area, the pixel data extraction unit 43 Pixel data of pixels, for example, RGB values and image coordinates are extracted from the stored storage unit 41 (step S310), and stored in a third area which is a different address area of the storage unit 41 (step S311). Here, the third area is an address area for storing pixel data of pixels that have reached two boundaries in the horizontal and vertical directions.

  After storing the pixel data in the third region, the pixel data extraction unit 43 increments c_y (increases the value by 1) (step S312), and repeats the determination in step S309 and the subsequent pixel data extraction process.

If the pixel data extraction unit 43 determines that the display Y coordinate of the pixel whose image coordinates are (c_x, c_y) is greater than or equal to Y _MIN as a result of repeatedly incrementing c_y in step S312 (S309: NO). The extraction unit 43 extracts the pixel data of the pixel from the storage unit 41 (step S313), and stores it in a first area that is a different address area of the storage unit 41 (step S314). Here, the first area is an address area for storing pixel data of a pixel that has reached one of the left and right boundaries. After storing the pixel data in the first area, the pixel data extraction unit 43 increments c_y (step S315). Thereafter, the pixel data extraction unit 43 determines whether c_y is larger than y _max that is the maximum value of the image y coordinate (step S316). In step S316, it is determined whether or not the loop processing related to c_y has reached the end point.

When it is determined that c_y is equal to or less than y _max (S316: NO), the pixel data extraction unit 43 returns the process to step S313 and sequentially repeats the extraction of pixel data in one column in the vertical direction. On the other hand, if the pixel data extraction unit 43 determines that c_y is greater than y _max as a result of repeating increments of c_y (S316: YES), the pixel data extraction unit 43 ends the loop processing for c_y. c_x is incremented (step S317), and the process returns to step S305. As a result, the pixel data extraction unit 43 sequentially repeats the extraction process for pixel data in a vertical direction with different image x coordinates.

When it is determined in step S308 that the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is greater than or equal to X _MIN (S308: NO), the pixel data extraction unit 43 sets c_x to the variable L _border . Is substituted (step S318), and L _border is updated. Results c_x was repeated incremented from the initial value, the image coordinates (c_x, c_y) value of c_x when the display X coordinate of the pixel which is is equal to X _MIN is, the position of the left boundary of the display area in the image It is. Then the pixel data extracting section 43, the image coordinates (c_x, c_y) display Y coordinate of the pixel is determines whether _{Y MIN} is smaller than (step S319).

When it determines with it being small (S319: YES), the pixel data extraction part 43 substitutes L _border for c_x (step S320), and initializes c_x. The pixel data extraction unit 43 extracts the pixel data of the pixel at the image coordinates (c_x, c_y) from the storage unit 41 (step S321), and stores the extracted pixel data in a second area that is a different address area of the storage unit 41 (step S322). . Here, the second area is an address area for storing pixel data of a pixel that has reached either the upper or lower boundary. After storing the pixel data in the second area, the pixel data extraction unit 43 increments c_x (step S323).

Thereafter, the pixel data extraction unit 43 determines whether c_x is larger than x _max that is the maximum value of the image X coordinates (step S324). In step S324, it is determined whether or not the loop processing related to c_x has reached the end point. If c_x is equal to or less than _{x max} (S324: NO), the pixel data extracting section 43 returns the process to step S321, sequentially repeating the extraction of pixel data in the horizontal direction line. Meanwhile, as a result of the increment of c_x is repeated, if the pixel data extracting section 43, the c_x is determined to be greater than _{x max} (Step S324: YES), the pixel data extraction unit 43 ends the loop processing for c_x C_y is incremented (step S325), and the process returns to step S319. That is, the extraction process is sequentially repeated with respect to pixel data in a horizontal row of different image y coordinates.

When it is determined that the display Y coordinate of the pixel whose image coordinates are (c_x, c_y) is greater than or equal to Y _MIN as a result of repeating the increment of c_y in step S325 (S319: NO), the pixel data extraction unit 43 finishes extracting the pixel data of all the pixels that have reached the boundary. In such a case, the pixel data extraction unit 43 substitutes the value of c_y for T _border (step S326), and updates T _border . As a result of repeating incrementing from the initial value of c_y, the value of c_y when the display Y coordinate of the pixel whose image coordinates are (c_x, c_y) becomes equal to Y _MIN is the position of the upper boundary of the display area in the image It is. The pixel data extraction unit 43 stores the updated T _border in the storage unit 41 (step S327). Subsequently, the pixel data extraction unit 43 stores L _border in the storage unit 41 (step S328).

After storing T _border and L _border , the pixel data extraction unit 43 ends the pixel data extraction process P2 which is a subroutine, and returns the process to the main routine.

In step S304, when it is determined that _{Y s} is equal to 0 (S304: = 0), the sliding direction of the image does not include a vertical component, the image is slid in the horizontal direction left (Figure 17). In such a case, the pixel data extraction unit 43 substitutes y _min for c_y (step S329). Accordingly, the pixel data extraction unit 43 performs a loop process in order from the minimum value on the image y coordinate. After substituting y _min for c_y, the pixel data extraction unit 43 determines whether or not the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is smaller than X _MIN (step S330). When it is determined that the pixel is small (S330: YES), that is, when the pixel has reached the left boundary of the display area, the pixel data extraction unit 43 stores the pixel data of the pixel at the image coordinates (c_x, c_y) as the storage unit 41. (Step S331) and stored in the first area of the storage unit 41 (step S332).

Thereafter, the pixel data extraction unit 43 increments c_y (step S333). Subsequently, the pixel data extraction unit 43 determines whether or not the incremented c_y is greater than y _max (step S334). When it is determined that c_y is equal to or less than y _max (S334: NO), the pixel data extraction unit 43 returns the process to step S331, and sequentially repeats the pixel data extraction process for the pixels in one column in the vertical direction. On the other hand, when the pixel data extraction unit 43 determines that c_y is larger than y _max as a result of repeating the increment of c_y (S334: YES), the pixel data extraction unit 43 increments c_x (Step S334). In step S335, the process returns to step S329. That is, the extraction process is sequentially repeated for pixel data in a vertical row of different image x coordinates.

When it is determined that the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is greater than or equal to X _MIN as a result of repeating the increment of c_x in step S335 (S330: NO), the pixel data extraction unit 43 finishes extracting the pixel data of all the pixels that have reached the boundary. In such a case, similarly to step S318, the pixel data extraction unit 43 assigns and updates the value of c_x to L _border (step S336), and stores the updated L _border in the storage unit 41 (step S328).

Pixel data extraction unit 43 in step S304 is, when it is determined that _{Y s} is greater than 0 (S304:> 0), the image is slid in the lower left direction. In such a case, the pixel data extraction unit 43 executes the processing from step S337 to S360. The processing from step S337 to S360 corresponds to the processing from step S305 to S328 when the image slides in the upper left direction. Therefore, common parts will be omitted and described briefly.

When it is determined that Y _s is greater than 0 (S304:> 0), the pixel data extraction unit 43 determines whether or not B _border is stored in the storage unit 41 (step S337). Here, B _border is a variable representing the position of the lower boundary of the display area in the slid image by the image y coordinate. When it is determined that B _border is stored (S337: YES), the pixel data extraction unit 43 substitutes B _border for the counter c_y related to the image y coordinate (step S338), while B _border is not stored. (S337: NO), the pixel data extraction unit 43 substitutes y _max for c_y (step S339). Here, y _max is the maximum value of the image y coordinate.

The determination in step S340 is the same as in step S308. Subsequently, in step S341 determines, even pixel data extracting section 43, the image coordinates (c_x, c_y) display Y coordinate of the pixel is as to whether or not greater than _{Y MAX.} Here, Y _MAX is the maximum value of the Y coordinate of the display area of the display unit 45, that is, the display Y coordinate of the lower boundary of the display area. By comparing the display Y coordinate of the pixel of the image with Y _MAX , the pixel data extraction unit 43 can determine whether or not the pixel has reached the lower boundary of the display area.

  Steps S342 to S344 correspond to steps S310 to S312. However, the difference is that c_y is decremented (the value is decreased by 1) in step S344. This is because the pixel data extraction unit 43 sequentially extracts pixel data from the lower left of the image.

In step S341, the processing of the image coordinates (c_x, c_y) from step S345 to the pixel data extraction unit 43 is performed when the display Y coordinate of the pixel which is is equal to or less than _{Y MAX} to S349 from step S313 S317 It corresponds to. However, the point to decrement the c_y, and that compares the c_y and _{y min} in step S348 differs in step S347.

In step S340, the processing of the image coordinates (c_x, c_y) from step S350 when the display X coordinate of the pixel is is determined to be _{X MIN} or more to S360 correspond to the S318 to S328.

Step S350 is the same as S318. In step S351, the image coordinates (c_x, c_y) display X coordinate of the pixel is determined whether greater than _{Y MAX.} Steps S352 to S356 are the same as steps S320 to S324, and a description thereof will be omitted. In step S357, the pixel data extraction unit 43 decrements c_y, and returns the process to step S351.

If it is determined in step S351 that the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is Y _MAX or less (step S351: NO), the pixel data extraction unit 43 sets the value of c_y to B _border. Substitution is performed (step S358), and B _border is updated. Subsequently, the pixel data extraction unit 43 stores the updated B _border in the storage unit 41 (step S359). Step S360 is the same as S328.

_{If X s} in step S300 is determined to be equal to 0 (S300: = 0), that is, when the slide of the image does not include a horizontal component, the pixel data extracting section 43 further compares the _{Y s} and 0 (Step S361).

If Y _s is determined smaller than 0 (S361: <0), the image is slid in the upward direction (Figure 18). In such a case, the pixel data extraction unit 43 determines whether or not T _border is stored in the storage unit 41 (step S362). When T _border is stored (S362: YES), that is, when pixel data extracted in the previous frame is stored, the pixel data extraction unit 43 substitutes T _border for c_y (step S363), and c_y. Is initialized. On the other hand, when it is determined that T _border is not stored (S362: NO), the pixel data extraction unit 43 substitutes y _min for c_y (step S364), and initializes c_y.

Subsequently, the pixel data extraction unit 43 substitutes x _min for c_x (step S365), and initializes c_x. After initialization of c_x and c_y, the pixel data extraction unit 43 determines whether or not the display Y coordinate of the pixel whose image coordinates are (c_x, c_y) is smaller than Y _MIN (step S366). When it determines with it being small (S366: YES), the pixel data extraction part 43 extracts the pixel data of the said pixel from the memory | storage part 41 (step S367), and memorize | stores it in the 2nd area | region of the memory | storage part 41 (step S368). Then, the pixel data extracting section 43 increments c_x (step S369), the incremented c_x is equal to or greater than _{x max} (step S370). When it determines with it not being large (S370: NO), the pixel data extraction part 43 returns a process to step S367, and repeats extraction of pixel data sequentially.

On the other hand, if the c_x is determined to greater than _{x max} (S370: YES), the pixel data extraction unit 43 increments the c_y (step S371). When it is determined that the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is greater than or equal to Y _MIN as a result of repeating the increment of c_y in step S371 (S366: NO), the pixel data extraction unit 43 substitutes the value of c_y at that time into T _border (step S372), and updates T _border . Thereafter, the pixel data extraction unit 43 stores the updated T _border in the storage unit 41 (step S373), ends the pixel data extraction process P2, and returns the process to the main routine.

In step S361, the pixel data extracting section 43, when it is determined that _{Y s} is equal to 0 (S361: = 0), the image will not slide. In such a case, the pixel data extraction unit 43 ends the pixel data extraction process P2, and returns the process to the main routine.

In step S361, the pixel data extracting section 43, when it is determined that _{Y s} is greater than 0 (S361:> 0), the image is slid in the downward direction. In such a case, the processing from steps S374 to S385 executed by the pixel data extraction unit 43 corresponds to steps S362 to S373 when sliding upward. Therefore, the common parts are omitted and will be described briefly.

In step S <b> 374, the pixel data extraction unit 43 determines whether or not B _border is stored in the storage unit 41. When it determines with having memorize | stored (S374: YES), the pixel data extraction part 43 substitutes B _border to c_y (step S375). When it is determined that it is not stored (S374: NO), the pixel data extraction unit 43 substitutes y _max for c_y (S376).

Step S377 is the same as step S365. Thereby, the pixel data extraction unit 43 extracts the pixel data in order from the lower left of the difference area. Subsequently, the pixel data extraction unit 43 determines whether or not the display Y coordinate of the pixel whose image coordinates are (c_x, c_y) is larger than Y _MAX (step S378).

  When it is determined that the value is large (S378: YES), the processing from step S379 to S382 is the same as the processing from S367 to S370. In step S383, unlike step S371, the pixel data extraction unit 43 decrements c_y.

In step S378, when it is determined that the display X coordinate of the pixel whose image coordinates are (c_x, c_y) is equal to or less than Y _MAX (S378: NO), the pixel data extraction unit 43 sets the value of c_y at that time to B _border. Substitution is performed (step S384), and B _border is updated. Thereafter, the pixel data extraction unit 43 stores the updated B _border value in the storage unit 41 (step S385), ends the pixel data extraction process P2, and returns the process to the main routine.

The process from step S386 to S445 when X _s is larger than 0 (S300:> 0) corresponds to the process from step S301 to S360 when X _s is smaller than 0. Therefore, common parts will be omitted and described briefly.

Steps S386 to S388 correspond to S301 to S303. However, in step S386, it is determined whether or not R _border is stored. In step S387, R _border is substituted for c_x. In step S388, x _max is substituted for c_x. That is, in steps S386 to S445, the point that the loop process for decrementing c_x from the initial value is performed is different from steps S301 to S360.

Step S389 is the same as step S304. Pixel data extraction unit 43 in step S389 is, _{if Y s} is determined smaller than 0 (S389: <0), the image is slid in the upper right direction. In such a case, the pixel data extraction unit 43 executes the processes from step S390 to S413. The processing from step S390 to S413 corresponds to the processing from step S305 to S328 when the image slides in the upper left direction.

Steps S390 to S397, except the point to be compared for _{X MAX} is greater than or not at step S393 is the same as the steps S305 to S312. Further, steps S398 to S402 are the same as steps S313 to S317 except that c_x is decremented in step S402.

In step S403, the pixel data extracting section _43, by substituting c_x to _{R border} updating _{R border.} From step S404 until step S410, that substitutes _{R border} to c_x at step S405, that decrements the c_x in step S408, and except for comparing the c_x and _{x min} in step S409, the steps S319 to S325 It is the same. Steps S411 and S412 are the same as steps S326 and S327. In step S413, the pixel data extraction unit 43 stores the R _border in the storage unit 41. Thereafter, the pixel data extraction unit 43 ends the pixel data extraction process P2 and returns the process to the main routine.

Pixel data extraction unit 43 in step S389 is, when it is determined that _{Y s} is equal to 0 (S389: = 0), the image is slid in the right direction. In such a case, the pixel data extraction unit 43 executes the processing from step S414 to S421. The processes from step S414 to S421 correspond to steps S329 to S336 when the image slides to the left.

Step S414 is the same as step S329. In step _S415, the point to be compared with the _{X MAX} is different from step S330. Steps S416 to S419 are the same as steps S331 to S334. In step S420, the pixel data extraction unit 43 decrements c_x, and then returns the process to step S414. In step S421, the pixel data extracting section 43 _updates by substituting c_x at that time _{R border,} stores _{R border} updated in step S413 in the storage unit 41. Thereafter, the pixel data extraction unit 43 ends the pixel data extraction process P2 and returns the process to the main routine.

Pixel data extraction unit 43 in step S389 is, when it is determined that _{Y s} is greater than 0 (S389:> 0), the image is slid in the lower right direction. In such a case, the pixel data extraction unit 43 executes the processing from step S422 to S445. The processes from step S422 to S445 correspond to steps S337 to S360 when the image slides in the lower left direction.

Steps S422 to S424 are the same as steps S337 to S339. Steps S425 to S429, except that the comparison with the _{X MAX} in step S425, is the same as the S340 to S344. Steps S430 to S434 are the same as steps S345 to S349 except that c_x is decremented in step S434.

In step S435, the pixel data extracting section 43 _substitutes c_x to _{R border,} to update the _{R border.} Step S436 is the same as step S351. In step S437, the pixel data extraction unit 43 substitutes R _border for c_x. Steps S438 to S442 are that decrements the c_x in step S440, and except for comparing the c_x and _{x min} in step S441, are the same as steps S353 to S357. Steps S443 and S444 are the same as steps S358 and S359. In step S445, the pixel data extraction unit 43 stores R _border in the storage unit 41. Thereafter, the pixel data extraction unit 43 ends the pixel data extraction process P2, and returns the process to the main routine.

  Through the above processing procedure, the pixel data extraction unit 43 executes the pixel data extraction process P2, extracts pixel data of pixels that have reached the boundary of the display area due to the slide from the image, and stores them in the storage unit 41.

  The control unit 40 reads the extracted pixel data from the storage unit 41 and outputs it to the communication unit 44, and the communication unit 44 transmits the input pixel data to the guest side via the network (step S5). At this time, when the data transmission destination guest is set to the left and right of the host, the control unit 40 reads the pixel data from the first area of the storage unit 41 and outputs the pixel data to the communication unit 44. Similarly, the control unit 40 reads out pixel data from the second area of the storage unit 41 when the data transmission destination guests are set above and below the host, and when the data transmission destination guest is set in an oblique direction, the storage unit 41 is read from the third area. After the communication unit 44 transmits the pixel data in step S5, the control unit 40 determines for each frame whether or not all the pixel data of the generated captured image has been transmitted to the guest side (step S6). When it is determined that all the pixel data of the captured image has been transmitted to the guest side (S6: YES), the control unit 40 ends the image transfer process. On the other hand, when it is determined that all of the pixel data of the captured image has not been transmitted to the guest side (S6: NO), the control unit 40 returns the process to step S3 and repeats the process for the next frame.

  Next, the procedure of guest side image reception processing will be described. The guest-side control unit 40 determines whether the communication unit 44 has received pixel data from the outside via the network (step S7). If it is determined that the communication unit 44 has not received pixel data (S7: NO), the control unit 40 repeats step S7 until it is determined that it has been received.

  When it is determined that the communication unit 44 has received pixel data (S7: YES), the control unit 40 displays a page composed of another image different from the captured image composed of the received pixel data on the display unit 45. It is determined whether it is displayed (step S8).

  In the present embodiment, since the guest is dedicated to viewing, the page displayed on the guest side is constituted by, for example, a single still image. The still image displayed on the guest side is, for example, a still image compressed by the same method as the captured image generated by the host side.

  When no other image is displayed on the display unit 45 (S8: NO), that is, as shown in FIG. 8, the control unit 40 skips steps S9 to S13 and is received by the communication unit 44. It is determined whether or not a part of the captured image constituted by the pixel data is displayed on the display unit 45 (step S14). When the control unit 40 determines that a part of the captured image is displayed on the display unit 45 (S14: YES), the image processing unit 42 displays the received captured pixel data on the displayed captured image. Therefore, the slide processing P1 of the received captured image is executed so as to slide from the right to the left by the necessary width (step S15).

  After sliding the captured image, the image processing unit 42 converts the received portion of the captured image based on the received pixel data into the already displayed portion in the display area vacated by sliding within one frame of the captured image. Connected and displayed (step S16). That is, the portion of the captured image displayed on the guest side display unit 45 is enlarged. On the other hand, when the control unit 40 determines that the captured image is not displayed on the display unit 45 (S14: NO), the image processing unit 42 captures the captured image based on the received pixel data at the host side end of the display area. Is received (step S16).

  After the image based on the received pixel data is displayed on the display unit 45, the control unit 40 determines whether or not the communication unit 44 has received all the pixel data of the captured image (step S17). When the communication unit 44 determines that not all of the pixel data of the captured image has been received (S17: NO), the control unit 40 returns the process to step S7, and further receives the pixel data and performs the process of the next frame. .

  On the other hand, when the communication unit 44 determines that all the pixel data of the captured image has been received (S17: YES), since the entire captured image is displayed on the display unit 45, the control unit 40 ends the image receiving process. To do. Through the processing on the host side and guest side described above, the page displayed on the host side is slid and transferred to the guest side (FIG. 19).

  Next, as shown in FIG. 20, image transfer processing when pages are displayed on the host side and the guest side of the multi-display system 100 will be described.

  Even when the page is displayed on the guest side, the operation on the host side is the same as when the page is not displayed on the guest side. Therefore, description of the operation on the host side is omitted. On the guest side, when it is determined in step S8 that the control unit 40 determines that a page composed of another image different from the captured image composed of the received pixel data is displayed on the display unit 45 (S8: YES), the image processing unit 42 executes a slide process P1 of another image, and slides the other image corresponding to the direction in which the captured image slides on the host side (step S9). Subsequently, the pixel data extraction unit 43 of the control unit 40 executes a pixel data extraction process P2 so as to extract pixels of other images that are no longer displayed because the slide has reached the boundary of the display area of the display unit 45. (Step S10).

  Thereafter, the control unit 40 determines whether or not the other guest is set to cooperate with the own apparatus adjacent to the opposite side of the host (step S11). That is, in the example shown in FIG. 20, it is determined whether or not another guest is set to cooperate with the left side of the guest.

  When the control unit 40 determines that another guest is set to cooperate with the own device (S11: YES), the communication unit 44 extracts the pixel data extracted by the pixel data extraction unit 43, It transmits to another adjacent guest (step S12). On the other hand, when it is determined that another guest is not set to cooperate adjacently to the own device (S11: NO), the control unit 40 stores the pixel data extracted by the pixel data extraction unit 43 in the storage unit 41. Store (step S13) (FIG. 21).

  Steps S14 to S17 are the same as when no other image is displayed on the guest side, and a description thereof will be omitted.

  With the above operation procedure, an image can be transferred from the host side to the guest side with a smooth slide display.

  In the multi-display system according to the present embodiment, it is possible to transfer and display the captured image once transferred from the host side to the guest side in the reverse direction from the guest side to the host side by a user operation. It is. Hereinafter, the reverse image transfer process from the guest side to the host side will be described with reference to FIGS.

  FIG. 22 is a flowchart showing an operation procedure on each of the host side and the guest side in the reverse image transfer process from the guest side to the host side in the multi-display system 100 according to the first embodiment. 23 to 27 are conceptual diagrams showing an operation image of the reverse image transfer process from the guest side to the host side in the multi-display system 100 according to the first embodiment. In order to start the image reverse transfer process, the captured image transferred from the host side needs to be displayed on the guest side of the multi-display system 100 as shown in FIG.

  First, the operation procedure on the guest side of the image reverse transfer process will be described. The control unit 40 of the information processing apparatus 1 serving as a guest determines whether or not a flick or swipe operation from the guest side to the host side in FIG. Determination is made (step S20). When it is determined that the flick or swipe operation has not been input (S20: NO), the control unit 40 repeats S20 until it is determined that the input has been performed.

  When the control unit 40 determines that a flick or swipe operation has been input (S20: YES), the image processing unit 42 causes the display unit 45 to slide-display the captured image from left to right within one frame time. The captured image slide process P1 is executed (step S21) (FIG. 23). The pixel data extraction unit 43 of the control unit 40 detects a portion of the captured image that is not displayed on the display unit 45 because it has reached the boundary of the display area of the display unit 45 during one frame due to the slide display. The pixel data extraction process P2 is executed so as to extract (step S22). The control unit 40 reads the extracted pixel data from the storage unit 41 and outputs it to the communication unit 44, and the communication unit 44 transmits the pixel data input from the control unit 40 to the host via the network (step S23). .

  Subsequently, the control unit 40 determines whether or not pixel data of another image that is no longer displayed in order to display the captured image transferred from the host side on the display unit 45 is displayed in the storage unit 41. (Step S24). Such pixel data is stored in the storage unit 41 in step S13 of the flowchart of FIG. When it is determined that the pixel data of another image is not stored in the storage unit 41 (S24: NO), that is, as shown in FIGS. 9 and 23, the control unit 40 skips steps S25 to S27. Then, the process proceeds to step S28.

  On the other hand, when it is determined that the pixel data of another image is stored in the storage unit 41 (S24: YES), that is, as shown in FIG. It is determined whether a part is displayed on the display unit 45 (step S25). When the control unit 40 determines that another stored image is not displayed on the display unit 45 (S25: NO), the image processing unit 42 selects one frame of the captured image in the display area of the display unit 45. Another image is displayed on the basis of the pixel data stored in the storage unit 41 in an area vacated by the slide inside (step S27).

  When the control unit 40 determines that a part of another image is displayed on the display unit 45 (S25: YES), the image processing unit 42 within one frame of the captured image in the display area of the display unit 45. The slide processing P1 of another image is executed so that the displayed part of the other images stored in the area vacated by the slide is displayed as a slide (step S26). Thereafter, the image processing unit 42 displays an image based on the pixel data stored in the storage unit 41 in an area vacated by a slide of a portion where another image is displayed in the display area of the display unit 45. The image is displayed so as to be connected to the already displayed portion (step S27) (FIG. 26).

  In step S27, after the image processing unit 42 displays another image based on the pixel data stored in the storage unit 41, or the control unit 40 does not store the pixel data of the other image in the storage unit 41. (S24: NO), the control unit 40 determines for each frame whether or not all the pixel data of the captured image has been transmitted to the host side (step S28). When it is determined that all of the pixel data of the captured image has not been transmitted to the host side (S28: NO), the control unit 40 returns the process to step S21 and performs the process for the next frame. If it is determined that all the pixel data of the captured image has been transmitted to the host side (S28: YES), the control unit 40 ends the image reverse transfer process on the guest side.

  Next, an operation procedure on the host side in the reverse image transfer process will be described. The control unit 40 of the information processing apparatus 1 serving as the host determines whether or not the communication unit 44 has received pixel data from the outside via the network (step S29). When it determines with the communication part 44 not receiving pixel data (step S29: NO), the control part 40 repeats step S29 until it determines with receiving pixel data.

  When it is determined that the communication unit 44 has received the pixel data (step S29: YES), the control unit 40 displays a part of the captured image constituted by the pixel data received by the communication unit 44 on the display unit 45. It is determined whether or not (step S30). When the control unit 40 determines that such a received captured image is not displayed on the display unit 45 (S30: NO), the image processing unit 42 receives the captured image on the display unit 45 based on the received pixel data. The part is displayed (step S32).

  On the other hand, when the control unit 40 determines that a part of the captured image is displayed on the display unit 45 (S30: YES), the image processing unit 42 regards the portion where the captured image is displayed as the captured image. The captured image slide process P1 is executed so that the slide is displayed on the display unit 45 within one frame time in the direction corresponding to the direction in which the user slides on the guest side (step S31). Subsequently, the image processing unit 42 converts the received part of the captured image based on the received pixel data to the same captured image up to the area vacated by the slide of the part where the captured image is displayed in the display area of the display unit 45. Are connected to the already displayed part (step S32) (FIG. 26).

  In step S32, after displaying the captured image based on the pixel data received by the image processing unit 42, the control unit 40 determines for each frame whether or not the communication unit 44 has received all the pixel data of the captured image. (Step S33). When the communication unit 44 determines that all of the pixel data of the captured image has not been received (S33: NO), the control unit 40 returns the process to step S29, further receives the pixel data, and performs the process for the next frame.

  When the control unit 40 determines that the communication unit 44 has received all the pixel data of the captured image (S33: YES), the entire captured image is displayed on the display unit 45 (FIG. 24). In such a case, the image processing unit 42 reconstructs the object included in the page from which the captured image is created based on the object data stored in the storage unit 41 and replaces the captured image with the captured image. It is displayed (step S34) (FIG. 27). As a result, the original page is reproduced with the object being editable. After the image processing unit 42 reconstructs and displays the object in step S34, the host-side control unit 40 ends the image reception process.

  Note that the reverse image transfer process from the guest side to the host side is not limited to the case where the entire captured image is transferred from the host side to the guest side. For example, during the image transfer process from the host side to the guest side, when the captured image is displayed across both display units 45, the user can use the guest side on the touch panel on either the host side or the guest side. In response to flicking or swiping the captured image in the direction from the host to the host, image reverse transfer processing from the guest to the host may be started. Or, conversely, during the reverse image transfer process from the guest side to the host side, when the captured image is displayed across both the display units 45, the user can select either the host side or the guest side touch panel. The image transfer processing from the host side to the guest side may be started in response to the flick or swipe operation performed at.

  With the above configuration and operation procedure, in a multi-display system including a plurality of information processing apparatuses, smooth image transfer processing by slide is realized between the information processing apparatuses, and the area in which images can be displayed in cooperation is expanded.

(Embodiment 2)
In the first embodiment, the guest is configured to be used only for browsing images. On the other hand, in the second embodiment, the user can create and edit a new object on the guest side. Hereinafter, the features of the present embodiment that are different from those of the first embodiment will be specifically described.

  Since the configuration of the multi-display system, the configuration of the information processing device, and the cooperation setting between the information processing devices according to the present embodiment are the same as those of the first embodiment, description thereof will be omitted.

  Hereinafter, the operation procedure of the multi-display system according to the present embodiment will be described with reference to the related drawings.

  28 and 29 are flowcharts showing operation procedures on the host side and the guest side in the image transfer processing from the host side to the guest side in the multi-display system 100 according to the second embodiment. 30 and 31 are conceptual diagrams showing an operation image of image transfer processing from the host side to the guest side in the multi-display system 100 according to the second embodiment.

  Steps S40 to S45 on the host side in FIG. 28 are part of the processing (FIG. 30) in which the host transfers an image to the guest. Since the procedure from step S40 to S45 is the same as that from step S1 to S6 of the first embodiment, the description thereof is omitted. If the host-side control unit 40 determines in step S45 that all pixel data of the captured image has been transmitted to the guest side (S45: YES), the control unit 40 stores the object data relating to the object of the original page. The data is read from the unit 41 and output to the communication unit 44, and the communication unit 44 transmits the input object data to the guest side via the network (step S46) (FIG. 31). Thereafter, the host-side control unit 40 ends the image transfer process.

  Next, the procedure of guest side image reception processing will be described. First, the control unit 40 on the guest side determines whether or not the communication unit 44 has received pixel data from the outside via the network (step S47). When it determines with the communication part 44 not receiving pixel data (S47: NO), the control part 40 repeats step S47 until it determines with having received.

  When it is determined that the communication unit 44 has received pixel data (S47: YES), the control unit 40 is configured by another captured image different from the captured image configured by the received pixel data, or one It is determined whether another page including an object different from the page is displayed on the display unit 45 (step S48). When it determines with the other page not being displayed on the display part 45 (S48: NO), the control part 40 skips step S49 to S55, and advances a process to step S56. Steps S56 to S59 are the same as steps S14 to S17 of the first embodiment, and thus description thereof is omitted.

  On the other hand, when it is determined that another page is displayed on the display unit 45 (S48: YES), the control unit 40 starts from the other page including the object on which the display unit 45 displayed the image processing unit 42. It is determined whether or not the original other page has been replaced by the captured image created by (i.e., whether the image has been replaced) (step S49). When the control unit 40 determines that the page displayed on the display unit 45 is not a replaced captured image (S49: NO), that is, when another page including an object is displayed on the display unit 45, the image The processing unit 42 captures another page including the displayed object, generates a captured image, and replaces the other original page with the generated captured image (step S50). At this time, the object data of the object displayed on the other original page is stored in the storage unit 41.

  After the image processing unit 42 generates and replaces the captured image, or when the control unit 40 determines that the other original page has already been replaced by the captured image (S49: YES), the control unit 40 Advances to step S51. Steps S51 to S59 are the same as steps S9 to S17 of the first embodiment, and thus description thereof is omitted.

  In step S59, when the control unit 40 determines that the communication unit 44 has received all the pixel data of the captured image (S59: YES), that is, the entire portion of the captured image is displayed on the display unit 45 based on the result. If so, the control unit 40 determines again whether or not another page is displayed on the display unit 45 at the time of step S48 (step S60). In step S60, since the same processing as step S48 is performed, for example, the determination result at the time of determination in step S48 may be managed by a flag and stored.

  When it determines with the other page being displayed on the display part 45 (S60: YES), the control part 40 determines whether the adjacent guest is set similarly to step S53 (step S61). When it is determined that an adjacent guest is set (S61: YES), the control unit 40 reads out the object data related to the object on the other page from the storage unit 41 and outputs the object data to the communication unit 44. The communication unit 44 The object data input from the control unit 40 is transmitted to the adjacent guest via the network (step S62).

  On the other hand, when it is determined that the adjacent guest is not set (S61: NO), the control unit 40 continues to store the object data in the storage unit 41 (step S63).

  After step S62 or S63, the communication unit 44 receives the object data of the object included in the page that is the basis of the received captured image (step S64). After that, the image processing unit 42 reconstructs the object included in the page from which the captured image is created based on the received object data, and displays it replacing the received captured image (step S65). (FIG. 31). As a result, since the object displayed on the transfer source host side is also displayed on the guest side, the user can newly create and edit the object also on the guest side. After reconstructing the object in step S65, the guest-side control unit 40 ends the image reception process.

  Also in the multi-display system 100 according to the present embodiment, similarly to the first embodiment, it is possible to transfer and display pages in the reverse direction from the guest side to the host side by a user operation. However, in this embodiment, since a new object can be created and edited on the guest side, a captured image is generated from the page also in the reverse image transfer process. Hereinafter, image reverse transfer processing from the guest side to the host side will be described with reference to related drawings.

  FIG. 32 is a flowchart showing an operation procedure on each of the host side and the guest side in the reverse image transfer process from the guest side to the host side in the multi-display system 100 according to the second embodiment. FIG. 33 is a conceptual diagram illustrating operation images on the host side and the guest side in the image reverse transfer process from the guest side to the host side in the multi-display system 100 according to the second embodiment. In order to start the image reverse transfer process, the page transferred from the host side needs to be displayed on the guest side of the multi-display system 100 as in the first embodiment (FIG. 9).

  The operation procedure of the information processing apparatus as a guest will be described. Steps S70 to S74 on the guest side are the same as steps S40 to S44 (or steps S1 to S5 in FIG. 5) on the host side in FIG. Further, steps S75 to S78 are the same as steps S24 to S27 in FIG.

  After the communication unit 44 transmits the pixel data in step S78, the control unit 40 transmits all the pixel data of the captured image by the communication unit 44, and all the other captured images stored in the storage unit 41 are stored. Whether or not the image is displayed on the display unit 45 is determined for each frame (step S79). When the control unit 40 determines that the communication unit 44 does not transmit all of the pixel data of the captured image or that all of the other captured images stored in the storage unit 41 are not displayed on the display unit 45 (S79: NO), the control unit 40 returns the process to step S72, and performs the process of the next frame.

  On the other hand, when the control unit 40 determines that the communication unit 44 transmits all the pixel data of the captured image and the other captured images stored in the storage unit 41 are displayed on the display unit 45. (S79: YES), the image processing unit 42 reconstructs the object based on the object data stored in the storage unit 41, and displays it by replacing it with another captured image (step S80) (FIG. 33). Thereafter, the control unit 40 on the guest side ends the image reverse transfer process.

  The operation procedure (steps S81 to S86) of the image reception process on the host side is the same as steps S29 to S34 of the first embodiment, and thus description thereof is omitted.

  With the above configuration and operation procedure, in a multi-display system composed of a plurality of information processing devices, smooth image transfer processing by sliding between the information processing devices is realized, and the user also creates and edits objects on the guest side. And the display area where the object can be edited is expanded.

(Embodiment 3)
In each of the above-described embodiments, when a captured image is transferred from the host side to the guest side, the image on the guest side is slid when the image is displayed on the guest side. In contrast, in the third embodiment, when a captured image is transferred from the host side to the guest side, if the image is displayed on the guest side, the transferred captured image overwrites the guest side image. Are displayed. Hereinafter, the features of the third embodiment that are different from those of the above-described embodiments will be specifically described.

  Since the configuration of the multi-display system, the configuration of the information processing device, and the cooperation setting between the information processing devices according to the present embodiment are the same as those of the above-described embodiments, description thereof will be omitted.

  Hereinafter, the operation procedure of the multi-display system according to the present embodiment will be described with reference to the related drawings.

  FIG. 34 is a flowchart showing an operation procedure on the guest side in image transfer processing from the host side to the guest side in the multi-display system 100 according to the third embodiment. FIG. 35 is a conceptual diagram illustrating an operation image of image transfer processing from the host side to the guest side in the multi-display system 100 according to the third embodiment. FIG. 36 is a flowchart showing an operation procedure of pixel data extraction processing P3 which is a subroutine in the flowchart of FIG.

  In this embodiment, for the sake of simplicity, a case where the guest is dedicated to browsing will be described as an example, but the present invention is not limited thereto. The user may be able to create and edit an object on the guest side.

  Since the operation procedure of the image transfer process on the host side is the same as that of the first embodiment, the description will be omitted and the image reception process on the guest side will be described.

  Step S90 of the guest-side image reception process is the same as step S7 of the first embodiment, and steps S91 and 92 are the same as steps S14 and 15, and a description thereof will be omitted. In step S92, when the image processing unit 42 slides a part of the received captured image that is displayed, the control unit 40 determines whether another image already displayed on the display unit 45 is overwritten. Determination is made (step S93). That is, it is determined whether or not the left end of the captured image has reached the right end of another image based on the received captured image slide.

  When the control unit 40 determines that another image has been overwritten when the received captured image slides (S93: YES), the pixel data extraction unit 43 newly performs a slide within one frame among the other images. A pixel data extraction process P3, which is a subroutine for extracting the pixel data of the overwritten portion, is executed (step S94).

With reference to FIG. 36, the process procedure of the pixel data extraction process P3 will be described. First, the pixel data extraction unit 43 acquires the slide width (X _s , Y _s ) of the received image from the pixel data transmitted from the host side (step S500). Subsequently, the pixel data extraction unit 43 specifies a newly overwritten difference area from (X _s , Y _s ) (step S501). After specifying the difference area, the pixel data extraction unit 43 extracts the pixel data of the specified difference area from the storage unit 41 (step S502) and stores it in a different address area of the storage unit 41 (step S503). A more detailed execution procedure of such processing is performed in a mode according to the pixel data extraction processing P2.

  Through the above processing procedure, the pixel data extraction unit 43 executes the pixel data extraction process P3, extracts the pixel data of the pixel that has reached the boundary of the display area by the slide from the image, and stores it in the storage unit 41.

  After executing the pixel data extraction process P3, the control unit 40 continues to store the extracted pixel data in the storage unit 41 (step S95) (FIG. 35). In step S95, after the control unit 40 stores the extracted pixel data in the storage unit 41, or when the control unit 40 determines that another image is not overwritten when the received captured image slides (S93). : NO), or when the control unit 40 determines in step S91 that the captured image constituted by the received pixel data is not displayed on the display unit 45 (S91: NO), the image processing unit 42 The received part based on the pixel data received in one frame of the image is connected to the already displayed part and displayed on the display unit 45 (step S96).

  After step S96, the control unit 40 determines for each frame whether or not the communication unit 44 has received all the pixel data of the captured image (step S97). When the communication unit 44 determines that not all of the pixel data of the captured image has been received (S97: NO), the control unit 40 returns the process to step S90, and further receives the pixel data and performs the process of the next frame. .

  On the other hand, when it is determined that the communication unit 44 has received all the pixel data of the captured image (S97: YES), that is, when the entire captured image is displayed on the display unit 45 as a result, the control unit 40 displays the image. The reception process ends. Through the host side and guest side processing described above, the page displayed on the host side is slid and transferred to the guest side (FIG. 35). The image displayed on the guest side when the page from the host side to the guest side is transferred is not displayed on the display unit 45 by being overwritten, but the pixel data is stored in the storage unit 41. Therefore, when a page transferred and displayed from the host side to the guest side is reversely transferred from the guest side to the host side, the overwritten image is restored and displayed based on the pixel data stored in the storage unit 41. obtain.

  With the above configuration and operation procedure, in a multi-display system including a plurality of information processing devices, smooth image transfer processing by sliding between information processing devices is realized, and a display area where images can be displayed in cooperation with each other is expanded.

(Embodiment 4)
In each of the above-described embodiments, the display resolution of the touch panel display of each information processing apparatus is the same, but is not limited thereto. In the fourth embodiment, when an image is transferred between information processing apparatuses, the information processing apparatus on the receiving side can display the received image in an enlarged or reduced manner. Hereinafter, the features of the fourth embodiment that are different from those of the above-described embodiments will be specifically described.

  Since the configuration of the multi-display system, the configuration of the information processing device, and the cooperation setting between the information processing devices according to the present embodiment are the same as those of the above-described embodiments, description thereof will be omitted. However, in the present embodiment, the display resolution of the touch panel display of each information processing apparatus constituting the multi-display system may be different.

  Hereinafter, the operation procedure of the multi-display system according to the present embodiment will be described with reference to the related drawings.

  FIG. 37 is a flowchart showing an operation procedure on the guest side in image transfer processing from the host side to the guest side in the multi-display system 100 according to the fourth embodiment. FIG. 38 is a conceptual diagram showing an operation image of image transfer processing from the host side to the guest side in the multi-display system 100 according to the fourth embodiment.

  In this embodiment, for the sake of simplicity, a case where the guest side is dedicated to browsing will be described as an example, but the present invention is not limited thereto. The user may be able to create and edit an object on the guest side.

  Since the operation procedure of the image transfer process on the host side is the same as that of the first embodiment, the description will be omitted and the image reception process on the guest side will be described.

  Steps S100 to S102 of the guest-side image reception process are the same as steps S90 to S92 of the third embodiment, and a description thereof will be omitted. In step S102, after sliding the displayed part of the captured image received by the image processing unit 42, or when the control unit 40 determines that the captured image is not displayed (S101: NO), the control unit 40 determines whether the resolution of the display area of the display unit 45 on the host side that has transmitted the received pixel data is the same as the resolution of the display area of the display unit 45 of the device itself (step S103).

  When the control unit 40 determines that the resolutions are the same (S103: YES), the image processing unit 42 captures on the display unit 45 based on the received pixel data in the same manner as in step S16 of the first embodiment. The received part of the image is displayed (step S104). On the other hand, when the control unit 40 determines that the resolution of the captured image composed of the received pixel data is different from the resolution of the display area of the display unit 45 (S103: NO), the image processing unit 42 A portion related to the received pixel data is enlarged or reduced in accordance with the resolution of the display area (step S105) (FIG. 38).

  After step S104 or S105, the control unit 40 advances the process to step S106. Since step S106 is the same as step S17 in the first embodiment, a description thereof will be omitted.

  With the above configuration and operation procedure, even in a multi-display system including a plurality of information processing apparatuses having different display resolutions, smooth image transfer processing by sliding between information processing apparatuses is realized, and images can be displayed in cooperation with each other. The display area is expanded.

(Embodiment 5)
In each of the above-described embodiments, the object included in the original page when generating the capture image is a still image, but is not limited to this configuration. In the fifth embodiment, even when an object is a moving image being reproduced, a captured image of the page can be generated and transferred, and the moving image can be continuously reproduced in the information processing apparatus after the transfer. Hereinafter, features of the fifth embodiment that are different from those of the above-described embodiments will be specifically described.

  Since the configuration of the multi-display system, the configuration of the information processing device, and the cooperation setting between the information processing devices according to the present embodiment are the same as those of the above-described embodiments, description thereof will be omitted.

  FIGS. 39 and 40 are flowcharts showing operation procedures on the host side and the guest side in image transfer processing from the host side to the guest side in the multi-display system 100 according to the fifth embodiment. First, the procedure of image transfer processing on the host side will be described. The determination of the flick or swipe operation in step S110 is the same as step S1 in the first embodiment, and thus the description thereof is omitted. When the input unit 46 detects a flick or swipe operation by the user (S110: YES), the control unit 40 determines whether or not a moving image object is included in the page displayed on the display unit 45 (step S111). ). When the control unit 40 determines that a moving image object is included (S111: YES), the image processing unit 42 pauses the reproduction of the moving image object (step S112). The image processing unit 42 captures a page in a state where the moving image is paused, generates a captured image, and replaces the page with the generated captured image (step S113).

  When the control unit 40 determines that the moving image object is not included in the page (S111: NO), the image processing unit 42 skips step S112 and replaces the page with a captured image (step S113). Steps S114 to S118 are the same as steps S42 to S46 of the second embodiment, and a description thereof will be omitted. Here, the object data transmitted by the communication unit 44 in step S118 includes stop position data, moving image data after the stop position, and the like necessary for reproducing the moving image from the pause position. After the communication unit 44 transmits the object data in step S118, the control unit 40 ends the image transfer process.

  Next, the procedure of guest side image reception processing will be described. Here, for simplicity, only the case where no other page is displayed on the guest side will be described. Step S119 of the processing procedure on the guest side is the same as step S47 of the second embodiment, and steps S120 to S123 are the same as steps S56 to S59, and S124 and S125 are the same as S64 and S65. Therefore, the description is omitted. However, in step S125, the moving image object is reconstructed in a state when it is temporarily stopped by the host side.

  After the image processing unit 42 reconstructs the object in step S125, the control unit 40 determines whether or not a moving image object is included in the reconstructed object (step S126). When the control unit 40 determines that the reconstructed object includes a moving image (S126: YES), the image processing unit 42 selects the moving image based on the object data of the moving image object received by the communication unit 44. Playback is started from the pause position (step S127), and the image reception process is terminated. Note that there is a large amount of data related to moving image reproduction, and it is possible that reception of all moving image data has not been completed when moving image reproduction is started in step S127. In such a case, a configuration may be employed in which playback of moving images and reception of moving image data are performed simultaneously in parallel.

  On the other hand, when it determines with a moving image object not being included by step S126 (S126: NO), the control part 40 skips step S127 and complete | finishes an image reception process.

  With the above configuration and operation procedure, in a multi-display system including a plurality of information processing devices, transfer processing of an image including a moving image is realized between the information processing devices.

  In the present invention, since the information processing apparatuses are connected to each other via a network, it is not necessary to arrange them at physically close positions. If it is placed physically close, it can be used as an electronic blackboard system, while it can be used for a remote education system, a remote conference system, or the like if it is placed at a physically separated point.

  In each of the above-described embodiments, the input unit of the information processing apparatus is a touch panel. However, an input device other than the touch panel may be included. For example, a mouse and a keyboard may be additionally provided. In addition to the touch panel, the input unit that detects the direction may be, for example, a camera that can recognize the user's gesture.

  According to a first aspect of the present invention, in an image display system 100 in which a plurality of information processing apparatuses 1 having a function of displaying an image is connected to each of the plurality of display apparatuses 30, each of the information processing apparatuses 1 receives an input. Input unit 46, communication unit 44 that performs data communication with other information processing devices, image processing unit 42 that performs image processing including generation of a captured image in units of pages, and an image displayed on the display device An image processing unit 42 of one information processing apparatus 1, and a storage unit 41 that stores pixel data and object data related to an object displayed on the display device. Is an operation that includes one captured image generated by capturing one page including the object displayed on the display device 30, including the direction input to the input unit 46. And the pixel data extracting unit 43 of the one information processing device displays the display by sliding the captured image. The pixel data of the part reaching the boundary of the display area of the device 30 is sequentially extracted, and the extracted pixel data is sequentially transmitted to the other information processing device 1 via the communication unit 44. The image processing unit 42 of the device 1 is characterized in that the pixel data received by the communication unit 44 is displayed on the display device 30 as an image.

  In the present invention, the plurality of information processing devices have a function of displaying an image on each of the plurality of display devices, and are connected to each other via, for example, a network. An operation is input to the input unit of each information processing apparatus. The communication unit transmits / receives data to / from other information processing apparatuses via the network. The image processing unit performs various image processing including generation of a captured image for each page. The pixel data extraction unit extracts pixel data of an image displayed on the display device. The storage unit stores pixel data and object data of an object displayed on the display device.

  The image processing unit of one information processing apparatus captures one page including the object and generates a capture image according to a user operation including the direction input to the input unit, and generates the captured image. Display on the display device so as to slide in the direction corresponding to the operation. The pixel data extraction unit sequentially extracts pixel data of a portion of the captured image that reaches the boundary of the display area. The communication unit sequentially transmits the pixel data extracted by the pixel data extraction unit to other information processing apparatuses.

  An image processing unit of another information processing apparatus to which the pixel data is transmitted causes the display device to sequentially display the pixel data received by the communication unit as an image.

  When transmitting a page from one information processing device to another information processing device, if the object displayed on the page is transmitted as data as it is, the page is being transferred when the number of objects increases. There is a possibility that the amount of calculation processing for sequentially processing the image of the object increases. Therefore, by temporarily converting a page including an object into a captured image and transmitting it, an increase in the amount of calculation processing can be suppressed even when the number of objects increases. Furthermore, smooth page movement is expressed between display devices by sequentially transmitting pixel data related to the portion that has reached the boundary of the display area when the page is slid and displaying it on the display device of another information processing device. obtain.

  In the second aspect of the present invention, when the communication unit 44 of the other information processing apparatus 1 receives the pixel data from the one information processing apparatus 1, the image processing unit 42 is displayed on the display device 30. The other pixel image is displayed so as to slide in the direction related to the operation, and the received pixel data is displayed as an image in an area vacated by the slide of the other image in the display area of the display device 30. It is characterized by being displayed.

  In the present invention, when the communication unit of another information processing apparatus receives the pixel data of the captured image from one information processing apparatus, when the other page is displayed on the display device, the image processing unit Slide the pages sequentially. The image processing unit further displays the received pixel data as an image in a display area vacated by sliding the page.

  When a page displayed on one information processing apparatus is transmitted to another information processing apparatus by slide, the other pages displayed on the other information processing apparatus slide in conjunction with each other. Therefore, a smooth page movement can be expressed as a whole.

  In the third aspect of the present invention, the pixel data extraction unit 43 of the other information processing apparatus 1 sequentially extracts and extracts the pixel data of the portion that has reached the boundary of the display area by the slide from the other images. The pixel data is stored in the storage unit 41.

  In the present invention, the pixel data extraction unit of the other information processing apparatus sequentially extracts the pixel data of the part that has reached the boundary of the display area by sliding among the other pages, and stores the extracted pixel data in the storage unit Remember. Therefore, it may be possible to restore other pages later and display them again.

  In a fourth aspect of the present invention, when the communication unit 44 of the one information processing apparatus 1 transmits the pixel data to the other information processing apparatus 1, the object data of the object is also transmitted. When the communication unit 44 receives the pixel data of the entire part of the captured image, the image processing unit 42 of the other information processing apparatus 1 extracts the object based on the object data received by the communication unit 44. The information is displayed on the display device 30.

  In the present invention, when all the pixel data of the captured image generated from one page including the object is transmitted from one information processing apparatus to another information processing terminal, the object data is also transmitted. Based on the object data received by the communication unit, the image processing unit of another information processing device reconstructs the object, replaces the captured image, and displays it on the display device. Therefore, the user can edit the object on the information processing apparatus to which the page is transmitted.

  According to a fifth aspect of the present invention, when the communication unit 44 of the other information processing apparatus 1 receives the pixel data from the one information processing apparatus 1, the image processing unit 42 displays on the display device 30. Another captured image generated by capturing another page including the object being displayed is displayed on the display device 30 so as to slide in the direction related to the operation, and the other of the display areas of the display device 30 The received pixel data is displayed as an image in an area vacated by slide of the captured image.

  In the present invention, in another information processing device, when the communication unit receives the pixel data of the captured image from one information processing device, when another page including the object is already displayed on the display device, The image processing unit generates other captured images from other pages and displays them in a slide display. Furthermore, an image is displayed based on the received pixel data in a display area of a display device vacated by another captured image slide. Therefore, smooth page movement can be expressed as a whole, and at the same time, an increase in the amount of calculation processing related to the slide of another page can be suppressed.

  In the sixth aspect of the present invention, the pixel data extraction unit 43 of the other information processing apparatus 1 extracts and extracts pixel data of a part of the other captured image that reaches the boundary of the display area by sliding. The pixel data and the object data of the object are stored in the storage unit 41.

  In the present invention, the pixel data extraction unit of another information processing apparatus extracts pixel data of a portion that has reached the boundary of the display area by slide from other captured images generated from other pages. The storage unit stores object data and extracted pixel data. Accordingly, it may be possible to display other pages that are no longer displayed due to the slide by restoring the object to an editable state later.

  In the seventh aspect of the present invention, when the other information processing apparatus 1 displays a part or all of the one captured image on the display device 30, the information is input to the one information processing apparatus 1. In response to an operation including a direction opposite to the direction related to the operation, the image processing unit 42 slides the captured image in the opposite direction, and the pixel data extraction unit 43 slides the captured image. Sequentially extract the pixel data of the part that has reached the boundary of the display area, and sequentially transmit the extracted pixel data to the one information processing apparatus 1 via the communication unit 44. When the communication unit 44 receives the pixel data, the image processing unit 42 of the device 1 slides the portion of the captured image that is already displayed on the display device 30 in the opposite direction. In addition, the received pixel data is displayed as an image in the display area of the display device 30 vacated by the slide, and the entire captured image is displayed on the display device 30 by receiving the pixel data. In this case, the image processing unit 42 is configured to display the object included in the one page on the display device 30 based on the object data stored in the storage unit 41. To do.

  In the present invention, one captured image generated from one page and displayed on the display device of another information processing device is slid in the opposite direction on the display device according to the operation, Pixel data can be transmitted from another information processing apparatus to one information processing apparatus. The image processing unit of the one information processing apparatus displays the one captured image on the display device based on the received pixel data. When the entire captured image is transmitted to the one information processing device by slide and displayed on the display device, the image processing unit reconstructs the object based on the object data stored in the storage unit. Display on the display device.

  Accordingly, since one page transmitted by sliding from one information processing apparatus to another information processing apparatus can be returned to one information processing apparatus by sliding in the opposite direction, the convenience of the user is improved. To do.

  In an eighth aspect of the present invention, when the other information processing apparatus 1 displays a part or all of the one captured image on the display device 30, the opposite direction input to the input unit 46 is displayed. The image processing unit 42 slides the one captured image in the opposite direction according to an operation including the pixel data extracting unit 43 by sliding the display device 30 in the one captured image. The pixel data of the part that has reached the boundary of the display area is sequentially extracted, and the extracted pixel data is sequentially transmitted to the one information processing apparatus 1 via the communication unit 44, and When a part of another image is displayed and pixel data relating to the remaining part is stored in the storage unit 41, the image processing unit 42 is empty by sliding the one captured image. The other image is slid in the opposite direction to the display area, and the pixel data stored in the storage unit 41 is displayed as an image in the display area vacated by the sliding of the other image. Or when the other image is not displayed on the display device 30 and pixel data of the other image is stored in the storage unit 41, the image processing unit 42 The pixel data stored in the storage unit 41 is displayed as an image in the display area vacated by a slide of one capture image, and the image processing unit 42 of the one information processing apparatus 1 includes: When the communication unit 44 receives the pixel data related to the one captured image, the part already displayed on the display device 30 in the one captured image is scanned in the opposite direction. In addition, the received pixel data is displayed as an image in the display area of the display device 30 that is vacated by sliding, and all of the one captured image is displayed on the display device 30 by receiving the pixel data. Is displayed, the image processing unit 42 displays the object included in the one page on the display device 30 based on the object data stored in the storage unit 41. It is characterized by.

  In the present invention, when one captured image generated from one page is slid from another information processing apparatus to one information processing apparatus by a user operation, the display device of the other information processing apparatus When a part of another captured image is displayed above and the pixel data of the remaining part is stored in the storage unit, the image processing unit displays another captured image in a display area vacated by the slide of one captured image. Slide. Furthermore, the image processing unit restores and displays the other captured image on the display area vacated by the slide of the other captured image based on the pixel data stored in the storage unit.

  On the other hand, when another captured image is not displayed on the display device of another information processing apparatus and the pixel data is stored in the storage unit, the image processing unit displays a display area that is vacant by the slide of the one captured image. In addition, other captured images are restored and displayed based on the pixel data stored in the storage unit.

  Furthermore, when the communication unit receives the pixel data of one captured image, the image processing unit of the one information processing apparatus slides a part of the already displayed one captured image, and a display area that is freed by the slide. In addition, the received pixel data is displayed as an image by connecting to the already displayed portion. When the entire captured image is displayed, the image processing unit reconstructs the object displayed on the original page based on the object data stored in the storage unit.

  That is, in conjunction with the slide of one captured image in the opposite direction, the displayed part of the other captured image is slid, and the remaining part based on the pixel data stored in the storage unit Is restored and displayed on another information processing apparatus. Therefore, when one page is returned to one information processing apparatus by slide, other pages can be returned to the other information processing apparatus at the same time by a smooth slide linked with each other, so that images are displayed on the display devices of the plurality of information processing apparatuses. Displayed in conjunction with each other, improving the convenience for the user.

  In the ninth aspect of the present invention, when the other information processing apparatus 1 is displaying the one page on the display device 30, according to an operation including the opposite direction input to the input unit 46, The image processing unit 42 is configured to display the captured image generated by capturing the one page on the display device 30 so as to slide in the opposite direction, and the pixel data extracting unit 43 Sequentially extract the pixel data of the portion of the captured image that has reached the boundary of the display area of the display device 30 by sliding, and sequentially transmit the extracted pixel data to the one information processing device 1 via the communication unit 44. When a part of the other captured image is displayed on the display device 30 and the pixel data relating to the remaining part is stored in the storage unit 41, the image processing is performed. The unit 42 slides the other image in the opposite direction to the display area vacated by the slide of the one capture image, and further stores the storage unit 41 in the display area vacated by the slide of the other capture image. The pixel data stored in the display device 30 is displayed as an image, or the other capture image is not displayed on the display device 30, and the pixel of the other capture image is displayed on the storage unit 41. When data is stored, the image processing unit 42 displays the pixel data stored in the storage unit 41 as an image in the display area vacated by the slide of the one captured image. Yes, when all of the other captured images are displayed in the display area vacated by the slide of the one captured image, The image processing unit 42 displays an object on the display device 30 based on the object data of the other page stored in the storage unit 41, and the image processing of the one information processing device 1 When the communication unit 44 receives the pixel data related to the one captured image, the unit 42 slides a portion of the one captured image that is already displayed on the display device 30 in the opposite direction, and further slides the portion. When the received pixel data is displayed as an image in the vacant display area of the display device 30 and when the whole of the one captured image is displayed on the display device 30 by receiving the pixel data, Based on the object data stored in the storage unit 41, the image processing unit 42 includes the object included in the one page. The display device 30 is configured to display the project on the display device 30.

  In the present invention, when another captured image is reproduced and displayed on the display device of another information processing apparatus based on the pixel data stored in the storage unit, the image processing unit is displayed in the storage unit. The object is reconstructed based on the stored object data relating to the object of the other page. Therefore, the object can be restored in an editable state not only on one information processing apparatus but also on another information processing apparatus, so that convenience for the user is improved.

  In a tenth aspect of the present invention, when the communication unit 44 of the other information processing apparatus 1 receives the pixel data from the one information processing apparatus 1, the image processing unit 42 receives the received pixel data. The information processing apparatus is characterized by being displayed as an image by overwriting another page displayed on the display device 30 of the information processing apparatus.

  In the present invention, when the communication unit of another information processing apparatus receives pixel data of one captured image, when another page is displayed on the display device, the image processing unit Instead of displaying the image as a slide, the received pixel data is overwritten on another page and displayed as an image. Accordingly, calculations related to slide display of other pages are omitted.

  In an eleventh aspect of the present invention, the pixel data extraction unit 43 of the other information processing apparatus 1 extracts pixel data of the overwritten portion of the other page, and stores the extracted pixel data in the storage unit 41. It is made to do so.

  In the present invention, the pixel data extraction unit of the other information processing apparatus sequentially extracts the pixel data of the portion of the other page overwritten by the one captured image. The storage unit stores the extracted pixel data. Therefore, the captured image that has been overwritten and is no longer displayed can be restored and displayed later, which improves the convenience for the user.

  In a twelfth aspect of the present invention, when the object included in the page is a moving image object, the image processing unit 42 pauses the reproduction of the moving image object in response to an operation including the direction input to the input unit. When the captured image is generated based on the stopped state, and the entire captured image is transmitted from one information processing device 1 to the other information processing device 1 and displayed on the display device 30 The image processing unit 42 reconstructs and displays the moving image object on the page, and starts reproduction from the time when it is paused.

  In the present invention, when the object included in the page is a moving image, the image processing unit pauses the reproduction of the moving image object according to the operation including the input direction, and captures based on the state. Generate an image. When the generated captured image is transmitted from one information processing device to another information processing device and displayed on the display device, the image processing unit reconstructs and displays the moving image object. Furthermore, the moving image object starts to be reproduced from the position where it is paused.

  Therefore, even when the object is a moving image, a captured image can be generated and transferred, so that an increase in the amount of calculation processing during page transfer can be suppressed.

  In a thirteenth aspect of the present invention, when a page is transmitted from one information processing apparatus 1 to another information processing apparatus 1 by a slide, the image processing unit 42 of the information processing apparatus 1 as a transmission destination The page can be enlarged or reduced in accordance with the resolution of 30 and displayed on the display device 30.

  In the present invention, the image processing unit can display an image related to the received page in an enlarged or reduced manner according to the resolution of the display device. Therefore, the image display system of the present invention can include a plurality of display devices having different resolutions.

  A fourteenth aspect of the present invention is characterized in that the display device 30 is a touch panel display, and the input unit 46 is a touch panel.

  In the present invention, the display device is a touch panel display, and the input unit is a touch panel. The operation including the direction input to the input unit is a flick operation. Therefore, the user can operate intuitively using a finger or a stylus pen.

  According to a fifteenth aspect of the present invention, an operation is input in the information processing apparatus 1 that has a function of displaying an image on the display apparatus 30 and can configure the image display system 100 by connecting a plurality of information processing apparatuses 1. An input unit 46, a communication unit 44 that performs data communication with other information processing devices, an image processing unit 42 that performs image processing including generation of a captured image in units of pages, and pixels of an image displayed on the display device A pixel data extraction unit 43 for extracting data; and a storage unit 41 for storing pixel data and object data relating to an object displayed on the display device 30. The image processing unit 42 is provided in the display device 30. In response to an operation including a direction input to the input unit 46, a captured image generated by capturing one page including the displayed object is selected as the operation. The pixel data extraction unit 43 is configured to display the pixel data of the portion of the captured image that has reached the boundary of the display area of the display device 30 by sliding in the captured image. When the extracted pixel data is sequentially transmitted to the other information processing apparatus 1 via the communication unit 44, the image processing unit 42 further receives the pixel data. The received pixel data is displayed on the display device 30 as an image.

  In the present invention, the information processing device has a function of displaying an image on the display device, and is connected to another information processing device via a network, for example. An operation is input to the input unit of the information processing apparatus. The communication unit transmits / receives data to / from other information processing apparatuses via the network. The image processing unit can perform various image processing including generation of a captured image in units of pages. The pixel data extraction unit extracts pixel data of an image displayed on the display device. The storage unit stores pixel data and object data of an object displayed on the display device.

  In response to a user operation including the direction input to the input unit, the image processing unit captures one page including the object to generate a captured image, and the generated captured image corresponds to the user operation. To be displayed on the display device. The pixel data extraction unit sequentially extracts pixel data of a portion of the captured image that reaches the boundary of the display area. The communication unit sequentially transmits the pixel data extracted by the pixel data extraction unit to other information processing apparatuses. In addition, the image processing unit causes the display device to display the pixel data received by the communication unit as an image.

  Therefore, by temporarily converting a page including an object into a captured image and transmitting it, even when the number of objects increases, an increase in the amount of calculation processing of the information processing apparatus can be suppressed. Furthermore, by smoothly transmitting the pixels of the part that has gone out of the display area when the page is slid and displaying it on other information processing devices, smooth page movement is expressed between the mutually connected information processing devices. obtain.

  A sixteenth aspect of the present invention is an information processing method of an image display system 100 in which a plurality of information processing devices 1 having a function of displaying an image on a display device 30 is connected, and each of the information processing devices 1 includes: An input unit 46 to which an operation is input, a communication unit 44 that performs data communication with another information processing device, an image processing unit 42 that performs image processing including generation of a captured image in units of pages, and a display on the display device A pixel data extraction unit 43 that extracts pixel data of an image to be processed, and a storage unit 41 that stores pixel data and object data related to an object displayed on the display device. The unit 42 captures a captured image generated by capturing one page including the object displayed on the display device 30, and operates the captured image including the direction input to the input unit 46. And the step of displaying on the display device 30 so as to slide in the direction related to the operation, and the pixel data extraction unit 43 of the one information processing device 1 of the display device 30 by the slide of the captured image. The step of sequentially extracting the pixel data of the part that has reached the boundary of the display area, sequentially transmitting the extracted pixel data to the other information processing apparatus 1 via the communication unit 44, and the image processing of the other information processing apparatus 1 The unit 42 includes a step of displaying the pixel data received by the communication unit 44 on the display device 30 as an image.

  In the present invention, information is processed by an image display system formed by connecting a plurality of information processing devices having a function of displaying an image on a display device. An operation is input to the input unit of the information processing apparatus. The communication unit transmits / receives data to / from another information processing apparatus via a network, for example. The image processing unit can perform various image processing including generation of a captured image in units of pages. The pixel data extraction unit extracts pixel data of an image displayed on the display device. The storage unit stores pixel data and object data of an object displayed on the display device.

  In the first step, the image processing unit of one information processing apparatus generates a captured image by capturing one page including the object in response to a user operation including the direction input to the input unit. The captured image is displayed on the display device so as to slide in the direction corresponding to the user's operation. In a subsequent step, the pixel data extraction unit sequentially extracts the pixel data of the portion of the captured image that has reached the boundary of the display area, and the communication unit extracts the pixel data extracted by the pixel data extraction unit to another information processing device. Sequentially. In the next step, the image processing unit of another image processing apparatus causes the display device to display the pixel data received by the communication unit as an image.

  Therefore, by temporarily converting a page including an object into a captured image and transmitting it, even when the number of objects increases, an increase in the amount of calculation processing of the information processing apparatus can be suppressed. Furthermore, by smoothly transmitting the pixels of the part that has gone out of the display area when the page is slid and displaying it on other information processing devices, smooth page movement is expressed between the mutually connected information processing devices. obtain.

  A seventeenth aspect of the present invention is a program that includes an input unit 46 and a communication unit 44 and causes the information processing apparatus 1 to display an image on the display device 30 to process information, and the direction received by the input unit 46 A step of capturing a page including an object displayed on the display device 30 in accordance with an operation to be generated to generate a captured image, and the display device 30 to slide the generated captured image in a direction related to the operation. The information processing apparatus is configured to execute the steps of displaying the image data and extracting the pixel data of the portion of the captured image that reaches the boundary of the display area of the display device 30 by sliding.

  In the present invention, an information processing apparatus that includes an input unit and a communication unit and displays an image on a display device causes information to be processed. In the first step of the program, a capture image is generated by capturing a page including an object displayed on the display device in accordance with an operation including a direction received by the input unit. In the subsequent step, the generated captured image is displayed on the display device so as to slide in the operated direction. In the next step, the pixel data of the part reaching the boundary of the display area of the display device by the slide is extracted from the captured image.

  Therefore, by temporarily converting a page including an object into a captured image and transmitting it, even when the number of objects increases, an increase in the amount of calculation processing of the information processing apparatus can be suppressed.

  The embodiment disclosed this time is to be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 2 Control apparatus 3 Touch panel display 20 CPU
21 Memory 22 Storage 23 Communication IF
24 Image output IF
25 Control I / O
30 Display (display device)
31 Display controller 32 Image input IF
33 Touch sensor 34 Touch panel I / O
40 Control Unit 41 Storage Unit 42 Image Processing Unit 43 Pixel Data Extraction Unit 44 Communication Unit 45 Display Unit 46 Input Unit 100 Multi-Display System (Image Display System)

Claims (17)

In an image display system formed by connecting a plurality of information processing devices having a function of displaying an image on each of a plurality of display devices,
Each of the information processing devices
An input unit for inputting an operation;
A communication unit that communicates data with other information processing devices;
An image processing unit that performs image processing including generation of a captured image in units of pages;
A pixel data extraction unit for extracting pixel data of an image displayed on the display device;
A storage unit for storing pixel data and object data relating to an object displayed on the display device;
The image processing unit of the one information processing device captures one page including the object displayed on the display device and generates one captured image according to an operation including the direction input to the input unit, Display on the display device to slide in the direction related to the operation,
The pixel data extraction unit of the one information processing device sequentially extracts pixel data of a portion of the one captured image that has reached the boundary of the display area of the display device by sliding, and the extracted pixel data is transmitted to the communication unit. And sequentially transmitting to other information processing devices via
The image processing unit of the other information processing apparatus displays the pixel data received by the communication unit as an image on a display device. When the communication unit of the other information processing device receives the pixel data from the one information processing device, the image processing unit moves the other image displayed on the display device in the direction related to the operation. 2. The received pixel data is displayed as an image in an area slid and displayed in an area vacated by sliding of the other image in the display area of the display device. The image display system described in 1. The pixel data extraction unit of the other information processing apparatus sequentially extracts the pixel data of the part of the other image that has reached the boundary of the display area by sliding, and stores the extracted pixel data in the storage unit. The image display system according to claim 2, wherein: When the communication unit of the one information processing apparatus transmits the pixel data to the other information processing apparatus, the object data of the object is also transmitted.
The image processing unit of the other information processing apparatus, when the communication unit receives the pixel data of the entire part of the captured image, the object to the display device based on the object data received by the communication unit. The image display system according to claim 1, wherein the image display system is displayed. When the communication unit of the other information processing device receives the pixel data from the one information processing device, the image processing unit captures another page including the object displayed on the display device. The generated other captured image is displayed on the display device so as to slide in the direction related to the operation, and the received pixel is further received in a region vacated by the slide of the other captured image in the display region of the display device. The image display system according to claim 4, wherein the data is displayed as an image. The pixel data extraction unit of the other information processing apparatus extracts pixel data of a part of the other captured image that reaches the boundary of the display area by sliding, and stores the extracted pixel data and the object data of the object The image display system according to claim 5, wherein the image display system is stored in a unit. When the other information processing apparatus displays a part or all of the one captured image on the display device, an operation including a direction opposite to the direction related to the operation input to the one information processing apparatus In response, the image processing unit slides the captured image in the opposite direction,
The pixel data extraction unit sequentially extracts pixel data of a portion of the captured image that has reached the boundary of the display area by sliding, and sequentially transmits the extracted pixel data to the one information processing apparatus via a communication unit. And
When the communication unit receives the pixel data, the image processing unit of the one information processing device slides a portion of the captured image that is already displayed on the display device in the opposite direction, and is further vacated by the slide. The pixel data received in the display area of the display device is displayed as an image,
When the entire captured image is displayed on the display device by receiving the pixel data, the image processing unit is included in the one page based on object data stored in a storage unit The image display system according to any one of claims 1 to 6, wherein an object is displayed on the display device. In a case where the other information processing apparatus displays a part or all of the one captured image on the display device, the image processing unit is responsive to an operation including the opposite direction input to the input unit. The one captured image is slid in the opposite direction,
The pixel data extraction unit sequentially extracts pixel data of a portion of the one captured image that has reached the boundary of the display area of the display device by sliding, and the extracted pixel data is transmitted to the one information via a communication unit. It is designed to send sequentially to the processing device,
When a part of the other image is displayed on the display device and the pixel data relating to the remaining part is stored in the storage unit, the image processing unit is vacated by the slide of the one captured image. The other image is slid in the opposite direction in the display area, and the pixel data stored in the storage unit is displayed as an image in the display area vacated by sliding the other image. ,
Alternatively, when the other image is not displayed on the display device and the pixel data of the other image is stored in the storage unit, the image processing unit performs the slide of the one captured image. In the vacant display area, the pixel data stored in the storage unit is displayed as an image,
When the communication unit receives the pixel data related to the one captured image, the image processing unit of the one information processing device moves a part of the one captured image that is already displayed on the display device in the opposite direction. The received pixel data is displayed as an image on the display area of the display device which is slid and further emptied by the slide,
When the entire captured image is displayed on the display device by receiving the pixel data, the image processing unit is included in the one page based on object data stored in a storage unit. The image display system according to claim 3, wherein the displayed object is displayed on the display device. When the other information processing apparatus displays the one page on the display device, the image processing unit displays the one page in response to an operation including the opposite direction input to the input unit. The captured image generated by capturing is displayed on the display device so as to slide in the opposite direction,
The pixel data extraction unit sequentially extracts pixel data of a portion of the one captured image that has reached the boundary of the display area of the display device by sliding, and the extracted pixel data is transmitted to the one information via a communication unit. It is designed to send sequentially to the processing device,
When a part of the other captured image is displayed on the display device and the pixel data relating to the remaining part is stored in the storage unit, the image processing unit is vacated by the slide of the one captured image The other image is slid in the display area in the opposite direction, and the pixel data stored in the storage unit is displayed as an image in the display area vacated by the slide of the other captured image. And
Alternatively, when the other captured image is not displayed on the display device and pixel data of the other captured image is stored in the storage unit, the image processing unit The pixel data stored in the storage unit is displayed as an image in the display area vacated by the slide,
When all of the other captured images are displayed in the display area vacated by the slide of the one captured image, the image processing unit is based on the object data of the other page stored in the storage unit The object is displayed on the display device,
When the communication unit receives the pixel data related to the one captured image, the image processing unit of the one information processing device moves a part of the one captured image that is already displayed on the display device in the opposite direction. The received pixel data is displayed as an image on the display area of the display device which is slid and further emptied by the slide,
When the entire captured image is displayed on the display device by receiving the pixel data, the image processing unit is included in the one page based on object data stored in a storage unit. The image display system according to claim 6, wherein the displayed object is displayed on the display device. When the communication unit of the other information processing device receives the pixel data from the one information processing device, the image processing unit displays the received pixel data on the display device of the own information processing device. The image display system according to claim 1, wherein the image display system overwrites another existing page and displays the image as an image. The pixel data extraction unit of the other information processing apparatus extracts pixel data of an overwritten portion of the other page, and stores the extracted pixel data in a storage unit. Item 15. The image display system according to Item 10. When the object included in the page is a moving image object, the image processing unit pauses playback of the moving image object according to an operation including the direction input to the input unit, and captures an image based on the stopped state. Is generated, and
When the entire captured image is transmitted from one information processing apparatus to another information processing apparatus and displayed on the display apparatus, the image processing unit reconstructs and displays the moving image object on the page. The image display system according to any one of claims 4 to 6, wherein reproduction is started from a point of time when the operation is paused. When a page is transmitted from one information processing apparatus to another information processing apparatus by slide, the image processing unit of the information processing apparatus that is a transmission destination enlarges or reduces the page according to the resolution of the display apparatus. The image display system according to any one of claims 1 to 12, wherein the image display system can display on the display device.   The image display system according to claim 1, wherein the display device is a touch panel display, and the input unit is a touch panel. In an information processing apparatus having a function of displaying an image on a display apparatus and capable of configuring an image display system by connecting a plurality of information processing apparatuses,
An input unit for inputting an operation;
A communication unit that communicates data with other information processing devices;
An image processing unit that performs image processing including generation of a captured image in units of pages;
A pixel data extraction unit for extracting pixel data of an image displayed on the display device;
A storage unit for storing pixel data and object data relating to an object displayed on the display device;
The image processing unit captures a captured image generated by capturing one page including an object displayed on the display device, and a direction related to the operation according to an operation including a direction input to the input unit. To display on the display device to slide
The pixel data extraction unit sequentially extracts pixel data of a portion of the captured image that has reached the boundary of the display area of the display device by sliding, and the extracted pixel data is transmitted to the other information processing device via the communication unit. Are sent sequentially,
The image processing unit is further configured to display the received pixel data as an image on the display device when the communication unit receives the pixel data. An information processing method for an image display system in which a plurality of information processing devices having a function of displaying an image on a display device are connected,
Each of the information processing devices includes an input unit to which an operation is input, a communication unit that performs data communication with other information processing devices, an image processing unit that performs image processing including generation of a captured image in units of pages, A pixel data extraction unit that extracts pixel data of an image displayed on the display device; and a storage unit that stores pixel data and object data relating to an object displayed on the display device,
The image processing unit of one information processing apparatus captures one page including the object displayed on the display device and generates the captured image according to the operation including the direction input to the input unit. Displaying on the display device to slide in the direction according to
The pixel data extraction unit of the one information processing device sequentially extracts pixel data of the captured image that has reached the boundary of the display area of the display device by sliding, and the extracted pixel data is transmitted via the communication unit. Sequentially transmitting to other information processing devices;
And an image processing unit of the other information processing apparatus including a step of displaying the pixel data received by the communication unit as an image on a display device. A program that includes an input unit and a communication unit and causes an information processing device to display an image on a display device to process information,
In response to an operation including a direction received by the input unit, capturing a page including an object displayed on the display device to generate a captured image;
Displaying the generated captured image on the display device so as to slide in the direction related to the operation;
A program causing the information processing apparatus to execute a step of extracting pixel data of a portion of the captured image that reaches a boundary of a display area of the display device by sliding.

Images