JP2008117270A - Information processing system, information processor, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently develop an application to which a panel compatible with multipoint output is applied. <P>SOLUTION: A panel emulator system is configured of a computer 61 as a server and one or more computers 71<SB>1</SB>, 71<SB>2</SB>as clients, and the one or more computers 71<SB>1</SB>, 71<SB>2</SB>transmit mouse information to the server, the computer 61 receives the mouse information transmitted from the one or more computers 71<SB>1</SB>, 71<SB>2</SB>, and generates necessary information required by the application to which the panel compatible with multipoint output is applied on the basis of the mouse information. This invention can be applied to a case or the like of emulating a digital camera adopting a display panel outputting information of a plurality of points on the panel. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program, and in particular, efficiently develops an application to which a panel that can output information about points on the panel at a plurality of points is applied. The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program.

  As a panel capable of outputting information on points on a panel for a plurality of points (hereinafter also referred to as a multi-point output panel), for example, an optical sensor is built in a liquid crystal display device, and external light input is performed by the optical sensor. There has been proposed a display panel (hereinafter also referred to as an input display panel) capable of inputting light by detection (see, for example, Patent Documents 1 and 2).

  In input display panels (including multi-point output panels), methods for detecting external input can be broadly divided into using an object with an external light source such as an LED (Light Emitting Diode) (for example, a pen). A method of detecting the light emitted from the object with an optical sensor and inputting light with a finger or pen that does not have such an external light source, and the light emitted from the liquid crystal display device, that is, a liquid crystal display There is a method in which light transmitted through the backlight of the device is reflected by a finger or pen touching the display surface of the liquid crystal display device and returned to the inside of the liquid crystal display device by a light sensor.

  Here, in a conventional electrostatic or pressure-sensitive touch panel, panel information that is information on a point on the touch panel where the input is made is output in response to an input from the outside. Only one point could be output.

  That is, for example, even if the user touches two points on the touch panel, one of the two points on the touch panel, such as the point with the higher pressure or the point touched first, is selected. Panel information is output only for points.

  On the other hand, the input display panel can output panel information on a plurality of points, and is expected to be applied to applications such as products to which a conventional touch panel is applied in the future.

JP 2000-019478 JP 2004-127272 A

  Since the input display panel is a new device, it may be difficult to efficiently develop an application to which such an input display panel is applied.

  In other words, software of an application to which the input display panel is applied needs to be designed so as to perform various processes based on panel information about a plurality of points that can be output by the input display panel.

  However, in the initial stage of application development, necessary hardware, for example, the input display panel itself is often not completed, and in this case, development of application software is performed using actual hardware. Can't do it.

  Also, if software development is started after the actual hardware is completed, the development period will be long.

  Further, when a problem occurs in the software, the cause of the problem is in the software of the input display panel, or other than that, for example, in the control unit (as upper software) that controls the entire application, etc. It is difficult to determine whether there is any.

  Therefore, an application that uses an input display panel using an emulator that can input a plurality of points and output panel information for the plurality of points (hereinafter referred to as multipoint output as appropriate). There is a way to do development.

  However, for example, on a computer (hereinafter also referred to as a PC as appropriate) equipped with a general operating system (OS) such as Windows (registered trademark) or Linux (registered trademark), a plurality of mice as pointing devices are connected. Even in this case, only one mouse cursor that can input (instruct) a point is displayed.

  Therefore, it is difficult to make a PC equipped with a general OS function as an emulator that can output multiple points without changing the OS.

  On the other hand, if a PC with a general OS is made to function as an emulator capable of multi-point output by changing the OS, it takes a huge cost to create the emulator.

  The present invention has been made in view of such a situation, and makes it possible to efficiently develop an application to which a panel capable of multipoint output is applied.

  An information processing system according to a first aspect of the present invention is an information processing system that includes a server and one or more clients, and that emulates the panel capable of outputting information about points on the panel for a plurality of points. Receiving means for transmitting pointing device information transmitted from one or more of the clients, wherein the client has transmitting means for transmitting pointing device information relating to a point designated by the pointing device to the server. And generating means for generating necessary information required by an application to which the panel is applied based on the pointing device information.

  In the first aspect as described above, the client transmits pointing device information regarding a point designated by the pointing device to the server. On the other hand, the server receives pointing device information transmitted from one or more clients, and generates necessary information required by an application to which the panel is applied based on the pointing device information.

  An information processing apparatus according to a second aspect of the present invention is an information processing apparatus constituting an information processing system that performs emulation of the panel capable of outputting information about points on a panel for a plurality of points, and the information processing system And receiving means for receiving pointing device information about a point designated by the pointing device, which is transmitted from one or more clients constituting the network, and an application to which the panel is applied based on the pointing device information Generating means for generating information.

  An information processing method or program according to a second aspect of the present invention provides an information processing method for an information processing apparatus constituting an information processing system that performs emulation of the panel capable of outputting information about points on the panel for a plurality of points. Or a program that causes a computer that configures the information processing system to execute information processing, and receives pointing device information regarding a point indicated by the pointing device that is transmitted from one or more clients that configure the information processing system. Generating necessary information required by an application to which the panel is applied based on the pointing device information.

  In the second aspect as described above, pointing device information relating to a point designated by the pointing device transmitted from one or more clients configuring the information processing system is received. Based on the pointing device information, necessary information required by an application to which the panel is applied is generated.

  An information processing apparatus according to a third aspect of the present invention is an information processing apparatus constituting an information processing system that performs emulation of the panel capable of outputting information about points on a panel for a plurality of points, and the information processing system Required information required by an application to which the panel is applied based on the pointing device information received from the pointing device information received from one or more information processing apparatuses constituting Transmitting means for transmitting the pointing device information to a server for generating

  An information processing method or program according to a third aspect of the present invention is an information processing method for an information processing apparatus constituting an information processing system that performs emulation of the panel capable of outputting information on points on the panel for a plurality of points. Or a program that causes a computer that constitutes the information processing system to execute information processing, and that points to a point indicated by a pointing device that is transmitted from one or more information processing apparatuses or computers that constitute the information processing system Receiving device information, and transmitting the pointing device information to a server that generates necessary information required by an application to which the panel is applied based on the pointing device information.

  In the third aspect as described above, the panel is based on pointing device information about a point designated by the pointing device, which is transmitted from one or more information processing apparatuses or computers constituting the information processing system. The pointing device information is transmitted to a server that generates necessary information required by the application to be applied.

  The program can be transmitted via a transmission medium, or can be recorded on a recording medium.

  According to the first to third aspects of the present invention, it is possible to efficiently develop an application to which a panel capable of multipoint output is applied.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

An information processing system according to the first aspect of the present invention includes:
A server (eg, server computer 61 of FIG. 5);
One or more clients (for example, client computers 71 ₁ and 71 ₂ in FIG. 5),
An information processing system (for example, the panel emulator system in FIG. 5) that emulates the panel (for example, the input display panel 2 in FIGS. 1 to 3) that can output information related to points on the panel for a plurality of points. ,
The client includes transmission means (for example, the transmission unit 303 in FIG. 10) that transmits the pointing device information regarding the point designated by the pointing device (for example, the mouse 207A in FIG. 8) to the server.
The server
Receiving means (for example, the acquiring unit 312 in FIG. 10) for receiving pointing device information transmitted from one or more clients;
Based on the pointing device information, generation means (for example, a generation unit 313 in FIG. 10) that generates necessary information required by an application to which the panel is applied.

An information processing apparatus according to the second aspect of the present invention provides:
An information processing system (for example, the panel emulator system in FIG. 5) that emulates the panel (for example, the input display panel 2 in FIGS. 1 to 3) that can output information about points on the panel for a plurality of points is configured. Information processing apparatus (for example, server computer 61 in FIG. 5),
Pointing device information regarding a point designated by a pointing device (for example, mouse 207A in FIG. 8) transmitted from one or more clients (for example, client computers 71 ₁ and 71 ₂ in FIG. 5) constituting the information processing system. Receiving means (for example, the acquisition unit 312 in FIG. 10),
Generation means (for example, the generation unit 313 in FIG. 10) that generates necessary information required by the application to which the panel is applied based on the pointing device information.

The information processing apparatus according to the second aspect includes
Storage means (for example, storage unit 314 in FIG. 16) for storing the pointing device information transmitted from one or more clients can be further provided.
In this case, the generation unit can generate necessary information from the pointing device information stored in the storage unit.

An information processing method or program according to the second aspect of the present invention includes:
An information processing system (for example, the panel emulator system in FIG. 5) that emulates the panel (for example, the input display panel 2 in FIGS. 1 to 3) that can output information about points on the panel for a plurality of points is configured. An information processing method of an information processing apparatus (for example, the server computer 61 in FIG. 5) or a program that causes a computer constituting the information processing system to execute information processing,
Pointing device information related to a point indicated by a pointing device (for example, the mouse 207A in FIG. 8) transmitted from one or more clients constituting the information processing system is received (for example, step S22 in FIG. 11).
Based on the pointing device information, necessary information required by an application to which the panel is applied is generated (for example, step S23 in FIG. 11).
Includes steps.

An information processing apparatus according to the third aspect of the present invention provides:
An information processing system (for example, the panel emulator system in FIG. 5) that emulates the panel (for example, the input display panel 2 in FIGS. 1 to 3) that can output information about points on the panel for a plurality of points is configured. Information processing apparatus (for example, the client computers 71 ₁ and 71 ₂ in FIG. 5),
Pointing device information related to a point indicated by a pointing device (for example, the mouse 207A in FIG. 8) transmitted from one or more information processing apparatuses constituting the information processing system is received, and based on the pointing device information, Transmission means for transmitting the pointing device information to a server (for example, the server computer 61 in FIG. 5) that generates necessary information required by the application to which the panel is applied (for example, the transmission unit 303 in FIG. 10). Is provided.

An information processing method or a program according to the third aspect of the present invention includes:
An information processing system (for example, the panel emulator system in FIG. 5) that emulates the panel (for example, the input display panel 2 in FIGS. 1 to 3) that can output information about points on the panel for a plurality of points is configured. An information processing method of an information processing apparatus (for example, the client computers 71 ₁ and 71 ₂ in FIG. 5) or a program that causes a computer constituting the information processing system to execute information processing,
Pointing device information about a point designated by a pointing device (for example, the mouse 207A in FIG. 8) transmitted from one or more information processing apparatuses or computers constituting the information processing system is received, and the pointing device information is included in the pointing device information. A step of transmitting the pointing device information to a server (for example, the server computer 61 of FIG. 5) that generates necessary information required by the application to which the panel is applied (for example, step S13 of FIG. 11). )including.

  Embodiments of the present invention will be described below with reference to the drawings.

  1 to 3 show an input display system to be emulated (a system is a logical collection of a plurality of devices, and whether or not the devices of each configuration are in the same casing) No configuration example).

  That is, FIG. 1 shows a first configuration example of an input display system as an application to which an input display panel is applied.

  Here, an input display system as an application to which the input display panel is applied includes, for example, a digital camera. In this case, the input display panel functions as, for example, a display unit that displays a so-called through image, a recorded image, and the like, and an input unit for a user to perform an operation input.

  In FIG. 1, the input display system includes an application unit 1 and an input display panel 2.

  Further, in FIG. 1, the application unit 1 includes a control unit 11 and controls the entire input display system including the input display panel 2.

  That is, the control unit 11 controls the input display panel 2 and other blocks (not shown) constituting the input display system. Further, the control unit 11 performs various processes based on information supplied from the input display panel 2. Specifically, the control unit 11 generates display data corresponding to an image to be displayed on the input display panel 2 based on information supplied from the input display panel 2 and supplies the display data to the input display panel 2.

  The input display panel 2 includes a display display data processing unit 21, an input / output display 22, a received light data processing unit 23, an image processing unit 24, and a generation unit 25, and is supplied from the application unit 1 (the control unit 11). An image is displayed based on the display data. Furthermore, when a finger or pen touches the display surface on which the image is displayed, the input display panel 2 supplies information related to one or more points where the touch has occurred to the application unit 1 (the control unit 11). To do.

  That is, the display data from the application unit 1 is supplied to the display display data processing unit 21. The display display data processing unit 21 controls the input / output display 22 based on the display data from the application unit 1, thereby causing the input / output display 22 to display an image corresponding to the display data.

  The input / output display 22 includes, for example, a liquid crystal display device and an optical sensor, and displays an image according to control from the display display data processing unit 21. The input / output display 22 detects external light input to the display surface for displaying an image, for example, at a predetermined frame period, and supplies light reception data as a detection result to the light reception data processing unit 23. To do.

  The light reception data processing unit 23 performs amplification processing and filter processing on the light reception data supplied from the input / output display 22 so that, for example, a finger or a pen is in contact with the entire display surface of the input / output display 22. Image data (hereinafter also referred to as contact point detection image data) in which the luminance of the portion and the portion where nothing is in contact is generated at a predetermined frame period and supplied to the image processing unit 24.

  The image processing unit 24 performs binarization processing, noise removal processing, labeling processing, and the like for each frame on the contact point detection image data from the light reception data processing unit 23, so that the input / output display 22 Point information relating to points on the display surface is generated for each frame for one or more points on the display surface that are in contact with a finger or pen as input from outside (touch input), and supplied to the generation unit 25. .

  Here, the point information includes position information (for example, the coordinates of the center of gravity of the touch area, the position information indicating the position of the point on the display surface on the area where the finger or pen is in contact (hereinafter also referred to as the touch area), The coordinates of the upper left point of the smallest rectangle surrounding the touch area, etc., and size information indicating the size of the touch area (for example, the area of the touch area and the x of the left and right sides of the smallest rectangle surrounding the touch area) Coordinates, and y-coordinates of the upper and lower sides).

  The generation unit 25 generates necessary information required by the application unit 1 (the control unit 11) from the point information from the image processing unit 24 and supplies the necessary information to the application unit 1.

  Here, when the application unit 1 changes the image displayed on the input / output display 22 in accordance with, for example, the locus drawn by the finger in contact with the input / output display 22 (display surface thereof), the input / output Information that can recognize a trajectory drawn by a finger in contact with the display 22 is necessary information.

  Specifically, for example, when a stroke drawn by a finger is referred to as a stroke from when the contact of the finger to the input / output display 22 is started until the contact is released, the stroke is recognized. Occurred, for example, when the contact of a finger was started, the contact of the finger was released, or a touch area of a certain frame moved by which touch area of the previous frame. Information (correspondence between the touch area of a certain frame and the touch area of the previous frame) is necessary, and such information becomes necessary information.

  Next, FIG. 2 shows a second configuration example of the input display system as an application to which the input display panel is applied.

  The input display system of FIG. 2 is configured in the same manner as the input display system of FIG. 1 except that the generation unit 25 is provided in the application unit 1 instead of the input display panel 2.

  Therefore, in the input display system of FIG. 1, necessary information output from the generation unit 25 is supplied from the input display panel 2 to the application unit 1, but in the input display system of FIG. 2, The point information output from the image processing unit 24 to the generating unit 25 is supplied from the input display panel 2 to the application unit 1.

  Next, FIG. 3 shows a third configuration example of the input display system as an application to which the input display panel is applied.

  The input display system of FIG. 3 is configured in the same manner as the input display system of FIG. 1 except that the image processing unit 24 and the generation unit 25 are provided in the application unit 1 instead of the input display panel 2.

  Therefore, in the input display system of FIG. 1, necessary information output from the generation unit 25 is supplied from the input display panel 2 to the application unit 1, but in the input display system of FIG. 3, From the input display panel 2, contact point detection image data output from the light reception data processing unit 23 to the image processing unit 24 is supplied to the application unit 1.

  Here, information relating to a point on the input / output display 22 where the input is input, which the input display panel 2 outputs in response to an external input such as a finger contact with the input / output display 22, is referred to as panel information. Accordingly, in FIG. 1, necessary information is panel information, in FIG. 2, point information is shown, and in FIG. 3, contact point detection image data is panel information.

  The input display panel 2 can output panel information about a plurality of points on the input / output display 22.

  Next, processing of the input display system of FIGS. 1 to 3 will be described with reference to FIG.

  If the input display system is, for example, a digital camera, the control unit 11 generates image data for displaying a recorded (photographed) image in a predetermined size, and displays it as display data. The data is supplied to the display data processing unit 21.

  The display display data processing unit 21 receives the image data from the control unit 11 and causes the input / output display 22 to display an image corresponding to the image data.

  On the other hand, the input / output display 22 detects light input from the outside with respect to the display surface for displaying an image at a predetermined frame period, and supplies light reception data as a detection result to the light reception data processing unit 23.

  The light reception data processing unit 23 generates contact point detection image data from the light reception data supplied from the input / output display 22 at a predetermined frame period, and supplies the image data to the image processing unit 24.

  The image processing unit 24 uses the contact point detection image data from the received light data processing unit 23 to display point information on a point (touch area) where a finger or a pen is in contact on the display surface of the input / output display 22. For each generation and supply to the generation unit 25.

  The generation unit 25 generates necessary information required by the control unit 11 from the point information from the image processing unit 24 and supplies the necessary information to the control unit 11.

  The control unit 11, for example, image data for displaying an image that has been enlarged, reduced, rotated, or moved from the image currently displayed on the input / output display 22 according to the necessary information supplied from the generation unit 25. Alternatively, it generates image data or the like for displaying an image recorded next to or before the image currently displayed on the input / output display 22 in a predetermined size, and supplies it to the input display panel 2 as display data. To do.

  Thereby, the display of the input / output display 22 is updated (changed) in accordance with the input (contact) to the input / output display 22.

  In this case, for example, with respect to the image displayed on the input / output display 22, the user moves the finger touching the input / output display 22 from right to left so as to proceed to display the next recorded image. Thus, it is possible to realize a user interface in which the display on the input / output display 22 is changed to the next recorded image.

  Next, FIG. 5 shows a configuration example of an embodiment of a panel emulator system that emulates the input display system of FIGS. 1 to 3.

In FIG. 5, the panel emulator system includes one computer (hereinafter also referred to as a server computer) 61 serving as a server and one or more, for example, two computers (hereinafter also referred to as client computers) 71 _{1 serving} as clients. , 71 ₂ , and the server computer 61 and each of the client computers 71 ₁ or 71 ₂ can communicate with each other via the network 81.

In FIG. 5, two computers 71 ₁ and 71 ₂ are provided as the computers serving as clients. However, the number of computers serving as clients may be one or three or more. The number of clients (assumed computers) is determined by, for example, how many points (touch area) output from the input display panel 2 to be emulated can be processed by the control unit 11. Etc.

The communication method performed by the server computer 61 and each of the client computers 71 ₁ or 71 ₂ is not particularly limited, and a wireless LAN (Local Area Network), a wired LAN, serial communication, or infrared light is used. Communication and others can be employed.

In the panel emulator system of FIG. 5, the server computer 31 and the client computers 71 ₁ and 71 ₂ execute various programs, and if necessary, between the server computer 31 and the client computer 71 ₁ or 71 ₂ , The input display system is emulated by exchanging data (information).

Here, hereinafter, the client computers 71 ₁ and 71 ₂ will be collectively referred to as a client computer 71 as appropriate.

FIG. 6 shows a program for causing the server computer 31 and the client computers 71 ₁ and 71 ₂ of FIG. 5 to function as a panel emulator system.

  A program as a panel emulator system includes a panel emulator 51, a control unit program 52, and a display program 53.

  The panel emulator 51 is composed of a server program 51A and a client program 51B, and emulates a panel that can output information on points on the panel at a plurality of points, specifically, for example, emulation of the input display panel 2. Do.

  That is, the server program 51A (more precisely, a server that is a computer executing the server program 51A) is transmitted from one or more client programs 51B (more precisely, a client that is a computer executing the client program 51B). The received pointing device information relating to the point indicated by the pointing device is received, and the application to which the input display panel 2 is applied based on the pointing device information, specifically, for example, FIG. 1 to FIG. Necessary information required by the control unit 11 constituting the application unit 1 of the input display system is generated and supplied to the control unit program 52.

  The client program 51B transmits pointing device information regarding the point indicated by the pointing device to the server program 51A.

  The control unit program 52 (more precisely, the computer executing the control unit program 52) is an application to which the input display panel 2 is applied, for example, the application unit 1 of the input display system of FIGS. Emulation of the control unit 11 to be configured is performed.

  That is, the control unit program 52 transmits display data corresponding to an image to be displayed on the input display panel 2 to the control unit 11 (FIGS. 1 to 3) based on necessary information supplied from the server program 51A of the panel emulator 51. Similarly, it is generated and supplied to the display program 53.

  The display program 53 (more precisely, the computer executing the display program 53) is the same as the image displayed on the input / output display 22 of the input display panel 2 corresponding to the display data supplied from the application unit 1. The image is displayed corresponding to the display data supplied from the control unit program 52.

  Next, FIG. 7 shows a hardware configuration example of the server computer 61 of the panel emulator system of FIG.

  The server computer 61 includes a CPU (Central Processing Unit) 102. An input / output interface 110 is connected to the CPU 102 via the bus 101, and the CPU 102 operates the input unit 107 including the mouse 107 </ b> A, the keyboard 107 </ b> B, and the like via the input / output interface 110. When a command is input as a result, a program stored in a ROM (Read Only Memory) 103 is executed accordingly. Alternatively, the CPU 102 also transfers from a program stored in the hard disk 105, a program transferred from a satellite or a network, received by the communication unit 108 and installed in the hard disk 105, or a removable recording medium 111 attached to the drive 109. The program read and installed in the hard disk 105 is loaded into a RAM (Random Access Memory) 104 and executed. Thereby, the CPU 102 performs processing according to a flowchart described later or processing performed according to a configuration of a block diagram described later. Then, the CPU 102 outputs (displays) the processing result from the output unit 106 configured with an LCD (Liquid Crystal Display), a speaker, or the like, for example, via the input / output interface 110, or communicates as necessary. The data is transmitted from the unit 108 and further recorded on the hard disk 105.

  The hard disk 105 of the server computer 61 has at least an OS, a server program 51A, a control unit program 52, and a display program 53 installed.

  Here, various programs can be recorded in advance on a hard disk 105 or a ROM 103 as a recording medium built in the server computer 61.

  Alternatively, the program is stored temporarily on a removable recording medium 111 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored permanently (recorded). Such a removable recording medium 111 can be provided as so-called package software.

  The program is installed in the server computer 61 from the removable recording medium 111 as described above, and is transferred from the download site to the server computer 61 via a digital satellite broadcasting artificial satellite or by a LAN (Local The network is transferred to the server computer 61 via a network such as an area network or the Internet. The server computer 61 receives the program transferred in this way by the communication unit 108 and installs it in the built-in hard disk 105. be able to.

  FIG. 8 shows a hardware configuration example of the client computer 71 of the panel emulator system of FIG.

  The client computer 71 includes a bus 201, a CPU 202, a ROM 203, a RAM 204, a hard disk 205, an output unit 206, a mouse 207A, a keyboard 207B, an input unit 207, a communication unit 208, a drive 209, an input / output interface 210, and a removable recording. A medium 211 is included.

  Note that the bus 201 or the removable recording medium 211 of the client computer 71 is configured in the same manner as the bus 101 or the removable recording medium 111 of the server computer 61 shown in FIG.

  However, the hard disk 205 of the client computer 71 has at least an OS and a client program 51B installed.

  Next, referring to FIG. 9, the OS, server program 51A, control unit program 52, and display program 53 are executed in the server computer 61, and the OS and client program 51B are executed in the client computer 71. Processing of the server computer 61 and the client computer 71 will be described.

  Note that the subject performing the process is a computer executing a program, but hereinafter, the subject performing the process is also referred to as a program as appropriate for convenience of explanation.

On the client computer 71 ₁ , a client window 72 ₁ and a cursor 73 ₁ are displayed.

Here, the client window 72 ₁ is displayed by the client program 51B (FIG. 6), and the cursor 73 ₁ is displayed by the OS.

In the client computer 71 ₁ , the cursor 73 ₁ is a pointing device, for example, moves according to the operation of the mouse 207A.

The client computer 71 _1, in response to a request from the server computer 61, that the mouse 207A a pointing device instructs, i.e., a point on the client window 72 ₁ cursor 73 ₁ moving according to the operation of the mouse 207A instructs Is acquired as pointing device information and transmitted to the server computer 61.

Any client computer _712, as with the client computer 71 _1, displays the client window 72 ₂ and the cursor 73 _2, the cursor 73 ₂ is moved according to the operation of the mouse 207A a pointing device.

Then, the client computer _712, like the client computers 71 _1, that the mouse 207A instructs, i.e., the mouse information about the client window 72 ₂ points on the cursor 73 ₂ moving in accordance with operation of the mouse 207A instructs In response to a request from the server computer 61, the data is transmitted to the server computer 61.

  On the other hand, the server computer 61 displays a server window 62, a display window 63, and a cursor 65.

  Here, the server window 62 is displayed by the server program 51A (FIG. 6), and the display window 63 is displayed by the display program. The cursor 65 is displayed by the OS.

  The server computer 61 requests mouse information from the client computer 71, and receives mouse information transmitted from the client computer 71 in response to the request.

In the server computer 61, the mouse information from the client computer 71 ₁ and the mouse information from the client computer 71 ₂ are the points (touch area) where the touch input to the input / output display 22 of the input display panel 2 has occurred. Treated as information.

That is, the server computer 61, based on the mouse data from the client computer 71 ₁ and 71 _2, the cursor 66 ₁ and 66 ₂ mice 207A of the client computers 71 ₁ and 71 ₂ is representative of a respective point of instruction, the server window 62 Display above.

Specifically, the server computer 61, based on the mouse data from the client computers 71 _1, the position on the server window 62 corresponding to the position of the cursor 73 ₁ on the client window _721, corresponding to the cursor 73 ₁ A cursor 66 ₁ is displayed. Further, the server computer 61, based on the mouse data from the client computer _712, the position on the server window 62 corresponding to the position of the cursor 73 ₂ on the client window 72 _2, the cursor 66 ₂ corresponding to the cursor 73 ₂ Is displayed.

  The server computer 61 displays an image corresponding to the display data on the display window 63.

  That is, in the server computer 61, the server program 51A (FIG. 6) receives mouse information from the client computer 71, and necessary information required by the control unit 11 (FIGS. 1 to 3) based on the mouse information. Is generated and supplied to the control unit program 52 (FIG. 6).

Where it is displayed on the server window 62, the cursor 66 ₁ corresponding to the cursor 73 _1, the cursor 66 ₂ corresponding to the cursor 73 _2, since they represent a touch input to the input-output display 22, a cursor 66 _1, or 66 ₂ is also referred to as a touch cursor 66 ₁ or 66 ₂ , respectively.

  The control unit program 52 generates display data corresponding to an image to be displayed on the input display panel 2 based on the necessary information supplied from the server program 51A, and supplies the display data to the display program 53 (FIG. 6).

  Then, the display program 53 displays an image corresponding to the display data supplied from the control unit program 52 on the display window 63.

In FIG. 9 (the same applies to FIGS. 13 to 15 described later), the display window 63 corresponds to the touch cursors 66 ₁ and 66 ₂ displayed on the server window 62 in addition to the image corresponding to the display data. Cursors 67 ₁ and 67 ₂ are displayed at positions on the display window 63 corresponding to the positions of the touch cursors 66 ₁ and 66 ₂ on the server window 62, respectively.

Accordingly, the cursors 67 ₁ and 67 ₂ displayed in the display window 63 represent the points designated by the mouse 207A as touch points on the client computers 71 ₁ and 71 ₂ , respectively. Referring to the display window 63, when a certain image is displayed on the display window 63, when the touch input is made at the point represented by the cursors 67 ₁ and 67 ₂ , the display of the display window 63 is displayed. , You can see how it will change.

Here, hereinafter, the client windows 72 ₁ and 72 ₂ will be collectively referred to as a client window as appropriate. Similarly, the cursors 73 ₁ and 73 ₂ are collectively referred to as a cursor 73, and the touch cursors 66 ₁ and 66 ₂ are collectively referred to as a touch cursor 66.

  Next, FIG. 10 shows a functional configuration example of the panel emulator system of FIG.

  The functional configuration example of FIG. 10 (same as the functional configuration example of FIG. 16 described later) is executed in the server computer 61 by the server program 51A, the control unit program 52, and the display program 53, and the client computer 71. This is realized (conceptually) by executing the client program 51B in FIG.

  In FIG. 10, the client program 51 </ b> B executed by the client computer 71 includes a display unit 301, an acquisition unit 302, and a transmission unit 303.

  In FIG. 10, the server program 51 </ b> A executed by the server computer 61 includes a display unit 311, an acquisition unit 312, and a generation unit 313. Further, the control unit program 52 executed by the server computer 61 is configured by the control unit 321, and the display program 53 executed by the server computer 61 is configured by the display unit 331.

  In the client program 51B, the display unit 301 displays the client window 72 (FIG. 9), and further, a cursor 73 (FIG. 9) that moves according to the operation of the mouse 207A based on the mouse information supplied from the acquisition unit 302 and others. Is displayed on the client window 72.

  The acquisition unit 302 acquires mouse information related to a point on the client window 72 indicated by the cursor 73 that moves according to the operation of the mouse 207A based on a signal supplied from the mouse 207A or the like, and displays the display unit 301. To the transmission unit 303.

  Here, the mouse information includes on / off information indicating whether the button of the mouse 207A is turned on or off (indicating whether the button of the mouse 207A is operated (pressed)). When the button of the mouse 207A is turned on, a touch input is made to the input / output display 22 of the input display panel 2, and when the button is turned off, a touch input is not made to the input / output display 22.

  Further, the mouse information includes, for example, coordinates (x, y) indicating the position on the client window 72 of the point indicated by the mouse 207A, that is, the cursor 73 that moves in accordance with the operation of the mouse 207A. The coordinates (x, y) represent the coordinates of the position of the touch input when the touch input is made on the input / output display 22 of the input display panel 2, that is, the coordinates of the center of gravity of the touch area, for example.

  In addition, the mouse information can include, for example, information on a predetermined area including the point indicated by the mouse 207A, that is, the position of the cursor 73 that moves in accordance with the operation of the mouse 207A.

  That is, when a finger or a pen touches the input / output display 22 of the input display panel 2 and a touch input is made to the input / output display 22, the contact portion between the finger or pen and the input / output display 22 is infinitely small. This is not a point of size, but an area of finite size (touch area). The mouse information includes information on the touch area of the finite size (hereinafter referred to as touch area information as appropriate), the area of the touch area, the x coordinates of the left and right sides of the smallest rectangle surrounding the touch area, and You can include y-coordinates for each of the top and bottom sides.

  Note that the point designated by the mouse 207A is not an area of a finite size such as the touch area, so it is difficult to obtain touch area information only from the point designated by the mouse 207A. Therefore, the touch area information can be set by the server program 51A or the client program 51B. Details of the setting of the touch area information will be described later.

  The transmission unit 303 transmits the mouse information from the acquisition unit 302 to the server program 51A in response to a request from the server program 51A (the acquisition unit 312).

In the server program 51A, the display unit 311 displays the server window 62 (FIG. 9). Further, the display unit 311, based on the mouse information supplied from the acquisition unit 312, a display touch cursor 66 ₁ corresponding to the cursor 73 _1, and the touch cursor 66 ₂ corresponding to the cursor 73 _2, on the server window 62 To do.

  The acquisition unit 312 requests mouse information from the client program 51B (the transmission unit 303). Furthermore, the acquisition unit 312 acquires the mouse information transmitted from the client program 51B in response to the request for the mouse information, and supplies the acquired information to the display unit 311 and the generation unit 313.

  The generation unit 313 generates necessary information required by the control unit 11 (FIGS. 1 to 3) of the application unit 1 from the mouse information from the acquisition unit 312 and supplies the necessary information to the control unit program 52 (the control unit 321). To do.

  In the control unit program 52, the control unit 321 performs the same processing as the control unit 11. That is, the control unit 321 generates display data according to the necessary information supplied from the generation unit 313 and supplies the display data to the display program 53 (the display unit 331 thereof).

In the display program 53, the display unit 331 displays the display window 63 (FIG. 9). In addition, the display unit 331 displays an image corresponding to the display data from the control unit 321 and the cursors 67 ₁ and 67 ₂ on the display window 63.

  Next, processing of the panel emulator system of FIG. 10 will be described with reference to the flowchart of FIG.

  First, the processing of the client program 51B will be described with reference to the first flowchart from the right in FIG.

  In the client program 51B, in step S11, the transmission unit 303 determines whether or not there is a request for mouse information from the server program 51A (the acquisition unit 312).

  If it is determined in step S11 that no mouse information is requested from the server program 51A, the process returns to step S11.

  If it is determined in step S11 that there is a request for mouse information from the server program 51A, the transmission unit 303 supplies the request for mouse information to the acquisition unit 302, and the process proceeds to step S12.

  In step S12, the acquisition unit 302 acquires mouse information regarding a point on the client window 72 indicated by the cursor 73 that moves in accordance with the operation of the mouse 207A in response to a request for mouse information from the transmission unit 303. Then, the process proceeds to step S13.

  In step S13, the transmission unit 303 transmits the mouse information from the acquisition unit 302 to the server program 51A, returns to step S11, and the same processing is repeated thereafter.

  Next, the processing of the server program 51A will be described with reference to the second flowchart from the right in FIG.

  In the server program 51A, in step S21, the acquisition unit 312 requests mouse information from the client program 51B (the transmission unit 303). Then, the acquisition unit 312 waits for the mouse information to be transmitted from the client program 51B, proceeds from step S21 to S22, acquires (receives) the mouse information from the client program 51B, and displays the display unit 311 and the generation unit 311. The process proceeds to step S23.

  In step S23, the generation unit 313 generates necessary information from the mouse information from the acquisition unit 312 and proceeds to step S24. In step S24, the generation unit 313 supplies the necessary information generated in step S23 to the control unit program 52 (the control unit 321), returns to step S21, and the same processing is repeated thereafter.

Note that the processing in steps S21 to S24 is performed for all clients (for example, client computers 71 ₁ and 71 _{2 in} FIG. 5) that constitute the panel emulator system.

The display unit 311 of the server program 51A is a server window 62, based on the mouse information supplied from the acquisition unit 312 in step S22, ₁ and touch cursor 66 corresponding to the cursor 73 _1, corresponding to the cursor 73 ₂ The touch cursor 66 ₂ is displayed on the server window 62. However, the display unit 311 displays (draws) the touch cursor 66 corresponding to the cursor whose on / off information included in the mouse information is on among the cursors 73 ₁ and 73 ₂ . Accordingly, in the server window 62, the touch cursor 66 corresponding to the cursor whose on / off information included in the mouse information is turned off is not displayed among the cursors 73 ₁ and 73 ₂ .

As described above, in the server window 62, the input display panel is displayed by displaying only the touch cursor 66 corresponding to the cursor whose on / off information included in the mouse information is turned on among the cursors 73 ₁ and 73 _2. It is possible to express a state in which touch input is performed on the two input / output displays 22.

  Next, the processing of the control unit program 52 will be described with reference to the third flowchart from the right in FIG.

  In the control unit program 52, the control unit 321 determines whether or not necessary information has been supplied (transmitted) from the server program 51A (the generation unit 313) in step S31.

  If it is determined in step S31 that necessary information has not been supplied, the process returns to step S31.

  If it is determined in step S31 that necessary information has been supplied, the process proceeds to step S32, and the control unit 321 generates display data according to the necessary information, and then proceeds to step S33.

  In step S33, the control unit 321 supplies the display data generated in step S32 to the display program 53 (the display unit 331), returns to step S31, and thereafter the same processing is repeated.

  Next, processing of the display program 53 will be described with reference to the third flowchart from the right in FIG. 11 (first from the left).

  In the display program 53, the display unit 331 determines whether or not display data has been supplied (transmitted) from the control unit program 52 (the control unit 321) in step S41.

  If it is determined in step S41 that display data has not been supplied, the process returns to step S41.

  If it is determined in step S41 that display data has been supplied, the process proceeds to step S42, and the display unit 331 updates the display of the display window 63 (FIG. 9) to an image corresponding to the display data. Returning to step S41, the same processing is repeated thereafter.

  As described above, the server computer 61 and one or more client computers 71 constitute a panel emulator system. The one or more client computers 71 transmit mouse information to the server. Since the mouse information transmitted from the client computer 71 is received and necessary information required by the control unit 11 of the input display system to which the input display panel 2 is applied is generated based on the mouse information. The input display panel 2 can be easily emulated without changing the general OS in which only one mouse cursor is displayed without changing the OS, and as a result, the input display panel 2 that can output multiple points. Application development (including multi-point output panels) It can be carried out efficiently.

  Note that the OS of the server computer 61 and the client computer 71 need not be the same OS.

  The information included in the mouse information transmitted from the client computer 71 to the server computer 61 includes on / off information, the coordinates (x, y) of the cursor 73 as the coordinates (x, y) of the center of gravity of the touch area, and Is information with a small amount of data such as touch area information, so that it is not necessary to apply a large load to the network 81 (FIG. 5).

Incidentally, in the embodiment of FIG. 9, the display window 63 displays the cursors 67 ₁ and 67 ₂ corresponding to the touch cursors 66 ₁ and 66 ₂ representing the touch input, together with the image corresponding to the display data. The cursors 67 ₁ and 67 ₂ can be prevented from being displayed on the display window 63, and in this case, the processing load by the display program 53 can be reduced.

However, if the cursors 67 ₁ and 67 ₂ are not displayed on the display window 63, the developer can display the display window 63 when touch input is performed on any part of the image displayed on the display window 63. However, it is difficult to confirm how it is updated (changed).

That is, in the case where the touch cursor 66 representing the point designated by the mouse 207A as the point where the touch input has been made is displayed in the server window 62 and the image corresponding to the display data is displayed in the display window 63, the server When the window 62 and the display window 63 are different windows, if the cursors 67 ₁ and 67 ₂ are not displayed in the display window 63, it is determined which part of the image displayed in the display window 63 has a touch input. It becomes difficult to confirm.

Therefore, when the cursors 67 ₁ and 67 ₂ are not displayed in the display window 63, the server window 62 and the display window 63 can be displayed as shown in FIG. 12, for example.

  That is, FIG. 12 shows a display example of the server window 62 and the display window 63.

  In FIG. 12, the server window 62 and the display window 63 are displayed in the same size (congruent shape), and the server window 62 is all or one part such as a part where the touch cursor 66 can be displayed. The part is displayed translucently.

  As described above, when the server window 62 and the display window 63 are displayed in the same size and the server window 62 is displayed in a translucent manner, the server window 62 or the display window 63 is replaced with the server window 62. By moving so as to overlap the entire display window 63, it is possible to easily confirm which part of the image displayed in the display window 63 is touched by the touch cursor 66 on the semi-transparent server window 62. Furthermore, it is possible to easily adjust the operational feeling by touch input.

  Note that the degree of transparency that makes the server window 62 translucent can be set by operating the keyboard 107B (FIG. 7) or the like, for example. The server window 62 can be made completely transparent.

  Next, setting of touch area information that can be included in mouse information will be described with reference to FIGS.

  As described above, the mouse information includes on / off information indicating whether or not touch input is being performed, and a touch area that is a portion of the input / output display 22 where the touch input is performed (a portion where a finger or a pen is in contact). The coordinates (x, y) of the position of the touch input such as the center of gravity of the touch area are included, and further, the area of the touch area, the x coordinate of the left and right sides of the smallest rectangle surrounding the touch area, and the upper and lower sides Touch area information such as y-coordinates can be included.

  This touch area information can be set by the server program 51A or the client program 51B.

  First, the case where the touch area information is set by the client program 51B will be described with reference to FIG.

  Hereinafter, the area of the touch area in the touch area information is also referred to as a touch area. Further, in the touch area information, the x coordinates of the left and right sides of the smallest rectangle surrounding the touch area, and the y coordinates of the upper and lower sides, for example, the coordinates of the center of gravity of the touch area (x, y) as a reference, the x coordinate of the left and right sides of the smallest rectangle surrounding the touch area is called the Left coordinate and the Right coordinate, respectively, and the y coordinate of the top and bottom sides is the Top coordinate and Bottom coordinate, respectively. That's it.

In Figure 13 the left side, in the client computer 71 _1, the touch region information Top coordinates -10, the Bottom coordinate +10, the Right coordinates +10, the Left coordinates -10, the touch area 400, respectively Is set. Further, in the client computer _712, the Top coordinate of the touch area information -5, the Bottom coordinate +5, the Right coordinates +5, the Left coordinates -5, the touch area 100, is set respectively Yes.

  When the touch area information as described above is set in the client computer 71, the client computer 71 includes the touch area information in the mouse information and transmits it to the server computer 61. In this case, the server computer 61 can display the touch cursor 66 corresponding to the cursor 73 on the server window 62 in a size specified from the touch area information included in the mouse information.

  The touch area information set in the client computer 71 is changed from the state shown on the left side of FIG. 13 to the state shown on the right side of FIG. 13 by the developer operating the keyboard 207B (FIG. 8), for example. Can do.

Here, in the 13 right, the client computer 71 _1, the Top coordinate of the touch area information -5, the Bottom coordinate +5, the Right coordinates +5, the Left coordinates -5, touch area 100 Respectively. Further, in the client computer _712, the Top coordinate of the touch area information 15, the Bottom coordinate +15, the Right coordinates +15, the Left coordinates -15, the touch area 900, it is set respectively Yes.

  When the mouse information including the touch area information as described above is transmitted from the client computer 71 to the server computer 61, the touch cursor 66 corresponding to the cursor 73 is displayed on the server computer 61 as shown on the right side of FIG. It is displayed on the server window 62 in a size specified from the touch area information included in the information.

  Next, with reference to FIGS. 14 and 15, a case where the touch area information is set by the server program 51A will be described.

When setting the touch region information in the server program 51A can either be to set a common touch region information about all of the client computers 71 ₁ and 71 _2, each of the client computers 71 ₁ and 71 _2, separate Touch area information can also be set.

FIG. 14 shows a case where common touch area information is set for all of the client computers 71 ₁ and 71 _{2 in} the server program 51A.

  On the left side of FIG. 14, in the server computer 61, the Top coordinate of the touch area information is set to -10, the Bottom coordinate is set to +10, the Right coordinate is set to +10, the Left coordinate is set to -10, and the touch area is set to 400. Has been.

  When the touch area information is set in the server computer 61, the server computer 61 includes the touch information in the mouse information transmitted from the client computer 71, and then sets the touch cursor 66 on the server based on the mouse information. It is displayed in the window 62.

  When touch information is included in the mouse information, as described with reference to FIG. 13, the server computer 61 has a size specified by the touch cursor 66 corresponding to the cursor 73 from the touch area information included in the mouse information. Can be displayed on the server window 62.

  The touch area information set in the server computer 61 is changed, for example, from the state shown on the left side of FIG. 14 to the state shown on the right side of FIG. 14 by operating the keyboard 107B (FIG. 8) or the like. Can do.

  Here, on the right side of FIG. 14, in the server computer 61, the Top coordinate of the touch area information is -5, the Bottom coordinate is +5, the Right coordinate is +5, the Left coordinate is -5, and the touch area is 100. , Each is set.

  When the touch area information as described above is set by the server computer 61, in the server computer 61, as shown on the right side of FIG. 14, the touch cursor 66 corresponding to the cursor 73 is obtained from the touch area information included in the mouse information. The server window 62 is displayed in the specified size.

Next, FIG. 15 shows a case where individual touch area information is set for each of the client computers 71 ₁ and 71 _{2 in} the server program 51A.

15 On the left, the server computer 61, as the touch area information Client0 of the client computer 71 _1, the Top coordinates -10, the Bottom coordinate +10, the Right coordinates +10, the Left coordinates -10, touch The area is set to 400 respectively. Further, the server computer 61, as the touch area information Client1 of the client computer _712, the Top coordinate -5, the Bottom coordinate +5, the Right coordinates +5, the Left coordinates -5, touch area 100 Respectively.

When the touch area information is set in the server computer 61, as described in FIG. 14, the server computer 61 includes the touch information in the mouse information transmitted from the client computer 71, and then based on the mouse information. The touch cursor 66 is displayed on the server window 62. Thus, the touch cursor 66 ₁ corresponding to the cursor 73 ₁ of the client computer 71 ₁ is displayed based on the mouse information including the touch region information Client0, touch cursor 66 ₂ corresponding to the cursor 73 ₂ of the client computer 71 _2, Displayed based on mouse information including touch area information Client1.

  When touch information is included in the mouse information, the server computer 61 identifies the touch cursor 66 corresponding to the cursor 73 from the touch area information included in the mouse information, as described with reference to FIG. The size can be displayed on the server window 62.

  The touch area information set in the server computer 61 is changed from the state shown on the left side of FIG. 15 to the state shown on the right side of FIG. 15 by the developer operating the keyboard 107B (FIG. 8), for example. Can do.

  Here, on the right side of FIG. 15, in the server computer 61, the Top coordinate of the touch area information Client0 is -5, the Bottom coordinate is +5, the Right coordinate is +5, the Left coordinate is -5, and the touch area is 100. Respectively. Further, the Top coordinate of the touch area information Client1 is set to -15, the Bottom coordinate is set to +15, the Right coordinate is set to +15, the Left coordinate is set to -15, and the touch area is set to 900, respectively.

  When the touch area information as described above is set by the server computer 61, in the server computer 61, as shown on the right side of FIG. 15, the touch cursor 66 corresponding to the cursor 73 is obtained from the touch area information included in the mouse information. The server window 62 is displayed in the specified size.

  When touch area information is set in both the server computer 61 and the client computer 71, which touch area information is valid is determined by the developer (a user who performs emulation using the panel emulator system). Can be selectable.

  Next, FIG. 16 shows another functional configuration example of the panel emulator system of FIG.

  In the figure, portions corresponding to those in FIG. 10 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  That is, the panel emulator system of FIG. 16 includes a display unit 301, an acquisition unit 302, a transmission unit 303, a display unit 311, an acquisition unit 312, a generation unit 313, a control unit 321, and a display unit 331. This is different from the panel emulator system of FIG. 10 in that a storage unit 314 is newly provided in common with the panel emulator system.

  That is, in the panel emulator system of FIG. 10, the server program 51A has the display unit 311, the acquisition unit 312, and the generation unit 313. However, in the panel emulator system of FIG. Part 314.

  Mouse information is supplied from the acquisition unit 312 to the storage unit 314. The storage unit 314 stores mouse information supplied from the acquisition unit 312.

  That is, the acquisition unit 312 supplies the mouse information from the client program 51B to the display unit 311, the generation unit 313, and the storage unit 314.

  As described above, in the display unit 311, the touch cursor 66 (FIG. 9) is displayed on the server window 62 based on the mouse information supplied from the acquisition unit 312.

  In addition, the generation unit 313 generates necessary information required by the control unit 11 (FIGS. 1 to 3) of the application unit 1 from the mouse information from the acquisition unit 312, and the control unit program 52 (the control unit 321). To be supplied.

  In the storage unit 314, the mouse information from the acquisition unit 312 is written in a file in a predetermined format at the same timing as the generation of necessary information by the generation unit 313.

  Here, FIG. 17 shows a format of a file in which mouse information is written in the storage unit 314 (hereinafter referred to as a mouse information file as appropriate).

  The mouse information is transmitted from the client computer 71 in response to the request when the server computer 61 requests the mouse information, but is transmitted from the client computer 71 to the server computer 61 in response to a single request. The incoming mouse information is written in one record (entry) of the mouse information file.

  At the beginning of the record, information on the time when the server computer 61 requested mouse information and a colon (:) indicating the end (separator) of the information on the time are arranged, followed by the effective number of mouse information, A colon representing the end of the effective number is placed.

Here, the effective number is the number of mouse information whose on / off information included in the mouse information indicates that the button of the mouse 207A is on. For example, as shown in FIG. 5, when the panel emulator system has two client computers 71 ₁ and 71 ₂ , the minimum value of the effective number is 0 and the maximum value is 2.

  After the colon indicating the end of the effective number, the coordinates (x, y) of the center of gravity of the touch area included in the mouse information, as well as the touch area, Top coordinate, Bottom coordinate, Left coordinate, and Right, which are touch area information Coordinates and a semicolon (;), which is a delimiter, are repeatedly arranged in effective numbers.

  That is, after the colon representing the end of the effective number, the coordinates (x, y) of the center of gravity of the touch area included in the mouse information whose on / off information represents the on state of the button of the mouse 207A, the touch area, the Top coordinate, the Bottom Coordinates, Left coordinates, Right coordinates, and a semicolon as a separator are repeatedly arranged.

  As described above, when the mouse information from the acquisition unit 312 is stored in the storage unit 314 (FIG. 16), the generation unit 313 generates necessary information from the mouse information supplied from the acquisition unit 312. In addition, for example, necessary information is sequentially generated from the mouse information stored in the storage unit 314 and supplied to the control unit program 52 in accordance with the operation of the keyboard 107B (FIG. 7) of the server computer 61. In this case, it is possible to easily perform debugging using the same touch input sequence repeatedly.

  Next, in the panel emulator system of FIG. 9, the server computer 61 displays a server window 62 and a display window 63. In the server window 62, a touch cursor 66 corresponding to the touch input is displayed. In the server computer 61, the server window 62 and the display window 63 are also used as the same window, that is, for example, the server window 62 has the display window 63 displayed. It is possible to capture the function and display only the server window 62 and display the touch cursor 66 corresponding to the touch input and the image corresponding to the display data in the server window 62.

  FIG. 18 shows a configuration example of such a panel emulator system.

  In FIG. 18, only the server window 62 is displayed on the server computer 61, and the touch cursor 66 corresponding to the touch input and the image corresponding to the display data are displayed on the server window 62.

  As described above, when the function of the display window 63 is taken into the server window 62 and only the server window 62 is displayed, the display program 53 is not necessary, but the server program 51A (FIG. 6) is replaced with the display program 53. It is also necessary to perform this process.

  The processing of the panel emulation system of FIG. 18 in which the server program 51A also performs the processing of the display program 53 will be described with reference to the flowchart of FIG.

  The first from the right in FIG. 19 is a flowchart for explaining the processing of the client program 51B executed by the client computer 71.

  In the client program 51B, in steps S111 to S113, processes similar to those in steps S11 to S13 in FIG. 11 are performed.

  The second from the right in FIG. 19 is a flowchart for explaining the processing of the server program 51 </ b> A executed by the server computer 61.

  In step S121, the server program 51A requests mouse information from the client program 51B. Then, the server program 51A waits for the mouse information to be transmitted from the client program 51B, proceeds from step S121 to S122, acquires (receives) the mouse information from the client program 51B, and proceeds to step S123.

  In step S123, the server program 51A generates necessary information from the mouse information, and the process proceeds to step S124. In step S124, the server program 51A supplies (transmits) the necessary information generated in step S123 to the control unit program 52, and the process proceeds to step S125.

  In step S125, the server program 51A determines whether or not display data has been supplied (transmitted) from the control unit program 52. That is, as will be described later, the control unit program 52 generates display data from the necessary information supplied to the control unit program 52 by the server program 51A in step S124 and supplies the display data to the server program 51A. In this way, it is determined from the control unit program 52 whether display data has been supplied.

  If it is determined in step S125 that display data has not been supplied, the process returns to step S125.

  If it is determined in step S125 that display data has been supplied, the process proceeds to step S126, and the server program 51A updates the display of the server window 62 (FIG. 18) to an image corresponding to the display data. Returning to step S121, the same processing is repeated thereafter.

  The third from the right in FIG. 19 is a flowchart for explaining the processing of the control unit program 52 executed by the server computer 61.

  In step S131, the control unit program 52 determines whether necessary information has been supplied from the server program 51A.

  If it is determined in step S131 that necessary information has not been supplied, the process returns to step S131.

  If it is determined in step S131 that necessary information has been supplied, the process proceeds to step S132, and the control unit program 52 generates display data according to the necessary information, and then proceeds to step S133.

  In step S133, the control unit program 52 supplies the display data generated in step S132 to the server program 51A, returns to step S131, and the same processing is repeated thereafter.

  Here, in the above embodiment, a series of processing is performed by causing a computer to execute software (program). However, the series of processing can also be performed by dedicated hardware.

  Also, in this specification, the processing steps for describing a program for causing a computer to perform various processes do not necessarily have to be processed in time series in the order described in the flowchart, and are executed in parallel or individually. Processing to be performed (for example, parallel processing or object processing) is also included.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  That is, in the present embodiment, the touch cursor 66 corresponding to the touch input is displayed in the server window 62, but the touch cursor 66 can be not displayed.

  In this embodiment, the client program 51B is installed in a computer 71 different from the computer 61 in which the server program 51A is installed. However, the client program 51B is installed in the computer 61 in which the server program 51A is installed. Can also be installed. In this case, the computer 61 in which both the server program 51A and the client program 51B are installed functions as both a server and a client.

  Furthermore, in the present embodiment, the server program 51A requests mouse information from the client program 51B, and in response to the request, the client program 51B transmits mouse information to the server program 51A. However, transmission of mouse information from the client program 51B to the server program 51A can be performed periodically or irregularly from the client program 51B without a request from the server program 51A.

  In the present embodiment, the mouse 207A is used as the pointing device. However, for example, a pen, a trackball, or the like can be used as the pointing device.

It is a block diagram which shows the 1st structural example of the input display system used as the object of emulation. It is a block diagram which shows the 2nd structural example of the input display system used as the object of emulation. It is a block diagram which shows the 3rd structural example of the input display system used as the object of emulation. It is a figure for demonstrating the process of an input display system. It is a figure which shows the structural example of one Embodiment of the panel emulator system to which this invention is applied. It is a figure explaining the program which makes a computer function as a panel emulator system. FIG. 3 is a block diagram illustrating a hardware configuration example of a server computer 61. FIG. 3 is a block diagram illustrating a hardware configuration example of a client computer 71. It is a figure for demonstrating the process of the server computer 61 and the client computer 71. FIG. It is a block diagram which shows the functional structural example of a panel emulator system. It is a flowchart explaining the process of a panel emulator system. It is a figure which shows the example of a display of the server window 62 and the display window 63. FIG. It is a figure for demonstrating the setting of the touch area information in the client computer. It is a figure for demonstrating the setting of the touch area | region information in the server computer 61. FIG. It is a figure for demonstrating the setting of the touch area | region information in the server computer 61. FIG. It is a block diagram which shows the other functional structural example of a panel emulator system. It is a figure which shows the format of the file which memorize | stores mouse information. It is a figure which shows the structural example of the panel emulator system which displays the touch cursor 66 and the image corresponding to display data in the server window 62. FIG. 10 is a flowchart illustrating processing of the panel emulator system that displays a touch cursor 66 and an image corresponding to display data in a server window 62.

Explanation of symbols

1 application section, 2 input display panel, 11 control section, 21 display display data processing section, 22 input / output display, 23 received light data processing section, 24 image processing section, 25 generation section, 51 panel emulator, 51A server program, 51B client program, 52 controller program, 53 display program, 61 computer, 62 server window 63 display window, 65, 66 _1, 66 _2, 67 _1, 67 ₂ cursors, 71 _1, 71 ₂ computer, 72 _1, 72 ₂ client window, 73 _1, 73 ₂ cursors, 81 network, 101 bus, 102 CPU, 103 ROM, 104 RAM, 105 hard disk, 106 output unit, 107 input unit, 107A mouse, 107B keyboard, 108 copies Unit, 109 drive, 110 input / output interface, 111 removable recording medium, 201 bus, 202 CPU, 203 ROM, 204 RAM, 205 hard disk, 206 output unit, 207 input unit, 207A mouse, 207B keyboard, 208 communication unit, 209 drive , 210 input / output interface, 211 removable recording medium, 301 display unit, 302 acquisition unit, 303 transmission unit, 311 display unit, 312 acquisition unit, 313 generation unit, 314 storage unit, 321 control unit, 331 display unit

Claims (18)

Server,
With one or more clients,
In an information processing system that emulates the panel capable of outputting information about points on the panel for a plurality of points,
The client has a transmission means for transmitting pointing device information relating to a point indicated by the pointing device to the server,
The server
Receiving means for receiving pointing device information transmitted from one or more of the clients;
An information processing system comprising: generating means for generating necessary information required by an application to which the panel is applied based on the pointing device information. In an information processing apparatus constituting an information processing system that performs emulation of the panel capable of outputting information about points on the panel for a plurality of points,
Receiving means for receiving pointing device information about a point designated by the pointing device, transmitted from one or more clients constituting the information processing system;
An information processing apparatus comprising: generating means for generating necessary information required by an application to which the panel is applied based on the pointing device information. The information processing apparatus according to claim 2, wherein the pointing device information includes information indicating a position of a point designated by the pointing device and information on a predetermined area including the point designated by the pointing device. The information processing apparatus according to claim 3, wherein the information on the predetermined area is set by the client. The information processing apparatus according to claim 3, wherein information on the predetermined area is set, and the information on the predetermined area is included in the pointing device information transmitted from one or more of the clients. The information processing apparatus according to claim 2, wherein a point indicated by the pointing device is displayed on a window based on the pointing device information. In the case where the panel performs display according to data supplied from the application, and the application generates display data corresponding to an image displayed by the panel based on the necessary information.
When an image corresponding to the display data generated by the application is displayed in the window,
The information processing apparatus according to claim 6, wherein a window in which a point indicated by the pointing device is displayed is different from a window in which an image corresponding to the display data is displayed. The information processing apparatus according to claim 7, wherein a part or all of a window in which a point indicated by the pointing device is displayed is transparent. The information processing apparatus according to claim 8, wherein a degree of transparency of a part or all of a window in which a point indicated by the pointing device is displayed can be set. Storage means for storing the pointing device information transmitted from one or more clients;
The information processing apparatus according to claim 2, wherein the generation unit generates necessary information from the pointing device information stored in the storage unit. In the information processing method of the information processing apparatus constituting the information processing system for performing emulation of the panel capable of outputting information on points on the panel for a plurality of points,
Receiving pointing device information about a point designated by the pointing device, transmitted from one or more clients constituting the information processing system;
An information processing method including a step of generating necessary information required by an application to which the panel is applied based on the pointing device information. In a program for causing a computer constituting an information processing system that performs emulation of the panel capable of outputting information on points on a panel to output a plurality of points, to execute information processing,
Receiving pointing device information about a point designated by the pointing device, transmitted from one or more clients constituting the information processing system;
A program for causing a computer to execute information processing including a step of generating necessary information required by an application to which the panel is applied based on the pointing device information. In an information processing apparatus constituting an information processing system that performs emulation of the panel capable of outputting information about points on the panel for a plurality of points,
An application is required that receives pointing device information transmitted from one or more information processing apparatuses constituting the information processing system and that is related to a point indicated by the pointing device, and to which the panel is applied based on the pointing device information. An information processing apparatus comprising: transmission means for transmitting the pointing device information to a server that generates necessary information. The information processing apparatus according to claim 13, wherein the pointing device information includes information indicating a position of a point designated by the pointing device and information on a predetermined region including the point designated by the pointing device. The information processing apparatus according to claim 14, wherein information on the predetermined area is set and included in the pointing device information. The information processing apparatus according to claim 14, wherein the information on the predetermined area is set by the server. In the information processing method of the information processing apparatus constituting the information processing system for performing emulation of the panel capable of outputting information on points on the panel for a plurality of points,
An application is required that receives pointing device information transmitted from one or more information processing apparatuses constituting the information processing system and that is related to a point indicated by the pointing device, and to which the panel is applied based on the pointing device information. The information processing method includes a step of transmitting the pointing device information to a server that generates the necessary information. In a program for causing a computer constituting an information processing system that performs emulation of the panel capable of outputting information on points on a panel to output a plurality of points, to execute information processing,
An application to which the panel is applied based on the pointing device information received from the pointing device information received from one or more computers constituting the information processing system is received. A program that causes a computer to execute information processing including a step of transmitting the pointing device information to a server that generates necessary information.

Images