JP2009282637A - Display method and display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display method and a display device, allowing an operator to easily perform operation even on a large-size display screen. <P>SOLUTION: An input processing part 13 specifies a position of a person from a position of a light reception part receiving infrared light when using the infrared light as a person authentication device 23. When not displaying a reduced screen, an video output part 16 is instructed to compose the reduced screen with the display screen so that the reduced screen is displayed in a position closest to the specified position to be displayed on a display device 21. The video output part 16 composes the display screen by actual screen information received from an actual screen creation part 14 and the reduced screen by reduction information received from a reduced screen creation part 15, and makes the display device 21 display it. When an operation instruction to the reduced screen is performed, the input processing part 13 performs processing corresponding to the operation instruction to an operation position after conversion wherein a position of the operation instruction is converted from reduced screen coordinates into actual screen coordinates by a coordinate conversion part 12, e.g. drawing. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

The present invention relates to a display method and a display device for displaying information such as image information.
About.

  Some display devices such as a display device (hereinafter also referred to as a “display”) according to the prior art have a multi-window function for simultaneously displaying a plurality of windows on a display screen. Further, when the entire image information cannot be displayed on the display screen because the amount of image information to be displayed is large, a reduced image window obtained by reducing the entire image is displayed on the display screen. Some have a function of indicating a display position in the entire image by indicating an image indicating the position of the image displayed on the screen.

  For example, the multi-window display method described in Patent Document 1 displays a part of display screen information on a window when the resolution of the window is not high enough to display the entire display screen information. Further, when the entire display screen information is reduced and displayed in another window, the overlapped portion of the two windows is displayed semi-transparently. By making it semi-transparent, it is possible to input characters, figures and the like considering the proportion of the entire screen while viewing the reduced display window in the window displaying a part of the display screen information.

  FIG. 10 is a diagram showing a display screen 90 of a map display according to a conventional technique. The display screen 90 is a screen that displays a part of the map information, and a window 91 that displays a reduced screen obtained by reducing the entire map information is displayed in the display screen 90. The reduced image 92 indicates the position of the map information displayed on the display screen 90 in the entire reduced map information displayed in the window 91. The user can change the range of the map information displayed on the display screen 90 by an operation in the window 91.

  FIG. 11 is a diagram showing a display screen 93 of a virtual desktop according to the conventional technique. The display screen 93 is a real desktop screen, and displays a part of the display screen of the virtual desktop. On the display screen 93, a window 94 in which the entire virtual desktop display screen is reduced is displayed. The reduced image 95 is an image obtained by reducing the screen displayed on the display screen 93, and shows a position in the reduced virtual desktop displayed on the window 94. When a plurality of screens are displayed on the virtual desktop displayed in the window 94, the user can move an application window displayed on any screen to another screen.

JP-A-7-210700

  FIG. 12 is a diagram illustrating a display device 96 having a touch panel function according to a conventional technique. Large display devices such as whiteboards equipped with a touch panel function on large displays have become available due to the increase in the size of thin displays. The display device 96 is a large display device having a touch panel function used for presentations or meetings. The operator can operate or draw the screen by touching the screen of the display device 96. However, if the screen is large, the operator cannot reach the top of the screen and cannot operate or draw. It is possible to display the same screen on another small display and operate or draw on the screen, but the operator sees another screen only by himself and is difficult to speak.

  Any of the above-described conventional techniques has a problem that when the display device becomes large, the operator cannot operate where the hand cannot reach, similarly to the display device 96 shown in FIG.

  An object of the present invention is to provide a display method and a display device that allow an operator to easily operate a large display screen on the display screen.

The present invention includes a display step of combining and displaying a reduced screen obtained by reducing a display screen for displaying display information with the display screen;
An operation instruction step for performing an operation instruction on the display screen;
A conversion step of converting the position where the operation instruction is given in the operation instruction step from a position based on the reduced screen to a position based on the display screen;
When the position where the operation instruction is performed in the operation instruction step is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted in the conversion step is performed, and the operation instruction is performed in the operation instruction step. When the performed position is a position in the remaining area excluding the reduced screen area in the display screen area, the operation instruction performed in the operation instruction step with respect to the position based on the display screen is performed. An operation execution step for performing a corresponding process.

The present invention also provides a display means for displaying a reduced screen obtained by reducing a display screen for displaying display information in combination with the display screen;
An operation instruction means for performing an operation instruction on the display screen;
Conversion means for converting the position where the operation instruction is given by the operation instruction means from a position based on the reduced screen to a position based on the display screen;
When the position where the operation instruction is given by the operation instruction means is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted by the conversion means is performed, and the operation instruction is given by the operation instruction means. When the performed position is a position in the remaining area of the display screen area excluding the reduced screen area, an operation is performed to perform a process corresponding to the operation instruction for a position based on the display screen A display device.

  According to the present invention, in the display step, a reduced screen obtained by reducing the display screen for displaying the display information is combined with the display screen and displayed. In the operation instruction step, an operation instruction on the display screen is performed. In the conversion step, the position where the operation instruction is given in the operation instruction step is converted from a position based on the reduced screen to a position based on the display screen. In the operation execution step, when the position where the operation instruction is performed in the operation instruction step is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted in the conversion step is performed, When the position where the operation instruction is performed in the operation instruction step is a position in the remaining area of the display screen area excluding the reduced screen area, the operation instruction for the position based on the display screen is Perform the corresponding process. Therefore, if the display method according to the present invention is applied, the operator can easily operate the display screen even on a large display screen.

  According to the invention, the display unit displays a reduced screen obtained by reducing the display screen for displaying the display information in combination with the display screen, and the operation instruction unit performs an operation instruction on the display screen. The position where the operation instruction is given by the operation instruction means is converted by the conversion means from the position based on the reduced screen to the position based on the display screen, and is operated by the operation instruction means by the operation execution means. When the position where the instruction is made is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted by the conversion means is performed, and the position where the operation instruction is issued by the operation instruction means The operation finger with respect to a position relative to the display screen when the position is in a remaining area of the display screen area excluding the reduced screen area. Processing corresponding to is performed. Therefore, even on a large display screen, the operator can easily operate on the display screen.

  FIG. 1 is a block diagram showing a configuration of a display device 1 according to an embodiment of the present invention. The display device 1 includes an information processing device 10, a display device 21, an input device 22, and a person recognition device 23. The display method according to the present invention is implemented by the display device 1. The information processing apparatus 10 includes a device recognition unit 11, a coordinate conversion unit 12, an input processing unit 13, a real screen creation unit 14, a reduced screen creation unit 15, a video output unit 16, a reduced screen position storage unit 17, an image storage unit 18, and the like. It is configured to include the functions of

  The information processing apparatus 10 is configured by a personal computer including a central processing unit (hereinafter referred to as “CPU”) (not shown) and a storage device (not shown) such as a semiconductor memory or a hard disk device. The storage device stores a control program executed by the CPU and control information used for controlling the information processing apparatus 10, for example, information stored in the reduced screen position storage unit 17 and the image storage unit 18. The CPU executes a control program stored in the storage device to thereby execute a device recognition unit 11, a coordinate conversion unit 12, an input processing unit 13, a real screen creation unit 14, a reduced screen creation unit 15, a video output unit 16, and a reduction. Functions such as the screen position storage unit 17 and the image storage unit 18 are realized.

  When receiving an input from the input device 22, the device recognition unit 11 recognizes the input device 22 that has received the input. The device recognition unit 11 recognizes the input device 22 by, for example, the device type of the input device 22 and an ID (Identification) for identifying each input device 22 of the same type, and device information representing the recognized device type, ID, and the like. Is sent to the input processing unit 13.

  The coordinate conversion unit 12 serving as a conversion unit receives operation position information indicating an operation position at which an operation instruction has been performed on the screen of the display device 21 from the input device 22. The operation instruction is, for example, an instruction for performing a process of moving the position of the displayed window, a process of closing the window, a process of drawing, and the like. Further, the coordinate conversion unit 12 converts the operation position indicated by the operation position information received from the input device 22 from a position based on the reduced screen to a position based on the display screen. Specifically, the coordinates of the operation position are converted from a reduced screen coordinate (hereinafter also referred to as “reduced screen coordinates”) system to a display screen coordinate (hereinafter also referred to as “real screen coordinates”) system. The coordinate conversion unit 12 sends converted position information representing the converted operation position to the input processing unit 13. In the embodiment shown in FIG. 1, the coordinate conversion unit 12 is provided in the information processing apparatus 10. However, the coordinate conversion unit 12 provided outside the information processing apparatus 10 is operated from the coordinate conversion unit 12 provided outside. You may receive the conversion position information which shows a position.

  The input processing unit 13, which is an operation execution unit, receives operation position information from the input device 22, receives device information from the device recognition unit 11, and conversion position information from the coordinate conversion unit 12. Based on the received operation position information and the received device information and conversion operation position information, the real screen creation unit 14 and the video output unit 16 are controlled.

  The real screen creation unit 14 creates real screen information representing a display screen to be displayed on the display device 21 and stores the created real screen information in the image storage unit 18. Further, in response to an instruction from the video output unit 16, the real screen information stored in the image storage unit 18 is read and sent to the video output unit 16. The reduced screen creation unit 15 reads the real screen information stored in the image storage unit 18 according to an instruction from the video output unit 16, and creates reduced information representing a reduced screen obtained by reducing the display screen based on the read real screen information. The generated reduction information is sent to the video output unit 16.

  The video output unit 16 combines a real screen information received from the real screen creation unit 14 or a screen obtained by combining the display screen based on the real screen information received from the real screen creation unit 14 and the reduced screen based on the reduction information received from the reduced screen creation unit 15. The composite information to be expressed is converted into a video signal and transmitted to the display device 21.

  The reduced screen position storage unit 17 displays display information indicating whether or not the reduced screen is displayed, display position information indicating the position where the reduced screen is displayed, size information indicating the size of the reduced screen, and the like. Store control information. The size information can be set in the reduced screen position storage unit 17 by the input device 22, for example. Display control information is written and read by the input processing unit 13. The image storage unit 18 stores the real screen information created by the real screen creation unit 14, and the stored real screen information is read by the real screen creation unit 14 and the reduced screen creation unit 15.

  The display device 21 is a large-sized display device, for example, more than 100 inches, receives a video signal from the video output unit 16, and displays a screen based on the received video signal. The real screen creation unit 14, the reduced screen creation unit 15, the video output unit 16, and the display device 21 are display means.

  The input device 22 that is an operation instruction unit is configured by a touch panel, a mouse, a tablet, or the like, and the operation position information that indicates the operation position where the user has issued the operation instruction among the positions in the screen of the display device 21 is converted into the coordinate conversion unit 12. And transmitted to the input processing unit 13. Further, device information such as the device type and ID of the input device 22 is transmitted to the device recognition unit 11.

The ID is identification information for identifying each input device 22, for example, USB (
Universal Serial Bus) is stored in the input device 22 itself, such as a connected device.

  The ID does not necessarily have to be stored in the input device 22 itself, but is stored in a device external to the input device 22 such as the ID of each touch pen for a touch panel, and the input device 22 receives infrared (hereinafter referred to as “IR”). May be transmitted by communication means such as

  The person recognition device 23 includes an infrared sensor, a camera, an IR transmitter, an IR receiver, and the like, recognizes the position of an operator near the display device 21, and inputs person position information representing the recognized position. To the unit 13. When an infrared sensor is used as the person recognition device 23, for example, an infrared light emitting unit and a light receiving unit are provided at several places below the display device 21, and the input processing unit 13 receives the infrared light detected by the operator. The position of the operator is specified by a signal from the unit. When a camera is used as the person recognition device 23, for example, a display screen is captured by the camera, and the input processing unit 13 recognizes the operator by image recognition from the images captured by the camera, and determines the position of the operator. Identify. When an IR transmitter and an IR receiver are used as the person recognition device 23, for example, an IR transmitter is built in an instruction bar or key holder held by an operator, and IR receivers are provided at several edge portions of the display device 21. . The input processing unit 13 specifies the position of the operator based on the signal from the IR receiver that has received the signal from the IR transmitter.

  FIG. 2 is a diagram illustrating an example of a display screen 30 that displays a reduced screen 31 near the operator 2. The display screen 30 is an example in which the reduced screen 31 is displayed near the operator 2 when the operator 2 approaches the display device 21. Since the reduced screen 31 is displayed near the operator 2, the operator 2 can issue an operation instruction to the display screen 30 by giving an operation instruction to the reduced screen 31.

  FIG. 3 is a diagram illustrating an example of the display screen 40 that displays the reduced screen 42 in a range that the operator 2 can reach. The display screen 40 is an example in which a reduced screen 42 is displayed in a reduced screen display area 41 that is within the reach of the operator 2. Since the remaining area excluding the reduced screen display area 41 in the entire area of the display screen 40 is an area that cannot be reached by human hands, the touch panel function may not be provided. When a resistive touch panel is used as the person recognition device 23, the accuracy decreases as the surface area increases, and the cost increases when a wide area is covered. When infrared rays or image recognition is used as the person recognition device 23, the recognition speed and accuracy are affected if a wide range is covered. Therefore, by limiting the area where the reduced screen is displayed within the reduced screen display area 41, the area where the touch panel function is provided can be reduced, so that cost reduction, accuracy improvement, or recognition speed improvement can be achieved.

  FIG. 4 is a diagram illustrating an example of a display screen 50 operated using two types of touch pens. The display screen 50 is an example in which the reduced screen 51 is displayed near the operator 2. The operator 2 uses a reduced screen touch pen 3a for giving an operation instruction on the reduced screen, and the operator 2b uses an ordinary touch pen 3b for giving an operation instruction on a display screen excluding the reduced screen.

  In the example shown in FIG. 4, the operator 2b draws with the normal touch pen 3b, and the operator 2 draws within the area of the reduced screen with the reduced screen touch pen 3a. The operator 2 can draw the same line as the line drawn on the normal touch pen 3b by the operator 2b outside the reduced screen area by drawing the reduced screen touch pen 3a in the reduced screen area.

  The normal touch pen 3 b and the reduced screen touch pen 3 a are assigned IDs, and each touch pen transmits the ID to the information processing apparatus 10. Specifically, for example, an operation button for transmitting an ID is provided at the tip of the touch pen, and the touch pen transmits the ID to the device recognition unit 11 by an IR signal when the operation button is operated.

In the example shown in FIG. 4, the operation instruction for the display screen and the operation instruction for the reduced screen are distinguished using two touch pens having different IDs, but the present invention is not limited to this. You may distinguish by the difference in a type. For example, the operation instruction for the display screen may be performed by operating the mouse, and the operation instruction for the reduced screen may be performed by the operation instruction of the touch panel. The normal touch pen 3b and the mouse are the first pointing devices. The reduced screen touch pen 3a and the touch panel are second instruction devices. FIG. 5 is a flowchart showing a first display process executed by the information processing device 10. The first display process is a process for displaying the reduced screen 31 near the operator as in the display screen 30 shown in FIG. 2 and enabling operation instructions on the displayed reduced screen 31. When the power of the display device 1 is turned on and becomes operable, the process proceeds to step A1.

  In step A1, a person is detected by a person recognition device 23, for example, a camera or a sensor using infrared rays. In step A <b> 2, the person recognition device 23 recognizes a person approaching the display device 21. In step A3, the input processing unit 13 determines whether or not a person can be recognized. When the person position information is received from the person recognition device 23, it is determined that the person can be recognized, and the process proceeds to step A4. When the person position information is not received from the person recognition device 23, the first display process is terminated. Then, after performing other processes, for example, the touch process shown in FIG. 6, the process returns to step A1.

  In step A4, the input processing unit 13 specifies the location of the person. For example, when a camera is used as the person recognizing device 23, the position of the person is specified from an image photographed by the camera. Identify the location.

  In step A5, the input processing unit 13 determines whether or not a reduced screen is being displayed. If the displayed information stored in the reduced screen position storage unit 17 indicates that the reduced screen is being displayed, it is determined that the reduced screen is being displayed, and the process proceeds to step A6, where the reduced screen position storage unit 17 is displayed. If the currently displayed information stored in does not indicate that the reduced screen is displayed, it is determined that the reduced screen is not being displayed, and the process proceeds to step A10. In step A6, the input processing unit 13 calls the reduced screen position. Specifically, the display position information stored in the reduced screen position storage unit 17 is read.

  In step A7, the input processing unit 13 determines whether the reduced screen is displayed within a certain distance from the person. If the position indicated by the display position information read from the reduced screen position storage unit 17 is within a certain distance from the position specified in step A4, for example, within the distance that a human hand can reach, the reduced screen is displayed within a certain distance from the person. The first display process is terminated, other processes are performed, and the process returns to step A1. If the position indicated by the display position information read from the reduced screen position storage unit 17 is not within a certain distance from the position specified in step A4, it is determined that the reduced screen is not displayed within a certain distance from the person, and step A8. Proceed to

  In step A8, the input processing unit 13 moves the reduced screen near the person. Specifically, the input processing unit 13 instructs the video output unit 16 to move the position where the reduced screen is displayed to the position closest to the position specified in step A4. The video output unit 16 combines the reduced screen based on the reduced information received from the reduced screen creating unit 15 at the position indicated by the input processing unit 13 in the display screen based on the real screen information received from the actual screen creating unit 14. The synthesized information representing the screen is converted into a video signal and transmitted to the display device 21. The display device 21 displays a screen based on the video signal received from the video output unit 16. In step A9, the input processing unit 13 stores the position where the reduced screen is moved in the reduced screen position storage unit 17 as new display position information, and then ends the first display process and performs other processing. Return to step A1.

  In step A10, the input processing unit 13 generates a reduced screen. Specifically, the input processing unit 13 instructs the video output unit 16 to combine the reduced screen with the display screen and display it on the display device 21. In response to an instruction from the input processing unit 13, the video output unit 16 instructs the reduced screen creation unit 15 to create reduced information. The reduced screen creation unit 15 reads the actual screen information stored in the image storage unit 18 in accordance with an instruction from the video output unit 16, creates the reduced information based on the read actual screen information, and displays the created reduced information as the video. The data is sent to the output unit 16. As for the size of the reduced screen, the size information stored in the reduced screen position storage unit 17 is read by the input processing unit 13 and sent to the reduced screen creation unit 15 via the video output unit 16.

  In step A11, the input processing unit 13 displays a reduced screen near the person. Specifically, the input processing unit 13 instructs the video output unit 16 to display the reduced screen generated in step A10 at a position closest to the position specified in step A4. The video output unit 16 combines the reduced screen based on the reduced information received from the reduced screen creating unit 15 at the position indicated by the input processing unit 13 in the display screen based on the real screen information received from the actual screen creating unit 14. The synthesized information representing the screen is converted into a video signal and transmitted to the display device 21. The display device 21 displays a screen based on the video signal received from the video output unit 16. In step A12, the input processing unit 13 stores the display position information indicating the position where the reduced screen is displayed in the reduced screen position storage unit 17, and further displays the in-display information indicating that the reduced screen is displayed. After storing in the position storage unit 17, the first display process is terminated, and after other processes are performed, the process returns to step A1. Step A8 or A11 is a display step. Steps A1 to A4 are detection steps.

  FIG. 6 is a flowchart illustrating a touch process executed by the information processing apparatus 10. The touch process is a process for performing a process corresponding to the operation instruction. After the first display process shown in FIG. 5, the process proceeds to step B1.

  In step B <b> 1, the input device 22 detects a position touched on the screen, and transmits operation position information indicating the detected position, for example, coordinate information representing coordinates in the real screen coordinate system to the input processing unit 13. In step B2, the input processing unit 13 determines whether or not the screen is touched. If the operation position information is received from the input device 22, it is determined that the screen is touched, and the process proceeds to step B3. If the operation position information is not received from the input device 22, it is determined that the screen is not touched, and the process returns to step B1. .

  In step B3, the input processing unit 13 determines whether or not a reduced screen is being displayed. If the displayed information stored in the reduced screen position storage unit 17 indicates that the reduced screen is being displayed, it is determined that the reduced screen is being displayed, and the process proceeds to step B4, where the reduced screen position storage unit 17 is displayed. If the currently displayed information stored in does not indicate that the reduced screen is displayed, it is determined that the reduced screen is not being displayed, and the process proceeds to step B7. In step B4, the input processing unit 13 calls a reduced screen display position. Specifically, display position information and size information stored in the reduced screen position storage unit 17 are read.

  In step B5, the input processing unit 13 determines whether or not the touched position is on the reduced screen. More specifically, the area of the reduced screen in which the position indicated by the operation position information received in step B2 is displayed at the position indicated by the display position information read from the reduced screen position storage unit 1 in the size indicated by the size information. If it is within the position, it is determined that the touched position is on the reduced screen, and the process proceeds to step B6. The position indicated by the operation position information received in step B2 is not a position within the area of the reduced screen displayed at the size indicated by the size information at the position indicated by the display position information read from the reduced screen position storage unit 1. Then, it is determined that the touched position is not on the reduced screen, and the process proceeds to Step B7.

  In step B6, the input processing unit 13 converts the reduced screen coordinates to the real screen coordinates. Specifically, the input processing unit 13 receives converted position information representing the converted operation position converted by the coordinate converting unit 12. In step B7, the input processing unit 13 calls the processing content for the coordinates. The coordinates are the coordinates of the position indicated by the converted position information when converted in step B6, and the coordinates in the actual screen coordinates of the position where the operation instruction has been made if not converted in step B6.

  The processing content for the coordinates is the content of the operation according to the operation instruction, for example, processing for moving the position of the displayed window, processing for closing the window, processing for drawing, and the like. For example, in the window closing process, if the operation instruction by the input device 22 is an operation instruction for the position where the operation button for closing the window is displayed, the window closing process is performed. The drawing process is, for example, a process of combining and displaying an image of a predetermined color sequentially at a position where an operation instruction is given from the input device 22. The processing content for the coordinates is stored in a storage device (not shown), for example, and the processing content corresponding to the coordinates is read from the storage device.

  In step B8, the input processing unit 13 executes processing for the coordinates. That is, the processing content called in step B7 is executed. In step B9, the input processing unit 13 reflects the result on the actual screen. Specifically, the real screen creation unit 14 is instructed to create real screen information based on the information reflecting the result. When the reduced screen is not displayed, a display screen based on the created real screen information is displayed. It is displayed on the display device 21 by the video output unit 16. In step B10, the input processing unit 13 determines whether or not a reduced screen is being displayed. If the displayed information stored in the reduced screen position storage unit 17 indicates that the reduced screen is being displayed, it is determined that the reduced screen is being displayed, and the process proceeds to step B11, where the reduced screen position storage unit 17 is displayed. If the currently displayed information stored in does not indicate that the reduced screen is being displayed, it is determined that the reduced screen is not being displayed, and the touch processing is terminated.

  In step B11, the input processing unit 13 reflects the result on the reduced screen and ends the touch process. Specifically, the video output unit 16 is instructed to synthesize a reduced screen obtained by reducing the display screen based on the real screen information in which the result is reflected in step B9, and to display the reduced screen on the display device 21. . In accordance with an instruction from the input processing unit 13, the video output unit 16 combines information that represents a screen obtained by combining the display screen based on the real screen information received from the real screen creation unit 14 and the reduced screen based on the reduced information received from the reduced screen creation unit 15. Is converted into a video signal and transmitted to the display device 21 for display. Steps B1 and B2 are operation instruction steps. Step B6 is a conversion step. Steps B7 to B11 are operation execution steps.

  FIG. 7 is a flowchart showing a second display process executed by the information processing apparatus 10. The second display process is a process of displaying a reduced screen at a position touched by the operator 2 and performing a process corresponding to the operation instruction. When the power of the display device 1 is turned on and becomes operable, the process proceeds to step C1. Steps C1 to C9 are the same as steps B1 to B9 shown in FIG. 6, respectively, and description thereof is omitted to avoid duplication.

  In step C10, the input processing unit 13 reflects the result on the reduced screen, ends the second display process, performs other processes, and then returns to step C1. Specifically, the video output unit 16 is instructed to synthesize a reduced screen obtained by reducing the display screen based on the real screen information in which the result is reflected in step C9, and to display the reduced screen on the display device 21. . In accordance with an instruction from the input processing unit 13, the video output unit 16 combines information that represents a screen obtained by combining the display screen based on the real screen information received from the real screen creation unit 14 and the reduced screen based on the reduced information received from the reduced screen creation unit 15. Is converted into a video signal and transmitted to the display device 21 for display.

  In step C11, the input processing unit 13 generates a reduced screen. Specifically, the input processing unit 13 instructs the video output unit 16 to combine the reduced screen with the display screen and display it on the display device 21. In response to an instruction from the input processing unit 13, the video output unit 16 instructs the reduced screen creation unit 15 to create reduced information. The reduced screen creation unit 15 reads the actual screen information stored in the image storage unit 18 in accordance with an instruction from the video output unit 16, creates the reduced information based on the read actual screen information, and displays the created reduced information as the video. The data is sent to the output unit 16. As for the size of the reduced screen, the size information stored in the reduced screen position storage unit 17 is read by the input processing unit 13 and sent to the reduced screen creation unit 15 via the video output unit 16.

  In step C12, the input processing unit 13 displays a reduced screen around the touch coordinates. Specifically, the input processing unit 13 causes the video output unit 16 to synthesize the reduced screen with the display screen in accordance with the position indicated by the operation position information received in step C <b> 2 in the display device 21. Instruct to display. The video output unit 16 matches the center of the reduced screen based on the reduced information received from the reduced screen creation unit 15 with the position indicated by the operation position information received in step C2, and receives the reduced screen from the real screen creation unit 14. Is displayed on the display device 21.

  In step C13, the input processing unit 13 stores the display position information indicating the position where the reduced screen is displayed in the reduced screen position storage unit 17, and further displays in-display information indicating that the reduced screen is displayed on the reduced screen. After storing in the position storage unit 17, the second display process is terminated, and after performing other processes, the process returns to step C1. Step C12 is a display step. Steps C1 and C2 are operation instruction steps. Step C6 is a conversion step. Steps C7 to C10 are operation execution steps.

  FIG. 8 is a flowchart illustrating a third display process executed by the information processing apparatus 10. The third display process is a process that enables operation instructions using two types of touch pens as in the display screen 50 shown in FIG. When the power of the display device 1 is turned on and becomes operable, the process proceeds to step D1. Steps D1, D2, D5 to D7, and D11 to D15 are the same as steps B1 to B10 shown in FIG. 6, respectively, and description thereof is omitted to avoid duplication.

  In step D3, the input processing unit 13 acquires a pen ID. Specifically, the ID transmitted from the touch pen is acquired from the device recognition unit 11. In step D4, it is determined whether or not the acquired pen ID is a reduction pen ID. If the ID received from the touch pen by the device recognition unit 11 is the ID of the reduction screen touch pen, it is determined that the acquired pen ID is the ID of the reduction pen, and the process proceeds to step D5, where the device recognition unit 11 receives from the touch pen. If the obtained ID is not the ID of the reduced screen touch pen, it is determined that the acquired pen ID is not the ID of the reduced pen, and the process proceeds to step D17.

  In step D8, the input processing unit 13 displays a reduced screen around the touch coordinates. Specifically, the input processing unit 13 causes the video output unit 16 to synthesize the reduced screen with the display screen in accordance with the position indicated by the operation position information received in step D <b> 2 in the display device 21. Instruct to display. The video output unit 16 matches the center of the reduced screen based on the reduced information received from the reduced screen creation unit 15 with the position indicated by the operation position information received in step D2, and receives the reduced screen from the real screen creation unit 14. Is displayed on the display device 21.

  In step D9, the input processing unit 13 stores the display position information indicating the position where the reduced screen is displayed in the reduced screen position storage unit 17, and further displays in-display information indicating that the reduced screen is displayed on the reduced screen. After storing in the position storage unit 17, the third display process is terminated, and after other processes are performed, the process returns to step D1.

  In step D10, the input processing unit 13 generates a reduced screen and proceeds to step D8. Specifically, the input processing unit 13 instructs the video output unit 16 to combine the reduced screen with the display screen and display it on the display device 21. In response to an instruction from the input processing unit 13, the video output unit 16 instructs the reduced screen creation unit 15 to create reduced information. The reduced screen creation unit 15 reads the actual screen information stored in the image storage unit 18 in accordance with an instruction from the video output unit 16, creates the reduced information based on the read actual screen information, and displays the created reduced information as the video. The data is sent to the output unit 16.

  In step D16, the input processing unit 13 reflects the result on the reduced screen, ends the third display process, performs other processes, and then returns to step D1. Specifically, the video output unit 16 is instructed to synthesize a reduced screen obtained by reducing the display screen based on the real screen information in which the result is reflected in step D14, and display the resultant on the display device 21. . In accordance with an instruction from the input processing unit 13, the video output unit 16 combines information that represents a screen obtained by combining the display screen based on the real screen information received from the real screen creation unit 14 and the reduced screen based on the reduced information received from the reduced screen creation unit 15. Is converted into a video signal and transmitted to the display device 21 for display.

  In step D17, the input processing unit 13 determines whether or not a reduced screen is being displayed. If the displayed information stored in the reduced screen position storage unit 17 indicates that the reduced screen is being displayed, it is determined that the reduced screen is being displayed, and the process proceeds to step D18 to reduce the reduced screen position storage unit 17. If the currently displayed information stored in does not indicate that the reduced screen is displayed, it is determined that the reduced screen is not being displayed, and the process proceeds to step D12. In step D18, the input processing unit 13 calls a reduced screen position. Specifically, display position information and size information stored in the reduced screen position storage unit 17 are read.

  In step D19, the input processing unit 13 determines whether or not the touched position is on the reduced screen. Specifically, the area of the reduced screen in which the position indicated by the operation position information received in step D2 is displayed in the size indicated by the size information at the position indicated by the display position information read from the reduced screen position storage unit 1 If it is within the position, it is determined that the touched position is on the reduced screen, and the process proceeds to step D20. The position indicated by the operation position information received in step D2 is not a position within the area of the reduced screen displayed at the size indicated by the size information at the position indicated by the display position information read from the reduced screen position storage unit 1. Then, it is determined that the touched position is not on the reduced screen, and the process proceeds to Step D12.

  In step D20, the input processing unit 13 deletes the reduced screen, ends the third display process, performs other processes, and then returns to step D1. Specifically, the input processing unit 13 instructs the video output unit 16 to display the display screen on the display device 21 without combining the reduced screen with the display screen. The video output unit 16 causes the display device 21 to display a display screen based on the real screen information received from the real screen creation unit 14. Steps D8 and D20 are display steps. Steps D1 and D2 are operation instruction steps. Step D11 is a conversion step. Steps D12 to D16 are operation execution steps. Steps D3 and D4 are determination steps.

  FIG. 9 is a flowchart illustrating a fourth display process executed by the information processing apparatus 10. The fourth display process is a process that enables operation instructions using two input devices 22 having different device types, for example, a touch panel and a mouse. When the power of the display device 1 is turned on and becomes operable, the process proceeds to step E1. Steps E5 to E17 are the same as steps D5 to D17 shown in FIG. 8, respectively, and description thereof is omitted to avoid duplication.

  In step E1, the input device 22 detects a position input by the touch panel or the mouse, and transmits operation position information indicating the detected position, for example, coordinate information indicating coordinates in the real screen coordinate system to the input processing unit 13. . In step E2, the input processing unit 13 determines whether or not there is an input. When the operation position information is received from the input device 22, it is determined that there is an input, and the process proceeds to step E3. When the operation position information is not received from the input device 22, it is determined that there is no input, and the process returns to step E1.

  In step E3, the input processing unit 13 confirms the input device. Specifically, device information is acquired from the device recognition unit 11. In step E4, the input processing unit 13 determines whether the device is a touch panel. If the device type indicated by the device information acquired from the device recognition unit 11 is a touch panel, it is determined that the device is a touch panel. The process proceeds to step E5, and the device type indicated by the device information acquired from the device recognition unit 11 is not a touch panel. It is determined that the device is not a touch panel, and the process proceeds to step E17.

  In step E18, the input processing unit 13 deletes the reduced screen and proceeds to step E12. Specifically, the input processing unit 13 instructs the video output unit 16 to display the display screen on the display device 21 without combining the reduced screen with the display screen. The video output unit 16 causes the display device 21 to display a display screen based on the real screen information received from the real screen creation unit 14. Steps E8 and E18 are display steps. Steps E1 and E2 are operation instruction steps. Step E11 is a conversion step. Steps E12 to E16 are operation execution steps. Steps E3 and E4 are determination steps.

  Thus, in step A8 or A11 shown in FIG. 5, step C12 shown in FIG. 7, steps D8 and D20 shown in FIG. 8, or steps E8 and E18 shown in FIG. 9, a display screen for displaying display information is displayed. A reduced screen obtained by reducing the size is combined with the display screen and displayed. In steps B1 and B2 shown in FIG. 6, steps C1 and C2 shown in FIG. 7, steps D1 and D2 shown in FIG. 8, or steps E1 and E2 shown in FIG. 9, operation instructions on the display screen are given. Do. In step B6 shown in FIG. 6, step C6 shown in FIG. 7, step D11 shown in FIG. 8, or step E11 shown in FIG. 9, steps B1 and B2 shown in FIG. 6, steps shown in FIG. C1, C2, positions D1 and D2 shown in FIG. 8, or positions E1 and E2 shown in FIG. 9 are used as a reference from the position on the basis of the reduced screen as a reference. Convert to position.

  Then, in steps B7 to B11 shown in FIG. 6, steps C7 to C10 shown in FIG. 7, steps D12 to D16 shown in FIG. 8, or steps E12 to E16 shown in FIG. 9, step B1 shown in FIG. , B2, steps C1 and C2 shown in FIG. 7, steps D1 and D2 shown in FIG. 8, or steps E1 and E2 shown in FIG. At a certain time, processing corresponding to the operation instruction for the position converted in step B6 shown in FIG. 6, step C6 shown in FIG. 7, step D11 shown in FIG. 8, or step E11 shown in FIG. Operation instructions are given in steps B1 and B2 shown in FIG. 6, steps C1 and C2 shown in FIG. 7, steps D1 and D2 shown in FIG. 8, or steps E1 and E2 shown in FIG. When the position is a position in the remaining area of the display screen area excluding the reduced screen area, the positions shown in steps B1, B2, and FIG. Steps C 1 and C 2, steps D 1 and D 2 shown in FIG. 8, or steps E 1 and E 2 shown in FIG. Therefore, if the display method according to the present invention is applied, the operator can easily operate the display screen even on a large display screen.

  Further, in steps A1 to A4 shown in FIG. 5, the position of the operator is detected. In step A8 or A11 shown in FIG. 5, the reduced screen is displayed at a position corresponding to the position of the operator detected in steps A1 to A4 shown in FIG. A reduced screen can be displayed near the operator.

  Further, in steps D3 and D4 shown in FIG. 8 or steps E3 and E4 shown in FIG. 9, whether the operation instruction is an operation instruction with a normal touch pen 3b or a mouse that performs an operation instruction on the display screen, It is determined whether the instruction is an operation instruction using the reduction screen touch pen 3a or the touch panel for giving an operation instruction to the reduction screen. Then, in step D11 shown in FIG. 8 or step E11 shown in FIG. 9, the operation instruction is reduced screen touch pen 3a or touch panel in steps D3 and D4 shown in FIG. 8 or steps E3 and E4 shown in FIG. And the position where the operation instruction is given in steps D1 and D2 shown in FIG. 8 or steps E1 and E2 shown in FIG. 9 is within the area of the reduced screen. The position where the operation instruction is performed is converted from a position based on the reduced screen to a position based on the display screen. Therefore, the operation instruction for the reduced image and the operation instruction for the display image in the area excluding the reduced image can be used properly by the two types of instruction devices.

  Further, in step D20 shown in FIG. 8 or step E18 shown in FIG. 9, the operation instruction is given by the normal touch pen 3b or the mouse in steps D3 and D4 shown in FIG. 8 or steps E3 and E4 shown in FIG. When the operation instruction is determined and the position where the operation instruction is given in steps D1 and D2 shown in FIG. 8 or steps E1 and E2 shown in FIG. 9 is within the area of the reduced screen, Since the display screen that does not synthesize the reduced screen is displayed, the reduced screen can be erased by giving an operation instruction to the area where the reduced screen is displayed with the normal touch pen 3b or the mouse.

  Further, in step D8 shown in FIG. 8 or step E8 shown in FIG. 9, the operation instruction is reduced to the reduced screen touch pen 3a or the touch panel in steps D3 and D4 shown in FIG. 8 or steps E3 and E4 shown in FIG. And the position where the operation instruction is given in steps D1 and D2 shown in FIG. 8 or steps E1 and E2 shown in FIG. 9 is the reduced screen in the display screen area. Since the reduced screen is displayed at the position where the operation instruction has been made when it is within the remaining area excluding the area, the position where the reduced image is displayed can be easily moved to a position where it can be easily operated.

  Further, a reduced screen obtained by reducing the display screen for displaying the display information is combined with the display screen and displayed by the real screen creating unit 14, the reduced screen creating unit 15, the video output unit 16, and the display device 21. The operation instruction on the display screen is performed, and the position where the operation instruction is performed by the input unit 22 by the coordinate conversion unit 12 is changed from the position based on the reduced screen to the position based on the display screen. When the position where the operation instruction is performed by the input processing unit 13 by the input processing unit 13 is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted by the coordinate conversion unit 12 is performed. The position where the operation instruction is given by the input device 22 is a position in the remaining area of the display screen area excluding the reduced screen area. Rutoki, processing corresponding to the operation instruction for position relative to the said display screen is performed. Therefore, even on a large display screen, the operator can easily operate on the display screen.

It is a block diagram which shows the structure of the display apparatus 1 which is one Embodiment of this invention. It is a figure which shows an example of the display screen 30 which displayed the reduction screen 31 near the operator 2. FIG. It is a figure which shows an example of the display screen 40 which displayed the reduction screen 42 in the range which the operator 2 can reach. It is a figure which shows an example of the display screen 50 operated using two types of touch pens. 4 is a flowchart illustrating a first display process executed by the information processing apparatus 10. 4 is a flowchart showing touch processing executed by the information processing apparatus 10. 5 is a flowchart showing a second display process executed by the information processing apparatus 10. 12 is a flowchart illustrating a third display process executed by the information processing apparatus 10. 12 is a flowchart illustrating a fourth display process executed by the information processing apparatus 10. It is a figure which shows the display screen 90 of the map display by a prior art. It is a figure which shows the display screen 93 of the virtual desktop by a prior art. It is a figure which shows the display apparatus 96 which has a touchscreen function by a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display apparatus 10 Information processing apparatus 11 Device recognition part 12 Coordinate conversion part 13 Input processing part 14 Real screen creation part 15 Reduced screen creation part 16 Image | video output part 17 Reduced screen position memory | storage part 18 Image memory | storage part 21 Display apparatus 22 Input device 23 Human recognition device

Claims (6)

A display step of combining and displaying a reduced screen obtained by reducing a display screen for displaying display information with the display screen;
An operation instruction step for performing an operation instruction on the display screen;
A conversion step of converting the position where the operation instruction is given in the operation instruction step from a position based on the reduced screen to a position based on the display screen;
When the position where the operation instruction is performed in the operation instruction step is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted in the conversion step is performed, and the operation instruction is performed in the operation instruction step. When the performed position is a position in the remaining area excluding the reduced screen area in the display screen area, the operation instruction performed in the operation instruction step with respect to the position based on the display screen is performed. An operation execution step of performing corresponding processing. A detection step for detecting the position of the operator;
The display method according to claim 1, wherein in the display step, the reduced screen is displayed at a position corresponding to the position of the operator detected in the detection step among the positions of the display screen. A determination step of determining whether the operation instruction is an operation instruction by a first instruction device that performs an operation instruction on the display screen or an operation instruction by a second instruction device that performs an operation instruction on the reduced screen; In addition,
In the conversion step, it is determined in the determination step that the operation instruction is an operation instruction by a second instruction device, and a position where the operation instruction is performed in the operation instruction step is within an area of the reduced screen. 2. The display method according to claim 1, wherein the position where the operation instruction is performed is converted from a position based on the reduced screen to a position based on the display screen.   In the display step, it is determined in the determination step that the operation instruction is an operation instruction by a first instruction device, and a position where the operation instruction is performed in the operation instruction step is within an area of the reduced screen. The display method according to claim 3, wherein a display screen that does not synthesize the reduced screen is displayed.   In the display step, it is determined in the determination step that the operation instruction is an operation instruction by a second instruction device, and a position where the operation instruction is performed in the operation instruction step is the region of the display screen. The display method according to claim 3, wherein the reduced screen is displayed at a position where the operation instruction is performed when the area is in a remaining area excluding the reduced screen area. Display means for combining and displaying a reduced screen obtained by reducing a display screen for displaying display information with the display screen;
An operation instruction means for performing an operation instruction on the display screen;
Conversion means for converting the position where the operation instruction is given by the operation instruction means from a position based on the reduced screen to a position based on the display screen;
When the position where the operation instruction is given by the operation instruction means is within the area of the reduced screen, processing corresponding to the operation instruction for the position converted by the conversion means is performed, and the operation instruction is given by the operation instruction means. When the performed position is a position in the remaining area of the display screen area excluding the reduced screen area, an operation is performed to perform a process corresponding to the operation instruction for a position based on the display screen And a display device.

Images