JP3504467B2 - Multi-window display device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-window display device for displaying a plurality of windows on a screen of an information device, and more particularly to a device for displaying windows as if they were placed in a virtual three-dimensional space.

[0002]

2. Description of the Related Art In an information terminal such as a computer, a multi-window display device is used to improve the operability of a man-machine interface. As a conventional multi-window display device that efficiently displays as many windows as possible on a screen having a limited area, see Japanese Patent Application Laid-Open No. 6-2228.
There is a multi-window display method described in 99.

FIG. 30 is a diagram showing a screen display according to the above conventional technique. In this conventional technique, windows are arranged in a virtual three-dimensional space, and the windows placed in the back of the screen are reduced and displayed, thereby improving the storage efficiency of the windows and making the screen feel psychologically wide. Is.

[0004]

However, in the above-mentioned prior art, since the window placed in the back of the screen is displayed in a reduced size, the entire display content of the window cannot be determined. is there. Therefore, the present invention has been made in view of the above problems, and is a device for displaying a window as if it were placed in a virtual three-dimensional space, which improves the storage efficiency of the window and makes the screen psychologically wider. It is an object of the present invention to provide a multi-window display device that makes a user feel and is less likely to have a problem that all the display contents cannot be determined.

[0005] To accomplish the above object, a multi-window display apparatus for displaying a plurality of windows on a screen, the window data holding means for holding the window data representing the display contents of the window, Misao
Input means for obtaining instructions from the author and the window
Based on the window data held in the data holding means
Of the contents displayed in the above window that meet certain conditions.
Window analysis means to detect places as important points,
Determining a virtual placement position in at least one of the windows that is not parallel to the screen in a three-dimensional space
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Determine the coordinates in the dimensional space as the placement position,
The location detected by the analysis means will be displayed in front.
A placement position determining means for determining a placement position of the window, perspective projection means for transforming the window data so as to perspective-project the window on the screen based on the determined placement position, and the transformed window data Is displayed on the screen.

As a result, at least one window is arranged tilted in the depth direction in the three-dimensional space,
Since the image is perspectively projected and displayed on the screen, even if the window is arranged in the back of the three-dimensional space, the entire size of the window is uniformly reduced, so that all the displayed contents cannot be determined. Technical problems can be avoided, the display area of the screen can be effectively utilized, and the screen can be made to feel psychologically large.

[0007]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. [First Embodiment] First, a multi-window display device according to a first embodiment will be described.

This apparatus is characterized in that the window is displayed in a three-dimensional space with the window tilted in the depth direction. The shape of the window itself is a quadrangular plane whose outer shape is specified by four vertices. (Structure) FIG. 1 is a block diagram showing the structure of the multi-window display device according to the first embodiment.

This apparatus comprises a program execution unit 101, a storage unit 102, an input unit 103, a three-dimensional position calculation unit 104, a texture mapping unit 105, a perspective projection unit 106, a frame memory unit 107 and an image display unit 108. . The program execution unit 101 is a CPU or the like that displays an ordinary (two-dimensional) window and activates and executes an application program that interacts with the user. Specifically, information about each window, that is, a display target of the window ( Code data (documents, characters, figures, etc.) representing objects) and image data representing objects in bit images are stored in the storage unit 102, and information regarding the window is exchanged with other components 102 to 108.

The image data is image data that does not depend on the size of the entire window in which each object is placed or the position of the window itself in the three-dimensional space, and is represented by a fixed size determined in advance. Data. The storage unit 102 is a memory that stores the code data and the image data in association with each window of each application program. It should be noted that all the windows stored in the storage unit 102 are not necessarily displayed on the image display unit 108, but the windows displayed on the image display unit 108 are not necessarily stored on the image display unit 108. It is limited to

The input unit 103 is a pointing device such as a mouse and receives an instruction from the user. The three-dimensional position calculation unit 104, according to the notification from the program execution unit 101 or the input unit 103, the arrangement position in the three-dimensional space for one window stored in the storage unit 102 (coordinates of four vertices of each window). Is calculated and stored. Specifically, when the program execution unit 101 receives the notification of the four vertex coordinates as the initial position, it is stored as it is, while the 3 of the already placed windows are stored.
When the notification of the movement displacement (type of movement and movement amount) in the dimensional space is received, a new vertex coordinate is calculated by affine transformation.

If the window layout position is not explicitly specified by the program execution unit 101 or the input unit 103, the three-dimensional position calculation unit 104 sets the default layout position, for example, the left side of the window. An arrangement position in which the right side of the axis of rotation is inclined by 45 degrees in the depth direction of the screen is generated and stored. Texture mapping unit 105
When the new placement position is calculated by the three-dimensional position calculation unit 104, the image data of the window is read from the storage unit 102, and the program execution unit 101 simultaneously enlarges or reduces the window size according to the placement position. The texture mapping is performed based on the instruction from, and the obtained texture data is sent to the perspective projection unit 106. Note that this texture data is
The data is determined only by the position of arrangement in the dimensional space, and does not depend on the position of the viewpoint of viewing the window.

The perspective projection unit 106 performs perspective projection on the texture data sent from the perspective projection unit 106,
The obtained perspective data is stored in the frame memory unit 107. Here, the “perspective projection” means that an object (window) placed in the above-mentioned three-dimensional space has a viewpoint placed in front of the display surface of the image display unit 108 as a projection center, and that two-dimensional display surface is the projection surface. Is an operation for generating a perspective view. Note that "front" and "back" in this specification
Is an expression based on the distance from the viewpoint of the user located in front of the display surface.

FIGS. 2A and 2B are views for explaining the concept of perspective projection by the perspective projection unit 106. FIG. 2A is a diagram showing the positional relationship between the projection center, the projection surface, and the object (window). Perspective projection unit 106
Is a perspective projection of an image projected on the display screen by placing a display screen of the image display unit 108 between a window virtually placed in a three-dimensional space and the user's viewpoint, and applying light that converges to the viewpoint to the window. Generated window.

FIG. 2B shows a perspective view when the object is a three-dimensional object. In this case, a point (vanishing point) where parallel lines intersect on a horizontal line at infinity occurs behind the three-dimensional object. The perspective projection unit 106 stores the coordinates of 256 vanishing points placed at predetermined positions in the three-dimensional space in order to perform perspective projection by multipoint perspective instead of single point perspective.

FIGS. 3A to 3C show the perspective projection unit 1.
It is a figure for demonstrating the specific processing content of the perspective projection by 06. FIG. 3A is a diagram showing the perspective of the user, the positional relationship between the display screen and the window, and the perspective-projected shape of the window when the window virtually placed in the three-dimensional space is parallel to the display screen. is there.

In this case, the perspectively projected window has a rectangular shape. FIG. 3B shows the user's viewpoint and the display when the window virtually placed in the three-dimensional space is not parallel to the display screen, that is, when the window is placed tilted in the depth direction of the display screen. It is a figure which shows the positional relationship of a screen and a window, and the shape of the window by perspective projection.

In this case, the perspectively projected window has a trapezoidal shape. FIG. 3C is a diagram illustrating a specific calculation process of perspective projection. A line segment parallel to the z-axis with the display screen as the xy plane and the depth direction as the z-axis.
The case of perspective projection of A0B0 is shown. Perspective projection unit 1
06 is a line segment A1B1 first calculated from the above 256 vanishing points by perspective transformation based on one vanishing point C specified by the program execution unit 101 or the input unit 103. Then, the final line segment A2B2 is calculated by projection conversion by parallel projection onto the xy plane.

The frame memory section 107 includes an image display section 1
08 is a video RAM that stores image data for one frame to be displayed on the screen. Image display unit 108
Is equipped with an image display control circuit, a CRT, etc., and reads out the image data placed in the frame memory unit 107 to perform CR.
Display on T. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

FIG. 4 is a flow chart showing the operation procedure of the multi-window display device of this embodiment. Figure 5
FIG. 9 is a diagram showing a window displayed on the image display unit 108 by the operation. First, the program execution unit 10
1 starts and executes an application program, and stores the code data and image data of the window generated by the application program in the storage unit 102 (steps S120 and S121).

Next, the three-dimensional position calculation unit 104, according to the notification from the program execution unit 101 or the input unit 103, arranges the one window stored in the storage unit 102 in the three-dimensional space (the position of each window). The coordinates of the four vertices are calculated and stored (step S12).
2). The coordinate system at this time is as shown in FIG.

Subsequently, the texture mapping unit 105
When the new placement position is calculated by the three-dimensional position calculation unit 104, the image data of the window is read from the storage unit 102, and the three-dimensional position calculation unit 1 reads it.
Texture mapping is performed to fit and paste in a window having a size determined by the four vertices calculated by 04 (step S123).

Then, the perspective projection unit 106 specifies the vanishing point based on the notification from the program executing unit 101 or the input unit 103 in the texture data generated by the texture mapping unit 105, and the perspective using the vanishing point. Projection is performed, and the obtained perspective data is overwritten on the corresponding position of the frame memory unit 107 (step S124). If there is no explicit designation of the vanishing point from the program execution unit 101 or the input unit 103, the perspective projection unit 106 determines that the x coordinate and the y coordinate are the same as the center of each window and the z coordinate is Perspective projection is performed using vanishing points that have a value equal to the width of the display screen.

Finally, the image display unit 108 reads the fluoroscopic data (image data) written in the frame memory unit 107 and displays it on the CRT (step S12).
5). Here, the program execution unit 101 or the input unit 10
When the 3D position calculation unit 104 is notified that the window layout position stored in the storage unit 102 is changed, the image display unit 108 is repeated by repeating the processes in steps S122 to S125. The contents displayed in are updated.

As described above, since the present multi-window display device specifies the coordinates of the four vertices of the window virtually placed in the three-dimensional space and projects it perspectively on the display screen, the window generated by the application program is When the arrangement position in the three-dimensional space is parallel to the display screen, it is displayed as the rectangular windows 130 and 131 shown in FIG.
If the positions of the windows are not parallel to the display screen, the trapezoidal windows 132 to 13 shown in FIG.
Displayed as 4.

That is, the two windows 130 and 131 on the left side of the screen shown in FIG. 5 are examples in which they are facing the front of the user, and the window 132 on the right side of the screen is
This is an example in which the upper side of the window is tilted in the depth direction with respect to the user, and the lower side is tilted in the depth direction with respect to the user. This is an example of tilting. (Summary) As described above, as a result of the window being tilted and displayed by the device, the user may have a little difficulty in recognizing the content of the characters and the like displayed in the tilted depth portion, but the user cannot see the contents of the tilted front part. Regarding the part, it is possible to fully understand the content. As a result, as compared with the case where the window is displayed in front, the display area of the screen in the depth portion can be reduced by the amount of tilting, so that the display area of the screen is effectively used.

In particular, in a portable information terminal such as a notebook computer, a PDA (Personal Digital Assistants) and an electronic notebook used for mobile use, the display area of the screen is limited. When doing so, the window is directed to the front, and when a plurality of windows are referred to when the windows are not operated, the windows are tilted and displayed, so that the screen can be effectively utilized. Further, the invention is also effective on a desktop computer when browsing a homepage on the Internet or activating a plurality of application programs.

In the present embodiment, the window itself is a quadrangular plane, but the present invention is not limited to this shape. For example, it may be a window having a thickness or a polygon. This is because, regardless of the shape of the window, the display surface can be tilted in the depth direction of the screen for perspective projection. Further, in the present embodiment, one side of the rectangular window is rotated as an axis, but two sides may be sequentially rotated as an axis.

Further, in the present embodiment, the perspective projection is performed after performing the texture mapping for all the display contents of the window. However, the order is reversed, and the window composed of four vertices of the window arranged in the three-dimensional space is used. A texture for projecting only the frame on the two-dimensional display surface by perspective projection in advance and pasting the image data of the window stored in the storage unit 102 to the projected window frame while performing linear transformation such as affine transformation. You may map. By switching the order of texture mapping and perspective projection, and performing two-dimensional linear conversion during texture mapping,
Although the strictness of the texture is impaired, the calculation load is reduced as the processing area becomes smaller.

Further, the multi-window display device of this embodiment is provided with a three-dimensional position calculation unit 104, a texture mapping unit 105 and a perspective projection unit 106 in addition to the program execution unit 101. Alternatively, the configuration may be such that a part of the processing is executed. Further, the storage unit 102 and the frame memory unit 107 may share the same storage device. [Second Embodiment] Next, a multi-window display device according to a second embodiment will be described.

The apparatus is characterized in that the window is displayed so that the important portion of the display content of the window is in the foreground. (Structure) FIG. 6 is a block diagram showing the structure of the multi-window display device according to the second embodiment.

The present apparatus further includes a window analysis unit 201 in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. Window analysis unit 201
Is the time when the application program is started by the program execution unit 101 and the code data and image data of one window is stored in the storage unit 102,
The code data of the window is read, an important part is specified from the code data, the position of the window is determined so that the part is displayed in front, and the three-dimensional position calculating unit 10 is determined.
Notify 4.

Specifically, the window analysis unit 201 is
The side of the window closest to the specified important point is set as the rotation axis, and the side opposite to the side is notified to be rotated by 45 degrees in the depth direction. The "important part" means the part of the window that the user is likely to pay attention to at first glance to determine the type and display contents of the window, and the specific judgment criteria are predetermined procedures described later. Follow

In this embodiment, the three-dimensional position calculation unit 104
In addition to the program execution unit 101 and the input unit 103,
The window analysis unit 201 also receives a notification about the initial placement position or movement displacement of the window, and calculates a new placement position. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, window code data and image data are generated (step S1 in FIG. 4).
21) and the calculation of the next three-dimensional arrangement position (step S in FIG. 4).
122) has been added to the processing for identifying the important part of the window. Therefore, the added processing will be described here.

FIG. 7 is a flow chart showing a procedure for the window analysis unit 201 to identify an important part of a window. FIG. 8 shows an example of a screen display when an important part is specified by the window analysis unit 201.
When the application program is started by the program execution unit 101 and the code data and image data of one window are stored in the storage unit 102 (step S121 of FIG. 4), the window analysis unit 201 reads the code data of the window. (Step S22
0).

Then, the window analysis unit 201 determines whether or not the code data has a title to be displayed in the title bar of the window (step S221), and if there is, determines that the title bar is an important part of the window, The three-dimensional position calculation unit 104 is notified that the upper side is the rotation axis (step S222, step S12 in FIG. 4).
2). The resulting screen display is as in the window 230 shown in FIG. Since the title 234 is present in the title bar 235, the title bar 235 is displayed so that the upper side is inclined toward the front. The detection of the title is determined by whether or not the code data includes a reserved word that declares the title of the window.

If there is no title, it is subsequently determined whether or not the code data includes a horizontally written sentence from left to right (step S2232). If it is included, the left side of the window is determined to be the important point, and the three-dimensional position calculation unit 104 is notified that the left side is the rotation axis (step S).
224, step S122 in FIG. 4). The resulting screen display is as in the window 231 shown in FIG.
The left side, which is the beginning of the sentence, is displayed so that it is tilted toward you.

When the horizontally written sentence is not included, it is subsequently determined whether or not the code data includes a vertically written sentence that is line-feeded from right to left (step S225). If it is included, the right side of the window is judged to be important,
The three-dimensional position calculation unit 104 is notified that the right side is the rotation axis (step S226, step S122 in FIG. 4).
The resulting screen display is the window 23 shown in FIG.
There are three. The right side, which is the beginning of the sentence, is displayed so that it is tilted toward you. It should be noted that horizontal writing and vertical writing are detected by determining whether or not the code data includes a reserved word that specifies the style of the writing.

If the vertically written sentence is not included, it is then determined whether the code data includes a drawing (data in a predetermined drawing format) (step S227). If it is included, the drawing is determined to be an important part, the side of the window that is closest to the layout position of the drawing (the center point of the rectangular area surrounding the drawing) is specified, and that side is used as the rotation axis. The calculation unit 104 is notified (step S228, step S122 in FIG. 4). The resulting screen display is as in window 232 shown in FIG. The right side closest to the drawing is displayed so that it is inclined toward you.

If the drawing is not included, the three-dimensional position calculation unit 104 indicates that the left side is the rotation axis by default.
(Step S229, step S12 of FIG. 4)
2). The resulting screen display is as in the window 231 shown in FIG. This is because, in general, the content of interest is often displayed in the upper left portion of the window. (Summary) As described above, this device identifies an important point in the window and tilts the window so that the point is in the foreground, making it difficult to read the display contents by perspective projection of the window. Is alleviated.

In the present embodiment, the window analysis unit 201 specifies the important portion based on the presence / absence of the title in the window, the text style, and the presence / absence of the drawing, but the display information is concentrated in addition to these judgment criteria. A portion or a portion in which a reserved word representing the content of the page is placed may be specified as the important portion. In addition, when all the display contents of the window are drawings, it is determined by the composition of the drawing. For example, in the case of a window including drawings including a person, the position of the face of the person is determined by the distribution of the skin color, and the face is easy to see. If you tilt the window so that it is in the foreground, or if it is a window composed of drawings that include mountains, analyze the distribution of green and brown of mountains and the distribution of blue of the sky, and if you place importance on mountains, It is also possible to tilt the window so that the area with a large distribution of green or brown in the mountain is in the foreground. This is because these pieces of display information can be easily discriminated from the image data, the character code, etc. before being developed into image data. [Third Embodiment] Next, a multi-window display device according to a third embodiment will be described.

This apparatus is characterized by automatically aligning a plurality of windows which are scattered and displayed on the screen. (Structure) FIG. 9 is a block diagram showing the structure of the multi-window display device according to the third embodiment.

The present apparatus is common in that it includes the constituent elements 101 to 108 included in the multi-window display device of the first embodiment, but that the three-dimensional position calculation unit 104 of the present apparatus has an automatic alignment unit 104a inside. different. Hereinafter, differences from the first embodiment will be described. The automatic alignment unit 104a is
For all windows stored in the storage unit 102, the arrangement positions are aligned by the first non-overlapping alignment method or the second overlapping alignment method.

Specifically, the automatic alignment section 104a determines that the target window is 1 in the first alignment method.
The final alignment positions where the windows do not overlap in each case of 1 to 16 are stored in advance. on the other hand,
Regarding the second alignment method, the left side of each window is in front, the right side is tilted in the depth direction, and the arrangement positions are arranged at equal intervals in the horizontal direction so that the right halves overlap.

When the automatic alignment unit 104a receives an instruction to specify the alignment method and the number of windows to be aligned from the three-dimensional position calculation unit 104, the texture mapping unit sequentially determines the layout positions of the windows. Send to 105. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In the calculation of the three-dimensional arrangement position in the first embodiment (step S122 in FIG. 4), the automatic arrangement unit 104a calculates a new arrangement position. Therefore, a specific procedure of automatic alignment by the automatic alignment unit 104a will be described here. FIG. 10 is a flowchart showing the procedure of automatic alignment by the automatic alignment unit 104a.

11 (a) and 11 (b) show an example of screen display when five windows are automatically aligned by the automatic alignment unit 104a, and FIG. 11 (a) shows the first non-overlapping screen. In the case of the alignment method of FIG.
(B) shows an example in the case of the overlapping second alignment method. First, the three-dimensional position calculation unit 104 receives a designation of the alignment method from the input unit 103, reads out and specifies the number n of windows to be aligned from the storage unit 102, and notifies them to the automatic alignment unit 104a ( Step S230).

Upon receipt of the notification, the automatic arranging unit 104a judges the arranging method (step S231). As a result, if the first arranging method does not overlap, it is stored in advance in correspondence with the notified number n. The arranged positions of the n windows thus created are sequentially read out (step S232) and notified to the texture mapping unit 105 (step S23).
4). The resulting screen display is as shown in FIG.

On the other hand, in the case of the overlapping second alignment method, the automatic alignment unit 104a overlaps the right half of the notified number n with the n windows tilted with the left side toward you. The arrangement positions of the n windows are sequentially calculated so that they are evenly arranged in the horizontal direction (step S233), and the texture mapping unit 10
5 is notified (step S234). The resulting screen display is as shown in FIG. (Summary) As described above, since many windows scattered and displayed on the screen are automatically aligned by the perspective projection mode, unnecessary areas on the screen are eliminated, and at a glance It is possible to understand the display position and contents of the window.

Here, after automatic alignment of the windows is once performed, it is possible to effectively utilize the arrangement position as a default position of each window. For example, the three-dimensional position calculation unit 104 stores the alignment position of each window obtained by the automatic alignment as a default position, and when one is designated by the input unit 103 from the automatically aligned window group, the window is sorted. The layout position is calculated so that the front side of the window is the front side of the rotation axis, and the front side of the window is a specific position (for example, the window arrangement button on the title bar).
A unit for calculating the arrangement position may be added so that when the window is designated by the input unit 103, the window is returned to the default position when the windows are automatically arranged.

FIGS. 12A to 12C are views showing an example in which the automatically arranged arrangement positions are effectively used as the default positions of the respective windows. FIG. 12A shows a screen display immediately after automatic alignment, and corresponds to FIG. 11B. 12
12B shows the input unit 1 in the screen display of FIG.
The screen display immediately after one window 241 is selected by 03 is shown.

Here, the three-dimensional position calculation unit 104 calculates a new layout position so that the selected side of the window 241 faces the front with the front side (left side) fixed. FIG. 12C shows the screen display immediately after the window organizing button 245 provided at the right end of the title bar of the window 241 is pressed by the input unit 103 in the screen display of FIG. 12B.

Here, the three-dimensional position calculation unit 104 reads the default position already stored for the window in which the window arrangement button 245 is pressed as a new arrangement position and sends it to the texture mapping unit 105.
As a result, the window 241 is tilted in the depth direction and returns to the position immediately after being automatically aligned. in this way,
By operating one button, the window can be turned to the front or returned to the automatically aligned position, so that the operability of the window is improved by applying it to a portable information terminal having a particularly small display area.

In the present embodiment, the plurality of windows are arranged so that the right halves thereof are overlapped by the overlapping second arrangement method. However, by further providing the window analysis unit 201 of the second embodiment, the windows are arranged. It is easy to deform so that the non-important parts of are overlapped. [Fourth Embodiment] Next, a multi-window display device according to a fourth embodiment will be described.

This device is characterized in that the positions of the title bar and the menu bar are changed according to the inclination of the window. (Structure) FIG. 13 is a block diagram showing the structure of the multi-window display device according to the fourth embodiment.

The present apparatus further includes a window information image position changing unit 301 and a window information image direction changing unit 302 in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. The window information image position changing unit 301 positions the window information image so that the window information image (the image of the title bar and the menu bar) is displayed on the front side of the window when the window is displayed tilted in the depth direction. To change. Specifically, the placement position of the window in the three-dimensional space calculated by the three-dimensional position calculation unit 104 is read, and the placement position is a position tilted in the depth direction with one side of the window in front, and the window information. If the image is not displayed on the front side, the position of the window information image is changed to the front side by rewriting the image data of the window stored in the storage unit 102.

The window information image direction changing unit 302
When the window is displayed tilted in the depth direction and the window information image is displayed on the near side, only the rectangular part of the window information image is bent and faces the front (parallel to the display screen). Change the data. Specifically, the image data of the entire window stored in the storage unit 102 is divided into the window information image and the remaining window body portion, and the window information image is notified to the three-dimensional position calculation unit 104 so as to be directed to the front. The three-dimensional position calculation unit 104 is notified that the remaining window body portion should be tilted in the depth direction. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, the processing for the window information image is added between the calculation of the three-dimensional arrangement position (step S122 in FIG. 4) and the next texture mapping (step S123 in FIG. 4). is there. Therefore, the added processing will be described here. 14
4 is a flowchart showing an operation procedure of the window information image position changing unit 301 and the window information image direction changing unit 302.

The window information image position changing unit 301
When the arrangement position of the window in the three-dimensional space is calculated by the three-dimensional position calculation unit 104 (step S12 in FIG. 4).
2) By referring to the arrangement position, it is determined whether or not the arrangement position is a position tilted in the depth direction with one side of the window as the front (step S320). as a result,
If the determination is affirmative, then it is determined whether the window information image is located on the front side (step S
321) If the window information image is not located on the front side, the image data of the window stored in the window information image storage unit 102 is rewritten so that the window information image is displayed on the front side (step S322).

FIG. 15 shows an example of a screen display when the window 330 before the position of the window information image is changed and the window 331 after the position is changed are directly displayed through texture mapping and perspective projection. Show. It can be seen that the title bar and menu bar (window information image) that should be displayed in the back of the screen are moved to the front side of the screen and are displayed.

Subsequently, the window information image direction changing unit 3
02 determines whether or not an instruction to direct the window information image to the front is made from the input unit 103 (step S323). If the instruction is received, the storage unit 10
Among the image data of the entire window stored in 2, the window information image and the remaining window body portion are divided and specified (step S324), and they are connected, and the window information image is directed to the front. The three-dimensional position calculating unit 104 is notified so that the remaining window body portion is tilted in the depth direction, and the three-dimensional position calculating unit 104 is notified so that the arrangement positions are calculated separately (step S325).

Thereafter, the window information image and the remaining window body portion are sequentially displayed on the screen through texture mapping and perspective projection (step S12 in FIG. 4).
3 to step S125). FIG. 16 shows the window 332 before the orientation of the window information image is changed in this way.
An example of the screen display of the window 333 after being changed to the front is shown.

It can be seen that, of the windows tilted in the depth direction, the title bar and menu bar (window information image) parts are bent and directed to the front. (Summary) As described above, the window information image for performing the window operation and the menu operation is always displayed in the foreground even when the entire window is tilted in the depth direction by the device. Further, only the window information image displayed in the foreground can be fixed so that it always faces the front. This avoids the problem that the window operation and the menu operation become difficult due to the tilted display of the window.

In the present embodiment, the processing is performed such that the entire planar window is bent into the window information image and the remaining portion, but it is assumed that the window itself is a three-dimensional plate model with a large thickness. A method of displaying a title bar or a menu bar on the side contacting the front side when tilted in the depth direction may be used. As a result, the window information image and the remaining portion can be handled as integrated data without being separated, which facilitates the process of changing the position and changing the direction.

Further, it is easy to see and operate the window, and at the same time, it is not necessary to display on the window surface, so that the window surface can be effectively utilized. Furthermore, if the direction may be changed at any time by the window operation, the title bar and the menu bar may be displayed in advance on all four side surfaces in contact with the window surface. [Fifth Embodiment] Next, a multi-window display device according to a fifth embodiment will be described.

The present apparatus is characterized in that the display of the scroll bar is controlled in association with the size of the indistinguishable display area caused by tilting the window. (Structure) FIG. 17 is a block diagram showing the structure of the multi-window display device according to the fifth embodiment.

The present apparatus further includes a scroll button position determination unit 401, a scroll bar addition unit 402, and an unidentifiable area determination unit 403 in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. The unidentifiable area determination unit 403 identifies an area that cannot be identified in the final display state because the window is arranged so as to be inclined in the depth direction. Specifically, prior to the processing by the texture mapping unit 105 and the perspective projection unit 106, the character size of the window stored in the storage unit 102 and the position of the window calculated by the three-dimensional position calculation unit 104, By referring to the conversion constants in the perspective projection unit 106 (vanishing point and viewpoint coordinates), the size of the character finally placed in a representative position on the window when it is finally displayed is calculated, When it is smaller than 5 points × 5 points, the area displayed behind the character is determined as the unidentifiable area.

When the unidentifiable area determination section 403 detects an unidentifiable area, the scroll button position deciding section 401 decides the position of the scroll button reflecting the area. Specifically, the side along the depth direction is set as the scroll direction, and the position is determined from the ratio of the scroll direction distance of the identifiable region and the scroll direction distance of the unidentifiable region. The position corresponds to the position when the unidentifiable area is not displayed in the window.

The scroll bar addition unit 402 adds the image of the scroll bar in which the scroll button is arranged at the position determined by the scroll button position determination unit 401 to the side along the depth direction of the window and displays it. Specifically, the image of the scroll bar is added to the image data of the window stored in the storage unit 102. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, the image processing for the scroll bar is added between the calculation of the three-dimensional arrangement position (step S122 in FIG. 4) and the next texture mapping (step S123 in FIG. 4). is there. Therefore, the added processing will be described here. FIG.
4 is a flowchart showing the operation procedure of the scroll button position determination unit 401, scroll bar addition unit 402, and unidentifiable area determination unit 403.

When the three-dimensional position calculation unit 104 calculates the arrangement position of the window in the three-dimensional space (step S122 in FIG. 4), the unidentifiable area determination unit 403 arranges the window in the depth direction. With that,
It is determined whether or not an unidentifiable region occurs in the finally displayed window (step S420),
If it occurs, specify the area.

If it is determined that an unidentifiable area occurs, then the scroll button position determination unit 401
Determines the position where the scroll button should be arranged based on the size of the area (step S421). That is, the position is determined when only the identifiable area is displayed in the window. Then, the scroll bar addition unit 40
2 adds the image of the scroll bar in which the scroll button is arranged at the position determined by the scroll button position determination unit 401 to the image data stored in the storage unit 102 (step S422).

As described above, when the content of the storage unit 102 is rewritten to the image data of a new window having a scroll bar, the texture mapping is performed on the new image data thereafter, as in the case of the first embodiment. Is displayed on the screen through perspective projection (steps S123 to S125 in FIG. 4). After that, the input unit 103
If you slide the scroll button by,
Although the program execution unit 101 performs a normal scrolling operation, since the arrangement position of the window itself does not change, the position of the scroll button shifted by the unidentifiable area does not change.

FIG. 19A shows a scroll button 430.
FIG. 14 is a diagram showing a general concept of a scroll bar 431 and shows an example of a conventional window display. The position of the scroll button on the scroll bar indicates the relative position of the contents currently displayed in the window in the entire display target. FIG. 19B shows an example of screen display when it is determined that the lower half of the window display area is the unidentifiable area 432.

Here, the scroll button 430 is displayed at a position where the lower half 432 of the display area is out of the window. Therefore, the input unit 10
When the scroll button 430 is moved downward by 3, the area that cannot be identified until then is moved to the identifiable position. (Summary) As described above, even if an unidentifiable region occurs because the window is tilted in the depth direction by this device, a scroll bar that slides the window in the depth direction and moves it toward you is displayed. Since it is attached, the entire display target can be identified by a simple operation of sliding the scroll button without moving the window itself.

In the present embodiment, a scroll bar is added when an unidentifiable area is detected. However, if there is a portion that is not displayed in the window regardless of whether there is an unidentifiable area in the first place. May have a configuration in which a scroll bar is already added. In this case, the position of the scroll button is the position in the case where the area obtained by adding the unidentifiable area to the protruding area is protruding. Further, in texture mapping, the operability for the user can be improved by distinguishing the indistinguishable region and the region protruding from the page by color-coding the scroll bar. [Sixth Embodiment] Next, a multi-window display device according to a sixth embodiment will be described.

The present apparatus is characterized in that when a window having a layer structure is displayed, each layer is separated and tilted in the depth direction for display. (Structure) FIG. 20 is a block diagram showing the structure of the multi-window display device according to the sixth embodiment.

The present apparatus further comprises a window layer separation section 501 and a layer window generation section 502 in addition to the constituent elements 101 to 108 included in the multi-window display apparatus of the first embodiment. Hereinafter, differences from the first embodiment will be described. The window layer separation unit 501 detects a window having a layer structure from the windows stored in the storage unit 102 and separates the window for each layer.

In an application program in which image data and text data are mixed and an application program in which plural image data are mixed, image data and text data are separated from each other like graphic drawing software and document layout software used for desktop publishing. There is an application program that has layers and can be edited and operated independently.
When such an application program is executed by the program execution unit 101, the storage unit 102 stores code data and image data of a window having a layer structure.

Specifically, the window layer separation unit 501 detects a window having a layer structure by analyzing the code data of each window stored in the storage unit 102, and detects the basic data from the image data of the window. Image data belonging to layers other than the (lowermost) layer is extracted. The layer window generation unit 502
An independent layer window composed of layers separated by the window layer separation unit 501 is generated. Specifically, the image data extracted by the window layer separation unit 501 is used as one independent window for each layer to generate new image data, and the storage unit 1
Stored in 02. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, window code data and image data are generated (step S1 in FIG. 4).
21) and the calculation of the next three-dimensional arrangement position (step S in FIG. 4).
122) and the processing for the layer window has been added. Therefore, the added processing will be described here.

FIG. 21 shows the window layer separating section 501.
9 is a flowchart showing an operation procedure of the layer window generation unit 502. When the program execution unit 101 stores the code data and the image data of the window having the layer structure in the storage unit 102 (step S121 in FIG. 4), the window layer separation unit 501 extracts the code data stored in the storage unit 102. After the window having the layer structure is specified and the image data is separated for each layer, only the basic layer is left and the image data of other layers is taken out (step S520).

FIG. 22A is a diagram conceptually showing the separation of layers (step S521) when there are two layers. It is shown that the composition window 530 stored in the storage unit 102 is separated into a base layer 531 and another upper layer 532. Subsequently, the layer window generation unit 502 generates image data having the upper layer 532 extracted by the window layer separation unit 501 as an independent window and stores the image data in the storage unit 102 (step S521). Then, the layer window generation unit 502 uses the newly generated layer window 5
For 32, as the arrangement position in the three-dimensional space,
The three-dimensional position calculation unit 104 is notified so that the position is parallel to the window of the basic layer 531 and is separated by a predetermined distance in the front direction when the basic window is viewed from the front (step S522).

Therefore, the three-dimensional position calculation unit 104 calculates the arrangement position of the window of the base layer 531 as specified by the program execution unit 101 or the input unit 103, while the window of the upper layer 532 is calculated as follows. Based on the designation from the layer window generation unit 502, the arrangement position of the window of the base layer 531 translated in parallel with the window surface by a predetermined distance is calculated (step S122 in FIG. 4).

FIG. 22B shows an example of a screen display when a window having a layer structure is arranged tilted in the depth direction. The upper layer 532 is displayed as if it floats above the base layer 531. (Summary) As described above, by tilting the window having the layer structure in the depth direction and separately displaying the information by the present apparatus, information that cannot be distinguished in the conventional front direction, that is, in which layer It is possible to grasp at a glance the information that an object is placed. As a result, instead of designating an object while switching to a different layer, it is possible to directly designate an object while displaying all layers, which facilitates editing operations across a plurality of layers.

In the present embodiment, the layer windows are transparent and all the overlapping layer windows are displayed. However, by making the layer windows of interest to the user semitransparent, only the layer windows of interest are displayed. It is highlighted and easy to see. [Seventh Embodiment] Next, a multi-window display device according to a seventh embodiment will be described.

The present apparatus is characterized in that the result of the keyword search is separated from the original window as a new layer window and is displayed in a floating manner. (Structure) FIG. 23 is a block diagram showing the structure of the multi-window display device according to the seventh embodiment.

The present apparatus further includes a keyword search unit 601, a search result output unit 602, and a layer window generation unit 603 in addition to the constituent elements 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. Keyword search unit 60
1 retrieves a keyword from the document displayed in the window. Specifically, the code data of the document stored in the storage unit 102 is searched for a character string that matches the keyword specified by the input unit 103, and if found, the window and the searched position and keyword. To the layer window generation unit 603.

The layer window generation unit 603 generates a new layer window in which only the searched keyword is arranged at the searched position based on the notification from the keyword search unit 601. Specifically, the image data in which only the character string retrieved from the image data of the window notified from the keyword retrieval unit 601 remains is stored as the image data of the new layer window in the storage unit 10.
Store in 2.

The search result output unit 602 notifies the three-dimensional position calculation unit 104 of the layout position of the layer window generated by the layer window generation unit 603. Specifically, the positional relationship between the original window to be searched and the generated new layer window is set to be the same as the positional relationship between the base layer 531 and the upper layer 532 in the sixth embodiment. Specify the position of the layer window. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, window code data and image data are generated (step S1 in FIG. 4).
21) and the calculation of the next three-dimensional arrangement position (step S in FIG. 4).
122) and the processing for the layer window has been added. Therefore, the added processing will be described here.

FIG. 24 is a flowchart showing the operation procedure of the keyword search unit 601, the search result output unit 602, and the layer window generation unit 603. When a keyword is input by the user via the input unit 103 (step S620), the keyword search unit 601 causes the storage unit 102 to operate.
A character string that matches the keyword is searched from the code data of all the documents stored in (step S62).
1).

If retrieved (step S62)
2) The keyword search unit 601 obtains the position searched for each window and the keyword from the layer window generation unit 6
Report to 03. According to the report, the layer window generation unit 603 generates the image data of the layer window including only the searched character string and stores it in the storage unit 102 (step S623).

Then, the search result output unit 602 sets the window generated by the layer window generation unit 603 so that the window is raised from the original window and displayed.
The placement position in the dimensional space is calculated by the three-dimensional position calculation unit 104 (step S624). Therefore, the three-dimensional position calculation unit 104 calculates the arrangement position of the original window that is the search target as specified by the program execution unit 101 or the input unit 103, and is generated by the layer window generation unit 603. With respect to the layer window, an arrangement position parallel to the original window and separated by a predetermined distance is calculated based on the designation from the search result output unit 602 (step S122 in FIG. 4).

FIG. 25 is a diagram showing an example of the above procedure and the final screen display. Original window 63 to be searched
If the keyword “window” is included in 0, the layer window generation unit 603 generates a new layer window 631 and the original window 630 is generated.
It is shown that it is raised and displayed. (Summary) As described above, since the original window and the window showing the search result of the keyword are tilted in the depth direction and separated from each other by this device, the search result can be grasped at a glance. . That is, in the conventional front facing process, it is necessary to perform processing such as expressing the search result in a different color, but in the present apparatus, such special processing is unnecessary.

It is easy to make a device corresponding to a plurality of keywords by displaying different keywords on different layers or coloring each layer semi-transparently. [Embodiment 8] Next, a multi-window display device according to Embodiment 8 will be described.

The present device is characterized in that a plurality of related windows are connected and displayed while being inclined in the depth direction. (Structure) FIG. 26 is a block diagram showing the structure of the multi-window display device according to the eighth embodiment.

The present apparatus further includes an inter-window connection unit 701, a window relative position storage unit 702, and a window relative position changing unit 703 in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. The inter-window connection unit 701 identifies a group of related windows from the plurality of windows that are currently scattered and displayed. Here, the related window refers to a window generated by the same application program in this embodiment. For example, these windows correspond to a case where a plurality of pages of the same document are simultaneously displayed in a plurality of windows by an application program such as a word processor.

Specifically, the inter-window connection unit 701
Based on the notification from the program execution unit 101 executing the application program, the storage unit 102
The window relative position storage unit 702 identifies the windows generated by the same application program from among the plurality of windows stored in the above, and orders them by analyzing the code data of each window.
To notify. For example, the page numbers are ordered in ascending order.

The window relative position storage unit 702 calculates the arrangement position of each window in the three-dimensional space when the group of windows specified and ordered by the inter-window connection unit 701 is connected in the horizontal direction so as to be bent in a zigzag manner. Connection information for creating and storing the connection information. Specifically, the arrangement position of the leading window tilted in the depth direction and the relative coordinates between sequentially connected windows are stored. The initial arrangement position is predetermined, and the inclination of each window is, for example, in a state in which the front side is the rotation axis and the opposite side is rotated by 45 degrees in the depth direction.

The window relative position changing unit 703 removes from the screen display by folding only a specific window from a series of windows displayed connected in zigzag. Specifically, the storage content of the window relative position storage unit 702 is changed so that the window specified by the input unit 103 is bypassed and connected. (Operation) Next, the operation of the multi-window display device of this embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, window code data and image data are generated (step S1 in FIG. 4).
21) and the calculation of the next three-dimensional arrangement position (step S in FIG. 4).
122) is added to the processing for connecting the window. Therefore, the added processing will be described here.

FIG. 27 is a flow chart showing the operation procedure of the inter-window connection unit 701, window relative position storage unit 702 and window relative position changing unit 703.
Now, the program execution unit 101 uses pages 1 to 1 of the same document based on the execution of the same application program.
It is assumed that the code data and the image data of the five windows that respectively display page 5 are stored in the storage unit 102, and the inter-window connection unit 701 is notified of this (FIG. 4).
Step S121).

The inter-window connection unit 701 specifies the five windows stored in the storage unit 102 based on the notification, and refers to their code data to store the window relative position in ascending order of page numbers. The unit 702 is notified (step S720). The window relative position storage unit 702 generates and stores initial connection information regarding the windows notified by the inter-window connection unit 701 so that the windows are connected in a zigzag manner in the notified order (step S721).

In FIG. 28 (a), the initial connection information is not changed (step S722), and the placement positions of the windows of page 1 to page 5 are determined by the three-dimensional position calculation unit 104 based on the connection information. The screen display when it is calculated and displayed (steps S122 to S125 in FIG. 4) is shown. The five windows are connected to each other with one side in common, and are alternately displayed in different rows in a horizontal row.

If the input unit 103 notifies the window relative position changing unit 703 of an instruction to fold a specific window (step S722),
The window relative position changing unit 703 changes the storage content (connection information) of the window relative position storage unit 702 so as to bypass the window and connect the subsequent window (step S723).

In FIG. 28B, the connection information is changed so that the window of page 3 is folded (step S72).
2, S723), and a screen when the layout positions of the windows of the subsequent pages 4 and 5 are newly calculated and displayed by the three-dimensional position calculation unit 104 based on the connection information (steps S122 to S125 in FIG. 4). An example of the display is shown.

Page 4 and page 5 are changed to be connected after page 2 while maintaining the relative position of page 4 with respect to page 3. (Summary) In this way, by connecting and displaying the sides of related page windows by tilting them by this device, the relationship between windows can be grasped at a glance by the connection state between windows. Becomes

An application program such as a word processor is often composed of a plurality of pages.
In this case, it is difficult to display a plurality of pages at the same time in the conventional window system in which the window is directed toward the front, and there is no choice but to overlap each page or display several pages. According to this device, many pages are arranged and displayed in a list without overlapping in a limited display area.

Further, according to a user's input, a window to be displayed and a window which need not be displayed are designated, and windows which do not need to be displayed are arranged in a folded position so as to overlap with other windows. By storing it, it becomes possible to keep only a plurality of windows of interest at the same time while maintaining the relation between windows such as continuous pages.

For windows arranged at positions folded so as to overlap other windows, the texture mapping unit 105 does not paste the window bitmaps, so that the texture mapping process is partially omitted. It becomes possible. Further, in the present embodiment, the windows generated by the same application program are related windows, but pages of another document belonging to the same project are often strongly related to each other, and different application programs are generated. Even in the case of the window to be displayed, the related windows may be clearly displayed in advance to connect and display them.

Further, a means for editing, such as copying or moving an object, may be provided across the windows displayed by being connected. 29 (a) and 29 (b),
It is an example of a screen display in the case where means for operating an object is provided across windows that are page-connected. FIG. 29A shows the screen display when the windows of page 1 to page 5 are displayed based on the initial connection information, and is the same drawing as FIG. 28A.

FIG. 29B shows a state in which the objects of page 1 are copied to page 4 after pages 1 and 4 are displayed face-to-face by folding page 2 and page 3. There is. This is the input unit 10
This can be realized by overwriting the page 5 code data and image data stored in the storage unit 102 on the page 5 so that the object of the copy source window designated by 3 is copied to the copy destination window.

[0115]

As is apparent from the above description, according to the multi-window display device of the present invention, it is a multi-window display device for displaying a plurality of windows on the screen, the window showing the display contents of the window. From the window data holding means that holds the data and the operator
Input means for obtaining the instruction of the
Based on the window data held by the holding means, the window
Of the display contents of the window, the part that satisfies certain conditions is important
And window analyzing means for detecting as a point, and determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window, the
The placement position is determined according to the instruction obtained by the input means.
And the three-dimensional sky of each vertex that specifies the outline of the window
The coordinates between the positions are determined as the arrangement position, and the analysis is performed.
The place detected by the means should be displayed in front
Placement position determining means for determining the placement position of the window ,
A perspective projection unit that transforms the window data so as to perspective-project the window on the screen based on the determined arrangement position; and a display unit that displays the transformed window data on the screen. To do.

As a result, at least one window is displayed tilted in the depth direction, so that the size of the entire window is reduced and the displayed contents cannot be determined when the window is arranged in the back of the three-dimensional space. Is avoided. In addition, since the display area can be saved in the depth part of the window tilt while keeping the information of the part in front of the window tilt visible, at least part of the display content of the window can be displayed while the screen display is maintained. A multi-window display device that effectively utilizes the area is realized.

[0117] By this, the operator is able to determine the degree of slope to the overall size and depth direction window.

[0118] By this, for example, windows in the case of square-planar, are identified position in the three-dimensional space of the window only by the coordinates of the four vertices
It

As a result, the window is displayed so as to be tilted in the depth direction so that the important part of the display content is in the foreground, so that the operator can display a plurality of windows even if they are displayed. It is possible to identify the type and display contents of each window at a glance. Further, the window analysis means may determine whether or not the window data includes a title of the window, and if the window data is included, a portion where the window title is displayed may be detected as the important portion.

As a result, it becomes easy to understand the type of window. Further, the window analysis means determines whether or not the window data includes a document and an instruction to display the document horizontally, and if so, the left side of the window is detected as the important portion. You can also This makes it easy to understand the type and outline of the horizontally written document displayed in the window.

Further, the window analysis means determines whether or not the window data includes a document and an instruction to display the document vertically, and if so, the right side of the window is located at the important portion. Can be detected as This makes it easy to understand the type and outline of the vertically written document displayed in the window.

Also, multiple windows may be displayed on the screen.
A multi-window display device comprising:
That holds the window data that represents the display contents of the
(C) Data holding means and input for obtaining instructions from the operator
Means and a tertiary for at least one of said windows
Virtual placement position that is not parallel to the screen in the original space
Position and determine the location according to the instruction obtained by the input means.
Each to determine the layout position and specify the outline of the window
Determine the coordinates of the vertex in the three-dimensional space as the placement position.
To determine the arrangement position determining means and the determined arrangement position
To perspective project the window onto the screen based on
Perspective projection means for converting the window data, and conversion
Means for displaying the displayed window data on the screen
And the arrangement position determining means includes a plurality of windows.
A predetermined alignment position for at least one
Make sure that the windows are not in parallel with the screen above.
Including an automatic alignment section for determining the position of the window
is doing. As a result, the plurality of windows that are scattered and displayed on the screen are aligned and displayed, so that the display area of the screen can be effectively used and all the windows that are currently displayed can be glanced.

Further, the arrangement position determining means further determines means for determining a new arrangement position so that the window displayed at the aligned position is displayed in the front direction in accordance with the instruction acquired by the input means, and in the front direction. It is also possible to include means for deciding a new arrangement position so as to return the displayed window to the aligned position. This facilitates the operation of directing only the window to be worked to the front and returning it to the original alignment position when the work is completed.

In addition, multiple windows are displayed on the screen.
A multi-window display device comprising:
That holds the window data that represents the display contents of the
(C) Data holding means and input for obtaining instructions from the operator
Means and a tertiary for at least one of said windows
Virtual placement position that is not parallel to the screen in the original space
Position and determine the location according to the instruction obtained by the input means.
Each to determine the layout position and specify the outline of the window
Determine the coordinates of the vertex in the three-dimensional space as the placement position.
To determine the arrangement position determining means and the determined arrangement position
To perspective project the window onto the screen based on
Perspective projection means for converting the window data, and conversion
Means for displaying the displayed window data on the screen
And the placement position determined by the placement position determining means.
Window information consisting of a title bar and a menu bar
Save the window data so that the report image is displayed in the foreground.
Window for editing window data held by holding means
(C) Information image position changing means is provided.

As a result, the title bar and the menu bar are displayed in the foreground even when the window is displayed in a tilted manner, which makes it easy to understand the type of window and operate the window. Further, the multi-window display device further controls the arrangement position determining means so that the window information image of the window on which the window information image is displayed is facing forward and the remaining window body portion is in an arrangement position not parallel to the screen. A window information image direction changing means may be provided.

As a result, the title bar and menu bar are displayed at the front position and in the front direction even when the window is displayed tilted.
It is easy to understand the type of window and operate the window. Also, a multi-window that displays multiple windows on the screen.
A window display device, and the display contents of the window
Window data storage that holds the window data that represents
Holding means and input means for obtaining instructions from the operator,
In at least one of the three windows
Determine a virtual placement position that is not parallel to the above screen
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
An arrangement that determines the coordinates in the dimensional space as the arrangement position.
Based on the placement position determining means and the determined placement position
The window is projected to project the window onto the screen.
Perspective projection means for converting the window data and the converted window
Display means for displaying window data on the screen;
The window data converted by the perspective projection means
The characters are displayed smaller than a predetermined size and cannot be identified.
Unidentifiable area determination means for determining whether an area occurs
And if it is determined that there is an unidentifiable area,
Said that area is not visible in the window
Scroll button that determines the position of the scroll button when
Button position determining means and scroll button at the determined position.
Add a scroll bar image with a window to the window
Scroll bar to edit the window data
-Additional means will be provided.

With this, even when there is an area in which the character or the like cannot be identified because the window is tilted in the depth direction and displayed, the character or the like can be slid to the identifiable display area. Also, multiple screens
It is a multi-window display device that displays windows.
Window data representing the display contents of the window
Window data holding means for holding the
Input means for obtaining instructions and at least the window
One of them is parallel to the screen in 3D space
No virtual placement position determined, the input means acquired
The placement position is determined according to the instructions given in
The coordinates in the three-dimensional space of each vertex that specifies the outline of
Placement position determining means for determining the placement position, and
The window based on the arranged position
To transform the window data into a perspective projection onto
The visual projection means and the converted window data are displayed on the screen.
Means for displaying on the screen, and the window data holding means
Window with layer structure held in the window
C) Data belonging to upper layers excluding the lowest layer from the data
Layer separation means for separating window data for each layer
And the window of the layer window corresponding to the separated layer.
Layer window generation means for generating window data
And the arrangement position determining means includes the layer window.
C is parallel to the lowest layer and has a predetermined position
A feature of determining the arrangement positions separated by a distance.
Multi-window display device.

As a result, each layer window is displayed as if it is three-dimensionally raised, so that the objects placed on each layer can be grasped at a glance, and the objects on each layer or across multiple layers can be grasped. Editing becomes easy. Also, display multiple windows on the screen
A multi-window display device for
A window that holds the window data that represents the displayed contents
Dow data storage means and input to acquire instructions from the operator
3 for force means and at least one of said windows
Virtual arrangement not parallel to the screen in dimensional space
Determine the position and follow the instructions obtained by the input means
Determine the layout position and specify the outline of the window
The coordinates of the respective vertices in the three-dimensional space are used as the arrangement positions.
Arrangement position determining means for determining, and the determined arrangement position
Perspective projection of the window to the screen based on
A perspective projection means for converting the window data,
A display that displays the changed window data on the screen.
And the window held by the window data holding means.
The key acquired by the input method for searching the window data
Keyword search method to search for a part that matches a word
And a new record with keywords placed in the searched
Generate window data for the ear window and
Layer window generation to store in window data holding means
Means for arranging the arrangement position,
The window that was searched as the location of the window
It is possible to determine the placement position that is parallel to
I am going to.

As a result, the window showing the search result is displayed as if it were raised from the original window that was the search target, so that the position of the searched keyword can be confirmed simply by focusing on the layer window. . It also allows you to display multiple windows on the screen.
A window display device for displaying the window
Window data that holds the window data that represents the contents
Data holding means and input means for obtaining instructions from the operator
And a 3D sky for at least one of the windows
The virtual layout position that is not parallel to the screen
Determine and arrange according to the instructions obtained by the input means
Each vertex that determines the position and identifies the outline of the window
Determine the coordinates in the three-dimensional space as the placement position
Based on the determined placement position.
To project the window onto the screen in perspective.
Is a perspective projection means to transform window data,
Display means for displaying the window data on the screen,
A related group from among the displayed windows
Related window identification means for identifying the
Bending zigzags through a set of defined windows
Generate connection information to attach frames to
Connection information storage means for storing
The setting means sets a group of windows based on the connection information.
The placement position is decided.

Thus, when a plurality of pages in a document are displayed in different windows, the related windows are connected and displayed, so that windows related to the same document can be easily specified. Further, the multi-window display device further comprises connection information changing means for changing the connection information so as to arrange the windows excluding some windows from the group of windows according to the instruction acquired by the input means. The arrangement position determining means may determine the arrangement position of a group of windows excluding the part of the windows based on the changed connection information.

As a result, it is possible to display only the windows of interest among the windows that are connected and displayed. Further, the multi-window display device further edits the window data held in the window data holding means so as to edit the display content across one and the other windows of the group according to the instruction acquired by the input means. It is also possible to provide an inter-window editing means for editing.

As a result, only the window of interest can be displayed close to each other among the windows that are connected and displayed, and the object can be edited over two windows. Therefore, the operability across multiple windows can be improved. Is improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a multi-window display device according to a first embodiment.

2A and 2B are perspective projection units 10. FIG.
6 is a diagram for explaining the concept of perspective projection according to FIG. FIG. 2A is a diagram showing the positional relationship between the projection center, the projection surface, and the object (window). FIG. 2B shows a perspective view when the object is a three-dimensional object.

FIG. 3A to FIG. 3C are perspective projection units 106.
FIG. 6 is a diagram for explaining the specific processing content of perspective projection by. FIG. 3A shows a case where a window virtually placed in the three-dimensional space is parallel to the display screen.
It is a figure which shows the user's viewpoint, the display screen, the positional relationship of a window, and the shape of the window by perspective projection. FIG. 3B shows a case where the window virtually placed in the three-dimensional space is not parallel to the display screen.
It is a figure which shows the user's viewpoint, the display screen, the positional relationship of a window, and the shape of the window by perspective projection. FIG. 3C is a diagram illustrating a specific calculation process of perspective projection.

FIG. 4 is a flowchart showing an operation procedure of the multi-window display device according to the first embodiment.

FIG. 5 is a window 130 arranged in parallel with the screen,
131 and the window 13 arranged in a non-parallel position
It is a figure which shows the example of the screen display of 2-133.

FIG. 6 is a block diagram showing a configuration of a multi-window display device according to a second embodiment.

FIG. 7 is a flowchart showing a procedure when the window analysis unit 201 identifies an important part of a window.

FIG. 8 shows an example of a screen display when an important part is specified by the window analysis unit 201.

FIG. 9 is a block diagram showing a configuration of a multi-window display device according to a third embodiment.

FIG. 10 is a flowchart showing a procedure of automatic alignment by the automatic alignment unit 104a.

FIG. 11A shows an example in the case of the first non-overlapping alignment method, and FIG. 11B shows an example in the case of the non-overlapping second alignment method.

12A is a screen display immediately after automatic alignment, and FIG. 12B is a screen display of FIG.
The screen display immediately after one window 241 is selected by 03. In FIG. 12C, the window arrangement button 245 provided on the right end of the title bar of the window 241 by the input unit 103 in the screen display of FIG.
An example of the screen display immediately after is pressed is shown.

FIG. 13 is a block diagram showing a configuration of a multi-window display device according to a fourth embodiment.

FIG. 14 is a flowchart showing an operation procedure of a window information image position changing unit 301 and a window information image direction changing unit 302.

FIG. 15 shows an example of screen display of a window 330 before the position of the window information image is changed and a window 331 after the position of the window information image is changed.

FIG. 16 shows an example of a screen display of a window 332 before the orientation of the window information image is changed and a window 333 after the orientation of the window information image is changed to the front.

FIG. 17 is a block diagram showing a configuration of a multi-window display device according to a fifth embodiment.

FIG. 18 is a scroll button position determination unit 401, a scroll bar addition unit 402, and an unidentifiable area determination unit 403.
3 is a flowchart showing the operation procedure of FIG.

FIG. 19A is a diagram showing a general concept of a scroll button 430 and a scroll bar 431. FIG. 19B shows an example of screen display when it is determined that the lower half of the window display area is the unidentifiable area 432.

FIG. 20 is a block diagram showing a configuration of a multi-window display device according to a sixth embodiment.

FIG. 21 is a flowchart showing an operation procedure of a window layer separation unit 501 and a layer window generation unit 502.

FIG. 22A is a diagram conceptually showing the process of layer separation (step S521) when the number of layers is two. FIG. 22B shows an example of a screen display when a window having a layer structure is arranged tilted in the depth direction.

FIG. 23 is a block diagram showing a configuration of a multi-window display device according to a seventh embodiment.

FIG. 24 is a keyword search unit 601 and a search result output unit 6
02 and a layer window generation unit 603 is a flowchart showing an operation procedure.

FIG. 25 is a diagram showing an example of a screen display when the result of keyword search is displayed as a layer window.

FIG. 26 is a block diagram showing a configuration of a multi-window display device according to an eighth embodiment.

FIG. 27 is an inter-window connection unit 701, window relative position storage unit 702, and window relative position change unit 703.
3 is a flowchart showing the operation procedure of FIG.

FIG. 28 (a) shows an example of a screen display in which the windows of page 1 to page 5 are displayed based on the initial connection information. FIG. 28B shows an example of a screen display when the connection information is changed so that the window of page 3 is folded.

FIG. 29 is an example of a screen display when a means for operating an object is provided across windows that are page-connected. FIG. 29A shows the screen display when the windows of page 1 to page 5 are displayed based on the initial connection information, and is the same drawing as FIG. 28A. FIG. 29
(B) is a diagram showing a state in which the object of page 1 is copied to page 4 after displaying page 1 and page 4 face to face by folding page 2 and page 3.

FIG. 30 is a diagram showing an example of a multi-window screen display according to a conventional technique.

[Explanation of symbols]

101 Program execution unit 102 storage 103 input section 104 3D position calculator 104a Automatic alignment section 105 texture mapping section 106 perspective projection unit 107 frame memory section 108 image display section 201 Window analysis part 301 Window information image position change unit 302 Window information image direction changing unit 401 Scroll button position determination unit 402 Scroll bar addition section 403 Unidentifiable area determination unit 501 window layer separation unit 502 Layer window generator 601 Keyword search section 602 Search result output section 603 Layer window generator 701 Connection between windows 702 window relative position storage unit 703 Window relative position change unit

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-10-283158 (JP, A) JP-A-6-266330 (JP, A) JP-A-6-222899 (JP, A) US Patent 5621906 (US , A) US Patent 5678015 (US, A) US Patent 5754809 (US, A) US Patent 5880733 (US, A) US Patent 5977973 (US, A) (58) Fields investigated (Int. Cl. ⁷ , DB name ) G06F 3/00 655 G06F 3/14 350 G09G 5/14

Claims (14)

(57) [Claims] 1. A multi-window display device for displaying a plurality of windows on a screen, comprising: window data holding means for holding window data representing display contents of the window; and input means for obtaining an instruction from an operator. , The window held by the window data holding means
Certain of the displayed contents of the window based on the data
A window that detects points that meet the conditions as important points
(C) Analyzing means and a virtual arrangement position in at least one of the windows that is not parallel to the screen in the three-dimensional space
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Determine the coordinates in the dimensional space as the placement position,
The location detected by the analysis means will be displayed in front.
An arrangement position determining means for determining an arrangement position of the window, a perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position, and the converted window data Is provided on the screen, and a multi-window display device is provided. 2. The window analysis means determines whether or not the window data includes a title of the window, and if the window data is included, detects a portion where the window title is displayed as the important portion. multi-window display apparatus according to claim 1, wherein. 3. The window analysis means determines whether or not the window data includes a document and an instruction to display the document in horizontal writing, and when it includes, the left side of the window is the important portion. The multi-window display device according to claim 1, wherein: 4. The window analysis means determines whether or not the window data includes a document and an instruction to display the document in a vertical writing mode. multi-window display apparatus according to claim 1, wherein the detecting the location. 5. A multi-window display device for displaying a plurality of windows on a screen, comprising window data holding means for holding window data representing display contents of the window, and input means for obtaining an instruction from an operator. , determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Arrangement position determining means for determining coordinates in a dimensional space as the arrangement position, and perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position. When, and display means for displaying the converted window data on the screen, the position determination means, pre plurality of windows
At least one window with a defined alignment position
Of each window so that it is not parallel to the screen
It is characterized by including an automatic aligning unit that determines a placement position.
Multi-window display device. 6. The arrangement position deciding means further comprises means for deciding a new arrangement position so that the window displayed at the aligned position is displayed in the front direction according to the instruction acquired by the input means, 6. The multi-window display device according to claim 5 , further comprising means for determining a new arrangement position so as to return the displayed window to the aligned position. 7. A multi-window display device for displaying a plurality of windows on a screen, comprising window data holding means for holding window data representing display contents of the window, and input means for obtaining an instruction from an operator. , determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Arrangement position determining means for determining coordinates in a dimensional space as the arrangement position, and perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position. When a display unit that displays the converted window data on the screen, data on the basis of the position where the position determination means has determined
Window information screen consisting of iter bar and menu bar
The window data holding hand so that the image is displayed in front
Window information that is held in columns and edits window data
And a multi-image position changing means.
Window display device. 8. The multi-window display device further determines the arrangement position so that the window information image of a window on which the window information image is displayed in the front is oriented in the front direction and the remaining window body portion is in an arrangement position not parallel to the screen. 8. The multi-window display device according to claim 7, further comprising window information image direction changing means for controlling the means. 9. A multi-window display device for displaying a plurality of windows on a screen, comprising window data holding means for holding window data representing display contents of the window, and input means for obtaining an instruction from an operator. , determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Arrangement position determining means for determining coordinates in a dimensional space as the arrangement position, and perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position. When a display unit that displays the converted window data on the screen, the window data converted by the perspective projection means
Of characters that are smaller than the specified size
Judgment whether or not a feasible area occurs
Method, and if it is determined that there is an unidentifiable area,
Area is not displayed in the window
Scroll button position that determines the position of the scroll button
Position determination means and scroll with scroll buttons placed at the determined position
Window to add the image of the louver to the window
C) A scroll bar adding means for editing data is provided.
A multi-window display device characterized by the above. 10. A multi-window display device for displaying a plurality of windows on a screen, comprising: window data holding means for holding window data representing display contents of the window; and input means for obtaining an instruction from an operator. , determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Arrangement position determining means for determining coordinates in a dimensional space as the arrangement position, and perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position. When the converted display means for displaying the window data to the screens, the window data holding means layer is held to the structure
From the window data of the window with
Layer the window data belonging to the upper layer excluding
Layer separation means for separating each layer and the window of the layer window corresponding to the separated layer
Equipped with layer window generation means for generating dough data
The arrangement position determining means is arranged to arrange the layer window.
Position is parallel to the lowest layer and only a predetermined distance
Multi-wiring characterized by deciding the disposition position
Display device. 11. A multi-window display device for displaying a plurality of windows on a screen, comprising window data holding means for holding window data representing display contents of the window, and input means for obtaining an instruction from an operator. , determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Arrangement position determining means for determining coordinates in a dimensional space as the arrangement position, and perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position. When a display unit that displays the converted window data on the screen, held in the window data holding means window
Keywords obtained by the input means for data retrieval
A keyword search method that searches for a part that matches with, and a new layer in which the keyword is placed in the searched part
Generate the window data of the window and
(C) Layer window generation means for storing in data holding means
And the arrangement position determining means arranges the arrangement of the layer window.
The location is parallel to the searched window and
It is characterized by deciding the placement positions separated by a fixed distance
Multi-window display device. 12. A multi-window display device for displaying a plurality of windows on a screen, comprising window data holding means for holding window data representing display contents of the window, and input means for obtaining an instruction from an operator. , determining at least one virtual position which is not parallel with the screen in 3-dimensional space for the window
Then, according to the instruction obtained by the input means, the arrangement position
3 of each vertex that determines the outline of the window
Arrangement position determining means for determining coordinates in a dimensional space as the arrangement position, and perspective projection means for converting the window data so as to perspectively project the window on the screen based on the determined arrangement position. A display means for displaying the converted window data on the screen, and a related group from the plurality of displayed windows.
Related window identification means to identify each window and the group of identified windows will be bent zigzag
Generates connection information to attach frames to each other and arrange them
And a connection information storage unit that stores the information, wherein the arrangement position determination unit is a group based on the connection information.
Is characterized by determining the placement position of the
Multi-window display device. 13. The multi-window display device further changes the connection information so that the connection information is arranged so that the group of windows excluding some of the windows is arranged according to the instruction acquired by the input means. 13. The multi-window display device according to claim 12 , further comprising means, wherein the arrangement position determining means determines arrangement positions of a group of windows excluding the part of the windows based on the changed connection information. . 14. The multi-window display device is further held by the window data holding means so as to edit display contents across one and the other windows of the group according to an instruction acquired by the input means. 14. The multi-window display device according to claim 13, further comprising inter-window editing means for editing window data.