JPH1165806A - Multiwindow display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase the storage efficiency of windows and to eliminate the trouble that display contents can not be all decided by arranging at least one window obliquely in its depth direction and see-through projecting and displaying it on a screen by transmission. SOLUTION: A three-dimensional position calculation part 104 calculates and stores the arrangement position (coordinates of four vertexes of each window) of one window stored in a storage part 102 in a three-dimensional space according to information from a program execution part 101 or input part 103. A texture mapping part 105 reads the image data of the window out of the storage part 102 and generates texture data. A see-through projection part 106 applies the see-through projection to the generated texture data and overwrites the obtained transmission data at corresponding positions in a frame memory part 107. Consequently, when the arrangement position of the window is not parallel to the display screen, it is displayed as a trapezoidal window.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-window display device for displaying a plurality of windows on a screen of an information device, and more particularly to a device for displaying windows as if they were placed in a virtual three-dimensional space.

[0002]

2. Description of the Related Art In an information terminal such as a computer, a multi-window display device is used to improve operability of a man-machine interface. As a conventional multi-window display device which efficiently displays as many windows as possible on a screen having a limited area, Japanese Patent Laid-Open No. 6-2228 discloses a conventional multi-window display device.
99 has a multi-window display system.

FIG. 30 is a view showing a screen display according to the above-mentioned conventional technique. This conventional technology arranges windows in a virtual three-dimensional space and displays the windows placed at the back of the screen in a reduced size, thereby increasing the storage efficiency of the windows and making the screen feel wider psychologically. It is.

[0004]

However, in the above-mentioned prior art, the window placed at the back of the screen has a problem that the entire display content of the window cannot be determined because the entire window is displayed in a reduced size. is there. Accordingly, the present invention has been made in view of such a problem, and is an apparatus for displaying a window as if it were placed in a virtual three-dimensional space. It is an object of the present invention to provide a multi-window display device that makes the user feel it and is less likely to have a problem that all display contents cannot be determined.

[0005]

To achieve the above object, the present invention provides a multi-window display device for displaying a plurality of windows on a screen, the window data holding window data representing the display contents of the windows. Holding means, arrangement position determining means for determining a virtual arrangement position not parallel to the screen in at least one of the windows in at least one of the windows, and placing the window on the screen based on the determined arrangement position. It is characterized by comprising a perspective projection unit for converting the window data so as to perform perspective projection, and a display unit for displaying the converted window data on the screen.

Thus, at least one window is arranged in a three-dimensional space so as to be inclined in the depth direction,
Conventionally, all the displayed contents cannot be determined because the size of the entire window is reduced evenly when the window is placed in the back of the three-dimensional space because it is projected and displayed on the screen. In addition to avoiding technical inconvenience, the display area of the screen is effectively utilized, and the screen can be made to feel wide psychologically.

[0007]

Embodiments of the present invention will be described below in detail with reference to the drawings. [First Embodiment] First, a multi-window display device according to a first embodiment will be described.

This apparatus is characterized in that a window is displayed in a three-dimensional space while being inclined in the depth direction. Note that the shape of the window itself is a rectangular plane whose outer shape is specified by four vertices. (Configuration) FIG. 1 is a block diagram showing the configuration of the multi-window display device according to the first embodiment.

This apparatus comprises a program execution unit 101, a storage unit 102, an input unit 103, a three-dimensional position calculation unit 104, a texture mapping unit 105, a perspective projection unit 106, a frame memory unit 107, and an image display unit 108. . The program execution unit 101 is a CPU or the like that starts and executes an application program that displays a normal (two-dimensional) window and interacts with a user. Specifically, information on each window, that is, a window display target ( The storage unit 102 stores code data (documents, characters, graphics, and the like) representing the object) and image data representing the object as a bit image, and exchanges information about the window with the other components 102 to 108.

The image data is image data that does not depend on the size of the entire window in which each object is placed or the position of the window itself in a three-dimensional space, and is represented by a predetermined fixed size. Data. The storage unit 102 is a memory that stores the code data and the image data in association with each window of each application program. Note that all windows stored in the storage unit 102 are not necessarily displayed on the image display unit 108, but windows displayed on the image display unit 108 are always stored on the image display unit 108. Limited to

The input unit 103 is a pointing device such as a mouse, and acquires an instruction from a user. The three-dimensional position calculation unit 104, in accordance with a notification from the program execution unit 101 or the input unit 103, arranges one window stored in the storage unit 102 in the three-dimensional space (the coordinates of four vertices of each window). Is calculated and stored. Specifically, when the notification of the coordinates of the four vertices is received from the program execution unit 101 as the initial position, the coordinates are stored as they are,
When the movement displacement (movement type and movement amount) in the dimensional space is notified, new vertex coordinates are calculated by affine transformation.

When the three-dimensional position calculating unit 104 does not receive the explicit designation of the window arrangement position from the program execution unit 101 or the input unit 103, the three-dimensional position calculation unit 104 determines the default arrangement position, for example, the left side of the window. An arrangement position in which the right side is inclined by 45 degrees in the depth direction of the screen as a rotation axis is generated and stored. Texture mapping unit 105
When a new arrangement position is calculated by the three-dimensional position calculation unit 104, the image data of the window is read out from the storage unit 102, and is expanded or reduced to a window size corresponding to the arrangement position, and at the same time, the program execution unit 101 Then, texture mapping is performed based on the instruction from the CPU, and the obtained texture data is sent to the perspective projection unit 106. Note that this texture data is stored in window 3
This data is determined depending only on the arrangement position in the dimensional space, and does not depend on the viewpoint position at which the window is viewed.

The perspective projection unit 106 performs perspective projection on the texture data sent from the perspective projection unit 106,
The obtained perspective data is stored in the frame memory unit 107. Here, the “perspective projection” means that an object (window) placed in the three-dimensional space is a projection center with a viewpoint placed in front of a display surface of the image display unit 108 and the two-dimensional display surface is a projection surface. Means an operation for generating a perspective view. In this specification, “front” and “back”
Is an expression based on the distance from the viewpoint of the user located in front of the display surface.

FIGS. 2A and 2B are views for explaining the concept of perspective projection by the perspective projection unit 106. FIG. FIG. 2A is a diagram illustrating a positional relationship among a projection center, a projection plane, and an object (window). Perspective projection unit 106
Puts a display screen of the image display unit 108 between a window virtually placed in a three-dimensional space and a user's viewpoint, and irradiates light converging on the viewpoint to the window to perspectively project an image projected on the display screen. Created window is created.

FIG. 2B is a perspective view when the object is a three-dimensional object. In this case, a point (a vanishing point) occurs at the rear of the three-dimensional object as if the parallel lines crossed on a horizontal line at infinity. The perspective projection unit 106 stores the coordinates of 256 vanishing points placed at predetermined positions in a three-dimensional space in order to perform perspective projection by multipoint perspective instead of single point perspective.

FIGS. 3A to 3C show the perspective projection unit 1. FIG.
FIG. 16 is a diagram for explaining specific processing contents of perspective projection by No. 06. FIG. 3A is a diagram showing the user's viewpoint, the positional relationship between the display screen and the window, and the shape of the perspectively projected window when the window virtually placed in the three-dimensional space is parallel to the display screen. is there.

In this case, the perspectively projected window has a rectangular shape. FIG. 3B shows the viewpoint and display of the user when the window virtually placed in the three-dimensional space is not parallel to the display screen, that is, when the window is placed at an angle in the depth direction of the display screen. It is a figure which shows the positional relationship of a screen and a window, and the shape of the window projected and projected.

In this case, the perspectively projected window has a trapezoidal shape. FIG. 3C is a diagram illustrating a specific calculation process of perspective projection. The display screen is an xy plane, the depth direction is the z axis, and a line segment parallel to the z axis
A case where A0B0 is perspectively projected is shown. Perspective projection unit 1
06 calculates a line segment A1B1 for the line segment A0B0 by first performing a perspective transformation based on one vanishing point C specified by the program execution unit 101 or the input unit 103 from the above 256 vanishing points. Then, a final line segment A2B2 is calculated by a projection transformation based on parallel projection onto the xy plane.

The frame memory unit 107 includes the image display unit 1
08 is a video RAM for storing one frame of image data to be displayed on the screen provided in the image RAM 08. Image display unit 108
Is provided with an image display control circuit, a CRT, and the like, reads out image data stored in the frame memory
Display on T. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

FIG. 4 is a flowchart showing the operation procedure of the multi-window display device of the present embodiment. FIG.
FIG. 7 is a diagram showing a window displayed on the image display unit 108 by the operation. First, the program execution unit 10
1 starts and executes the application program, and stores the code data and image data of the window generated by the application program in the storage unit 102 (steps S120 and S121).

Next, the three-dimensional position calculation unit 104, in accordance with a notification from the program execution unit 101 or the input unit 103, arranges one window stored in the storage unit 102 in the three-dimensional space (the position of each window). Calculate and store the coordinates of the four vertices (step S12)
2). The coordinate system at this time is as shown in FIG.

Subsequently, the texture mapping unit 105
When a new arrangement position is calculated by the three-dimensional position calculation unit 104, the image data of the window is read out from the storage unit 102, and is read out from the storage unit 102.
Texture data representing one window is generated by performing texture mapping to fit and paste into a window of a size determined by the four vertexes calculated by step 04 (step S123).

Then, the perspective projection unit 106 specifies a vanishing point based on the notification from the program executing unit 101 or the input unit 103 with respect to the texture data generated by the texture mapping unit 105, and performs the perspective using the vanishing point. The projection is performed, and the obtained perspective data is overwritten on the corresponding position in the frame memory unit 107 (step S124). When there is no explicit designation of the vanishing point from the program execution unit 101 or the input unit 103, the perspective projection unit 106 determines that the x coordinate and the y coordinate are the same as the center of each window, and the z coordinate is Perspective projection is performed using a vanishing point having a value equal to the width of the display screen.

Finally, the image display unit 108 reads out the fluoroscopic data (image data) written in the frame memory unit 107 and displays it on the CRT (step S12).
5). Here, the program execution unit 101 or the input unit 10
When the three-dimensional position calculation unit 104 is notified of changing the arrangement position of the window stored in the storage unit 102, the processing in steps S122 to S125 is repeated, and the image display unit 108 is updated. The content displayed in is updated.

As described above, the present multi-window display device specifies the coordinates of four vertices of a window virtually placed in a three-dimensional space and perspectively projects the coordinates on a display screen. When the arrangement position in the three-dimensional space is parallel to the display screen, it is displayed as rectangular windows 130 and 131 shown in FIG.
If the position of the window is not parallel to the display screen, the trapezoidal windows 132 to 13 shown in FIG.
It is displayed as 4.

That is, the two windows 130 and 131 on the left side of the screen shown in FIG. 5 are examples facing the user, and the window 132 on the right side of the screen is
This is an example in which the upper side of the window is a rotation axis and the lower side is inclined in the depth direction with respect to the user. Two windows 134 and 133 on the lower right side of the screen have the right side in the depth direction with respect to the user with the left side of the window as the rotation axis. This is a tilted example. (Summary) As a result of the window being tilted and displayed by the present apparatus, the user has difficulty in recognizing the contents of characters and the like displayed in the tilted depth part, but the character before the tilted window is displayed. As for the part, it is possible to fully understand the content. As a result, as compared with the case where the window is displayed facing the front, the display area of the screen in the depth portion can be reduced by an amount corresponding to the inclination, and the display area of the screen is effectively used.

Particularly, in a portable information terminal such as a notebook computer, a PDA (Personal Digital Assistants), and an electronic organizer used for portable applications, the display area of the screen is limited. In such a case, the window is turned to the front, and when the user refers to a plurality of windows when no operation is performed, the windows are tilted and displayed, so that the screen can be effectively used. The invention is also effective when a desktop computer is used for browsing an Internet homepage, or when a plurality of application programs are running.

In this embodiment, the window itself is a rectangular plane, but the present invention is not limited to this shape. For example, it may be a thick window or a polygon. This is because any shape of window can be perspectively projected by tilting the display surface in the depth direction of the screen. In this embodiment, the rectangular window is rotated around one side as an axis, but it may be rotated around two sides in order.

In this embodiment, perspective projection is performed after texture mapping is performed on all display contents of the window. However, this order is reversed, and a window composed of four vertices of a window arranged in a three-dimensional space is obtained. Only the frame is projected onto the two-dimensional display surface by perspective projection in advance, and the image data of the window stored in the storage unit 102 is pasted to the projected window frame while performing linear transformation such as affine transformation. It may be mapped. By changing the order of texture mapping and perspective projection and performing two-dimensional linear conversion during texture mapping,
Although the strictness of the texture is impaired, the calculation load is reduced by the reduced processing area.

The multi-window display device of the present embodiment includes a three-dimensional position calculation unit 104, a texture mapping unit 105, and a perspective projection unit 106 separately from the program execution unit 101. Alternatively, a configuration for executing a part of the processing may be adopted. The storage unit 102 and the frame memory unit 107 may share the same storage device. [Second Embodiment] Next, a multi-window display device according to a second embodiment will be described.

The present apparatus is characterized in that the window is tilted and displayed so that an important part of the display content of the window is in front. (Configuration) FIG. 6 is a block diagram showing the configuration of the multi-window display device according to the second embodiment.

The present apparatus further includes a window analyzing unit 201 in addition to the components 101 to 108 included in the multi-window display device according to the first embodiment. Hereinafter, differences from the first embodiment will be described. Window analysis unit 201
When the application program is started by the program execution unit 101 and the code data and the image data of one window are stored in the storage unit 102,
The code data of the window is read out, an important part is specified from the code data, and the arrangement position of the window is determined so that the part is displayed in the foreground.
Notify 4.

Specifically, the window analysis unit 201
The side of the window closest to the specified important point is set as a rotation axis, and a side opposing the side is notified to be rotated 45 degrees in the depth direction. The “important part” refers to a part of the window that is considered to be noticed by the user in order to determine at a glance the type and display contents of the window, and the specific criterion is a predetermined procedure described later. Obey.

In the present embodiment, the three-dimensional position calculator 104
Is, in addition to the program execution unit 101 and the input unit 103,
A notification about the initial arrangement position or movement displacement of the window is also received from the window analysis unit 201, and a new arrangement position is calculated. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The difference between the operation of this apparatus and the first embodiment is that
In addition to the operations in the first embodiment, generation of window code data and image data (step S1 in FIG. 4)
21) and calculation of the next three-dimensional arrangement position (step S in FIG. 4)
122) is added to the process for specifying an important part of the window. Therefore, here, the added processing will be described.

FIG. 7 is a flowchart showing a procedure when the window analysis unit 201 specifies an important part of a window. FIG. 8 shows an example of a screen display when an important portion is specified by the window analysis unit 201.
When the application program is started by the program execution unit 101 and the code data and image data of one window are stored in the storage unit 102 (step S121 in FIG. 4), the window analysis unit 201 reads the code data of the window. (Step S22
0).

Then, the window analysis unit 201 determines whether or not the code data has a title to be displayed on the title bar of the window (step S221). If there is, the title bar is determined to be an important part of the window. The three-dimensional position calculation unit 104 is notified that the upper side is the rotation axis (step S222, step S12 in FIG. 4).
2). The screen display of the result is as shown in a window 230 shown in FIG. Since the title bar 235 has the title 234, the title bar 235 is displayed so as to be tilted so that the upper side is in front. The title is detected by determining whether or not the code data includes a reserved word for declaring the title of the window.

If there is no title, subsequently, it is determined whether or not the code data includes a horizontally written sentence from left to right (step S2232). If it is included, the left side of the window is determined to be an important point, and the three-dimensional position calculation unit 104 is notified that the left side is to be the rotation axis (step S
224, step S122 in FIG. 4). The screen display of the result is as shown in window 231 shown in FIG.
The text is displayed so that the left side, which is the start of the sentence, is in front.

If a horizontally written sentence is not included, subsequently, it is determined whether or not the code data includes a vertically written sentence that causes a line feed from right to left (step S225). If so, the right side of the window is considered important,
The three-dimensional position calculation unit 104 is notified that the right side is the rotation axis (step S226, step S122 in FIG. 4).
The resulting screen display is shown in window 23 shown in FIG.
3 The text is displayed so that the right side, which is the start of the sentence, is in front. The detection of a horizontally written sentence or a vertically written sentence is determined based on whether or not a reserved word for specifying the style of the sentence is included in the code data.

If no vertically written text is included, it is then determined whether or not the code data includes a drawing (data in a predetermined drawing format) (step S227). If it is included, the drawing is determined to be an important point, the side of the window closest to the layout position of the drawing (the center point of the rectangular area surrounding the drawing) is specified, and the three-dimensional position indicating that the side is used as the rotation axis. The calculation unit 104 is notified (step S228, step S122 in FIG. 4). The resulting screen display is as shown in window 232 shown in FIG. The right side closest to the drawing is tilted and displayed.

If no drawing is included, the three-dimensional position calculation unit 104 indicates that the left side is the rotation axis as a default.
(Step S229, Step S12 in FIG. 4)
2). The screen display of the result is as shown in window 231 shown in FIG. This is because, in general, the content to be focused on is often displayed in the upper left portion of the window. (Summary) As described above, the important point in the window is specified by the present apparatus, and the window is displayed with the window tilted so that the point is in the foreground. Is alleviated.

In the present embodiment, the window analysis unit 201 specifies an important part based on the presence / absence of a title in a window, a sentence style, and the presence / absence of a drawing. However, display information is concentrated in addition to these criteria. A portion or a portion where a reserved word representing the content of a page is placed may be specified as an important portion. In addition, when all the display contents of the window are drawings, the determination is made based on the composition of the drawing. For example, in the case of a window including a drawing including a person, the position of the face of the person is determined based on the distribution of the skin color, and the face is easy to see. If the window is tilted so that it is in the foreground, or if the window is composed of drawings that include mountains, analyze the distribution of green and brown in the mountains and the distribution of blue in the sky. It is also possible to tilt the window so that the region where the green or brown distribution of the mountain is mostly in front is in front. This is because such display information can be easily determined from image data, character codes, and the like before being expanded into image data. Third Embodiment Next, a multi-window display device according to a third embodiment will be described.

The present apparatus is characterized in that a plurality of windows scattered and displayed on a screen are automatically arranged. (Configuration) FIG. 9 is a block diagram showing the configuration of the multi-window display device according to the third embodiment.

The present apparatus is common in that it has the components 101 to 108 provided in the multi-window display apparatus of the first embodiment, but in that the three-dimensional position calculation unit 104 of the apparatus has an automatic alignment unit 104a inside. different. Hereinafter, differences from the first embodiment will be described. The automatic alignment unit 104a
For all windows stored in the storage unit 102, the arrangement positions are aligned by the first non-overlapping alignment method or the overlapping second alignment method.

Specifically, the automatic sorting unit 104a determines that the target window is one for the first sorting method.
The final alignment positions where the windows do not overlap in each of the 16 pieces are stored in advance. on the other hand,
In the second alignment method, the left side of each window is set to the front, the right side is inclined in the depth direction, and the arrangement positions are arranged at equal intervals in the horizontal direction so that the right halves overlap.

When the automatic alignment unit 104a receives an instruction for specifying the alignment method and a notification of the number of windows to be aligned from the three-dimensional position calculation unit 104, the automatic mapping unit 104a sequentially arranges the arrangement positions of the windows in the texture mapping unit. Send to 105. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The operation of this apparatus is different from that of the first embodiment in that
In the calculation of the three-dimensional arrangement position in the first embodiment (step S122 in FIG. 4), the automatic alignment unit 104a calculates a new arrangement position. Therefore, here, a specific procedure of automatic alignment by the automatic alignment unit 104a will be described. FIG. 10 is a flowchart illustrating a procedure of automatic alignment by the automatic alignment unit 104a.

FIGS. 11A and 11B show an example of a screen display when five windows are automatically aligned by the automatic alignment unit 104a, and FIG. 11A shows the first non-overlapping window. FIG. 11
(B) shows an example in the case of the overlapping second alignment method. First, the three-dimensional position calculating unit 104 receives designation from the input unit 103 about the sorting method, reads out and specifies the number n of windows to be sorted from the storage unit 102, and notifies the automatic sorting unit 104a of them ( Step S230).

The automatic sorting unit 104a, having received the notification, determines the sorting method (step S231). As a result, if the first sorting method does not overlap, the automatic sorting unit 104a stores in advance the number corresponding to the notified number n. The arrangement positions of the n windows thus set are sequentially read out (step S232), and notified to the texture mapping unit 105 (step S23).
4). The resulting screen display is as shown in FIG.

On the other hand, in the case of the overlapping second alignment method, the automatic alignment unit 104a overlaps the right halves from the notified number n in a state where the n windows are inclined with the left side in front. The arrangement positions of the n windows are sequentially calculated so as to be arranged at equal intervals in the horizontal direction (step S233), and the texture mapping unit 10
5 (step S234). The resulting screen display is as shown in FIG. (Summary) As described above, since many windows scattered and displayed on the screen by the present apparatus are automatically aligned in a perspectively projected manner, useless areas on the screen are eliminated, and all at a glance. It is possible to grasp the display position and contents of the window.

Here, after the windows are automatically arranged, the arrangement position can be effectively used as a default position of each window. For example, the three-dimensional position calculation unit 104 stores the alignment position of each window obtained by the automatic alignment as a default position, and when one of the automatically aligned windows is designated by the input unit 103, the window is displayed. Is calculated so as to face the front with the side in front of the as the rotation axis, while a specific portion of the window facing the front (for example, a window arrangement button provided in the title bar)
May be added to calculate the arrangement position so that the window is returned to the default position when the windows are automatically arranged when the input unit 103 is designated by the input unit 103.

FIGS. 12A to 12C are diagrams showing examples in which the automatically arranged arrangement positions are effectively used as the default positions of the respective windows. FIG. 12A shows a screen display immediately after the automatic alignment, and corresponds to FIG. 11B. FIG.
FIG. 12B shows the input unit 1 in the screen display of FIG.
03 shows a screen display immediately after one window 241 is selected.

Here, the three-dimensional position calculation unit 104 calculates a new arrangement position of the selected window 241 so that the selected window 241 faces forward with the front side (left side) fixed. FIG. 12C illustrates a screen display immediately after the window arrangement button 245 provided on the right end of the title bar of the window 241 is pressed by the input unit 103 in the screen display of FIG.

Here, the three-dimensional position calculation unit 104 reads the default position already stored for the window for which the window arrangement button 245 has been pressed as a new arrangement position, and sends it to the texture mapping unit 105.
As a result, the window 241 is tilted in the depth direction and returns to the position immediately after the automatic alignment. in this way,
By operating the button, the window can be turned to the front or returned to the automatically aligned position, so that the operability of the window is improved by applying the invention to a portable information terminal having a small display area.

In the present embodiment, the plurality of windows are arranged so that the right halves overlap with each other by the overlapping second arrangement method. However, by further providing the window analyzing unit 201 of the second embodiment, the plurality of windows are arranged. It is easy to deform such that the unimportant portions overlap. [Fourth Embodiment] Next, a multi-window display device according to a fourth embodiment will be described.

This apparatus is characterized in that the positions of the title bar and the menu bar are changed according to the inclination of the window. (Configuration) FIG. 13 is a block diagram showing the configuration of the multi-window display device according to the fourth embodiment.

The present apparatus further includes a window information image position changing unit 301 and a window information image direction changing unit 302 in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. The window information image position change unit 301 controls the position of the window information image so that the window information image (the image of the title bar and the menu bar) is displayed on the front side of the window when the window is displayed inclined in the depth direction. To change. Specifically, the arrangement position of the window in the three-dimensional space calculated by the three-dimensional position calculation unit 104 is read out, and the arrangement position is a position inclined with respect to one side of the window in the depth direction and the window information. If the image is not displayed on the near side, the position of the window information image is changed to the near side by rewriting the image data of the window stored in the storage unit 102.

The window information image direction changing unit 302
When a window is displayed tilted in the depth direction and a window information image is displayed on the near side, only the rectangular portion of the window information image is bent so that it faces forward (parallel to the display screen). Change data. More specifically, the image data of the entire window stored in the storage unit 102 is divided into a window information image and a remaining window body portion, and the window information image is notified to the three-dimensional position calculation unit 104 so as to face the front. Then, the three-dimensional position calculation unit 104 is notified that the remaining window main body is inclined in the depth direction. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The operation of the present apparatus is different from that of the first embodiment in that
In addition to the operation in the first embodiment, the processing for the window information image is added between the calculation of the three-dimensional arrangement position (step S122 in FIG. 4) and the next texture mapping (step S123 in FIG. 4). is there. Therefore, here, the added processing will be described. FIG.
9 is a flowchart showing an operation procedure of the window information image position changing unit 301 and the window information image direction changing unit 302.

The window information image position changing unit 301
When the arrangement position of the window in the three-dimensional space is calculated by the three-dimensional position calculation unit 104 (step S12 in FIG. 4).
2) By referring to the arrangement position, it is determined whether or not the arrangement position is a position inclined in the depth direction with one side of the window as the near side (step S320). as a result,
If the determination is affirmative, subsequently, it is determined whether the window information image is located on the near side (step S).
321) If the window information image is not located on the near side, the window image data stored in the window information image storage unit 102 is rewritten so that the window information image is displayed on the near side (step S322).

FIG. 15 shows an example of a screen display when the window 330 before the position of the window information image is changed and the window 331 after the change are displayed as they are via texture mapping and perspective projection. Show. It can be seen that the title bar and menu bar (window information image) to be displayed at the back of the screen have been moved to the front side of the screen and displayed.

Subsequently, the window information image direction changing unit 3
02 determines whether or not an instruction has been given from the input unit 103 to turn the window information image to the front (step S323).
The window information image and the remaining window body portion of the image data of the entire window stored in the window 2 are divided and specified (step S324). The arrangement position is calculated separately by notifying the three-dimensional position calculation unit 104 so that the remaining window main body part is inclined in the depth direction with respect to the remaining window body (step S325).

Thereafter, the window information image and the remaining window body portion are displayed on the screen through texture mapping and perspective projection in order (step S12 in FIG. 4).
3 to Step S125). FIG. 16 shows the window 332 before the orientation of the window information image is changed as described above.
7 shows an example of the screen display of the window 333 after the front view has been changed.

It can be seen that the title bar and the menu bar (window information image) of the window tilted in the depth direction are bent and turned to the front. (Summary) As described above, the window information image for performing the window operation and the menu operation is always displayed on the front side even when the entire window is displayed in the depth direction by the present apparatus. Alternatively, only the window information image displayed in the foreground may be fixed so as to always face the front. This avoids the problem that window operations and menu operations become difficult due to tilted windows displayed.

In this embodiment, the processing is performed such that the entire planar window is bent into two parts, the window information image and the remaining part. However, the window itself is assumed to be a thick three-dimensional plate model. A title bar or menu bar may be displayed on a side surface that is in contact with the front side when tilted in the depth direction. As a result, the window information image and the remaining portion can be handled as integrated data without being separated, so that the position change and the direction change can be easily performed.

In addition, the window can be easily viewed and the window operation can be easily performed. At the same time, it is not necessary to display the window on the window, so that the window can be effectively used. Further, when the direction may be changed at any time by a window operation, a title bar or a menu bar may be displayed in advance on all four side surfaces in contact with the window surface. [Fifth Embodiment] Next, a multi-window display device according to a fifth embodiment will be described.

The present apparatus is characterized in that the display of the scroll bar is controlled in conjunction with the size of the indistinguishable display area caused by tilting the window. (Configuration) FIG. 17 is a block diagram showing the configuration of the multi-window display device according to the fifth embodiment.

The present apparatus further includes a scroll button position determining unit 401, a scroll bar adding unit 402, and a non-identifiable area determining unit 403, in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. The non-identifiable area determination unit 403 specifies an area that cannot be identified in the final display state because the window is arranged inclined in the depth direction. Specifically, prior to the processing by the texture mapping unit 105 and the perspective projection unit 106, the size of the character of the window stored in the storage unit 102, the arrangement position of the window calculated by the three-dimensional position calculation unit 104, and By referring to the conversion constant (the vanishing point and the coordinates of the viewpoint) in the perspective projection unit 106, the size at which the character previously placed at a representative position on the window is finally displayed is calculated, If it is smaller than 5 points × 5 points, an area displayed behind the character is determined to be an indistinguishable area.

When a non-identifiable area is detected by the non-identifiable area determination section 403, the scroll button position determining section 401 determines the position of the scroll button reflecting the area. Specifically, the side along the depth direction is defined as the scroll direction, and the position is determined from the ratio of the distance in the scroll direction of the identifiable area to the distance in the scroll direction of the non-identifiable area. The position corresponds to a position where an unidentifiable area is not displayed in the window.

The scroll bar adding section 402 displays an image of a scroll bar in which scroll buttons are arranged at the positions determined by the scroll button position determining section 401, by adding the image to the side along the depth direction of the window. More specifically, a scroll bar image is added to the window image data stored in the storage unit 102. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operation in the first embodiment, the image processing for the scroll bar is added between the calculation of the three-dimensional arrangement position (step S122 in FIG. 4) and the next texture mapping (step S123 in FIG. 4). is there. Therefore, here, the added processing will be described. FIG.
9 is a flowchart showing the operation procedure of the scroll button position determination unit 401, the scroll bar addition unit 402, and the non-identifiable area determination unit 403.

When the arrangement position of the window in the three-dimensional space is calculated by the three-dimensional position calculation unit 104 (step S122 in FIG. 4), the non-identifiable area determination unit 403 arranges the window in a direction inclined in the depth direction. With that,
It is determined whether an indistinguishable region occurs in the finally displayed window (step S420),
If it occurs, specify the area.

If it is determined that an indistinguishable area is generated, then the scroll button position determining unit 401
Determines the position where the scroll button should be placed based on the size of the area (step S421). That is, the position where only the identifiable area is displayed in the window is determined. Subsequently, the scroll bar adding unit 40
2 adds a scroll bar image in which scroll buttons are arranged at the positions determined by the scroll button position determination unit 401 to the image data stored in the storage unit 102 (step S422).

As described above, when the content of the storage unit 102 is rewritten to image data of a new window having a scroll bar, thereafter, as in the first embodiment, texture mapping is performed on the new image data. Is displayed on the screen through perspective projection (steps S123 to S125 in FIG. 4). After that, the input unit 103
If you slide the scroll button by
Although the normal scrolling operation is performed by the program execution unit 101, the arrangement position of the window itself does not change, so that the point where the position of the scroll button is shifted by the unidentifiable area remains unchanged.

FIG. 19A shows the scroll button 430.
FIG. 4 is a diagram for illustrating a general concept of a scroll bar 431 and an example of a conventional window display. The position of the scroll button on the scroll bar indicates the relative position of the content currently displayed in the window at that point in the entire display target. FIG. 19B illustrates an example of a screen display when the lower half of the display area of the window is determined to be the unidentifiable area 432.

Here, a scroll button 430 is displayed at a position where the lower half 432 of the display area protrudes from the window. Therefore, the input unit 10
When the scroll button 430 is moved downward by 3, an area that could not be identified until now is moved to an identifiable position. (Summary) As described above, even if an indistinguishable area is generated because the window is tilted and displayed in the depth direction by the present apparatus, the scroll bar for sliding the window in the depth direction and moving the window to the front is provided. Since it is attached, the entire display target can be identified by a simple operation of sliding the scroll button without moving the window itself.

In the present embodiment, a scroll bar is added when an unidentifiable area is detected. However, regardless of the presence or absence of an unidentifiable area, there is a case where a portion not displayed in the window exists. May have a configuration in which a scroll bar is already added. In this case, the position of the scroll button is a position in a case where an area obtained by adding a non-identifiable area to the protruding area is protruding. Further, in the texture mapping, the operability for the user can be improved by distinguishing the indistinguishable area from the area that is off the page and color-coding the scroll bar. [Sixth Embodiment] Next, a multi-window display device according to a sixth embodiment will be described.

The present apparatus is characterized in that when a window having a layer structure is displayed, each layer is separated and displayed in a tilted depth direction. (Configuration) FIG. 20 is a block diagram showing the configuration of the multi-window display device according to the sixth embodiment.

The present apparatus further includes a window layer separating section 501 and a layer window generating section 502 in addition to the components 101 to 108 included in the multi-window display apparatus of the first embodiment. Hereinafter, differences from the first embodiment will be described. The window layer separation unit 501 detects a window having a layer structure from the windows stored in the storage unit 102, and separates the window for each layer.

In an application program in which image data and text data are mixed or an application program in which a plurality of image data are mixed, image data and text data are separated from each other as in graphic drawing software and document layout software used for desktop publishing. An application program has a layer structure that can be edited and operated independently.
When such an application program is executed by the program execution unit 101, the storage unit 102 stores code data and image data of a window having a layer structure.

More specifically, the window layer separating section 501 detects a window having a layer structure by analyzing the code data of each window stored in the storage section 102, and detects a window from the image data of the window. The image data belonging to other layers except the (lowest) layer is extracted. The layer window generation unit 502
An independent layer window including the layers separated by the window layer separation unit 501 is generated. Specifically, the image data extracted by the window layer separation unit 501 is generated as one independent window for each layer, and new image data is generated.
02 is stored. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operations in the first embodiment, generation of window code data and image data (step S1 in FIG. 4)
21) and calculation of the next three-dimensional arrangement position (step S in FIG. 4)
122) is added to the processing for the layer window. Therefore, here, the added processing will be described.

FIG. 21 shows a window layer separating section 501.
6 is a flowchart showing an operation procedure of a layer window generation unit 502. When the code data and the image data of the window having the layer structure are stored in the storage unit 102 by the program execution unit 101 (step S121 in FIG. 4), the window layer separation unit 501 converts the code data stored in the storage unit 102 After specifying a window having a layer structure and separating the image data for each layer, the image data of another layer is taken out except for the base layer (step S520).

FIG. 22A is a diagram conceptually showing the separation of layers (step S521) when there are two layers. A state in which the combining window 530 stored in the storage unit 102 is separated into the base layer 531 and another upper layer 532 is shown. Subsequently, the layer window generating unit 502 generates image data for using the upper layer 532 extracted by the window layer separating unit 501 as an independent window, and stores the image data in the storage unit 102 (Step S521). Then, the layer window generation unit 502 generates the newly generated layer window 5
For 32, as the arrangement position in the three-dimensional space,
The three-dimensional position calculation unit 104 is notified that the position is parallel to the window of the basic layer 531 and is a position separated by a predetermined distance in the front direction when the basic window is viewed from the front (step S522).

Therefore, the three-dimensional position calculation unit 104 calculates the arrangement position of the window of the basic layer 531 as specified by the program execution unit 101 or the input unit 103, while the three-dimensional position calculation unit 104 calculates the arrangement position of the window of the upper layer 532. Based on the designation from the layer window generation unit 502, an arrangement position where the window of the basic layer 531 is translated by a predetermined distance in a direction perpendicular to the window surface is calculated (step S122 in FIG. 4).

FIG. 22B shows an example of a screen display when a window having a layer structure is arranged to be inclined in the depth direction. The upper layer 532 is displayed as if it were floating above the base layer 531. (Summary) As described above, the window having the layer structure is separated and displayed by being tilted in the depth direction by the present apparatus, so that information which cannot be determined in the conventional front view, that is, It is possible to grasp at a glance the information as to whether the object is placed. Thus, the object can be directly specified while displaying all the layers, instead of specifying the object while switching to a different layer, so that an editing operation or the like over a plurality of layers becomes smooth.

In the present embodiment, the layer windows are transparent, and all the overlapping layer windows are displayed. However, by making the layer windows that the user wants to see translucent, only the layer windows that the user wants to pay attention to are displayed. It is emphasized and displayed so that it is easy to see. Seventh Embodiment Next, a multi-window display device according to a seventh embodiment will be described.

The present apparatus is characterized in that the result of the keyword search is displayed as a new layer window, separated from the original window and floated. (Configuration) FIG. 23 is a block diagram showing the configuration of the multi-window display device according to the seventh embodiment.

The present apparatus further includes a keyword search unit 601, a search result output unit 602, and a layer window generation unit 603, in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. Keyword search unit 60
1 retrieves a keyword from the document displayed in the window. More specifically, a character string that matches the keyword specified by the input unit 103 is searched from the code data of the document stored in the storage unit 102. If a search is performed, the window, the searched position, and the keyword Is reported to the layer window generation unit 603.

The layer window generating section 603 generates a new layer window in which only the searched keywords are arranged at the searched positions based on the notification from the keyword searching section 601. More specifically, the storage unit 10 stores, as image data of a new layer window, image data in which only the character string searched from the image data of the window notified from the keyword search unit 601 is left.
2 is stored.

The search result output unit 602 notifies the three-dimensional position calculation unit 104 of the layout position of the layer window generated by the layer window generation unit 603. More specifically, the new window is searched so that the positional relationship between the original window to be searched and the newly generated layer window is the same as the positional relationship between the windows of the base layer 531 and the upper layer 532 in the sixth embodiment. Specify the layout position of the layer window. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operations in the first embodiment, generation of window code data and image data (step S1 in FIG. 4)
21) and calculation of the next three-dimensional arrangement position (step S in FIG. 4)
122) is added to the processing for the layer window. Therefore, here, the added processing will be described.

FIG. 24 is a flowchart showing the operation procedure of the keyword search unit 601, the search result output unit 602, and the layer window generation unit 603. When a keyword is input from the user via the input unit 103 (step S620), the keyword search unit 601 causes the storage unit 102
Is searched for a character string that matches the keyword from the code data of all the documents stored in step S62.
1).

When the search is performed (step S62)
2) The keyword search unit 601 converts the position and keyword searched for each window into the layer window generation unit 6
Report to 03. In accordance with the report, the layer window generating unit 603 generates layer window image data consisting of only the searched character strings, and stores the image data in the storage unit 102 (step S623).

Then, the search result output unit 602 generates a three-dimensional data so that the window generated by the layer window generating unit 603 is displayed as being raised from the original window.
The three-dimensional position calculation unit 104 calculates the arrangement position in the three-dimensional space (step S624). Therefore, the three-dimensional position calculation unit 104 calculates the arrangement position of the original window as the search target as specified by the program execution unit 101 or the input unit 103, while the three-dimensional position calculation unit 104 generates the arrangement position by the layer window generation unit 603. With respect to the layer window, an arrangement position parallel to the original window and separated by a predetermined distance is calculated based on the designation from the search result output unit 602 (step S122 in FIG. 4).

FIG. 25 is a diagram showing an example of the above procedure and a final screen display. Original window 63 to be searched
If the keyword “window” is included in 0, a new layer window 631 is generated by the layer window generation unit 603, and the original window 630 is generated.
It is shown that it is displayed by being raised from the display. (Summary) As described above, since the original window and the window showing the search result of the keyword are displayed separated and inclined in the depth direction by the present apparatus, the search result can be grasped at a glance. . That is, in the conventional front view, processing such as expressing the search result in another color is required, but such special processing is unnecessary in the present apparatus.

It is also easy to provide an apparatus that supports a plurality of keywords by displaying different keywords for different layers or coloring each layer translucently. [Eighth Embodiment] Next, a multi-window display device according to an eighth embodiment will be described.

The present apparatus is characterized in that a plurality of related windows are connected and displayed while being tilted in the depth direction. (Structure) FIG. 26 is a block diagram showing the structure of the multi-window display device according to the eighth embodiment.

The present apparatus further includes an inter-window connection unit 701, a window relative position storage unit 702, and a window relative position change unit 703, in addition to the components 101 to 108 included in the multi-window display device of the first embodiment. Hereinafter, differences from the first embodiment will be described. The inter-window connection unit 701 specifies a group of related windows from among a plurality of windows currently scattered and displayed. Here, the related window is a window generated by the same application program in this embodiment. For example, these windows correspond to a case where a plurality of pages of the same document are simultaneously displayed in a plurality of windows by an application program such as a word processor.

More specifically, the inter-window connection unit 701
Is stored in the storage unit 102 based on a notification from the program execution unit 101 executing the application program.
Are identified as those generated by the same application program from among the plurality of windows stored in the window, the code data of each window is analyzed, and the windows are ordered.
Notify. For example, ordering is performed in ascending order of page numbers.

The window relative position storage unit 702 calculates the arrangement position of each window in a three-dimensional space when a group of windows specified and ordered by the inter-window connection unit 701 are connected in a horizontal direction so as to be bent in a zigzag manner. Generate and store connection information for connection. Specifically, the arrangement position of the top window tilted in the depth direction and the relative coordinates between the sequentially connected windows are stored. Note that the initial arrangement position is predetermined, and the inclination of each window is, for example, a state in which the opposite side is rotated by 45 degrees in the depth direction with the near side as a rotation axis.

The window relative position changing unit 703 collapses only a specific window from a series of windows connected and displayed in a zigzag manner to erase the window from the screen display. Specifically, the storage contents of the window relative position storage unit 702 are changed so that the window specified by the input unit 103 is bypassed and connected. (Operation) Next, the operation of the multi-window display device of the present embodiment configured as described above will be described.

The operation of this apparatus differs from that of the first embodiment in that
In addition to the operations in the first embodiment, generation of window code data and image data (step S1 in FIG. 4)
21) and calculation of the next three-dimensional arrangement position (step S in FIG. 4)
122) is added. Therefore, here, the added processing will be described.

FIG. 27 is a flowchart showing the operation procedure of the window connection unit 701, window relative position storage unit 702, and window relative position change unit 703.
Now, based on the execution of the same application program, the program execution unit 101 sets pages 1 to 5 of the same document.
It is assumed that the code data and the image data of the five windows that respectively display page 5 are stored in the storage unit 102, and the fact is notified to the inter-window connection unit 701 (FIG. 4).
Step S121).

The inter-window connection unit 701 specifies the five windows stored in the storage unit 102 based on the notification, and refers to their code data to store the window relative position in ascending page number order. The notification is made to the unit 702 (step S720). The window relative position storage unit 702 generates and stores initial connection information in which the windows notified from the inter-window connection unit 701 are connected in a zigzag manner in the notified order (step S721).

FIG. 28A shows that the arrangement positions of the windows of page 1 to page 5 are determined by the three-dimensional position calculation unit 104 based on the connection information without changing the initial connection information (step S722). FIG. 5 shows a screen display when the calculated and displayed (Steps S122 to S125 in FIG. 4). The five windows are connected with one side in common, and alternately tilted in different directions to the depth and are displayed side by side in a row.

If the input unit 103 notifies the window relative position changing unit 703 of an instruction to collapse a specific window (step S722),
The window relative position change unit 703 changes the storage contents (connection information) of the window relative position storage unit 702 so as to bypass the window and connect the subsequent window (step S723).

In FIG. 28B, the connection information is changed so that the window of page 3 is folded (step S72).
2, S723), the screen in the case where the arrangement positions of the subsequent windows of pages 4 and 5 are newly calculated and displayed by the three-dimensional position calculation unit 104 based on the connection information (steps S122 to S125 in FIG. 4). The example of a display is shown.

The pages 4 and 5 are changed so as to be connected after the page 2 while maintaining the relative position of the page 4 with respect to the page 3. (Summary) In this way, by connecting the sides of the related page windows to each other and displaying them at an angle by the present apparatus, it is possible to grasp the relevance between the windows at a glance by the connection state between the windows. Becomes

An application program such as a word processor often includes a plurality of pages.
In this case, in a conventional window system in which a window is directed to the front, it is difficult to display a plurality of pages at the same time, and there is no choice but to overlap each page or display several pages. According to the present apparatus, many pages are arranged without overlapping the limited display area and displayed in a list.

Further, a window to be displayed and a window which does not need to be displayed are designated in accordance with a user's input, and a window which does not need to be displayed is arranged at a position where it is folded so as to overlap another window. By storing the information, it is possible to keep only a plurality of windows of interest at the same time while maintaining the relationship between windows such as continuous pages.

Note that the texture mapping section 105 does not paste the bitmap of the window with respect to the window arranged at the position where it is folded so as to overlap the other window, so that the texture mapping process is partially omitted. It becomes possible. Also, in the present embodiment, windows generated by the same application program are used as related windows. However, pages of another document belonging to the same project are often strongly related between pages, and different application programs are generated. The related windows may be connected and displayed by specifying the related windows in advance even if the windows are to be displayed.

Further, there may be provided means for editing such as copying or moving an object across windows that are connected and displayed. FIGS. 29 (a) and 29 (b)
It is an example of a screen display in a case where means for operating an object is provided across windows connected to pages. FIG. 29A shows a screen display when windows of page 1 to page 5 are displayed based on the initial connection information, and is the same drawing as FIG. 28A.

FIG. 29B shows a state in which the objects of page 1 are copied to page 4 after page 1 and page 4 are displayed facing each other by folding page 2 and page 3. I have. This is the input unit 10
The code data and image data of page 1 stored in the storage unit 102 are overwritten on page 5 so that the object of the copy source window designated by 3 is copied to the copy destination window.

[0115]

As is apparent from the above description, according to the multi-window display device of the present invention, there is provided a multi-window display device for displaying a plurality of windows on a screen, the window representing the display contents of the windows. Window data holding means for holding data, layout position determining means for determining a virtual layout position in at least one of the windows that is not parallel to the screen in a three-dimensional space, and based on the determined layout position It is characterized by comprising a perspective projection unit for converting the window data so that the window is perspectively projected on the screen, and a display unit for displaying the converted window data on the screen.

As a result, since at least one window is displayed inclined in the depth direction, the size of the entire window is reduced when placed in the depth of the three-dimensional space, and the display content cannot be determined. Is avoided. In addition, since the display area can be saved in the depth portion of the window while the information of the portion before the window tilt can be grasped, at least a part of the display content of the window is displayed so as to be identifiable. A multi-window display device that effectively utilizes the area is realized.

Here, the multi-window display device further comprises input means for obtaining an instruction from an operator,
The arrangement position determining means may determine the arrangement position according to an instruction obtained by the input means. As a result, the operator can determine the size of the entire window and the degree of inclination in the depth direction.

The arrangement position determining means may determine the coordinates of each vertex for specifying the outline of the window in a three-dimensional space as the arrangement position. Thus, for example, when the window is a plane rectangle, the arrangement position of the window in the three-dimensional space is specified only by the coordinates of the four vertices. In addition, the multi-window display device further includes a window analysis unit that detects a portion that satisfies a certain condition among display contents of the window as an important portion based on the window data held in the window data holding unit, The arrangement position determination means may determine the arrangement position of the window so that the position detected by the analysis means is displayed on the near side.

As a result, the window is displayed tilted in the depth direction such that the important part of the display content is in the foreground. Therefore, even if a plurality of windows are displayed, the operator can view the window. The type and display contents of each window can be identified at a glance. Further, the window analyzing means may determine whether or not the window data includes the title of the window, and if the window data is included, may detect a location where the window title is displayed as the important location.

This makes it easy to grasp the type of window. Further, the window analysis means determines whether or not the window data includes a document and an instruction to display the document horizontally, and if so, determines that the left side of the window is the important part. You can also. This makes it easy to grasp the type and outline of the horizontally written document displayed in the window.

The window analyzing means determines whether or not the window data includes a document and an instruction to display the document in vertical writing. Can also be detected. This makes it easy to grasp the type and outline of the vertically written document displayed in the window.

Further, the arrangement position determining means includes an automatic arrangement unit for determining an arrangement position of each window such that at least one window is a predetermined arrangement position for a plurality of windows and is at a position not parallel to the screen. It can also be. As a result, the plurality of windows scattered and displayed on the screen are displayed side by side, so that the display area of the screen is effectively utilized and all the windows currently displayed can be seen at a glance.

Further, the arrangement position determining means further comprises: means for determining a new arrangement position such that the window displayed at the arrangement position is displayed in front according to the instruction obtained by the input means. Means for determining a new arrangement position so as to return the displayed window to the arrangement position. This facilitates the operation of turning only the window to be worked toward the front, and returning to the original alignment position when the work is completed.

Further, the multi-window display device further holds the window information image including the title bar and the menu bar in the window data holding means so as to be displayed on the near side based on the arrangement position determined by the arrangement position determining means. And a window information image position changing means for editing the window data.

As a result, even when the window is displayed at an angle, the title bar and the menu bar are displayed at the front, so that it is easy to grasp the type of the window and to operate the window. Further, the multi-window display device further controls the arrangement position determining means so that the window information image is directed to the front with respect to the window in which the window information image is displayed in the foreground, and the remaining window main body is arranged at a position not parallel to the screen. Window information image direction changing means.

As a result, even when the window is displayed at an angle, the title bar and the menu bar are displayed at the front and in the front, so that the type of the window and the window operation can be grasped. It will be easier.
In addition, the multi-window display device further includes a non-identifiable area determining unit that determines whether an unidentifiable area in which characters smaller than a predetermined size are displayed in the window data converted by the perspective projection unit occurs. Scroll button position determining means for determining the position of the scroll button when it is determined that there is an unidentifiable area, and determining that the area is not displayed in the window; and scrolling to the determined position. Scroll bar adding means for editing the window data so as to add an image of a scroll bar on which buttons are arranged to a window may be provided.

Thus, even when an area where characters and the like cannot be identified due to the display of the window tilted in the depth direction, the characters and the like can be slid to the display area where the characters and the like can be identified. Further, the multi-window display device further includes a layer separating unit that separates window data belonging to an upper layer excluding a lowest layer from window data of a window having a layer structure held by the window data holding unit, for each layer. Layer window generating means for generating window data of a layer window corresponding to the separated layer, wherein the arrangement position determining means arranges the layer window as an arrangement position in parallel with the lowermost layer and separated by a predetermined distance. The position can also be determined.

As a result, each layer window is displayed in a three-dimensionally raised manner, so that the objects placed on each layer can be grasped at a glance, and the objects of each layer or objects extending over a plurality of layers can be grasped. Editing becomes easy. In addition, the multi-window display device further includes a keyword search unit that searches the window data held in the window data holding unit for a portion that matches the keyword acquired by the input unit, and a keyword in the searched portion. Layer window generating means for generating window data of a newly arranged layer window and storing the generated window data in the window data holding means, wherein the arrangement position determining means is a window which has been searched as an arrangement position of the layer window. It is also possible to determine an arrangement position that is parallel to and separated by a predetermined distance.

As a result, the window showing the search result is displayed as if it were raised from the original window as the search target, so that the position of the searched keyword can be confirmed only by focusing on the layer window. . Further, the multi-window display device further comprises:
Related window specifying means for specifying a group of related windows from among a plurality of displayed windows, and connection information for arranging the specified group of windows together so as to bend in a zigzag manner and generating connection information Connection information storage means for storing, and the arrangement position determination means may determine the arrangement position of the group of windows based on the connection information.

Thus, when a plurality of pages in a document are displayed in different windows, the related windows are connected and displayed, so that, for example, windows relating to the same document can be easily specified. Further, the multi-window display device further includes connection information changing means for changing the connection information so as to arrange the group of windows except for some of the windows according to an instruction acquired by the input means. The arrangement position determination means may determine an arrangement position of a group of windows excluding the some windows based on the changed connection information.

As a result, it is possible to display only the window of interest out of the connected and displayed windows. In addition, the multi-window display device may further include a window data held by the window data holding unit so as to edit display content across one of the group of windows and another one according to an instruction obtained by the input unit. An inter-window editing means for editing may be provided.

As a result, it is possible to display only the window of interest from the connected and displayed windows in close proximity, and to edit the object over two windows. Is improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a multi-window display device according to a first embodiment.

FIGS. 2A and 2B are perspective projection units 10;
6 is a diagram for explaining the concept of perspective projection by No. 6; FIG. FIG. 2A is a diagram illustrating a positional relationship among a projection center, a projection plane, and an object (window). FIG. 2B shows a perspective view when the object is a three-dimensional object.

FIGS. 3A to 3C are perspective projection units 106;
FIG. 7 is a diagram for explaining the specific processing contents of perspective projection by using FIG. FIG. 3A shows a case where a window virtually placed in a three-dimensional space is parallel to a display screen.
FIG. 4 is a diagram illustrating a user's viewpoint, a positional relationship between a display screen and a window, and a shape of a perspectively projected window. FIG. 3B shows a case where the window virtually placed in the three-dimensional space is not parallel to the display screen.
FIG. 4 is a diagram illustrating a user's viewpoint, a positional relationship between a display screen and a window, and a shape of a perspectively projected window. FIG. 3C is a diagram illustrating a specific calculation process of perspective projection.

FIG. 4 is a flowchart illustrating an operation procedure of the multi-window display device according to the first embodiment.

FIG. 5 shows a window 130 arranged in parallel with the screen;
131 and the window 13 arranged in a non-parallel position
It is a figure which shows the example of the screen display of 2-133.

FIG. 6 is a block diagram illustrating a configuration of a multi-window display device according to a second embodiment.

FIG. 7 is a flowchart illustrating a procedure when the window analysis unit 201 specifies an important part of the window.

8 shows an example of a screen display when an important part is specified by the window analysis unit 201. FIG.

FIG. 9 is a block diagram illustrating a configuration of a multi-window display device according to a third embodiment.

FIG. 10 is a flowchart illustrating a procedure of automatic alignment by the automatic alignment unit 104a.

11A shows an example in the case of a first alignment method without overlapping, and FIG. 11B shows an example in a case of a second alignment method with overlapping.

12A is a screen display immediately after automatic alignment, and FIG. 12B is a screen display of FIG.
12 is a screen display immediately after one window 241 is selected by the user. FIG. 12C is a window display button 245 provided at the right end of the title bar of the window 241 by the input unit 103 in the screen display of FIG.
An example of a screen display immediately after is pressed is shown.

FIG. 13 is a block diagram illustrating a configuration of a multi-window display device according to a fourth embodiment.

FIG. 14 is a flowchart illustrating an operation procedure of a window information image position changing unit 301 and a window information image direction changing unit 302.

FIG. 15 shows an example of screen display of a window 330 before the position of the window information image is changed and a window 331 after the position is changed.

FIG. 16 shows an example of screen display of a window 332 before the direction of the window information image is changed and a window 333 after the direction of the window information image is changed to the front.

FIG. 17 is a block diagram illustrating a configuration of a multi-window display device according to a fifth embodiment.

FIG. 18 shows a scroll button position determining unit 401, a scroll bar adding unit 402, and a non-identifiable area determining unit 403.
6 is a flowchart showing the operation procedure of FIG.

FIG. 19A is a diagram illustrating a general concept of a scroll button 430 and a scroll bar 431. FIG. 19B illustrates an example of a screen display when the lower half of the display area of the window is determined to be the unidentifiable area 432.

FIG. 20 is a block diagram illustrating a configuration of a multi-window display device according to a sixth embodiment.

FIG. 21 is a flowchart showing an operation procedure of a window layer separation unit 501 and a layer window generation unit 502.

FIG. 22A is a diagram conceptually illustrating a process of separating layers (step S521) when there are two layers. FIG. 22B shows an example of a screen display when a window having a layer structure is arranged to be inclined in the depth direction.

FIG. 23 is a block diagram illustrating a configuration of a multi-window display device according to a seventh embodiment.

FIG. 24 shows a keyword search unit 601 and a search result output unit 6
11 is a flowchart illustrating an operation procedure of the layer window generation unit 02 and the layer window generation unit 603.

FIG. 25 is a diagram illustrating an example of a screen display when a result of a keyword search is displayed as a layer window.

FIG. 26 is a block diagram illustrating a configuration of a multi-window display device according to an eighth embodiment.

FIG. 27 shows a window connection unit 701, a window relative position storage unit 702, and a window relative position change unit 703.
6 is a flowchart showing the operation procedure of FIG.

FIG. 28A illustrates an example of a screen display in which windows of pages 1 to 5 are displayed based on initial connection information. FIG. 28B shows an example of a screen display when the connection information is changed so that the window of page 3 is folded.

FIG. 29 is an example of a screen display when means for operating an object is provided across windows connected to pages. FIG. 29A shows a screen display when windows of page 1 to page 5 are displayed based on the initial connection information, and is the same drawing as FIG. 28A. FIG.
FIG. 4B is a diagram showing a state in which the objects of page 1 are copied to page 4 after folding pages 2 and 3 to display page 1 and page 4 face-to-face.

FIG. 30 is a diagram showing an example of a multi-window screen display according to the related art.

[Explanation of symbols]

 Reference Signs List 101 program execution unit 102 storage unit 103 input unit 104 three-dimensional position calculation unit 104a automatic alignment unit 105 texture mapping unit 106 perspective projection unit 107 frame memory unit 108 image display unit 201 window analysis unit 301 window information image position change unit 302 window information Image direction change unit 401 Scroll button position determination unit 402 Scroll bar addition unit 403 Indistinguishable area determination unit 501 Window layer separation unit 502 Layer window generation unit 601 Keyword search unit 602 Search result output unit 603 Layer window generation unit 701 Connection between windows Unit 702 window relative position storage unit 703 window relative position change unit

Claims (17)

[Claims] 1. A multi-window display device for displaying a plurality of windows on a screen, comprising: window data holding means for holding window data representing display contents of the window; and at least one of the windows in a three-dimensional space. Placement position determining means for determining a virtual placement position that is not parallel to the screen, and perspective projection means for converting the window data to perspective project the window on the screen based on the determined placement position. A display unit for displaying the converted window data on the screen. 2. The multi-window display device further comprises input means for acquiring an instruction from an operator, wherein the arrangement position determination means determines the arrangement position according to the instruction acquired by the input means. The multi-window display device according to claim 1, wherein 3. The arrangement position determining means determines coordinates of each vertex for specifying an outer shape of the window in a three-dimensional space as the arrangement position.
The multi-window display device as described in the above. 4. The multi-window display device further includes a window analysis unit that detects a portion satisfying a predetermined condition in the display contents of the window as an important portion based on the window data held in the window data holding unit. 4. The multi-window display device according to claim 3, wherein the arrangement position determination unit determines the arrangement position of the window so that a position detected by the analysis unit is displayed on the near side. 5. 5. The window analyzing means determines whether or not the window data includes a title of the window, and when the window data is included, detects a location where a window title is displayed as the important location. The multi-window display device according to claim 4, wherein: 6. The window analysis means determines whether or not the window data includes a document and an instruction to display the document in a horizontal direction. 5. The multi-window display device according to claim 4, wherein 7. The window analysis means determines whether or not the window data includes a document and an instruction to display the document in vertical writing. The multi-window display device according to claim 4, wherein the position is detected as a location. 8. The arrangement position determination means, wherein at least one arrangement position is predetermined for a plurality of windows.
4. The multi-window display device according to claim 3, wherein one of the windows includes an automatic alignment unit that determines an arrangement position of each window so as not to be parallel to the screen. 9. The arrangement position determining means further determines a new arrangement position such that the window displayed at the arrangement position is displayed in a front view in accordance with an instruction obtained by the input means, and 9. The multi-window display device according to claim 8, further comprising: means for determining a new arrangement position to return the displayed window to the arrangement position. 10. The multi-window display device further holds the window information image including a title bar and a menu bar in the window data holding unit based on the arrangement position determined by the arrangement position determination unit. 4. The multi-window display device according to claim 3, further comprising a window information image position changing means for editing the window data. 11. The multi-window display device further determines the arrangement position such that the window information image is directed to the front with respect to the window on which the window information image is displayed in the foreground, and the remaining window main body is arranged at a position not parallel to the screen. 11. The multi-window display device according to claim 10, further comprising a window information image direction changing unit for controlling the unit. 12. The multi-window display device further determines whether or not a non-identifiable area in which characters smaller than a predetermined size are displayed in the window data converted by the perspective projection unit occurs. Region determining means; and scroll button position determining means for determining the position of the scroll button when it is determined that there is an indistinguishable area when the area is not displayed in the window. 4. The multi-window display device according to claim 3, further comprising: a scroll bar adding unit that edits the window data so as to add a scroll bar image having a scroll button at a position to the window. 13. The multi-window display device further includes a layer separation unit that separates window data belonging to an upper layer excluding a lowermost layer from window data of a window having a layer structure held in the window data holding unit for each layer. Means, and a layer window generating means for generating window data of a layer window corresponding to the separated layer, wherein the arrangement position determining means is arranged at a predetermined distance in parallel with the lowermost layer as an arrangement position of the layer window. 4. The multi-window display device according to claim 3, wherein the arrangement positions separated by a distance are determined. 14. The multi-window display device further includes: a keyword search unit configured to search a window data held in the window data holding unit for a portion matching the keyword acquired by the input unit, as a search target; Layer window generating means for generating window data of a new layer window in which a keyword is arranged and storing the generated window data in the window data holding means, wherein the arrangement position determining means determines a search object as an arrangement position of the layer window. 4. The multi-window display device according to claim 3, wherein an arrangement position which is parallel to the changed window and is separated by a predetermined distance is determined. 15. The multi-window display device further comprises: related window specifying means for specifying a related group of windows from the plurality of displayed windows; and a frame for bending the specified group of windows in a zigzag manner. Connection information storage means for generating and storing connection information for attaching and arranging the groups, and wherein the arrangement position determination means determines the arrangement position of the group of windows based on the connection information. The multi-window display device according to claim 3. 16. The connection information change device according to claim 1, wherein the multi-window display device further changes the connection information so as to arrange only a group of windows except for some of the windows according to an instruction obtained by the input unit. 16. The multi-window display device according to claim 15, wherein the arrangement position determination unit determines an arrangement position of a group of windows excluding the some windows based on the changed connection information. . 17. The multi-window display device further includes a window data holding unit configured to edit display content across one of the group of windows and another one according to an instruction obtained by the input unit. 17. The multi-window display device according to claim 16, further comprising inter-window editing means for editing window data.