JPH07154684A - Window display controller - Google Patents

Abstract

PURPOSE:To provide a function of panning and zooming economically in order to display a portrait image received from a camera in the center of a window with ease of observation with respect to the window display control method for a video communication terminal equipment. CONSTITUTION:In a motion picture frame memory control part for a window display device displaying a digital video image onto a personal computer or a work station, a read start address designation part 8 designates a read start address on a frame memory under the control of a system control part 10 to read digital video data at an optional address in the motion picture frame memory part 4. The group window display controller is provided with plural frame memory control parts, a system control part, a picture element data arbitration part and a display control signal generating part and the display control signal generating part is provided with a means setting its area depending on a display position of a group window.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention economically performs panning (moving up and down, left and right) and zooming (moving into perspective) as a function for displaying a human image input from a camera in the center of a moving image window in an easy-to-see manner. The present invention relates to a display control method of a video communication terminal including a display control means to be provided.

Recently, N-ISDN (narrow band 64 Kbps) and B
-As a new communication service that utilizes the communication capacity of ISDN (wideband 150 Mbps), people in remote areas can exchange multimedia information including videos in real time.
Video communication systems that support group collaboration such as meetings and group decision making have been drawing attention. This is a service that uses a personal computer, a workstation, or the like to display the image of the person who is participating in the conference in a moving image window.

The present invention also relates to a multi-window display method for grouping a plurality of windows and performing display control in cooperation with each other, and a clipping (capturing) display control device for group windows in the apparatus. The multi-window display is an essential function for constructing a human interface such as a workstation as it provides a work environment in which a user can efficiently perform various tasks interactively. For such multi-window display, there is a need for a technology that satisfies the required functions of high-speed display and display with high flexibility.

[0004]

2. Description of the Related Art There are two possible methods for displaying a person's image of the other party in a moving image window on an image terminal in an easily viewable manner. First
Method is realized by the video communication terminal equipped with a movable camera that can be electronically controlled, and the video terminals of each other send control of panning and zooming to the camera of the other party via a communication line. .

In the second method, the video communication terminal is equipped with a fixed and inexpensive camera, and the video of the user himself is displayed in the moving picture window for checking so as not to be off the screen or too close to the camera. The purpose can be achieved by consciously adjusting the position of one's body.

Further, due to the sophistication of the human interface, a "multi-window display control method and apparatus (Patent Application Publication No. 4-127195)" in which a plurality of windows are defined as one group window and display control is performed in group window units has been proposed. As a technology, as shown in Figure 17,
An example of basic operation of a group window realized by the present invention will be shown. (a) create / disappear, (b) change overlay,
(c) shows the state of each operation screen during movement.

In the conventional example, the display control signal generation section performs centralized control of the group window, and each frame memory control section performs display control of individual windows forming the group window. When changing the display position of the group window within the range in which the entire area of the group window in which the windows are grouped is displayed, it is only necessary to change the control parameters to the display control signal generator to control the individual frame memory controllers. There is an advantage that the display control is simple because it is not necessary to change the parameters.

[0008]

The first method has a problem that a large and expensive camera is required because it is movable. The second method is to look at the document that is the subject of the discussion and the participants while looking at the video window that is showing you, so that it appears in the center of the window and in the same size as the other participants. You need to adjust the distance from. However, keeping the position of the body constant while arguing has caused a problem of a mental burden.

The present invention provides a window display control device that enables panning and zooming in a moving image window of a person's image of a partner even in a video communication terminal equipped with a fixed inexpensive camera. The purpose is to

FIG. 18 shows three types of display examples regarding the group window. Display example (a) is when the entire area of the group window is displayed, and display example (b) is
The left side of the defined group window is in the invisible state, and the display example (c) is in the invisible state on the right side. In this way, if it is attempted to realize the display control in which a part of the group window is not displayed in the conventional example, not only the change of the control parameter to the display control signal generation unit but also the control parameter of each individual frame memory control unit There is a problem in that the display state management and the display control regarding the group window and the individual windows in the group window are complicated because they have to be changed.

[0011]

FIG. 1 shows the principle configuration of a window display control device according to the present invention. The figure shows a moving picture frame memory control unit of a window display device for displaying a digital video image on a personal computer or a workstation. In the figure, 1 is a video frame memory control unit,
2 is a scaling unit, 3 is a write address generation unit, 4 is a moving image frame memory unit, 5 is a write control unit, 6 is a write start address designation unit, 7 is a read control unit, 8 is a read start address designation unit, Reference numeral 9 represents a read address generation unit, and 10 represents a system control unit.

The moving picture frame memory unit 4 is a normal RAM.
The row and column address data are selected as in the case of the memory device of FIG. The digital video data input to the scaling unit 2 is thinned and interpolated so that the reduction / enlargement ratio designated by the system control unit 10 is achieved, and the digital video data is output to the moving image frame memory unit 4.

The scaling unit 2 also outputs a vertical synchronizing signal, a horizontal synchronizing signal, and a scaled dot clock to the write address generating unit 3. The write address generating unit 3 generates an address for each of the digital video data written in the moving image frame memory unit 4 based on these. At this time, the write start address designating section 6 designates the address to start writing in the frame memory where the digital video data is to be written, to the write controlling section 5 by the designation of the system control section 10.

Under the control of the system control unit 10, the read start address designating unit 8 designates the read start address on the frame memory. This makes it possible to read digital video image data at an arbitrary address in the moving image frame memory unit 4.

FIG. 2 is a block diagram showing the principle of the group window display control device of the present invention. In the figure, 31 is a system control unit that controls the entire apparatus, 32 is a display control signal generation unit that generates signals related to display control, 33 _{1 to} 33 _n are frame memory control units, and 34 is pixel data arbitration that selects pixel data. Indicates a part.

The display frame memory controller 33 ₁ ~ 33 _n is, means for providing an offset to the display address created from the display address creation area signal, the frame memory storing the image data to be displayed in each window The display signal generator 32 is provided with means for giving the start coordinates, and means for setting the area by the display position of the group window.

[0017]

In the moving image window display control device, the received phase manual image data is processed by the scaling unit 2, the write start address designating unit 6, and the read start address designating unit 8 to pan the window for displaying the phase manual image. , Realize the zooming function. (Claim 1) In the moving picture window display control device, with the write start address to the moving picture frame memory unit 4 fixed,
A panning function is realized by changing the read start address as needed by the read start address designating unit 8.
(Claim 2) In the moving picture window display control device, with the read start address to the moving picture frame memory unit 4 fixed,
The panning function is realized by changing the write start address by the write start address designating section 6 as needed.
(Claim 3) In the moving picture window display control device, with the display start address of the moving picture frame memory section 4 fixed, the scaling section 2 performs enlarging / reducing processing at any time, and the write address generating section 3 writes at any writing address. A zooming function is realized by adding an offset to.
(Claim 4) In the moving picture window display control device, with the write start address to the moving picture frame memory unit 4 fixed,
The scaling unit 2 changes the enlargement / reduction instruction for the digital video data at any time, and the read start address designating unit 8 changes the read start address synchronized with this at any time to provide a zooming function. (Claim 5) In the group window display control device, even if the display position of the group window on the display is changed, the group window and the individual windows belonging to the group window are not changed, and the correction means is added to the group window. Controls the display of windows. (Claim 6) In the group window display control device, the frame memory control unit 33 comprises means for giving an offset to the display display address and means for giving a display start coordinate to the frame memory. However, a means for setting the area by the display position of the group window is provided. (Claim 7) In a group window display control device, group window display position determination processing, group window display area setting processing, individual window effective display area setting processing, individual window display position determination processing, individual window start coordinate correction The display is controlled by processing. (Claim 8) By applying the frame memory control unit of the window display device according to claim 1 to the frame memory control unit of the group window display device according to claim 6, a group window display device with a mixture of moving image windows is created. To do. (Claim 9)

[0018]

EXAMPLE An example of a block diagram of an image communication terminal is shown in FIG. In the figure, 11 is a system control unit, 12 is a line interface unit, 13 is a codec unit, 14 is a moving picture frame memory control unit, 15 is a frame memory control unit, 16 is a combining unit, 17 is a display, 18 is a camera, 19 is An animation window is shown.

The system control unit 11 controls the entire system, and the codec unit 13 band-compresses the video signal from the camera 18 and transfers it to the line interface unit 12.
It is transmitted to the video communication terminal of the communication partner via a network such as SDN. On the other hand, the band compression encoded data received by the line interface unit 12 from the partner terminal is decoded by the codec unit 13, and the moving image frame memory control unit 14
Transferred to.

The system control unit 11 writes the image data generated by the application program in the frame memory control unit 15, and the composition unit 16 is the moving image frame memory control unit.
By combining the display signals from 14 and the frame memory control unit 15, the moving image window 19 can be displayed on the display 17 in an overlapping manner.

The operation of the moving picture frame controller 14 will be described below with reference to the principle block diagram of FIG. Since the other parts are not deeply related to the present invention, description thereof will be omitted. An example of panning (moving vertically and horizontally) and zooming (moving perspective) will be described with reference to FIGS. 4 and 5 below.

An embodiment of the panning function is shown in FIG. In the display of the moving image window on the display, the person is not in the center of the window, so this is an example of panning to the right. Examples (a) and (b) show how to achieve this. Example (a) is a reduction / enlargement ratio of the scaling unit,
The addressing of the write addressing unit is fixed and the address of the reading addressing unit is changed at any time to realize the panning function. The embodiment (b) realizes the panning function by fixing the reduction / enlargement ratio of the scaling unit and the address designation of the read start address designating unit and changing the address of the write start address designating unit at any time. (Claims 2 and 3) An embodiment of the zooming function is shown in FIG. In the video window display on the display, the person is too large, so
This is an example of a case where a reduced display is performed. Examples (c) and (d) show how to achieve this. The embodiment (c) realizes the zooming function by fixing the address designation of the read start address designating section and changing the reduction / enlargement ratio of the scaling section and the address of the write start address designating section in synchronization with each other as needed. . The embodiment (d) realizes the zooming function by fixing the address designation of the write start address designating section and changing the reduction / enlargement ratio of the scaling section and the address of the read start address designating section in synchronization with each other as needed. . (Claim 4,
5) Next, an embodiment of the group window display device shown in FIG. FIG. 6 shows a block diagram of the display control signal generator 32. In the figure, 41 is an effective display address generating unit for generating display addresses in the horizontal direction X and vertical direction Y which can be displayed on the display, and 42 is a display address and a set value from a processor connected to the system bus. Shows a group display area generation unit that generates a display area for each group window, 43 is a display priority sorting switch unit that switches the group window display area signals in descending order of display priority according to an instruction from the processor, 44 is this display priority Of the plurality of valid group window area signals output from the degree sorting switch section, the one having the highest priority is determined, and the corresponding output signal is determined as the display priority determining section, 45 is the display. While the signal output from the priority determination unit is valid, the group number set by the processor The group number register unit for output is shown.

The group window display area signal is a signal indicating the maximum range that can be occupied by the grouped windows, and the area is as shown in (a) of FIG. 7 (a) for explaining the signal of the display control signal generator. In the case of the rectangular area GW0, the signal Sx in the X direction and the signal Sy in the Y direction are combined according to the horizontal synchronizing signal and the vertical synchronizing signal.

The signal output from the display priority sorting switch section 43 shown in FIG. 6 is a signal in which the group window area signals GW1 and GW2 are arranged in the order of display priority, as shown in FIG. 7B, for example. . When the group window area signals GW1 and GW2 overlap, only the group window area signal having the highest priority becomes valid by the display priority determining section 44, as shown in FIG. 7C.
The group number register unit 45 rewrites these group window area signals GW1 and GW2 into group numbers GN1 and GN2 and outputs them as shown in FIG.

The group window display area generating unit 42 shown in FIG. 6 is constructed, for example, as shown in the block diagram of FIG. The group window display area generating unit 42 includes area generating circuits 1 to n for each group. These have the same structure, and the group window display start X
Coordinate register GWSx, end of group window display X
Coordinate register GWEx, group window display start Y
Coordinate register GWSy, end of group window display Y
It has a coordinate register GWEy. The upper left start coordinates and the lower right end coordinates of the rectangular area are set in these registers by the processor via the system bus.

The comparator compares the address generated by the effective display area address generator 41 with the value of the group window display start X coordinate register GWSx, and if they match, the flip-flop is set. If the X address matches the value in the group window display end X coordinate register GWEx, the flip-flop is reset. With respect to Y, the comparator / comparator similarly controls the setting / resetting of the flip-flop. If both flip-flops are set, AN
The group window display address creation area signal GW # i (i = 0 to n) is output via the D gate.

FIG. 9 shows a block diagram of the frame memory controller 33 of FIG. In the figure, 131 is a bus interface unit that interfaces with address / data signals and control signals of the system bus, 132 is a frame memory unit that stores image data, 133 is a refresh address generation unit that generates addresses and control signals at the time of refresh, 134
Is a serial port address signal generator that generates addresses and control signals for the serial port of the frame memory that outputs pixel data, 135 is an address selector that selects an address, 136 is the refresh address generator, and serial port address signal generator And an address switching control unit that arbitrates the timing between address signals created by access from the processor and controls the address selector. 137 is a multiplexing unit that multiplexes the digital pixel data output from the frame memory up to the pixel clock frequency. 138 is a window effective display area signal generation section for generating a signal indicating a display area of a window which is allowed to be displayed from the group window display address generation area signal output from the display control signal generation section 32 of FIG. Has a window 7 shows a gate unit that allows pixel data output from the multiplexing unit to pass through only the effective region by the window effective display region signal output from the effective display region signal generation unit.

Serial port address signal generator 134
Is a display start X coordinate register that sets the display start address of the frame memory that is actually displayed in the window, a display start Y coordinate register, and the start coordinates of the frame memory that is actually displayed by giving an offset value to the set value of the register. It is composed of a display start coordinate offset register and a display start Y coordinate offset register which can be changed.

Next, FIG. 10 shows a block diagram of the window effective display area signal generator 138 of FIG. The group window display area signal from the display control signal generating section 32 of FIG. 2 is input, and the X address in the group window display area is generated by this and the horizontal synchronizing signal and the pixel clock which are the display control data from the pixel data arbitration section 34. G
Generate WX and Y address GWY.

A signal obtained by ANDing the group window display area signal and the pixel clock by an AND gate is input to the clock input of the counter, and the group window display area signal is input to the reset signal, and the counter is set only between the group window display areas. Validate. A signal obtained by ANDing a group window display area signal and a horizontal synchronizing signal by an AND gate and a vertical synchronizing signal as a reset signal are input to the clock input of the counter so that the Y address is reset every vertical synchronizing signal. .

The processor transmits a group window display offset X coordinate register GWOx and a group window display offset Y coordinate register GWO via the system bus.
The offset address value set to y is added to the GWX and GWY signals by an adder, and a comparator,
,,, are input.

On the other hand, via the system bus, the window display start relative X coordinate register WSx, the window display end relative X coordinate register WEx, the window display start relative Y coordinate register WSy, and the window display end relative Y coordinate register WEy are connected to the processor. Sets relative start coordinates and end coordinates in the coordinate system of the group window area.

Output of adder, and registers WSx, W
Ex, WSy, WEy are compared by a comparator ,,, and when the start coordinate is located, the flip-flop F
F is set, and when the end coordinates match, the flip-flop is reset. A window effective display area signal is generated by ANDing the output of the flip-flop with an AND gate. The window effective display area signal is input to the gate 139 of FIG. 9, and causes the pixel data arbitration unit 34 to output the effective period pixel data.

FIG. 11 shows the relationship between the display and the frame memory. An example in which one group window consisting of windows and windows is displayed on the display is shown in (a). The number of pixels beside the display is D
w, the number of vertical pixels is Dh, and the upper left coordinates of the group window in the display coordinate system are [GWSx1, GWS
y1], and the lower right coordinates are [GWEx1, GWEy1]. These values are the group window display start X coordinate register GWSx, the group window display start Y coordinate register GWSy, and the group window display end X coordinate G in the group window display area generator 42 shown in FIG.
WEx and group window display end Y coordinate GWEy are set.

The coordinates at the upper left of the windows forming the group window are [WSx11, WSy11] = (0, in the relative address system in the group window.
0), the lower right coordinate is [WEx11, WEy11], the upper left coordinate of the window is [WSx12, WSy12] in the relative address system in the group window, and the lower right coordinate is [WEx12, WEy12]. The window display start relative X coordinate register WSx, the window display start relative Y coordinate register WSy, the window display end relative X coordinate register WEx, the window provided in the window effective display area signal generation unit 138 in the allocated frame memory control unit 33. The display end relative Y coordinate register WEy is set.

The window and the image data displayed in the window are stored in the frame memory and the frame memory shown in (b) and (c) of FIG. 11, respectively. Window display start address [WCx11, WCy
11] and [WCx12, WCy12] are the display start X coordinate register WCx and the display start Y coordinate register WCy in the serial port address signal generator 134 provided in the frame memory controller 33 to which they are respectively assigned.
Is set to.

FIG. 7 is a flowchart of the above display control process.
Shown in 12. 13 shows the details of the display position determination method and processing of the group window, FIG. 14 shows the display position of the group window (case classification), FIG. 15 shows the details of the display position determination method and processing of the individual window in the group window, and FIG. The display position of the individual window in the group window is shown.

In the group window display position determination processing, the display position of the group window in the absolute display coordinate system of the display is divided into cases as shown in FIG. In the group window display area setting process, based on this, the group window display start X coordinate register GWSx, the group window display end X coordinate register GWEx, the group window display start Y coordinate register GWSy, and the group window display in the group window area generation unit 42 are displayed. A value is set in the end Y coordinate register GWEy.

The window effective display area correction processing is as follows.
Group window display offset X coordinate register GW
A value is set in Ox and the group window display offset Y coordinate register GWOy.

In the window display position determination processing, the display positions of the individual windows constituting the group window are divided into cases as shown in FIG. In the display start coordinate correction processing of the frame memory, based on this, values are set in the display start X coordinate offset register WOx and the display start Y coordinate offset register WOy in the serial port address signal generator 134.

, Are repeated for the number of windows belonging to the same group.

[0042]

As described above, according to the present invention, a movable and expensive camera is used to perform panning and zooming control of a window displaying a partner video by processing the received partner video data. Since it can be realized by using a fixed inexpensive camera, there is an effect that an economical system can be provided.

Further, the display control in which a part of the group window in which a plurality of windows are grouped is cut off from the screen is controlled by the parameters separated from the setting parameters for displaying all the group windows, so that the control is simple. There is an effect that.

Further, by mixing the frames for moving picture display in the frame memories corresponding to the individual windows forming the group window, for example, the effect that the multimedia document including the moving image can be displayed by the group window There is.

[Brief description of drawings]

FIG. 1 is a block diagram showing the principle of a window display control device of the present invention.

FIG. 2 is a principle configuration diagram of a group window display control device of the present invention.

FIG. 3 is an example of a block configuration diagram of an image communication terminal.

[Fig. 4] Example of panning function

FIG. 5 Example of zooming function

FIG. 6 is a block configuration diagram of a display control signal generator.

FIG. 7 is a signal explanatory diagram of a display control signal generator.

FIG. 8 is a block diagram of a group window display area signal generation unit.

FIG. 9 is a block configuration diagram of a frame memory control unit.

FIG. 10 is a block configuration diagram of a window effective display area signal generation unit.

[Fig. 11] Relationship between display and frame memory

[Fig. 12] Display control processing flowchart

[Fig. 13] Details of display position determination method and processing of group window

[Fig.14] Display position of group window (case classification)

[Fig. 15] Details of display position determination method and processing of individual window in group window

[Figure 16] Display position of individual windows in the group window

[Figure 17] Basic operation example of the group window

[Fig.18] Display example of group window

[Explanation of symbols]

1,14 Video frame memory control unit 2 Scaling unit 3 Write address generation unit 4 Video frame memory unit 5 Write control unit 6 Write start address designation unit 7 Read control unit 8 Read start address designation unit 9 Read address generation unit 10 , 11, 31 System control unit 12 Line interface unit 13 Codec unit 15 Frame memory control unit 16 Compositing unit 17 Display 18 Camera 19 Video window 32 Display control signal generation unit 33 _{1 to} 33 _n Frame memory control unit 34 Pixel data arbitration unit 41 Effective display address generation unit 42 Group window display area generation unit 43 Display priority sorting switch unit 44 Display priority determination unit 45 Group number register unit 131 Bus interface unit 132 Frame memory unit 133 Refresh address generation unit 134 Serial port address signal generation unit 135 address selector 136 address Rikae control unit 137 multiplexing unit 138 windows the effective display area signal generator 139 gate portion

Claims (9)

[Claims] 1. A video communication terminal capable of moving picture communication, wherein a moving picture frame memory control section (1) constituting the video communication terminal includes a scaling section (2) for scaling a digital video signal, and a digital video image. A moving picture frame memory unit (4) for storing data, a write address generation unit (3) for generating a write address for the frame memory unit, and a writing for controlling writing of digital video data to the frame memory unit. A control unit (5), a write start address designating unit (6) for designating a start address of a frame memory to write digital video data to the write control unit, and a read address for the frame memory unit are generated. Read address generator (9) and stored digital video data from frame memory It has a read control section (7) for controlling the output and a read start address designating section (8) for designating to the read control means the start address of the frame memory from which the digital video data should be read. By processing the phase manual image data by the scaling unit (2), the write start address designating unit (6), and the read start address designating unit (8), the panning and zooming functions of the window displaying the phase manual image are realized. A window display control device characterized by: 2. The moving picture frame memory control unit (1) changes the reading start address at any time by the reading start address designating unit (8) while the writing start address to the moving picture frame memory unit (4) is fixed. The window display control device according to claim 1, wherein a panning function is realized by doing so. 3. In the moving picture frame memory control unit (1), the write start address is designated by the write start address designating unit (6) at any time while the read start address to the moving picture frame memory unit (4) is fixed. The window display control device according to claim 1, wherein the panning function is realized by changing the panning function. 4. In the moving picture frame memory control unit (1), with the display start address of the moving picture frame memory unit (4) being fixed, the scaling unit (2) enlarges / updates at any time.
2. The window display control device according to claim 1, wherein the zooming function is realized by performing a reduction process and adding an offset to the write address as needed by the write address generating unit (3). 5. In the moving picture frame memory control unit (1), a scaling unit (2) gives an enlargement / reduction instruction to digital video data in a state in which a write start address to the moving picture frame memory unit (4) is fixed. Change at any time,
2. The window display control device according to claim 1, wherein a zooming function is provided by changing the read start address synchronized with the read start address designating unit (8) at any time. 6. A group window display area signal indicating a display area of a group window on which a plurality of windows are grouped, a pixel data selection signal designating pixel data to be displayed on the display, and at least a horizontal synchronization signal. And a display control signal generating section (32) for outputting a display control signal including a vertical synchronizing signal and a clock,
A plurality of frame memory control units (33 _{1 to} 33) that create a display address from the group window display area signal and output the image data from the frame memory that stores the image data.
_n ) and an image data arbitration unit (34) for switching the image data output from the frame memory control unit by an image data selection signal, the display position of the group window on the display is changed. Even if the group window and individual windows belonging to the group window are not changed, the display of the group window is controlled by adding correction means to the group window display control device. 7. In the group window display control device, the frame memory control section (33) includes means for giving an offset to a display address and means for giving a display start coordinate to the frame memory, and a display control signal generating unit is provided. Division (32)
7. The group window display control device according to claim 6, further comprising means for setting the area according to the display position of the group window. 8. In the group window display control device, group window display position determination processing, group window display area setting processing, individual window effective display area setting processing, individual window display position determination processing, individual window display The group window display control device according to claim 6, wherein display control is performed by a start coordinate correction process. 9. The moving picture window in the group window display control device, wherein the frame memory control unit of the window display device according to claim 1 is applied to the frame memory control unit of the group window display device according to claim 6. A group window display control device characterized by creating a mixed group window display device.