JP2000270306A - Image processing unit, its method and served medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a sight line of a video camera in matching with that of a user. SOLUTION: A projector 44 displays a picture on a screen 45 to allow a user 51 to view the picture. The screen 45 is made transparent at a time when no picture is displayed and a video camera 42 photographs the user 51 through the screen 45. The picture photographed by the video camera 42 is sent to an opposite terminal, where a projector 44 displays the picture on the screen 55.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and a providing medium. The present invention relates to an image processing apparatus and method, and a providing medium.

[0002]

2. Description of the Related Art A video telephone system and a video conference system are known as systems for communicating images, voices, data, and the like between a plurality of persons located at remote locations to communicate with each other. These systems are already in practical use and are becoming popular, especially in business applications.

[0003] In these systems, a video camera and a microphone are provided to capture a human face and capture its voice. Further, a monitor and a speaker are provided to output images and sounds transmitted from the other party. Typically, video cameras are located around the monitor (eg, at the top, bottom, or sides).

In a video conference system, when a conference is held by a large number of people, images of the faces of all the participants are tiled on a monitor due to reasons such as cost problems and a narrow office environment. Often displayed side by side. Then, when the number of attendees increases, the size of the image is reduced (the size of one tile is reduced), and images of many participants are displayed.

[0005]

However, when the video camera is arranged around the monitor, the center of the monitor does not coincide with the line of sight of the video camera. As a result, in general, the user looks at the monitor, not the video camera, and talks to the person on the monitor, for example,
When the video camera is installed on the upper part of the monitor, the face of the other party displayed on the monitor becomes an image slightly inclined. When the video camera is installed at the lower part of the monitor, on the contrary, the face of the other party displayed on the monitor becomes an upward and downward image. In either case, the image becomes unnatural and the user may feel uncomfortable.

[0006] Further, when the images of the participants are displayed in a tile shape, it is felt as if all the people are talking to themselves, and it is impossible to obtain a sense of reality as in an actual conference. was there.

Further, with the increase in the number of participants,
When the size of the image is reduced, the face of the participant becomes difficult to see, and there is a problem that natural communication cannot be achieved. The present invention has been made in view of such a situation, and aims to make the line of sight of a participant coincide with the line of sight of a camera.

[0008] The present invention also enables a large number of participants to participate in a conference with a natural sense of reality.

[0009]

An image processing apparatus according to a first aspect of the present invention includes a display unit for displaying an image of another user, an imaging unit for imaging the user via the display unit, and an image processing apparatus. Control means for controlling the display means with respect to the display timing of the image and the timing of the image pickup by the image pickup means via the display means, and transmission of transmitting the image of the user picked up by the image pickup means to another user Means.

According to a fifth aspect of the present invention, in the image processing method, a control step of controlling the display means is performed with respect to a timing of displaying an image on the display means and a timing of imaging by the imaging means via the display means. Transmitting the captured image of the user to another user.

According to a sixth aspect of the present invention, there is provided the providing medium, wherein a control step of controlling the display means is performed with respect to a timing of displaying an image on the display means and a timing of imaging by the imaging means via the display means. And a transmission step of transmitting the image of the user to another user.

According to a seventh aspect of the present invention, in the image processing apparatus, a unit is configured by a predetermined number of terminals, and the unit is disposed at a predetermined position in the virtual space. Control means for controlling the display of an image on each terminal of the unit.

An image processing method according to a twelfth aspect corresponds to an arrangement step in which a unit is constituted by a predetermined number of terminals and the unit is arranged at a predetermined position in a virtual space, and the arrangement by the arrangement step processing. And controlling the image display of each terminal of the unit.

According to a thirteenth aspect of the present invention, there is provided a providing medium in which a unit is constituted by a predetermined number of terminals, and the unit is arranged at a predetermined position in a virtual space, and the arrangement is performed by the arrangement step. And a control step of controlling display of an image on each terminal of the unit.

An image processing apparatus according to claim 1, wherein
In the image processing method described in the item (1) and the providing medium described in the item (6), the user is imaged through the display means.

An image processing apparatus according to claim 7, wherein
In the image processing method according to the second aspect and the providing medium according to the thirteenth aspect, a unit is configured by a predetermined number of terminals, and the unit is arranged at a predetermined position in the virtual space.

[0017]

FIG. 1 is a diagram showing a configuration example of an image processing system to which the present invention is applied. In this configuration example, the plurality of terminals 1-1 to 1-6 are connected to the network 3
Is connected to the master server 2. The network 3 can be configured by an intranet, a LAN (Local Area Network), or the like in addition to the Internet.

FIG. 2 shows a terminal 1 (hereinafter, terminals 1-1 to 1).
When it is not necessary to distinguish -6 individually, it is simply referred to as terminal 1. The same applies to other devices). The detection device 11 detects the direction of the face or the direction of the line of sight of the user 51 shown in FIG. The detection result of the detection device 11 is subjected to A / D conversion by the A / D converter 15 and then input to the arithmetic device 19. The will input device 12
For example, it is constituted by a keyboard, a mouse, a joystick and the like, and is operated by a user as needed. The signal input from the will input device 12 is input to the A / D converter 16.
After the A / D conversion, the data is input to the arithmetic unit 19.

The sound input device 13 is composed of, for example, a microphone and captures a user's voice. The audio signal input from the audio input device 13 is A / D converted by the A / D converter 17 and then input to the arithmetic unit 19. The image input device 14 includes, for example, a video camera,
The image signal input from the image input device 14 is A / D converted by the A / D converter 18 and then input to the arithmetic device 19.

The audio reproducing device 22 is composed of, for example, a speaker. The digital audio signal input from the arithmetic device 19 is D / A converted by the D / A converter 20 and then input. The audio reproducing device 22 outputs the input audio signal. The image display device 23 is, for example, an LCD (Liquid Crystal).
Display) and a projector for projecting an image on the screen.
The digital image data output from the D / A converter 21
Is input after D / A conversion. Video display device 2
3 displays the input video signal. Further, as described later with reference to FIG. 3, the image input device 14 captures an image of the user via the video display device 23. Therefore, the arithmetic device 19 controls the video display device 23 to control the timing at which the image is displayed and the timing at which the image is captured by the image input device 14.

The arithmetic unit 19 can be constituted by, for example, a personal computer or the like. Transceiver 2
Reference numeral 4 includes a modem, for example, and communicates with the master server 2 via the network 3.

FIG. 3 shows an arrangement of each device constituting the terminal 1. The user 51 of the terminal 1 is
The image of the user of the other terminal displayed on the screen 45 is observed by the projector 44 that constitutes. Behind the screen 45, a microphone 41 constituting the sound input device 13, a video camera 42 constituting the image input device 14, and a speaker 4 constituting the audio reproducing device 22
3 are arranged. The microphone 41 is connected to the user 51
The speaker 43 outputs an audio signal from another user to be transmitted to the user 51.

The video camera 42 captures an image of the user 51 via the screen 45. Screen 45 is an LCD
And is controlled by the arithmetic unit 19 so as to perform a time-division operation. That is, the screen 45 is controlled so as not to transmit light when displaying the image of the projector 44, and is controlled so as to transmit light when imaging the user 51 from the video camera 42. Arithmetic unit 1
When the image 9 is displayed on the screen 45 by the projector 44, the imaging by the video camera 42 is stopped, the image emitted from the projector 44 is displayed on the screen 45, and the user 51 is observed. In addition, at the timing when the image of the user 51 is captured by the video camera 42, the arithmetic device 19 stops the operation of the projector 44 and controls the screen 45 to transmit light. Therefore, the video camera 42 is connected to the screen 45.
, An image of the user 51 can be captured.

FIG. 4 shows a configuration example of the master server 2. The input device 31 includes a keyboard, a mouse, and the like, and is operated when an administrator inputs various commands. The arithmetic device 32 executes various processes in response to a command from the input device 31. The storage device 34 is configured by a hard disk, a magnetic disk, a magneto-optical disk, and stores various data and programs executed by the arithmetic device 32. The transmission / reception device 33 is configured by, for example, a modem, is controlled by the arithmetic device 32, and communicates with the terminal 1 via the network 3.

In this example, as shown in FIG. 5, the terminal 1 has a total of four video cameras 42F, 42R, 42L,
42B. Video cameras 42F, 42R, 4
The 2L is disposed behind the screen 45 when viewed from the user 51. The video camera 42F is arranged at a position where the face of the user 51 is imaged from the front, and the video camera 4F
The 2R is arranged at a position where the face of the user 51 is imaged from the right side, and the video camera 42L is arranged at a position where the face of the user 51 is imaged from the left side. Video camera 42B
Is arranged behind the user 51, and images the user 51 from behind.

In each terminal 1, each video camera is similarly arranged.

The arithmetic unit 32 of the master server 2 constitutes one unit by arranging the four terminals 1 in a square shape as shown in FIG. 6 in a virtual space managed by itself. In the example of FIG. 6, the video camera of the terminal 1-1 is arranged at the lower center, the video camera of the terminal 1-2 is arranged at the upper right, and the video camera of the terminal 1-3 is arranged at the upper side. , The video camera of the terminal 1-4 is arranged on the upper left side. Of course, in practice, each terminal 1-
1 to 1-4 are arranged at independent positions in the virtual space, and the arrangement shown in FIG.
This is an arrangement in the virtual space managed by the master server 2 only.

In other words, the arithmetic unit 32 is connected to the terminal 1-1.
As shown in FIG. 7, by arranging the tables 1-4 in this way, a table T and a seat S
The state where 1 to S4 are arranged is managed as one unit. In other words, the arithmetic unit 32 has the terminals 1-1 to 1
-4 seats S1 to S in the virtual space, respectively.
4 and controls the display of images on the screens 45-1 to 45-4 assuming a state of having a meeting.

Further, the arithmetic unit 32 arranges three tables T1 to T3 in the virtual space 60, as shown in FIG. In this example, the table T1 is arranged in the lower center of the virtual space 60, the table T2 is arranged on the right side, and the table T3 is arranged on the left side. In FIG. 8, at the end of the numbers of each video camera and the screen, -1 is added to the table T1, -2 is added to the table T2, and -2 is added to the table T3. -3 is added.

FIG. 9 is a simplified illustration of the virtual space 60 of FIG. 8 managed by the arithmetic unit 32. The table T1 has four seats A to D, the table T2 has four seats E to H, and the table T3 has four seats I to L. When the user of each terminal 1 accesses the master server 2, the user is provided with a table layout diagram of the virtual space 60 as shown in FIG. 9 and designates a seat of a desired table. Attend a meeting in.

For example, when the user of the terminal 1 acquires the seat A of the table T1 as his / her own seat, the arithmetic unit 32 of the master server 2 moves the screen 45 of the terminal 1 as shown in FIG. Display the image. That is, the screen 45 displays the seat A on which the user is sitting.
The images of the users b to d of the other terminals 1 sitting on the other seats B to D of the table T1 are displayed around the table T1. Similarly, around the table T2, images e to h of the users of the terminals 1 sitting in the seats E to H are displayed, and around the table T3, the users sitting in the seats I to L are displayed. Are displayed.

Next, the operation of the video conference system will be described with reference to the flowcharts of FIGS. When the user instructs to attend the video conference system by operating the will input device 12, the arithmetic device 19 of the terminal 1 controls the transmitting / receiving device 24 in step S 1 and the master server 2 via the network 3. , And introduce the current connection status of the video conference system. The arithmetic unit 19 determines whether or not there is a response to the connection status inquiry from the master server 2 in step S2. If there is no response, the process proceeds to step S3, in which the number of connection requests is a predetermined number. It is determined whether it is smaller than the threshold value N. If the number of connection requests is smaller than the threshold value N, the process returns to step S1 to execute the connection status inquiry process again.

As described above, if there is no response to the master server 2 after inquiring the connection status N times, the arithmetic unit 19 of the terminal 1 terminates the processing.

When the arithmetic unit 32 of the master server 2 receives a connection status inquiry request from the terminal 1 via the transmission / reception device 33, the arithmetic unit 32 stores the virtual space 6 stored in the storage unit 34.
The current connection status of “0” is checked to determine whether there is a vacant seat. If there is no vacant seat, the arithmetic unit 32 does not respond to the terminal 1. Therefore, in such a case, the terminal 1 cannot attend the video conference.

When there is a vacant seat in the virtual space 60, the arithmetic unit 32 of the master server 2 generates an image indicating the current connection status in step S4, and transmits the image to the terminal 1 from the transmitting / receiving device 33 to the terminal 1. To send over. When receiving the response from the master server 2 via the transmission / reception device 24, the arithmetic device 19 of the terminal 1 converts the image corresponding to the received data into a D / A in step S4.
The data is supplied to the projector 44 constituting the video processing device 23 via the converter 21 and displayed on the screen 45.
As a result, the positions of the tables and seats are displayed on the screen 45, for example, as shown in FIG. Each seat is displayed so that a vacant seat can be distinguished from a seat where another user is already sitting. In the example of FIG.
Seats indicated by hatching are seats where other users are sitting, and seats not hatched are vacant seats. Therefore, in the example of FIG. 13, the seats A, D, and G are vacant, and the other seats are already used by other users.

The user looks at the display shown in FIG. 13, and in step S5, operates the intention input device 12 to input a seat desired to be obtained from the available seats and requests a login. . For example, when the user designates the seat A, the key of the character A is operated on the keyboard. The arithmetic device 19 transmits data corresponding to the input from the will input device 12 to the master server 2 via the transmission / reception device 24.
Send to

In step S6, the arithmetic unit 19
It is determined whether or not a login OK signal has been received from the master server 2. If a login OK signal has not been received, the process proceeds to step S 7, where the number of connection requests exceeds a predetermined threshold N set in advance. It is determined whether it is small.
If the number of connection requests is smaller than the threshold N, step S
5 and the subsequent processing is repeatedly executed. If the number of connection requests is equal to or greater than the threshold value N, the computing device 19 terminates the processing.

When the arithmetic unit 32 of the master server 2 determines that the login is OK, the process proceeds to step S8, in which the terminal 1 is allocated to the seat designated by the terminal 1 in step S5. Change the connection. Thus, for example, the state in which the new terminal 1 is allocated to the seat A shown in FIG. 13 is the current connection state of the virtual space 60. The arithmetic unit 32 of the master server 2 stores an image corresponding to the connection status in the storage unit 34.
And transmits it to the terminal 1 via the transmission / reception device 33.

The arithmetic unit 19 of the terminal 1 includes a transmitting / receiving unit 24
When this data is received via the PC, a corresponding image is displayed on the screen 45. Thus, in this case, an image as shown in FIG. 14 is displayed on the screen 45 in this case. That is, images i to l of the users of the terminals sitting in the four seats I to L are displayed on the table T3. Around the table T2, images e, of the users sitting in the seats E, F, and H excluding the seat G,
f and h are displayed. In the table T1, images b and c of the users sitting in the seats B and C, respectively, are displayed.
Since the seats D and H are empty, no image is displayed. Images b and c of the seats on the table T1 where you are located
Are displayed larger than the images e, f, h to l of the other tables T2, T3.

When the user attends a video conference, the detection device 11 detects the direction of the user's face or line of sight. The detection result is sent from the arithmetic unit 19 to the transmitting / receiving unit 20.
Is transmitted to the master server 2 via

The arithmetic unit 32 of the master server 2 receives the detection data of the direction from the terminal 1 and determines in step S9 whether the user of the terminal 1 is watching the same direction for a predetermined time or more. Determine whether or not. If the same direction is not watched for a certain period of time or more, the process of step S9 is repeatedly executed.

Which direction the user is facing is
As shown in FIG. 14, the angle θ1 detected as the direction in which the user is facing is converted into the virtual space 6 as shown in FIG.
The determination is made by converting the angle into an angle θ2 above zero.

The arithmetic unit 32 determines that the user 51 of the terminal 1 has been gazing in a predetermined direction for a predetermined time or more.
Proceeding to step S10, the sound volume from the table where the watcher (user watching the predetermined direction) is located is limited. For example, when a user sitting in the seat A of the table T1 watches the seat G of the table T2 for a certain period of time or more, the user judges that the user is not interested in the conversation of the table T1 now, Seat B of table T1
Alternatively, the volume of the sound from the user sitting in the seat C is reduced. Alternatively, the image b of the user sitting on the seat B and the image c of the user sitting on the seat C shown in FIG. Similarly, the volume of the voice of the user of the seat A provided to the terminal 1 of the user sitting on the seats B and C is reduced, and the image a is reduced. Thereby, for example, the user sitting in the seats B and C of the table T1 can know that the user sitting in the seat A is not interested in the conversation right now.

Next, the operation proceeds to step S11, where the arithmetic unit 3
2 causes the image of the watcher on the display image for the user of the watched table to be displayed larger. In this case,
Since the user of the seat A is watching the seat G of the table T2, the image a of the user A viewed from the user using the seats E, F, and H of the table T2 is compared with other images. Display larger.

That is, an image as shown in FIG. 16 is displayed on the screen 45 of the terminal of the user sitting on the seat F of the table T2. The images e and h of the user sitting at the table T2 where they are sitting are displayed larger than the images a to d and i to l of the users sitting at the seats of the distant tables T1 and T3. I have. Seat A
Is also an image of the distant table T1, and unless the user of the seat A gazes at the table T2, the image a is displayed in the same size as the other images b to d. However, when the user of the seat A gazes at the table T2 for a certain period of time or more, the image a of the user of the seat A is indicated by a two-dot chain line in FIG. 16 so that the user of the table T2 can notice this. It is displayed large as shown. At this time, the sound from the seat A may also be output loudly to the user of the table T2. This allows
The user of the table T2 is the user of the seat A.
It can be inferred beforehand that it may move to 2.

It is to be noted that, besides displaying the image a large, it may be displayed brighter or blinked as compared with other displays.

In step S12, the arithmetic unit 32
Determines whether the user is gazing in the same direction for a certain period of time or more. If the user stops gazing halfway, the process returns to step S9, and the subsequent processing is repeatedly executed. If the user remains gazing for a certain period of time, the process proceeds to step S13, and the arithmetic unit 32 determines whether there is an empty seat in the direction in which the user is gazing. If there is no vacant seat, the process returns to step S9, and the subsequent processes are repeatedly executed. If there is a vacant seat, the process proceeds to step S14,
The arithmetic unit 19 of the terminal 1
The same processing as when the movement to the predetermined seat is requested from the will input device 12 is executed. That is, the arithmetic unit 19
Requests the master server 2 to move to the seat G.

When receiving the request for relocation from the computing device 19 of the terminal 1, the computing device 32 of the master server 2 determines in step S15 whether or not the transfer is OK.
If another user has already applied for the acquisition of the seat, this request is rejected, the process returns to step S9, and the subsequent processes are executed. If the seat is vacant, the arithmetic unit 32 of the master server 2 assigns the seat to the terminal 1 in response to the request. Therefore, the process returns to step S8, and the arithmetic unit 32 executes a process of allocating the seat G to the user of the terminal 1.

As a result, on the screen 45 of the terminal 1 having moved to the seat G, the image as shown in FIG. 14 is switched to the image as shown in FIG.

Note that gaze for a certain time or more may be detected at each terminal. Also, the intention to move is
The direction may be performed by a gesture indicated by the user with a finger, or may be performed by a mechanical operation such as a keyboard, a mouse, or a joystick.

In the above, one screen 45 is used, and images of all the video conference attendees in the virtual space 60 are displayed on the screen 45. For example, as shown in FIG. Screen 45B displaying an image of a user sitting on the seat of table T1 with the seat
L, 45BC, and BR are stored in other tables T2 to T4.
May be independent of the screen 45A that displays the image of the user sitting on the seat. By doing so, a more realistic system can be realized.

In the example shown in FIG. 18, the image of the table T1 where one is sitting is centered on oneself.
An image c of the user sitting on the seat opposite to the user is displayed on the front screen 45BC, and an image b of the user sitting on the right seat is displayed on the right screen 45BR. , The screen 45BL on the left side
Is displayed on the screen 45A, for example, the table T is used.
As shown in FIG. 2, images g and h of the user sitting in the two seats G and H are displayed as front images, and images e and f of the user sitting in the two seats E and F in the foreground. Are an image viewed from the right side and an image viewed from the left side, respectively.

As described above, when displaying an image, the arithmetic unit 32 of the master server 2 manages the virtual space 60 as shown in FIG. 19, for example. The image c displayed on the screen 45BC of the table T1 is a video camera 42F-3.
-1, an image b displayed on the screen 45BR is an image captured by the video camera 42L-2-1, and an image d displayed on the screen 45BL is an image captured by the video camera 42R-4-1. And In contrast, for example, the image g of the table T2 is an image captured by the video camera 42F-3-2, the image h is an image captured by the video camera 42F-4-2, and the image e is
The image f may be an image captured by the video camera 42R-1-2, and the image f may be an image captured by the video camera 42L-2-2.

In this way, there is no backward image, and it is possible to confirm the attendee more easily.
Therefore, in this case, the video camera 42B for imaging the user from behind becomes unnecessary.

As shown in FIG. 10, for example, the image of the own table is displayed on one screen 45 similarly to the images of the other tables. If the seat arrangement may be slightly different, the video camera 42B behind each terminal 1 can be omitted.

FIG. 20 shows another example of the arrangement of the projector, video camera, and screen in each terminal 1. In this example, a video camera 42F is arranged in front of the user 51, a video camera 42B is arranged in the rear, a video camera 42L is arranged in the front left side, and a video camera 42R is arranged in the front right side. This is the same as the case in FIG. 5, but in this example, the screen 45 is a surround screen and is arranged so as to surround the user 51. Then, a projector 44A is arranged in front of the user 51, a projector 44B is arranged on the left rear side, and a projector 44C is arranged on the right rear side so that an image can be displayed on the surround screen. As a result, an image is displayed on the surround screen at least in a range that can be visually recognized by the user 51. By doing so, a more realistic system can be realized.

In the above, the case where the present invention is applied to a video conference system has been described as an example. However, the present invention can be applied to a party other than a video conference, for example, a party in a virtual space.

As described above, at least the image of the user of the table on which the user is located is displayed larger than the images of the users located on other tables, so that even if the number of tables increases. At least, the image of the user of the table on which he or she is located can be surely confirmed. In addition, since it is possible to move to another table in the virtual space, even if an image of another table is displayed small, by moving to that table, the face of the attendee of that table is displayed large. And confirm it. Therefore, as in the case of an actual conference or party, communication with a large number of people can be easily realized.

In the above description, the screen 45 is operated in a time-sharing manner in order to realize a translucent screen. For example, as shown in FIG.
Can be a translucent screen such as a half mirror, with one surface being a diffuse reflection surface 81 and the other being a translucent surface 82. In this case, as shown in FIG. 21, the irregular reflection surface 81 is arranged on the projector 44 side,
The translucent surface 82 is arranged on the video camera 42 side.

On the irregular reflection surface 81, as shown in FIG.
The square irregular reflection portions 91 and the transparent portions 92 are arranged in a checkered pattern. The surface of the irregular reflection section 91 is processed so as to irregularly reflect incident light. The transparent section 92 is configured to transmit the incident light as it is.

The opposite side of the diffuse reflection section 91 (video camera 42
23) is a black portion 93 as shown in FIG. The black portion 93 is colored black. The irregular reflection portion 91 and the transparent portion 92, and therefore, the black portion 93 and the transparent portion 9
2 is a sufficiently small area.

When the light emitted from the projector 44 irradiates the irregular reflection surface 81, the light is irregularly reflected by the irregular reflection portion 91, and the user 51 can visually recognize an image constituted by the irregularly reflected light. Can be. The light incident on the transparent portion 92 of the irregular reflection surface 81 is directly transmitted to the video camera 4.
Transmitted to two sides.

On the other hand, the video camera 42 captures an image of the user 51 via the transparent section 92. Since the video camera 42 is focused on the user 51 instead of the screen 45, the video camera 42 can capture an image of the user 51, although the image is slightly dark. Transparent part 9
The light transmitted from the projector 44 through the projector 2 can be substantially restricted by providing the video camera 42 with directivity or providing a lens hood.

In the example shown in FIGS. 22 and 23, the irregular reflection portion 91, the black portion 93, and the transparent portion 92 are each formed in a square shape and arranged in a checkered pattern.
As shown in FIG. 4, the irregular reflection portion 91 (black portion 93) is formed in a hexagonal shape, and the transparent portion 92 is formed as the irregular reflection portion 91 (black portion 93).
May be formed around the periphery.

Further, as shown in FIG.
(Black portion 93) is formed in a square shape, and a transparent portion 92
May be arranged. In the present specification, the term “system” refers to an entire device including a plurality of devices. In addition, as a providing medium for providing a user with a computer program that performs the above-described processing, a recording medium such as a magnetic disk, a CD-ROM, and a solid-state memory, as well as a communication medium such as a network and a satellite can be used. .

[0066]

As described above, the image processing apparatus according to the first aspect, the image processing method according to the fifth aspect, and the sixth aspect.
According to the provision medium described in (1), the user is imaged via the display means, so that the user's line of sight can be matched with the line of sight of the imaging means.

An image processing apparatus according to claim 7, wherein
According to the image processing method described in Item 2 and the providing medium described in Item 13, a unit constituted by a predetermined number of terminals is arranged at a predetermined position in a virtual space, and each terminal of the unit is arranged. Is controlled so that a large number of people can communicate with each other in the same manner as in a real space.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of a system to which the present invention is applied.

FIG. 2 is a block diagram illustrating a configuration example of a terminal in FIG. 1;

FIG. 3 is a diagram illustrating an arrangement of each device constituting the terminal of FIG. 2;

FIG. 4 is a block diagram illustrating a configuration example of a master server in FIG. 1;

FIG. 5 is a diagram illustrating a planar arrangement of each device constituting the terminal of FIG. 2;

FIG. 6 is a diagram illustrating a configuration of a unit.

FIG. 7 is a diagram illustrating a configuration of a unit in a virtual space.

FIG. 8 is a diagram illustrating an arrangement of units in a virtual space.

FIG. 9 is a diagram illustrating an arrangement of units in a virtual space.

FIG. 10 is a diagram illustrating a display example of a screen.

FIG. 11 is a flowchart illustrating the operation of the system in FIG. 1;

FIG. 12 is a flowchart illustrating the operation of the system in FIG. 1;

FIG. 13 is a diagram illustrating a display example of a screen.

FIG. 14 is a diagram illustrating gaze of a screen.

FIG. 15 is a diagram illustrating gaze in a virtual space.

FIG. 16 is a diagram illustrating a display example of a screen on the terminal side that is watched.

FIG. 17 is a diagram illustrating a display example of a screen after moving in a virtual space.

FIG. 18 is a diagram showing another display example of the screen.

FIG. 19 is a diagram illustrating an arrangement of units in a virtual space for realizing the display example of FIG. 18;

FIG. 20 is a diagram showing another example of a planar arrangement of each device constituting the terminal.

FIG. 21 is a diagram showing another example of the screen.

FIG. 22 is a diagram showing a configuration of a diffuse reflection surface of the screen of FIG. 21.

FIG. 23 is a diagram showing a configuration of a translucent surface of the screen of FIG. 21.

24 is a diagram illustrating another configuration example of the screen in FIG. 21.

25 is a diagram illustrating another configuration example of the screen in FIG. 21.

[Explanation of symbols]

1-1 to 1-6 terminals, 2 master server, 3
Network, 11 detection devices, 14 image input devices, 22 audio playback devices, 23 video display devices,
19 arithmetic units, 32 arithmetic units, 34 storage units, 41 microphones, 42 video cameras,
43 speaker, 44 projector, 45 screen

Claims (13)

[Claims] A display unit that displays an image of another user; an imaging unit that captures an image of the user via the display unit; a display timing of an image on the display unit; A control unit that controls the display unit, and a transmission unit that transmits an image of the user captured by the imaging unit to the other user. Image processing apparatus. 2. The image processing apparatus according to claim 1, wherein the image capturing unit captures images of at least the front, left side, and right side of the user. 3. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus that is connected to another image processing apparatus via a network and forms a virtual space. 4. The image processing apparatus according to claim 3, wherein the display unit displays the image of the user assuming that the user is located in the virtual space. 5. An image processing method for an image processing apparatus, comprising: display means for displaying an image of another user; and imaging means for imaging a user via the display means. And a control step of controlling the display unit with respect to the timing of the imaging by the imaging unit via the display unit; and transmitting the image of the user captured by the imaging unit to the other user. Transmitting the image data. 6. An image processing apparatus comprising: display means for displaying an image of another user; and imaging means for imaging the user via the display means; A control step of controlling the display unit with respect to a timing of imaging by the imaging unit via the display unit; and a transmitting step of transmitting an image of the user captured by the imaging unit to the other user. And a computer-readable program for executing a process including: 7. An image which is connected to a network and provides a virtual space, together with a plurality of terminals including display means for displaying an image of another user, and imaging means for imaging a user via the display means. In the processing device, a unit is configured by a predetermined number of the terminals, an arrangement unit that arranges the unit at a predetermined position in the virtual space, and an image of each terminal of the unit corresponding to the arrangement by the arrangement unit. Control means for controlling display of the image. 8. The image processing apparatus according to claim 7, further comprising moving means for moving the terminal in the virtual space. 9. The image processing apparatus according to claim 7, wherein the moving unit detects a movement input of a user of the terminal, and moves the terminal based on a detection result. 10. The apparatus according to claim 9, wherein the movement unit detects the movement input from a line of sight, a face direction, a gesture, or a mechanical operation input of a predetermined position of a user of the terminal. An image processing apparatus according to claim 1. 11. The image processing apparatus according to claim 9, wherein the control unit controls an image or a sound of the unit at a movement destination based on a detection result of the movement input. 12. An image which is connected to a network and provides a virtual space, together with a plurality of terminals including display means for displaying an image of another user, and imaging means for imaging a user via the display means. In the image processing method of the processing apparatus, a unit is configured by a predetermined number of the terminals, and an arrangement step of arranging the unit at a predetermined position in a virtual space; Controlling a display of an image of each terminal of the unit. 13. An image connected to a network and provided with a plurality of terminals including display means for displaying an image of another user and imaging means for imaging the user via the display means, and providing a virtual space. A processing device, a unit is configured by a predetermined number of the terminals, an arrangement step of arranging the unit at a predetermined position in the virtual space, and a terminal of the unit corresponding to the arrangement by the processing of the arrangement step. Providing a computer-readable program for executing a process including a control step of controlling display of an image.