JP2010213133A - Conference terminal device, display control method, and display control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a conference terminal device which can display a photographed image or an avatar by switching on a display according to the size of a display region, to provide a display control method, and to provide a display control program. <P>SOLUTION: In the conference terminal device, a plurality of photographed image are received from another conference terminal device. A display area (first parameter) when the received photographed images are arranged side by side on the display and a resolution thereof (second parameter) are obtained (S55, S57). When the obtained first parameter is equal to or greater than a first threshold value (S59:YES) and the second parameter is equal to or greater than a second threshold value (S61:YES), the photographed image is selected (S63). When each parameter does not meet a condition (S59:NO, S61:NO), the avatar image created based on the photographed image is selected (S65). The selected image is displayed on the display (S69). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a conference terminal device, a display control method, and a display control program. More specifically, the present invention relates to a conference terminal device, a display control method, and a display control program that can switch a camera image and an avatar image and display them on a display.

  Usually, in a video conference system, a conference participant is photographed by a camera. The captured image captured by the camera is transmitted to the video conference terminal installed at each base. The video conference terminal that has received the captured image displays the received captured image on the display. A user of the video conference system can visually recognize a captured image captured by a video conference terminal installed at another base via a display. In this way, the video conference is executed. In addition, when a video conference is performed between a plurality of video conference terminals, the captured images are displayed side by side on the display.

  Here, when the number of video conference terminals participating in the video conference increases, a large number of captured images must be displayed on the display, so that a display area where one captured image can be displayed becomes small. For this reason, the visibility of each captured image deteriorates. Under such circumstances, there has been proposed a video conference system that performs a video conference by displaying an avatar created based on a captured image of a conference participant on a display instead of a captured image (for example, Patent Document 1). reference). Even when a normal avatar is displayed in a limited display area, it does not deteriorate the visibility of the user. Therefore, even when the number of video conference terminals participating in the video conference increases and the display area where one captured image can be displayed becomes small, the visibility of the user can be maintained well.

JP 2000-244886 A

  However, in the above video conference system, the avatar is always displayed on the display. Therefore, even if the number of video conference terminals participating in the video conference is small and a large display area can be secured and the captured image itself can be satisfactorily displayed on the display, the avatar is always displayed. There is a problem.

  The present invention has been made to solve the above-described problems, and a conference terminal device and a display control method capable of switching a captured image and an avatar to be displayed on a display according to the size of a display area. An object of the present invention is to provide a display control program.

  In order to solve the above-mentioned problem, the conference terminal device of the invention according to claim 1 is a conference terminal device that is connectable to a network and performs a remote conference by communicating via the network, A first reception control unit that receives a captured image that is an image captured by a capturing unit of another conference terminal device, and an avatar image that is a character image is added to the captured image received by the first reception control unit. Of the display area of the display means for displaying the avatar assignment means to be assigned and the photographed image received by the first reception control means or the avatar image assigned by the avatar assignment means, the photographed image or the avatar Region parameter acquisition means for acquiring parameters related to an area where an image can be displayed, and the area parameter acquisition means On the basis of the parameters, an image selection unit that selects any one of the captured image and the avatar image as a display image to be displayed on the display unit, and the display image selected by the image selection unit, Display control means for displaying in a displayable area of the display area of the display means.

  In addition to the configuration of the invention according to claim 1, the conference parameter device of the invention according to claim 2 is characterized in that the region parameter acquisition means sets a parameter of an area of a region where the display image can be displayed as a first parameter. And the image selection means selects the display image to be displayed on the display means based on the first parameter.

  In addition to the configuration of the invention described in claim 2, the conference terminal device of the invention according to claim 3 is characterized in that the region parameter acquisition means sets a second parameter for the resolution parameter of the region in which the display image can be displayed. And the image selection means selects the display image to be displayed on the display means based on the first parameter and the second parameter.

  According to a fourth aspect of the present invention, there is provided the conference terminal device according to the second aspect of the invention, wherein the region parameter acquisition means includes at least one of brightness and saturation of a region in which the display image can be displayed. One of the parameters is acquired as a second parameter, and the image selection unit selects the display image to be displayed on the display unit based on the first parameter and the second parameter.

  According to a fifth aspect of the present invention, in the conference terminal device according to the second aspect, in addition to the configuration of the second aspect of the invention, the image selection means has the first parameter acquired by the area parameter acquisition means predetermined. Selecting the captured image as the display image when the threshold value is equal to or greater than a first threshold value, and selecting the avatar image as the display image when the first parameter is less than the first threshold value. Features.

  According to a sixth aspect of the present invention, there is provided the conference terminal device according to the third aspect, wherein the first parameter acquired by the area parameter acquisition unit is the image selection unit. When the second parameter is equal to or greater than a second threshold that is a predetermined threshold that is equal to or greater than a first threshold that is a predetermined threshold, the captured image is selected as the display image, and the first parameter is the The avatar image is selected as the display image when it is less than a first threshold value or when the second parameter is less than the second threshold value.

  According to a seventh aspect of the present invention, in the conference terminal device according to the fifth aspect, in addition to the configuration of the fifth aspect of the invention, the first parameter acquired by the area parameter acquiring unit is a predetermined value. When the first image is equal to or greater than the first threshold value, the captured image is selected as the display image, and the state where the first parameter is equal to or greater than the first threshold value is equal to or greater than the first time. When not continuous, the photographed image is selected as the avatar image.

  According to an eighth aspect of the present invention, there is provided the conference terminal device according to the sixth aspect of the invention, in addition to the configuration of the sixth aspect of the invention, wherein the first parameter acquired by the area parameter acquiring unit is a predetermined value. When the second parameter is equal to or greater than the first threshold continuously for a first time that is a time, and the second parameter is equal to or greater than the second threshold for a second time that is a predetermined time, The photographed image is selected as the display image, and the state in which the first parameter is not less than the first threshold is not continuous for the first time or the state in which the second parameter is not less than the second threshold. The avatar image is selected as the display image when it does not continue for the second time or longer.

  In addition to the configuration of the invention according to any one of claims 1 to 8, the conference terminal device according to the invention according to claim 9 is a face included in the photographed image received by the first reception control means. The facial expression specifying means for specifying the facial expression and the avatar image assigned by the avatar assigning means are converted based on the facial expression specified by the facial expression specifying means. Avatar conversion means, wherein the image selection means selects either the photographed image or the avatar image converted by the avatar conversion means as a display image to be displayed on the display means. And

  A conference terminal device according to a tenth aspect of the invention, in addition to the configuration of the invention according to any one of the first to ninth aspects, receives the avatar image transmitted from the other conference terminal device. A reception control unit, wherein the region parameter acquisition unit acquires the parameter related to a region in which the captured image or the avatar image received by the second reception control unit can be displayed, and the image selection unit includes: And selecting one of the captured image and the avatar image received by the second reception control unit as the display image based on the parameter acquired by the region parameter acquisition unit. To do.

  In addition to the configuration of the invention of claim 10, the conference terminal device of the invention according to claim 11 provides a request notification which is a notification for requesting the avatar image to the other conference terminal device. Transmission control means for transmitting, wherein the second reception control means receives the avatar image transmitted from the other conference terminal device in response to the request notification transmitted by the transmission control means. And

  A display control method according to a twelfth aspect of the present invention is a display control method capable of connecting to a network and performing a remote conference by communicating via the network, and is a photographing means of another conference terminal device A first reception control step of receiving a photographed image that is a photographed image at the first reception control step, an avatar assignment step of assigning an avatar image that is a character image to the photographed image received by the first reception control step, and the first Of the display area of the display means for displaying the photographed image received in the one reception control step or the avatar image assigned in the avatar assignment step, related to the area where the photographed image or the avatar image can be displayed. Region parameter acquisition step for acquiring parameters to be performed, and region parameter acquisition step Based on the obtained parameters, an image selection step for selecting either the captured image or the avatar image as a display image to be displayed on the display means, and the display image selected in the image selection step A display control step of displaying in a displayable area of the display area of the display means.

  A display control program according to a thirteenth aspect of the invention drives a computer as each processing means in the conference terminal device according to any of the first to eleventh aspects.

  In the conference terminal device according to the first aspect of the present invention, the display means displays either a captured image captured and transmitted by another conference terminal device or an avatar image assigned to the received captured image. It selects as a display image and displays on a display means. One of the captured image and the avatar image is selected based on the parameter related to the region where the display image can be displayed among the display regions of the display means. As a result, it is possible to select a display image with good visibility for the user and display it on the display means.

  Moreover, in the conference terminal device of the invention according to claim 2, in addition to the effect of the invention of claim 1, the area of the region where the display image can be displayed is acquired as the first parameter. And based on the acquired area, either a picked-up image and an avatar image are selected as a display image displayed on a display means. As a result, a display image that is optimal for the area of the displayable region can be selected from either the photographed image or the avatar image and displayed on the display means. As a result, it is possible to display on the display means a display image having good visibility for the user.

  Moreover, in the conference terminal device of the invention according to claim 3, in addition to the effect of the invention of claim 2, the resolution of the area where the display image can be displayed is acquired as the second parameter. And based on the acquired 1st parameter and 2nd parameter, either a picked-up image and an avatar image are selected as a display image displayed on a display means. As a result, a display image that is optimal for the resolution of the displayable region can be selected from the photographed image and the avatar image and displayed on the display means. As a result, it is possible to display on the display means a display image having good visibility for the user.

  Moreover, in the conference terminal device of the invention according to claim 4, in addition to the effect of the invention of claim 2, at least one of brightness and saturation of a region where the display image can be displayed is acquired as the second parameter. To do. And based on the acquired 1st parameter and 2nd parameter, either a picked-up image and an avatar image are selected as a display image displayed on a display means. As a result, a display image that is optimal for at least one of the brightness and saturation of the displayable region can be selected from either the photographed image or the avatar image and displayed on the display means. As a result, it is possible to display on the display means a display image having good visibility for the user.

  Moreover, in the conference terminal device of the invention according to claim 5, in addition to the effect of the invention according to claim 2, when the area of the displayable area is acquired as a parameter, the captured image is displayed when the area is large. When an area is small, an avatar image can be selected. As a result, it is possible to display on the display means a display image having good visibility for the user.

  Moreover, in the conference terminal device of the invention according to claim 6, in addition to the effect of the invention according to claim 3 or 4, when the resolution of the displayable area is acquired as a parameter, the image is taken when the resolution is large. When an image is selected and the resolution is small, an avatar image can be selected. When at least one of brightness and saturation of the displayable area is acquired as a parameter, a captured image is selected when the selected value is high, and an avatar image is selected when the selected value is low . As a result, it is possible to display on the display means a display image having good visibility for the user.

  Moreover, in the conference terminal device of the invention according to claim 7, in addition to the effect of the invention according to claim 5, the display image is displayed when the first parameter continuously exceeds the first threshold for the first time or more. Select the captured image as. Accordingly, it is possible to prevent a state in which the display image is frequently switched between the captured image and the avatar image due to a change in the parameter.

  Moreover, in the conference terminal device of the invention according to claim 8, in addition to the effect of the invention of claim 6, the first parameter is continuously equal to or greater than the first threshold for the first time, When the second parameter is continuously equal to or greater than the second threshold for the second time or longer, the captured image is selected as the display image. Accordingly, it is possible to prevent a state in which the display image is frequently switched between the captured image and the avatar image due to a change in the parameter.

  Moreover, in the conference terminal device of the invention according to claim 9, in addition to the effect of the invention according to any one of claims 1 to 8, the avatar image can be assigned based on the facial expression included in the photographed image. It becomes possible.

  Moreover, in the conference terminal device of the invention according to claim 10, in addition to the effect of the invention according to any one of claims 1 to 9, the conference terminal device receives a captured image and an avatar image from another conference terminal device. Receive. Then, based on the parameters acquired by the area parameter acquisition means, either the photographed image or the avatar image is selected as a display image and displayed on the display means. This eliminates the time required for the process of converting a person image into an avatar image, so that the avatar image can be quickly displayed on the display means.

  Moreover, in the conference terminal device of the invention according to claim 11, in addition to the effect of the invention of claim 10, the conference terminal device may request other conference terminal devices to transmit an avatar image. It becomes possible. This makes it possible to request conversion and transmission of an avatar image from another conference terminal device only when an avatar image is necessary, thereby reducing the amount of communication data. Moreover, it becomes possible to reduce the processing load of another conference terminal device.

  In the display control method of the invention according to claim 12, any one of a photographed image captured and transmitted by another conference terminal device and an avatar image assigned to the received photographed image is used as a display means. A display image to be displayed is selected and displayed on the display means. One of the captured image and the avatar image is selected based on the parameter related to the region where the display image can be displayed among the display regions of the display means. As a result, it is possible to select a display image with good visibility for the user and display it on the display means.

  In the display control program of the invention according to claim 13, a computer can be driven as each processing means in the conference terminal device according to any of claims 1 to 11.

1 is a schematic diagram showing a configuration of a video conference system 1. FIG. 3 is a schematic diagram showing an electrical configuration of a conference terminal device 3. FIG. 3 is a schematic diagram showing various storage areas of an HDD 31. FIG. 3 is a schematic diagram showing a login table 3111. FIG. 10 is a schematic diagram showing a display image information table 3121. FIG. It is a flowchart which shows a picked-up image transmission process. It is a flowchart which shows a picked-up image reception process. It is a flowchart which shows an avatar allocation process. It is a flowchart which shows a display process. It is an example of a display screen. It is an example of a display screen. It is a flowchart which shows the avatar allocation process in a 1st modification. It is a flowchart which shows the display process in a 2nd modification. It is a flowchart which shows the avatar preparation instruction process in a 3rd modification. It is a flowchart which shows the picked-up image transmission process in a 3rd modification. It is a flowchart which shows the picked-up image reception process in a 3rd modification.

  Hereinafter, conference terminal devices 3 to 6 according to an embodiment of the present invention will be described with reference to the drawings. These drawings are used for explaining the technical features that can be adopted by the present invention, and the configuration of the apparatus described, the flowcharts of various processes, etc., unless otherwise specified. It is not intended to be limited to that, but merely an illustrative example.

  With reference to FIG. 1, the structure of the video conference system 1 which has the conference terminal devices 3-6 as a component is demonstrated. The video conference system 1 includes a network 2 and a plurality of conference terminal devices 3, 4, 5, and 6 that are connected to each other via the network 2 and are provided for each base. In the video conference system 1, a video conference is performed by transmitting and receiving images and sounds between the conference terminal devices 3 to 6 via the network 2.

  A person who wishes to participate in the video conference logs in to the video conference by operating one of the conference terminal devices 3 to 6 and inputting ID information. In the state of participating in the video conference, for example, the conference participant is photographed by the camera 34 (see FIG. 2) connected to the conference terminal device 3. The captured images of the conference participants (hereinafter referred to as “captured images”) are transmitted from the conference terminal device 3 to the conference terminal devices 4 to 6. In the conference terminal devices 4 to 6 that have received the captured image, the received captured image is displayed on the display 28. When captured images are received from a plurality of conference terminal devices, the received plurality of captured images are displayed side by side on the display 28. Thereby, the conference participant can hold the video conference while visually recognizing the captured images of the other conference participants captured by each of the plurality of conference terminal devices.

  In the video conference system 1 described above, when the number of conference terminal devices that log in to a video conference increases and the number of captured images to be received increases, all of these must be displayed side by side on the display 28. For this reason, the size of the display area of each captured image becomes small. In this case, the visibility of the captured image is deteriorated. In this embodiment, when the visibility of the captured image deteriorates, the image displayed on the display 28 is switched from the captured image to the avatar image. The avatar image is a character image created based on the captured image (details will be described later). The avatar image is composed of relatively few pixels (for example, 100 × 100 pixels). Therefore, the avatar image can be viewed well even when the display area is small compared to the captured image. The present embodiment is characterized in that an image to be displayed is switched between a captured image and an avatar image according to parameters such as the size of the display area of the captured image.

  In the above description, a person who wishes to participate in a video conference logs in to the video conference by inputting ID information. However, the present invention is not limited to this method, and various other methods can be used. For example, a method of logging in by causing the reader to read the RFID can be used.

  The electrical configuration of the conference terminal device 3 will be described with reference to FIG. Since all the conference terminal devices 3 to 6 have the same configuration, only the configuration of the conference terminal device 3 will be described here, and the description of the other conference terminal devices 4 to 6 will be omitted.

  The conference terminal device 3 is provided with a CPU 20 as a controller that controls the conference terminal device 3. Connected to the CPU 20 are a ROM 21 that stores BIOS, a RAM 22 that temporarily stores various data, and an I / O interface 30 that mediates data transfer. A hard disk drive 31 (hereinafter referred to as “HDD 31”) having various storage areas is connected to the I / O interface 30.

  The I / O interface 30 captures a communication device 25 for communicating with the network 2, a mouse 27, a video controller 23, a key controller 24, a camera 34 for photographing a user, and a user's voice. A microphone 35 and a CD-ROM drive 26 are connected to each other. A display 28 is connected to the video controller 23. A keyboard 29 is connected to the key controller 24. The CD-ROM 114 inserted into the CD-ROM drive 26 stores the main program of the conference terminal device 3 and the communication control program of the present invention. When the CD-ROM 114 is introduced, these various programs are set up from the CD-ROM 114 to the HDD 31 and stored in a program storage area 314 (see FIG. 3) described later.

  Various storage areas of the HDD 31 will be described with reference to FIG. The HDD 31 is provided with at least a login table storage area 311, a display image information table storage area 312, an image storage area 313, a program storage area 314, and other information storage areas 315.

  The log-in table storage area 311 stores a log-in table 3111 (see FIG. 4) that manages conference participants who are logged-in to the video conference and conference terminal devices used by the conference participants. The display image information table storage area 312 stores a display image information table 3121 (see FIG. 5) that stores information related to a captured image to be displayed on the display 28 (hereinafter referred to as “display image information”). The image storage area 313 stores a photographed image photographed by the camera 34 and an avatar image created based on the photographed image. The program storage area 314 stores a main program of the conference terminal device 3 and a communication control program for executing a video conference with the other conference terminal devices 4 to 6. The other information storage area 315 stores other information used in the conference terminal device 3. When the conference terminal device 3 is a dedicated machine that does not include the HDD 31, various programs are stored in the ROM 21.

  The login table 3111 will be described with reference to FIG. The login table 3111 stores the user ID of the conference participant who logged in to the video conference and the terminal ID of the conference terminal device 3 used by the conference participant.

  As the user ID, ID information input via the keyboard 29 of the conference terminal device 3 at the time of login is stored. In the terminal ID, identification information for specifying the conference terminal devices 3 to 6 is stored. As the terminal ID, the MAC address or IP address of the conference terminal device 3 can be used. The user ID and the terminal ID are included in the terminal information transmitted from the conference terminal devices 3 to 6 when logging in to the video conference. The conference terminal device 3 extracts the user ID and the terminal ID included in the terminal information when it receives the above-described terminal information from the other conference terminal devices 4 to 6 while it is logged in to the video conference. . The extracted user ID and terminal ID are stored in the login table 3111.

  In the example shown in FIG. 4, the person specified by the user ID “A0001” (terminal ID: 0001), the person specified by the user ID “B0001” (terminal ID: 0002), and the user ID “C0001”. A login table 3111 in a state in which a total of four persons identified by the identified person (terminal ID: 0003) and the person identified by the user ID “D0001” (terminal ID: 0004) participated in the video conference is shown. Note that the conference terminal device identified by the terminal ID: 0001 corresponds to the conference terminal device 3 in FIG. Similarly, the conference terminal device specified by the terminal ID: 0002 corresponds to the conference terminal device 4. The conference terminal device specified by the terminal ID: 0003 corresponds to the conference terminal device 5. The conference terminal device identified by the terminal ID: 0004 corresponds to the conference terminal device 6.

  The display image information table 3121 will be described with reference to FIG. The display image information table 3121 includes a “user ID”, a file name of the photographed image (corresponding to “photographed image” in the figure), a file name of the avatar image (corresponding to “avatar image” in the figure), a photographed image, and an avatar image. "Selection flag" indicating which of the images is displayed on the display 28, "display area" of the display area of the captured image or avatar image on the display 28, "display location" of the captured image or avatar image on the display 28, display Information on the resolution of the display area of the captured image or avatar image in 28 (corresponding to “display area parameter (resolution)” in the figure) is stored.

  As the “user ID”, ID information (ID information input at the time of login) of a conference participant who is participating in a video conference using the conference terminal devices 4 to 6 in which the captured images are captured is stored. As the “captured image”, the file name of the captured image received from the other conference terminal devices 4 to 6 is stored. As the “avatar image”, the file name of the avatar image created based on the photographed image is stored. As the “selection flag”, flag information for specifying either a photographed image or an avatar image is stored. In the example illustrated in FIG. 5, “avatar image” is shown as a state in which flag information for specifying an avatar image is stored as a selection flag.

  As the “display area”, the display area of the display area of the captured image or the avatar image on the display 28 is stored as “length in the vertical direction × length in the horizontal direction”. As the “display location”, information on the upper left position of the window for displaying the captured image or the avatar image on the display 28 is based on the length from the origin when the upper left of the entire display 28 is the origin (0, 0). Stored as coordinate information. As the “display area parameter (resolution)”, the resolution of the display area of the captured image or the avatar image is stored.

  In the example shown in FIG. 5, a total of three “avatar images” of user ID: B0001 (conference terminal device 4), user ID: C0001 (conference terminal device 5), and user ID: D0001 (conference terminal device 6) are displayed. 28 shows a display image information table 3121 of the conference terminal device 3 in the state displayed on the screen 28. Among these, for example, the file name of the captured image received from the conference terminal device 4 is “BBBB.cif”. The file name of the avatar image created based on this photographed image is “BBBB.jpg”. The avatar image is displayed on the display 28 with a display area of “10 cm × 10 cm”. The upper left position of the displayed window is the same as the upper left position of the display 28. The resolution of the display area where the avatar image is displayed is “256 × 256”.

  Various processes (captured image transmission process and captured image reception process) executed by the CPU 20 of the conference terminal device 3 will be described with reference to FIGS. These processes are started and executed by the CPU 20 when a user logs in to the video conference in the conference terminal device 3 and an application for executing the video conference is started. Note that these processes are appropriately switched and executed based on a predetermined cycle or a priority set in advance for each process. This avoids a state where one process is continuously executed and the other process is not executed. When the other process is executed while one process is being executed, the process in the execution state is temporarily interrupted. The other process is executed instead. The interrupted process is restarted from the point of interruption at the next execution timing.

  The captured image transmission process will be described with reference to FIG. In the captured image transmission process, a process for transmitting the captured image captured by the camera 34 to the other conference terminal devices 4 to 6 is executed.

  As shown in FIG. 6, when the photographed image transmission process is activated, it is determined whether the camera 34 is operating and photographing a conference participant or the like (S11). If the camera 34 is not in operation and no conference participants are photographed (S11: NO), the process returns to S11 without performing any special processing, and the camera 34 is continuously monitored.

  When the camera 34 is operating and a conference participant or the like is being captured (S11: YES), a captured image is acquired from the operating camera 34 (S13). The login table 3111 stored in the login table storage area 311 is referred to, and the other conference terminal devices 4 to 6 logged in to the video conference are specified. And the acquired picked-up image is transmitted with respect to the other conference terminal devices 4-6 which have logged in to the video conference (S15). Next, in order to wait until the transmission timing of the next photographed image, a predetermined time (for example, 1 ms) is waited (S17). After a predetermined time has elapsed, the process returns to S11 and the above-described processing is repeatedly executed. In this way, when the camera 34 is in operation, the captured image is repeatedly transmitted to the conference terminal devices 4 to 6 at a constant period.

  The captured image reception process will be described with reference to FIG. In the captured image reception process, an avatar image is created when a captured image is received from the conference terminal devices 4 to 6. Then, one of the photographed image and the avatar image is selected and displayed on the display 28.

  As shown in FIG. 7, when the captured image reception process is started, it is determined whether captured images are received from other conference terminal devices 4 to 6 via the network 2 (S21). When the captured image has not been received (S21: NO), the process returns to S21 without performing any special processing, and reception monitoring of the captured image is continuously performed.

  When the captured image is received (S21: YES), the received captured image is stored in the image storage area 313 (S23). Next, information related to the received captured image is stored in the display image information table 3121 (S25). Specifically, it is information included in the received captured image, and information on user IDs of conference participants who log in to the video conference by operating the conference terminal devices 4 to 6 as the transmission source is stored. In addition, the file name of the received captured image is stored.

  In addition, when the picked-up image of the same file name has already been received and the information of the picked-up image of the same file name is stored in the image storage area 313 and the display image information table 3121, these stored information The received captured image information is overwritten and updated.

  Next, an image of a person portion (hereinafter referred to as “person image”) among the captured images stored in the image storage area 313 is specified (S27). As a method for specifying a person image from an image, various conventionally known methods can be used. For example, the following method can be used. First, the color, geometric shape, shading pattern, movement, etc. of the captured image are extracted as parameters. Next, a database of person image parameters stored in advance in the other information storage area 315 of the HDD 31 is referred to. Then, matching processing between the extracted parameters and the parameters of the person image stored in the database is executed. If necessary, the matching process is executed by a combination of a plurality of parameters. The portion of the photographed image from which the parameter that matches well with the parameter of the person image stored in the database is extracted is specified. The image of the identified part is recognized as a person image.

  After the person image is specified from the photographed image, a process for obtaining an avatar image (avatar assignment process, see FIG. 8) based on the specified person image is executed (S29).

  The avatar assignment process will be described with reference to FIG. In the avatar assignment process, first, in particular, the feature portion of the face portion is extracted from the person image specified in S27 (see FIG. 7) (S41). As a method for extracting the feature portion, various conventionally known methods can be used. For example, an algorithm such as a Harris operator or SIFT (Scale Invariant Feature Transform) can be used.

  Next, a process for converting a person image into an avatar image is executed based on the extracted feature part of the face part (S43). Specifically, for example, the following method can be used. Of the extracted feature parts, a part (eyebrows, eyes, mouth, etc.) that can sufficiently express the facial expression and a part that represents the outline of the face are selected. Other features are deleted. Edge detection is performed on the selected feature portion. The detected edge portion is converted into a line drawing. The color of the selected feature is detected. The detected line is colored with the detected line drawing.

  In addition, the method of converting a person image into an avatar image is not limited to the above-mentioned method, Various conventionally well-known methods are employable. For example, the following methods can be used. A plurality of patterns of avatar images are stored in the other information storage area 315 of the HDD 31 in advance. By the above-described method, a part that can sufficiently express an expression and a part that expresses the outline of a face are selected as characteristic parts. A matching process between the selected feature portion and the feature portion of the avatar image stored in the other information storage area 315 is executed. The avatar image when both are most similar is selected. Details will be described in a first modification.

  Next, the facial expression of the person image is specified based on the extracted feature portion of the face portion (S45). Various methods known in the art can be used as a method for specifying a facial expression. For example, the following method can be used. Of the extracted feature portions, portions (eyebrows, eyes, mouth, etc.) that can sufficiently express the facial expression are selected. By comparing the tendency of the characteristic part peculiar to the expressions of "anger", "disgust", "fear", "happiness", "sadness", and "surprise" with the tendency of the extracted characteristic part, one of the facial expressions is a person Specified as a facial expression. For example, when “eyebrows” are pulled up and curved in the extracted feature portion, “surprise” is specified as the facial expression of the person.

  Next, the avatar image obtained in S43 is deformed based on the specified facial expression (S47). Then, the avatar assignment process ends, and the process returns to the captured image reception process (see FIG. 7).

  As shown in FIG. 7, after the avatar assignment process is completed, in the captured image reception process, the avatar image obtained by the avatar assignment process is stored in the image storage area 313 of the HDD 31. In addition, the file name of the avatar image obtained in the display image information table storage area 312 is stored (S31). Next, a process for displaying the captured image or the avatar image on the display (display process, see FIG. 9) is executed (S35).

  The display process will be described with reference to FIG. In the display process, either the received captured image or an avatar image created based on the captured image is selected based on the condition of the display area. Then, the selected image is displayed on the display 28.

  As shown in FIG. 9, in the display process, the display image information table 3121 is referred to first. Then, based on the number of captured images received from the conference terminal devices 4 to 6, the display area and display location when each captured image is displayed on the display 28 are determined. The determined display area and display location are stored in the display image information table 3121 (S51).

  For example, the case where the display area and the display location are determined based on information in the display image information table 3121 shown in FIG. 5 will be specifically described. Since the total number of captured images stored in the display image information table 3121 is three, the display area on the display 28 is one area when the entire display area of the display 28 is divided into four. The determined vertical length (10 cm) and horizontal length (10 cm) of the display area are stored in the display image information table 3121 as information indicating the display area.

  Next, the display location is determined. As the display location, the upper left coordinates of the divided areas of the display 28 determined in the above are sequentially assigned to each captured image. In the example shown in FIG. 5, the upper left coordinate (0, 0) of the upper left display area in the display area of the display 28 divided into four is assigned to the photographed image “BBBB.cif”. The upper left coordinate (10, 0) of the upper right display area in the display area of the display 28 divided into four is assigned to the captured image “CCCC.cif”. The upper left coordinates (0, −10) of the lower left display area in the display area of the display 28 divided into four are assigned to the photographed image “DDDD.cif”. The assigned coordinate information is stored as the display location of the display image information table 3121.

  In the present embodiment, an example in which the total number of captured images is three has been described. However, for example, when the total number of captured images is one, the entire display area of the display 28 is set as the display area. The When the total number of photographed images is two, the entire display area of the display 28 is divided into two areas. When the total number of photographed images is 4, the entire display area of the display 28 is set as one area when divided into four.

  In the present embodiment, the display area and the display location are determined based on the total number of captured images stored in the display image information table 3121 as described above, but the present invention is not limited to this method. Other methods can be used. For example, the number of user IDs is calculated from the number of user IDs stored in the login table 3111 (see FIG. 4). Then, based on the calculated number, the display area and the display location may be specified by the same method as described above.

  Next, based on the display area and display location information determined in S51, the resolution of the corresponding area portion of the display area of the display 28 is acquired (S53). The calculated resolution is stored as a display area parameter (resolution) in the display image information table 3121.

  Next, the display area obtained above is read out from the display image information table 3121 (S55), and then the display area parameter (resolution) obtained above is read out from the display image information table 3121. (S57). Hereinafter, the read display area is referred to as “first parameter”, and the read resolution is referred to as “second parameter”.

  Based on the called first parameter and second parameter, either the received captured image or the avatar image obtained based on the received captured image is selected as an image to be displayed on the display 28. First, it is determined whether or not the first parameter is equal to or greater than a first threshold that is a predetermined threshold (S59). As the first threshold, a minimum display area (for example, 15 × 15 cm) necessary for displaying a captured image with high visibility is selected. When the first parameter is less than the first threshold (S59: NO), the display condition is such that the visibility is deteriorated when the captured image is displayed. Therefore, the avatar image is selected as the image to be displayed on the display 28. (S65). Then, the process proceeds to S67.

  On the other hand, when the first parameter is equal to or greater than the first threshold (S59: YES), it is then determined whether the second parameter is equal to or greater than a second threshold that is a predetermined threshold (S61). As the second threshold value, a minimum resolution (for example, 352 × 288 pixels) necessary for displaying a captured image with high visibility is selected. When the second parameter is less than the second threshold (S61: NO), the display condition is such that visibility is deteriorated when the captured image is displayed, and thus an avatar image is selected as an image to be displayed on the display 28. (S65). Then, the process proceeds to S67.

  When the first parameter is greater than or equal to the first threshold value and the second parameter is greater than or equal to the second threshold value (S61: YES), it is a display condition that allows a captured image to be displayed with good visibility. A captured image is selected as an image to be displayed on the display 28 (S63). Then, the process proceeds to S67.

  In S67, the information in the display image information table 3121 is updated. When an avatar image is selected, information indicating that the avatar image is selected is stored as a selection flag in the display image information table 3121. When the photographed image is selected, information indicating that the photographed image is selected is stored as a selection flag in the display image information table 3121 (S67). The file name of the selected image, its display area, and display location are determined based on the display area, display location, and selection flag information in the display image information table 3121. Based on the determined file name, the corresponding file is read out from the image storage area 313 and displayed at a desired position on the display 28 (S69). Then, the display process ends, and the process returns to the captured image reception process (see FIG. 7).

  As shown in FIG. 7, in the captured image reception process, either one of the captured image and the avatar image is displayed on the display 28 in the display process, and then the process returns to S21. Then, the above process is repeatedly executed. Here, as described in the captured image transmission process (see FIG. 6), captured images are repeatedly transmitted from the conference terminal devices 4 to 6 at a predetermined cycle. In the captured image reception process, every time a captured image is received, the image displayed on the display 28 is updated with the received captured image or avatar image.

  With reference to FIGS. 10 and 11, an example of a display screen displayed on the display 28 as a result of the above-described processing will be described.

  In FIG. 10, there is only one conference terminal device logged in to the video conference except for the conference terminal device 3, and an example of a display screen of the display 28 when a captured image is received only from the conference terminal device. It is shown. In this case, there is only one captured image stored in the display image information table 3121. Accordingly, the entire display area of the display 28 is set as a display area for displaying an image (S51, see FIG. 9), and this area is stored in the display image information table 3121 as a display area. Under this condition, the first parameter (display area) is equal to or greater than the first threshold (S59: YES, see FIG. 9), and the second parameter (resolution) is equal to or greater than the second threshold (S61: YES, see FIG. 9). . Therefore, a photographed image is selected as an image to be displayed on the display 28 (S63, see FIG. 9). As a result, as shown in FIG. 10, the selected captured image 41 is displayed on the display 28 (S69, see FIG. 9).

  On the other hand, in FIG. 11, the call terminal devices logged in to the video conference are three units (for example, the conference terminal devices 4 to 6) except for the conference terminal device 3, and all of these conference terminal devices 4 to 6 are used. An example of a display image on the display 28 in a case where a captured image is received from is shown. In this case, a total of three captured images are stored in the display image information table 3121 as in the example shown in FIG. Therefore, each area in a state where the entire display area of the display 28 is divided into four is set as a display area for displaying each image (see S51, FIG. 9), and this area is used as a display area and is a captured image file. It is associated with the name and stored in the display image information table 3121.

  Also, under the above conditions, is the first parameter (display area) less than the first threshold (S59: NO, see FIG. 9), or the second parameter (resolution) is less than the second threshold (S61: NO, FIG. 9). In this case, since the visibility deteriorates when the captured image is displayed on the display 28, the avatar image is selected as the image to be displayed on the display 28 (S65, see FIG. 9). Then, as shown in FIG. 11, the selected avatar images 42, 43, and 44 are displayed on the display 28 (S69, see FIG. 9).

In the example illustrated in FIG. 11, the avatar image 42 indicates the avatar image “BBBB.jpg” created based on the captured image “BBBB.cif” transmitted from the conference terminal device 4. Similarly, the avatar image 43 indicates an avatar image “CCCC.jpg” created based on the photographed image “CCCC.cif” transmitted from the conference terminal device 5.
The avatar image 44 indicates an avatar image “DDDD.jpg” created based on the photographed image “DDDD.cif” transmitted from the conference terminal device 6.

  The avatar image is obtained based on the characteristics and facial expressions of the face portion (see FIG. 8), in which a person image is extracted from the received photographed image (S27, see FIG. 7). Accordingly, the displayed avatar image reflects the facial expression of the person included in the base captured image. In the example shown in FIG. 11, the avatar image 42 shows a smiling expression, the avatar image 43 shows a smiling expression, and the avatar image 44 shows a crying expression.

  The conference terminal devices 4 to 6 repeatedly transmit captured images at a predetermined cycle. The conference terminal device 3 receives these captured images respectively. An avatar image is created based on the received photographed image and displayed on the display 28. Therefore, when the facial expression included in the received captured image changes, the facial expression of the avatar image also changes in accordance with the change of the captured image.

  As described above, in the conference terminal device 3 in the present embodiment, an avatar image is created based on the captured images received from the other conference terminal devices 4 to 6. Then, when displayed on the display 28, an image with good visibility is selected from any one, and the selected image is displayed on the display 28. As a result, the conference participants can hold a video conference while viewing images with better visibility.

  In addition, this invention is not limited to the said embodiment, A various change is possible. In the above-described embodiment, the resolution of the display area is acquired as the second parameter, and the image to be displayed on the display 28 is selected based on the first parameter (display area) and the second parameter (resolution). However, the present invention is not limited to this method. Therefore, for example, an image to be displayed on the display 28 may be selected based on only the first parameter (display area).

  Moreover, in the said embodiment, the resolution was acquired as a 2nd parameter and the image displayed on the display 28 was selected based on the resolution. However, the present invention is not limited to this method. For example, instead of the resolution, at least one of brightness and saturation of the display area may be acquired and used as the second parameter. Then, an image to be displayed on the display 28 may be selected based on the first parameter (display area) and the second parameter (at least one of brightness and saturation). As a result, an image optimal for the brightness and saturation of the display area can be selected from either the photographed image or the avatar image and displayed on the display 28. Note that, for example, a value of 30% is used as the second threshold value used when determining the brightness. For example, a value of 30% is used as the second threshold value used when determining the saturation.

  In the above-described embodiment, the display area and the display location for display on the display 28 are determined according to the number of received captured images, and the image is displayed based on the determined information. However, the present invention is not limited to this display method. Accordingly, a configuration may be adopted in which the size and display location of the image can be changed by the conference participant operating the keyboard 29 and the mouse 27 on the image once displayed on the display 28. When the image size or display location is changed, the display area and display location in the display image information table 3121 are changed based on the operation content. Therefore, even when the image size or the display location is changed, the visibility of the captured image and the avatar image is changed based on the first parameter (display area) and the second parameter (resolution) of the image after the change. A good image is selected. As a result, the user can hold a video conference while viewing an image with high visibility.

  As described above, when the display area, the display location, and the resolution value are updated in the display image information table 3121 as a result of changing the size and the display location of the image displayed on the display 28, It is assumed that the processing of S51 (see FIG. 9) (display area and display location determination processing based on the number of stored captured images) is not performed. Thereby, the display state after the change is maintained.

  If a new captured image is received and information is stored in the display image information table 3121, the process of S51 (see FIG. 9) is resumed. As a result, when the number of received captured images increases, the captured images can be displayed on the display 28 so as not to overlap each other.

  In the above-described embodiment, in the captured image transmission process (see FIG. 6), the acquired captured image is transmitted to the other conference terminal devices 4 to 6 logged in to the video conference (S15, (See FIG. 6). In the captured image reception process (see FIG. 7), the captured images transmitted from the other conference terminal devices 4 to 6 were received (S21, see FIG. 7). However, the present invention is not limited to this configuration and may have other configurations. For example, when an MCU (Multipoint Control Unit) that controls the entire video conference is connected to the network 2, the captured image is transmitted to the MCU and received in the captured image transmission process (see FIG. 6). In the processing (see FIG. 7), a configuration may be adopted in which a captured image is received from the MCU.

In the above-described embodiment, the avatar image is created based on the captured images received from the other conference terminal devices 4 to 6. However, the present invention is not limited to this method. For example, a plurality of avatar images are stored in the HDD 31 in advance. And you may determine the avatar image corresponding to a picked-up image by selecting one avatar image based on a picked-up image. Hereinafter, a first modification of the present invention will be described.
<First modification>

  The avatar assignment process in the first modification of the present invention will be described with reference to FIG. In the first modification, a plurality of avatar images are stored in the other information storage area 315 of the HDD 31. For example, avatar images having different “sex”, “hairstyle”, “expression”, “face outline” and the like are stored in the other information storage area 315. Of the configuration of the video conference system 1, the electrical configuration of the conference terminal devices 3 to 6, the storage area of the HDD 31, and various processes executed by the CPU 20, the processes other than the captured image reception process are described above. In the following, the description is omitted or simplified.

  When the avatar assignment process (see S29) is called in the captured image reception process (see FIG. 7), as shown in FIG. 12, first, in particular, the feature of the face portion in the human image specified in S27 (see FIG. 7). A part is extracted (S191). Next, based on the extracted feature portion of the face portion, “sex”, “hairstyle”, “expression”, “face outline”, and the like are determined. Then, an avatar image having parameters that match these determined parameters is selected from the other information storage area 315 (S193).

  Next, the facial expression of the person image is specified based on the extracted feature portion of the face portion (S195). Next, the avatar image obtained in S193 is deformed based on the specified facial expression (S197). Then, the avatar assignment process ends, and the process returns to the captured image reception process (see FIG. 7).

  As described above, a plurality of avatar images are prepared in advance, and an avatar image is assigned to a captured image by selecting one avatar image. Thereby, processing and time required for creating an avatar image can be suppressed. When the expression of the person image in the photographed image changes, the change in expression can be quickly reflected in the avatar image.

In the above-described embodiment, when the first parameter is equal to or greater than the first threshold and the second parameter is equal to or greater than the second threshold, the captured image is selected as an image to be displayed on the display 28. In addition, when the first parameter is less than the first threshold value or the second parameter is less than the second threshold value, an avatar image is selected as an image to be displayed on the display 28. However, the present invention is not limited to this selection method. Hereinafter, a second modification of the present invention will be described.
<Second modification>

  A second modification of the present invention will be described with reference to FIG. In the second modification, the first parameter is continuously greater than or equal to the first threshold for a predetermined time (first time) and the second parameter is continuously greater than or equal to the predetermined time (second time). When the value is equal to or greater than the threshold value, the captured image is selected. The avatar image is selected when the state where the first parameter is equal to or higher than the first threshold does not continue for the first time or when the state where the second parameter is higher than the second threshold is not continuous for the second time or longer. This prevents the image displayed on the display 28 from frequently switching between the captured image and the avatar image.

  With reference to FIG. 13, the display process in a 2nd modification is demonstrated. The configuration of the video conference system 1, the electrical configuration of the conference terminal devices 3 to 6, the storage area of the HDD 31, the display image transmission process executed by the CPU 20, and the display image reception process excluding the display process are described above. Therefore, the description is omitted or simplified below. The display process shown in FIG. 13 is called from the display image reception process activated by the CPU 20 when an application for executing a video conference is activated.

  As shown in FIG. 13, in the display process in the second modification, first, the display image information table 3121 is referred to, and the display area and the display location are determined. Information on the determined display area and display location is stored in the display image information table 3121 (S101). The display area obtained as described above is read out from the display image information table 3121 and used as the first parameter (S103). And it is judged whether a 1st parameter is more than a 1st threshold value (S105). When the first parameter is less than the first threshold (S105: NO), the display condition is such that the visibility is deteriorated when the captured image is displayed. Therefore, the avatar image is selected as the image to be displayed on the display 28. (S121). Then, the process proceeds to S123.

  On the other hand, when the first parameter is greater than or equal to the first threshold (S105: YES), the display image information table 3121 is referred to. The resolution is acquired based on the information on the display area and the display location (S109) and stored in the display image information table 3121. The stored resolution is read as the second parameter (S111). Then, it is determined whether the second parameter is greater than or equal to the second threshold (S113). When the second parameter is less than the second threshold (S113: NO), the display condition is such that the visibility is deteriorated when the captured image is displayed, and thus the avatar image is selected as the image to be displayed on the display 28. (S121). Then, the process proceeds to S123.

  If the first parameter is greater than or equal to the first threshold and the second parameter is greater than or equal to the second threshold (S113: YES), the time (elapsed time) after starting the determination of the first parameter and the second parameter It is determined whether or not (time) is equal to or longer than a predetermined time (first time) (S115). For example, a value of 10 seconds is set as the first time. When the elapsed time is less than the first time (S115: NO), the process returns to S103, the first parameter (display area) is repeatedly read from the display image information table 3121, and the above-described processing is repeatedly executed.

  When the above process is repeated, a conference participant operates the keyboard 29 or the mouse 27 to change the size of the image (captured image, avatar image) displayed on the display 28. The value of the display area in the image information table 3121 is changed and updated. In this case, the changed display area (first parameter) is read out (S103). And when it becomes less than a 1st threshold value (S105: NO), an avatar image will be selected as an image displayed on a display (S121).

  On the other hand, when the elapsed time is equal to or longer than the first time (S115: YES), it is determined whether the elapsed time is equal to or longer than a second time which is a predetermined time longer than the first time (S117). For example, a value of 20 seconds is set as the second time. When the elapsed time is less than the second time (S117: NO), the process returns to S109, the second parameter (resolution) is repeatedly read from the display image information table 3121, and the above-described processing is repeatedly executed.

  In addition, while the above-described processing is repeatedly performed, the conference participant operates the keyboard 29 or the mouse 27 to change the size of the image (captured image, avatar image) displayed on the display 28. In this case, the resolution value in the display image information table 3121 is changed and updated. In this case, the changed resolution (second parameter) is read (S109). And when it becomes less than a 2nd threshold value (S113: NO), an avatar image will be selected as an image displayed on a display (S121).

  When the state where the first parameter is equal to or greater than the first threshold continues for the first time or more and the state where the second parameter is equal to or greater than the second formula continues for the second time or longer (S117: YES), the display 28 displays A captured image is selected as an image to be displayed (S119). Then, the process proceeds to S123.

  In S123, the information of the selection flag in the display image information table 3121 is updated based on the image selected above (S123). Then, based on the updated information on the selection flag and the information on the display area and the display location, the selected image (one of the captured image and the avatar image) is displayed at a desired position on the display 28 (S125). . Then, the display process ends, and the process returns to the captured image reception process (see FIG. 7).

  As described above, in the second modification, the state where the first parameter is equal to or greater than the first threshold continues for the first time or more, and the state where the second parameter is equal to or greater than the second threshold is equal to or greater than the second time. When the images are consecutive, a captured image is selected as an image to be displayed on the display 28. Once the avatar image is selected, the image displayed on the display 28 is not switched to the captured image unless at least the second time has elapsed. Thereby, for example, when the size of the image displayed on the display 28 is changed by the operation of the conference participant and the display area (first parameter) becomes a value close to the first threshold, the resolution (second parameter) ) Becomes a value close to the second threshold value, it is possible to prevent the image displayed on the display 28 from frequently switching between the photographed image and the avatar image.

In addition, this invention is not limited to the said embodiment and modification, A various change is possible. In the above description, the conference terminal device 3 that has received the captured image has created the avatar image based on the received captured image, but the present invention is not limited to this method. For example, it is good also as a structure which produces an avatar image in the conference terminal devices 4-6 of the side which transmits a picked-up image, and transmits with respect to the conference terminal device 3 with a picked-up image. Hereinafter, a third modification of the present embodiment will be described.
<Third modification>

  A third modification of the present invention will be described with reference to FIGS. In the third modification, the conference terminal devices 4 to 6 on the side of transmitting the captured image create an avatar image as necessary and transmit the avatar image to the conference terminal device 3 together with the captured image. As a result, the processing load on the conference terminal device 3 on the side that receives the captured image can be reduced, and the avatar image can be smoothly displayed on the display 28.

  Of the configuration of the video conference system 1, the electrical configuration of the conference terminal devices 3 to 6, the storage area of the HDD 31, and the display image reception processing executed by the CPU 20, the portions other than the display processing are described in the above embodiment. Therefore, the description is omitted below. In addition, in the following captured image transmission process and captured image reception process, the description of the same part as the above-described embodiment is omitted or simplified.

  The flag used in the third modification will be described. In the third modification, a request flag and a requested flag are used. The request flag is used to indicate whether or not it is necessary to create an avatar image based on the captured image when the conference terminal device 3 transmits the captured image to the conference terminal devices 4 to 6. Flag. “1” is stored when it is necessary to create an avatar image, and “0” is stored when it is not necessary to create an avatar image. A request flag corresponding to each of the other conference terminal devices 4 to 6 logged in to the conference system is stored in the RAM 22.

  The requested flag is a flag used to indicate whether or not an instruction to create an avatar image has been given to the conference terminal devices 4 to 6. “1” is stored when an instruction to create an avatar image is issued, and “0” is stored when an instruction to create an avatar image is not issued. Requested flags corresponding to each of the other conference terminal devices 4 to 6 logged in to the conference system are stored in the RAM 22.

  Hereinafter, “1” stored in the flag is referred to as “flag is turned on”, and “0” is stored in the flag is referred to as “flag is turned off”. These flags are all turned off and initialized when the CPU 20 is powered on and activated.

  The avatar creation instruction process will be described with reference to FIG. As shown in FIG. 14, when the avatar creation instruction process is activated, first, a notification signal (hereinafter referred to as “request notification”) that requests creation of an avatar image from the conference terminal devices 4 to 6 via the network 2. Is received (S131). When the request notification is received (S131: YES), the request flag corresponding to the conference terminal devices 4 to 6 that transmitted the request notification is turned ON (S135). Then, the process proceeds to S141.

  If a request notification has not been received (S131: NO), whether or not a notification signal (hereinafter referred to as “rejection notification”) for rejecting creation of an avatar image has been received from the conference terminal devices 4 to 6 via the network 2. Determination is made (S137). When the rejection notification is received (S137: YES), the request flag corresponding to the conference terminal devices 4 to 6 that transmitted the rejection notification is turned OFF (S139). Then, the process proceeds to S141. On the other hand, if neither the request notification nor the rejection notification has been received (S137: NO), the process proceeds to S141 without performing any special processing.

  In S141, it is determined whether an instruction to create an avatar image and transmit an avatar image to the conference terminal devices 4 to 6 is given by the conference participant (S141). When a conference participant who operates the conference terminal device 3 inputs an avatar image creation and avatar image transmission instruction via the keyboard 29 and the mouse 27 (S141: YES), specific other conference terminal devices 4 to 4 are used. A request notification is transmitted to 6 (S143). Next, the requested flag corresponding to the conference terminal devices 4 to 6 as the transmission destination of the request notification is turned on (S145). Then, the process returns to S131 and the above-described processing is repeatedly executed.

  When the instruction for creating the avatar image and the transmission of the avatar image is not input (S141: NO), an instruction for prohibiting the creation of the avatar image and the transmission of the avatar image to the conference terminal devices 4 to 6 is performed by the conference participant. It is determined whether it has been made (S147). When an instruction prohibiting assignment of avatar images and transmission of avatar images is input by a conference participant who operates the conference terminal device 3 via the keyboard 29 and the mouse 27 (S147: YES), specific other conference terminals A rejection notification is transmitted to the devices 4 to 6 (S149). Next, the requested flag corresponding to the conference terminal devices 4 to 6 as the transmission destination of the request notification is turned off (S151). Then, the process returns to S131 and the above-described processing is repeatedly executed. On the other hand, when no input is made by the conference participant (S147: NO), the process returns to S131 without going through special processing, and the above-described processing is repeatedly executed.

  The captured image transmission process will be described with reference to FIG. As shown in FIG. 15, when the captured image transmission process is activated, it is first determined whether or not the camera 34 is operating and capturing a conference participant or the like (S11). When the camera 34 is not operating (S11: NO), the process returns to S11 without performing any special processing, and the camera 34 is continuously monitored. When the camera 34 is operating (S11: YES), a captured image is acquired from the operating camera 34 (S13), and the acquired captured image is transmitted to the conference terminal devices 4 to 6 via the network 2. (S15).

  Next, the requested flag is referenced (S161). When the requested flag is OFF (S161: NO), a request notification has not been received from the conference terminal devices 4 to 6 corresponding to this flag, and processing for creating an avatar image based on the captured image is performed. Not receiving instructions. In this case, the process proceeds to S167 without performing special processing.

  On the other hand, when the requested flag is ON (S161: YES), since the request notification is received from the conference terminal devices 4 to 6 corresponding to this flag, an avatar image is created based on the photographed image and photographed. It is necessary to transmit to this conference terminal device 4-6 with an image. Therefore, in this case, a person image is specified among the acquired captured images (S162). Next, a process for obtaining an avatar image based on the person image (avatar assignment process, see FIG. 8) is executed (S163). Details of the avatar assignment process are omitted. The avatar image obtained after the avatar image assignment processing is completed is transmitted to the conference terminal devices 4 to 6 corresponding to the requested flag that is turned on (S165). Then, the process proceeds to S167.

  In S167, in order to wait until the transmission timing of the next photographed image, a predetermined time (for example, 1 ms) is waited (S167). After a predetermined time has elapsed, the process returns to S11 and the above-described processing is repeatedly executed. In this way, when the camera 34 is in operation, captured images are repeatedly transmitted to the conference terminal devices 4 to 6 at a constant period. In addition, the avatar image assigned to the captured image is repeatedly transmitted at regular intervals to the conference terminal devices 4 to 6 that have received the request notification.

  The captured image reception process will be described with reference to FIG. As shown in FIG. 16, when the captured image reception process is activated, it is determined whether captured images are received from the conference terminal devices 4 to 6 via the network 2 (S21). When the captured image has not been received (S21: NO), the process returns to S21 without performing any special processing, and reception monitoring of the captured image is continuously performed. When the captured image is received (S21: YES), the received captured image is stored in the image storage area 313 (S23). Next, information related to the received captured image is stored in the display image information table 3121 (S25).

  The request flag is then referenced. When the request flag is ON (S171: YES), a request notification is transmitted to the conference terminal devices 4 to 6 corresponding to this flag, and an instruction to perform a process of creating an avatar image based on the captured image Has been made. In this case, since the conference terminal devices 4 to 6 are transmitting avatar images, the transmitted avatar images are received (S179). Then, the process proceeds to S181.

  On the other hand, when the request flag is OFF (S171: NO), since the request notification is not transmitted to the conference terminal devices 4 to 6 corresponding to this flag, the user creates an avatar image based on the captured image. It is necessary to perform processing. Therefore, a person image is specified among the captured images stored in the image storage area 313 (S173). Then, a process for obtaining an avatar image based on the person image (avatar assignment process, see FIG. 8) is executed (S175). Details of the avatar assignment process are omitted. The obtained avatar image is stored in the image storage area 313 of the HDD 31 (S177). Then, the process proceeds to S181.

  In S181, a process for displaying the photographed image or the avatar image on the display 28 (display process, see FIG. 9) is executed (S181). Details of the display process are omitted.

  As described above, in the third modification, the conference terminal devices 4 to 6 that transmit the captured image can be caused to execute a process of creating an avatar image from the captured image. This eliminates the time required for the avatar image assignment process, so that the avatar image can be quickly displayed on the display 28.

  7 and 16 corresponds to the “first reception control means” of the present invention, and the CPU 20 that performs the processes of S27 and S29 in FIG. 7 and S173 and S175 in FIG. Corresponds to “Avatar assigning means”. The CPU 20 that performs the processes of S55 and S57 of FIG. 9 and S103 and S111 of FIG. 13 corresponds to the “region parameter acquisition unit” of the present invention. The CPU 20 that performs the processes of S59, S61, S63, S65 of FIG. 9 and S105 to S121 of FIG. 13 corresponds to the “image selection means” of the present invention, and the CPU 20 that performs the processes of S69 of FIG. 9 and S125 of FIG. It corresponds to “display control means” of the present invention.

  The CPU 20 that performs the process of S45 in FIG. 8 corresponds to the “expression specifying unit” of the present invention. The CPU 20 that performs the process of S179 in FIG. 16 corresponds to the “second reception control unit” of the present invention. The CPU 20 that performs the process of S143 in FIG. 14 corresponds to the “transmission control means” of the present invention.

  7 and 16 correspond to the “first reception control step” of the present invention, and S27 and S29 correspond to the “avatar allocation step” of the present invention. S55 and S57 in FIG. 9 and S173 and S175 in FIG. 16 correspond to the “region parameter acquisition step” of the present invention. S59, S61, S63, S65 in FIG. 9 and S105 to S121 in FIG. 13 correspond to the “image selection step” of the present invention, and S69 in FIG. 9 and S125 in FIG. 13 correspond to the “display control step” in the present invention. To do.

1 Video conference system 2 Network 3, 4, 5, 6 Conference terminal device 20 CPU

Claims (13)

A conference terminal device that is connectable to a network and performs a remote conference by communicating via the network,
First reception control means for receiving a photographed image that is an image photographed by the photographing means of another conference terminal device;
An avatar assigning means for assigning an avatar image which is a character image to the photographed image received by the first reception control means;
Of the display area of the display means for displaying the photographed image received by the first reception control means or the avatar image assigned by the avatar assignment means, an area capable of displaying the photographed image or the avatar image Area parameter acquisition means for acquiring parameters related to
Based on the parameter acquired by the region parameter acquisition unit, an image selection unit that selects either the captured image or the avatar image as a display image to be displayed on the display unit;
A conference terminal device comprising: display control means for displaying the display image selected by the image selection means in a displayable area of the display area of the display means. The region parameter acquisition means includes
Obtain the area parameter of the area where the display image can be displayed as a first parameter,
The image selection means includes
2. The conference terminal apparatus according to claim 1, wherein the display image to be displayed on the display unit is selected based on the first parameter. The region parameter acquisition means includes
Obtaining a resolution parameter of an area where the display image can be displayed as a second parameter;
The image selection means includes
The conference terminal apparatus according to claim 2, wherein the display image to be displayed on the display unit is selected based on the first parameter and the second parameter. The region parameter acquisition means includes
Obtaining at least one parameter of brightness and saturation of a region capable of displaying the display image as a second parameter;
The image selection means includes
The conference terminal apparatus according to claim 2, wherein the display image to be displayed on the display unit is selected based on the first parameter and the second parameter. The image selection means includes
When the first parameter acquired by the region parameter acquisition unit is equal to or greater than a first threshold that is a predetermined threshold, the captured image is selected as the display image, and the first parameter is less than the first threshold The conference terminal device according to claim 2, wherein the avatar image is selected as the display image. The image selection means includes
When the first parameter acquired by the area parameter acquisition unit is equal to or greater than a first threshold that is a predetermined threshold and the second parameter is equal to or greater than a second threshold that is a predetermined threshold An image is selected as the display image, and the avatar image is selected as the display image when the first parameter is less than the first threshold value or the second parameter is less than the second threshold value. The conference terminal device according to claim 3, wherein the conference terminal device is a conference terminal device. The image selection means includes
When the first parameter acquired by the region parameter acquisition unit is continuously equal to or greater than the first threshold for a first time that is a predetermined time, the captured image is selected as the display image, The conference terminal device according to claim 5, wherein the captured image is selected as the avatar image when a state where one parameter is equal to or greater than the first threshold does not continue for the first time or longer. The image selection means includes
The first parameter acquired by the region parameter acquiring means is a case where the first parameter is continuously equal to or greater than the first threshold which is a predetermined time, and the second parameter is a predetermined time. When the captured image is selected as the display image for two or more hours continuously, the state in which the first parameter is equal to or greater than the first threshold does not continue for the first time or more. The conference terminal according to claim 6, wherein the avatar image is selected as the display image when a state where the second parameter is equal to or greater than the second threshold does not continue for the second time or longer. apparatus. Facial expression specifying means for specifying facial expressions by extracting facial features included in the captured image received by the first reception control means;
Avatar conversion means for converting the avatar image assigned by the avatar assigning means based on the facial expression specified by the facial expression specifying means;
The image selection means includes
9. The display device according to claim 1, wherein one of the photographed image and the avatar image converted by the avatar conversion unit is selected as a display image to be displayed on the display unit. Conference terminal device. A second reception control means for receiving the avatar image transmitted from the other conference terminal device;
The region parameter acquisition means includes
Obtaining the parameter related to the area where the captured image or the avatar image received by the second reception control means can be displayed;
The image selection means includes
Based on the parameter acquired by the region parameter acquisition unit, either the captured image or the avatar image received by the second reception control unit is selected as the display image. The conference terminal device according to claim 1. Transmission control means for transmitting a request notification that is a notification for requesting the avatar image to the other conference terminal device,
The second reception control means includes
The conference terminal device according to claim 10, wherein the avatar image transmitted from the other conference terminal device is received in response to the request notification transmitted by the transmission control unit. A display control method capable of connecting to a network and performing a remote conference by communicating via the network,
A first reception control step of receiving a photographed image that is an image photographed by the photographing means of another conference terminal device;
An avatar assignment step of assigning an avatar image which is a character image to the captured image received by the first reception control step;
Of the display area of the display means for displaying the captured image received in the first reception control step or the avatar image allocated in the avatar allocation step, the captured image or the avatar image can be displayed. An area parameter acquisition step for acquiring parameters related to
Based on the parameter acquired in the area parameter acquisition step, an image selection step of selecting either the captured image or the avatar image as a display image to be displayed on the display means;
A display control method comprising: a display control step of displaying the display image selected in the image selection step in a displayable area of the display area of the display means.   The display control program which drives a computer as each process means in the conference terminal device in any one of Claims 1 thru | or 11.

Images