JP6268510B2 - COMMUNICATION DEVICE, COMMUNICATION METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to a communication device, a communication method, and a program. More specifically, the present invention relates to a communication device, a communication method, and the communication device used for information sharing by transmitting and receiving images and sounds via a communication network between at least three bases. Relates to the program used in

  2. Description of the Related Art Conventionally, there is known a video conference terminal device used at any one site for performing a video conference (information sharing) by transmitting and receiving images and sounds via a communication network between at least three sites (for example, Patent Documents). 1).

  In this video conference terminal device, an image sent from at least one other site where there is no speaker is displayed on the display in a predetermined size, and an image sent from one other site where the speaker is present is made larger than the predetermined size. Is also displayed on the display in a larger enlarged size.

  However, in the video conference terminal device disclosed in Patent Document 1, when a speaker is present at a plurality of other bases, the speaker cannot be easily recognized.

The present invention is a communication apparatus used at any one site for sharing information by transmitting and receiving images and sounds via a communication network between at least three sites, and at least two other devices different from the one site Based on information on at least two voices individually transmitted from the base via the communication network, a determination unit that determines the presence or absence of a speaker at the other base, and the determination unit determines that there is no speaker The image sent from the other site via the communication network is displayed on the display screen in a predetermined size, and sent from the other site determined by the judging means that the speaker is present via the communication network. display means for displaying an image on the display surface at a greater enlargement in size than the predetermined size, the said determination means, said at least two other base volume level at each threshold or higher When that condition is considered to have continued for the predetermined time, determines that there is a speaker in said other bases, the display means has been determined that there is a speaker in a plurality of other locations in the determination unit The plurality of other sites that are individually transmitted via the communication network from the plurality of other sites and are displayed together on the display surface in the enlarged size, and the plurality of other sites determined to have a speaker. Of images from other sites where the speaker who is currently speaking is displayed in a first enlarged size larger than the predetermined size, and images from other sites where the speaker is suspended Is displayed in a second enlarged size smaller than the first enlarged size and larger than the predetermined size .

  According to this, when a speaker is present at a plurality of other bases, the speaker can be easily recognized.

It is a figure which shows the structure of the conference system which concerns on one Embodiment. FIGS. 2A and 2B are perspective views (No. 1 and No. 2), respectively, in the reference form of the video conference apparatus. It is a top view in the standard form of a video conference apparatus. It is a block diagram which shows the structure of control of a video conference apparatus. It is a side view (the 1) in the standard form of a video conference apparatus. It is a side view (the 2) in the standard form of a video conference apparatus. It is a bottom view in the standard state of a video conference apparatus. It is a side view in the opening form of a video conference apparatus. It is a perspective view in the maximum opening form of a video conference apparatus. It is a figure which shows the aspect of the video conference performed using a video conference apparatus in each base. It is a flowchart for demonstrating a series of processes performed when the image from another base is displayed on the display of a video conference apparatus. It is a figure which shows the state by which the image from one other base is displayed on the display of a video conference apparatus by the enlarged size, and the image from the remaining other bases is displayed by the reduced size. It is a figure which shows the state by which the image from two other bases is displayed on the display of a video conference apparatus by the enlarged size, and the image from the remaining other bases is displayed by the reduced size. It is a figure which shows the state from which the image from three other bases are displayed by the enlarged size, and the image from the other other bases is displayed by the reduced size on the display of a video conference apparatus. It is a figure which shows the state in which the image from all the other bases is displayed on the display of a video conference apparatus by the reduced size. FIG. 16A and FIG. 16B each show an image from one other site in an enlarged size on the display of the video conference apparatus of the comparative example, and images from the remaining other sites in a reduced size. It is a figure (the 1 and the 2) which shows the state currently performed. The image from one other site displayed in the enlarged size is enlarged on the display of the video conference apparatus, the image from another one site displayed in the enlarged size is reduced, and the image from the other other site is displayed. It is a figure which shows the state by which the image of 1 is displayed by reduction size. The image from one other site displayed in the enlarged size is enlarged on the display of the video conference apparatus, the images from the other three other sites displayed in the enlarged size are reduced, and the image from the other other sites is displayed. It is a figure which shows the state by which the image of 1 is displayed by reduction size. 19A to 19C each show a process for identifying images from two other bases on the display of the video conference apparatus in the same size and having the same size. FIG. 7 is a diagram illustrating a state in which images from other remaining bases are displayed in a reduced size.

  Hereinafter, an embodiment will be described with reference to FIGS. FIG. 1 schematically shows a conference system 1000 as an example of a communication system according to an embodiment. The conference system 1000 is used for a video conference (information sharing) via a network (communication network) between at least three sites (for example, 10 sites), for example, the Internet. Here, the video conference is a system for sharing information using moving images and sound.

  As shown in FIG. 1, the conference system 1000 includes, for example, a video conference apparatus 100 as a plurality of (for example, 10) communication apparatuses.

  As an example, the ten video conference apparatuses 100 have substantially the same configuration and function, and are connected to be able to communicate bidirectionally via the Internet.

  FIGS. 2A and 2B are perspective views showing an appearance of one form (hereinafter referred to as a reference form) of the video conference apparatus 100 viewed from different directions.

  As shown in FIGS. 2 (A) and 2 (B), the video conference apparatus 100 as a whole has, for example, a substantially rectangular plate-like outer shape (for example, a thickness of 20 mm to 40 mm) of A4 size as a whole. have.

  Thus, the video conference apparatus 100 is designed to be thin and compact in the standard form. Further, as will be described in detail later, the video conference apparatus 100 is between a reference form and an open form suitable for use in a state of being placed on an upper surface (mounting surface) such as a desk or table. The form can be shifted.

  FIG. 3 shows a state in which the video conference device 100 in the standard form is placed on a horizontal placement surface. Hereinafter, the longitudinal direction of the video conference apparatus 100 is the X axis direction, the direction orthogonal to the X axis direction in the horizontal plane is the Y axis direction, and the direction (vertical direction) orthogonal to any of the X axis direction and the Y axis direction is the Z axis. This will be described as a direction. Further, the video conference apparatus 100 is assumed to be in a reference form unless otherwise specified.

  As shown in FIGS. 2A to 3, the video conference apparatus 100 includes a first housing 10, a second housing 12, a display 14, an electronic camera 16, a speaker 18, a microphone 20, and a control device 24 (see FIG. 2). 4).

  As an example, as shown in FIGS. 2A and 2B, the first housing 10 includes a control device housing portion 10a in which the control device 24 is housed and a plurality of operation members. And an operation panel unit 10b.

  The control device 24 encodes or decodes image data and audio data, and controls transmission and reception of images and audio via the Internet.

  As shown in FIG. 4, the control device 24 includes a main board 62 as a control board, a sub board 63 as a voice processing and operation board, and the like.

  As an example, the main board 62 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 104, an HDD (Hard Disk Drive) 106 (or a storage or recording device), and data storage. Each component of an output I / F (Interface) 108, a network I / F 110, an image input I / F 112, a volume level determination unit 160, a display processing unit 140, and an image output I / F 114 is mounted. The CPU 101, ROM 102, RAM 104, HDD 106, data input / output I / F 108, network I / F 110, image input I / F 112, and volume level determination unit 160 communicate bidirectionally via a bus line 116 such as an address bus or a data bus. Connected as possible. Note that the above-described image data is data of moving images or intermittent images (still images with a fixed time interval).

  The CPU 101 controls the overall operation of the video conference apparatus 100 based on a predetermined program (video conference apparatus program). A series of operations related to bidirectional communication of audio and images via a network (for example, the Internet) in accordance with a command from the CPU 101 according to the TV conference device program will be described later.

  The ROM 102 stores a program used for driving the CPU 101 such as an IPL (Initial Program Loader). The RAM 104 is used as a work area for the CPU 101.

  The HDD 106 stores various data such as the video conference device program, image data, and audio data. In addition, not only HDD106 but SSD (Solid State Drive) etc. may be used, for example. The program for the video conference apparatus may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium such as a recording medium. The video conference device program may be stored in the ROM 102 instead of the HDD 106. The HDD 106 controls reading or writing of various data with respect to the HDD 106 according to the control of the CPU 101.

  The data input / output I / F 108 controls reading or writing (storage) of data with respect to a recording medium M such as a flash memory connected to a plurality of USB terminals, which will be described later, and for example a PC (personal computer) connected to the USB terminal. The transmission of data from the computer) to the video conference device at the other site is controlled.

  The recording medium M is detachably attached to one of the plurality of USB terminals. For example, if the recording medium M is an SD memory, a compact flash (registered trademark) or the like that cannot be directly attached to or detached from the USB terminal, the memory reading drive may be attached to or detached from the USB terminal. The video conference device 100 may be provided with a memory reading drive in advance. The recording medium M is not limited to a flash memory as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 101, and may be an EEPROM (Electrically Erasable and Programmable ROM) or the like.

  The network I / F 110 includes a LAN terminal 38 (for example, Ethernet (registered trademark) terminal) described later, and performs input / output (transmission / reception) of data (image data and audio data) via the Internet. The network I / F 110 may be provided so as to be connectable to a wired LAN terminal, or may be a wireless LAN network I / F that can be connected to a wireless LAN.

  The image input I / F 112 takes in an image signal of a subject output from an electronic camera 16 described later as predetermined image data.

  The volume level determination unit 160 determines whether or not the volume level of the voice data from the communication partner (other base) captured via the network I / F 110 is equal to or higher than a predetermined threshold, and displays the determination result. The data is sent to the processing unit 140.

  As will be described in detail later, the display processing unit 140 uses the image data from another base captured via the network I / F 110 as an image based on the determination result of the volume level determination unit 160 as a display surface of the display 14. The size and position to be displayed are set, and the image data is sent to the image output I / F 114. In the present embodiment, as will be described in detail later, an image from another site where no speaker is present is reduced on the display surface of the display 14 by a display unit including the display processing unit 140 and the CPU 101. The image from the other site where the speaker is present is displayed in an enlarged size larger than the reduced size.

  The image output I / F 114 converts the image data from the display processing unit 140 into an image signal that can be displayed on the display 14 and outputs the image signal to the display 14. As a result, an image corresponding to the image data is displayed on the display surface of the display 14.

  Examples of the image signal include an analog RGB signal (VGA), a component video signal, an HDMI (registered trademark) (High-Definition Multimedia Interface) signal, and a DVI (Digital Video Interactive) signal.

  On the sub board 63, the respective components of the plurality of operation terminals 118, the voice input / output I / F 120, and the voice control unit 122 corresponding to the plurality of operation buttons described above are mounted. The plurality of operation terminals 118 and the audio control unit 122 are connected to each other via the bus line 116 so as to be capable of bidirectional communication with each other, and are mounted on the main board 62 and connected via the bus line 116. It is connected so that it can communicate with the other.

  The voice input / output I / F 120 takes in the voice signal input from the microphone 20 as predetermined voice data, sends the voice data to the voice control unit 122, and receives it through the network I / F 110 and the voice control unit 122. The voice data from the other site is converted into a voice signal that can be reproduced by the speaker 18.

  The sound control unit 122 adjusts the volume of sound output from the speaker 18 when a pair of volume buttons 62a and 62b described later is operated, and a microphone mute button 64 described later is pressed down (ON / OFF). ), The sound input / non-input by the microphone 20 is switched.

  In addition, the voice control unit 122 receives sound output from the speaker 18 through the microphone 20 during bidirectional communication with the video conference device 100 at another site, and generates sound waves with the video conference device 100 at the other site. It has an echo cancellation processing function that suppresses echoes and howling that are generated by forming a loop, and a noise cancellation processing function that reduces sound input by the microphone 20 such as an operating sound of an indoor air conditioner.

  As can be seen from a combination of FIGS. 5 and 6, the control device accommodating portion 10 a includes, as an example, a box-shaped portion having a substantially rectangular flat plate-like outer shape whose longitudinal direction is the X-axis direction.

  In addition, as shown in FIG. 5, four openings are formed in the side wall on the + X side of the control device housing portion 10a, and the four openings are for communication mounted on the main board 62, respectively. LAN terminal 38, HDMI (registered trademark) image output terminal 40, VGA image output terminal 42, and power terminal 44 (power jack) are fitted. Instead of or in addition to the image output terminals 40 and 42, a DVI-I terminal, a component terminal, a D terminal, an S terminal, and a composite terminal may be provided.

  As shown in FIG. 6, as an example, two openings are formed at the −X side end of the −Y side side wall of the control device housing portion 10 a, and the two openings are respectively connected to the main opening. USB terminals 28 and 30 mounted on the board 62 are fitted. Also, as an example, three openings are formed at the + X side end of the −Y side side wall of the control device accommodating portion 10 a, and the three openings are respectively mounted on the main board 62. A headphone connection terminal 32, a microphone connection terminal 34, and a USB terminal 36 are fitted.

  Further, as shown in FIG. 7, as an example, four openings are formed in the four corners of the −Z side wall (bottom wall) of the control device housing portion 10 a, and the four openings include: Four connectors 46 mounted on the main board 62 for connecting an external battery are fitted.

  As can be understood from FIG. 3, FIG. 5 and FIG. 6, the operation panel portion 10b is composed of a box-shaped portion having an elongated, substantially rectangular, flat plate shape whose longitudinal direction is the X-axis direction. It is continuous to the + Z side end of the −Y side end.

  As shown in FIG. 3, in the region on the −X side of the + Z side wall of the operation panel unit 10b, as an example, five openings are formed side by side in the X-axis direction. Five operation members are individually fitted. As an example, these five operation members are a power button 48, a menu button 50, a cursor 52, a decision button 54, and a disconnect button 56 for disconnecting the Internet line with the communication partner in order from the -X side to the + X side. ing.

  The power button 48 is an operation member for turning on / off the power to the video conference apparatus 100. In the vicinity of the power button 48 in the operation panel unit 10b, a power lamp 49 that is turned on when the power is turned on is provided.

  The menu button 50 is an operation member for calling a menu screen on the display 14.

  The cursor 52 is an operation member for selecting an item displayed in the menu screen displayed on the display 14. That is, by operating the cursor 52, items in the menu screen can be selected.

  The determination button 54 is an operation member for determining an item selected on the menu screen. As an item selected and determined in the menu screen, for example, a destination of the video conference apparatus 100 at another base can be cited.

  The disconnect button 56 is an operation member for disconnecting the Internet line with the video conference device 100 of the communication partner.

  For example, in the + X side region of the + Z side wall of the operation panel unit 10b, five openings are formed side by side in the X-axis direction, and five operation members are individually fitted into the five openings. ing. As an example, these five operation members are a view switching button 58, a brightness adjustment button 60, a pair of volume buttons 62a and 62b, and a microphone mute button 64 in order from the -X side to the + X side.

  The view switching button 58 is an operation member for switching an image to be transmitted to another site among images captured by the electronic camera 16. That is, for example, when there are a plurality of users (conference participants) at a certain base, by pressing the view switching button 58, one of a plurality of images selectively displayed by at least one of the plurality of users is displayed. Can be sent to other locations.

  The brightness adjustment button 60 is an operation member for adjusting the brightness of an image displayed on the display 14. By appropriately pressing the brightness adjustment button 60 according to the ambient brightness of the video conference apparatus 100, the brightness of the screen of the display 14 can be adjusted to an appropriate brightness with good visibility. Here, as an example, the lightness adjusted by the lightness adjustment button 60 is set to N levels (N is a natural number), and the lightness gradually increases or decreases each time the lightness adjustment button 60 is pressed. When the brightness adjustment button 60 is pressed N times, the original brightness is restored.

  The pair of volume buttons 62 a and 62 b are operation members for adjusting the volume of the sound output from the speaker 18. The volume can be lowered by pressing the -X side volume button 62a of the pair of volume buttons 62a and 62b, and the volume can be raised by pressing the + X side volume button 62b. .

  The microphone mute button 64 is an operation member for switching ON / OFF of the microphone 20. Note that when the microphone 20 is ON, it means a state in which sound is input via the microphone 20, and when the microphone 20 is OFF, it means a state where no sound is input via the microphone 20.

  As shown in FIGS. 2A and 2B, the second housing 12 accommodates the display 14, a part of the electronic camera 16, a part of the speaker 18, and a part of the microphone 20. It has the 1st accommodating part 12a, the 2nd accommodating part 12b in which the remainder (except the imaging lens 16a) of the electronic camera 16, the remainder of the speaker 18, and the remainder of the microphone 20 are accommodated.

  As can be understood from FIGS. 3, 5, and 6, the first housing portion 12 a is composed of a box-shaped portion having a substantially rectangular flat plate-like outer shape whose longitudinal direction is the X-axis direction. It is on the + Z side and is located on the + Y side of the operation panel unit 10b. Here, as an example, when the video conference device 100 is in the reference form, the −Z side surface of the first housing portion 12a is in contact with the + Z side surface of the control device housing portion 10a.

  The display 14 is accommodated in a state parallel to the XY plane at the approximate center in the first accommodating portion 12a.

  As an example, the display 14 includes a liquid crystal panel having a display surface on which an image is displayed, two electrodes disposed at a position sandwiching the liquid crystal panel, and two polarizing plates disposed at a position sandwiching the two electrodes. And a backlight disposed on one side of the liquid crystal structure, and has a flat plate-like outer shape as a whole. That is, the display 14 is a so-called liquid crystal display.

  As an example, a rectangular opening that is one or more times larger than the display 14 is formed on the surface on the + Z side of the first housing portion 12a, and a cover panel 26 made of reinforced plastic is fitted into the rectangular opening. . That is, the display 14 is covered from the + Z side by the cover panel 26. The cover panel 26 has a transparent portion corresponding to (opposed to) the display 14, and a portion surrounding the portion is, for example, black. Hereinafter, the transparent portion of the cover panel 26 is referred to as a clear panel portion CP, and the black portion of the cover panel 26 is referred to as a black panel portion BP. In this case, the liquid crystal panel of the display 14 is visible through the clear panel portion CP from the + Z side.

  As can be understood from FIGS. 5 and 6, the second housing portion 12 b is composed of a substantially rectangular plate-like box-shaped portion whose longitudinal direction is the X-axis direction, and the end portion on the + Y side of the first housing portion 12 a. It continues to the end on the -Z side. That is, the 2nd accommodating part 12b protrudes in the -Z side rather than the -Z side end of the 1st accommodating part 12a.

  As shown in FIG. 3, FIG. 5, and FIG. 6, an electronic camera is provided at the center in the X-axis direction of the space formed by the + Y side end of the first housing portion 12 a and the second housing portion 12 b. A portion excluding the 16 photographing lenses 16 a is accommodated, and the photographing lens 16 a protrudes outside the second housing 12 from an opening formed in the black panel portion BP.

  As the photographic lens 16a, for example, a substantially hemispherical wide-angle lens having a viewing angle in the X-axis direction of 170 ° and a viewing angle in the Y-axis direction of 135 ° is used.

  As shown in FIGS. 3, 5, and 6, a pair of lens protection protrusions 66 a for protecting the photographing lens 16 a are provided near the + Y side and the −Y side of the photographing lens 16 a in the black panel portion BP. 66b protrudes. The outer surface of each lens protection projection is formed of a curved surface so that it will not be injured when touched by a finger or the like.

  The electronic camera 16 captures an image of a subject (for example, a user, a document, etc.) through the photographing lens 16a, converts the captured image into an image signal (electric signal), and outputs the image signal to the image input I / F 112 (FIG. 4). reference). As an example of the image pickup device of the electronic camera 16, a CCD, a CMOS, or the like is used.

  Also, as shown in FIG. 3, a speaker 18 is connected to the sound output direction at the −X side end of the space formed by the + Y side end of the first storage portion 12a and the second storage portion 12b. Are arranged in the + Z direction.

  Here, as the speaker 18, for example, a full range type round shape is adopted, but other types may be used. The speaker 18 is connected to the voice input / output I / F 120, and outputs a voice signal transmitted from the voice input / output I / F 120 as voice (see FIG. 4).

  In a portion corresponding to the speaker 18 in the black panel portion BP, a sound emission port 68 configured to include a plurality of small through holes for discharging sound output from the speaker 18 to the outside is formed ( (See FIG. 3).

  In addition, in the space on the + X side of the electronic camera 16 in the space formed by the + Y side end of the first housing portion 12a and the second housing portion 12b, the microphone 20 has a voice input direction approximately in the −Z direction. It is arranged to be.

  Here, for example, a small omnidirectional microphone is adopted as the microphone 20, but a microphone having directivity may be used. The microphone 20 is connected to the voice input / output I / F 120 and transmits the input voice to the voice input / output I / F 120 as a voice signal (see FIG. 4).

  A portion corresponding to the microphone 20 of the black panel portion BP is formed with a sound intake port 70 including at least one small through hole for capturing sound input via the microphone 20. .

  Further, at the -X side and -Y side corners of the black panel portion BP, there are a remaining amount lamp 72 that displays the remaining amount of charge of the external battery, and a line lamp 74 that displays the connection status of the Internet line. Is provided.

  Hereinafter, as shown in FIG. 8, an axis orthogonal to the X axis and parallel to the cover panel 26 is referred to as an α axis, and an axis orthogonal to the X axis and the α axis is referred to as a β axis. Therefore, when the video conference apparatus 100 is in the reference form, the α axis coincides with the Y axis, and the β axis coincides with the Z axis.

  Here, the end portion on the −Y side of the first housing portion 12a, that is, the end portion on the −α side, is connected to the first housing 10 via a torque hinge (not shown) whose axial direction is the X-axis direction. ing. The liquid crystal panel of the display 14 is visible from the + β side, the photographing lens 16a is exposed to the + β side, the sound output direction of the speaker 18 is approximately + β side, and the sound input direction of the microphone is approximately −β. On the side.

  That is, the second housing 12 is separated from the first housing 10 by a contact position (see FIGS. 5 and 6) where the second housing 12 is in contact with the first housing 10 around the X axis. It can be rotated relative to the position (see FIG. 8) about the X axis. Therefore, in the following, the form of the video conference apparatus 100 when the second housing 12 is located at the separated position is also referred to as an open form.

  The rotation angle θ around the X axis of the second housing 12 with respect to the first housing 10 is θ = 0 when the video conference apparatus 100 is in the reference form, that is, when the second housing 12 is located at the contact position. For example, 0 ° ≦ θ ≦ 90 ° is set. In FIG. 9, θ = 90 °, and the video conference apparatus 100 is in the maximum opening form. The second casing 12 can be held at a rotational angle θ around the X axis with respect to the first casing 10 by the action of the torque hinge. In addition, the form of the video conference apparatus 100 when the 2nd housing | casing 12 is located in a contact position is a reference | standard form. Therefore, hereinafter, the reference form is also referred to as a closed form.

  An example of a video conference using the conference system 1000 configured as described above will be described below. Here, the video conference is performed using, for example, the video conference apparatus 100 arranged at each site between 10 sites (see FIG. 1). The number of users (conference participants) at each site is, for example, one.

  As shown in FIG. 10, the user at each base is seated with the table T side facing the -Y side of the table T arranged in one room, for example.

  The user places the video conference device 100 on the upper surface of the table T, for example, in the standard form so that the operation panel unit 10b is positioned on the near side (−Y side), and the second state with respect to the first housing 10. The housing 12 is rotated around the X axis by an angle φ (for example, 60 ° ≦ φ ≦ 90 °). As a result, the display 14 is in a state in which the first casing 10 is pedestal and stands substantially facing the −Y side, that is, the display surface is visible from the −Y side. Further, in this state, the photographing lens 16a, the speaker 18 and the microphone 20 are located at substantially the same height as the height of the user's face. Further, in this state, the photographing lens 16a is generally directed to the -Y side, the sound output direction of the speaker 18 is approximately the -Y side, and the sound input direction of the microphone is approximately the + Y side.

  In addition to the placement of the video conference apparatus 100 on the table T, the user can connect wiring related to electricity and communication to the video conference apparatus 100 (for example, connection between the LAN terminal 38 and a terminal connected to the Internet, The power terminal 44 and the external power source are connected).

  Next, the user presses the power button 48 to activate the video conference device 100. At this time, a menu screen is displayed on the display 14. In this menu screen, various items such as various adjustments, start of a conference (start of bidirectional communication), and the like are displayed with icons and text information. Therefore, the user operates the cursor 52 to select an item related to the start of the conference on the menu screen, and presses the determination button 54 to determine the start of the conference.

  When the start of the conference is determined, the destination list of nine other sites is displayed on the menu screen of the display 14 of the video conference device 100 at any one site, and the user at one site operates the cursor 52 to From the destination list, another site where a video conference (two-way communication) is to be performed is selected, and the decision button 54 is pressed to decide. Here, nine other bases are selected and determined. As a result, a request for bidirectional communication is transmitted from the video conference device 100 at one site to the video conference devices 100 at nine other sites.

  When the two-way communication request arrives, the video conference apparatus 100 at each other site displays items relating to the acceptance and rejection of the request on the menu screen displayed on the display 14.

  Therefore, the user at each other site operates the cursor 52 and the determination button 54 of the video conference apparatus 100 at the other site to select and determine any of the items regarding the acceptance and rejection. Here, the items relating to acceptance are selected and determined, and as a result, the ten video conference apparatuses 100 are connected to each other via the Internet so as to be capable of bidirectional communication.

  When two-way communication is started between the ten video conference apparatuses 100, the image of the user at one site captured by the electronic camera 16 of the video conference apparatus 100 at any one site is displayed via the Internet. The video is transmitted to the video conference device 100 at the base and displayed on the display 14 of the video conference device 100.

  In addition, the voice of the user at the one location captured by the microphone 20 of the video conference device 100 at any one location is transmitted to the nine video conference devices 100 at other locations via the Internet. Output from the speaker 18.

  In this manner, a video conference is performed between the 10 sites by bidirectional image and audio communication.

  By the way, in general, in a video conference apparatus, it is necessary to display images from a plurality of other bases on the screen of a display device (for example, a monitor) so as to be visible at the same time. It is necessary to reduce the size of the image from the base. In this case, it is difficult to visually recognize the image, and it is not always easy to recognize the other site where the speaker is present only by voice. Therefore, it is desirable to make the image easily visible. In addition, when there are speakers at each of a plurality of other bases, that is, when a plurality of other bases are in a conversation state, it is also desirable to make it easy to visually recognize which other bases are in a conversation state. In addition, the location where the speaker is located in order to make it easier to grasp the atmosphere of the place when a video conference is recorded and recorded on a medium as a conference record, for example, when a person who has not participated in the video conference plays and views the medium. It is desirable to make it easy to visually recognize the above.

  Therefore, in the video conference apparatus 100, the control (a series of processes) shown in FIG. 11 is performed. The flowchart in FIG. 11 corresponds to the processing algorithm of the CPU 101. This control will be described in detail below.

  In the first step S1, it is determined whether image data and audio data from at least one other site have been received. If the network I / F 110 has received image data and audio data from at least one other site, the determination in step S1 is affirmed and the process proceeds to step S3. On the other hand, if the network I / F 110 has not received image data and audio data from another site, the determination in step S1 is denied and the flow ends. Note that the CPU 101 constantly monitors the reception status of image data and audio data in the network I / F 110.

  In step S3, based on the determination result of the volume level determination unit 160, whether or not there is another site whose volume level is equal to or higher than a predetermined threshold value, that is, the volume level of audio data from at least one other site is equal to or higher than the threshold value. It is determined whether or not. This threshold is a value that serves as an index (reference) for determining the presence or absence of a speaker at each site. If the determination in step S3 is affirmed, the process proceeds to step S5. On the other hand, if the determination in step S3 is negative, the process proceeds to step S17.

  In step S5, it is determined whether a predetermined time (for example, several seconds to several tens of seconds) has elapsed. If the determination in step S5 is affirmed, the process proceeds to step S7. On the other hand, if the determination in step S5 is negative, the same determination is repeated. In addition, the time measurement here is performed using the timer which the control apparatus 24 has, for example.

  In step S7, based on the determination result in the volume level determination unit 160, whether or not there is another site whose volume level is equal to or higher than the threshold value, that is, the volume level of the audio data from at least one other site is equal to or higher than the threshold value. It is determined whether or not there is. If the determination in step S7 is affirmative, the process proceeds to step S8. On the other hand, if the determination in step S7 is negative, the process proceeds to step S17.

  In step S8, it is determined whether or not the volume level of at least one other site among the other sites whose volume level is equal to or higher than the threshold value at the time of determination in step S7 is equal to or higher than the threshold value for the predetermined time. It is determined whether or not the volume level at the other site at the time is equal to or higher than the threshold at the time of determination at step S3. When the determination in step S8 is affirmed, the process proceeds to step S9. On the other hand, if the determination in step S8 is negative, the process returns to step S5.

  Since it is estimated that there is a speaker at another site whose volume level continues for the predetermined time or more than the above threshold, the other site is also referred to as “another site where the speaker is present”. In this case, for example, a very short voice due to competing, coughing, sneezing, etc. means that it does not correspond to utterance even if the volume level is above the threshold. Actually, it is also assumed that the volume level becomes lower than the threshold value during the utterance (within the predetermined time), but if the volume level is equal to or higher than the threshold value at any time of determination in step S3 and step S7. For example, it may be considered that the volume level is continuously higher than the threshold value for the predetermined time, that is, the utterance is considered to be continued for the predetermined time.

  In step S9, the number of other bases (other bases where the speaker is present) whose volume level continues for the predetermined time or more than the threshold is acquired.

  In the next step S11, the size of the enlarged size and the position of the image to be displayed in the enlarged size are set according to the number of other sites whose volume level continues for the predetermined time or longer than the threshold value.

  As an example, as shown in FIG. 12, when the number of other bases where the speaker is present among the nine other bases is 1, the enlarged size is the entire image from one other base (for example, another base D). Is set to a size that can be displayed on the display surface, and the position where the enlarged image is displayed is set to an arbitrary position (for example, the left portion of the display surface).

  Further, as an example, when the number of other bases where the speaker is present is two of nine other bases as shown in FIG. 13, the size of the enlarged size and the position where the image of the enlarged size is displayed. Is set so that the entire image from each of the two other bases (for example, the other bases D and F) is displayed together at, for example, the same size at different positions (for example, the left part and the right part of the display surface).

  Further, as an example, when the number of other bases where the speaker is present is three among nine other bases as shown in FIG. 14, the size of the enlarged size and the position where the image of the enlarged size is displayed. Are displayed together in the same size, for example, at different positions (for example, the upper part, the lower left part, and the lower right part) of the entire image from each of the three other bases (for example, other bases B, E, and H). Set to

  In the next step S13, the sound volume level continues and images from other sites that are equal to or greater than the threshold value are displayed at the set position in the set enlarged size (see FIGS. 12 to 14).

  In the next step S15, based on the determination result in the volume level determination unit 160, whether or not there is another site whose volume level at the time of determination in step S7 is less than the above threshold, that is, at least 1 at the time of determination in step S7. It is determined whether or not the volume level of the audio data from one other site is less than the threshold value. Since it can be estimated that there is no speaker at other sites whose volume level is less than the above threshold at the time of the determination in step S7, the other sites are also referred to as “other sites without speakers” below. Therefore, a determination unit that includes the volume level determination unit 160 and the CPU 101 and determines the presence or absence of a speaker at another base is configured. If the determination in step S15 is affirmed, the process proceeds to step S17. On the other hand, if the determination in step S15 is negative, the process returns to step S1. That is, the other bases where the speaker is present and the number of the other bases, and the other bases where the speaker is not present and the number of the other bases are detected at any time, and according to the detection result, the other bases where the speaker is The other sites where there is no speaker are displayed on the display surface of the display 14 in a reduced size. In this way, the size of the image displayed on the display surface is switched between the enlarged size and the reduced size at the own site (one site) depending on the presence or absence of the speaker.

  In step S17, the number of other sites whose volume level is less than the threshold is acquired.

  In the next step S19, the size of the reduced size and the position of the image to be displayed in the reduced size are set according to the number of other sites whose volume level is less than the threshold value.

  As an example, as shown in FIG. 15, when the number of other bases where there is no speaker among nine other bases is nine, the size of the reduced size and the position of the image displayed in the reduced size are nine. Nine images from other bases (for example, other bases A to I) are set to be located at different positions on the display surface. In FIG. 15, nine images from nine other sites are arranged in a 3 × 3 matrix on the display surface as an example.

  As an example, as shown in FIG. 12, when the number of other sites where no speaker is present among the nine other sites is 8, the size of the reduced size and the position of the image displayed in the reduced size are Eight images from eight other bases (for example, other bases A, B, C, E, F, G, H, and I) are set to be located at different positions on the display surface. In FIG. 12, a part of the image from each of the other bases A, E, and G is hidden in the shadow of the image from the other base D.

  As an example, as shown in FIG. 13, when the number of other sites where no speaker is present among the nine other sites is 7, the size of the reduced size and the position of the image displayed in the reduced size are Seven images from seven other sites (for example, other sites A, B, C, E, G, H, and I) are set to be located at different positions on the display surface. In FIG. 13, a part of the image from each of the other sites A, B, C, E, G, H, and I is hidden in the shadow of the image from the other site D or F.

  As an example, as shown in FIG. 14, when the number of other bases where no speaker is present among nine other bases is 6, the size of the reduced size and the position of the image displayed in the reduced size are Six images from six other bases (for example, other bases A, C, D, F, G, and I) are set to be located at different positions on the display surface. In FIG. 14, a part of the image from each of the other bases A, C, D, F, G, and I is hidden by the shadow of the image from at least one of the other bases B, E, and H.

  In the next step S21, images from other locations whose volume level is less than the threshold are displayed at the set position with the set reduced size (see FIGS. 12 to 15). After step S21 is executed, the flow returns to step S1.

  As described above, the video conference apparatus 100 according to the present embodiment performs a video conference (information sharing) by transmitting and receiving images and sounds via the Internet (communication network) between at least three sites (for example, 10 sites). Communication devices used at any one site, and at least two (for example, nine) individually transmitted via the Internet from at least two other sites (for example, nine other sites) different from the one site. Based on the voice data (information related to the voice), the determination means including the volume level determination unit 160 for determining the presence or absence of a speaker at the other base, and the other base determined by the determination means that there is no speaker An image sent via the Internet is displayed on the display screen in a reduced size (predetermined size), and the other means determined by the determination means that there is a speaker Displaying an image sent via the Internet on the display surface at a greater enlargement in size than the reduced size, and a, a display means including a display processing unit 140. The display means displays a plurality of images individually sent from the plurality of other sites together on the display surface in the enlarged size when a speaker is present at each of the plurality of other sites.

  In this case, when a speaker is present at each of a plurality of other bases, a plurality of images from the plurality of other bases are displayed together in an enlarged size on the display surface.

  As a result, when a speaker is present at a plurality of other bases, the speaker can be easily recognized (understood). That is, when a plurality of other bases are in a conversation state, the plurality of other bases in the conversation state can be easily recognized (understood).

  Further, the determination means determines that there is a speaker at the other site when it is considered that the state where the volume level at each of at least two other sites is equal to or greater than the threshold value continues for a predetermined time. High reliability of judgment. On the other hand, if it is determined that another site whose volume level is equal to or higher than the threshold at one time point is a site where the speaker is uniformly present, for example, a conference participant who is a speaker who is not a speaker is also a speaker. Therefore, the reliability of the determination is low.

  Further, the display means sets the size of the enlarged size according to the number of other bases determined to have a speaker by the determining means, so that the other bases are independent of the number of the other bases. It is possible to make it easy to visually recognize the image from.

  On the other hand, in the comparative example shown in FIG. 16A and FIG. 16B, another base where the speaker is located at another base with the highest volume level among the other bases (for example, another base A ′ or other base B). '), Only the other bases are displayed in an enlarged size, and the remaining other bases are displayed in a reduced size. In this case, when a speaker is present at a plurality of other sites, images displayed in an enlarged size are frequently switched. Therefore, it is difficult to recognize (understand) a plurality of other sites where the speaker is actually present. In addition, it is difficult to recognize (understand) which other bases are in a conversation state. Also, when a video conference is recorded and recorded on a medium, and the medium is played back and viewed, it is difficult to recognize (understand) which other bases are in a conversation state.

  Further, in the above embodiment, the enlarged sizes of the images from a plurality of other bases where the speaker is present are set to the same size. However, the present invention is not limited to this. For example, as shown in FIG. May be. Specifically, the image from the other base where the speaker is currently speaking (for example, another base D) is displayed in a large enlarged size, and the image from the other base where the speaker is interrupting the speech (for example, another base F) is displayed. May be displayed in a small enlarged size. Further, the utterance time at another site where the speaker is present may be measured, and the other site with a longer utterance time may be displayed in a larger enlarged size. In this case, the size of the enlarged size of the image from another base where the speaker is present may be changed gradually or stepwise according to the speech time. In this case, it is preferable to increase the enlargement size as the other site has a longer utterance time. In other words, it is preferable to reduce the enlargement size as the other site has a shorter utterance time.

  Moreover, in the said embodiment, although the image from several other bases with a speaker is displayed in the position where a display surface differs in an enlarged size, it is not restricted to this, For example, as shown in FIG. You may display so that at least one part may overlap. Specifically, an image from another site that is currently speaking (for example, another site E) is displayed in the foreground in a large enlarged size, and three other sites (for example, other sites F, H, and I) that are suspending the speech are displayed. It is also possible to display the image from the back with a small enlarged size. That is, a plurality of enlarged-size images may be displayed so that at least a part thereof overlaps. In this case, the utterance time at a plurality of other sites where the speaker is present may be measured, and the other sites with longer utterance time may be displayed on the front surface with a larger enlarged size. In this case, the size of the enlarged size of the image from another base where the speaker is present may be changed gradually or stepwise according to the speech time. In this case, it is preferable to increase the enlargement size as the other site has a longer utterance time. In other words, it is preferable to reduce the enlargement size as the other site has a shorter utterance time.

  In practice, it is assumed that the volume level becomes lower than the threshold value during the utterance (for example, during the utterance). That is, even when there is a speaker at a certain other base, it is assumed that the volume level is equal to or higher than the threshold at the time of determination in step S3 and the volume level is lower than the threshold at the time of determination in step S7. . Therefore, a series of steps similar to the series of steps including step S5 and step S7 may be performed at least once between step S3 and step S5, or between step S7 and step S8.

  Actually, when there are speakers at each of a plurality of other bases, the speakers are often talking with each other, and the time zone of the utterance is assumed to be different for each speaker. Therefore, as shown in FIGS. 19A to 19C, for example, when a plurality of images from a plurality of other bases where a speaker is present are displayed in the same size and enlarged size, the plurality of images are displayed. It is also possible to perform processing for distinguishing an image of another site where the speaker is currently speaking from other sites from an image of another site. In this case, for example, the other base having the highest volume level at the time of determination in step S7 among the plurality of other bases where the speaker is present may be regarded as the other base where the speaker is actually speaking. As a result, the other site where the speaker is actually speaking can be easily visually recognized.

  Specifically, the frame of an image from another base where the speaker is currently speaking may be different from the frame of an image from another base. For example, as shown in FIG. 19A, the frame of the image from the other site F where the speaker is currently speaking may be displayed thicker than the frame of the image from the other site D. Further, for example, the line of the image frame from another site where the speaker is currently speaking may be different from the line of the image frame from another site. Specifically, the frame of an image from another site where the speaker is currently speaking may be a solid line or the like, and the frame of an image from another site may be a broken line or the like. Further, only the frame of the image from the other base where the speaker is currently speaking may be a double line or the like. Further, for example, the color of the frame of the image from another site where the speaker is currently speaking may be different from the color of the frame of the image from another site.

  Further, for example, as shown in FIG. 19B, the resolution of the image from another site D may be lower than the resolution of the image from the other site F where the speaker is currently speaking. Further, for example, the brightness of an image from another base may be made lower than the brightness of an image from another base where the speaker is currently speaking. Further, the frame rate of an image from another site may be lower than the frame rate of an image from another site where the speaker is currently speaking. In this case, it is possible to reduce the load on the CPU 101 of the video conference apparatus 100 at one site, and thus reduce power consumption.

  Further, for example, as shown in FIG. 19C, a graphic, a symbol, a letter, a number, a pattern, a combination thereof, etc. A process of adding a figure circled) may be performed.

  In the above-described embodiment, it is determined whether or not the volume level of the audio data from each other site is equal to or greater than the threshold value at two points in time (substantially continuing for a predetermined time) at one site. Not limited to this. For example, at another site, it is determined whether or not the volume level of the audio data at the other site is substantially equal to or greater than a threshold value for a predetermined time, and the determination result is transmitted to one site via the Internet. It's also good. In this case, since the determination result of the presence / absence of the speaker at the other site is sent to the video conference apparatus at one site, there is no need for a determination means for determining the presence / absence of the speaker, and the configuration and control can be simplified. it can.

  Moreover, in the said embodiment, although the video conference is performed between 10 bases, it is not restricted to this, In short, what is necessary is just to be held between at least 3 bases. Also in this case, it is preferable to arrange the video conference apparatus 100 at each site.

  Moreover, in the said embodiment, although the display surface on which a display means displays an image is made into the display surface of the display 14, it is not restricted to this. For example, it may be the surface of a screen (for example, when a projector is connected to a video conference device), an external monitor screen, a television screen, a personal computer screen, or the like. In this case, the display 14 is not essential. Further, the size of the display surface is not particularly limited.

  Moreover, in the said embodiment, what is called a liquid crystal display is employ | adopted as a display, However, It is not restricted to this, For example, you may employ | adopt other displays, such as a plasma display and an organic electroluminescent (EL) display.

  Moreover, in the said embodiment, although an electronic camera, a microphone, and a speaker are provided in the housing | casing integrally, at least 1 of these may be a different body from a housing | casing.

  Moreover, in the said embodiment, although the video conference apparatus 100 is used in one room, it is not limited to this. Since the video conference apparatus 100 is excellent in portability as described above, it is not necessary to be installed in a specific conference room, and it can be expected to be freely carried and used in various places.

  In the above embodiment, the present invention is applied to a so-called portable type (portable) video conference apparatus 100. However, the present invention can also be applied to a so-called stationary type video conference apparatus.

  Moreover, in the said embodiment, although the number of users of each base is made into one person, it is not restricted to this, A multiple person may be sufficient. In this case, for example, an image in which all the users at each site are displayed may be sent to another site, or the view switching button 58 is operated to display an image in which some users at the site are displayed. You may send it to the base.

  Further, for example, a program for executing a series of processes shown in FIG. 11 used in the video conference apparatus (also referred to as a communication apparatus) of the above-described embodiment is an installable format or executable format file and is a CD-ROM. , Recorded on a computer-readable recording medium such as a flexible disk (FD), a CD-R, and a DVD (Digital Versatile Disk).

  Further, for example, a program for executing a series of processes shown in FIG. 11 used in the video conference apparatus (also referred to as a communication apparatus) of the above embodiment is stored on a computer connected to a network such as the Internet, and the network You may comprise so that it may provide by making it download via. In addition, a program for executing a flow executed by the video conference apparatus (also referred to as a communication apparatus) of the above embodiment may be provided or distributed via a network such as the Internet.

  Further, for example, a program for executing a series of processes shown in FIG. 11 used in the video conference apparatus (also referred to as a communication apparatus) according to the above-described embodiment may be provided by being incorporated in advance in a ROM or the like. .

  In the above embodiment, the present invention has been described with reference to an example in which the present invention is applied to a dedicated device for video conferences. However, the present invention is applicable to any device having a camera and a voice input function, such as a personal computer and a smartphone. be able to.

  Moreover, although the video conference apparatus 100 used for the video conference has been described as an example of the communication apparatus in the above embodiment, the present invention is not limited to this. That is, the communication device of the present invention can be used in various modes as a communication tool used for information sharing by transmitting and receiving images and sounds between at least three bases.

  DESCRIPTION OF SYMBOLS 14 ... Display (part of display means), 100 ... Video conference apparatus (communication apparatus), 114 ... Image output I / F (part of display means), 140 ... Display processing part (part of display means) ... Volume level determination unit (part of determination means), 1000 ... conference system.

JP-A-9-261608

Claims (8)

A communication device used at any one site for sharing information by transmitting and receiving images and sounds via a communication network between at least three sites,
A determination means for determining presence / absence of a speaker at the other base based on information on at least two voices individually transmitted via the communication network from at least two other bases different from the one base;
An image sent from the other site determined to have no speaker by the determination unit via the communication network is displayed on a display surface with a predetermined size, and the other site determined to have a speaker by the determination unit Display means for displaying an image sent from the communication network from the display screen in an enlarged size larger than the predetermined size,
The determination means determines that there is a speaker at the other base when it is considered that the state where the volume level at each of the at least two other bases is equal to or higher than a threshold value continues for a predetermined time;
The display means,
When it is determined by the determination means that there are speakers at a plurality of other sites, a plurality of images individually sent from the plurality of other sites via the communication network are combined with the display surface in the enlarged size. Displayed on the
Of the images from the plurality of other sites determined to have a speaker, an image from the other site where the speaker who is currently speaking is displayed in a first enlarged size larger than the predetermined size, The communication apparatus which displays the image from the other base where the speaker who is interrupting is displayed in the second enlarged size smaller than the first enlarged size and larger than the predetermined size . The communication apparatus according to claim 1 , wherein the display unit sets the size of the enlarged size in accordance with the number of other bases determined by the determination unit that there is a speaker. The display means, wherein when displaying the image from the other bases that are displayed in a predetermined size in the expanded size, the communication device according to claim 1 or 2, characterized in that larger and larger the image. The display means, when displaying the images from other locations that are displayed in the enlarged size in the predetermined size, in any one of claim 1 to 3, characterized in that gradually reduce the image The communication device described. The display means sets the enlarged size of a plurality of images from the plurality of other bases to the same size when the judgment means determines that there are speakers at the plurality of other bases, and claim 1 any one of 4, wherein the handle is subjected to image visually distinguishable from the other base image of another from speaker other bases currently in speech out of the image The communication apparatus as described in. A communication method used at any one site for sharing information by transmitting and receiving images and sounds via a communication network between at least three sites,
Determining the presence or absence of a speaker at the other site based on information about at least two voices individually sent via the communication network from at least two other sites different from the one site;
An image sent via the communication network from another site determined that there is no speaker in the determining step is displayed on the display surface in a predetermined size, and it is determined that there is a speaker in the determining step. Displaying together an image sent from another base via the communication network on the display surface in an enlarged size larger than the predetermined size,
In the determining step, when it is considered that the state where the volume level at each of the at least two other sites is equal to or higher than a threshold value is continued for a predetermined time, it is determined that there is a speaker at the other site;
In the displaying step ,
When it is determined in the determining step that a speaker is present at a plurality of other sites, a plurality of images individually sent from the plurality of other sites via the communication network are displayed on the display surface in the enlarged size. displayed together,
Of the images from the plurality of other sites determined to have a speaker, an image from the other site where the speaker who is currently speaking is displayed in a first enlarged size larger than the predetermined size, A communication method for displaying an image from another base where a speaker who is interrupting the speech is interrupted in a second enlarged size smaller than the first enlarged size and larger than the predetermined size. Wherein in the determining step, depending on the number of other sites where it is determined that there are calling speaker The communication method according to claim 6, characterized in that setting the size of said larger size. A program used in a communication device used in any one site in order to share information by transmitting and receiving images and sounds via a communication network between at least three sites,
A procedure for determining the presence or absence of a speaker at the other base based on information about at least two voices individually sent from at least two other bases different from the one base;
An image sent via the communication network from another site determined to have no speaker in the determining procedure is displayed on the display screen in a predetermined size, and it is determined that there is a speaker in the determining procedure . A procedure for displaying an image sent from another site via the communication network on the display surface in an enlarged size larger than the predetermined size,
In the determination procedure, when it is considered that the state where the volume level at each of the at least two other sites is equal to or greater than a threshold value continues for a predetermined time, it is determined that there is a speaker at the other site,
In the displaying procedure ,
When it is determined in the determining procedure that there are speakers at a plurality of other sites, a plurality of images individually sent from the plurality of other sites are displayed together on the display surface in the enlarged size ,
Of the images from the plurality of other sites determined to have a speaker, an image from the other site where the speaker who is currently speaking is displayed in a first enlarged size larger than the predetermined size, A program for displaying an image from another base where a speaker who is interrupting the voice is interrupted in a second enlarged size smaller than the first enlarged size and larger than the predetermined size .