JP4862645B2 - Video conferencing equipment - Google Patents

Description

  The present invention relates to a video conferencing apparatus that communicates video, images, and audio used in a video conference between conference rooms separated from each other.

  Conventionally, when a video conference is performed between a plurality of points distant from each other, a video conference device (video conference device) as shown in Patent Document 1 is arranged at each point, and a conference person exists so as to surround the video conference device. Sit and hold a meeting.

In the video conference apparatus of Patent Document 1, each conference person is equipped with a microphone with a radio wave generator, and radio waves are radiated from the microphone that picks up the highest level of sound. The person photographing camera detects the speaker direction by receiving the radio wave, directs the camera toward the speaker direction, and captures an image centered on the speaker. The video data and audio data are encoded and transmitted to the destination video conference apparatus.
JP-A-6-276514

  When a video conference is performed, there are cases where it is desired to refer not only to the video of a conference person such as a speaker as described above but also to materials in common between distant points. With the apparatus of Patent Document 1, it is possible to switch and acquire a speaker's video, but it is not possible to project a document as it is. For this reason, in order to project a document using the configuration of Patent Document 1, it is sufficient for a conference person to manually view the document in front of the camera, but the document cannot be completely fixed, so the image is blurred. End up. In addition, the document cannot be captured as it is (original image) due to the influence of the curvature of the lens. In addition, as an alternative method to refer to the material in common, it is possible to send the material as data, but it provides intuitive and flexible material such as writing and explaining during the meeting. I can't.

  Accordingly, an object of the present invention is to provide a video conferencing apparatus capable of accurately and clearly transmitting even a highly flexible material together with audio and video.

This invention produces a camera for imaging a predetermined region, and the image data generation means for generating image data based on the video imaged in the camera, the picked-up sound data picked up ambient sounds own device Communication data is formed by sound collection means, sound emission means for emitting sound emission data, a housing provided with the sound collection means and sound emission means, and the sound collection sound data and the video data. A communication means for transmitting the communication data to the outside, acquiring sound emission sound data from the communication data from the outside, and providing the sound emission means to the sound emission means; and a support means for supporting the camera with respect to the casing. A video conferencing apparatus, wherein the camera simultaneously images a conference person imaging area and an area close to the camera in the vicinity of the housing, and the video data generating means Corresponds to the imaging area 1 partial video data, by cutting out only the azimuth area corresponding to the picked-up sound data, the first partial image data cut out by compensation by the first correction processing, a second partial image corresponding to the region close to the camera The data is corrected by a second correction process different from the first correction process.

With this configuration, the video conferencing apparatus simultaneously acquires the first partial video data corresponding to the conference person imaging area and the second partial video data corresponding to the area where the material close to the camera is arranged by one camera. Is done. In the first partial video data, only the azimuth area corresponding to the collected sound data is cut out and appropriately corrected by the first correction processing. The second partial video data is adjusted so as to be easily viewable by the corresponding second correction processing.

  Thereby, a conference person image and a still image such as a document are acquired at the same time, and are corrected according to each shooting specification. As a result, it is also possible to simultaneously transmit the conference person video and the material image appropriately corrected to the destination apparatus.

  The video conference apparatus according to the present invention includes selection means for selecting partial video data used for communication data. The video data generation means of the video conference apparatus gives the partial video data selected by the selection means to the communication means.

  In this configuration, any one selected from the conference video and the still image is transmitted. As a result, a still image that hardly changes with time can be transmitted only when necessary, so that no extra load is applied to the communication system.

  In the video conference apparatus of the present invention, the camera has a fish-eye lens, the center area of the area imaged by the fish-eye lens is an area close to the camera, and at least the peripheral area outside the center area is the conference person imaging area. It is characterized by that.

  In the video conferencing apparatus having this configuration, a fisheye lens is used as a specific camera specification. Then, an area corresponding to the center of the fisheye lens is set as an area close to the camera, and correction is appropriately performed by correction processing according to the area. As the conference person imaging area, the center area may be used when the mode is changed, but the peripheral area is mainly used. Therefore, the video in the conference area is appropriately corrected by the correction processing according to the selected area in each case. Thereby, even if the video (image) of the proximity region near the camera and the video of the conference person imaging region are imaged via the fisheye lens, the respective images are appropriately corrected.

The video data generating means of the video conference apparatus of the present invention is formed integrally with the camera. The communication unit of the video conferencing system of the present invention is integrally formed in the housing with a sound unit and sound pickup means release. The video data generating means of the video conferencing system of the present invention is integrally formed in the housing with a sound unit and sound pickup means release. As a result, the video conference apparatus is configured in a compact manner.

  The video conference apparatus according to the present invention further includes a display monitor for reproducing video data. The communication means of this video conference apparatus acquires video data included in the communication data and gives it to the display monitor.

  As a result, the conference video and materials can be easily shared by both parties simply by arranging and connecting the video conference apparatus of the present invention at each point where the communication conference is performed.

  According to the present invention, with a simple camera direction operation, the video of the speaker is corrected by the correction processing according to the video of the speaker, and the image of the material is corrected by the correction processing according to the image of the material. Both the speaker video and the material image can be transmitted to the counterpart device accurately and clearly. As a result, in the video conference using the present apparatus, it is possible to easily realize a conference that is more realistic and easy to understand.

A video conference apparatus according to a first embodiment of the present invention will be described with reference to the drawings.
1 and 2 are external views of the video conference apparatus according to the present embodiment, in which (A) is a plan view and (B) is a side view. 1 and 2 show only the structure of the sound emission / collection device, camera, and stay that are mechanically characteristic, and the communication terminals, the sound emission / collection device, and the cable that electrically connects the camera are not shown. . FIG. 1 shows the mechanism state in the conference shooting mode, and FIG. 2 shows the mechanism state in the document shooting mode.
FIG. 3 is a block diagram showing the main configuration of the video conference apparatus according to the present embodiment.

In FIG. 1, FIG. 2, FIG. 3, and the drawings referred to in this specification, the microphone is represented or generically represented by “MC”, and the speaker is represented or generically represented by “SP”. .
The video conference apparatus according to the present embodiment includes a sound emitting and collecting apparatus 1 having a disk shape in plan view, a camera 2 having an imaging function and a video data generating function, and the camera 2 at a predetermined position with respect to the sound emitting and collecting apparatus 1. And a stay 3 to be installed. Although not shown in FIGS. 1 and 2, the sound emitting and collecting apparatus 1 and the camera 2 are electrically connected, and the video conference apparatus is electrically connected to the sound emitting and collecting apparatus 1 and the camera 2. A communication terminal to be connected is provided.

  The communication terminal 5 demodulates the communication data received from the communication terminal of the other party's video conferencing apparatus connected via the network 500, and obtains the sound output sound signal, the other party apparatus ID, and the speaker orientation data. Then, it is given to the sound emitting and collecting apparatus 1 on the own apparatus side connected by the cable. Further, the communication terminal 5 generates communication data based on the collected sound signal and the speaker position data received from the sound emitting and collecting device 1 on the own device side and the video data received from the camera 2. The communication terminal 5 transmits the generated communication data to the communication terminal of the partner video conference device. Moreover, the communication terminal 5 mediates transmission / reception of the speaker position data between the sound emitting and collecting apparatus 1 and the camera 2 depending on the case.

  The sound emission and collection device 1 includes a disk-shaped housing 11. Specifically, the casing 11 has a circular shape in plan view, the area between the top surface and the bottom surface is narrower than the area of the middle part in the vertical direction, and the shape in side view has a ceiling from one point in the height direction. It has a shape that narrows toward the surface and narrows from the one point toward the bottom surface. That is, it has a shape having inclined surfaces on the upper side and the lower side from the one point. The top surface of the housing 11 is formed with a recess 110 that is narrower than the top surface and has a predetermined depth, so that the center of the recess 110 in plan view coincides with the center of the top surface. Is set.

  The 16 microphones MC1 to MC16 are installed inside the top surface of the housing 11 along the side surface of the recess 110, and each of the microphones MC1 to MC16 has a center in the plan view of the sound emitting and collecting device 1 as a rotation center. They are arranged at an equiangular pitch (in this case, an interval of about 22.5 °). At this time, the microphone MC1 is in the direction of θ = 0 °, and the microphones MC1 to MC16 are arranged along the direction in which θ increases by 22.5 ° in order. For example, the microphone MC5 is disposed in the θ = 90 ° direction, the microphone MC9 is disposed in the θ = 180 ° direction, and the microphone MC13 is disposed in the θ = 270 ° direction. Further, each of the microphones MC1 to MC16 has a single directivity, and each microphone is arranged so as to have a strong directivity in the central direction as viewed from above. For example, the microphone MC1 has the direction of θ = 180 ° as the center of directivity, the microphone MC5 has the direction of θ = 270 ° as the center of directivity, and the microphone MC9 has the direction of θ = 0 (360) ° as the center of directivity. The microphone MC13 has the direction of θ = 90 ° as the center of directivity. The number of microphones is not limited to this, and may be set as appropriate according to specifications.

  The four speakers SP1 to SP4 are respectively installed so that the inclined surface on the lower side of the housing 11 and the sound emitting surface coincide with each other, and each speaker SP1 to SP4 has a center in a plan view of the sound emitting and collecting device 1. The rotation centers are arranged at equiangular pitches (in this case, intervals of about 90 °). At this time, the speaker SP1 is arranged in the same θ = 0 ° direction as the microphone MC1, the speaker SP2 is arranged in the same θ = 90 ° direction as the microphone MC5, and the speaker SP3 is arranged in the same θ = 180 ° direction as the microphone MC9. The speaker SP4 is arranged in the same θ = 270 ° direction as the microphone MC13. The speakers SP1 to SP4 have strong directivity in the front direction of the sound emitting surface. The speaker SP1 emits sound around the θ = 0 ° direction, and the speaker SP2 centers around the θ = 90 ° direction. The speaker SP3 emits sound around the θ = 180 ° direction, and the speaker SP4 emits sound around the θ = 270 ° direction.

  As described above, the speakers SP1 to SP4 are arranged on the lower side of the casing 11, the microphones MC1 to MC16 are arranged on the upper side of the casing 11, and the sound collection direction of the microphones MC1 to MC16 is the center direction of the casing 11. This makes it difficult for the microphones MC1 to MC16 to collect the wraparound sound from the speakers SP1 to SP4. For this reason, it becomes difficult to be influenced by the wraparound sound in the speaker position detection described later, and the speaker position can be detected with higher accuracy.

The operation unit 111 is installed on an inclined surface on the upper side of the housing 11 and includes various operation buttons and a liquid crystal display panel (not shown).
The input / output I / F 102 (not shown in FIGS. 1 and 2) is an inclined surface on the lower side of the housing 11 and is installed at a position where the speakers SP1 to SP4 are not installed. Audio data and various control data Is provided with a terminal capable of communicating. Then, by connecting the terminal of the input / output I / F 102 and the communication terminal with a cable or the like, communication is performed between the sound emitting and collecting apparatus 1 and the communication terminal.

The sound emission and collection device 1 has a functional configuration as shown in FIG. 3 in addition to such a structural configuration.
The control unit 101 performs general control such as setting, sound collection, and sound emission of the sound emitting and collecting apparatus 1 and gives control to each unit of the sound emitting and collecting apparatus 1 based on the operation instruction content input by the operation unit 111.

(1) Sound emission The input / output I / F 102 outputs sound emission sound signals S1 to S3 received from the communication terminal 5 to the channels CH1 to CH3, respectively. Note that channel assignment may be set as appropriate according to the number of received sound signals for sound emission. Further, the input / output I / F 102 receives the counterpart device ID from the communication terminal 5 and allocates a channel CH for each counterpart device ID. For example, when there is one connected counterpart device, the audio data from the counterpart device is assigned to the channel CH1 as the sound output sound signal S1. Further, when there are two connected counterpart devices, the audio data from the two counterpart devices are individually assigned to the channels CH1 and CH2 as sound emission sound signals S1 and S2, respectively. Similarly, when there are three connected counterpart devices, the audio data from the three counterpart devices are individually assigned to channels CH1, CH2, and CH3 as sound emission sound signals S1, S2, and S3, respectively. . Channels CH <b> 1 to CH <b> 3 are connected to the sound emission control unit 103 via the echo cancellation unit 107.
Further, the input / output I / F 102 extracts the speaker orientation data Py at the other party sound emission and collection device from the communication terminal 5 and supplies it to the sound emission control unit 103 together with the channel information.

  The sound emission control unit 103 generates speaker output signals SPD1 to SPD4 to be given to the speakers SP1 to SP4 based on the sound emission sound signals S1 to S3 and the speaker orientation information Py.

  The D / A-AMP 104 performs digital-analog conversion on the speaker output signals SPD1 to SPD4, amplifies the signals with a constant amplification factor, and supplies them to the speakers SP1 to SP4, respectively. The speakers SP1 to SP4 convert the given speaker output signals SPD1 to SPD4 into sounds and emit the sounds.

  By performing such sound emission processing, the sound emitted from the speakers SP1 to SP4 has a predetermined delay relationship and amplitude relationship, so that the conference person feels as if the sound is emitted from the set virtual sound source. Can be given to.

(2) Sound Collection The microphones MC1 to MC16 collect sound from the outside such as the sound generated by the conference person and generate sound collection signals MS1 to MS16. Each A / D-AMP 105 amplifies the corresponding collected sound signals MS <b> 1 to MS <b> 16 with a predetermined amplification factor, performs analog-digital conversion, and outputs the result to the collected sound control unit 106.

  The sound collection control unit 106 synthesizes the acquired sound collection signals MS1 to MS16 with different delay control patterns and amplitude patterns, and generates sound collection beam signals having different directions in the central direction of directivity. For example, with the sound emitting and collecting apparatus 1 as the center, eight sound collecting beam signals are generated in which the entire circumference 360 ° is divided into eight angles, that is, the central direction of the directivity is shifted every 45 °. The sound collection control unit 106 compares the amplitude levels of these sound collection beam signals, selects the sound collection beam signal MBS having the highest amplitude level, and outputs it to the echo cancellation unit 107. The sound collection control unit 106 acquires the speaker orientation corresponding to the selected sound collection beam signal, generates the speaker orientation information Pm, and provides it to the input / output I / F 102.

  The echo cancellation unit 107 generates a pseudo regression sound signal based on the sound output sound signals S1 to S3 for the input sound collection beam signal MBS, and a pseudo regression sound from the sound collection beam signal MBS. It consists of a post processor that subtracts the signal. The echo cancellation circuit subtracts the pseudo-regression sound signal from the output sound collection beam signal MBS while sequentially optimizing the filter coefficient of the adaptive filter, so that the speakers SP1 to SP4 included in the output sound collection beam signal MBS are connected to the microphone. The wraparound component to MC1 to MC16 is removed. The collected sound beam signal MBS from which the wraparound component has been removed is output to the input / output I / F 102.

  The input / output I / F 102 associates the collected sound beam signal MBS from which the return sound has been removed by the echo canceling unit 107 with the speaker orientation information Pm from the collected sound control unit 106 and outputs it to the communication terminal 5.

  As shown in FIGS. 1 and 2, the camera 2 is installed at a position fixed to the sound emitting and collecting apparatus 1 by the stay 3. At this time, the camera 2 is installed by the stay 3 so as to be rotatable between a horizontal direction (direction facing the camera 2 shown in FIG. 1) and a vertical downward direction (direction facing the camera 2 shown in FIG. 2). Yes.

  The stay 3 includes a main body portion 31, a camera support portion 32, a main body support portion 33, and a sound emitting and collecting device attachment portion 34. The main body portion 31 is formed of a linear member having a predetermined width and is installed in a shape extending in a direction of a predetermined angle with respect to the vertical direction by the main body support portion 33. A camera support portion 32 is installed at one end in the extending direction of the main body portion 31 via a hinge 203, and a sound emission and sound collection device mounting portion 34 is installed at the other end. The sound emitting and collecting device mounting portion 34 is formed of a flat plate having an opening portion in which the leg portion 12 of the housing 11 is fitted, and is integrally formed with the main body portion 31, for example.

  The end portion of the main body 31 on the camera support portion 32 side has a shape in which only both end walls in the width direction remain and the center portion in the width direction opens. The opening has a shape that does not come into contact with the main body 31 when the camera 2 installed on the camera support 32 rotates between the horizontal direction and the vertical downward direction.

  The hinge 203 has a structure in which the camera support portion 32 is installed so as to be rotatable with respect to the main body portion 31. In addition, the hinge 203 and the camera holding unit 32 have a structure that is semi-fixed when the camera 2 and the camera support unit 32 face in the horizontal direction and in the vertical downward direction. For example, the hinge 203 is fixed to the main body 31, and the recesses are formed at the horizontal position and the vertical downward position of the hinge 203, respectively. The camera support portion 32 has a hinge-side end provided with a convex portion that fits into the concave portion, and is provided with a shape that biases the convex portion from within the camera support portion 32 with a spring or the like. Thereby, the camera 2 can be rotated between the horizontal direction and the vertical downward direction, and can maintain a mechanical state in the horizontal direction and the vertical downward direction.

  The mechanism unit including the hinge 203 and the camera support unit 32 functions as the switch 4. For example, an electrode is installed in each of the concave portion and the convex portion, and the conduction and release of these are detected electrically. At this time, the connection or detection signal is set so that different signals are obtained between the horizontal recess and the vertical downward recess. The switch 4 is formed by such a structure, and the detection result of the switch 4 is given to the camera 2. Thereby, the camera 2 can identify whether the camera 2 is facing the horizontal direction or the vertical downward direction, and can acquire the video.

  The camera 2 includes an imaging unit 21 and a video processing unit 22. The imaging unit 21 includes a fisheye lens, and images an area from an infinite distance to the fisheye lens installation surface in all directions with the front direction of the camera 2 as the center. The imaging data is given to the video processing unit 22.

  The video processing unit 22 acquires the direction (hereinafter referred to as a shooting direction) of the camera 2 detected from the switch 4 (the hinge 203 and the camera support unit 32) of the stay 3. Based on the acquired shooting direction and the speaker orientation data Pm from the sound emitting and collecting apparatus 1 via the communication terminal 5, the video processing unit 22 extracts only the necessary part from the imaged data and corrects the image, thereby converting the video data. Generate. The generated video data is given to the communication terminal 5.

  Next, the usage method of the video conference apparatus and the video data generation method in the video processing unit 22 will be described more specifically. In the following description, the case where there are five conference members on the own device side is shown, but the number of conference members is not particularly limited to this.

FIG. 4 is a diagram illustrating a situation in which the video conference apparatus according to the present embodiment is arranged and a video conference is performed with another point connected to the network, and the case where the camera 2 captures the participants 601 to 605 is illustrated. FIG.
FIG. 5 is an explanatory diagram used for explaining video data generation. (A) shows a video (image) taken through a fisheye lens, and (B) and (C) show image correction concepts for each conference direction. Show.

FIG. 6 is a diagram illustrating a situation in which the video conference apparatus according to the present embodiment is arranged to perform a video conference with another point connected to the network, and is a diagram illustrating a case where the camera 2 captures the material 650. .
FIGS. 7A and 7B are explanatory diagrams used for explaining video data generation. FIG. 7A shows a video (image) captured through a fisheye lens, and FIG. 7B shows a concept of image correction at the time of document imaging.

  When a video conference is performed, the conference participants 601 to 605 are seated at a position excluding one end in the longitudinal direction with respect to the oval table 700. On the table 700, an integrated member of a circular sound emitting and collecting apparatus 1 and a camera 2 fixed to the same by a stay 3 is installed. At this time, the camera 2 is installed so that the axis parallel to the longitudinal direction of the table 700 coincides with the central axis of the fisheye lens in a state of being oriented in the horizontal direction. Under the table 700, the communication terminal 5 is installed. The communication terminal 5 is electrically connected to the sound emission and collection device 1 and the camera 2, and is connected to the network 500. The communication terminal 5 is electrically connected to the display 6. The display 6 includes a liquid crystal display, for example, and is installed near the end of the table 700 on the side where the participants 601 to 605 are not seated. At this time, the display 6 is installed so that the display surface faces in the direction of the table 700.

  When a video conference is performed in such a state, the video conference device including the sound emission and collection device 1, the camera 2, and the communication terminal 5 transmits the video of the conference to the destination video conference device in two modes.

(1) Conference participant shooting mode When any of the conference participants 601 to 605 sets the camera 2 in the horizontal direction, the video processing unit 22 of the camera 2 selects the conference participant shooting mode based on the detection signal from the switch 4. Detect that. When the video processing unit 22 detects the conference person shooting mode, the video processing unit 22 gives a selection signal of the mode to the communication terminal 5.

  The imaging unit 21 of the camera 2 acquires imaging data obtained by imaging all the conference participants 601 to 605 present on the device side through the fisheye lens, and outputs the acquired imaging data to the video processing unit 22. Here, since the imaging data passes through the fisheye lens, the imaging area is circular as shown in FIG. When the conference shooting mode is selected, the video processing unit 22 acquires, with respect to the circular imaging data, a coordinate system in which the horizontal direction that is curved in an arc shape is represented by an azimuth angle φ and the vertical direction is represented by an elevation angle ψ. . That is, in the front direction of the fish lens, the same height as the lens axis is set to φ = 0 ° and ψ = 0 °. Further, φ is set so that φ increases in the negative direction in the direction spreading leftward from the coordinates, and φ increases in the positive direction in the direction spreading rightward. Therefore, from the leading edge of the fisheye lens of the camera 2, the direction perpendicular to the axis of the fisheye lens to the left with respect to the photographing direction is φ = −90 °, and from the leading edge of the fisheye lens of the camera 2 to the right with respect to the photographing direction. Thus, the direction perpendicular to the axis of the fisheye lens is φ = + 90 °. In addition, ψ increases in the positive direction in the direction extending upward from the coordinates, and ψ increases in the negative direction in the direction extending downward. Therefore, from the leading edge of the fisheye lens of the camera 2, the direction upward with respect to the photographing direction and perpendicular to the axis of the fisheye lens is ψ = + 90 °, and from the leading edge of the fisheye lens of the camera 2 to the downward direction with respect to the photographing direction. The direction perpendicular to the axis of the fisheye lens is ψ = −90 °.

  The sound emission and collection device 1 obtains the voice of the conference participant who is speaking by the above-described processing, detects the conference direction, and provides the communication terminal 5 with the collected sound data and the speaker orientation information θ. For example, if the conference person 601 shown in FIG. 4 speaks, the sound emitting and collecting apparatus 1 detects the orientation θ1 of the conference person 601, and the sound collection voice data and the speaker orientation information based on the voice from the conference party 601 direction. θ1 is given to the communication terminal 5. Further, if the conference person 605 speaks, the sound emitting and collecting apparatus 1 detects the orientation θ2 of the conference person 605, and transmits the collected sound data based on the voice from the direction of the conference person 605 and the speaker orientation information θ2 to the communication terminal. Give to 5. The communication terminal 5 gives the speaker orientation information θ to the video processing unit 22 of the camera 2.

  The video processing unit 22 corrects the imaging data based on the speaker orientation information θ from the communication terminal 5. The video processing unit 22 stores in advance the relationship between the speaker orientation information θ and the orientation angle φ set in the imaging data. When the video processing unit 22 receives the speaker orientation information θ, the video processing unit 22 reads the corresponding orientation angle φ. For example, when the video processing unit 22 receives the speaker azimuth information θ1 for the conference 601, the video processing unit 22 reads the corresponding azimuth angle φ = 0 °. For example, when the video processing unit 22 receives the speaker orientation information θ <b> 2 for the conference 605, the video processing unit 22 reads the corresponding orientation angle φ = −90 °.

  The video processing unit 22 sets an image extraction azimuth angle range including a predetermined azimuth angle width including the read azimuth angle φ. Further, the video processing unit 22 sets an image extraction elevation angle range having a predetermined elevation angle width including the elevation angle ψ = 0 °. Then, the video processing unit 22 determines an image extraction region based on the set azimuth angle range and elevation angle range, and acquires imaging data corresponding to the region as image data.

  For example, when the video processing unit 22 reads out the azimuth angle φ = 0 °, the video processing unit 22 sets the range of the azimuth angle φ1 to the azimuth angle φ2 (φ1 <0 ° <φ2) including φ = 0 ° as the azimuth angle range. Further, the video processing unit 22 sets the range of elevation angle ψ1 to elevation angle ψ2 (ψ1 <ψ2) including ψ = 0 ° as the elevation angle range. Then, the video processing unit 22 acquires the image data 621 by setting an image extraction region with the azimuth angle ranges φ1 to φ2 and the elevation angle ranges φ1 to φ2. Further, for example, when the image processing unit 22 reads out the azimuth angle φ = −90 °, the azimuth angle range falls within a range of azimuth angle φ3 to azimuth angle φ4 (φ3 <−90 ° <φ4) including φ = −90 °. Set to. Further, the video processing unit 22 sets the range of elevation angle ψ3 to elevation angle ψ4 (ψ3 <ψ4) including ψ = 0 ° as the elevation angle range. Then, the video processing unit 22 sets the image extraction region based on the azimuth angle range φ3 to φ4 and the elevation angle range φ3 to φ4, and acquires the image data 622.

  The video processing unit 22 performs image correction conversion for each acquired image extraction region. More specifically, correction conversion is performed so that each pixel defined by two angular directions φ direction and ψ direction is applied to a pixel of an orthogonal two-dimensional plane coordinate (XY coordinate system). At this time, the video processing unit 22 stores in advance a conversion processing table between the φ-ψ coordinate system and the XY coordinate system, and calculates the XY coordinates based on the obtained φ-ψ coordinates of each pixel. Calculate and compensate. Note that the video processing unit 22 stores a coordinate conversion calculation formula in advance, and may perform correction conversion using the coordinate conversion calculation formula.

  For example, as shown in FIG. 5B, the video processing unit 22 uses the plane coordinate system and the horizontal direction as the X axis for the image data 621 set in the azimuth angle range φ1 to φ2 and the elevation angle range φ1 to φ2. The image data is converted into corrected image data 621 ′ set by x1 to x2 and y1 to y2 with the vertical direction as the Y axis. By this conversion, the person image 611 of the conference 601 acquired in the φ-ψ coordinate system is converted into a corrected person image 631 in the XY coordinate system (planar coordinate system). By converting to the XY coordinate system in this way, the corrected person image 631 becomes close to the natural body image of the conference 601.

  Also, for example, as shown in FIG. 5C, the video processing unit 22 converts the image data 622 set in the azimuth angle range φ3 to φ4 and the elevation angle range ψ3 to ψ4 into a plane coordinate system in the horizontal direction X. The image is converted into corrected image data 622 ′ set by x3 to x4 and y3 to y4 with the vertical direction as the axis. By this conversion, the person image 615 of the conference person 605 acquired in the φ-ψ coordinate system is converted into a corrected person image 635 in the XY coordinate system (planar coordinate system). By converting to the XY coordinate system in this way, the corrected person image 635 becomes close to the natural body image of the conference 601.

  The video processing unit 22 attaches time information to the corrected image data including the corrected human image approaching the natural body in this way, and outputs the corrected image data to the communication terminal 5 as video data. Such generation and output of the corrected image data are sequentially performed. If the received speaker orientation information θ changes, the center direction of the corrected image data is switched in accordance with the change.

  The communication terminal 5 generates communication data by associating the video data from the video processing unit 22, the collected sound data, and the speaker orientation information θ, and transmits the communication data to the video conference device of the other party via the network 500. Accordingly, it is possible to provide a conference person who is present around the other party's video conference apparatus with an image close to the natural body of the conference participant who is speaking and the speech of the conference participant.

(2) Document shooting mode When any of the participants 601 to 605 sets the camera 2 vertically downward as shown in FIG. 6, the video processing unit 22 of the camera 2 uses the detection signal from the switch 4 to Detect that the document shooting mode is selected. When the video processing unit 22 detects the material photographing mode, the video processing unit 22 gives a selection signal of the mode to the communication terminal 5.

  In addition, any of the conference participants 601 to 605 places the material 650 around the vertical downward position of the hinge 203 in the table 700. At this time, if the material placement marking is performed on the table 700 in advance, the material 650 can be placed easily and appropriately.

  The imaging unit 21 of the camera 2 acquires imaging data obtained by imaging the material 650 placed on the table 700 through the fisheye lens, and outputs the acquired imaging data to the video processing unit 22. Here, since the imaging data passes through the fisheye lens, the imaging area is circular as shown in FIG.

  When the document photographing mode is selected, the image processing unit 22 for circular image data, the distance r extending in the radial direction from the origin with the center of the image data as the origin, and a predetermined direction (in FIG. 7, from the origin). It is acquired in an r-η coordinate system expressed by an angle η with respect to the imaging data (the right direction is the 0 ° direction). The video processing unit 22 cuts out image data 680 in a preset range from the acquired imaging data.

  The video processing unit 22 corrects the image data 680 in the r-η coordinate system by converting it into corrected image data 680 'in the XY plane coordinate system. At this time, the video processing unit 22 stores in advance a coordinate conversion processing table in which the center coordinates of the r-η coordinate system and the XY coordinate system are matched, and is based on the acquired r-η coordinates of each pixel. X-Y coordinates are calculated and correction conversion is performed. Note that the video processing unit 22 stores a coordinate conversion calculation formula in advance, and may perform correction conversion using the coordinate conversion calculation formula.

  By this conversion, the material image 660 of the material 650 acquired in the r-η coordinate system is converted into a corrected material image 670 in the XY coordinate system (planar coordinate system). By converting into the XY coordinate system in this way, the corrected material image 670 becomes close to the natural body image of the material 650. That is, the image data of the material 650 in which the characters are not distorted can be acquired.

  The communication terminal 5 generates communication data including the image data of the material 650 acquired from the video processing unit 22 and transmits the communication data to the partner video conference apparatus via the network 500. Thereby, it is possible to provide clear and easy-to-view material images to the conference attendees who are present around the other party's video conference apparatus. At this time, if the collected sound data is acquired from the sound emission and collection device 1, the communication terminal 5 generates and transmits communication data including the collected sound data together with the image data of the material 650. Also good.

  As described above, by using the configuration and processing of the present embodiment, it is possible to acquire and transmit a conference person's video and material image in a state suitable for each specification. At this time, the video corresponding to the specifications of the conference video and the document image can be easily acquired by simply changing the camera in two directions, the horizontal direction and the vertical downward direction.

Next, a video conference apparatus according to the second embodiment will be described with reference to the drawings.
FIG. 8 is an external view of an assembly member including the sound emitting and collecting apparatus 1, the camera 2, and the support 7 in the video conference apparatus of the present embodiment, (A) is a plan view, and (B) is a side view. It is.
FIG. 9 is a diagram illustrating a situation of a video conference apparatus using the video conference apparatus of the present embodiment, where (A) is a plan view and (B) is a side view. 8 and 9, the cables connected to the sound emission and collection device 1 and the camera 2 are not shown.

10A and 10B are diagrams for explaining generation of video data by the video conferencing apparatus according to the present embodiment. FIG. 10A is a diagram illustrating imaging data, FIG. 10B is a conceptual diagram of image correction at the center of the imaging data, C) is a conceptual diagram of image correction around the image data.
In the video conference apparatus of the present embodiment, the configurations and processes of the sound emitting and collecting apparatus 1 and the communication terminal 5 are the same as those of the video conference apparatus of the first embodiment. On the other hand, the video conferencing apparatus of the present embodiment is different from the first embodiment in the installation structure of the camera 2, that is, the structure of the support 7, and the video processing method in the video processing unit 22 of the camera 2, and the switch 4 is omitted. It has been done.

  As shown in FIG. 8, a support 7 is disposed around the disc-shaped sound emitting and collecting apparatus 1. The support 7 has four vertical support shafts extending in the vertical direction, two horizontal support shafts arranged at a distance h1 from the upper surface of the sound emitting and collecting device 1, and a distance from the upper surface of the sound emitting and collecting device 1. It consists of four horizontal spindles arranged at the position of h2 (> h1). The two horizontal support shafts arranged at the distance h1 have a structure that intersects at a substantially central position when the sound emitting and collecting apparatus 1 is viewed in plan, and are held at the distance h1 by the four vertical support shafts. The horizontal support shafts arranged at the distance h2 are assembled so as to be substantially square in a plan view, and are held at the distance h2 by four vertical support shafts.

  The camera 2 is installed at the intersection of two horizontal spindles at a distance h1. The camera 2 is installed so that the imaging direction is vertically upward.

  The mounting table 8 is supported by four horizontal support shafts at a distance h2, and the mounting table 8 is formed of a highly transmissive glass, an acrylic plate, or the like. At this time, the mounting table 8 and the camera 2 are installed so that the center of the mounting table 8 and the axis of the fisheye lens of the camera 2 substantially coincide with each other in a plan view.

  On the mounting table 8, the material 650 is placed with the printing surface in a vertically downward direction, that is, in a direction in contact with the mounting table 8.

  Here, the height of the camera 2 and the height of the mounting table 8, that is, the distances h1 and h2, as shown in FIG. 9, at least the faces of the conference participants 601 to 604 can be photographed by the camera 2, and The horizontal support shaft that supports the mounting table 8 may be set so as not to be hidden.

  When the video conferencing apparatus having such a configuration is used, the imaging data acquired by the imaging unit 21 of the camera 2 is as shown in FIG. That is, since the imaging data is taken through a fish-eye lens, the entire imaging area is circular all-area image data 610, and the document image 660 of the document 650 is projected at the center, and each conference participant 601 is located in the periphery. ˜604 person images 641 to 644 are displayed.

  For the circular image data, the image processing unit 22 sets the center of the image data as the origin, a distance r extending in the radial direction from the origin, and a predetermined direction (in FIG. 10, the right direction from the origin to the image data is 0 °. It is obtained in an r-η coordinate system expressed by an angle η with respect to (direction). The video processing unit 22 cuts out image data 681 in a preset range from the acquired imaging data.

  The video processing unit 22 corrects the image data 681 in the r-η coordinate system by converting it into corrected image data 681 'in the XY plane coordinate system. At this time, the video processing unit 22 stores in advance a coordinate conversion processing table in which the center coordinates of the r-η coordinate system and the XY coordinate system are matched, and is based on the acquired r-η coordinates of each pixel. X-Y coordinates are calculated and correction conversion is performed. Note that the video processing unit 22 stores a coordinate conversion calculation formula in advance, and may perform correction conversion using the coordinate conversion calculation formula.

  By this conversion, as shown in FIG. 10B, the material image 660 of the material 650 acquired in the r-η coordinate system is converted into a corrected material image 670 in the XY coordinate system (planar coordinate system). By converting into the XY coordinate system in this way, the corrected material image 670 becomes close to the natural body image of the material 650. That is, the image data of the material 650 in which the characters are not distorted can be acquired.

  In addition, the video processing unit 22 acquires peripheral image data 682 obtained by removing the image data 681 near the center from the entire region image data 610. The video processing unit 22 sets a region to be extracted based on the speaker position information acquired from the sound emitting and collecting apparatus 1 via the communication terminal 5 as in the first embodiment. That is, the video processing unit 22 extracts a region including the image of the conference participant who is speaking, and acquires partial image data 683. At this time, the video processing unit 22 acquires partial image data in the r-η coordinate system. Specifically, as shown in FIG. 10C, the video processing unit 22 determines the coordinates of the four corners of the fan-shaped corner including the image of the corresponding conference, based on the speaker orientation information ( r10, η10), (r10, η20), (r20, η20), (r20, η10) are set and acquired.

  The video processing unit 22 performs correction conversion on the acquired partial image data 683. Specifically, correction conversion is performed so that each pixel defined in the r-η coordinate system is applied to a pixel in an orthogonal two-dimensional plane coordinate (XY coordinate system). At this time, the video processing unit 22 stores in advance a conversion processing table between the r-η coordinate system and the XY coordinate system, and calculates the XY coordinates based on the acquired r-η coordinates of each pixel. Calculate and compensate. Note that the video processing unit 22 stores a coordinate conversion calculation formula in advance, and may perform correction conversion using the coordinate conversion calculation formula.

  For example, as shown in FIG. 10C, the video processing unit 22 converts the partial image data 683 set in the distance range r10 to r20 and the azimuth angle range η10 to η20 into the plane coordinate system and the horizontal direction as the X axis. Is converted into corrected image data 683 ′ set by x10 to x20 and y10 to y20 with the vertical direction as the Y axis. By this conversion, the person image 644 of the conference person 604 acquired in the r-η coordinate system is converted into a corrected person image 654 in the XY coordinate system (planar coordinate system). By converting to the XY coordinate system in this way, the corrected person image 654 becomes close to the natural body image of the conference person 604.

  The video processing unit 22 attaches time information to the corrected image data including the acquired correction material image 670 and the corrected image data including the corrected human image 654, and outputs the image data to the communication terminal 5. Such generation and output of the corrected image data are sequentially performed. If the received speaker orientation information θ changes, video data in which only the corrected image data including the corrected person image is switched according to the change. Is output.

  The communication terminal 5 generates communication data by associating the video data from the video processing unit 22, the collected sound data, and the speaker orientation information θ, and transmits the communication data to the video conference device of the other party via the network 500. Thus, a conference image can be provided to a conference person who is present in the vicinity of the video conference apparatus of the other party, together with a video close to the natural state of the conference participant who is speaking and the speech of the conference participant.

  As described above, by using the configuration and processing of the present embodiment, a video conference apparatus that simultaneously acquires and transmits a conference participant's video and a material image that are speaking can be realized with a relatively simple structure.

  In the present embodiment, an example in which a conference person image and a document image are acquired and transmitted at the same time has been shown. However, acquisition of a document image is not performed regularly, but is performed temporarily and at this timing. You may make it transmit only. In this case, since the material image does not change except when the material is replaced, the content of information transmitted to the other party is not reduced even when compared with the case of constantly transmitting the material image. On the other hand, while the material image is not transmitted, the processing and the network load are reduced by the amount of data of the material image, so that the processing and transmission can be performed at higher speed. Note that the material image acquisition timing may be such that when the new material is placed, an acquisition operation input may be performed from the operation unit, an image analysis unit is provided, and the acquired image is different from the previous image. A new acquisition timing may be set.

  In each of the above-described embodiments, an example in which a video processing unit is provided in the camera has been shown. However, the video processing unit can be realized by a device independent of the camera, or installed in a sound emitting and collecting device or a communication terminal. May be. Accordingly, since the camera has a simpler structure, a general-purpose video camera can be used as long as it has a lens capable of photographing the necessary area described above.

  In the above description, the example in which the communication terminal is provided independently of the sound emission and collection device has been described. However, the function of the communication terminal may be provided in the sound emission and collection device. As a result, the number of components of the video conference apparatus is reduced, so that a simpler and smaller video conference apparatus can be realized.

It is an external view at the time of the conference person photographing mode of the video conference apparatus of the first embodiment. It is an external view at the time of document photography mode of the video conference apparatus of 1st Embodiment. It is a block diagram which shows the main structures of the video conference apparatus of 1st Embodiment. It is a figure which shows the condition (conference person imaging | photography mode) which arrange | positions the video conference apparatus of 1st Embodiment and performs a video conference with the other points connected to the network. It is explanatory drawing used for description of the video data generation at the time of a conference person photographing mode. It is a figure which shows the condition (material imaging | photography mode) which arrange | positions the video conference apparatus of 1st Embodiment and performs a video conference with the other points connected to the network. It is explanatory drawing used for description of the video data generation at the time of document photography mode. It is an external view of the assembly member which consists of the sound emission and collection apparatus 1, the camera 2, and the support body 7 in the video conference apparatus of 2nd Embodiment. It is the figure which showed the condition of the video conference apparatus using the video conference apparatus of 2nd Embodiment. It is a figure explaining the production | generation of the video data by the video conference apparatus of 2nd Embodiment.

Explanation of symbols

1- Sound emitting and collecting device, 2-camera, 3-stay, 4-switch, 5-communication terminal, 6-display, 7-support, 8-mounting table, 11-housing, 12-leg, 21 -Imaging unit, 22-Video processing unit, 31- Main unit, 32-Camera support unit, 33- Main unit support unit, 34- Sound output device mounting unit, 102- Input / output I / F, 103- Sound output control unit , 105-A / D-AMP, 106-sound collection control unit, 107-echo cancellation unit, 110-recess, 111-operation unit, 203-hinge, 500-network, 601-605-conference, 610-all areas Image data, 611, 615-person image, 621-correction image data, 622-correction image data, 631,635-correction person image, 641-644-person image, 650-material, 654-correction image, 660-material Image, 670-Compensation Ryozo, 680,681- compensation image data, 682- periphery image data, 683- partial image data, 700- Table

Claims (7)

A camera for imaging a predetermined area;
Video data generating means for generating video data based on video captured by the camera;
Sound collecting means for collecting sound around the device and generating collected sound data ;
A sound emission means for emitting the emitted sound data ;
A housing provided with the sound collecting means and the sound emitting means ;
Communication means that forms communication data with the collected sound data and the video data, transmits the communication data to the outside, and obtains sound emission sound data from the communication data from the outside and gives the sound emission means to the sound emission means; ,
Support means for supporting the camera with respect to the housing;
A video conferencing apparatus comprising:
The camera simultaneously images a conference person imaging area and an area close to the camera in the vicinity of the housing,
The video data generation means includes
Only the azimuth area corresponding to the collected sound data is cut out from the first partial video data corresponding to the conference person imaging area, and the cut out first partial video data is corrected by the first correction processing,
A video conference apparatus that corrects second partial video data corresponding to an area close to the camera by a second correction process different from the first correction process. Comprising a selection means for selecting partial video data used for communication data;
The video conference apparatus according to claim 1 , wherein the video data generation unit provides the communication unit with the partial video data selected by the selection unit. 3. The camera according to claim 1, wherein the camera has a fish-eye lens, a center area of an area imaged by the fish-eye lens is an area close to the camera, and at least a peripheral area outside the center area is the conference person imaging area. The video conferencing device described. The video data generating means, video conferencing apparatus according to any one of claims 1 to 3, wherein said camera and are integrally formed. The video data generating means, video conferencing apparatus according to any one of claims 1 to 3 which is integrally formed on the housing together with the release sound means and said sound collecting means. Wherein the communication means includes a video conferencing device according to any one of claims 1 to 5 which is integrally formed on the housing together with the release sound means and said sound collecting means. It has a display monitor that plays back video data,
It said communication means acquires image data included in the communication data, video conferencing apparatus according to any one of claims 1 to 6, applied to the display monitor.

Images