JP2010114768A - Video processing unit, system, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent difficulty in seeing photographed pictures caused by reflection of projection light of a projector to a camera without sacrificing easiness of installation and convenience of portability of the projector. <P>SOLUTION: A video conference system includes: the projector 1; a camera 2 installed opposite to the projector 1; and a video processing unit 6 connected to the projector 1 and the camera 2. The camera 2 is installed at this side of a screen 3, thus setting a dark region 52 in a region overlapping the camera 2 if the screen 3 is seen from the projector 1 on a picture 51 projected by the projector 1 when the picture 51 projected by the projector 1 overlaps with the camera 2, thereby reducing the reflection of the projection light to the camera 2. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video processing apparatus, system, and program suitable for use in a video conference system in which an imaging apparatus is installed so as to face a projection apparatus.

  In recent years, video conference systems that perform bidirectional video communication using a network have been commercialized. In a video conference system, not only a CRT and a liquid crystal display but also a projector that can be easily carried and can be displayed on a large screen is used as a device for displaying an image.

  In this type of video conference system, it is important for communication to perform a conference or the like by matching the line of sight while looking at each other's faces. In order to match the line of sight, it is necessary to install a camera near the place where the image is displayed. However, when a projector is used as a device for displaying an image, there is a problem that the projected light of the projector is reflected on the camera, making it difficult to see the captured image.

  In order to realize line-of-sight matching using a projector, Patent Document 1 discloses an example in which a camera is embedded in a screen and the projector is installed and irradiated at a place where the projector projection light is outside the field of view of the camera. However, this example can be realized by a configuration in which the camera is embedded in the screen, and considering the case of projecting on a place other than the dedicated screen such as a wall, it is not easy to install and convenient to carry.

  Further, Patent Document 2 discloses an example in which a projector and a camera are installed behind the screen using a transmission screen in order to match the line of sight. However, since this example requires a special screen, it is not easy to install and convenient to carry as in the example of Patent Document 1.

Japanese Patent Laid-Open No. 2004-357036 JP-A-6-233291

  As described above, in a video conference system that uses a projector as an image display device, when the camera is installed so that the line of sight matches, the projected light of the projector is reflected on the camera, and the captured image is difficult to see. There is a problem. The techniques disclosed in Patent Documents 1 and 2 can solve this problem, but there is a problem that the ease of installation of the projector and the convenience of carrying are impaired.

  The present invention has been made in view of the above points, and the projected light of the projector is reflected on the camera without impairing the ease of installation and carrying around of the projector, making it difficult to view the captured image. The purpose is to prevent.

The video processing apparatus of the present invention is a video processing apparatus that is connected to a projection apparatus installed so as to face the imaging apparatus, and includes a dark area setting unit that sets a dark area on an image projected by the projection apparatus. It is characterized by that.
The video processing system of the present invention is a video processing system comprising a projection device, an imaging device installed so as to face the projection device, and a video processing device connected to the projection device, wherein the projection device According to the present invention, there is provided dark area setting means for setting a dark area on the projected image.
The program of the present invention is a program used in a video processing apparatus connected to a projection apparatus installed so as to face an imaging apparatus, and performs processing for setting a dark region on a video projected by the projection apparatus. Let it run.

  According to the present invention, by setting a dark region on the image projected by the projection device, it is possible to reduce the reflection of the projection light on the imaging device and to prevent the captured image from being difficult to see. . Further, since image processing is performed on the image projected by the projection device, no special projection device or screen is required, and the ease of installation of the projection device and the convenience of carrying are not impaired.

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. In each embodiment, an example in which the present invention is applied to a video conference system that is a video processing system will be described.
(First embodiment)
FIG. 1 is a diagram showing the arrangement of projectors and cameras that constitute the video conference system of the present embodiment. In the video conference system, a projector (projection device) 1 projects an image on a screen 3.

  When the user 4 is photographed by the camera (imaging device) 2, as shown in FIGS. 1A and 1B, the camera 2 is installed so as to face the user 4, that is, the camera 2 faces the projector 1. Need to be installed. In particular, in order to match the line of sight between the user 4 at the local site and the user 5 at the other site, it is necessary to install the camera 2 at a height close to the line of sight of the user 4 as shown in FIG. .

  In this state, the projection light of the projector 1 is reflected on the camera 2, and as shown in FIG. 2, the luminance in the captured image 14 of the camera 2 is saturated, and the image 14 becomes difficult to see. FIG. 2 is a diagram schematically showing a captured image 14 when the projection light of the projector 1 is reflected on the camera 2. In the captured image 14, the periphery of the projection port 12 of the projector 1 has high brightness and the reflected light 13 is generated, so that the captured image 14 is difficult to see.

  FIG. 3 is a diagram illustrating a configuration example of the video conference system according to the present embodiment. The video conference system of this embodiment includes a projector 1, a camera 2, and a video processing device 8 connected to the projector 1 and the camera 2. The counterpart device 21 on the other side has the same configuration, and the video processing device 8 is connected to the counter device 21 via the network 7 to transmit and receive video and audio.

  The video processing device 8 includes an audio output unit 31, an audio input unit 32, a display control unit 33, an operation unit 34, a video output unit 35, a video input unit 36, a communication unit 37, and an overall control unit 38. With. Each unit 31 to 37 is connected to the overall control unit 38 via an internal bus or the like, and each control is performed by the overall control unit 38.

  The speaker 41 is connected to the sound output unit 31 and outputs the sound transmitted by the opposing device 21. The microphone 42 is connected to the voice input unit 32 and inputs the voice of the user at the local site.

  The display 43 is connected to the display control unit 33 and displays an operation screen of the video processing device 8. The display 43 includes a display device such as a CRT or a liquid crystal display, but is not limited to a specific display device. The mouse 44 is connected to the operation unit 34 and inputs a user operation. Of course, in addition to the mouse 44, an input device such as a keyboard or a touch panel may be connected.

  The video output unit 35 is connected to the projector 1 and outputs a video to be projected onto the projector 1. The video input unit 36 is connected to the camera 2 and receives a captured video from the camera 2. Examples of the connection form between the projector 1 and the video output unit 35 and the connection form between the camera 2 and the video input unit 36 include USB and IEEE 802.11, but are not limited to a specific connection form.

  The communication unit 37 is connected to the opposite device 21 via the network 7 and transmits and receives video and audio. The network 7 is not limited to the Internet, and includes various networks such as an intranet and a LAN. Further, the number of the opposing devices 21 connected via the network 7 is not limited to one.

  The video processing apparatus 8 configured as described above can be configured using a general-purpose personal computer including a CPU, a ROM, a RAM, and the like, for example.

  Next, the basic principle of image processing to which the present invention is applied will be described with reference to FIG. When the camera 2 is installed between the projector 1 and the screen 3, by setting the dark region to be superimposed on the image projected by the projector 1, the reflection of the projection light on the captured image of the camera 2 is reduced. Is. Here, the dark region is a dark color region and is not limited to a specific color.

  FIG. 4A is a diagram when the screen (projection position) 3 is viewed from the projector 1. Since the camera 2 is installed in front of the screen 3, the video 51 projected by the projector 1 and the camera 2 overlap. Therefore, as shown in FIG. 4B, on the video 51 projected by the projector 1, a portion corresponding to the camera 2, specifically, a region overlapping the camera 2 when the screen 3 is viewed from the projector 1 is a dark region. The dark region 52 is set on the projected video 51 by projecting so as to be arranged. Thereby, the reflection of the projection light on the camera 2 can be reduced.

  Next, a dark region setting method will be described with reference to FIGS. As shown in FIG. 5, a dark area setting screen 61 is displayed on the display 43. In the dark region setting screen 61, the video 62 is a video projected by the projector 1 onto the screen 3, and includes, for example, a video received from the opposing device 21. A video 63 is a video captured by the camera 2.

  On the dark area setting screen 61, the dark area 64 can be set on the video 62 projected by the projector 1 while visually recognizing the captured video 63 of the camera 2. At this time, attributes such as the position and size of the dark area 64 are manipulated and adjusted so as to reduce the influence of the reflection on the camera 2.

  As shown in FIG. 6, the position and size of the dark area 64 are set by dragging the operation portion 66 set around the dark area 64 with the mouse pointer 65, as in a general window system. Do that. For example, the width of the dark area 64 can be changed by moving the mouse pointer 65 to the left and right while dragging the operation portion 66, and the height of the dark area 64 can be changed by moving the mouse pointer 65 up and down. be able to. Further, if the mouse pointer 65 is moved obliquely, the height and width of the dark area 64 can be changed simultaneously.

  Although the position and size have been described as the attributes of the dark region 64 here, the shape, color, pattern, density, etc. may be arbitrarily set. For example, the shape of the dark region 64 is not limited to a rectangular shape, and may be a circle, an ellipse, a quadrangle, a pentagon, or the like, or may be selected from a plurality of preset shapes.

  Next, the procedure for setting the dark region 64 will be described with reference to the flowchart of FIG. FIG. 7 shows a software processing procedure executed by the overall control unit 38 when the dark region 64 is set. The following processing procedure is a processing example of the dark region setting means and display control means in the present invention. The overall control unit 38 corresponds to a CPU, and executes the process of FIG. 7 by referring to a program read from a ROM, which is a computer-readable medium, to a RAM.

  First, in step S101, the overall control unit 38 acquires a video from the camera 2, and receives a video (opposite device video) from the opposing device 21 via the communication unit 37 in step S102. Next, in step S103, as shown in FIG. 5, the overall control unit 38 causes the display 43 to simultaneously display the image 62 including the facing device image and the captured image 63 of the camera 2 (dark region setting screen). In step S <b> 104, the overall control unit 38 projects the video 62 on the projector 1.

  Next, in step S <b> 105, the overall control unit 38 arranges the dark region 64 so as to overlap the video 62 on the display 43. At this time, the video on which the dark region 64 is superimposed is projected from the projector 1.

  In step S106, the overall control unit 38 adjusts the position, size, shape, and the like of the dark region 64 according to the procedure described with reference to FIG. At this time, it is visually confirmed whether or not the influence of the reflection of the projection light on the camera 2 is minimal while viewing the captured image 63 of the camera 2 on the display 43. If it is determined that the minimum value is reached, the process ends.

  In the above description, the video processing device 8 and the counter device 21 are connected via a network. However, the counter device 21 is not necessary in the present invention. For example, there is a case where the camera 2 is arranged as shown in FIG.

  As described above, in the first embodiment, when the camera 2 is installed in a place where the projection light of the projector 1 is reflected, the dark area is set on the image projected by the projector 1 to Reflection of projection light can be reduced. As a result, the display image (including the image captured by the camera 2) on the opposite device 21 is easier to see than the conventional image without dark region superimposition.

(Second Embodiment)
In the second embodiment, an example will be described in which the setting of the dark region 64 is automatically optimized by using a luminance histogram in the captured image of the camera 2 installed so as to face the projector 1. The configuration of the entire video conference system and the configuration of each device are the same as in the first embodiment. Also in the present embodiment, control and creation of an image to be projected is performed by software operating on the overall control unit 38.

  First, the basic principle will be described. In the present embodiment, a method for optimizing each of the attributes of the dark region using the position and size will be described. The attributes of the dark region are not limited to the position and size, and other attributes can be optimized by the same method.

  In the second embodiment, a two-stage process is executed: a first stage for determining the position of the dark area 64 on the image 62 projected by the projector 1 and a second stage for determining the size of the dark area 64. Thereby, the dark region 64 is set in a portion (camera portion) corresponding to the camera 2 on the video 51 projected by the projector 1.

  In the first stage, a dark area of a predetermined size is arranged on the white image created by the overall control unit 38 and projection is performed from the projector 1. That is, a dark area is temporarily set on the image projected by the projector 1. Next, an image in which the temporarily set dark region is arranged is captured and acquired by the camera 2, and a luminance histogram is created.

Then, the dark regions are rearranged, a luminance histogram is created, and the process is repeated until the luminance histogram is created for the entire image. Here, the rearrangement of the dark areas stores the sum Z _n of the frequencies from the th class to the highest luminance class as shown in FIG. Regarding the setting of the value of th, a valley of a histogram is found and set using a discriminant analysis method or the like. The discriminant analysis method is a method for determining a threshold for separating distributions. As for the value of th, the value determined first is used each time.

The sum of frequencies Z _n is obtained by the following equation (1). In Expression (1), I _max indicates the maximum of the histogram class relating to luminance, and I _th indicates the _th th class. H _i is the frequency of the histogram of luminance i.

FIG. 9A is a graph of the frequency sum Z _n . In FIG. 9A, the horizontal axis indicates the histogram number and corresponds to a change in the position of the dark region. The vertical axis indicates the value Z of the frequency sum.

  In FIG. 9A, when the dark region is arranged in the camera portion, the sum value Z of the frequencies becomes smaller than in the case where the dark region is not arranged. For this reason, if the position of the dark region when the value Z is minimized is determined as the dark region placement location, the dark region can be placed at a position corresponding to the camera portion.

In the second stage, the size of the dark region is adjusted. On the white image created by the overall control unit 38, a dark region is arranged at the position determined in the first step and projection is performed from the projector 1. Next, the entire control unit 38 acquires a captured image of the camera 2 and creates a luminance histogram. Then, the dark region is reduced, and the creation of the luminance histogram is repeated until either the length or the length of the dark region becomes zero. Further, every time a luminance histogram is created, the sum Z _n of the frequencies from the th class to the highest luminance class is obtained and stored using equation (1). Here, the setting of the value of th is the same as in the first stage. Then, using equation (2), a change amount br _m of the component sum Z _m of the partial histogram per one reduction of the dark region is obtained. In the formula (2), Z _m represents the sum of the frequencies according to the m-th formula (1).

FIG. 9B is a graph of the change amount br _m of the component sum Z _m . In FIG. 9 (b), the horizontal axis represents the number of iterations of the loop, and the vertical axis represents the value br variation component sum Z _m.

  When the dark area is smaller than the size of the camera 2, the projection light is reflected on the camera 2, and the high-luminance points in the captured image increase rapidly, so the sum value Z of the frequencies increases. For this reason, when the amount of change br of Z in FIG. 9B is the maximum, the size corresponding to the camera portion can be obtained by setting the size of the dark region.

  The above procedure is shown in the operation processing flowchart of the overall control unit 38 shown in FIG. After starting the setting of the dark area, the overall control unit 38 optimizes the position of the dark area in step S201 (first stage), and optimizes the size of the dark area in step S202 (second stage). In step S203, the overall control unit 38 stores the position and size of the dark area in the RAM and ends.

  Next, the procedure for optimizing the position of the dark region defined in step S201 will be described with reference to FIG. Here, for the sake of explanation, a black rectangle having the same aspect ratio as that of the projected image and an area of 1 / s times is used.

In step S301, the overall control unit 38 causes the projector 1 to project an image in which a dark region is arranged on a white image. In step S <b> 302, the overall control unit 38 acquires a captured video of the camera 2. In step S303, after creating the luminance histogram, the overall control unit 38 calculates and stores the frequency sum Z _n as the feature amount in step S304. Here, the initial arrangement of the dark region is the upper left corner on the white image. The overall control unit 38 moves the dark region to the right when rearranging the dark region, and returns to the left end and moves the dark region downward when reaching the right end. In step S <b> 305, the overall control unit 38 determines to rearrange the dark areas until the dark areas are arranged with respect to the entire image. As a result, histograms of a plurality of videos are generated.

After that, in step S306, the overall control unit 38 compares the frequency sum Z _n stored in step S304, determines the dark area placement location where the sum is minimum, and stores the position in the memory in step S307. To do.

  Next, the procedure for optimizing the size of the dark region defined in step S202 will be described with reference to FIG. In addition, although the process of step S201 and the process of step S202 are demonstrated using the same flowchart, an actual process differs.

In step S301, the overall control unit 38 causes the projector 1 to project an image in which a dark region is arranged on a white image. In step S <b> 302, the overall control unit 38 acquires a captured video of the camera 2. In step S303, after creating the luminance histogram, the overall control unit 38 calculates the frequency sum Z _n and stores it in the memory in step S304. Here, the initial position and size of the dark area determined by step S201 are used, and the dark area reduction ratio is reduced pixel by pixel from the top, bottom, left, and right of the dark area. In step S305, the overall control unit 38 determines that the dark area is rearranged until either the vertical or horizontal size of the dark area becomes zero. As a result, histograms of a plurality of videos are generated.

  Thereafter, in step S306, the overall control unit 38 calculates the change amount br of the frequency using the frequency stored in step S304, obtains the size of the dark region where the change amount br is maximum, and in step S307. Store size in memory.

  In this embodiment, the luminance histogram is used, but the histogram to be used is not limited to the luminance histogram. For example, in the case of the RGB color model, it is possible to set the dark region by using each of the RGB components, combinations, or all histogram components.

(Third embodiment)
As a third embodiment, a case where the video processing apparatus 8 does not have the display 43 as shown in FIG. 11 will be described. The present invention can obtain an effect even in a configuration without the display 43. In the video processing apparatus 8 shown in FIG. 11, the configuration other than the display 43 and the display control unit 33 is the same as that of the video processing apparatus 8 shown in FIG. 3, and the description thereof is omitted here.

  In the case of the video processing apparatus 8 that does not have the display 43, there is no means for confirming the captured video 63 of the camera 2 at the local site shown in FIG. Therefore, simultaneously with the original projected video 62, the captured video 63 is arranged in the projected video. Then, the dark area of the screen 3 is set. That is, the screen 3 is used as display means.

  FIG. 12 shows a projected image in this case. The overall control unit 38 displays the original projected video 101 and then superimposes and displays the captured video 102 of the camera 2. At this time, the captured image 102 is arranged so as not to be superimposed on the camera 2 portion of the image 101 projected by the projector 1. Then, the position, size, shape, and the like to be superimposed are set by operating the dark area 103 with the mouse cursor 104. Since the dark region setting method is the same as that of the first embodiment, description thereof is omitted.

(Fourth embodiment)
As a fourth embodiment, composition of dark areas by the projector 1 will be described. In the present invention, the dark region arrangement processing and setting processing for the projected video can be performed by the projector 1. Specifically, a position for setting a dark region is designated by the operation unit 34 of the video processing device 8. Next, the position information for disposing the dark region and the projection video information are individually transmitted to the projector 1, and after the composition process is performed in the projector 1, the information is projected on the screen 3.

  Alternatively, the dark area setting itself may be performed in the projector 1. That is, when the projector 1 has an operation unit, the position where the dark region is set in the projector 1 is designated. Next, in the projector 1, a dark region is synthesized at a position corresponding to the designated position on the projection image transmitted from the image processing device 8.

(Fifth embodiment)
As a fifth embodiment, a method for preventing deterioration in operability due to the presence of a dark region will be described. Regarding the configuration and arrangement of the devices, either the first embodiment or the third embodiment may be used.

  When the dark region 64 is set by operating with the mouse pointer 65, the dark region is only the frame of the region or semi-transparent. Thereby, it is possible to visually recognize the background of the dark region during the setting operation.

  Even after the dark area is set, if a predetermined operation using an input device such as the mouse pointer 65 is performed, the dark area is made to be a frame only, or translucent or transparent. Thereby, even after the dark region is set, the background of the dark region can be visually recognized, and as a result, the operability can be prevented from being lowered.

  In addition to the method of making the dark area only a frame or semi-transparent (or transparent), there is a method for making the dark area transparent every other pixel, a method of making the dark area transparent every other second, etc. Yes, it is not limited to a specific method.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus. In this case, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, the functions of the above-described embodiments are not limited to being realized by executing the program code read by the computer. For example, an OS (basic system or operating system) running on the computer performs part or all of the actual processing based on an instruction of the program code, and the functions of the above-described embodiments are realized by the processing. May be.

  Further, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. In this case, after being written in the memory, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the function of the above-described embodiment is performed by the processing. Is realized.

It is a figure which shows arrangement | positioning of the projector and camera which comprise a video conference system. It is the figure which represented typically the picked-up image when the projection light of a projector was reflected on the camera. It is a figure which shows the structural example of the video conference system of 1st Embodiment. It is a figure for demonstrating the setting position of a dark area. It is a figure which shows the example of a dark area setting screen. It is a figure for demonstrating the adjustment method of a dark area. It is a flowchart which shows the setting procedure of the dark area in 1st Embodiment. It is a characteristic view which shows the example of a brightness | luminance histogram. It is a characteristic view which shows the graph of the calculated value at the time of dark area setting using a brightness | luminance histogram. It is a flowchart which shows the setting procedure of the dark area in 2nd Embodiment. It is a figure which shows the structural example of the video conference system of 3rd Embodiment. It is a figure which shows the example of a screen which superimposes and displays the picked-up image of a camera on a projection image | video.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Projector 2 Camera 3 Screen 7 Network 8 Video processing apparatus 31 Audio | voice output part 32 Audio | voice input part 33 Display control part 34 Operation part 35 Image | video output part 36 Image | video input part 37 Communication part 38 Overall control part 41 Speaker 42 Microphone 43 Display 44 Mouse

Claims (11)

A video processing device connected to a projection device installed to face the imaging device,
An image processing apparatus comprising dark area setting means for setting a dark area on an image projected by the projection apparatus.   The video processing device according to claim 1, wherein a dark region is set in a region overlapping with the imaging device when a projection position is viewed from the projection device.   3. The apparatus according to claim 1, wherein at least one of a position, a shape, a size, a color, a pattern, and a density of a dark region is included as an attribute, and the attribute can be arbitrarily set. Video processing equipment.   The video processing apparatus according to any one of claims 1 to 3, wherein a dark region can be set while visually recognizing a captured image of the imaging apparatus. As a dark area setting means, it has a display control means for displaying on the display means a dark area setting screen for simultaneously displaying an image projected by the projection device and a captured image of the imaging device,
The dark region can be set on the image projected by the projection device while visually recognizing the image captured by the imaging device on the dark region setting screen. The video processing apparatus according to the item.   As dark area setting means, means for provisionally setting a dark area on an image projected by the projection device, means for photographing and obtaining an image in which the temporarily set dark area is arranged, and the provisional setting A means for generating histograms of a plurality of videos obtained by changing the attributes of the dark areas, and a means for determining the attributes of the dark areas by comparing feature amounts obtained from the plurality of histograms. The video processing apparatus according to any one of claims 1 to 3.   The video processing apparatus according to claim 1, further comprising a unit that makes it possible to visually recognize a background of a dark area set on an image projected by the projection apparatus.   The video processing apparatus according to claim 1, further comprising a synthesizing unit that synthesizes a dark area on an image projected by the projection apparatus. Means for transmitting information of the dark area set by the dark area setting means to the projection device;
The video processing apparatus according to claim 1, wherein the projection apparatus causes the projection apparatus to perform a composition process for synthesizing a dark region on an image projected by the projection apparatus. A video processing system comprising a projection device, an imaging device installed to face the projection device, and a video processing device connected to the projection device,
An image processing system comprising dark region setting means for setting a dark region on an image projected by the projection device. A program used in a video processing device connected to a projection device installed to face an imaging device,
A program for causing a computer to execute processing for setting a dark region on an image projected by the projection device.

Images