JP5171560B2 - Video processing apparatus, system, and program - Google Patents

Description

  The present invention relates to a video processing apparatus, system, and program suitable for use in a video conference system in which an imaging apparatus is installed so as to face a projection apparatus.

  In recent years, video conference systems that perform bidirectional video communication using a network have been commercialized. In a video conference system, not only a CRT and a liquid crystal display but also a projector that can be easily carried and can be displayed on a large screen is used as a device for displaying an image.

  In this type of video conference system, it is important for communication to perform a conference or the like by matching the line of sight while looking at each other's faces. In order to match the line of sight, it is necessary to install a camera near the place where the image is displayed. However, when a projector is used as a device for displaying an image, there is a problem that the projected light of the projector is reflected on the camera, making it difficult to see the captured image.

JP-A-10-11263 Japanese Patent No. 03463711

  As described above, in a video conference system that uses a projector as an image display device, when the camera is installed so that the line of sight matches, the projected light of the projector is reflected on the camera, and the captured image is difficult to see. There is a problem.

  Therefore, a configuration in which a dark region is set on an image projected by a projector can be considered. However, when the dark area is arranged, an image behind the dark area cannot be displayed. In particular, when the video behind the dark area is a conversation person, it is difficult to communicate while looking at the other party. In addition, there may be a problem that an object such as a window, an image, or a button for performing an operation behind a dark area cannot be displayed. In particular, when there is an object such as a button on the window, the operability is deteriorated because it cannot be visually operated.

  In a multi-window system that displays a plurality of windows on a screen, Patent Document 1 discloses an example in which an overlapping area is detected as an overlapping area, and the overlapping display area is avoided and the display area is reduced. ing. In this example, after reducing the display area, the display area is reconstructed and displayed. Further, Patent Document 2 discloses an example in which when a window overlaps, one of the overlapping windows is moved to increase the visible area of the window.

  According to the examples disclosed in Patent Document 1 and Patent Document 2, when windows overlap each other, information on the window behind can be displayed by reducing the window. Further, by moving the window, it is possible to display more information about the window behind.

  However, in these examples, reduction and movement are performed regardless of information in the window such as a button for performing an operation arranged in the window and an area for displaying an image. When these examples are applied to a video conference in which a dark area is arranged, the size of the window to be displayed becomes small, and it becomes difficult to see the other party's video. Further, even if the window that overlaps the dark area is moved and the visible area is enlarged, it does not necessarily overlap with the button for performing the operation or the area for displaying the video. For this reason, in the examples disclosed in Patent Document 1 and Patent Document 2, convenience is not necessarily improved in the video conference system.

  Therefore, in the present invention, when a dark region is set on the image projected by the projector, it is possible to avoid overlapping the dark region with a window or image on the image projected by the projector, an operation button, or the like. The purpose is to do so.

Video processing apparatus of the present invention, for example, an image processing apparatus for connecting the image pickup device, in the installed Ru projection device so as to face the image pickup device, the area on the projection image projected by the projection device And a dark region setting means for setting a dark region for reducing the reflection of projection light on a captured image of the imaging device, and setting a superimposition prohibition region on a display portion of the projection image projected by the projection device The superimposition prohibition area setting means moves the display part where the superposition prohibition area setting means sets the superimposition prohibition area or the size thereof so that the superimposition of the dark area and the superposition prohibition area is avoided. And a superposition avoiding means for changing .
Also, for example, the video processing apparatus of the present invention is a video processing apparatus connected to a projection apparatus and an imaging apparatus installed between the projection apparatus and a screen so as to face the projection apparatus, and the projection apparatus Dark area setting means for setting a dark area in an area overlapping with the imaging apparatus when the screen is viewed from the projection apparatus among areas on the projected image projected on the screen, and projection projected by the projection apparatus A superposition prohibition region setting means for setting a superposition prohibition region on the video, and changing the projection position of the projection video, or changing the size of the projection video, so that the dark region and the superposition prohibition region are And a superposition avoiding means for avoiding superposition.
Further, for example, the video processing device of the present invention is a video processing device connected to an imaging device and a projection device installed so as to face the imaging device, on the projection video projected by the projection device. A dark region setting unit that sets a region as a dark region for reducing the reflection of projection light on a captured image of the imaging device, and a display portion of the projection image projected by the projection device is superimposed on the dark region When the first superposition prohibition area and the second superposition prohibition area are set by the superposition prohibition area setting means and the superposition prohibition area setting means, the first superposition prohibition area is set. By changing the layout of the first display portion set in the region and the second display portion set in the second superimposition prohibition region, the first and second superposition prohibition regions and the dark region are changed. Turn over Moving the first and second display portions so as to be, or, characterized in that it comprises a superimposing avoiding means for changing the size.

  According to the present invention, by setting the superimposition prohibition area on the image projected by the projection device, the superimposition of the dark region and the superimposition prohibition region is avoided, so the window or image on the image projected by the projection device, Visibility and operability can be improved by preventing the operation buttons and the dark region from overlapping.

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. In each embodiment, an example in which the present invention is applied to a video conference system that is a video processing system will be described.
(First embodiment)
FIG. 1 is a diagram showing the arrangement of projectors and cameras that constitute the video conference system of the present embodiment. In the video conference system, a projector (projection device) 1 projects an image on a screen 3.

  When the user 4 is photographed by the camera (imaging device) 2, as shown in FIGS. 1A and 1B, the camera 2 is installed so as to face the user 4, that is, the camera 2 faces the projector 1. Need to be installed. In particular, in order to match the line of sight between the user 4 at the local site and the user 5 at the other site, it is necessary to install the camera 2 at a height close to the line of sight of the user 4 as shown in FIG. .

  In this state, the projection light of the projector 1 is reflected on the camera 2, and as shown in FIG. 2, the luminance in the captured image 14 of the camera 2 is saturated, and the image 14 becomes difficult to see. FIG. 2 is a diagram schematically showing a captured image 14 when the projection light of the projector 1 is reflected on the camera 2. In the captured image 14, the periphery of the projection port 12 of the projector 1 has high brightness and the reflected light 13 is generated, so that the captured image 14 is difficult to see.

  FIG. 3 is a diagram illustrating a configuration example of the video conference system according to the present embodiment. The video conference system of this embodiment includes a projector 1, a camera 2, and a video processing device 8 connected to the projector 1 and the camera 2. The counterpart device 21 on the other side has the same configuration, and the video processing device 8 is connected to the counter device 21 via the network 7 to transmit and receive video and audio.

  The video processing device 8 includes an audio output unit 31, an audio input unit 32, a display control unit 33, an operation unit 34, a video output unit 35, a video input unit 36, a communication unit 37, and an overall control unit 38. With. Each unit 31 to 37 is connected to the overall control unit 38 via an internal bus or the like, and each control is performed by the overall control unit 38.

  The speaker 41 is connected to the sound output unit 31 and outputs the sound transmitted by the opposing device 21. The microphone 42 is connected to the voice input unit 32 and inputs the voice of the user at the local site.

  The display 43 is connected to the display control unit 33 and displays an operation screen of the video processing device 8. The display 43 includes a display device such as a CRT or a liquid crystal display, but is not limited to a specific display device. The mouse 44 is connected to the operation unit 34 and inputs a user operation. Of course, in addition to the mouse 44, an input device such as a keyboard or a touch panel may be connected.

  The video output unit 35 is connected to the projector 1 and outputs a video to be projected onto the projector 1. The video input unit 36 is connected to the camera 2 and receives a captured video from the camera 2. Examples of the connection form between the projector 1 and the video output unit 35 and the connection form between the camera 2 and the video input unit 36 include USB and IEEE 802.11, but are not limited to a specific connection form.

  The communication unit 37 is connected to the opposite device 21 via the network 7 and transmits and receives video and audio. The network 7 is not limited to the Internet, and includes various networks such as an intranet and a LAN. Further, the number of the opposing devices 21 connected via the network 7 is not limited to one.

  The video processing apparatus 8 configured as described above can be configured using a general-purpose personal computer including a CPU, a ROM, a RAM, and the like, for example.

  Next, the basic principle of image processing to which the present invention is applied will be described with reference to FIG. When the camera 2 is installed between the projector 1 and the screen 3, by setting the dark region to be superimposed on the image projected by the projector 1, the reflection of the projection light on the captured image of the camera 2 is reduced. Is. Here, the dark region is a dark color region and is not limited to a specific color.

  FIG. 4A is a diagram when the screen (projection position) 3 is viewed from the projector 1. Since the camera 2 is installed in front of the screen 3, the video 51 projected by the projector 1 and the camera 2 overlap. Therefore, as shown in FIG. 4B, on the video 51 projected by the projector 1, a portion corresponding to the camera 2, specifically, a region overlapping the camera 2 when the screen 3 is viewed from the projector 1 is a dark region. The dark region 52 is set on the projected video 51 by projecting so as to be arranged. Thereby, the reflection of the projection light on the camera 2 can be reduced.

  Next, a dark region setting method will be described with reference to FIGS. As shown in FIG. 5, a dark area setting screen 61 is displayed on the display 43. In the dark region setting screen 61, the video 62 is a video projected by the projector 1 onto the screen 3, and includes, for example, a video received from the opposing device 21. A video 63 is a video captured by the camera 2.

  On the dark area setting screen 61, the dark area 64 can be set on the video 62 projected by the projector 1 while visually recognizing the captured video 63 of the camera 2. At this time, attributes such as the position and size of the dark area 64 are manipulated and adjusted so as to reduce the influence of the reflection on the camera 2.

  As shown in FIG. 6, the position and size of the dark area 64 are set by dragging the operation portion 66 set around the dark area 64 with the mouse pointer 65, as in a general window system. Do that. For example, the width of the dark area 64 can be changed by moving the mouse pointer 65 to the left and right while dragging the operation portion 66, and the height of the dark area 64 can be changed by moving the mouse pointer 65 up and down. be able to. Further, if the mouse pointer 65 is moved obliquely, the height and width of the dark area 64 can be changed simultaneously.

  Although the position and size have been described as the attributes of the dark region 64 here, the shape, color, pattern, density, etc. may be arbitrarily set. For example, the shape of the dark region 64 is not limited to a rectangular shape, and may be a circle, an ellipse, a quadrangle, a pentagon, or the like, or may be selected from a plurality of preset shapes.

  Next, the procedure for setting the dark region 64 will be described with reference to the flowchart of FIG. FIG. 7 shows a software processing procedure executed by the overall control unit 38 when the dark region 64 is set. The following processing procedure is a processing example of the dark region setting means referred to in the present invention. The overall control unit 38 corresponds to a CPU, and executes the process of FIG. 7 by referring to a program read from a ROM, which is a computer-readable medium, to a RAM.

  First, in step S101, the overall control unit 38 acquires a video from the camera 2, and receives a video (opposite device video) from the opposing device 21 via the communication unit 37 in step S102. Next, in step S103, as shown in FIG. 5, the overall control unit 38 causes the display 43 to simultaneously display the image 62 including the facing device image and the captured image 63 of the camera 2 (dark region setting screen). In step S <b> 104, the overall control unit 38 projects the video 62 on the projector 1.

  Next, in step S <b> 105, the overall control unit 38 arranges the dark region 64 so as to overlap the video 62 on the display 43. At this time, the video on which the dark region 64 is superimposed is projected from the projector 1.

  In step S106, the overall control unit 38 adjusts the position, size, shape, and the like of the dark region 64 according to the procedure described with reference to FIG. At this time, it is visually confirmed whether or not the influence of the reflection of the projection light on the camera 2 is minimal while viewing the captured image 63 of the camera 2 on the display 43. If it is determined that the minimum value is reached, the process ends.

  In the above description, the video processing device 8 and the counter device 21 are connected via a network. However, the counter device 21 is not necessary in the present invention. For example, there is a case where the camera 2 is arranged as shown in FIG.

  As described above, when the camera 2 is installed at a location where the projection light of the projector 1 is reflected, the dark region 52 is set on the video 51 projected by the projector 1 so that the projection light is reflected on the camera 2. Can be reduced. As a result, the display image (including the image captured by the camera 2) on the opposite device 21 is easier to see than the conventional image without dark region superimposition.

  So far, the configuration in which the dark region 52 is set on the video 51 projected by the projector 1 has been described. Hereinafter, a configuration that avoids overlapping of the dark area 52 with the window and the video on the video 51, buttons for operation, and the like will be described. Here, the video 51 projected by the projector 1 will be described on the assumption that a plurality of display objects (display portions in the present invention) are arranged hierarchically.

  As shown in FIG. 8, when a dark region 52 is set on a video 51 projected by the projector 1, a display object (a window 81, a video (for example, a conversation person) 82 in the window 81), and a button 83 for performing an operation. ) May be hidden in the dark region 52. In this case, when the conversation is performed while looking at the face of the interlocutor in the window 81, the dark area 52 becomes annoying. Moreover, it becomes difficult to visually observe the button 83, and the operability is lowered. Therefore, as described below, the dark area 52 is not overlapped with the portion of the window 81 that needs to be displayed, such as the image 82 and the button 83.

  In detail, first, the dark area 52 is arranged as described above, and the portion where the person's video 82, the button 83 for performing the operation, and the like are arranged on the window 81 is set as the superposition prohibition area 84. To do. At this time, an image projected on the screen 3 is shown in FIG. Next, the window 81 is arranged so as to avoid the overlap between the dark region 52 and the overlap prohibition region 84 by moving the window 81 or changing (reducing) the size. At this time, an image projected on the screen 3 is shown in FIG.

  Here, with reference to FIG. 10, the setting method of the superimposition prohibition area | region 84 is demonstrated. The following processing procedure is a processing example of the superposition prohibition area setting means referred to in the present invention. FIG. 10 shows a setting screen 1001 for the superposition prohibition area displayed on the display 43. Although not shown in FIG. 10, a window 81 (see FIG. 9) of the video 51 projected by the projector 1 on the screen 3 is displayed on the setting screen 1001 for prohibiting superimposition, for example.

  First, on the setting screen 1001, by operating the mouse pointer 1002, a rectangular area 1003 is set in a portion where it is necessary to avoid overlapping with the dark area 52. The position and size of the rectangular area 1003 are set by dragging the operation portion 1004 set around the rectangular area 1003 with the mouse pointer 1002, as in a general window system. For example, the width of the rectangular area 1003 can be changed by moving the mouse pointer 1002 left and right while dragging the operation portion 1004, and the height of the rectangular area 1003 can be changed by moving the mouse pointer 1002 up and down. be able to. Further, if the mouse pointer 1002 is moved obliquely, the height and width of the rectangular area 1003 can be changed simultaneously.

  The rectangular area 1003 may be used as the superposition prohibition area 84 as it is, but the shape of the superposition prohibition area 84 may be set. In this case, for example, it may be possible to select from a plurality of preset shapes. Specifically, as shown in FIG. 10, in addition to an ellipse, there are a circle, a pentagon, and the like, but the present invention is not limited to a specific shape.

  When the superposition prohibition area 84 is set on the window 81 in this way, when the window 81 moves, the superposition prohibition area 84 also moves corresponding to the movement of the window 81. Further, when the size of the window 81 changes, the superposition prohibition area 84 also changes corresponding to the change in the size of the window 81.

  Further, if the mouse pointer 1002 approaches the dark area 52 while setting the superposition prohibition area 84, the display of the dark area 52 is temporarily stopped. Thereby, even in the vicinity of the dark region 52, the superposition prohibition region 84 can be set while being visually observed.

  FIG. 11 is a diagram showing the coordinates of a video range (video display range) 1101 to be output to the video output unit 35 of the video processing device 8, a window 1102, a superposition prohibition area 1103, and a dark area 1104. The window 1102, the superposition prohibition area 1103, and the dark area 1104 correspond to the window 81, the superposition prohibition area 84, and the dark area 52 shown in FIGS. FIG. 12 is a flowchart showing a process for arranging the window 1102 so as to avoid the overlap between the dark area 1104 and the overlap prohibited area 1103.

  As shown in FIG. 12, after the start, first, in step S1, the dark region 1104 is arranged. Next, in step S <b> 2, the overlap prohibition area 1103 on the window 1102 is set. In step S3, superimposition detection between the dark area 1104 and the superposition prohibition area 1103 is performed. If it is determined in step S4 that the dark region 1104 and the superimposition prohibited region 1103 are superimposed by the superimposition detection in step S3, the process proceeds to step S5. In step S5, the window 1102 is operated, and superposition detection is performed again in step S3. On the other hand, if it is determined in step S4 that the dark region 1104 and the superposition prohibition region 1103 are not superimposed by the superimposition detection in step S3, this processing is terminated.

  Next, with reference to FIG. 13, the superposition detection process in step S3 of FIG. 12 will be described. First, in step S11, the position, size, and shape of the overlapping prohibition area 1103 are acquired. Next, in step S12, the position, size, and shape of the dark region 1104 are acquired. In step S13, the position, size, and shape acquired in steps S11 and S12 are used to detect the overlap between the dark region 1104 and the overlap prohibition region 1103, and the process ends.

  Next, the window operation processing in step S5 in FIG. 12 will be described with reference to FIGS. FIG. 14 shows the distance Dist necessary for avoiding the overlap between the dark area 1104 and the overlap prohibition area 1103 and the movable distance Sur of the window 1102. In Dist and Sur, the y-axis minus direction is U, the x-axis plus direction is R, the y-axis plus direction is B, and the x-axis minus direction is L.

  FIG. 15 is a flowchart showing the window operation process. First, in step S21, the coordinates of the video display range 1101 are acquired. In step S22, the coordinates of the window 1102 are acquired. In step S23, the coordinates of the overlapping prohibition area 1103 are acquired. In step S24, the coordinates of the dark region 1104 are acquired.

  Next, in step S25, a distance Dist necessary for avoiding the overlap between the dark area 1104 and the overlap prohibition area 1103 and a movable distance Sur of the window 1102 are obtained.

  In step S31, it is determined whether there is a movable direction of the window 1102. If there is no movable direction, the process proceeds to step S32, and the window 1102 is reduced. Here, the window 1102 is reduced by multiplying the vertical and horizontal lengths by α.

  On the other hand, if there is a movable direction, the process proceeds to step S33, and a direction that is movable and has the smallest movement distance, that is, a direction with the smallest Dist is selected. In step S34, it is determined whether the window 1102 is movable by comparing Dist and Sur. If it is determined in step S34 that the window 1102 is not movable, the process proceeds to step S35, and the selected direction is not movable. If it is determined in step S34 that the window 1102 can be moved, the process proceeds to step S36, the window 1102 is moved by Dist in the selected direction, and this process is terminated.

  As described above, in the first embodiment, when the overlap prohibition area 84 is set on the window 81, the window 81 is moved and reduced so as to avoid the overlap between the dark area 52 and the overlap prohibition area 84. . Thereby, it is possible to avoid hiding the video 82 in the window 81 and the button 83 for performing the operation behind the dark region 52.

  In the present embodiment, a single window has been described, but the present invention is not limited to an operation on a single window. With respect to a plurality of windows, it is possible to operate the windows so as to avoid the overlapping of the dark area and the overlapping prohibition area by the same method.

(Second Embodiment)
FIG. 16 is a diagram illustrating a configuration example of the video conference system according to the present embodiment. In the video conference system of the present embodiment, the video processing device 8 ′ includes a projection device control unit 39. The projector control unit 39 is connected to the projector 1 and controls the projection position and the projection size of the projector 1. The overall control unit 38 is connected to the projection device control unit 39 via an internal bus or the like, and controls the projection control unit 39. The structure of each other part is the same as that of the video processing apparatus 8 of FIG.

  In the present embodiment, when the dark region 52 is set on the image 51 projected by the projector 1, the projector 1 is mechanically displaced, and then the dark region 52 is superimposed on the image 51.

  FIG. 17A shows a state in which the dark region 52 and the overlap prohibition region 84 are overlapped. At this time, the projection device control unit 39 of the video processing device 8 ′ controls the projector 1 to change the projection position of the video 51 and projects it within the projectable range 1105 (see FIG. 11). Thereby, as shown in FIG. 17B, the overlap between the dark region 52 and the overlap prohibition region 84 is avoided. Further, the projection device control unit 39 may avoid superimposition of the dark region 52 and the superimposition prohibition region 84 by changing the size of the image 51 to be projected.

  Further, when the projection position of the image 51 by the projector 1 is changed, it is conceivable that the image 51 projected on the screen 3 is distorted into a trapezoid. In the present embodiment, when the projected video 51 is distorted, the projector 1 performs correction.

  FIG. 18 is a flowchart showing a process for controlling the projector 1 so as to avoid the overlapping of the dark area 1104 and the overlapping prohibited area 1103. In FIG. 18, steps S1 to S4 are the same as steps S1 to S4 (see FIG. 12) of the first embodiment. If it is determined in step S4 that the dark region 1104 and the superimposition prohibited region 1103 are superimposed by the superimposition detection in step S3, the process proceeds to step S6. In step S6, the projector 1 is controlled by the projector control unit 39, and superposition detection is performed again in step S3.

  Next, the projector control process in step S6 in FIG. 18 will be described with reference to FIGS. FIG. 14 shows the distance Dist necessary for avoiding the overlap between the dark area 1104 and the overlap prohibition area 1103 and the movable distance Sur_a of the video display range 1101. In Dist and Sur_a, the negative y-axis direction is U, the positive x-axis direction is R, the positive y-axis direction is B, and the negative x-axis direction is L.

  FIG. 19 is a flowchart showing the projector control process. First, in step S111, the coordinates of the video display range 1101 are acquired. In step S112, the coordinates of the projectable range 1105 are acquired. In step S113, the coordinates of the overlap prohibition area 1103 are acquired. In step S114, the coordinates of the dark region 1104 are acquired.

  Next, in step S115, a distance Dist necessary for avoiding the overlap between the dark area 1104 and the overlap prohibition area 1103 and a movable distance Sur_a of the video display range 1101 are obtained.

  In step S121, it is determined whether there is a movable direction of the video display range 1101. If there is no movable direction, the process proceeds to step S122, and the video display range 1101 is reduced. Here, the video display range 1101 is reduced by multiplying the vertical and horizontal lengths by α.

  On the other hand, if there is a movable direction, the process proceeds to step S123, and a direction that is movable and has the smallest movement distance, that is, a direction with the smallest Dist is selected. In step S124, it is determined whether the video display range 1101 is movable by comparing Dist and Sur_a. If it is determined in step S124 that the video display range 1101 is not movable, the process proceeds to step S125, and the selected direction is not movable. If it is determined in step S124 that the video display range 1101 can be moved, the process proceeds to step S126, where the video display range 1101 is moved in the selected direction by Dist, and this process ends.

  As described above, in the second embodiment, the superimposition of the dark region 52 and the superimposition prohibition region 84 is avoided by controlling the projection position and the projection size of the projector 1. Thereby, it is possible to avoid hiding the video 82 in the window 81 and the button 83 for performing the operation behind the dark region 52.

(Third embodiment)
As the third embodiment, an example in which the first embodiment and the second embodiment are combined will be described. The configuration of the video processing apparatus is the same as that of the second embodiment. In this case, as described in the first embodiment, it is possible to avoid the overlap between the dark region and the overlap prohibition region by moving and reducing the window.

  First, as described in the first embodiment, an attempt is made to avoid superimposition of a dark region and a superimposition prohibition region by a window moving operation. Here, when the superimposition cannot be avoided, as described in the second embodiment, it is possible to avoid the superimposition of the dark region and the superimposition prohibited region by the operation of changing the projection position of the projected video by the projector 1. Try. Again, if superimposition cannot be avoided, both (that is, the window movement operation described in the first embodiment and the projector control described in the second embodiment) are executed simultaneously, and the dark region and the superposition prohibition region are performed. Attempt to avoid superimposition with.

(Fourth embodiment)
As a fourth embodiment, a method for changing the layout in the window in order to avoid the overlap between the dark region and the overlap prohibition region will be described. The configuration of the video processing apparatus is the same as that of the first embodiment.

  First, the basic principle will be described with reference to FIG. As shown in FIG. 20A, here, a superposition prohibition area 84 a and a superposition prohibition area 84 b are set on a single window 81. These superposition prohibition areas 84a and 84b are set in the video 82 and the button 83, which are display objects, respectively.

  At this time, as shown in FIG. 20B, by moving the display object 82 and the display object 83, the dark region 52 and the superposition prohibition regions 84a and 84b are prevented from overlapping. The movement of the display objects 82 and 83 is executed while maintaining the positional relationship set between the display objects 82 and 83. In addition, it is possible to avoid the dark region and the overlapping prohibited region from being overlapped on the plurality of windows by the same operation as described above.

  The flowchart related to the movement of the display object is the same as that in the flowchart of FIG. 15 of the first embodiment in which the window and the display object are replaced and the positional relationship between the display objects is added to the condition of the movable range.

  As described above, in the fourth embodiment, the display object in which the superposition prohibition area 84 is set is moved to avoid the superimposition with the dark area 52. Thereby, it is possible to avoid hiding the video 82 in the window 81 and the button 83 for performing the operation behind the dark region 52.

(Fifth embodiment)
As a fifth embodiment, an example in which the setting of the dark region 64 is automatically optimized by using a luminance histogram in a captured image of the camera 2 installed so as to face the projector 1 will be described. The configuration of the entire video conference system and the configuration of each device are the same as in the first embodiment. Also in the present embodiment, control and creation of an image to be projected is performed by software operating on the overall control unit 38.

  First, the basic principle will be described. In the present embodiment, a method for optimizing each of the attributes of the dark region using the position and size will be described. The attributes of the dark region are not limited to the position and size, and other attributes can be optimized by the same method.

  In the fifth embodiment, a two-stage process is executed: a first stage for determining the position of the dark area 64 on the image 62 projected by the projector 1 and a second stage for determining the size of the dark area 64. Thereby, the dark region 64 is set in a portion (camera portion) corresponding to the camera 2 on the video 51 projected by the projector 1.

  In the first stage, a dark area of a predetermined size is arranged on the white image created by the overall control unit 38 and projection is performed from the projector 1. That is, a dark area is temporarily set on the image projected by the projector 1. Next, an image in which the temporarily set dark region is arranged is captured and acquired by the camera 2, and a luminance histogram is created.

Then, the dark regions are rearranged, a luminance histogram is created, and the process is repeated until the luminance histogram is created for the entire image. Here, the rearrangement of the dark regions stores the sum Z _n of the frequencies from the th class to the highest luminance class as shown in FIG. Regarding the setting of the value of th, a valley of a histogram is found and set using a discriminant analysis method or the like. The discriminant analysis method is a method for determining a threshold for separating distributions. As for the value of th, the value determined first is used each time.

The sum of frequencies Z _n is obtained by the following equation (1). In Expression (1), I _max indicates the maximum of the histogram class relating to luminance, and I _th indicates the _th th class. H _i is the frequency of the histogram of luminance i.

FIG. 22A is a graph of the frequency sum Z _n . In FIG. 22A, the horizontal axis indicates the histogram number and corresponds to a change in the position of the dark region. The vertical axis indicates the value Z of the frequency sum.

  In FIG. 22A, when the dark area is arranged in the camera portion, the sum value Z of the frequencies becomes smaller than in the case where the dark area is not arranged. For this reason, if the position of the dark region when the value Z is minimized is determined as the dark region placement location, the dark region can be placed at a position corresponding to the camera portion.

In the second stage, the size of the dark region is adjusted. On the white image created by the overall control unit 38, a dark region is arranged at the position determined in the first step and projection is performed from the projector 1. Next, the entire control unit 38 acquires a captured image of the camera 2 and creates a luminance histogram. Then, the dark region is reduced, and the creation of the luminance histogram is repeated until either the length or the length of the dark region becomes zero. Further, every time a luminance histogram is created, the sum Z _n of the frequencies from the th class to the highest luminance class is obtained and stored using equation (1). Here, the setting of the value of th is the same as in the first stage. Then, using equation (2), a change amount br _m of the component sum Z _m of the partial histogram per one reduction of the dark region is obtained. In the formula (2), Z _m represents the sum of the frequencies according to the m-th formula (1).

FIG. 22B is a graph of the change amount br _m of the component sum Z _m . In FIG. 22 (b), the horizontal axis represents the number of iterations of the loop, and the vertical axis represents the value br variation component sum Z _m.

  When the dark area is smaller than the size of the camera 2, the projection light is reflected on the camera 2, and the high-luminance points in the captured image increase rapidly, so the sum value Z of the frequencies increases. For this reason, when the amount of change br of Z in FIG. 22B is the maximum, the size corresponding to the camera portion can be obtained.

  The above procedure is shown in the operation processing flowchart of the overall control unit 38 shown in FIG. After starting the setting of the dark area, the overall control unit 38 optimizes the position of the dark area in step S201 (first stage), and optimizes the size of the dark area in step S202 (second stage). In step S203, the overall control unit 38 stores the position and size of the dark area in the RAM, and the process ends.

  Next, the procedure for optimizing the position of the dark area defined in step S201 will be described with reference to FIG. Here, for the sake of explanation, a black rectangle having the same aspect ratio as that of the projected image and an area of 1 / s times is used.

In step S301, the overall control unit 38 causes the projector 1 to project an image in which a dark region is arranged on a white image. In step S <b> 302, the overall control unit 38 acquires a captured video of the camera 2. In step S303, after creating the luminance histogram, the overall control unit 38 calculates and stores the frequency sum Z _n as the feature amount in step S304. Here, the initial arrangement of the dark region is the upper left corner on the white image. The overall control unit 38 moves the dark region to the right when rearranging the dark region, and returns to the left end and moves the dark region downward when reaching the right end. In step S <b> 305, the overall control unit 38 determines to rearrange the dark areas until the dark areas are arranged with respect to the entire image. As a result, histograms of a plurality of videos are generated.

After that, in step S306, the overall control unit 38 compares the frequency sum Z _n stored in step S304, determines the dark area placement location where the sum is minimum, and stores the position in the memory in step S307. To do.

  Next, the procedure for optimizing the size of the dark region defined in step S202 will be described with reference to FIG. In addition, although the process of step S201 and the process of step S202 are demonstrated using the same flowchart, an actual process differs.

In step S301, the overall control unit 38 causes the projector 1 to project an image in which a dark region is arranged on a white image. In step S <b> 302, the overall control unit 38 acquires a captured video of the camera 2. In step S303, after creating the luminance histogram, the overall control unit 38 calculates the frequency sum Z _n and stores it in the memory in step S304. Here, the initial position and size of the dark area determined by step S201 are used, and the dark area reduction ratio is reduced pixel by pixel from the top, bottom, left, and right of the dark area. In step S305, the overall control unit 38 determines that the dark area is rearranged until either the vertical or horizontal size of the dark area becomes zero. As a result, histograms of a plurality of videos are generated.

  Thereafter, in step S306, the overall control unit 38 calculates the change amount br of the frequency using the frequency stored in step S304, obtains the size of the dark region where the change amount br is maximum, and in step S307. Store size in memory.

  In this embodiment, the luminance histogram is used, but the histogram to be used is not limited to the luminance histogram. For example, in the case of the RGB color model, it is possible to set the dark region by using each of the RGB components, combinations, or all histogram components.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus. In this case, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, the functions of the above-described embodiments are not limited to being realized by executing the program code read by the computer. For example, an OS (basic system or operating system) running on the computer performs part or all of the actual processing based on an instruction of the program code, and the functions of the above-described embodiments are realized by the processing. May be.

  Further, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. In this case, after being written in the memory, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the function of the above-described embodiment is performed by the processing. Is realized.

It is a figure which shows arrangement | positioning of the projector and camera which comprise a video conference system. It is the figure which represented typically the picked-up image when the projection light of a projector was reflected on the camera. It is a figure which shows the structural example of the video conference system of 1st Embodiment. It is a figure for demonstrating the setting position of a dark area. It is a figure which shows the example of a dark area setting screen. It is a figure for demonstrating the setting method of a dark area. It is a flowchart which shows the setting procedure of the dark area in 1st Embodiment. It is a figure which shows the state where the display object is hidden in the dark area. It is a figure which shows the state by which a window is arrange | positioned so that superimposition with a dark area | region and a superimposition prohibition area | region may be avoided. It is a figure for demonstrating the setting method of a superimposition prohibition area | region. It is a figure which shows the coordinate of a video display range, a window, a superimposition prohibition area | region, and a dark area. It is a flowchart which shows the process which arrange | positions a window so that the superimposition with a dark area | region and a superimposition prohibition area | region may be avoided in 1st Embodiment. It is a flowchart which shows a superimposition detection process. It is a figure which shows the coordinate of a video display range, a window, a superimposition prohibition area | region, and a dark area. It is a flowchart which shows a window operation process. It is a figure which shows the structural example of the video conference system of 2nd Embodiment. It is a figure which shows the state from which the projection position of an image | video is changed so that superimposition with a dark area | region and a superimposition prohibition area | region may be avoided. 10 is a flowchart illustrating a process of controlling the projector so as to avoid the superimposition of a dark region and a superposition prohibition region in the second embodiment. It is a flowchart which shows a projector control process. It is a figure which shows the state from which a layout is changed so that superimposition with a dark area | region and a superimposition prohibition area | region may be avoided. It is a characteristic view which shows the example of a brightness | luminance histogram. It is a characteristic view which shows the graph of the calculated value at the time of dark area setting using a brightness | luminance histogram. It is a flowchart which shows the setting procedure of the dark area in 5th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Projector 2 Camera 3 Screen 7 Network 8 Video processing apparatus 31 Audio | voice output part 32 Audio | voice input part 33 Display control part 34 Operation part 35 Image | video output part 36 Image | video input part 37 Communication part 38 Overall control part 39 Projection apparatus control part 41 Speaker 42 Microphone 43 Display 44 Mouse

Claims (11)

A video processing apparatus for connecting the image pickup device, in the installed Ru projection device so as to face the image pickup device,
A dark region setting means for setting a region on the projected image projected by the projection device as a dark region for reducing the reflection of projection light on the captured image of the imaging device ;
A superimposition prohibition region setting means for setting a superposition prohibition region on a display part of a projection image projected by the projection device;
Superimposition avoidance means for moving the display portion where the superposition prohibition area setting means sets the superposition prohibition area or changing the size so that the superimposition of the dark area and the superposition prohibition area is avoided. An image processing apparatus comprising the image processing apparatus. A video processing device connected to a projection device and an imaging device installed so as to face the projection device between the projection device and a screen;
A dark region setting means for setting a dark region in a region overlapping with the imaging device when the screen is viewed from the projection device among regions on a projected image projected onto the screen by the projection device;
A superimposition prohibition region setting means for setting a superimposition prohibition region on a projection image projected by the projection device;
An image comprising: a superposition avoiding means for avoiding superposition of the dark region and the superposition prohibition region by changing a projection position of the projection video or changing a size of the projection video. Processing equipment. An image processing device connected to an imaging device and a projection device installed to face the imaging device,
A dark region setting means for setting a region on the projected image projected by the projection device as a dark region for reducing the reflection of projection light on the captured image of the imaging device;
A superimposition prohibition region setting means for setting a display portion of a projection image projected by the projection device as a superposition prohibition region for avoiding superimposition with the dark region;
When the first superposition prohibition area and the second superposition prohibition area are set by the superposition prohibition area setting means, the first display portion set in the first superposition prohibition area and the second superposition prohibition area The first and second display parts are moved so as to avoid the overlap between the first and second overlap prohibited areas and the dark area by changing the layout with the set second display part. Or a superimposition avoiding means for changing the size of the image processing apparatus. The dark region setting means, the position of the dark regions, the shape, size, color, pattern, according to claim 1 to 3, characterized in that to be able to set at least one of concentration arbitrary The video processing device according to any one of claims. The superimposed forbidden area setting means, the position of the superimposed prohibited area, shape, any one of claims 1 to 3, characterized in that as at least any one can be set to arbitrary of the magnitude The video processing apparatus described in 1. A projection device, wherein an imaging device that will be installed so as to face the projection device, a video processing system comprising a video processing device connected to the projector,
A dark region setting means for setting a region on a projection image projected by the projection device as a dark region for reducing the reflection of projection light from the projection device on the imaging device ;
A superimposition prohibition region setting means for setting a superposition prohibition region on a display part of a projection image projected by the projection device;
Moves the display portion where the said superimposed forbidden area setting means so superposition between the dark region and the overlapped forbidden area can be avoided to set the superposition prohibition region, or a superposition avoiding means for changing the size A video processing system characterized by that. A video processing system comprising: a projection device; an imaging device installed between the projection device and a screen so as to face the projection device; and a video processing device connected to the projection device,
A dark region setting means for setting a dark region in a region overlapping with the imaging device when the screen is viewed from the projection device among regions on a projected image projected onto the screen by the projection device;
A superimposition prohibition region setting means for setting a superimposition prohibition region on a projection image projected by the projection device;
An image comprising: a superposition avoiding means for avoiding superposition of the dark region and the superposition prohibition region by changing a projection position of the projection video or changing a size of the projection video. Processing system. A video processing system comprising a projection device, an imaging device installed to face the projection device, and a video processing device connected to the projection device,
A dark region setting means for setting a region on the projected image projected by the projection device as a dark region for reducing the reflection of projection light on the captured image of the imaging device;
A superimposition prohibition region setting means for setting a display portion of a projection image projected by the projection device as a superposition prohibition region for avoiding superimposition with the dark region;
When the first superposition prohibition area and the second superposition prohibition area are set by the superposition prohibition area setting means, the first display portion set in the first superposition prohibition area and the second superposition prohibition area By changing the layout with the set second display portion, the first and second superimposing areas can be avoided from overlapping with the first and second superimposing prohibition areas. An image processing system comprising: a superimposition avoiding unit that moves a display part or changes a size thereof. An imaging device, a computer connected to the installed Ru projection device so as to face the image pickup device,
A dark region setting procedure for setting a region on a projection image projected by the projection device as a dark region for reducing the reflection of projection light on a captured image of the imaging device ;
A superposition prohibition region setting procedure for setting a superposition prohibition region on a display part of a projection image projected by the projection device;
A superposition avoidance procedure that moves or changes the size of the display portion in which the superposition prohibition region is set in the superposition prohibition region setting procedure so that superimposition of the dark region and the superposition prohibition region is avoided. program to be run. A computer connected to an imaging apparatus installed between the projection apparatus and the projection apparatus and a screen so as to face the projection apparatus;
A dark region setting procedure for setting a dark region in a region overlapping with the imaging device when the screen is viewed from the projection device among regions on a projected image projected onto the screen by the projection device;
A superposition prohibition region setting procedure for setting a superposition prohibition region on a projection image projected by the projection device;
A program for executing a superposition avoidance procedure for avoiding superposition of the dark region and the superposition prohibition region by changing the projection position of the projection video or changing the size of the projection video. An imaging device and a computer connected to a projection device installed to face the imaging device;
A dark region setting procedure for setting a region on a projection image projected by the projection device as a dark region for reducing the reflection of projection light on a captured image of the imaging device;
A superposition prohibition region setting procedure for setting a display portion of a projection image projected by the projection device as a superposition prohibition region for avoiding superimposition with the dark region;
When the first superposition prohibition area and the second superposition prohibition area are set in the superposition prohibition area setting procedure, the first display portion set in the first superposition prohibition area and the second superposition prohibition area By changing the layout with the set second display portion, the first and second superimposing areas can be avoided from overlapping with the first and second superimposing prohibition areas. A program for executing a superposition avoidance procedure for moving the display part or changing the size.

Images