JP2009055094A - Video system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To overcome the problem that when there is only one imaging apparatus in the conventional video conference system, the video image displayed on a display is restricted to one video taken from one place with one camera angle and a strange video image is presented depending on the standing position of a viewer, and the problem that although the position of the imaging apparatus can be changed actively depending on the standing position of a viewer but it cannot cope with a plurality of viewers, and to provide a system which can display videos taken at a plurality of camera angles, simultaneously. <P>SOLUTION: The video system includes a plurality of imaging apparatus for taking the image of an imaging space at different camera angles, and a multiview display for displaying the videos obtained from respective imaging apparatus simultaneously in different visual directions depending on the camera angle of the imaging apparatus. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video system characterized in that videos obtained by imaging at different camera angles with a plurality of imaging devices are simultaneously displayed in different viewing directions.

  In recent years, with the increase in the size of televisions, display close to the actual size has become possible. In addition, so-called video conferencing systems, etc., for communicating with a person at a remote place as if they are on the spot by using a combination of an imaging device such as a video camera and a display device are becoming common. is there.

  There are various forms of video conferencing and there are cases where a single screen is used or a multi-screen is used. In addition to business scenes, the use of video conferencing for communication with distant relatives and family members is also increasing in ordinary homes.

In addition, as a method of always displaying an image with the actual size of the subject without limiting the position of the subject, the video conference device uses the camera unit to determine the distance (imaging distance) between the user who is the subject and the camera unit. By measuring, it is possible to always grasp the imaging distance, and even if the user moves and the distance to the camera unit changes, the actual size of the user can be recognized based on the measured imaging distance, and There is an invention in which a user's image is always projected on a screen of a video conference apparatus with a life size regardless of the position. (Patent Document 1)
JP-A-2005-303683

  However, in such a video conference system or the like, when there is one image pickup device, the video displayed on the display device is limited to the camera angle taken from one place. It will be a video with a sense of incongruity and you will not be able to feel as if you are there.

  Also, if there are multiple shooting devices, they will be displayed on the corresponding multiple display devices, and even if individual display is optimized, it will not be unified as a whole, and it will also improve the sense of reality. Can not.

  Also, even if the position and direction of the imaging device are actively changed depending on the position of the viewer, it is only optimized for one viewer, and multiple viewers move to different positions However, it is impossible to display an image that is optimal for both viewers, and it is not possible to obtain a video with little discomfort.

  Therefore, even when a plurality of viewers view from different positions, a system with a sense of presence as if they are present is required by displaying images from a plurality of camera angles.

  In order to solve the above problems, the present invention proposes the following first to seventh inventions.

  According to a first aspect of the present invention, there are provided a plurality of imaging devices for imaging an imaging space at different camera angles, and a multi-display that displays images obtained from the respective imaging devices simultaneously in different viewing directions according to the camera angles of the imaging devices. A video system including a view display device.

  A second invention relates to the first invention, and the relative arrangement of the imaging device with respect to the imaging space is relatively equivalent to the scheduled viewer arrangement according to different viewing directions scheduled by the multi-view display device. It is a video system that is in the order of.

  A third invention is an image system according to the first or second invention, wherein an area of an imaging region captured by the imaging apparatus is substantially the same as a display area of the multi-view display apparatus.

  A fourth invention according to any one of the first to third inventions, wherein the multi-view display device is a video system capable of dual view display.

  A fifth invention according to any one of the first to third inventions, wherein the multi-view display device is a video system capable of triple view display.

  FIG. 1 is a conceptual diagram of a video system according to the present invention. FIG. 1 is a view of the space 0105 and the space 0106 as viewed from above. A space 0105 is an imaging space, and is a space in which the subject (0103a, 0103b) is imaged by a plurality of imaging devices (0101a, 0101b, 0101c). On the other hand, the space 0106 is a display-side space, and the video captured in the imaging space 0105 is displayed on the multi-view display device 0102, and a plurality of viewers (0104a, 0104b, 0104c) watch the video. The purpose of this video system is that a plurality of viewers (0104a, 0104b, 0104c) in the space 0106 can see the subject (0103a, 0103b) in another space 0105 as if they were in the same space. is there. For this purpose, first, the imaging device 0101a captures an image at a camera angle facing the subject (0103a, 0103b), and displays the image on the multi-view display device 0102 so that the viewer 0104a can visually recognize the image. Similarly, the video captured by the imaging device 0101b is displayed on the multi-view display device 0102 so that the viewer 0104b can visually recognize the video and the video captured by the imaging device 0101c is visible to the viewer 0104c. Although a technique for simultaneously displaying a plurality of different images on one multi-view display device will be described later, by simultaneously displaying images captured at different camera angles on the multi-view display device 0102 in this way, It is possible to show an image that allows the person to feel the person more realistically according to the position where the user is standing.

  In the present invention, in a video conference, even though a plurality of viewers feel as if they are watching one video of one display device, each viewer views the video with the most suitable viewing angle at the same time. Therefore, it is possible to communicate with a person in a remote place as if they were present.

The best mode for carrying out the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to these embodiments, and can be implemented in various modes without departing from the spirit of the present invention. The first embodiment will mainly describe claims 1, 2, 4, 5, 6, 7 and the like. The second embodiment will mainly describe claim 3 and the like.
(Embodiment 1)

  (Embodiment 1: Overview) In this embodiment, a plurality of imaging devices for imaging an imaging space at different camera angles and images obtained from the respective imaging devices are simultaneously different depending on the camera angle of the imaging device. A video system including a multi-view display device that displays in the viewing direction will be described.

  (Embodiment 1: Configuration) FIG. 2 illustrates a configuration diagram of a video system according to this embodiment. The video system (0200) includes a plurality of imaging devices (0201a, 0201b, 0201c,...) And a multi-view display device (0202).

  Note that the constituent elements of the present invention described in detail below are configured by hardware, software, or both hardware and software. For example, as an example for realizing these, there are hardware constituted by a CPU, a bus, a memory, an interface, a peripheral device, and the like, and software executable on the hardware. As software, the functions of each unit are realized by sequentially executing a program developed on the memory, and processing, storing, and outputting data on the memory and data input via the interface. (The same applies throughout the specification.)

  An “imaging device” (such as 0201a) captures an imaging space at different camera angles. In addition, a plurality of imaging devices exist in one video system (0200). The “imaging space” is a space where an image is picked up by the imaging device, and a subject exists in this space. FIG. 3 shows a specific example of “different camera angles”. It is assumed that two persons (subjects) (0103a, 0103b) are imaged by three imaging devices (0101a, 0101b, 0101c) in the imaging space 0105 as illustrated in FIG. Also, subjects 0303a and 0303b shown in FIG. 3 are subjects 0103a and 0103b in FIG. 1, respectively. Here, FIG. 3A shows a camera angle in the imaging apparatus 0101a. When the optical axis of the imaging device 0101a is directed toward the subject (0303a, 0303b), the subject 0303a is directed to the right rather than the vertical direction with respect to the wall on which the imaging device 0101a is installed. The image is taken to be closer. Similarly, FIG. 3C shows the camera angle of the imaging device 0101c. When the optical axis of the imaging device 0101c is directed toward the subject (0303a, 0303b), the subject 0303b is closer to the subject 0303a. The image is captured as if it were. 3B shows the camera angle of the image pickup apparatus 0101b. As shown in FIG. 1, since the subjects (0103a, 0103b) are almost in front of the image pickup apparatus 0101b, FIG. Thus, the subject 0303a and the subject 0303b are imaged with almost the same sense of distance.

  Further, as described above, the optical axis of each imaging device may be fixed in a direction facing the subject. Fixing in a direction facing the subject may be made to match that direction after the subject's position is determined, or fixed in advance in the direction of the position where the subject is expected to stand. It may be left. Also, the optical axis direction of the imaging devices 0101a, 0101b, and 0101c shown in FIG. 1 and the line-of-sight direction when the viewers 0104a, 0104b, and 0104c face the subject when it is assumed that the subjects 0103a and 0103b exist in the same space. It is expected that the sense of reality will increase further when When such an optical axis direction is set, for example, the position of the subject is detected using a position detection sensor or the like, and the optical axis direction of the imaging apparatus is changed to the detected position direction of the subject. May be. In addition, the setting method of the optical axis of each imaging device is not limited to the setting method as described above. For example, the optical axes of all the imaging devices may be always fixed in the vertical direction with respect to the installed wall, and when displaying images picked up with such fixed optical axes on the display device The image may be displayed after image editing is performed and the same image as when the image is taken with the optical axis of the imaging device directed toward the subject. The advantage of performing image editing in this way is that there is no need for a mechanical mechanism for changing the optical axis direction of the imaging device each time the position of the subject is changed in the imaging space, and the optical axis of the imaging device. It is not necessary to consider the time lag when changing. In addition, when performing such image editing, for example, the position of the subject is detected using a position detection sensor or the like, and image editing is performed according to the detected position of the subject and the angle in the optical axis direction of the imaging device. Good.

  The “multi-view display device” (0202) displays images obtained from the respective imaging devices in different viewing directions at the same time depending on the camera angle of the imaging device. “Each imaging device” refers to a plurality of imaging devices (such as 0201a) that capture an imaging space. “Different viewing directions” mean directions in which a viewer can visually recognize video displayed on the multi-view display device. For example, as illustrated in FIG. 4, the viewing direction when the viewer 0104 a visually recognizes the display of the multi-view display device 0102 in the space 0106 is a direction indicated by an arrow 0402. For example, in the example shown in FIG. 1, the viewing direction is assumed to be three viewing directions viewed from three viewers 0104a, 0104b, and 0104c. In such a case, one multi-view display device is used. Three different images are displayed at 0102 at the same time. Note that the number of viewing directions is three in FIG. 1, but is not limited thereto, and may be a plurality (two or more). That is, the multi-view display device may be capable of dual view display or triple view display.

  “Display in different viewing directions at the same time depending on the camera angle of the imaging device” means that a plurality of images with different camera angles obtained by imaging with a plurality of imaging devices can be viewed in different viewing directions. Is to display at the same time. For example, as shown in FIG. 4, an image captured with the optical axis fixed in the direction 0401 from the imaging device 0101a to the subject (0103a, 0103b) in the imaging space 0105 is captured by, for example, a viewer on the multi-view display device (0102). It is displayed in the viewing direction 0402 from 0104a. Note that the method of “displaying in different viewing directions simultaneously according to the camera angle of the imaging device” is not limited to such a case, and for example, an image captured from the imaging device 0101a is displayed in the viewing direction from the viewer 0104c. And so on. Further, if the optical axis direction 0401 and the visual recognition direction 0402 are substantially the same direction, the sense of reality is further enhanced. In addition, although the number of imaging devices (the number of images with different camera angles) and the number of viewing directions are the same in the above example, they may be different. For example, different images captured by five imaging devices may be displayed simultaneously in three viewing directions.

  Further, a technique for simultaneously displaying a plurality of different videos on one multi-view display device will be described with reference to FIG. In FIG. 5, a case where different images are displayed in two viewing directions will be described. The display unit 0502 of the multi-view display device includes display portions 0502d and 0502e for displaying two images with different camera angles. That is, the video imaged from one imaging device is sliced in the vertical direction with a predetermined width, and the two sliced videos are alternately arranged to form one display video image. Note that the display unit 0502 may be physically divided into portions 0502d and 0502e, or is physically a single display panel, and an image to be displayed on the display panel is obtained by combining two images. The image may be composed of composite data. In addition, a barrier 0501 having a slit is provided between the display portion 0502 and the viewer (0504d, 0504e). Accordingly, the viewer 0504d can view only the video displayed on the display portion 0502d, and the viewer 0504e can view only the video displayed on the display portion 0502e.

  In addition, the relative arrangement of the imaging devices with respect to the imaging space and the scheduled viewer arrangement according to different viewing directions scheduled by the multi-view display device may be in a relatively equivalent arrangement order. In the example of FIG. 1, “the scheduled viewer arrangement according to different viewing directions scheduled by the multi-view display device” has three viewing directions, and the “scheduled viewer arrangement according to the viewing direction”. "Corresponds to the positions indicated by the viewers 0104a, 0104b, and 0104c. In addition, “relatively equivalent arrangement order” means that the video imaged by the imaging device 0101a is displayed in the viewing direction from the scheduled viewer arrangement 0104a, and the video imaged by the imaging device 0101b is from the scheduled viewer layout 0104b. The order of arrangement of the corresponding imaging device and the scheduled viewer arrangement is relatively equivalent, such as displaying the video displayed in the viewing direction and displaying the image captured by the imaging device 0101c in the viewing direction from the scheduled viewer arrangement 0104c. Means that.

  Further, the imaging device (0201 etc.) and the multi-view display device (0202) may be physically integrated or may be separate. In the case of separate bodies, for example, it is assumed that both sides transmit and receive video data by communication. The communication method is not limited to a specific one. For example, wireless communication or wired communication may be used. In the case of wireless communication, infrared communication, Bluetooth (registered trademark), or the like is assumed. In the case of wired communication, Internet communication, LAN (Local Area Network) communication, peer-to-peer communication, and the like are assumed.

  (Embodiment 1: Processing on Hardware) FIGS. 6 and 7 show specific examples of processing on hardware of a video system according to this embodiment. FIG. 6 shows a specific example of processing on the hardware of the imaging apparatus. The imaging device images the imaging space by the imaging unit (0601), and the video data obtained by imaging is stored in the temporary storage memory (0602) and then stored in the storage device (0603) such as a hard disk or a semiconductor memory. Is done. The video data is output to the multi-view display device through an output interface such as a communication interface. More specifically, the imaging unit (0601), for example, receives light from an object through a lens with an imaging element such as a CCD (Charge Coupled Device), and receives CDS (Correlated Double Sampling) or AGC (AGC). After the processing in Auto Gain Control), the video data can be obtained by converting it into an electrical signal by an A / D converter or the like.

  FIG. 7 shows a specific example of processing on the hardware of the multi-view display device. As described with reference to FIG. 6, the video data output via the output interfaces of the plurality of imaging devices is received by an input interface such as a communication interface of the multi-view display device. Here, it is assumed that video data A (0701a), video data B (0701b), and video data C (0701c) are received from three imaging devices. These video data are data constituting videos with different camera angles as described above. When each video data is received via the input interface, it is stored in a temporary storage memory. Also, one composite data (0702) to be displayed on one multi-view display device is generated based on the video data stored in the temporary storage memory. The generated composite data is sent to the Video RAM (VRAM) (0703) and then displayed on the display (0704).

  Note that the processing on the hardware described with reference to FIGS. 6 and 7 is merely an overview of the processing.

  (Embodiment 1: Process Flow) FIG. 8 illustrates a flowchart showing a process flow in a video system according to this embodiment. The flowchart of (a) shows the process in an imaging device, and the flowchart of (b) shows the process in a multi view display apparatus.

  First, (a) the imaging device captures an imaging space (S0801). Next, the video data obtained by imaging is output to the multi-view display device (S0802). The series of processes described above are executed in each of the plurality of imaging devices.

  Next, (b) the multi-view display device receives video data from each imaging device (S0803). That is, each image data output in step S0802 is received by each imaging device. Next, the received video data are synthesized (S0804). That is, the display data for combining the video data and displaying them on the display is generated. Next, the generated composite data (display data) is displayed on the display (S0805).

  Note that the flowchart of FIG. 8 can be regarded as a process flowchart of a program executed by a computer. Further, such a program can be recorded on a medium such as a flexible disk. (The same applies throughout the specification.)

(Embodiment 1: Effect) According to the video system according to the present embodiment, since a plurality of viewers can each view a video with the most appropriate viewing angle, it is possible to communicate with a person in a remote place with a sense of presence. It becomes possible to take.
(Embodiment 2)

  (Embodiment 2: Overview) In the present embodiment, an image system is described in which the area of the imaging region captured by the imaging apparatus is substantially the same as the display area of the multi-view display apparatus.

  (Embodiment 2: Configuration) The configuration of the video system according to this embodiment is the same as the configuration of the video system described with reference to FIG. However, the area of the imaging region imaged by the imaging apparatus is substantially the same as the display area of the multi-view display apparatus. The “area of the imaging region” means the actual area of the imaging space at the focused depth. For example, as shown in FIG. 9, when imaging is performed with the focus at the time of imaging aligned with the subject 0103c, the imaging space captured by the imaging device 0101c is between point A and point A ′, and the imaging device 0101b. The imaging space captured by is between point B and point B ′. In other words, the area of the imaging region corresponds to the actual area when the imaging space is cut from point A to point A ′ or from point B to point B ′. That is, the distance from the point A to the point A ′ or the distance from the point B to the point B ′ is substantially the same as the length L of the display (0102s) of the multi-view display device, and the height is also substantially the same. . As shown in FIG. 10, if the imaging device 0101c is focused on the subject 0103d, the distance from the point C to the point C ′ is almost the same as the length L of the display (0102s) of the multi-view display device. It becomes. In this way, when the “area of the imaging region imaged by the imaging device” and the area of the display (0102s) of the multi-view display device are substantially the same, it is possible to display the subject at almost full size. Because there is, you can expect the effect of more realistic. Further, “substantially the same” means substantially the same. Strictly speaking, even if these areas are slightly different, it does not appear to be a problem particularly when the multi-view display device is large. Further, the area from the point A to the point A 'and the area from the point B to the point B' are slightly different, but such a difference does not matter.

  (Embodiment 2: Processing on Hardware) Processing on hardware of the video system according to this embodiment is the same as the processing described above.

  (Embodiment 2: Process Flow) The process flow in the video system according to this embodiment is the same as the process flow described above.

  (Embodiment 2: Effect) According to the video system according to the present embodiment, a more realistic video can be shown to the viewer.

Conceptual diagram of a video system according to the present invention Functional block diagram of the video system according to the first embodiment Example of camera angle Diagram explaining camera angle and viewing direction Example of video display method in multi-view display device FIG. 3 is a diagram illustrating an example of processing operations on hardware of the video apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of processing operations on hardware of the multi-view display device according to the first embodiment. FIG. 3 is a flowchart showing a processing flow of the video system according to the first embodiment. The figure explaining the area of an imaging region The figure explaining the area of an imaging region

Explanation of symbols

0101a Imaging device 0101b Imaging device 0101c Imaging device 0102 Multi-view display device 0103a Subject 0103b Subject 0104a Viewer 0104b Viewer 0104c Viewer 0105 Imaging space 0106 Display-side space

Claims (7)

A plurality of imaging devices for imaging the imaging space at different camera angles;
A multi-view display device that displays images obtained from the respective imaging devices in different viewing directions simultaneously according to the camera angle of the imaging device;
A video system consisting of   2. The video system according to claim 1, wherein the relative arrangement of the imaging device with respect to the imaging space and the scheduled viewer arrangement according to different viewing directions scheduled by the multi-view display device are in a relatively equivalent arrangement order. .   The video system according to claim 1, wherein an area of an imaging region captured by the imaging apparatus is substantially the same as a display area of the multi-view display apparatus.   The video system according to claim 1, wherein the multi-view display device is capable of dual view display.   The video system according to any one of claims 1 to 3, wherein the multi-view display device is capable of triple view display. An imaging step of imaging the imaging space with imaging devices of different camera angles;
A display step for displaying images obtained from the respective imaging devices in a multi-view manner in different viewing directions at the same time according to the camera angle of the imaging device;
A display method for a video system.   In the display step, the relative arrangement of the imaging device with respect to the imaging space and the scheduled viewer arrangement corresponding to different viewing directions scheduled for the multi-view display are displayed in a relatively equivalent arrangement order. 7. The video system display method according to claim 6, further comprising an arrangement order dependent display sub-step.

Images