JP2009055318A - Stereo image photographing method, stereo image photographing system, and stereo image photographing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform wide-angle photographing using one mirror without being affected by vignetting. <P>SOLUTION: A camera unit 100 includes a camera 10 and a mirror 11, and the camera 10 is installed at the ceiling and disposed at a right angle to the mirror 11. The camera 10 then photographs an image in a state that a half of the image is inverted with respect to the other half of the image. The address control section 222 of a partial mirror image inverting section 220 controls addresses of a memory 221 so as to read camera images in the reverse order of addresses, only for left-half images, when reading the camera images from the memory 221. Thus, the image inverting section 220 inverts any half of the image photographed by the camera 10 and outputs a stereo image. A stereo image processing section 223 then detects an object (e.g., a human being) included in the stereo image from the partial mirror image inverting section 220 and outputs a detection result. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a stereo image capturing method, a stereo image capturing system, and a stereo image capturing program for capturing a stereo image using one camera.

  In general, it is performed to detect a person from an image, track the person in the image, measure the position and the number of persons, and perform image processing. In the case of detecting a human from an image, conventionally, a method of calculating a background difference or an inter-frame difference is used. However, when these methods are used, they are easily affected by shadows, external light, and the like. Therefore, there is a need for a detection method that can detect a person in an image without being affected by these disturbances. Therefore, at present, a method of detecting a person by stereo shooting the floor direction from a camera installed on a ceiling to obtain a height image is used.

  In general, in order to photograph a human from a camera installed on the ceiling, it is necessary to perform wide-angle photographing because the photographing distance is short. For example, even if the height of the ceiling is 2500 mm, when the human height is 1800 mm, the shooting distance from the camera to the position of the human head is only 700 mm. In addition, as a stereo imaging technique for detecting a person in an image, a method of capturing an image from the ceiling to the floor using two wide-angle cameras is usually used.

  Further, for example, Patent Document 1 discloses a stereo image processing apparatus capable of performing accurate distance measurement by image processing using a stereo method even in a blooming region as another application technique of a stereo camera including a plurality of cameras. Are listed.

JP 2005-91173 A

  When taking a stereo image, since two cameras must be taken simultaneously, an external synchronization camera is required separately from the main camera. However, since an inexpensive camera that is easily available and does not have external synchronization, an expensive camera must be used to shoot a stereo image.

  When a stereo image is taken, the image processing apparatus also requires two components from the input unit for inputting an image to the memory for storing the image. Therefore, it becomes expensive compared with a monocular system using a conventional background difference or the like.

  In addition, there is a technique that uses a plurality of mirrors to form a left and right stereo image into one image using a single camera, but it is not for wide-angle shooting. Although it is conceivable to apply this technique to wide-angle imaging, it cannot be used because vignetting occurs in reality.

  For example, as shown in FIG. 9, by using a plurality of mirrors (four mirrors in this example), only one real camera 910 is used, and the two cameras 911 and 912 are used for stereo images. Can be taken. In this case, a part of the image photographed by the real camera 910 is reflected by the mirror 1 and further reflected by the mirror 2 and photographed within the screen of the left half of the camera 910. Further, another part of the image photographed by the real camera 910 is reflected by the mirror 3, further reflected by the mirror 4, and photographed within the right half screen of the camera 910. By doing so, a stereo image can be taken with the left half image and the right half image in the image. However, in this case, as the lens of the camera becomes a wide angle, vignetting occurs because the size of the mirror must be increased. For this reason, wide-angle shooting cannot be performed.

  Accordingly, an object of the present invention is to provide a stereo image capturing method, a stereo image capturing system, and a stereo image capturing program capable of performing wide-angle capturing using a single mirror without being affected by vignetting.

  The stereo image photographing method according to the present invention photographs a stereo image by performing a wide-angle photographing by arranging a mirror at a right angle to the plane on which the photographing means is disposed (for example, by placing a mirror at the wall or at the entrance / exit). A stereo image capturing method, wherein the image capturing means captures an image in which half of the image is inverted with respect to the other half of the image, and in the image captured by the image capturing means An image inversion step of inverting any one of the images and outputting a stereo image.

  A stereo image capturing system according to the present invention is a stereo image capturing system for capturing a stereo image by performing wide-angle capturing by disposing a mirror at a right angle to a plane on which the capturing unit is disposed. A photographed image input means (for example, realized by the partial mirror image reversing unit 220) for inputting an image photographed in a state in which half of the images are reversed with respect to the other half of the images; An image reversing unit (for example, realized by the address control unit 222) for reversing any half of the captured images and outputting a stereo image is provided.

  In addition, the stereo image capturing system includes a memory (for example, realized by the memory 221) that stores an image input by the captured image input unit, and the image inversion unit performs memory address control when the image is read from the memory. By performing this, one half of the image captured by the image capturing means may be reversed. According to such a configuration, an image can be easily inverted by performing address control.

  In the stereo image capturing system, the image inverting means may read out only half of the images stored in the memory in the reverse address order.

  Further, the stereo image capturing system may include an image detection unit (for example, realized by the stereo image processing unit 223) that detects an object included in the stereo image output by the image inversion unit.

  A stereo image shooting program according to the present invention is a stereo image shooting program for shooting a stereo image by performing a wide-angle shooting by arranging a mirror at a right angle to a plane on which shooting means are arranged. The image capturing means inputs a photographed image input process in which a half image in the image is inverted with respect to the other half of the image, and either half of the image captured by the image capturing means The image inversion processing for inverting the image and outputting a stereo image is executed.

  In the present invention, when a stereo image is captured using one camera, if a mirror enters only one of the left and right images, only one of the left and right images is reversed, so that the stereo image is not formed. Therefore, as a pre-process for stereo image processing, mirror image inversion processing by image processing is added to only one of the left and right images in the image. If another mirror is added, it causes vignetting in wide-angle shooting, so image processing without vignetting is realized using only one mirror.

  Further, by arranging the mirror and the camera at a right angle, it is possible to take a stereo image using almost all the angle of view of the camera. In this case, the mirror is usually large in order to increase the angle of view, and it is usually an obstacle to install on the ceiling. In the present invention, by arranging the mirror and the camera at right angles, the camera unit can be installed in contact with the wall surface, and a camera unit having an unobstructed shape can be configured. When used for measuring the number of people passing by or the like, it is common to measure at the entrance and exit, and it is desirable to install a mirror on the wall.

  However, an image that can be acquired (captured) in the present invention is different from a normal stereo image. In a normal stereo image, there is a viewpoint at the center of each of the left image and the right image in the image. Hereinafter, this image is referred to as an all stereo image. The stereo image of the present invention is a semi-stereo image having a viewpoint at the left end of each of the left image and the right image. Even in such a semi-stereo image, if the viewpoint is handled correctly, the parallax can be obtained from the semi-stereo image, and the distance and height can be calculated.

  According to the present invention, when a stereo image is captured by disposing a mirror at right angles to the plane on which the image capturing unit is disposed and performing wide-angle image capturing, the image capturing unit displays half of the image. Take an image that is inverted with respect to the other half of the image. Then, any half of the images photographed by the photographing means is inverted.

  With the configuration described above, it is possible to realize wide-angle stereo shooting by preventing the vignetting by using the mirror image inversion processing of the image processing together and limiting the number of mirrors to one. Furthermore, by arranging the mirror at right angles to the camera, it is possible to install a large mirror so that it does not get in the way when it is in contact with the wall surface. Therefore, the angle of view can be covered without waste and the camera unit can be configured compactly.

  Further, with the configuration as described above, wide-angle stereo shooting with one camera can be realized. Compared to the conventional two-camera system, the amount of components for constituting the camera unit is also halved, so that the cost can be halved. Further, if one mirror and one camera are used, a camera unit can be configured, and the mirror is arranged at right angles to the camera, so that the adjustment items of the camera can be reduced. For example, in the conventional two-camera method, it takes a lot of time to adjust the position and orientation between cameras, but according to the present embodiment, only one camera needs to be adjusted. Further, since there is one camera, external synchronization is unnecessary, and a camera unit can be configured using an inexpensive camera.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of a configuration of a stereo image capturing system using a stereo image capturing method according to the present invention. In the present embodiment, an example will be described in which the camera 10 is installed on a ceiling having a height of 2500 mm (2.5 m) and stereo shooting is performed at an angle of about 60 degrees with respect to the downward direction from the camera 10. Further, in the present embodiment, the stereo image capturing system can be used, for example, when measuring the number of people passing through a doorway in a room.

  As illustrated in FIG. 1, the stereo image capturing system includes an input unit 210, a display unit 211, an overall control unit 212, a partial mirror image inversion unit 220, a stereo image processing unit 223, and a display control unit 224. As shown in FIG. 1, the stereo image capturing system captures a stereo image using a camera unit 100. Note that the stereo image photographing system is realized by, for example, an information processing apparatus such as a personal computer, or various circuits constituting each processing unit / control unit.

  The camera unit 100 includes a camera 10 and a mirror 11. FIG. 2 is an explanatory diagram illustrating an example of the camera unit 100. As shown in FIG. 2, the mirror 11 is installed on a wall surface such as a room. Further, as shown in FIG. 2, the camera 10 is installed on the ceiling and arranged in a direction perpendicular to the mirror 11. In other words, the camera 10 is arranged so as to be able to photograph a direction directly below the ceiling.

  The partial mirror image reversing unit 220 has a function of inputting from the camera 10 an image captured in a state where half of the image is reversed with respect to the other half of the image. The partial mirror image reversing unit 220 has a function of reversing any half of the images captured by the camera 10. Further, as shown in FIG. 1, the partial mirror image inversion unit 220 includes a memory 221 and an address control unit 222.

  The memory 221 is, for example, a memory installed in the information processing apparatus. The memory 221 stores a camera image taken by the camera 10.

  The address control unit 222 is realized by a circuit having an address control function, for example. Note that the address control unit 222 may be realized by a CPU of an information processing apparatus that operates according to a program, for example. The address control unit 222 has a function of controlling the address of the memory 221 when reading an image from the memory 221. In the present embodiment, the address control unit 222 inverts one half of the images captured by the camera 10 by performing address control of the memory 221 when reading an image from the memory 221.

  The stereo image processing unit 223 is realized by a circuit having an image processing function, for example. Note that the stereo image processing unit 223 may be realized by a CPU of an information processing apparatus that operates according to a program, for example. The stereo image processing unit 223 has a function of executing predetermined stereo image processing. For example, the stereo image processing unit 223 detects an object (for example, a human) included in the stereo image from the partial mirror image inversion unit 220 and outputs the detection result.

  The display control unit 224 is realized by a CPU of an information processing device that operates according to a program, for example. The display control unit 224 has a function of displaying a stereo image from the partial mirror image inversion unit 220 on the display unit 211 such as a monitor. The display control unit 224 has a function of causing the display unit 211 to display the processing result of the stereo image processing executed by the stereo image processing unit 223.

  The overall control unit 212 is realized by, for example, a CPU of an information processing apparatus that operates according to a program. The overall control unit 212 has a function of causing the display unit 211 to display various setting screens when taking an image. The overall control unit 212 has a function of inputting various operation information from the input unit 210 in accordance with a user operation.

  The display unit 211 is realized by a display device such as a display device, for example. The display unit 211 has a function of displaying an image according to an instruction from the display control unit 224 or displaying various setting screens according to an instruction from the overall control unit 212.

  The input unit 210 is realized by an input device such as a keyboard and a mouse, for example. The input unit 210 has a function of inputting various operation information and outputting it to the overall control unit 212 in accordance with a user operation.

  In the present embodiment, a storage device (not shown) of an information processing apparatus that implements a stereo image capturing system stores various programs for capturing a stereo image using one camera 10. For example, the storage device of the information processing apparatus includes a captured image input process in which a photographing unit inputs an image photographed in a state in which half of the image is inverted with respect to the other half of the image. A stereo image shooting program for executing image inversion processing for inverting any half of the images captured by the image capturing means and outputting a stereo image is stored.

  When the processing of the stereo image capturing method described above is realized by using an information processing apparatus (for example, a personal computer), the stereo image capturing described above is used when actually used for the purpose of measuring the number of passing people. Human tracking is performed by continuously processing the method. In this case, it is essential for the information processing apparatus to execute processing in real time, and the information processing apparatus needs to process an image of 15 to 30 frames per second. That is, in general, in the NTSC standard, the output signal of the surveillance camera is about 30 frames per second. Therefore, in order to perform human tracking, it is necessary to perform processing once every frame or every two frames. A high-performance information processing apparatus is required to process an image of 15 to 30 frames.

  Next, the operation will be described. FIG. 3 is a flowchart showing an example of processing in the case of performing stereo shooting at an angle of about 60 degrees from the camera 10 with respect to the direction directly below. In the present embodiment, the camera 10 is set so that the shooting angle of view of the camera 10 itself is 120 degrees, and the left and right angles are 60 degrees from the ceiling.

  When image capturing by the camera 10 is started, an image on the right half of the image captured by the camera 10 (that is, an image obtained by capturing an area opposite to the wall surface on which the mirror 11 is installed) The captured image is input. On the other hand, an image on the left half of an image captured by the camera 10 (that is, an image obtained by capturing an area on the wall surface on which the mirror 11 is installed) is captured through the mirror 11 (that is, an area on the opposite side to the wall surface) The image is taken on the mirror 11). Therefore, the left half image and the right half image are images that are reflected in a state where mirror images (for example, humans) in the images are reversed.

  By doing as described above, using only one camera 10 is equivalent to placing another camera virtually on the surface of the camera 10 with the mirror 11 in between. Therefore, the apparent camera interval between the camera 10 and the other virtual camera is twice the interval between the mirror and the camera 10, which is effective for reducing the size of the camera unit 100.

  For example, in the case of a configuration in which a stereo image or a semi-stereo image (a stereo image captured so that the viewpoint is not in the middle of the image but at the end (for example, the left end)) is used without using the mirror 11, as illustrated in FIG. Thus, the camera unit must be configured using two cameras. Therefore, it is necessary to secure a space for setting two cameras, and the scale of the camera unit increases.

  On the other hand, in this embodiment, as shown in FIG. 5, the camera unit 100 is configured using the mirror 11, so that only one camera 10 is used and the mirror 11 is placed on the opposite side. A stereo image similar to that of another camera 10A (virtual camera) can be taken. Therefore, a stereo image can be taken only by using one camera 10, and the camera unit 100 can be downsized. As shown in FIG. 5, the camera interval between the camera 10 and the virtual camera 10 </ b> A can be 2d, which is twice the distance d between the camera 10 and the mirror 11.

  In order to be able to photograph the left half image up to the limit where the photographing area is the same as the photographing area of the right half image, the mirror 11 must have a size that reaches the floor as a mirror 11. However, in order to use the stereo image capturing system at the doorway (for example, when used for measuring the number of people passing through the doorway), the mirror 11 cannot be installed at the doorway. Will be limited in size. In this case, as shown in FIG. 6, if the height of the doorway is 2000 mm, the height to the ceiling is 2500 mm, so that the mirror 11 having a length of 500 mm is installed on the wall surface.

  Note that the mirror 11 does not need to be a surface mirror in particular because it does not require fine accuracy in applications where a person passing through the doorway is detected. In view of safety, an acrylic mirror may be used as the mirror 11. However, in order to obtain a stereo image, a mirror having a small surface curvature must be used.

  When the partial mirror image reversing unit 220 inputs a camera image taken by the camera 10 (step S401), first, as shown in FIG. 7, the distance from the camera 10 to the mirror 11 is set as d, using equation (1). An effective angle of view is calculated (step S402).

Left half angle of view effective angle of view = 60 degrees−ATAN (d [mm] / 500 [mm]) Formula (1)

  For example, in step S402, when d = 25 mm, the stereo image processing unit 223 obtains the effective field angle of the left half image as shown in Expression (2).

Left half image effective angle of view = 60 degrees-ATAN (25 [mm] / 500 [mm])
≒ 57 degrees Formula (2)

  Further, in this case, the apparent camera interval between the camera 10 and the camera 10A is 50 mm because it is twice d.

  As a matter of course, the camera 10 cannot be placed on the ceiling unless the size from the end to the center of the camera 10 is 25 mm or less. For example, if a 38 mm square board camera is used as the camera 10, the distance from the end to the camera center is 19 mm, so that the camera 10 can be disposed without any problem.

  The image captured by the camera 10 is an image obtained by inverting the left half image with respect to the right half image. Next, the partial mirror image inversion processing unit 220 inverts the left half image of the input camera image, and outputs a stereo image in which the left half image and the right half image have the same direction (step S403).

  The partial mirror image inversion processing unit 220 captures the input camera image as it is in the memory 221 and stores the camera image in the memory 221 (step S404).

  In addition, when reading the camera image from the memory 221, the address control unit 222 of the partial mirror image inverting unit 220 controls the address of the memory 221 so that only the left half image is read in the reverse address order (step S405).

  Specifically, the address control unit 222 performs address control so as to give a read address with a down-count from 319 to 0 when reading the left half image for the line 640 dots. Further, the address control unit 222 performs address control so as to give a read address with an up-count from 320 to 639 when reading the right half image.

  Next, the stereo image processing unit 223 performs predetermined stereo image processing based on the stereo image from the partial mirror image inversion unit 220 (step S406). For example, when detecting a human being from a stereo image, the stereo image processing unit 223 sets the range in which an object at a height of 1500 mm, which is almost the height of a human shoulder, as an effective length of the stereo image. Performs processing to detect humans from inside. In this case, the stereo image processing unit 223 can detect an object (for example, a human being) having a height of 1500 mm within the range represented by Expression (3) when there is an angle of view of 60 degrees with respect to the wall surface.

(2500 [mm] -1500 [mm]) × TAN 60 degrees = 1732 [mm] Formula (3)

In addition, when the vertical angle of view is 90 degrees (= 120 degrees × 3/4) with respect to the wall surface, the captured image can be used as it is, so that the stereo image processing unit 223 is within the range represented by the equation (4). An object with a height of 1500 mm can be detected.
(2500 [mm] -1500 [mm]) × TAN (90 degrees / 2) = 2000 [mm]
Formula (4)

  That is, as shown in FIG. 8, the stereo image processing unit 223 can detect a person who is in a rectangular area having a size of approximately 2.0 m wide by 1.7 m long from the wall.

  Then, the display control unit 224 causes the display unit 211 such as a monitor to display the stereo image from the partial mirror image inversion unit 220 and the processing result of the stereo image processing executed by the stereo image processing unit 223 (step S407). For example, the display control unit 224 causes the display unit 211 to display the number of people who have passed through an entrance included in the image within a predetermined time as a processing result of the stereo image processing.

  As described above, according to the present embodiment, a stereo image is shot by performing wide-angle shooting by arranging the mirror 11 at a right angle to the plane on which the camera 10 is placed. In this case, the camera 10 captures an image in a state where half of the image is inverted with respect to the other half of the image. Further, the partial mirror image inversion unit 220 inverts any half of the images taken by the camera 10.

  By configuring as described above, it is possible to realize wide-angle stereo shooting by preventing the vignetting by combining the mirror image reversal processing of the image processing and suppressing the mirror 11 to one. Furthermore, by arranging the mirror 11 at a right angle to the camera 10, it is possible to install a large mirror so as not to get in the way in contact with the wall surface. Therefore, it is possible to cover the angle of view without waste and to configure the camera unit 100 in a compact manner.

  Further, according to the present embodiment, it is possible to realize wide-angle stereo shooting with one camera by configuring as described above. According to the present embodiment, the amount of components for constituting the camera unit is substantially halved with respect to the conventional two-camera system, so that the cost can be substantially halved. In addition, if one mirror 11 and one camera 10 are used, the camera unit 100 can be configured, and the mirror 11 is disposed at a right angle with respect to the camera 10, so that adjustment items for the camera 10 can also be reduced. For example, in the conventional two-camera system, it takes a lot of time to adjust the position and orientation between cameras, but according to the present embodiment, only one camera 10 needs to be adjusted. Further, since there is one camera, external synchronization is unnecessary, and a camera unit can be configured using an inexpensive camera.

  INDUSTRIAL APPLICABILITY The present invention can be applied to uses such as a system for measuring the number of people passing through a doorway in a room.

It is a block diagram which shows an example of a structure of the stereo image imaging system using the stereo image imaging method by this invention. 2 is an explanatory diagram illustrating an example of a camera unit 100. FIG. It is a flowchart which shows an example of a process in the case of carrying out stereo imaging | photography at an angle of about 60 degree | times with respect to the downward direction from the camera. It is explanatory drawing which shows the structure of the camera unit which image | photographs a full stereo image and a semi-stereo image. It is explanatory drawing which shows the structure of the camera unit which image | photographs a stereo image using one camera. It is explanatory drawing which shows the relationship of the length of the mirror which can be installed with respect to the height of an entrance / exit, and the height to a ceiling. It is explanatory drawing which shows the calculation method of an effective angle of view. It is explanatory drawing explaining the area | region which can detect a human. It is explanatory drawing which shows the imaging | photography method of the stereo image using one conventional camera.

Explanation of symbols

10 camera 11 mirror 100 camera unit 210 input unit 211 display unit 212 overall control unit 220 differential mirror image inversion unit 221 memory 222 address control unit 223 stereo image processing unit 224 display control unit

Claims (6)

A stereo image photographing method for photographing a stereo image by performing wide-angle photographing by arranging a mirror at a right angle to a plane on which photographing means is disposed,
A step of photographing the image in a state in which half of the image is inverted with respect to the other half of the image;
An image inversion step of inverting any half of the images captured by the imaging means and outputting a stereo image. A stereo image photographing system for photographing a stereo image by performing a wide-angle photographing by arranging a mirror at a right angle to a plane on which photographing means is disposed,
The photographing means inputs a photographed image input means for inputting an image photographed in a state where a half image in the image is inverted with respect to the other half image in the image;
A stereo image photographing system comprising: an image reversing unit that inverts any half of the images photographed by the photographing unit and outputs a stereo image. A memory for storing an image input by the photographed image input means;
The stereo image photographing system according to claim 2, wherein the image reversing unit performs address control of the memory when the image is read from the memory, thereby reversing any half of the images photographed by the photographing unit.   The stereo image photographing system according to claim 3, wherein the image reversing means reads out only one half of the images stored in the memory in reverse address order.   The stereo image photographing system according to any one of claims 2 to 4, further comprising an image detection unit that detects an object included in a stereo image output by the image reversing unit. A stereo image shooting program for shooting a stereo image by performing a wide-angle shooting by arranging a mirror at a right angle to a plane on which shooting means are arranged,
On the computer,
A shooting image input process in which the shooting means inputs an image shot in a state in which half of the image is inverted with respect to the other half of the image;
A stereo image photographing program for executing image reversal processing for reversing any half of the images photographed by the photographing means and outputting a stereo image.

Images