JP2016058886A - Imaging device, imaging system, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device, an imaging system, an image processing method and a program that can obtain an image suitable for user's browsing according to the setup orientation of an imaging unit.SOLUTION: An imaging device 1 has an imaging unit for picking up an image, a detector for detecting the tilt of the imaging unit with respect to the vertical direction V of an actual space, and an image processor for performing image processing on an image picked up by the imaging unit. The image processor has a selection part and a conversion part. On the basis of the tilt detected by the detector, the selection part selects a projection face from plural kinds of projection faces 101, 102 which are virtual faces set in the actual space and different in shape from each other. The conversion unit converts an image picked up by the imaging unit so that the image picked up by the imaging unit is mapped on the projection face selected by the selection unit.SELECTED DRAWING: Figure 1

Description

  The present invention generally relates to an imaging apparatus, an imaging system, an image processing method, and a program, and more specifically, an imaging apparatus, an imaging system, an image processing method, and an image processing apparatus that perform appropriate image processing on an image generated by an imaging unit. And program.

  2. Description of the Related Art Conventionally, an imaging apparatus capable of photographing all directions (omnidirectional sphere) at once by using a fisheye lens (super wide angle lens) having an angle of view of 180 degrees or more and using two imaging elements (so-called two eyes). Has been proposed (see, for example, Patent Document 1).

  The imaging apparatus described in Patent Literature 1 includes an inclination detection unit that detects an inclination with respect to a vertical direction, conversion data that converts a plane coordinate value into a spherical coordinate value, and a correction unit that corrects the conversion data according to the inclination. It is out. When generating an omnidirectional (omnidirectional) image, this imaging apparatus detects the vertical direction and corrects the conversion table used for image processing according to the vertical direction. Thereby, the imaging device can generate a correct omnidirectional (omnidirectional) image in the vertical direction in response to an arbitrary inclination of the device with respect to the vertical direction.

  This imaging apparatus is basically used in the vertical direction (vertical direction), but when tilted with respect to the vertical direction, the zenith portion of the captured image may be shifted and the horizontal line may be distorted. Even in such a case, the image pickup apparatus described in Patent Document 1 corrects the conversion data according to the inclination with respect to the vertical direction, and thus the distortion of the horizontal line is equivalent to the case where it is installed without being inclined with respect to the vertical direction. A suppressed image can be obtained.

JP 2013-214947 A

  By the way, in the imaging device described in Patent Document 1, the orientation of the converted image is uniquely determined with respect to the vertical direction so that the zenith portion of the converted image matches the zenith portion of the real space. Therefore, for example, if the imaging device has a horizontal angle of view of 360 degrees, an image with a horizontal angle of view of 360 degrees is the same regardless of whether the imaging device is attached to the ceiling or the imaging device is attached to a wall. Will be obtained. Then, when the imaging device is attached to the wall, only the wall is reflected in about half of the image, and there is a possibility that the image is not suitable for the user's browsing.

  The present invention has been made in view of the above reasons, and provides an imaging apparatus, an imaging system, an image processing method, and a program capable of obtaining an image suitable for a user's browsing according to the direction in which the imaging unit is installed. For the purpose.

  An imaging apparatus according to the present invention includes an imaging unit that captures an image, a detection unit that detects an inclination of the imaging unit with respect to a vertical direction of real space, and an image processing unit that performs image processing on an image captured by the imaging unit. The image processing unit selects a projection plane based on the inclination detected by the detection unit from a plurality of types of projection planes that are virtual planes set in the real space and have different shapes. And a conversion unit that converts the image captured by the imaging unit so that the image captured by the imaging unit is mapped to the projection plane selected by the selection unit. And

  The imaging system of the present invention acquires data from an imaging unit that captures an image, a detection unit that detects an inclination of the imaging unit with respect to a vertical direction of real space, and the imaging unit and the detection unit to obtain data. An image processing unit that performs image processing on the captured image, and the image processing unit is a virtual plane set in the real space, and the detection unit is selected from a plurality of types of projection planes having different shapes. A selection unit that selects a projection plane based on the inclination detected in step (b), and an image captured by the imaging unit so that an image captured by the imaging unit is mapped to the projection plane selected by the selection unit. And a conversion unit for converting the image.

  The image processing method of the present invention selects a projection plane based on the inclination of the imaging unit with respect to the vertical direction of the real space from a plurality of types of projection planes that are virtual planes set in the real space and have different shapes. And a second step of converting the image captured by the imaging unit so that the image captured by the imaging unit is mapped onto the projection plane selected in the first step. It is characterized by that.

  The program according to the present invention allows a computer connected to an image capturing unit that captures an image to be a virtual plane set in the real space and a plurality of types of projection planes having different shapes from each other with respect to the vertical direction of the real space. A selection unit that selects a projection plane based on the tilt of the imaging unit, and an image captured by the imaging unit so that an image captured by the imaging unit is mapped to the projection plane selected by the selection unit To function as a conversion unit for converting.

  The present invention has an advantage that an image suitable for viewing by the user can be obtained according to the direction in which the imaging unit is installed.

1A is a perspective view of a state where the imaging apparatus according to Embodiment 1 is attached to a ceiling, and FIG. 1B is a perspective view of a state where the imaging apparatus according to Embodiment 1 is attached to a wall. 1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment. 3A is a side view of the imaging apparatus according to Embodiment 1 installed in a downward direction, FIG. 3B is a side view of the imaging apparatus according to Embodiment 1 installed in an upward direction, and FIG. It is a side view of the state installed sideways of the imaging device. 4A is a schematic diagram of a converted image in the comparative example, and FIG. 4B is a schematic diagram of a converted image obtained by the imaging apparatus according to the first embodiment. 6 is a perspective view showing an imaging apparatus according to Embodiment 2. FIG. 6 is a flowchart illustrating an operation of the imaging apparatus according to the second embodiment. 6 is a perspective view showing an imaging apparatus according to Embodiment 2. FIG. 8A is a perspective view of a state in which the imaging device according to the modification is installed upward, FIG. 8B is a perspective view of a state in which the imaging device according to the modification is installed obliquely upward, and FIG. 8C is an imaging device according to the modification. It is a perspective view of the state installed diagonally downward. It is a block diagram which shows the structure of the imaging system which concerns on embodiment.

(Embodiment 1)
As shown in FIG. 2, the imaging apparatus 1 according to the present embodiment includes an imaging unit 2 that captures an image, and a detection unit 3 that detects the inclination of the imaging unit 2 with respect to the vertical direction V of real space (see FIG. 1A). The image processing unit 4 performs image processing on the image captured by the image capturing unit 2.

  The image processing unit 4 includes a selection unit 41 and a conversion unit 42 that converts an image captured by the imaging unit 2. Here, as shown in FIGS. 1A and 1B, the selection unit 41 is detected by the detection unit 3 from a plurality of types of projection planes 101 and 102 which are virtual planes set in the real space and have different shapes. A projection plane is selected based on the tilt. The conversion unit 42 converts the image (captured by the imaging unit 2) so that the image captured by the imaging unit 2 is mapped to the projection plane selected by the selection unit 41.

  According to this configuration, the imaging apparatus 1 according to the present embodiment determines which projection plane to map an image from among a plurality of types of projection planes that are virtual planes set in real space and have different shapes. The selection is made based on the inclination of the imaging unit 2 with respect to the vertical direction V of the real space. In other words, the imaging device 1 performs image processing on the image captured by the imaging unit 2 by the image processing unit 4 so that the image is mapped to any one of the projection planes. Has been converted. Here, the projection plane onto which the image captured by the imaging unit 2 is mapped is automatically selected based on the inclination of the imaging unit 2 with respect to the vertical direction V of the real space. Therefore, the imaging apparatus 1 of the present embodiment has an advantage that an image suitable for the user's browsing can be obtained according to the direction in which the imaging unit 2 is installed.

  Hereinafter, the imaging device 1 of the present embodiment will be described in detail. However, the imaging apparatus 1 described below is only an example of the present invention, and the present invention is not limited to the following embodiment, and the technical idea according to the present invention is not limited to this embodiment. As long as it does not deviate from the above, various changes can be made according to the design and the like.

  In the following embodiments, for the purpose of crime prevention or the like, a case where the imaging device 1 is attached to the ceiling or wall of a building (for example, a detached house, an apartment house, an office, a store, a factory, etc.) as a monitoring camera will be described as an example. To do. In addition, the imaging apparatus 1 is assumed to be a camera that captures a moving image here, but may be a camera (still camera) that captures a still image.

<Configuration of imaging device>
As shown in FIG. 2, the imaging apparatus 1 according to the present embodiment includes the imaging unit 2, the detection unit 3, and the image processing unit 4 described above in one housing 5.

  The imaging unit 2 has one imaging element 21 and one optical system 22. The image sensor 21 is a two-dimensional image sensor such as a charge coupled devices (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor. The imaging element 21 forms an image of light from the imaging target on the imaging surface (light receiving surface) by the optical system 22, converts it into an electrical signal, and outputs it. In this embodiment, the optical system 22 includes a fisheye lens (super wide angle lens) having an angle of view of 180 degrees or more.

  As a result, an image with an angle of view of 180 degrees is projected through the optical system 22 onto the imaging surface of the imaging element 21. In the present embodiment, the image projected on the imaging surface of the imaging device 21 is an orthographic image obtained by projecting an image on the hemisphere, which is the surface of the fisheye lens, perpendicularly to the plane, which is the bottom surface of the hemisphere. That is, an image picked up by the image pickup unit 2 (hereinafter referred to as “original image”) is an image projected onto a hemispherical virtual surface set around the image pickup unit (image pickup element 21) 2 in real space. Become. In this way, a virtual surface in real space in which an image obtained by imaging is regarded as being projected is hereinafter referred to as a projection surface, and in particular, a projection surface corresponding to the original image (the virtual surface on the hemisphere described above). Is called the original projection plane 100 (see FIG. 1A).

  In the present embodiment, an XYZ orthogonal coordinate system defined by the X axis and the Y axis included in the imaging surface of the imaging device 21 and the Z axis corresponding to the optical axis of the imaging unit 2 is a camera coordinate system, and the center of the original image The coordinates are (x, y, z) = (0, 0, 0). The camera coordinate system here is a coordinate system set on the real space with reference to the orientation of the imaging unit 2. The camera coordinate system (XYZ orthogonal coordinate system) is converted to the UV coordinate system in the original image as follows. Here, the U axis is parallel to the X axis, the V axis is parallel to the Y axis, and (u, v) = (α, β) is the center position of the original image ((x, y) = (0, 0). )).

  First, the position of the point P on the original projection plane 100 is represented by (θ, φ, R) by converting the camera coordinate system, which is an orthogonal coordinate system, into a polar coordinate system. Here, θ is an angle formed by a straight line connecting the center of the original image and the point P on the original projection plane 100 and the optical axis (Z axis) of the imaging unit 2, and φ is the point P and the bottom surface of the original projection plane 100. An angle formed by a straight line connecting the point projected perpendicularly to the plane and the center of the original image and the X axis, R is the radius of the original projection plane 100.

  Furthermore, if the above θ, R and orthographic projection functions are used, the distance (radius) r of the P point projected on the original image from the center of the original image is represented by r = R × sin θ. Therefore, the position (u, v) of the point P in the UV coordinate system in the original image is expressed as follows using the above r and φ.

u = r cosφ + α
v = r sinφ + β
However, the projection method of the fisheye lens is not limited to the orthographic projection as described above, and may be, for example, an equisolid angle projection, an orthographic projection, or an equidistant projection. In the case of a projection method other than orthographic projection, since the projection function changes, the expression representing r in the above description changes. That is, r = R × 2 sin (θ / 2) for equisolid angle projection, r = R × 2 tan (θ / 2) for stereo projection, and r = R × θ for throw distance projection. Will be represented.

  The imaging unit 2 can use a captured image (original image) as, for example, JPEG format or H.264. H.264 or H.264 The image data is compressed by an image compression method such as H.265, and is output to the image processing unit 4 as image data. The optical system 22 is not limited to a fisheye lens, and may be configured using another wide-angle lens or a curved mirror such as an omnidirectional mirror.

  The detection unit 3 measures at least the magnitude (inclination angle) of the imaging unit 2 with respect to the vertical direction (vertical direction) V of the real space. For the detection unit 3, for example, an acceleration sensor is used. In the present embodiment, a triaxial acceleration sensor is used for the detection unit 3 as an example.

  The detection unit 3 measures an angle formed by the optical axis (Z axis) of the imaging unit 2 and the vertical direction V of the real space as the first angle θ1, and a value corresponding to the first angle θ1 is measured by the image processing unit 4. Output to. In other words, the detection unit 3 is configured to detect the angle formed by the optical axis of the imaging unit 2 and the vertical direction V of the real space in the above inclination.

  For example, as shown in FIG. 3A, when the optical axis (Z-axis) of the imaging unit 2 coincides with the vertical direction V and faces downward (the direction in which gravity acts), the first measured by the detection unit 3. The angle θ1 of 1 is 0 degree. As shown in FIG. 3B, when the optical axis (Z-axis) of the imaging unit 2 coincides with the vertical direction V and faces upward, the first angle θ1 measured by the detection unit 31 is 180 degrees. . On the other hand, as shown in FIG. 3C, when the optical axis (Z axis) of the imaging unit 2 coincides with the horizontal direction, the first angle θ1 measured by the detection unit 31 is 90 degrees.

  In the present embodiment, the image processing unit 4 further includes a determination unit 43, a storage unit 44, and an output unit 45 in addition to the selection unit 41 and the conversion unit 42 described above, as shown in FIG. The image processing unit 4 is mainly composed of a computer including a CPU (Central Processing Unit) and a memory such as a microcomputer (microcomputer). The image processing unit 4 causes the computer to function as the selection unit 41 and the conversion unit 42 by executing a program stored in the memory by the CPU. The program may be written in the memory in advance, or may be provided by being recorded through a telecommunication line or in a recording medium such as a memory card.

  The determination unit 43 determines the orientation of the imaging unit 2 in the real space based on the output value of the detection unit 3. In the present embodiment, the determination unit 43 determines whether the imaging unit 2 is “vertical” or “horizontal”. Here, “vertical” is a state in which the optical axis (Z-axis) of the imaging unit 2 is along the vertical direction (upward or downward) V in real space, and “lateral” is the optical axis (Z-axis) of the imaging unit 2. Is a state along the horizontal direction in real space.

  Specifically, the determination unit 43 compares the first angle θ1 measured by the detection unit 3 with a predetermined threshold value, and determines whether it is “vertical” or “horizontal” based on the comparison result. Here, the threshold value to be compared with the first angle θ1 is not limited to one value, and may be two or more values, for example. In the present embodiment, binary values of 45 degrees (first threshold value) and 135 degrees (second threshold value) are set as threshold values. Then, the determination unit 43 determines that the first angle θ1 is smaller than the first threshold (0 ≦ θ1 <45) and the first angle θ1 is larger than the second threshold (135 <θ1 ≦ 180). ), It is determined that the imaging unit 2 is “vertical”. On the other hand, when the first angle θ1 is greater than or equal to the first threshold and less than or equal to the second threshold (45 ≦ θ1 ≦ 135), the determination unit 43 determines that the imaging unit 2 is “landscape”.

  The determination unit 43 is not limited to the two-stage determination of “vertical” and “horizontal”, and may be configured to perform determination of three or more stages.

  The selection unit 41 selects a projection plane from a plurality of types of projection planes having different shapes based on the determination result of the determination unit 43. That is, the selection unit 41 receives the determination result of the determination unit 43 and selects different projection planes depending on whether the imaging unit 2 is “vertical” or “horizontal”. In the present embodiment, the selection unit 41 selects a projection plane from two types of projection planes, which are a first projection plane 101 (see FIG. 1A) and a second projection plane 102 (see FIG. 1B), which will be described later.

  Here, the selection unit 41 is configured to select, from a plurality of types of projection planes, a projection plane having a shape along the vertical direction V of the real space and curved in the horizontal plane of the real space. Therefore, the selection unit 41 selects the first projection plane 101 if the imaging unit 2 is “vertical”, and selects the second projection plane 102 if the imaging unit 2 is “horizontal”. Specific shapes of the first projection plane 101 and the second projection plane 102 will be described later.

  The conversion unit 42 converts the original image so that the original image captured by the imaging unit 2 is mapped to the projection plane selected by the selection unit 41. Actually, the conversion unit 42 performs coordinate conversion of the original image using a conversion formula corresponding to the projection plane selected by the selection unit 41. At this time, the conversion unit 42 performs distortion correction and interpolation processing together with conversion of the original image so as to suppress distortion of the converted image (hereinafter referred to as “converted image”).

  The storage unit 44 stores in advance information corresponding to a plurality of types of projection planes having different shapes in association with the determination result of the determination unit 43. In the present embodiment, the storage unit 44 stores the conversion formula used by the conversion unit 42 in association with the determination result of the determination unit 43. As a result, the selection unit 41 receives the determination result of the determination unit 43 and reads the conversion formula corresponding to the determination result of the determination unit 43 from the storage unit 44, thereby performing projection after conversion based on the determination result of the determination unit 43. A face can be selected.

  The output unit 45 is configured to be able to communicate with an external device. The output unit 45 transmits the image after image processing, that is, the image converted by the conversion unit 42 (converted image) to an external device via a network including a local area network (LAN), for example. The external device is a storage device that stores the converted image, a monitor device that displays the converted image, or the like. The output unit 45 is not limited to the configuration for transmitting the converted image to the external device, but may be configured to write the converted image to a recording medium such as a built-in memory or a memory card.

<Operation of imaging device>
The imaging device 1 configured as described above generates a converted image by performing image processing on the original image obtained by the imaging unit 2 by the image processing unit 4. At this time, as shown in FIG. 1A and FIG. 1B, the image processing unit 4 moves the virtual plane in the real space where the image is assumed to be projected, that is, the projection plane onto which the image is mapped, from the original projection plane 100. , Switching to the first projection plane 101 or the second projection plane 102. Here, the image processing unit 4 selects a projection plane based on the inclination of the imaging unit 2 detected by the detection unit 3 from a plurality of types of projection planes having different shapes.

  In the present embodiment, the image processing unit 4 selects the first projection plane 101 (see FIG. 1A) based on the inclination of the imaging unit 2 detected by the detection unit 3 if the imaging unit 2 is “vertical”. In the case of “landscape”, the second projection plane 102 (see FIG. 1B) is selected.

  That is, when the imaging unit 2 is “vertical”, the imaging device 1 has a cylindrical shape (circumferential shape of a column) having the optical axis (Z axis) of the imaging unit 2 as a central axis as shown in FIG. 1A. A projection plane (first projection plane) 101 is selected by the selection unit 41. FIG. 1A illustrates a state in which the imaging device 1 is attached to the ceiling of a building, and the imaging device 1 illustrated in FIG. 1A is in a state in which the optical axis of the imaging unit 2 is directed downward. Accordingly, the first projection plane 101 is set around the central axis in the vertical direction V passing through the imaging unit 2 below the imaging device 1 in real space. In other words, in the state of FIG. 1A, the first projection plane 101 is a projection plane having a shape along the vertical direction V of the real space and curved in the horizontal plane of the real space.

  In this case, the imaging device 1 converts the original image by the conversion unit 42 so that the original image captured by the imaging unit 2 is mapped to the first projection plane 101. That is, the conversion unit 42 converts the original image using a conversion formula corresponding to the first projection plane 101 among the conversion formulas stored in the storage unit 44. Here, the converted image output from the output unit 45 is a two-dimensional image obtained by expanding the image projected on the cylindrical first projection plane 101 into a planar shape. Thereby, the imaging apparatus 1 can output a panoramic image having a horizontal angle of view of 360 degrees in the real space from the output unit 45 as a converted image.

  On the other hand, if the imaging unit 2 is “landscape”, a projection that intersects the optical axis (Z-axis) of the imaging unit 2 and is orthogonal to the vertical direction V of the real space is an arc shape as shown in FIG. 1B. The plane (second projection plane) 102 is selected by the selection unit 41. Here, the second projection surface 102 is a curved surface that faces the XY plane including the imaging surface of the imaging unit 2 and is curved so as to approach the XY plane as the distance from the Y axis increases in the X axis direction. FIG. 1B shows a state where the imaging device 1 is attached to the wall of the building, and the imaging device 1 shown in FIG. 1B is in a state where the optical axis of the imaging unit 2 is directed forward (front direction of the wall). Therefore, the second projection plane 102 is set around the central axis in the vertical direction V passing through the imaging unit 2 in front of the imaging device 1 in real space. In other words, in the state of FIG. 1B, the second projection plane 102 is a projection plane having a shape along the vertical direction V of the real space and curved in the horizontal plane of the real space.

  In this case, the imaging apparatus 1 converts the original image by the conversion unit 42 so that the original image captured by the imaging unit 2 is mapped to the second projection plane 102. That is, the conversion unit 42 converts the original image using a conversion formula corresponding to the second projection plane 102 among the conversion formulas stored in the storage unit 44. Here, the converted image output from the output unit 45 is a two-dimensional image obtained by expanding the image projected on the curved second projection plane 102 into a planar shape. Thereby, the imaging device 1 can output a panoramic image having a horizontal angle of view of about 180 degrees in the real space from the output unit 45 as a converted image.

  By the way, it is not indispensable for the detection unit 3 to measure the magnitude (angle) of the inclination as in the present embodiment, and the detection unit 3 simply switches the opening and closing of the contact depending on whether it is “vertical” or “horizontal”, for example. It may be configured as follows. In this way, in the image processing unit 4, the determination unit 43 can be omitted if the output of the detection unit 3 is switched depending on “vertical” or “horizontal”.

  Alternatively, the detection unit 3 may be configured such that the detection unit 3 with respect to the vertical direction V detects an inclination by image processing instead of the sensor. In this case, the detection unit 3 extracts a horizontal plane such as a floor from the original image, for example, and specifies the vertical direction V from the normal vector of the horizontal plane.

  The imaging device 1 may include an input unit that receives an operation input by a user, and may be configured to select a projection plane according to an input from the input unit. In this case, the user can manually select a projection plane different from the projection plane automatically selected by the selection unit 41.

<Effect>
As described above, the imaging apparatus 1 according to the present embodiment determines which projection plane to map an image from among a plurality of types of projection planes that are virtual planes set in real space and have different shapes. The selection is made based on the inclination of the imaging unit 2 with respect to the vertical direction V of the real space. In other words, the imaging device 1 performs image processing on the image captured by the imaging unit 2 by the image processing unit 4 so that the image is mapped to any one of the projection planes. Convert. Here, the projection plane onto which the image captured by the imaging unit 2 is mapped is based on the inclination of the imaging unit 2 with respect to the vertical direction V of the real space, from the first projection plane 101 and the second projection plane 102 having different shapes. Automatically selected. Therefore, the image pickup apparatus 1 according to the present embodiment performs different image processing (conversion) on the original image based on the inclination of the image pickup unit 2 with respect to the vertical direction V of the real space. As a result, there is an advantage that the imaging apparatus 1 can obtain an image suitable for the user's browsing according to the direction in which the imaging unit 2 is installed.

  Further, as in the present embodiment, the selection unit 41 selects a projection surface that is curved along the vertical direction V of the real space and curved in the horizontal plane of the real space from among a plurality of types of projection surfaces. It is preferable to be configured. According to this configuration, the imaging apparatus 1 can generate a converted image (image converted by the converting unit 42) in which straight line distortion along the vertical direction V of the real space is reduced. In order to explain this point, consider the case where the imaging device 1 captures an image of the object 200 standing upright in the vertical direction V in real space.

  In this case, if the selection unit 41 selects a projection surface that is curved along the horizontal direction of the real space and is curved in the vertical plane of the real space, as shown in FIG. 4A, the object in the converted image Im1 The distortion of 200 is relatively large. In particular, the distortion of the object 200 increases at positions near both ends in the horizontal direction in the converted image Im1. On the other hand, as shown in FIG. 4B, when the selection unit 41 selects a projection surface having a curved shape in the horizontal plane of the real space along the vertical direction V of the real space as in the present embodiment, the conversion is performed. The distortion of the object 200 in the image Im2 is reduced. In this way, the distortion of the object 200 standing upright in the vertical direction V in the real space can be reduced in the imaging apparatus 1 that uses an upright person as the object, such as a surveillance camera installed for the purpose of crime prevention or the like. It is particularly useful.

  Further, as in the present embodiment, the detection unit 3 sets the angle formed by the optical axis of the imaging unit 2 and the vertical direction V of the real space to the inclination (the inclination of the imaging unit 2 with respect to the vertical direction V of the real space). It is preferable to be configured to include and detect. According to this configuration, for example, different image processing (conversion) is performed on the original image when the imaging device 1 is attached to the ceiling and when the imaging device 1 is attached to the wall. Therefore, when the imaging device 1 is attached to a wall, it is possible to avoid a situation in which only the wall appears in about half of the image, and an image suitable for user browsing can be obtained.

  Further, as in the present embodiment, the selection unit 41 selects a projection plane from a plurality of types of projection planes based on a comparison result between the magnitude of the inclination detected by the detection unit 3 and a predetermined threshold value. It is preferable that it is comprised. According to this configuration, even when the magnitudes of inclinations detected by the detection unit 3 are different, the selection unit 41 selects the same projection plane as long as the comparison result between the magnitude of the inclination and the threshold value is the same. become. Therefore, even when the magnitude of the inclination of the imaging unit 2 with respect to the vertical direction V slightly changes with time, the imaging apparatus 1 is sufficient for the original image to perform the same image processing. Can be suppressed.

(Embodiment 2)
In the imaging apparatus 1 according to the present embodiment, the detection unit 3 has a reference direction that is set for the imaging unit 2 and is orthogonal to the optical axis (Z-axis) of the imaging unit 2 and a vertical direction V in real space. The angle formed is included in the inclination of the imaging unit 2 with respect to the vertical direction V of the real space and is detected. Hereinafter, the same configurations as those of the first embodiment are denoted by common reference numerals, and description thereof will be omitted as appropriate.

<Configuration of imaging device>
In the present embodiment, as an example, the Y-axis direction of the camera coordinate system is set as the reference direction. The detection unit 3 measures the angle between the Y-axis direction as the reference direction and the vertical direction V of the real space as the second angle θ2, and outputs a value corresponding to the second angle θ2 to the image processing unit 4. To do.

  For example, as shown in FIG. 5, when the optical axis (Z-axis) of the imaging unit 2 coincides with the horizontal direction (when the first angle θ1 is 90 degrees), the Y-axis direction of the imaging unit 2 is horizontal. If it coincides with the direction, the second angle θ2 measured by the detector 31 is 90 degrees. On the other hand, when the optical axis (Z-axis) of the imaging unit 2 matches the horizontal direction, the Y-axis of the imaging unit 2 matches the vertical direction V and faces downward (the direction in which gravity acts). The second angle θ2 measured by the detection unit 3 is 0 degree. Further, when the optical axis (Z axis) of the imaging unit 2 matches the horizontal direction, if the Y axis of the imaging unit 2 matches the vertical direction V and faces upward, it is measured by the detection unit 3. The second angle θ2 is 180 degrees.

  When the imaging unit 2 is “landscape”, the determination unit 43 performs “rotation” around the optical axis based on the angle between the reference direction detected by the detection unit 3 and the vertical direction V of the real space, It is determined whether it is “no rotation” around the axis. Here, the “with rotation” state is a state in which the reference direction is along the horizontal direction of the real space, and the “without rotation” state is a state in which the reference direction is along the vertical direction V of the real space.

  Specifically, the determination unit 43 compares the second angle θ2 measured by the detection unit 3 with a predetermined threshold value, and determines whether “with rotation” or “without rotation” based on the comparison result. Here, the threshold value compared with the second angle θ2 is not limited to one value, and may be two or more values, for example. In the present embodiment, binary values of 45 degrees (third threshold value) and 135 degrees (fourth threshold value) are set as threshold values. Then, the determination unit 43 determines that the second angle θ2 is smaller than the third threshold (0 ≦ θ2 <45) and the second angle θ2 is larger than the fourth threshold (135 <θ2 ≦ 180). ), “Rotation” is determined. On the other hand, when the second angle θ2 is not less than the third threshold and not more than the fourth threshold (45 ≦ θ2 ≦ 135), the determination unit 43 determines “no rotation”.

  The selection unit 41 selects a projection plane from a plurality of types of projection planes having different shapes based on the determination result of the determination unit 43. That is, the selection unit 41 receives the determination result of the determination unit 43, and when the imaging unit 2 is “vertical” or “horizontal”, or “horizontal”, “rotates” around the optical axis. Different projection planes are selected depending on whether or not “no rotation”. In the present embodiment, the selection unit 41 has three types of projections: a first projection plane 101 (see FIG. 1A), a second projection plane 102 (see FIG. 1B), and a third projection plane 103 (see FIG. 7) described later. Select a projection plane from the plane.

  Here, the selection unit 41 is configured to select, from a plurality of types of projection planes, a projection plane having a shape along the vertical direction V of the real space and curved in the horizontal plane of the real space. Therefore, the selection unit 41 selects the first projection plane 101 if the imaging unit 2 is “vertical”, and selects the second projection plane 102 if the imaging unit 2 is “horizontal”. The specific shape of the third projection plane 103 will be described later.

<Operation of imaging device>
The imaging device 1 configured as described above generates a converted image by performing image processing on the original image obtained by the imaging unit 2 by the image processing unit 4. At this time, the image processing unit 4 converts the virtual plane in the real space where the image is projected, that is, the projection plane onto which the image is mapped, from the original projection plane 100 to the first projection plane 101 and the second projection. Switch to the plane 102 or the third projection plane 103. Here, the image processing unit 4 selects a projection plane based on the inclination of the imaging unit 2 detected by the detection unit 3 from a plurality of types of projection planes having different shapes.

  In the present embodiment, as illustrated in FIG. 6, the image processing unit 4 is based on the inclination of the imaging unit 2 detected by the detection unit 3 and if the imaging unit 2 is “vertical” (S1: Yes). The first projection plane 101 is selected (S3). On the other hand, when the imaging unit 2 is “landscape” (S1: No), the image processing unit 4 is “rotated” or “rotated” around the optical axis based on the inclination of the imaging unit 2 detected by the detection unit 3. It is determined whether it is “none” (S2). At this time, if the image processing unit 4 is “with rotation” (S2: Yes), the third projection plane 103 is selected (S4). If it is “without rotation” (S2: No), the second projection is performed. The surface 102 is selected (S5).

  That is, the imaging device 1 intersects with the optical axis (Z axis) of the imaging unit 2 as shown in FIG. 7 if the imaging unit 2 is “laterally” and “rotates” around the optical axis, and The selection unit 41 selects a projection plane (third projection plane) 103 whose cross section perpendicular to the vertical direction V of the real space is an arc. Here, the third projection surface 103 is a curved surface that faces the XY plane including the imaging surface of the imaging unit 2 and is curved so as to approach the XY plane as the distance from the X axis increases in the Y axis direction. FIG. 7 illustrates a state in which the imaging device 1 is attached to the wall of the building, and the imaging device 1 illustrated in FIG. 7 is in a state in which the optical axis of the imaging unit 2 is directed forward (front direction of the wall). Therefore, the third projection plane 103 is set around the central axis in the vertical direction V passing through the imaging unit 2 in front of the imaging device 1 in real space. In other words, in the state of FIG. 7, the third projection plane 103 is a projection plane having a shape along the vertical direction V of the real space and curved in the horizontal plane of the real space.

  In this case, the imaging device 1 converts the original image by the conversion unit 42 so that the original image captured by the imaging unit 2 is mapped to the third projection plane 103. That is, the conversion unit 42 converts the original image using a conversion formula corresponding to the third projection plane 103 among the conversion formulas stored in the storage unit 44. Here, the converted image output from the output unit 45 is a two-dimensional image obtained by expanding the image projected on the curved third projection plane 103 into a planar shape. Thereby, the imaging device 1 can output a panoramic image having a horizontal angle of view of about 180 degrees in the real space from the output unit 45 as a converted image.

<Effect>
As described above, in the imaging device 1 according to the present embodiment, the detection unit 3 is set for the imaging unit 2 and the reference direction that is one direction orthogonal to the optical axis (Z axis) of the imaging unit 2 and the real space. The angle formed with the vertical direction V is detected by being included in the inclination of the imaging unit 2 with respect to the vertical direction V in the real space. According to this configuration, the image processing unit 4 differs from the original image depending on whether the image pickup unit 2 is “vertical” or “horizontal”, and “rotation” or “no rotation” around the optical axis. Image processing (conversion) can be performed. Therefore, the imaging device 1 of the present embodiment can obtain an image that is more suitable for the user's browsing.

(Modification)
In the imaging device 1 according to the first and second embodiments, the shape of the projection plane that the selection unit 41 selects based on the inclination detected by the detection unit 3 is not limited to the above example. It can be changed as appropriate.

  For example, when the optical axis (Z axis) of the imaging unit 2 is directed upward in the vertical direction V as illustrated in FIG. 8A, the imaging device 1 uses the optical axis (Z axis) of the imaging unit 2 as a central axis. The selection unit 41 may select a cylindrical projection surface (fourth projection surface) 104 to be cylindrical. The fourth projection plane 104 is set around the central axis in the vertical direction V passing through the imaging unit 2 above the imaging device 1 in real space.

  8B, when the optical axis (Z axis) of the imaging unit 2 is directed obliquely upward as shown in FIG. 8B, the imaging device 1 intersects the optical axis (Z axis) of the imaging unit 2 and the X axis. The selection unit 41 selects a projection plane (fifth projection plane) 105 in which a cross section perpendicular to the direction (horizontal direction) has an arc shape. Here, the fifth projection surface 105 is a curved surface that faces the XY plane including the imaging surface of the imaging unit 2 and is curved so as to approach the XY plane as the distance from the X axis increases in the Y axis direction.

  Furthermore, when the optical axis (Z-axis) of the imaging unit 2 is directed obliquely downward as illustrated in FIG. 8C, the imaging device 1 intersects the optical axis (Z-axis) of the imaging unit 2 and the X-axis. The selection unit 41 selects a projection plane (sixth projection plane) 106 in which a cross section perpendicular to the direction (horizontal direction) has an arc shape. Here, the sixth projection surface 106 is a curved surface that faces the XY plane including the imaging surface of the imaging unit 2 and is curved so as to approach the XY plane as the distance from the X axis increases in the Y axis direction.

  This modification can be combined with any of the configurations of the first and second embodiments.

(Imaging system)
Functions similar to those of the imaging apparatus 1 described above can be realized even in an imaging system 10 in which a plurality of units are combined as shown in FIG.

  The imaging system 10 illustrated in FIG. 9 includes an imaging unit 2 that captures an image, a detection unit 3 that detects the inclination of the imaging unit 2 with respect to the vertical direction of real space, and an image processing unit 40. The image processing unit 40 is configured to acquire data from the imaging unit 2 and the detection unit 3 and to perform image processing on the image captured by the imaging unit 2.

  The image processing unit 40 includes a selection unit 41 and a conversion unit 42 that converts an image captured by the imaging unit 2. Here, the selection unit 41 selects a projection plane based on the inclination detected by the detection unit 3 from a plurality of types of projection planes that are virtual planes set in the real space and have different shapes. . The conversion unit 42 converts the image (captured by the imaging unit 2) so that the image captured by the imaging unit 2 is mapped to the projection plane selected by the selection unit 41.

  Since the basic configuration of the imaging system 10 is the same as that of the imaging device 1 described above, the same configuration as the imaging device 1 is denoted by the same reference numeral, and description thereof is omitted as appropriate.

  In the example of FIG. 8, the imaging system 10 includes an imaging unit 20 in which the imaging unit 2 and the detection unit 3 are integrated. The housing 51 of the imaging unit 20 is separate from the housing 52 of the image processing unit 40. Therefore, the imaging unit 20 is configured to be able to communicate with the image processing unit 40. The communication method between the imaging unit 20 and the image processing unit 40 may be a wired method or a wireless method. Further, the imaging unit 20 may be connected to the image processing unit 40 via a network such as the Internet. The image processing unit 40 includes an acquisition unit 46, and the acquisition unit 46 acquires the image data of the original image from the imaging unit 2 and the output value of the detection unit 3.

  According to this configuration, the imaging system 10 determines which of the plurality of types of projection planes, which are virtual planes set in the real space and have different shapes, is mapped to the vertical plane in the real space. The selection can be made based on the inclination of the imaging unit 2 with respect to the direction V. Therefore, according to this imaging system 10, there exists an advantage that the image suitable for a user's browsing can be obtained according to the direction in which the imaging part 2 is installed.

(Image processing method)
An image processing method that can be applied to the imaging apparatus 1 and the imaging system 100 described above is a virtual plane set in the real space, and a plurality of types of projection planes having different shapes from each other with respect to the vertical direction V of the real space. A first step of selecting a projection plane based on the inclination of the imaging unit 2 is included. Furthermore, the image processing method includes a second step of converting the image captured by the imaging unit 2 so that the image captured by the imaging unit 2 is mapped to the projection plane selected in the first step. .

  According to this method, an imaging unit with respect to the vertical direction V of the real space, to which of the plurality of types of projection planes that are set in the real space and have different shapes is mapped. Can be selected based on the slope of 2. Therefore, according to this method, there is an advantage that an image suitable for user browsing can be obtained according to the direction in which the imaging unit 2 is installed.

(program)
A program that can be applied to the imaging apparatus 1 and the imaging system 100 causes a computer connected to the imaging unit 2 that captures an image to function as the selection unit 41 and the conversion unit 42 that converts the image captured by the imaging unit. . The selection unit 41 selects a projection plane based on the inclination of the imaging unit 2 with respect to the vertical direction of the real space from a plurality of types of projection planes that are virtual planes set in the real space and have different shapes. The conversion unit 42 converts the image (captured by the imaging unit 2) so that the image captured by the imaging unit 2 is mapped to the projection plane selected by the selection unit 41.

  In addition, the computer here is implement | achieved by the image processing part 4 of the imaging device 1 mentioned above, or the image processing unit 40 of the imaging system 100, for example.

DESCRIPTION OF SYMBOLS 1 Imaging device 10 Imaging system 101,102,103,104,105,106 Projection surface 2 Imaging part 3 Detection part 4 Image processing part 40 Image processing unit 41 Selection part 42 Conversion part

Claims (8)

An imaging unit that captures an image;
A detection unit for detecting an inclination of the imaging unit with respect to a vertical direction of the real space;
An image processing unit that performs image processing on an image captured by the imaging unit;
The image processing unit
A selection unit that selects a projection plane based on the inclination detected by the detection unit from a plurality of types of projection planes that are virtual planes set in the real space and have different shapes;
An imaging apparatus comprising: a conversion unit that converts an image captured by the imaging unit so that an image captured by the imaging unit is mapped to the projection plane selected by the selection unit. The selection unit is configured to select, from the plurality of types of projection surfaces, a projection surface having a curved shape along a vertical direction of the real space and within a horizontal plane of the real space. The imaging apparatus according to claim 1. The said detection part is comprised so that the angle which the optical axis of the said imaging part and the perpendicular direction of the said real space make may be included in the said inclination, The detection part of Claim 1 or 2 characterized by the above-mentioned. Imaging device. The detection unit is configured to detect an angle formed by a reference direction which is set for the imaging unit and is perpendicular to the optical axis of the imaging unit and a vertical direction of the real space, included in the inclination. The imaging apparatus according to any one of claims 1 to 3, wherein the imaging apparatus is configured. The selection unit is configured to select a projection plane from the plurality of types of projection planes based on a comparison result between the magnitude of the inclination detected by the detection unit and a predetermined threshold value. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. An imaging unit that captures an image;
A detection unit for detecting an inclination of the imaging unit with respect to a vertical direction of the real space;
An image processing unit that acquires data from the imaging unit and the detection unit and performs image processing on an image captured by the imaging unit;
The image processing unit includes:
A selection unit that selects a projection plane based on the inclination detected by the detection unit from a plurality of types of projection planes that are virtual planes set in the real space and have different shapes;
An imaging system comprising: a conversion unit that converts an image captured by the imaging unit such that an image captured by the imaging unit is mapped to the projection plane selected by the selection unit. A first step of selecting a projection plane based on the inclination of the imaging unit with respect to the vertical direction of the real space from a plurality of types of projection planes that are virtual planes set in real space and have different shapes;
And a second step of converting the image picked up by the image pickup unit so that the image picked up by the image pickup unit is mapped onto the projection plane selected in the first step. Processing method. A computer connected to an imaging unit that captures an image,
A selection unit that selects a projection plane based on an inclination of the imaging unit with respect to a vertical direction of the real space from a plurality of types of projection planes that are virtual planes set in real space and have different shapes from each other,
A program that functions as a conversion unit that converts an image captured by the imaging unit so that an image captured by the imaging unit is mapped to the projection plane selected by the selection unit.

Images