JP2012256274A - Image generation device, image generation method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image generation device which achieves improved computational speed by reducing load on computational processing for generating a plane image of a given area, as a cut-out area, in an omnidirectional image.SOLUTION: An image generation device 10 generates a plane image that corresponds to a cut-out area cut out from an omnidirectional image with distortion and that is projected onto a virtual camera plane. The image generation device 10 includes: an omnidirectional image acquisition unit 11 which acquires an omnidirectional image; a conversion table storage unit 13 which stores therein a conversion table for projecting portions in divided areas of the omnidirectional image onto corresponding intermediate planes; an intermediate plane image generation unit 12 which projects, by using the conversion table, the portions in divided areas of the omnidirectional image onto the respective intermediate planes, so as to generate intermediate plane images of the respective divided areas; a cut-out area specification unit 15 which specifies a cut-out area; and a plane image generation unit 14 which projects intermediate plane images of the cut-out area onto the virtual camera plane, so as to generate a plane image.

Description

  The present invention relates to an image generation apparatus and an image generation method for generating a planar image by extracting a partial range of an omnidirectional image having distortion as an extraction range, and in particular, extracting an arbitrary range in the entire periphery image as an extraction range. The present invention relates to an image generation apparatus and an image generation method for generating a planar image.

  Conventionally, an all-around camera has been used as a monitoring camera or the like. The omnidirectional camera is also referred to as a 360-degree camera, and uses a fisheye lens or a mirror to shoot an omnidirectional image with a horizontal angle of view of 360 degrees and a vertical angle of view of 180 degrees to obtain a circular or donut-shaped image (hereinafter, referred to as “round-angled camera”). "All-around image") can be obtained.

  By correcting the omnidirectional image to a planar image, an image similar to that taken with a normal camera can be obtained. In particular, if the all-around image is developed for the entire circumference, a 360-degree panoramic image can be obtained by one shooting with one camera.

  In addition, a part of the entire surrounding image was cut out and corrected to a flat image, and the cutout range was changed (moved or enlarged / reduced), so that the camera was shot while panning, tilting, and zooming. A plurality of similar planar images (or moving images) are obtained. In this way, the control for changing the cutout range of the all-around image so as to obtain a plurality of planar images obtained by shooting a normal camera while panning, tilting, and zooming is performed using the electronic PTZ ( (pan, tilt, zoom) control.

  Examples of applications that cut out a partial range from the entire surrounding image and change the cut-out range include, for example, those that change the cut-out range manually in a surveillance camera, those that automatically change the cut-out range in a predetermined trajectory, In addition, there is one that detects a moving object from the entire surrounding image and automatically changes the extraction range so as to track the detected moving object.

  FIG. 14A is a diagram illustrating an example of an all-around image, and FIG. 14B is a diagram illustrating a planar image generated by cutting out a partial range PA from the all-around image in FIG. 14A. is there. As shown in FIG. 14A, the omnidirectional image is captured using a fish-eye lens, for example, by an equidistant projection method in which the distance from the center of the omnidirectional image is proportional to the elevation angle. Yes. Therefore, in order to extract the partial range PA from the entire surrounding image and obtain the planar image shown in FIG. 14B, it is necessary to perform distortion correction on the partial range PA from the entire surrounding image (for example, Patent Documents). 1).

  15 and 16 are diagrams for explaining generation of a planar image. FIG. 15A is a virtual hemispherical surface (hereinafter referred to as “virtual hemispherical surface”) VS having the center of the fisheye lens as its center and the focal length as its radius, and a plane for projecting the entire surrounding image. It is a figure which shows the relationship with a certain virtual camera plane VP, FIG.15 (b) is a figure which shows the omnidirectional image projected on image sensor IS. 16 (a) and 16 (b) are planes that pass through the center O of FIG. 15 (a) and intersect perpendicularly to the virtual camera plane VP. It is the figure which cut | disconnected virtual camera plane VP (Hereafter, FIG. 1, 4, 5, 6, 11, 12 is also the same). FIG. 16A shows the virtual camera plane VP, and FIG. 16B shows the virtual camera plane VP ′ when the cutout range is moved.

  The process of cutting out a partial range from the entire surrounding image as a cutout range and performing distortion correction (i.e., projecting onto the virtual camera plane) to generate a planar image requires a large amount of computation and memory access, and has a heavy processing load. Therefore, processing is performed by using a conversion table (look-up table) for generating a flat image by performing distortion correction on the cut-out range of the omnidirectional image (converting the cut-out range of the omnidirectional image into a flat image). It is possible to reduce the load.

  For example, Patent Document 1 discloses that a part of the distortion correction calculation (trigonometric function or the like) is performed using a conversion table. Japanese Patent Application Laid-Open No. 2004-228688 discloses storing a calculation result obtained by performing distortion correction on a specific cutout range of the entire surrounding image as a conversion table.

JP 2000-235645 A JP 2008-31890 A

  When converting from a omnidirectional image to a planar image using a conversion table, the calculation processing load is small and the calculation is fast, but in order to realize an electronic PTZ using an arbitrary range in the omnidirectional image as a cutout range Must hold an infinite number of conversion tables, which is not realistic.

  The present invention provides an image generation apparatus capable of generating a plane image for an arbitrary range in an all-around image and cutting out the calculation processing load and improving the calculation speed. With the goal.

  In order to solve the above-described conventional problems, an image generation apparatus according to the present invention is an image generation apparatus that generates a planar image obtained by projecting a cutout range cut out from an omnidirectional image having distortion onto a virtual camera plane. A omnidirectional image acquisition unit that acquires a peripheral image, and a conversion that stores a conversion table for projecting the omnidirectional image acquired by the omnidirectional image acquisition unit onto a predetermined plane corresponding to each of a plurality of division ranges Using the table storage unit and the conversion table stored in the conversion table storage unit, projecting the all-around image acquired by the all-around image acquiring unit onto the predetermined plane for each division range An intermediate plane image generation unit that generates an intermediate plane image for each of the divided ranges, a cutout range specification unit that specifies the cutout range, and an intermediate plane image of the cutout range as the virtual plane By projecting the camera plane has a configuration in which a flat image generator that generates the planar image.

  According to this configuration, when generating a planar image of an arbitrary cutout range from the entire surrounding image, the entire surrounding image is projected onto a predetermined plane (intermediate plane) for each fixed division range using the conversion table. Then, when the cutout range for generating the planar image is specified, the planar image is generated by projecting the intermediate planar image projected onto the intermediate plane onto the virtual camera plane for generating the planar image. . Here, since the conversion table stored in advance is used for the conversion from the omnidirectional image to the intermediate plane image, the calculation processing load is light and the calculation processing can be speeded up. Further, since the process of projecting the intermediate plane image onto the virtual camera plane is also a projection from the plane to the plane, the projection position can be obtained by multiplication of a fixed value, and the load of the calculation process can also be reduced in this respect.

  In the image generation device, the plane image generation unit may project each point of the intermediate plane image onto a leg of a perpendicular drawn from the point to the virtual camera plane.

  In the image generation device, the plane image generation unit may project each point of the intermediate plane image onto the virtual camera plane so as to enlarge all the points at an equal magnification.

  In the image generation apparatus, the intermediate plane image generation unit may generate an intermediate plane image of only the divided range including at least a part of the cutout range specified by the cutout range specifying unit.

  According to this configuration, since the intermediate plane image is generated only in a range necessary for generating the plane image, it is possible to reduce the amount of arithmetic processing compared with the above-described embodiment, and therefore, the processing load is further increased. This can be reduced and the processing speed can be increased.

  In the above image forming apparatus, the intermediate plane image generation unit may generate the intermediate plane image only for the divided range corresponding to a specified partial region in the omnidirectional image, and the cutout range specification unit May specify the cutout range in the partial area.

  According to this configuration, since the cutout range is specified in the region where the intermediate plane image is generated, the intermediate plane image is generated in advance in that region, and immediately after the cutout range is specified, the plane image generation unit A plane image can be generated at, so that it is possible to increase the speed from the designation of the extraction range to the output of the plane image.

  The image forming apparatus may further include an intermediate plane image storage unit that stores the intermediate plane image generated by the intermediate plane image generation unit, and the plane image generation unit stores the intermediate plane image storage unit. The plane image may be generated by reading out the intermediate plane image stored in the unit and projecting it on the virtual camera plane.

  According to this configuration, since the intermediate plane image is generated and stored in advance before the plane image is generated, the intermediate plane image is generated after an arbitrary clipping range is specified in order to generate the plane image. There is no need, and a plane image can be immediately generated using the intermediate plane image that has already been generated and stored, so that the time required for creating the plane image can be shortened.

  In the image generation apparatus, the planar image generation unit may set the virtual camera plane according to the cutout range specified by the cutout range specifying unit.

  In the image generation apparatus, the planar image generation unit may cause the projection distance at the center of the cutout range specified by the cutout range specifying unit to be shorter than the projection distance at the edge of the cutout range. In addition, the virtual camera plane may be set.

  According to this configuration, it is possible to generate a planar image with little distortion, particularly in the central portion.

  In the image generation apparatus, the planar image generation unit may be parallel to the predetermined plane corresponding to any of the divided ranges included in the cutout range specified by the cutout range specifying unit. The virtual camera plane may be set.

  According to this configuration, it is not necessary to convert the intermediate plane image in the divided range parallel to the virtual camera plane to project it onto the virtual camera plane, and the processing load for generating the plane image can be reduced. In addition, the distortion in the planar image can be reduced for the divided range.

  Further, in the above image generation device, the conversion table storage unit converts the entire range of the omnidirectional image acquired by the omnidirectional image acquisition unit with each of a plurality of different dividing methods. You may remember the table.

  According to this configuration, it is possible to select and use a more appropriately divided conversion table with respect to the position of the cutout range for generating a planar image, and a plurality of distortion distribution patterns in the planar image. You can choose from.

  The image generation method of the present invention is an image generation method for generating a planar image obtained by projecting a cutout range cut out from a distorted omnidirectional image onto a virtual camera plane, and acquiring the omnidirectional image Using the conversion table for projecting the omnidirectional image acquired in the omnidirectional image acquisition step onto a predetermined plane corresponding to each of a plurality of division ranges, and acquired in the omnidirectional image acquisition step. Further, by projecting the entire peripheral image onto the predetermined plane for each of the divided ranges, an intermediate plane image generating step for generating an intermediate plane image for each of the divided ranges, and a cutout range specifying step for specifying the cutout range; And a plane image generation step of generating the plane image by projecting the intermediate plane image of the cutout range onto the virtual camera plane. It is.

  Even with this configuration, when generating a planar image of an arbitrary cutout range from the entire surrounding image, the entire surrounding image is projected onto a predetermined plane (intermediate plane) for each divided range fixed using the conversion table, When a cutout range for generating a planar image is designated, a planar image is generated by projecting the intermediate planar image projected onto the intermediate plane onto the virtual camera plane for generating the planar image. Here, since the conversion table stored in advance is used for the conversion from the omnidirectional image to the intermediate plane image, the calculation processing load is light and the calculation processing can be speeded up. Further, since the process of projecting the intermediate plane image onto the virtual camera plane is also a projection from the plane to the plane, the projection position can be obtained by multiplication of a fixed value, and the load of the calculation process can also be reduced in this respect.

  The present invention generates a planar image by converting an omnidirectional image into an intermediate planar image and converting the intermediate planar image into a planar image. At this time, the conversion from the omnidirectional image to the intermediate planar image is stored in advance. Since the converted conversion table is used, the processing load is light and the processing speed can be increased. Further, since the conversion from the intermediate plane image to the plane image is also a projection from the plane to the plane, the projection position can be obtained by multiplication of a fixed value, and the load of the arithmetic processing can be reduced also in this respect.

The figure for demonstrating the principle which produces | generates the plane image in embodiment of this invention The figure which shows the relationship between the cutting-out range for producing | generating the plane image in embodiment of this invention, and the division | segmentation range for producing | generating an intermediate | middle plane image The block diagram which shows the structure of the image generation apparatus in embodiment of this invention The figure for demonstrating the 1st projection method which projects the intermediate plane image in a virtual camera plane in embodiment of this invention The figure for demonstrating the 1st projection method which projects the intermediate plane image in a virtual camera plane in embodiment of this invention The figure for demonstrating the 2nd projection method which projects the intermediate | middle plane image in embodiment of this invention on a virtual camera plane FIG. 3 is a flowchart of an image generation method executed by the image generation apparatus according to the embodiment of the present invention. The block diagram which shows the structure of the image generation apparatus in the 1st modification of embodiment of this invention. The flowchart of the image generation method performed with the image generation apparatus of the 1st modification of embodiment of this invention The block diagram which shows the structure of the image generation apparatus of the 2nd modification of embodiment of this invention. The figure for demonstrating the 3rd modification of embodiment of this invention. The figure for demonstrating the 4th modification of embodiment of this invention. The figure for demonstrating the 4th modification of embodiment of this invention. (A) The figure which shows the example of an all-around image (b) The figure which shows the plane image produced | generated by cutting out the partial range PA from the all-around image (A) The figure which shows the relationship between a virtual hemisphere and a virtual camera plane (b) The figure which shows the omnidirectional image projected on the image sensor (A) The figure which shows the relationship between a virtual hemisphere and a virtual camera plane (b) The figure which shows the relationship between a virtual hemisphere and a virtual camera plane when a cutting-out range moves

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. First, with reference to FIG. 1 and FIG. 2, a basic principle of generating a planar image in the embodiment of the present invention will be described. In the present embodiment, the planar image refers to an image obtained by projecting a distortion image captured by a fisheye lens or the like onto a virtual camera plane.

  FIG. 1 is a diagram for explaining the principle of generating a planar image in the embodiment of the present invention. FIG. 1 shows a virtual hemispherical surface VS, divided ranges CA11 to CA15, intermediate planes VP11 to VP15 that are planes that project the divided ranges, intermediate plane images RP11 to RP15, a cutout range CA1, and a virtual camera plane VP. The relationship with the plane image FP is shown.

  The planar image FP is generated as follows. In the virtual hemispherical surface VS, division ranges CA11 to CA15 are defined in advance, and when an all-around image is photographed, intermediate plane images RP11 corresponding to the division ranges CA11 to CA15 respectively for the photographed all-around image. ~ RP15 is generated. At this time, the intermediate plane images RP11 to RP15 are generated by projecting the divided ranges CA11 to CA15 of the entire peripheral image onto the intermediate planes VP11 to VP15. The intermediate planes VP11 to VP15 are defined as planes in contact with the virtual hemispherical surface VS at the centers of the cutout ranges CA11 to CA15.

  For the sake of brevity, FIG. 1 shows only CA11 to CA15 for the divided range, VP11 to VP15 for the intermediate plane, and only RP11 to RP15 for the intermediate plane image. The division range and the intermediate plane are set over the entire virtual hemisphere VS, and the intermediate plane image may be generated for all the set division ranges. Hereinafter, a set of intermediate planes is referred to as an intermediate plane group.

  When the virtual camera plane VP for generating the plane image FP is designated, an intermediate plane image including an image portion projected onto the virtual camera plane VP is projected onto the virtual camera plane VP. Then, the intermediate plane images projected onto the virtual camera plane VP are connected to each other to generate a plane image FP.

  FIG. 2A is a diagram of the intermediate plane group viewed from diagonally above, and FIG. 2B is a diagram of the intermediate plane group viewed from directly above. For convenience of explanation, FIG. 1 illustrates an example in which a plurality of divided ranges and intermediate planes are arranged in the horizontal direction. However, since the virtual hemispherical surface VS is actually three-dimensional, the intermediate plane group is a polyhedron. Become. Each intermediate plane may have any shape, but in the present embodiment, as shown in FIGS. 2A and 2B, the intermediate plane is a regular pentagon or a regular hexagon, and these are virtual hemispheres. A hemispherical portion of a truncated icosahedron is arranged on the surface VS.

  In the example of FIG. 2, the plane in contact with the virtual hemisphere at the center of each divided range is the intermediate plane of each divided range. By fixing the division range and the intermediate plane, a conversion table for projecting each division range onto the intermediate plane can be stored in advance as a fixed value.

  When an omnidirectional image is obtained, each divided range in the omnidirectional image is converted by a corresponding conversion table, so that it can be projected onto an intermediate plane, and a light processing load (i.e., conversion equation can be calculated). An intermediate plane image of each divided range can be generated (without determination). When a cutout range for generating a plane image is specified, an intermediate plane image (generally, a plurality) of divided ranges included in the cutout range is projected onto a virtual camera plane (see VP in FIG. 1). These are combined into a planar image.

  As described above, in the present embodiment, an intermediate plane image is generated from the entire surrounding image, and then a necessary intermediate plane image is projected onto the virtual camera plane corresponding to the cutout range to generate a plane image. To do. Since the intermediate plane image is generated by converting the entire surrounding image with the conversion table, the processing load can be reduced. Further, since the range (division range) for generating the intermediate plane image is fixed, it is not necessary to prepare an infinite conversion table. Furthermore, since the process of projecting the intermediate plane image onto the virtual camera plane is a projection from the plane to the plane, the processing load can be reduced. Therefore, the overall processing load for generating a planar image is reduced.

  FIG. 3 is a block diagram showing the configuration of the image generation apparatus according to the embodiment of the present invention. The image generation apparatus 10 includes an omnidirectional image acquisition unit 11, an intermediate plane image generation unit 12, a conversion table storage unit 13, a plane image generation unit 14, a cutout range designation unit 15, and an output unit 16.

  The omnidirectional image acquisition unit 11 acquires an omnidirectional image of the subject. In the present embodiment, the omnidirectional image acquisition unit 11 acquires an omnidirectional image from an external moving image capturing device. This moving image capturing device captures a moving image of a subject and outputs continuous image frames to the omnidirectional image acquisition unit 11. Moreover, the omnidirectional image acquisition part 11 may acquire the still image of the omnidirectional image obtained continuously.

  The omnidirectional image acquisition unit 11 projects a subject image onto the image sensor by an equidistant projection method using a fisheye lens, and acquires an omnidirectional image having a horizontal field angle of 360 degrees and a vertical field angle of 180 degrees. This all-around image is a circular image, and the degree of distortion increases as the distance from the center increases. The omnidirectional image acquired by the omnidirectional image acquisition unit 11 is output to the intermediate plane image generation unit 12.

  The conversion table storage unit 13 stores a conversion table for projecting the omnidirectional image onto a predetermined plane corresponding to each of a plurality of division ranges. Referring to FIG. 1, the conversion table storage unit 13 stores conversion tables for projecting the divided ranges CA11 to CA15 onto the intermediate planes VP11 to VP15, respectively. Here, as described above, the intermediate plane is set as a plane in contact with the virtual hemisphere at the center of the division range. Note that the divided ranges may overlap each other.

  The intermediate plane image generation unit 12 performs coordinate conversion with reference to the conversion table stored in the conversion table storage unit 13 for each divided range of the omnidirectional image acquired by the omnidirectional image acquisition unit 11. Thus, an intermediate plane image is generated. Referring to FIG. 1, the intermediate plane image generation unit 12 generates intermediate plane images RP11 to RP15 for the divided ranges CA11 to CA15, respectively.

  The cutout range designation unit 15 designates a cutout range (a range to be cut out in order to generate a planar image from the entire surrounding image). If it demonstrates with reference to FIG. 1, the extraction range CA1 will be designated by the extraction range designation | designated part 15. FIG. In FIG. 1, the virtual hemispherical surface VS is shown in a plan view. However, since the virtual hemispherical surface VS is actually a solid, the cutout range is actually a two-dimensional range. When the cutout range is a rectangle, the cutout range is specified by four vectors respectively directed to four points on the virtual hemisphere, and the range surrounded by the four vectors is the cutout range.

  When the aspect ratio and orientation of the cutout range are fixed, the cutout range is defined by a vector from the center of the virtual hemisphere to the center of the cutout range and the size parameter of the cutout range. The horizontal angle of this vector corresponds to the pan angle of a normal camera, and the vertical angle of the vector corresponds to the tilt angle of a normal camera. The size of the cut-out range (cut-off angle of view) corresponds to a normal camera zoom. Therefore, the designation of the extraction range is, in other words, designation of pan, tilt, and zoom in the electronic PTZ control.

  The cutout range designation unit 15 may designate a cutout range in accordance with a user operation. In this case, the image generation apparatus 10 is provided with an operation unit, and the user can designate the extraction range by operating the operation unit. The cutout range designation unit 15 may automatically move the cutout range so that the cutout range draws a predetermined trajectory.

  Further, the cut-out range specifying unit 15 may detect a moving object from the entire surrounding image and specify the cut-out range so as to track the moving object. In this case, the image generation apparatus 10 is provided with a motion detection unit that detects a motion in the entire surrounding image, and the extraction range designation unit 15 is notified of the range in which the motion has been detected from the motion detection unit. The range designation unit 15 designates the cutout range so that the range in which movement has occurred is the center.

  Further, the size (zoom magnification) of the cutout range specified by the cutout range specifying unit 15 may be fixed or variable. Further, the cutout range designation unit 15 designates a plurality of vectors directed to a plurality of arbitrary points on the virtual hemisphere when designating the cutout range, so that a rectangle other than a quadrangle or any arbitrary on the virtual hemisphere can be specified. The shape can also be specified as the cutout range.

  The plane image generation unit 14 generates a plane image of the cutout range specified by the cutout range specifying unit 15. For this purpose, the planar image generation unit 14 first sets a virtual camera plane corresponding to the specified cutout range. In the present embodiment, a plane that is in contact with the virtual hemisphere at the center of the specified cutout range is defined as a virtual camera plane. When the virtual camera plane is set, the plane image generation unit 14 projects the intermediate plane image included in the cutout range onto the virtual camera plane.

  The plane image generation unit 14 compares the cutout range with each divided range, selects an intermediate plane image of the divided range that is entirely or partially included in the cutout range, and projects it onto the virtual camera plane. The planar image generation unit 14 projects only the portion included in the cutout range onto the virtual camera plane for the intermediate plane image of the divided range that includes only part of the cutout range. The plane image generation unit 14 connects (synthesizes) the projected intermediate plane images to generate a plane image.

  Referring to FIG. 1, the plane image generation unit 14 projects a part of the intermediate plane images RP11 and RP15 and the entire intermediate plane images RP12 to RP14 included in the cutout range CA1 onto the virtual camera plane VP. Then, a part of the projected intermediate plane images RP11 and RP15 and the whole of the intermediate plane images RP12 to RP14 are connected to generate a plane image FP.

  The method of projecting the intermediate plane image onto the virtual camera plane by the plane image generation unit 14 is not limited to one. Below, two methods are demonstrated as an example.

  FIG. 4 is a diagram for explaining a first projection method for projecting the intermediate plane image onto the virtual camera plane. In the first projection method, each point of the intermediate plane image is projected in a direction perpendicular to the virtual camera plane. Referring to FIG. 4, arbitrary points p41, p42, p43, p44, p45,... On the intermediate plane images RP41, RP42 are perpendicular feet (virtual camera planes) drawn from the respective points to the virtual camera plane VP. (Intersections of perpendicular lines drawn down to VP and virtual camera plane VP) are projected onto P41, P42, P43, P44, P45,.

  FIG. 5 is a diagram showing another example of the first projection method. Arbitrary points p51, p52, p53,... Of the intermediate plane image RP5 are projected onto perpendicular legs P51, P52, P53,... Drawn from the respective points to the virtual camera plane VP. According to the first projection method, since the projection from the intermediate plane image to the virtual camera plane is a projection from the plane to the plane, the projection position can be obtained by multiplication of a fixed value.

  FIG. 6 is a diagram for explaining a second projection method for projecting the intermediate plane image onto the virtual camera plane. In the second projecting method, the intermediate plane image is uniformly enlarged in the entire area and projected onto the virtual camera plane. That is, the intermediate plane image generated by the intermediate plane image generation unit 12 and the intermediate plane image projected onto the virtual camera plane have a similar shape.

  Referring to FIG. 6, arbitrary points p61, p62, p63, p64,... Of the intermediate plane image RP6 are points P61, P62 on the virtual camera plane VP while maintaining the ratio of the distance between them. , P63, P64,... As a result, the ratio of the distances between the points P61, P62, P63, P64,... Is equal to the ratio of the distances between the points p61, p62, p63, p64,.

  The output unit 16 outputs the planar image generated by the planar image generation unit 14. The output unit 16 may be a display unit that displays the generated planar image on a display panel, and may be a recording unit that records the generated planar image on a recording medium. It may be a transmission unit for transmitting via.

  FIG. 7 is a flowchart of an image generation method executed by the image generation apparatus 10. Hereinafter, the image generation method of the present embodiment will be described with reference to FIG. This image generation method may be realized by a computer executing a computer program.

  In the image generation method of the present embodiment, first, an omnidirectional image is acquired by the omnidirectional image acquisition unit 11 (step S71). When the omnidirectional image is acquired, the intermediate plane image generation unit 12 generates an intermediate plane image for each divided range of the omnidirectional image using the conversion table stored in the conversion table storage unit 13 (step S72). ).

  Then, the cutout range designation unit 15 designates the cutout range (step S73). At this time, the cutout range designation unit 15 may designate the cutout range based on the content of the all-around image (for example, by detecting movement from the all-around image), or the cutout range may be set regardless of the all-around image. May be specified.

  Next, the plane image generation unit 14 sets a virtual camera plane corresponding to the specified cutout range (step S74), and projects the intermediate plane image included in the cutout range onto the virtual camera plane (step S75). Then, when there are a plurality of intermediate plane images projected onto the virtual camera plane, the plane image generation unit 14 generates a plane image by combining (joining) them (step S76). The output unit 16 outputs the generated planar image (step S77).

  As described above, according to the image generation apparatus 10 of the present embodiment, when generating a planar image of an arbitrary cutout range from the entire surrounding image, the entire surrounding image is obtained for each divided range fixed using the conversion table. When the cut-out range for generating a plane image is specified, the intermediate plane image projected onto the intermediate plane is projected onto the virtual camera plane for generating the plane image. A plane image is generated. Here, since the conversion table stored in advance is used for the conversion from the omnidirectional image to the intermediate plane image, the calculation processing load is light and the calculation processing can be speeded up. Further, since the process of projecting the intermediate plane image onto the virtual camera plane is also a projection from the plane to the plane, the projection position can be obtained by multiplication of a fixed value, and the load of the calculation process can also be reduced in this respect.

  Although one embodiment of the present invention has been described above, various modifications can be made to the above embodiment. Hereinafter, various modifications will be described.

(First modification)
FIG. 8 is a block diagram illustrating a configuration of the image generation apparatus according to the first modification. In the image generation apparatus 20, the same components as those in the image generation apparatus 10 of the above embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Compared to the image generation apparatus 10 of the above-described embodiment, the image generation apparatus 20 is provided with an intermediate plane image generation area specifying unit 27.

  When the cutout range is specified by the cutout range specifying unit 15, the intermediate plane image generation region specifying unit 27 receives the information and specifies a divided range that is partly or entirely included in the cutout range. The intermediate plane image generation area designating unit 27 also identifies a partial area when the partial area is included in the cutout area. Then, the intermediate plane image generation area designating unit 27 outputs the specified area to the intermediate plane image generation section 12.

  The intermediate plane image generation unit 12 generates an intermediate plane image only for the region specified by the intermediate plane image generation region specification unit 27 and outputs the intermediate plane image to the plane image generation unit 14. The plane image generation unit 14 refers to the range specified by the cutout range specification unit 15 and synthesizes (connects) the intermediate plane images generated by the intermediate plane image generation unit 12 to generate the plane image. Generate.

  FIG. 9 is a flowchart of an image generation method executed by the image generation apparatus 20 according to the first modification. In the flowchart of FIG. 9, the same processing as the processing of the image generation method of the above embodiment is given the same step number, and detailed description is omitted. In the image generation method of the first modified example, when an all-around image is acquired (step S71), before the intermediate plane image is generated, first, the cutout range specifying unit 15 generates a plane image. A cut-out range is designated (step S73).

  Next, the intermediate plane image generation area specifying unit 27 specifies an area for generating an intermediate plane image based on the cutout range specified by the cutout range specifying unit 15 (step S98). Then, the intermediate plane image generation unit 12 generates an intermediate plane image for each divided range in the intermediate plane image generation region specified by the intermediate plane image generation region specification unit 27 (step S92). The subsequent processing is the same as the image generation method of the above-described embodiment described with reference to FIG.

  According to the first modified example, since the intermediate plane image is generated only in a range necessary for generating the plane image, the amount of arithmetic processing can be reduced as compared with the above-described embodiment, and thus the processing is performed. The load can be further reduced and the processing speed can be increased.

  In the first modification described above, the intermediate plane image generation area specifying unit 27 generates an intermediate plane image within the cutout range based on the cutout range specified by the cutout range specifying unit 15. In contrast, the intermediate plane image generation area is designated, but conversely, the intermediate plane image generation area designating unit 27 first designates a division range in which the intermediate plane image is generated, and the cutout range designation unit 15 is used to designate the area including the division range. From this, it is possible to specify the cutout range, and the intermediate plane image generation unit 12 may generate an intermediate plane image for this divided range.

  In this case, in the omnidirectional image, the region specified by the intermediate plane image generation region specifying unit 27 (the region where the intermediate plane image is generated by the intermediate plane image generation unit 12) is displayed as the plane image generation possible range. The user designates a cutout range from the plane image generation possible range. According to this configuration, since the intermediate plane image is generated in advance as in the above-described embodiment, the plane image is immediately synthesized by the plane image generation unit 14 as soon as the cutout range is specified. The speed from the range specification to the output of the planar image can be increased, and the processing load can be reduced by the first modification described above.

(Second modification)
FIG. 10 is a block diagram illustrating a configuration of an image generation apparatus according to a second modification. In the image generation apparatus 30, the same components as those in the image generation apparatus 10 of the above embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Compared with the image generation apparatus 10 of the above-described embodiment, the image generation apparatus 30 has an intermediate plane image storage unit 38 added thereto.

  As described above, the omnidirectional image acquisition unit 11 acquires image frames of successive omnidirectional images that constitute a moving image. In the second modification, every time the image frame of the omnidirectional image is acquired by the omnidirectional image acquisition unit 11, when it is not necessary to immediately generate the plane image, that is, the image frame of the omnidirectional image is acquired. This is effective when a sufficient time can be secured between the time of shooting and the time of monitoring when the flat image is generated.

  At the time of shooting, when the omnidirectional image acquisition unit 11 acquires an image frame of the omnidirectional image, the intermediate plane image generation unit 12 generates an intermediate plane image for the image frame and stores it in the intermediate plane image storage unit 38. Remember. When the omnidirectional image acquisition unit 11 acquires the next image frame, the intermediate plane image generation unit 12 also generates an intermediate plane image and stores it in the intermediate plane image storage unit 38.

  In this way, the intermediate plane image 12 generates an intermediate plane image of the all-around image every time an image frame of the all-around image is input from the all-around image acquisition unit 11, and stores it in the intermediate plane image storage unit 38. Remember. At this time, each intermediate plane image is stored in the intermediate plane image storage unit 38 in association with information for specifying the image frame. The intermediate plane image storage unit 38 also stores an image frame of the all-around image. The intermediate plane image generation unit 12 may generate an intermediate plane image for the entire area of the entire surrounding image. Further, the image frame of the all-around image in which the intermediate plane image is stored in the intermediate plane image storage unit 38 may be stored in a storage area different from that of the intermediate plane image storage unit 38.

  At the time of monitoring, for each image frame in which the intermediate plane image is stored in the intermediate plane image storage unit 38, the extraction range is specified by the extraction range specification unit 15. When the cutout range is designated by the cutout range designation unit 15, the planar image generation unit 14 sets a virtual camera plane. Then, the plane image generation unit 14 reads out the necessary intermediate plane image in the corresponding image frame from the intermediate plane image storage unit 38, projects them onto the set virtual camera plane, and connects them to each other, thereby A plane image cut out from the entire image of the frame is generated.

  As described above, according to the image generation device of the second modification example, when it is not necessary to generate a planar image in real time for a captured entire surrounding image, each image frame is prior to monitoring. By generating an intermediate plane image in advance, a plane image can be generated immediately when an arbitrary cutout range is specified during monitoring, so that the time required for creating the plane image can be shortened.

(Third Modification)
FIG. 11 is a diagram for explaining the third modification. In the above embodiment, when a cutout range for generating a flat image is specified by the cutout range specifying unit 15, the flat image generation unit 14 sets a virtual camera plane. At this time, the flat image generation is performed. The unit 14 sets a plane in contact with the virtual hemisphere at the center of the cutout range as a virtual camera plane. In FIG. 11, a virtual camera plane VP11 indicated by a broken line is a virtual camera plane set in this way.

  However, when the virtual camera plane is set as described above, the virtual camera plane generally does not overlap with any intermediate plane (not parallel). In the example of FIG. 11, the virtual camera plane VP11 does not overlap with any of the associated intermediate planes VP111 and VP112. In such a case, in order to generate a plane image, both the intermediate plane image RP111 projected onto the intermediate plane VP111 and the intermediate plane image RP112 projected onto the intermediate plane VP112 must be projected onto the virtual camera plane VP11. I must.

  Therefore, in the third modification, when the plane image generation unit 14 sets a virtual camera plane for generating a plane image in the cutout range, the cutout range of the intermediate plane images included in the cutout range is set. The virtual camera plane is set so as to be parallel to the intermediate plane of the intermediate plane image including the center. Referring to FIG. 11, the plane image generation unit 14 is configured so that the virtual camera plane is parallel to the intermediate plane VP112 of the intermediate plane image RP112 including the center of the cutout range CA11 specified by the cutout range specifying unit 15. Set VP11 '.

  By doing in this way, it is not necessary to perform coordinate conversion processing for projecting the intermediate plane image of the intermediate plane parallel to the virtual camera plane onto the virtual camera plane, and the processing load can be reduced.

(Fourth modification)
12 and 13 are diagrams for explaining a fourth modification. In the above embodiment, the intermediate plane group is a set of a plurality of regular pentagons and regular hexagons forming the hemispherical portion of the truncated icosahedron, and the conversion table storage unit 13 includes such a Intermediate plane groups and division ranges corresponding to these intermediate plane groups are set, and a conversion table for converting a corresponding portion of the entire peripheral image into an intermediate plane image is stored for each division range. In FIG. 12, the solid line indicates the division range related to this conversion table.

  However, when the center O of the cutout range for generating a flat image is close to the cut of the divided range as in the cutout range CA12 shown in FIG. 12, the image quality of the central portion of the flat image is deteriorated. Therefore, in the fourth modified example, the conversion table storage unit 13 includes two types of conversion tables, that is, a conversion table related to the intermediate plane group indicated by a solid line in FIG. 12 and a conversion table for the intermediate plane group indicated by a broken line in FIG. Store the conversion table. FIG. 13 is a diagram three-dimensionally showing two types of intermediate plane groups corresponding to two types of conversion tables.

  When the cutout range is specified by the cutout range specifying unit 15, the intermediate plane image generation unit 12 selects a conversion table that has no break at the center of the specified cutout range from the two types of stored conversion tables. Then, an intermediate plane image is generated. In the example of FIG. 12, when the cutout area CA12 is designated, the intermediate plane image generation unit 12 does not extract the solid intermediate plane group conversion table in which the cut of the intermediate plane image comes to the center of the cutout area CA12. A conversion table of a broken-line intermediate plane group having no breaks in the intermediate plane image at the center of the range CA12 is selected.

  Specifically, when the cut-out range is designated, the intermediate plane image generation unit 12 obtains a center-to-center distance between the center of the cut-out range and the center of the intermediate plane including the center of the cut-out range. Among the intermediate plane groups, the conversion table of the intermediate plane group having the smallest center distance is selected. In the example of FIG. 12, the distance d2 between the center O of the cutout range CA12 and the center O2 of the intermediate plane RP2 of the middle plane group indicated by the broken line including the center O of the cutout range CA12 is the center O of the cutout range CA12 and its center O2. Since the distance d1 between the solid plane intermediate plane group including the center O of the cutout range CA12 and the center O1 of the intermediate plane RP1 is smaller, the conversion table of the dashed intermediate plane group is selected. Note that the types of conversion tables are not limited to two, and may be three or more.

(Other variations)
In the above embodiment, when only a part of the intermediate plane image is included in the cutout range, the flat image generation unit 14 projects only the part of the intermediate plane image included in the cutout range onto the virtual camera plane. Then, the plane image of the cutout range is generated by combining them, but the plane image generation unit 14 also performs the whole of the intermediate plane image even when only a part of the intermediate plane image is included in the cutout range. After projecting the image onto the virtual camera plane and synthesizing them, an unnecessary peripheral portion may be trimmed according to the cutout range to generate a cutout plane image.

  As described above, according to the present invention, it is possible to reduce the load of calculation processing for generating a planar image by extracting an arbitrary range in the entire surrounding image as the extraction range. This is useful as an image generation device that generates a planar image by cutting out as a cutout range.

DESCRIPTION OF SYMBOLS 10, 20, 30 Image generation apparatus 11 Whole periphery image acquisition part 12 Intermediate plane image generation part 13 Conversion table memory | storage part 14 Plane image generation part 15 Cutout range designation | designated part 16 Output part 27 Intermediate plane image generation area designation part 38 Intermediate plane image Memory

Claims (12)

An image generation device that generates a planar image obtained by projecting a cutout range cut out from an omnidirectional image having distortion onto a virtual camera plane,
An omnidirectional image acquisition unit for acquiring the omnidirectional image;
A conversion table storage unit that stores a conversion table for projecting the omnidirectional image acquired by the omnidirectional image acquisition unit onto a predetermined plane corresponding to each of a plurality of division ranges;
By using the conversion table stored in the conversion table storage unit, projecting the omnidirectional image acquired by the omnidirectional image acquisition unit to the predetermined plane for each of the divided ranges, the divided range An intermediate plane image generator for generating an intermediate plane image for each
A cutout range designating unit for designating the cutout range;
By projecting the intermediate plane image of the cutout range onto the virtual camera plane, a plane image generation unit that generates the plane image;
An image generation apparatus comprising:   The image generation apparatus according to claim 1, wherein the plane image generation unit projects each point of the intermediate plane image onto a leg of a perpendicular drawn from the point to the virtual camera plane.   The image generation apparatus according to claim 1, wherein the planar image generation unit projects each point of the intermediate planar image onto the virtual camera plane so as to be enlarged at an equal magnification.   The intermediate plane image generation unit generates an intermediate plane image of only the divided range including at least a part of the cutout range specified by the cutout range specifying unit. An image generation apparatus according to claim 1. The intermediate plane image generation unit generates the intermediate plane image only for the divided range corresponding to a specified partial region in the entire surrounding image,
The image generation apparatus according to any one of claims 1 to 3, wherein the cut-out range specifying unit specifies the cut-out range in the partial area. An intermediate plane image storage unit for storing the intermediate plane image generated by the intermediate plane image generation unit;
The plane image generation unit generates the plane image by reading out the intermediate plane image stored in the intermediate plane image storage unit and projecting the intermediate plane image onto the virtual camera plane. The image generation apparatus as described in any one of Claims.   The image generation according to claim 1, wherein the planar image generation unit sets the virtual camera plane according to the cutout range specified by the cutout range specifying unit. apparatus.   The planar image generation unit sets the virtual camera plane so that a projection distance at a center portion of the cutout range specified by the cutout range specification unit is shorter than a projection distance at an edge portion of the cutout range. The image generating apparatus according to claim 7.   The planar image generation unit sets the virtual camera plane so as to be parallel to the predetermined plane corresponding to any division range included in the cutout range specified by the cutout range specifying unit. The image generation apparatus according to claim 7.   The conversion table storage unit stores the conversion table for each of cases where the omnidirectional image acquired by the omnidirectional image acquisition unit is divided by a plurality of different dividing methods. Item 10. The image generation device according to any one of Items 1 to 9. An image generation method for generating a plane image obtained by projecting a cutout range cut out from an omnidirectional image having distortion onto a virtual camera plane,
An all-around image obtaining step for obtaining the all-around image;
Using the conversion table for projecting the omnidirectional image acquired in the omnidirectional image acquisition step to a predetermined plane corresponding to each of a plurality of division ranges, the acquired in the omnidirectional image acquisition step An intermediate plane image generation step for generating an intermediate plane image for each of the divided ranges by projecting the entire surrounding image onto the predetermined plane for each of the divided ranges;
A cutting range specifying step for specifying the cutting range;
A plane image generation step of generating the plane image by projecting the intermediate plane image of the cutout range onto the virtual camera plane;
An image generation method comprising:   A computer program for causing a computer to execute the image generation method according to claim 11.

Images