JP2008028606A - Imaging device and imaging system for panoramically expanded image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To photograph a 360° panoramic expanded image without a break of a subject. <P>SOLUTION: An imaging device 1 for panoramic expanded image generates the 360° panoramic expanded image by imaging. Then the imaging device has a memory 41 which stores positions or sizes of an inner circle boundary 42 and an outer circle boundary 43 defining the range of a ring shape to be used to generate the 360° panoramic expanded image in picked-up images generated by imaging means 27 and 36 through imaging by a fish-eye lens 22, a user updating means 55 of updating the position or size of at least one of the inner circle boundary 42 and outer circle boundary 43 stored in the memory 41 through user's operation, and a panoramic expanded image generating means 52 of generating the 360° panoramic expanded image based upon an image within the range of the ring shape sandwiched between the inner circle boundary 42 and outer circle boundary 43 stored in the memory 41 in the picked-up images generated by the imaging means 27 and 36. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a panorama developed image imaging apparatus and a panorama developed image imaging system.

  Patent Document 1 discloses an omnidirectional photographing apparatus. The omnidirectional photographing device develops and displays or records a video photographed by one camera module having a lens capable of directly taking a 360-degree annular video.

JP 2006-148787 A (summary, detailed description of the invention, drawings, especially paragraph 0024, paragraph 0049, FIG. 8, FIG. 9, etc.)

  The lens that directly captures a 360-degree annular image in Patent Document 1 is a special convex glass lens, specifically, a zenith reflecting surface and a substantially annular lens surface formed so as to surround the periphery. And a substantially annular rear reflection surface disposed below the lens surface, and a transmission surface surrounded by the rear reflection surface. Patent Document 1 suggests that a fish-eye lens can be used instead of this special convex glass lens.

  However, in the omnidirectional imaging device of Patent Document 1, there is a limit to the image height that can be captured in a 360-degree annular image. In this annular image, the upper portion of the lens surface is refracted by the lower portion of the lens surface, reflected from the outer peripheral portion of the back reflecting surface, and further incident from the direction reflected by the outer peripheral portion of the zenith reflecting surface. It is possible to obtain only an image in the range of angle of view that is transmitted and reflected by the inner peripheral part of the back-side reflecting surface and further incident light from the direction reflected by the inner peripheral part of the zenith reflecting surface. At the center of the ring image, the zenith reflecting surface is photographed in a circle. Since the omnidirectional imaging apparatus of Patent Document 1 uses such a special convex glass lens, the image height that can be captured is limited.

  Since the image height is limited, the omnidirectional imaging device of Patent Document 1 has a practical limit on the distance between the glass lens and the subject. For example, if the subject gets too close to the glass lens, the head of the subject will be cut off in a 360-degree annular image. In order to prevent the subject's head from being cut off, the distance of the glass lens to the subject must be secured.

  In particular, when the omnidirectional photographing apparatus is placed in the center of the table and an attempt is made to capture a conference scene or the like, this practical distance limitation greatly impairs the convenience of the omnidirectional photographing apparatus. For example, in a small meeting room, the distance of the glass lens with respect to the subject cannot be secured, and there is a possibility that photographing cannot be performed so that the head of the subject is not cut off.

  It is an object of the present invention to obtain a panorama developed image imaging apparatus and a panorama developed image imaging system that can capture a 360 ° panorama developed image without cutting off the subject.

  The panoramic expanded image imaging apparatus according to the present invention generates a 360-degree panoramic expanded image by imaging. The panoramic developed image imaging device includes a fish-eye lens, an imaging unit that forms a circular image with the fish-eye lens on a light-receiving surface on which a plurality of light-receiving elements are arranged, and generates captured image data having the circular image; Among them, a memory for storing the position or size of the inner circle boundary and the outer circle boundary that defines the range of the ring shape used for generating the 360 ° panoramic development image, and the memory based on the user's operation It is sandwiched between the inner circle boundary and the outer circle boundary stored in the memory in the captured image generated by the imaging means and the user update means that updates the position or size of at least one of the inner circle boundary and the outer circle boundary Panorama unfolded image generating means for generating a panorama unfolded image of 360 degrees based on an image within the range of the ring shape.

  If this configuration is adopted, a circular image formed by a fisheye lens is formed on the imaging means. The imaging unit generates a captured image having a circular image. The circular image is included without the subject's head being cut off.

  Further, the panorama developed image generating means generates a 360 ° panorama developed image based on an image within a ring shape range sandwiched between the inner circle boundary and the outer circle boundary in the captured image. The positions and sizes of the inner circle boundary and the outer circle boundary can be updated by the user update unit based on the user's operation. Therefore, for example, even if it is not possible to physically secure the distance of the fisheye lens to the subject, the subject in the circular image can be changed into a ring shape by updating the position of the inner circle boundary or the outer circle boundary by a user operation. It can be adjusted so as to be within the range. As a result, it is possible to generate a 360-degree panoramic developed image in which the head of the subject is not cut off.

  The panoramic developed image capturing apparatus according to the present invention has the following features in addition to the configuration of the invention described above. That is, the fisheye lens is of a three-dimensional projection system. The memory stores an image height correction table for correcting the balance of the imaging size of the subject obtained based on the relationship between the incident angle of the stereoscopic lens fisheye lens and the image height. The panoramic developed image generation means refers to the image height correction table and generates a 360-degree panoramic developed image in which the balance of the imaging size of the subject is adjusted.

  If this configuration is adopted, the peripheral portion in the image formed by the fisheye lens is less distorted and the amount of information in the peripheral portion is larger than when a general equidistant projection type fisheye lens is used. . Therefore, when the panoramic developed image pickup device is placed on the table with the fisheye lens facing upward, it is possible to obtain an image of a person, a blackboard, or the like around it with high resolution. Blackboard characters and the like can be reproduced in a 360-degree panoramic image. Moreover, a well-balanced image can be obtained as an image of a person or blackboard around the fisheye lens.

  The panoramic developed image capturing apparatus according to the present invention has the following characteristics in addition to the above-described components of the present invention. That is, the user update means updates the image height correction table stored in the memory after updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory, and the updated inner circle boundary and Updating is performed based on the outer circle boundary and the relationship between the angle of incidence and the image height of the stereoscopic lens fisheye lens.

  When the interval between the inner circle boundary and the outer circle boundary changes due to the change, the division ratio of the plurality of pixels in the image space between them changes. Even if only the positions of the inner circle boundary and the outer circle boundary move, the division ratio of the plurality of pixels in the image space between them changes. If this configuration is adopted, the image height correction table can be kept compatible with these.

  The panoramic developed image capturing apparatus according to the present invention has the following characteristics in addition to the above-described components of the present invention. That is, the user update means stores the change in the inner circle boundary or the outer circle boundary in the memory based on the change instruction of the inner circle boundary or the outer circle boundary when the size after the change of the inner circle boundary or the outer circle boundary is a radius greater than a predetermined number of pixels. Update inner circle boundary or outer circle boundary.

  For example, if the radius of the inner circle boundary becomes one pixel, the same pixel in the captured image is used many times in order to obtain the pixel values of the plurality of display pixels at the uppermost stage of the 360 ° panorama development image. Become. In this case, the generation load of the 360-degree panorama development image increases without improving the image quality of the 360-degree panorama development image. By preventing the radius of the inner circle boundary or outer circle boundary from becoming smaller than a predetermined number of pixels as in this configuration, it is possible to prevent such disadvantages from occurring.

  The panoramic developed image capturing apparatus according to the present invention has the following characteristics in addition to the above-described components of the present invention. That is, the panorama development image generation means calculates the distance and direction from the center of the circular image of each display pixel of the 360 degree panorama development image, and uses the calculated distance and direction to capture the captured image of each display pixel. And the pixel value of the pixel of the captured image located at each calculated coordinate is acquired as the pixel value of each display pixel.

  By adopting this configuration, it is possible to generate a 360-degree panoramic unfolded image using the pixel values of the circular image pixels in the captured image as they are. It is not necessary to obtain the pixel value of each display pixel of the 360 degree panoramic image by calculation. As in the case of obtaining the pixel value of each display pixel by calculation, image quality deterioration due to the calculation does not occur, and the generation time of one panoramic developed image can be shortened. As a result, a 360-degree panoramic image can be generated in a short time. In addition, by continuously repeating the panorama development image generation process, it is possible to obtain a moving image with continuous motion using a 360-degree panorama development image.

  The panoramic developed image capturing apparatus according to the present invention has the following characteristics in addition to the above-described components of the present invention. In other words, the panoramic developed image generation unit divides the range between the inner circle boundary and the outer circle boundary in the captured image into two ranges every 180 degrees, and generates two horizontally long images for each divided image range. A panorama divided development image is generated. In addition, there is provided display data generating means for generating display data for displaying two horizontally long panoramic divided developed images side by side.

  By adopting this configuration, it is possible to effectively use the display screen of the display means for displaying the display data, for example, as compared with the case where these are one developed image. A 360-degree panoramic developed image can be displayed large on the display means.

  A panoramic unfolded image capturing system according to the present invention includes a panoramic unfolded image capturing apparatus according to each configuration of the invention described above, a computer device that displays or stores a panorama unfolded image generated by the panoramic unfolded image capturing apparatus, and It is what has.

  If this configuration is adopted, a 360-degree panoramic image can be displayed or saved.

  In the present invention, a 360-degree panoramic image can be taken without the subject being cut off.

  Hereinafter, a panorama developed image imaging apparatus and a panorama developed image imaging system according to embodiments of the present invention will be described with reference to the drawings. The panorama development image capturing apparatus will be described by taking a conference 360-degree video camera apparatus as an example. The panorama development image capturing system will be described by taking an example in which a 360-degree video camera device and a personal computer are connected.

  FIG. 1 is a perspective view showing a panoramic developed image imaging system according to an embodiment of the present invention. The panoramic developed image imaging system includes a 360-degree video camera device 1 as a panoramic developed image imaging device and a PC (personal computer) 3 as a computer device. The 360-degree video camera device 1 is connected to the PC 3 by a USB (Universal Serial Bus) cable 2.

  The PC 3 includes a USB connector 11, a liquid crystal display device 12, a speaker 13, an input device 14, and the like. Further, a CPU (Central Processing Unit) (not shown) reads a program from a storage device (not shown) and executes the program, so that the PC 3 has a communication processing unit 15, a reproduction processing unit 16, and a command generation unit 17. Etc. are realized.

  The communication processing unit 15 controls data communication using the USB connector 11. The reproduction processing unit 16 controls the content displayed on the liquid crystal display device 12 and causes the speaker 13 to output sound. The command generation unit 17 generates a command based on input data input from the input device 14. Examples of the input device 14 include a keyboard and a pointing device.

  The 360 degree video camera device 1 has a cubic-shaped housing 21. The housing 21 is provided with a fisheye lens 22, a USB connector 23, a video output connector 24, an audio output connector 25, and the like. The fisheye lens 22 is disposed on the upper surface of the housing 21. A vent hole 20 for the microphone 26 is formed on the upper surface of the housing 21. The USB connector 23, the video output connector 24, and the audio output connector 25 are disposed on the side surface of the housing 21.

  FIG. 2 is a block diagram showing an electric circuit of the 360 degree video camera apparatus 1 in FIG. An electric circuit for generating a 360-degree panoramic developed image by imaging is incorporated in the housing 21 of the 360-degree video camera device 1.

  In addition to the fish-eye lens 22, the USB connector 23, the video output connector 24, and the audio output connector 25, the 360-degree video camera apparatus 1 includes an image sensor 27, an FPGA (Field Programmable Gate Array) 28, and a DSP (DSP) as a part of the imaging unit. It has a digital signal processor (Digital Signal Processor) 29, an external memory 30, an audio IC (Integrated Circuit) 31, a video encoder 32, and the like.

  FIG. 3 is an explanatory diagram showing an optical arrangement of the fisheye lens 22 and the image sensor 27.

  The image sensor 27 includes, for example, a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The image sensor 27 has a light receiving surface 33. On the light receiving surface 33, a plurality of light receiving elements (not shown) are arranged in a matrix at a ratio of, for example, 3: 4 in length and width. Each light receiving element outputs a value corresponding to the amount of received light. The image sensor 27 generates luminance distribution data having a plurality of received light amount values output from the plurality of light receiving elements. The image sensor 27 generates luminance distribution data at a predetermined cycle.

  The fish-eye lens 22 has a wide viewing angle of, for example, 180 degrees or more, and is a stereoscopic projection system. The three-dimensional projection type fisheye lens 22 is less distorted in the peripheral part of the image formed by the image, and the amount of information in the peripheral part is larger than that of a general equidistant projection type fisheye lens. As a projection method of the fisheye lens 22, an equi-stereo projection method, an orthographic projection method, or the like can be adopted in addition to the stereoscopic projection method and the equidistant projection method. However, when the 360-degree video camera device 1 is placed on a table in a conference room with the fisheye lens 22 facing upward, the amount of information such as a person or blackboard that appears in the periphery increases, so that the stereoscopic projection method is used. Is the best.

  The fisheye lens 22 is disposed above the light receiving surface 33 of the image sensor 27. Thereby, a circular image (hereinafter referred to as a circular image) by the fisheye lens 22 is formed on the light receiving surface 33 of the image sensor 27. The image sensor 27 periodically generates luminance distribution data and outputs it to the FPGA 28. In the FPGA 28, a color conversion processing unit 36 as a part of the imaging unit is realized.

  The color conversion processing unit 36 replaces the data of each pixel of the luminance distribution data using a color conversion table (not shown). The color conversion processing unit 36 replaces, for example, pixel data in a circular image with predetermined color data using the pixel value and the peripheral pixel values in the luminance distribution data. Thereby, captured image data having appropriate color data is generated.

  FIG. 4 is a diagram illustrating an example of a captured image generated by the color conversion processing unit 36 based on the luminance distribution data of the image sensor 27. As shown in FIG. 4, the captured image has a circular image at the center thereof. An image of the subject is formed inside the circular image. The color conversion processing unit 36 outputs the generated captured image data to the DSP 29.

  The microphone 26 generates a waveform signal corresponding to the sound. The waveform signal is converted into an audio signal by the audio IC 31 and supplied to the audio output connector 25. For example, a speaker unit or headphones can be connected to the audio output connector 25. Audio can be heard through a speaker unit, headphones, or the like connected to the audio output connector 25. In addition, a voice storage processing unit 37 is realized in the voice IC 31. The sound storage processing unit 37 samples the waveform signal supplied from the microphone 26 and stores the sound data 38 generated by the sampling in the external memory 30.

  The DSP 29 includes an EEPROM (Electrically Erasable Programmable Read-Only Memory) 41 as a memory. The EEPROM 41 stores inner circle boundary data 42, outer circle boundary data 43, an incident angle image height table 44, an image height correction table 45, a direction table 46, and the like.

  FIG. 5 is an explanatory diagram showing an image in which a circular mark 47 indicating the position of the inner circle boundary and a circular mark 48 indicating the position of the outer circle boundary are superimposed on the captured image. The inner circle boundary and the outer circle boundary are boundaries that specify an image range to be developed as a panorama developed image. An image portion between the inner circle boundary and the outer circle boundary is developed as a panoramic development image. The inner circle boundary data 42 is data indicating the position and size of the inner circle boundary in the captured image. The outer circle boundary data 43 is data indicating the position and size of the outer circle boundary in the captured image.

  FIG. 6 is an image height (field angle difference) characteristic diagram of the fisheye lens 22 of the stereoscopic projection method. The horizontal axis is the relative field angle when the optical axis direction of the fisheye lens 22 is set to 0 degree, and the vertical axis is the image height (field angle difference). 6 is used, for example, the image height (angle of view difference) of the subject existing in the direction of the angle of view of the fisheye lens 22 is about 0.1. Then, the image height (angle of view difference) of the subject existing in the direction of the angle of view of 90 degrees is about 0.2. This means that the subject in the peripheral direction of the fisheye lens 22 forms an image with a size about twice that of the subject in the central direction. Further, when the subject is photographed from the peripheral part to the central part, it means that the balance is lost. The incident angle image height table 44 is data of image height (view angle difference) characteristics with respect to the view angle of the fisheye lens 22.

  When a fisheye lens is used, an image of one subject is generally distorted as it goes to the periphery. A three-dimensional projection method exists as a method for reducing the degree of distortion and increasing the amount of information in the vicinity. When this stereoscopic projection type fish-eye lens 22 is used, the subject is imaged more greatly when imaged at the peripheral part than when imaged at the central part in terms of angle of view. The subject imaged in the central part forms an image with a smaller angle of view than the subject imaged in the peripheral part. This is because, as shown in FIG. 6, the four light receiving elements near the center of the image sensor 27 and the four light receiving elements near the peripheral part, that is, a portion near the angle of view of the image sensor 27, that is, the zenith. This means that an image with a wide angle of view is formed on the nearby light receiving elements, compared with a portion close to the angle of view of 90 degrees of the image sensor 27, that is, the light receiving element in the lateral direction.

  As described above, the stereoscopic projection method has an image with less blind spots and less distortion than the conventional equidistant projection fish-eye lens because there is more peripheral information. However, distortion is reduced, but not zero. On the other hand, in the panorama developed image, it is desirable that the subject is displayed with a balance of the original size. The image height correction table 45 stores correction data for obtaining a developed image in which a subject appears as an original balance from an image formed on the image sensor 27 by the fisheye lens 22 of the three-dimensional projection method.

  The DSP 29 has a CPU (not shown). When the CPU executes the program, the DSP 29 generates a still image storage processing unit 51, a decompressed still image generating unit 52 as a panorama expanded image generating unit, a compressed still image generating unit 53, and streaming generation as a display data generating unit. A unit 54, a processing management unit 55 as a user updating unit, a communication processing unit 56, and the like are realized.

  Further, an external memory 30 is connected to the DSP 29. The external memory 30 is configured by a storage device such as SRAM (Static RAM) or DDR-SDRAM (Double Data Rate SDRAM), and is accessible to the CPU of the DSP 29.

  The external memory 30 stores still image data 39, audio data 38, two compressed and expanded still image data 61 and 62, and the like. The external memory 30 has a first VRAM (Video RAM) area 63 and a second VRAM area 64. One of the two compressed and decompressed still image data 61 and 62 is stored in the first VRAM area 63, and the other is stored in the second VRAM area 64. In FIG. 2, the first VRAM area 63 stores n−1 compressed decompressed still image data 61, and the second VRAM area 64 stores nth compressed expanded still image data 62.

  When new captured image data is supplied from the color conversion processing unit 36, the still image storage processing unit 51 realized in the DSP 29 stores the captured image data as still image data 39 in the external memory 30.

  The expanded still image generation unit 52 generates expanded still image data from the still image data 39 stored in the external memory 30. The developed still image data is image data obtained by expanding a circular image in the captured image into a 360-degree panoramic image. The developed still image generating unit 52 generates and stores expanded still image data alternately in the first VRAM area 63 and the second VRAM area 64.

  The compressed still image generation unit 53 compresses the expanded still image data stored in the first VRAM area 63 or the second VRAM area 64. For compressing the developed still image data, for example, an image compression method such as a JPEG (Joint Photographic Coding Experts Group) compression method or an MPEG (Moving Picture Coding Experts Group) compression method may be used. As a result, the compressed expanded still image data 61 is stored in the first VRAM area 63, and the compressed expanded still image data 62 is stored in the second VRAM area 64.

  The streaming generation unit 54 reads the compressed and expanded still image data 61 and 62 and the audio data 38 from the external memory 30, and generates streaming data having these content data. As the streaming data, a streaming format such as an MPEG system or a div-X (divix) system may be adopted.

  The communication processing unit 56 controls data communication using the USB connector 23.

  The process management unit 55 manages the execution of the still image storage processing unit 51, the developed still image generation unit 52, the compressed still image generation unit 53, the streaming generation unit 54, the communication processing unit 56, and the like. The process management unit 55 instructs the still image storage processing unit 51, the developed still image generation unit 52, the compressed still image generation unit 53, the streaming generation unit 54, the communication processing unit 56, and the like to start or stop them.

  The video encoder 32 reads the compressed decompressed still image data 61 and 62 from the external memory 30 and generates a video signal. Video signals include, for example, those of NTSC (National TV Standards Committee) system and PAL (Phase Alternating Line) system. The video encoder 32 outputs the generated video signal to the video output connector 24. A television receiver or the like can be connected to the video output connector 24. With a television receiver or the like connected to the video output connector 24, the video signal can be reproduced and the video can be viewed.

  Next, the operation of the panoramic developed image imaging system having the above configuration will be described.

  When power is supplied to the 360-degree video camera device 1 from, for example, the USB cable 2, the DPS 29 and the like operate, and the 360-degree video camera device 1 includes a color conversion processing unit 36, an audio storage processing unit 37, and a still image storage processing unit. 51, a developed still image generation unit 52, a compressed still image generation unit 53, a streaming generation unit 54, a process management unit 55, and the like are realized. The process management unit 55 instructs the still image storage processing unit 51, the developed still image generation unit 52, the compressed still image generation unit 53, and the streaming generation unit 54 to start them.

  On the image sensor 27 of the 360-degree video camera device 1, an image formed by the light collected by the fisheye lens 22 is formed. The image sensor 27 generates luminance distribution data including the luminance distribution of the circular image. The color conversion processing unit 36 generates captured image data having a circular image as illustrated in FIG. 4 from the luminance distribution data using a color conversion table (not shown). The still image storage processing unit 51 stores the captured image data as still image data 39 in the external memory 30.

  Further, the image sensor 27 periodically generates luminance distribution data. Therefore, the still image data 39 in the external memory 30 is updated to new captured image data every predetermined period.

  When the still image data 39 in the external memory 30 is updated, the expanded still image generation unit 52 generates expanded still image data from the updated still image data 39.

  The expanded still image generation unit 52 first reads the inner circle boundary data 42 and the outer circle boundary data 43 from the EEPROM 41, and generates a ring shape for generating a panoramic expanded image in a captured image based on the updated still image data 39. Determine the range. Further, the developed still image generation unit 52 determines two division ranges obtained by equally dividing the ring-shaped image range into two at 180 degrees. FIG. 5 shows dotted lines 71 and 72 for horizontally dividing the ring-shaped image range into two. The developed still image generation unit 52 divides the ring-shaped image between the outer circle boundary and the inner circle boundary into two at 180 degrees, for example, by the two dotted lines 71 and 72, and the two divided image ranges on the upper and lower sides. A divided development image generation process is executed for each of 73 and 74.

  FIG. 7 is a flowchart showing the flow of the divided expanded image generation process by the expanded still image generation unit 52 for one divided image range 73 (74). The developed still image generation unit 52 first checks whether or not the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 have been updated after the last generation of the developed image (step ST1).

  When the inner circle boundary data 42 and the outer circle boundary data 43 are updated after the last developed image is generated (No in step ST), the developed still image generating unit 52 displays each developed pixel sequence of the divided developed image. The direction in the divided image range 73 (74) is calculated, and the direction table 46 is generated (step ST2).

  FIG. 8 is a diagram illustrating an example of an array of a plurality of display pixels of the divided development image. In order to simplify the description, a plurality of display pixels are arranged in 3 rows × 4 columns in the divided development image. In this case, the developed still image generation unit 52 generates a ring-shaped image for each of four developed pixel columns (1, Z) 81, (2, Z) 82, (3, Z) 83, and (4, Z) 84. Is calculated (direction in the divided image range 73 (74)).

  FIG. 9 is an explanatory diagram illustrating directions in the ring-shaped image (directions in the divided image range 73 (74)) of the expanded pixel columns 81, 82, 83, and 84 in FIG. The developed still image generation unit 52 calculates the angle obtained by equally dividing 180 degrees (= 360 degrees / 2) by the number of columns (= 4) assuming that the intersection of the two axes in FIG. 9 is the center of the circular image. .

  As shown in FIG. 9, for example, if the developed pixel column (1, Z) 81 of the first column in FIG. 8 is in the direction of the angle θ1, the developed still image generation unit 52 uses the developed pixel column ( 2, the angle of Z) 82 is calculated as “θ1 + uniform division angle”. The expanded still image generation unit 52 calculates the angle of the third expanded pixel column (3, Z) 83 as “θ1 + equal division angle × 2”. The developed still image generation unit 52 calculates the angle of the fourth developed pixel row (4, Z) 84 as “θ1 + equal division angle × 3”.

  The developed still image generating unit 52 calculates the respective directions for each of the developed pixel columns 81, 82, 83, and 84 as described above. The developed still image generating unit 52 generates a direction table 46 in which each direction is associated with a plurality of developed pixel columns 81, 82, 83, 84. The developed still image generation unit 52 stores the generated direction table 46 in the EEPROM 41. The direction table 46 may be stored in the external memory 30 or the like.

  After generating the direction table 46, the expanded still image generating unit 52 specifies the first display pixel of the divided expanded image (step ST3). The expanded still image generation unit 52 specifies, for example, the upper left display pixel ((1, 1) in FIG. 8) of the divided expanded image. Even when the inner circle boundary data 42 and the outer circle boundary data 43 are not updated (Yes in step ST1), the developed still image generation unit 52 specifies the first display pixel of the divided developed image (step ST3). ).

  Then, the developed still image generation unit 52 specifies the pixel in the captured image corresponding to the specified display pixel, acquires the pixel value of the pixel in the specified captured image, and displays the acquired pixel value Save as the pixel value of the pixel.

  Specifically, the developed still image generation unit 52 first refers to the image height correction table 45 and calculates the distance R from the center of the original image (circular image) for the specified display pixel (step ST4). . The developed still image generation unit 52 compares, for example, one developed pixel row of the divided developed image with a radial pixel arrangement within the ring shape range of the captured image while referring to the image height correction table 45, A distance R from the center of the donut at the position of the original image (circular image) corresponding to the specific display pixel is calculated.

  Next, the developed still image generation unit 52 calculates the abscissa in the original image (circular image) of the specific display pixel from the direction of the specific display pixel stored in the direction table 46 and the calculated distance R. (Step ST5). The developed still image generation unit 52 calculates the ordinate in the original image (circular image) of the specific display pixel from the direction of the specific display pixel stored in the direction table 46 and the calculated distance R (step ST6). ).

  FIG. 10 is an explanatory diagram of coordinate positions in the original image (circular image) of the specific display pixel. In FIG. 10, the center (IOX, IOY) of the original image (circular image) is the center of the inner circle boundary and the center of the outer circle boundary. That is, (IOX, IOY) is the center of the donut by the inner circle boundary and the outer circle boundary. When the distance from the center of the specific display pixel is R and the direction is θ, the coordinate position (IX, IY) of the specific display pixel in the original image (circular image) is expressed by the following formulas 1 and 2. Is calculated by The coordinate position (IX, IY) is based on the upper left of the original image (circular image).

IX = IOX + R × cos θ Formula 1
IY = IOY−R × sin θ Equation 2

  When the coordinate position (IX, IY) in the original image (circular image) of the specific display pixel is calculated, the developed still image generation unit 52 obtains the pixel value of the pixel 91 of the original image (circular image) at that coordinate. And stored as the pixel value of the specified display pixel. The developed still image generation unit 52 stores the pixel value of the specified display pixel in the first VRAM area 63 or the second VRAM area 64 of the external memory 30 (step ST7).

  Through the processes from step ST4 to ST7, the developed still image generation unit 52 determines and stores the pixel value of one display pixel specified in step ST3. Thereafter, the developed still image generation unit 52 determines whether or not there remains a display pixel for which the pixel value is not determined among the plurality of display pixels of the divided expanded image, and the display pixel for which the pixel value is not determined is determined. The determination process of the pixel value of the above display pixels is repeated until there is no more.

  Specifically, for example, the developed still image generating unit 52 first determines whether or not the determination of the pixel value for the display pixels for one column of the divided developed image has been completed (step ST8). If one column has not been completed (No in step ST8), the developed still image generation unit 52 specifies the display pixel of the next row in the same column (step ST9), and the pixel of the specific display pixel The value is determined (steps ST4 to ST7).

  When the determination of the pixel values for the display pixels for one column has been completed (Yes in step ST8), the expanded still image generation unit 52 has further completed the determination of the pixel values for all the columns of the divided expanded image. Is determined (step ST10). If all columns have not been completed (No in step ST10), the developed still image generation unit 52 specifies the display pixel at the upper end (uppermost row) of the next column (step ST11). The pixel value of the specific display pixel is determined (steps ST4 to ST7).

  When the pixel values have been determined for all the columns of the divided expanded image (Yes in step ST10), the expanded still image generating unit 52 ends the divided expanded image generation process of FIG. Thus, the divided developed image generation process for one divided image range 73 (74) is completed.

  Thereafter, the developed still image generation unit 52 executes a divided expanded image generation process for the remaining one divided image range 74 (73). As a result, two divided expanded images based on the still image data 39 are stored in the first VRAM area 63 or the second VRAM area 64 of the external memory 30.

  When two divided expanded images are stored in the first VRAM area 63 or the second VRAM area 64 of the external memory 30, the compressed still image generation unit 53 compresses the two divided expanded images. Thereby, the compressed and expanded still image data 61 and 62 are stored in the first VRAM area 63 or the second VRAM area 64.

  The streaming generation unit 54 sequentially reads the compressed and expanded still image data 61 and 62 from the first VRAM area 63 and the second VRAM area 64, and reads the audio data 38 from the external memory 30, and has these content data. Generate streaming data. In the streaming data, the two divided development images are arranged one above the other.

  The streaming generation unit 54 supplies the generated streaming data to the communication processing unit 56. The communication processing unit 56 of the 360 degree video camera apparatus 1 transmits streaming data to the communication processing unit 15 of the PC 3 through the USB connector 23, the USB cable 2, and the USB connector 11 of the PC 3.

  The communication processing unit 15 of the PC 3 supplies the received streaming data to the reproduction processing unit 16. The playback processing unit 16 of the PC 3 extracts and decodes the compressed expanded still image data from the streaming data, and supplies the data of the two divided expanded images to the liquid crystal display device 12 as display data. The liquid crystal display device 12 displays two divided development images. Further, the reproduction processing unit 16 of the PC 3 extracts audio data from the streaming data and supplies it to the speaker 13. The speaker 13 outputs a sound based on the extracted audio data.

  FIG. 11 is a display example of a 360-degree panoramic developed image displayed on the liquid crystal display device 12 of the PC 3 by imaging. On the liquid crystal display device 12, a 360-degree panoramic developed image is displayed by two divided developed screens 101 and 102 arranged vertically. Since the two divided development screens 101 and 102 are arranged one above the other, the display screen of the liquid crystal display device 12 can be used more effectively than when these are one development image, for example. In addition, a 360-degree panoramic image can be displayed large on the liquid crystal display device 12.

  Note that two divided development images 101 and 102 in FIG. 11 display two divided development images at 180 degrees obtained by dividing the ring-shaped image range of FIG. 5 into two by dotted lines 71 and 72. Has been. The two divided development images are continuous with each other at both ends, and as a whole, a 360-degree panoramic image is displayed without a break. With this display, the user of the PC 3 can browse a 360-degree panoramic image captured by the 360-degree video camera device 1. Further, the user of the PC 3 can hear the sound acquired by the 360 degree video camera apparatus 1 by the sound emitted from the speaker 13.

  As described above, the image sensor 27 of the 360 degree video camera apparatus 1 generates luminance distribution data for each period. The color conversion processing unit 36 generates captured image data from the luminance distribution data. The still image storage processing unit 51 updates the still image data 39 in the external memory 30 with the captured image data generated every cycle. The expanded still image generation unit 52 reads the still image data 39 from the external memory 30, and alternately generates two divided expanded image data based on the read still image data 39 in the first VRAM area 63 and the second VRAM area 64. Write. The streaming generation unit 54 reads the data of two compressed divided expanded images written in the first VRAM area 63 or the second VRAM area 64, and generates streaming data.

  Therefore, while the streaming generation unit 54 reads the data of two compressed divided expanded images from one of the first VRAM area 63 and the second VRAM area 64, the expanded still image generating unit 52 Two divided expanded images can be generated in the other VRAM area, and the compressed still image generating unit 53 can compress the two divided expanded images in the other VRAM area. The streaming generation unit 54 can allocate the two divided development images based on the luminance distribution data generated by the image sensor 27 for each period to the streaming data without causing image omission (frame omission). As a result, a moving image based on the luminance distribution data periodically generated by the image sensor 27 is displayed on the liquid crystal display device 12 of the PC 3.

  Next, an operation for auto panning a panorama developed image will be described.

  The command generation unit 17 of the PC 3 generates an autopan instruction command based on input data input from the input device 14. The communication processing unit 15 of the PC 3 transmits an auto pan instruction command to the communication processing unit 56 of the 360 degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360 degree video camera apparatus 1 supplies the received autopan instruction command to the process management unit 55.

  When the auto pan instruction command is supplied, the process management unit 55 periodically updates the direction table 46 stored in the EEPROM 41. Specifically, the process management unit 55 periodically repeats the process of adding a predetermined constant angle to the data in a plurality of directions for each developed pixel column in the direction table 46.

  Thereby, the direction of the specific display pixel is periodically increased by a predetermined constant angle in the flowchart of FIG. For example, the angle θ1 of the developed pixel row (1, Z) 81 in the first row in FIG. 9 periodically increases by a predetermined constant angle. The direction of the specific display pixel used in steps ST5 and ST6 in FIG. 7 also periodically increases by a predetermined constant angle.

  As a result, the coordinate position of each specific display pixel in the original image (circular image) rotates by a certain angle, and the pixel value of each specific display pixel also changes according to the rotation. As a result, in the divided developed image, the subject in the original image (circular image) moves sideways. An auto pan operation is realized in a 360-degree panoramic image.

  Next, the operation for adjusting the height of the panoramic image will be described.

  The command generation unit 17 of the PC 3 generates a change mode start instruction command based on input data input from the input device 14. The communication processing unit 15 of the PC 3 transmits a change mode start instruction command to the communication processing unit 56 of the 360 degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360 degree video camera apparatus 1 supplies the received change mode start instruction command to the process management unit 55.

  FIG. 12 is a flowchart showing the operation of the process management unit 55 in the change mode. When the change mode start instruction command is supplied (step ST21), the process management unit 55 supplies a boundary display instruction to the streaming generation unit 54 (step ST22). Thereafter, the process management unit 55 waits to receive a boundary change instruction command and a change mode end instruction command (steps ST23 and ST24).

  The streaming generation unit 54 to which the boundary display instruction is supplied reads the still image data 39 from the external memory 30. Every time the still image data 39 is read, the streaming generation unit 54 reads the inner circle boundary data 42 and the outer circle boundary data 43 from the EEPROM 41 following the reading. As shown in FIG. 5, the streaming generation unit 54 generates a display image in which the inner circle boundary mark 47 and the outer circle boundary mark 48 are superimposed on the captured image of the still image data 39. Note that the streaming generation unit 54 may read the inner circle boundary data 42 and the outer circle boundary data 43 each time the still image data 39 is read a plurality of times.

  Further, the streaming generation unit 54 generates streaming data including the generated display image data and supplies the streaming data to the communication processing unit 56. The communication processing unit 56 of the 360 degree video camera apparatus 1 transmits streaming data to the communication processing unit 15 of the PC 3 via the USB cable 2.

  The communication processing unit 15 of the PC 3 supplies the received streaming data to the reproduction processing unit 16. The reproduction processing unit 16 of the PC 3 extracts display image data from the streaming data and supplies it to the liquid crystal display device 12 as display data. As shown in FIG. 5, the liquid crystal display device 12 displays a captured image in which a circular mark 47 on the inner circle boundary and a circular mark 48 on the outer circle boundary are superimposed.

  The command generation unit 17 of the PC 3 moves the position of the inner circle boundary and the outer circle boundary, and expands / reduces the radius of the inner circle boundary and the outer circle boundary based on a user operation on the input device 14. Generate a command. The communication processing unit 15 of the PC 3 transmits a boundary change instruction command to the communication processing unit 56 of the 360 degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360 degree video camera apparatus 1 supplies the received boundary change instruction command to the process management unit 55.

  When the boundary change instruction command is received (Yes in step ST23), the process management unit 55 first determines whether or not the instruction is appropriate (step ST25). Specifically, for example, the process management unit 55 determines whether the radius of the inner circle boundary after the change or the radius of the outer circle boundary after the change is equal to or larger than a predetermined radius (for example, 144 pixels).

  If it is determined that the instruction is appropriate (Yes in step ST25), the process management unit 55 updates the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 based on the change instruction. (Step ST26). Further, the process management unit 55 updates the image height correction table 45 stored in the EEPROM 41 based on the new combination of the inner circle boundary and the outer circle boundary (step ST27).

  When the interval between the inner circle boundary and the outer circle boundary changes due to the change, the number of pixels between the inner circle boundary and the outer circle boundary in the captured image changes. As a result, the range of the space imaged by the plurality of pixels changes, and the division ratio of the imaging space by the plurality of pixels also changes.

  Further, when the fisheye lens 22 of the three-dimensional projection method is used, the subject forms an image with different sizes depending on the angle of view. Therefore, as described with reference to FIG. 6, even when the distance between the inner circle boundary and the outer circle boundary is not changed by the update, only the positions of the inner circle boundary and the outer circle boundary move, and the imaging space The division ratio of a plurality of pixels changes.

  The process management unit 55 updates the image height correction table 45 in accordance with the change in the division ratio of the plurality of pixels accompanying the change between the inner circle boundary and the outer circle boundary. Thereafter, the process management unit 55 returns to the reception waiting state for the boundary change instruction command and the change mode end instruction command (steps ST23 and ST24).

  If the instruction is not appropriate (No in step ST25), for example, if the radius of the inner circle boundary is smaller than the predetermined radius, the process management unit 55 ignores the change instruction, and changes the boundary change instruction command and change mode end instruction. Returning to the command reception waiting state (steps ST23 and ST24).

  By the way, the streaming generation unit 54 supplied with the boundary display instruction reads the inner circle boundary data 42 and the outer circle boundary data 43 from the EEPROM 41 every time the still image data 39 is read from the external memory 30. Therefore, when the process management unit 55 updates the inner circle boundary data 42 and the outer circle boundary data 43, the streaming generation unit 54 also updates the position and size of the overlap of the circular marks 47 and 48 with respect to the captured image. The positions and sizes of the circular marks 47 and 48 displayed on the liquid crystal display device 12 so as to overlap the captured image are changed according to the boundary change instruction command. The user can know that the inner circle boundary data 42 and the outer circle boundary data 43 have been updated by the display on the liquid crystal display device 12.

  The command generation unit 17 of the PC 3 generates a change mode end instruction command based on a user operation on the input device 14. The communication processing unit 15 of the PC 3 transmits a change mode end instruction command to the communication processing unit 56 of the 360 degree video camera apparatus 1 via the USB cable 2. The communication processing unit 56 of the 360 degree video camera apparatus 1 supplies the received change mode end instruction command to the process management unit 55. When the change mode end instruction command is received (Yes in step ST24), the process management unit 55 ends the change mode process of FIG. After the processing in this change mode, the inner circle boundary data 42, the outer circle boundary data 43, and the image height correction table 45 are changed to new ones, and the developed still image generation unit 52 of the 360-degree video camera device 1 performs this change. A 360-degree panoramic development image is generated using the processed data.

  As described above, according to this embodiment, a circular image is formed by the fisheye lens 22 on the image sensor 27 of the 360-degree video camera device 1. The still image storage processing unit 51 generates captured image data having a circular image based on the luminance distribution data of the image sensor 27. This circular image is included without the subject's head being cut off.

  In addition, the developed still image generation unit 52 has a 360 degree image obtained by two divided developed images based on an image within a ring-shaped range sandwiched between the inner circle boundary and the outer circle boundary in the captured image. A panorama development image is generated. The position and size of the inner circle boundary and the outer circle boundary can be freely set by the user by the process management unit 55.

  Therefore, by updating the position or size of the inner circle boundary or the outer circle boundary by the processing management unit 55, it is possible to adjust the subject in the circular image to be within the ring shape range. As a result, for example, even when the subject exists at a position close to the fisheye lens 22 and the physical distance of the fisheye lens 22 to the subject cannot be ensured, by adjusting the position of the inner circle boundary, etc. It is possible to generate a 360-degree panoramic image with no head cut. Further, for example, it is possible to generate a 360-degree panoramic developed image in which a person's head and body as a subject are displayed with good balance.

  In addition, two horizontally long divided developed images every 180 degrees are displayed vertically on the liquid crystal display device 12 of the PC 3. Therefore, for example, the display screen of the liquid crystal display device 12 can be used more effectively than when a 360-degree panoramic image is a single expanded image. A 360-degree panoramic image can be displayed large on the liquid crystal display device 12.

  In this embodiment, the fisheye lens 22 is of a three-dimensional projection system. The three-dimensional projection type fisheye lens 22 is less distorted in the peripheral portion in the image formed by the fisheye lens 22 than the case where a general equidistant projection type fisheye lens is used, and the amount of information in the peripheral portion is small. Become more. The EEPROM 41 also stores an image height correction table 45 for correcting the balance of the imaging size of the subject obtained based on the relationship between the incident angle of the stereoscopic lens 22 and the image height. The developed still image generation unit 52 refers to the image height correction table 45 and generates a 360-degree panoramic developed image in which the balance of the imaging size of the subject is adjusted.

  Therefore, when the 360 degree video camera device 1 is placed on the table with the fisheye lens 22 facing upward, it is possible to obtain an image of a person or blackboard around the high-resolution image. Blackboard characters and the like can be reproduced in a 360-degree panoramic image. Further, a well-balanced image can be obtained as an image of a person or blackboard around the fisheye lens 22.

  In this embodiment, the image height correction table 45 stored in the EEPROM 41 is updated every time the inner circle boundary data 42 or the outer circle boundary data 43 is updated. When the interval between the inner circle boundary and the outer circle boundary changes due to the change, the division ratio of the plurality of pixels in the image space between them changes. Even if only the positions of the inner circle boundary and the outer circle boundary move, the division ratio of the plurality of pixels in the image space between them changes. Thus, by updating the image height correction table 45 each time the inner circle boundary data 42 or the outer circle boundary data 43 is updated, the image height correction table 45 is maintained to be compatible with these.

  In this embodiment, the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 are updated when the changed size is a radius of 144 pixels or more. For example, if the radius of the inner circle boundary becomes one pixel, the same pixel in the captured image is used many times in order to obtain the pixel values of the plurality of display pixels at the uppermost stage of the 360 ° panorama development image. Become. In this case, the image quality of the 360-degree panorama development image is not improved, and the generation load of the 360-degree panorama development image increases. By preventing the inner circle boundary or outer circle boundary radius from becoming smaller than 144 pixels as in this embodiment, it is possible to prevent such disadvantages from occurring.

  Further, in this embodiment, the developed still image generating unit 52 calculates the distance R and the direction from the center of the circular image of each display pixel of the 360 ° panorama developed image, and uses the calculated distance R and direction. Then, the coordinates (IX, IY) in the captured image of each display pixel are calculated, and the pixel value of the pixel 91 of the captured image located at each calculated coordinate (IX, IY) is used as the pixel value of each display pixel. get.

  Therefore, it is possible to generate a 360-degree panoramic image using the pixel values of the circular image pixels in the captured image as they are. It is not necessary to obtain the pixel value of each display pixel of the 360 degree panoramic image by calculation. As in the case where the pixel value of each display pixel is obtained by computation, image quality degradation due to computation does not occur. In addition, the generation time of one panorama development image can be shortened. As a result, a 360-degree panoramic image can be generated in a short time. In addition, by continuously repeating the panorama development image generation process, it is possible to obtain a moving image with continuous motion using a 360-degree panorama development image.

  As described above, this embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the invention. Variations and changes are possible.

  For example, in the above-described embodiment, the fish-eye lens 22 uses a stereoscopic projection type. In addition to this, for example, a common equidistant projection method, an isostereoscopic projection method, an orthographic projection method, or the like may be used as the fisheye lens 22.

  In the embodiment described above, the developed still image generation unit 52 generates two divided developed images every 180 degrees as a 360 degrees panoramic developed image. In addition to this, for example, the developed still image generation unit 52 may generate a single 360-degree panoramic developed image, divide into three, or divide into four.

  In the above embodiment, the process management unit 55 updates both the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 based on a user operation. In addition, for example, the process management unit 55 may update only one of the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 based on a user operation.

  In the above embodiment, the PC camera 3 is connected to the 360 degree video camera apparatus 1 by the USB cable 2. In addition to this, for example, an input board for outputting a command may be connected to the 360 degree video camera apparatus 1 by a USB cable 2 or the like.

  In the above embodiment, the PC 3 connected to the 360 degree video camera apparatus 1 displays a moving image based on a 360 degree panoramic image on the liquid crystal display device 12. In addition to this, for example, the PC 3 may store a moving image based on a 360-degree panoramic image in a storage device (not shown).

  INDUSTRIAL APPLICABILITY The present invention can be widely used as a panorama developed image capturing apparatus and a panorama expanded image capturing system for capturing a meeting or the like.

It is a perspective view which shows the panoramic expansion image imaging system which concerns on embodiment of this invention. It is a block diagram which shows the electric circuit of the 360 degree video camera apparatus in FIG. It is explanatory drawing which shows the optical arrangement | positioning of the fisheye lens and image sensor in FIG. It is a figure which shows an example of the captured image which the color conversion process part in FIG. 1 produces | generates based on the luminance distribution data of an image sensor. FIG. 3 is an explanatory diagram showing an image obtained by superimposing a circular mark indicating a position of an inner circle boundary and a circular mark indicating a position of an outer circle boundary on a captured image in the 360-degree video camera device of FIG. 1. FIG. 2 is an image height (angle of view difference) characteristic diagram of the three-dimensional projection type fisheye lens in FIG. 6 is a flowchart showing a flow of a divided developed image generation process by a developed still image generation unit for one divided image range generated in the 360-degree video camera apparatus of FIG. 1. It is a figure which shows an example of the arrangement | sequence of several display pixels in the division | segmentation expansion | deployment image produced | generated in the 360 degree video camera apparatus of FIG. FIG. 9 is an explanatory diagram illustrating directions in a ring-shaped image (directions in a divided image range) of each developed pixel row in FIG. 8. It is explanatory drawing of the coordinate position in the original image (circular image) of the specific display pixel in the 360 degree video camera apparatus of FIG. 3 is a display example of a 360-degree panoramic unfolded image displayed by imaging, which is displayed on the liquid crystal display device of the PC in FIG. 1. 3 is a flowchart showing an operation of a process management unit in a change mode of the 360 degree video camera apparatus of FIG. 1.

Explanation of symbols

1 360 degree video camera device (panoramic unfolding image pickup device)
3 Personal computer (computer device)
22 Fisheye lens 27 Image sensor (part of imaging means)
33 Light receiving surface 36 Color conversion processing unit (part of imaging means)
41 EEPROM (memory)
45 Image Height Correction Table 52 Expanded Still Image Generation Unit (Panorama Expanded Image Generation Unit)
54 Streaming generator (display data generator)
55 Process Management Unit (User Update Unit)

Claims (7)

A panoramic unfolded image imaging device that generates a 360-degree panoramic unfolded image by imaging,
With a fisheye lens,
An imaging unit that forms a circular image by the fisheye lens on a light receiving surface on which a plurality of light receiving elements are arranged, and generates data of a captured image having a circular image;
A memory for storing the position or size of the inner circle boundary and the outer circle boundary defining the range of the ring shape used for generating the 360-degree panoramic development image in the captured image;
User update means for updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory based on a user operation;
A 360-degree panoramic unfolded image is generated based on an image within a ring-shaped range sandwiched between the inner circle boundary and the outer circle boundary stored in the memory in the captured image generated by the imaging means. Panoramic unfolded image generating means,
A panoramic developed image imaging device characterized by comprising: The fisheye lens is a three-dimensional projection method,
The memory stores an image height correction table for correcting a balance of the imaging size of a subject obtained based on a relationship between an incident angle of the stereoscopic projection type fisheye lens and an image height,
The panoramic unfolded image generating means refers to the image height correction table, and generates a 360 degree panoramic unfolded image in which the balance of the imaging size of the subject is adjusted;
The panoramic unfolded image imaging apparatus according to claim 1.   The user update means updates the image height correction table stored in the memory after updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory. 3. The panoramic unfolded image capturing apparatus according to claim 2, wherein updating is performed based on the inner circle boundary and the outer circle boundary, and a relationship between an incident angle and an image height of the stereoscopic eye fisheye lens.   The user update means, when the size of the inner circle boundary or the outer circle boundary after the change is a radius of a predetermined number of pixels or more, based on the change instruction of the inner circle boundary or the outer circle boundary, The panoramic developed image imaging apparatus according to claim 1, wherein the inner circle boundary or the outer circle boundary stored in a memory is updated.   The panoramic expanded image generation means calculates a distance and a direction of each display pixel of the 360 ° panoramic expanded image from the center of the circular image, and uses the calculated distance and direction of each display pixel, 2. The panorama development according to claim 1, wherein coordinates in the captured image are calculated, and pixel values of the pixels of the captured image located at the calculated coordinates are acquired as pixel values of the display pixels. Imaging device. The panoramic expanded image generation means divides a range between the inner circle boundary and the outer circle boundary in the captured image into two ranges every 180 degrees, and two ranges for the divided image ranges Generate a horizontally long panoramic split image,
Having display data generation means for generating display data for displaying the two horizontally long panoramic divided developed images side by side;
The panoramic unfolded image imaging apparatus according to claim 1. A panoramic unfolded image capturing apparatus according to any one of claims 1 to 6;
A computer device for displaying or storing a 360-degree panoramic image generated by the panoramic image-capturing device;
A panoramic unfolded image capturing system characterized by comprising:

Images