WO2008010345A1

WO2008010345A1 - Panorama image photographing system and panorama image photographing method

Info

Publication number: WO2008010345A1
Application number: PCT/JP2007/060193
Authority: WO
Inventors: Masayuki Ito
Original assignee: Opt Corporation; Shinya Makoto
Priority date: 2006-07-20
Filing date: 2007-05-18
Publication date: 2008-01-24
Also published as: JP2008028606A

Abstract

A 360-degree panorama image of an object is photographed without its discontinuity. A panorama image photographing device (1) generates a photographed 360-degree spread panorama image. The panorama image photographing device (1) is comprised of a memory (41) for storing positions and sizes of an inner circle boundary (42) and an outer circle boundary (43) to define a ring shaped region used for generating the 360-degree spread panorama image in a photographed image generated by photographing means (27, 36) based on an image formed by a fish-eye lens (22), a user updating means for updating the position or size of at least either the inner circle boundary (42) or the outer circle boundary (43) stored in the memory (41) in accordance with user's operations, and a panorama image generating means (52) for generating the 360-degree spread panorama image based on the image in the ring shaped region put between the inner circle boundary (42) and the outer circle boundary (43) stored in the memory (41).

Description

Specification

NOV llama unfolding image capturing device and panoramic unfolding image capturing system

[0001] The present invention relates to a panorama developed image imaging apparatus and a panorama developed image imaging system.

Background art

[0002] Patent Document 1 discloses an omnidirectional photographing apparatus. The omnidirectional imaging device develops and displays or records the video captured by one camera module that has a lens that can directly capture a 360-degree annular image.

Patent Document 1: Japanese Patent Application Laid-Open No. 2006-148787 (abstract, detailed description of the invention, drawings, particularly, paragraph 0024, paragraph 0049, FIG. 8, FIG. 9, etc.)

Disclosure of the invention

Problems to be solved by the invention

[0004] In Patent Document 1, a lens that directly captures a 360-degree annular image is a special convex glass lens, specifically, a zenith reflecting surface and a shape surrounding the zenith reflecting surface. A substantially annular lens surface; a substantially annular back reflecting surface disposed below the lens surface; and a transmission surface surrounded by the back reflecting surface. Patent Document 1 suggests that a fish-eye lens can be used in place of the special convex glass lens.

However, with the omnidirectional imaging device of Patent Document 1, there is a limit to the image height that can be captured in a 360-degree annular image. In this annular image, the upper part of the lens surface is refracted by the lower part of the lens surface, reflected from the outer peripheral part of the rear reflective surface, and further incident light from the direction reflected by the outer peripheral part of the zenith reflective surface. Therefore, it is not possible to obtain an image force in the range of the angle of view from the direction of the incident light from the direction reflected by the inner peripheral portion of the back reflecting surface and further reflected by the inner peripheral portion of the zenith reflecting surface. In the center of the ring image, the zenith reflection surface is photographed in a circle. Since the omnidirectional imaging device of Patent Document 1 uses such a special convex glass lens, the image height that can be captured is limited. [0006] Because there is a restriction on the image height, the omnidirectional photographing apparatus of Patent Document 1 imposes a restriction on the distance between the glass lens and the subject in practice. For example, if the subject gets too close to the glass lens, the subject's head will be cut off in the 360-degree annular image. In order to prevent the subject's head from being cut off, the distance of the glass lens to the subject must be secured.

[0007] In particular, when the omnidirectional photographing apparatus is placed in the center of the table and it is intended to capture a conference scene or the like, this practical distance limitation greatly impairs the convenience of the omnidirectional photographing apparatus. For example, in a small conference room, it may not be possible to secure the distance of the glass lens to the subject, and the subject's head may be cut off, making it impossible to shoot.

[0008] An object of the present invention is to obtain a panorama unfolded image capturing apparatus and a panorama unfolded image capturing system capable of capturing a 360-degree panorama unfolded image without causing the subject to be cut.

Means for solving the problem

[0009] The V Lamma development image capturing apparatus according to the present invention generates a 360 degree V Lama development image by imaging. The panoramic developed image imaging device includes a fish-eye lens, an imaging unit that forms a circular image by the fish-eye lens on a light-receiving surface on which a plurality of light-receiving elements are arranged, and generates captured image data having the circular image; The 360 degree panorama is stored in the memory based on the user's operation and the memory that stores the position or size of the inner and outer circle boundaries that define the range of the ring shape used to generate the unfolded image. A user update unit that updates the position or size of at least one of the inner circle boundary and the outer circle boundary, and the inner circle boundary and outer circle boundary stored in the memory in the captured image generated by the imaging unit. Panorama unfolded image generating means for generating a 360-degree non-linear raster image based on an image within the range of the ring shape.

If this configuration is adopted, a circular image formed by a fisheye lens is formed on the imaging means. The imaging means generates a captured image having a circular image. The circular image is included without the subject's head being cut off.

[0011] Further, the no-V llama unfolded image generating means includes an inner circle boundary and an outer circle boundary in the captured image. Based on the image in the range of the ring shape sandwiched between the two, a 360 degree V-Rama expansion image is generated. The positions and sizes of the inner circle boundary and the outer circle boundary can be updated by the user update unit based on the user's operation. Therefore, for example, even if it is not possible to physically secure the distance of the fisheye lens to the subject, the position of the inner circle boundary is updated by the user's operation so that the position of the outer circle boundary is updated. Subject force Can be adjusted so that it is within the range of the S-ring shape. As a result, the subject's head is cut off, and a 360-degree panoramic image can be generated.

In addition to the configuration of the above-described invention, a panoramic developed image imaging device according to the present invention has the following characteristics. That is, the fisheye lens is of a three-dimensional projection system. The memory stores an image height correction table for correcting the balance of the imaging size of the subject obtained based on the relationship between the incident angle of the stereoscopic lens fisheye lens and the image height. The panorama spread image generation means refers to the image height correction table, and generates a 360-degree V-Rama development image in which the tolerance of the imaging size of the subject is adjusted.

[0013] If this configuration is adopted, the peripheral portion in the image formed by the fisheye lens is less distorted and the amount of information in the peripheral portion than when a general equidistant projection fisheye lens is used. Will increase. Therefore, when the panoramic developed image pickup device is placed on the table with the fisheye lens facing upward, it is possible to obtain an image of a person or a blackboard with a high resolution. The characters on the blackboard can be reproduced in a 360-degree panoramic image. In addition, it is possible to obtain a well-balanced image such as a person or a blackboard with a fisheye lens.

[0014] A panoramic developed image capturing apparatus according to the present invention has the following features in addition to the above-described components of the present invention. That is, the user update means updates the image height correction table stored in the memory after updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory. It is updated based on the outer circle boundary and the relationship between the angle of incidence and the image height of the cuboid projection fisheye lens.

[0015] When the interval between the inner circle boundary and the outer circle boundary changes due to the change, the division ratio of the plurality of pixels in the image space between them changes. Even if the position of the inner circle boundary and the outer circle boundary only moves, the division ratio of the image space between them by multiple pixels Changes. If this configuration is adopted, the image height correction table can be kept compatible with them.

[0016] The panoramic developed image capturing apparatus according to the present invention has the following features in addition to the above-described components of the present invention. In other words, the user update means, when the size after the change of the inner circle boundary or the outer circle boundary is a radius greater than or equal to the predetermined number of pixels, Update the inner circle boundary or outer circle boundary stored in memory.

[0017] For example, if the radius of the inner circle boundary becomes one pixel, the same pixel in the captured image is used many times in order to obtain pixel values of a plurality of display pixels at the top of the 360-degree panorama development image. Will be. In this case, the load of generating a 360-degree panorama development image increases without improving the image quality of the 360-degree rama development image. By preventing the radius of the inner circle boundary or outer circle boundary from becoming smaller than the predetermined number of pixels as in this configuration, it is possible to prevent such disadvantages from occurring. Can do.

[0018] The panoramic developed image capturing apparatus according to the present invention has the following features in addition to the above-described components of the present invention. That is, the no-rama development image generation means calculates the distance and the direction from the center of the circular image of each display pixel of the 360-degree panorama development image, and uses the calculated distance and the direction of each display pixel. Then, the coordinates in the captured image are calculated, and the pixel value of the pixel of the captured image located at each calculated coordinate is acquired as the pixel value of each display pixel.

By adopting this configuration, it is possible to generate a 360-degree NORMAL development image using the pixel values of the pixels of the circular image in the captured image as they are. There is no need to calculate the pixel value of each display pixel in the 360-degree llama image. As in the case of obtaining the pixel value of each display pixel by calculation, it is possible to reduce the generation time of a single V-Rama development image without causing image quality degradation due to calculation. As a result, a 360-degree panoramic image can be generated in a short time. In addition, by continuously repeating the generation process of the no-V llama development image, it is possible to obtain a moving image with a 360-degree no-rama development image.

[0020] A panoramic developed image imaging device according to the present invention includes the following components in addition to the above-described components of the present invention. It has the following characteristics. In other words, the no-rama development image generation means divides the range between the inner circle boundary and the outer circle boundary in the captured image into two ranges every 180 degrees, and 2 for each range of the divided images. Two horizontally long panoramic divided development images are generated. In addition, there is a display data generating means for generating display data for displaying two horizontally long panoramic divided developed images side by side.

By adopting this configuration, it is possible to effectively use the display screen of the display means for displaying the display data as compared with, for example, a case where these are one developed image. The display means can display a large 360-degree image of the llama.

[0022] A NORMAL development image imaging system according to the present invention displays a panorama development image imaging device that generates a 360-degree panorama development image by imaging, and a NORMAL development image generated by the panorama development image imaging device. Or it has a computer apparatus which preserve | saves. Then, the panoramic developed image imaging device includes a fish-eye lens, an imaging unit that forms a circular image by the fish-eye lens on a light-receiving surface on which a plurality of light-receiving elements are arranged, and generates data of a captured image having a circular image; A memory that stores the position or size of the inner and outer circle boundaries that define the range of the ring shape used to generate a 360-degree image of the 360 ° llama in the image, and a memory based on user operations. Is the position of at least one of the inner circle boundary and the outer circle boundary stored in the !! is the user update means for updating the size, and the inner circle boundary stored in the memory in the captured image generated by the imaging means Panorama unfolded image generating means for generating a 360 degree NOV llama unfolded image based on an image within a range of a ring shape sandwiched between the outer circle boundaries.

[0023] By adopting this configuration, it is possible to display or save a 360-degree image of a llama developed.

The invention's effect

[0024] In the present invention, a 360-degree panoramic developed image can be taken without the subject being cut off.

Brief Description of Drawings

FIG. 1 is a perspective view showing a panoramic developed image imaging system according to an embodiment of the present invention. 2 is a block diagram showing an electric circuit of the 360-degree video camera device in FIG.

FIG. 3 is an explanatory view showing an optical arrangement of the fisheye lens and the image sensor in FIG.

4 is a diagram illustrating an example of a captured image generated by the color conversion processing unit in FIG. 1 based on luminance distribution data of the image sensor.

5 is an explanatory diagram showing an image obtained by superimposing a circular mark indicating the position of the inner circle boundary and a circular mark indicating the position of the outer circle boundary on the captured image in the 360-degree video camera device of FIG. is there.

FIG. 6 is an image height (field angle difference) characteristic diagram of the three-dimensional projection type fisheye lens in FIG.

FIG. 7 is a flow chart showing the flow of a divided expanded image generation process by the expanded still image generation unit for one divided image range generated in the 360-degree video camera device of FIG.

8 is a diagram showing an example of an array of a plurality of display pixels in a divided expanded image generated in the 360-degree video camera device of FIG.

FIG. 9 is an explanatory diagram illustrating directions in a ring-shaped image (directions in a divided image range) of each developed pixel row in FIG. 8.

FIG. 10 is an explanatory diagram of coordinate positions in an original image (circular image) of a specific display pixel in the 360-degree video camera device of FIG.

[FIG. 11] A display example of a 360-degree panoramic developed image displayed on the liquid crystal display device of the PC in FIG.

FIG. 12 is a flowchart showing the operation of the process management unit in the change mode of the 360-degree video camera device of FIG. Explanation of symbols

1 360-degree video camera device (Panorama image capturing device)

3 Personal computer (computer equipment)

22 Fisheye lens

27 Image sensor (part of imaging means)

33 Photosensitive surface

36 color conversion processing part (part of imaging means) 41 EEPROM (memory)

45 Image height correction table

52 Expanded still image generator (panorama expanded image generator)

54 Streaming generator (display data generator)

55 Process Management Department (User update means)

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, a panorama developed image imaging apparatus and a panorama developed image imaging system according to an embodiment of the present invention will be described with reference to the drawings. The panorama development image capturing apparatus will be described by taking a conference 360-degree video camera apparatus as an example. The V-Lama unfolding imaging system will be described using an example of a 360-degree video camera device connected to a personal computer.

[0028] FIG. 1 is a perspective view showing a panoramic developed image imaging system according to an embodiment of the present invention. The NOV llama unfolded image capturing system includes a 360 degree video camera device 1 as a NOV llama unfolded image capturing device and a PC (personal computer) 3 as a computer device. The 360-degree video camera device 1 is connected to the PC 3 by a USB (Universal Serial Bus) cable 2.

[0029] The PC 3 includes a USB connector 11, a liquid crystal display device 12, a speaker 13, an input device 14, and the like. In addition, a CPU (Central Processing Unit) (not shown) reads and executes a storage device power program (not shown) so that the PC 3 has a communication processing unit 15, a reproduction processing unit 16, a command generation unit. 17 is realized.

The communication processing unit 15 controls data communication using the USB connector 11. The reproduction processing unit 16 controls the content displayed on the liquid crystal display device 12 and causes the speaker 13 to output sound. The command generation unit 17 generates a command based on input data to which the input device 14 is also input. Examples of the input device 14 include a keyboard and a pointing device.

The 360-degree video camera device 1 has a cubic-shaped housing 21. The housing 21 is provided with a fisheye lens 22, a USB connector 23, a video output connector 24, an audio output connector 25, and the like. The fisheye lens 22 is disposed on the upper surface of the housing 21. housing A ventilation hole 20 for the microphone 26 is formed on the upper surface of 21. The USB connector 23, the video output connector 24 and the audio output connector 25 are arranged on the side surface of the housing 21.

FIG. 2 is a block diagram showing an electric circuit of 360-degree video camera device 1 in FIG. In the housing 21 of the 360 ° video camera device 1, an electric circuit for generating a 360-degree expanded image of a 360-degree llama by imaging is incorporated.

[0033] In addition to the fish-eye lens 22, the USB connector 23, the video output connector 24, and the audio output connector 25, the 360-degree video camera device 1 includes an image sensor 27 as a part of the imaging means, an FPG A (Field Programmable Gate Array). 28, DSP (Digital Signal Processor) 29, external memory 30, audio IC (Integrated Circuit) 31, video encoder 32, etc.

FIG. 3 is an explanatory diagram showing an optical arrangement of the fisheye lens 22 and the image sensor 27.

The image sensor 27 is, for example, a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The image sensor 27 has a light receiving surface 33. On the light receiving surface 33, a plurality of light receiving elements (not shown) are arranged in a matrix at a ratio of, for example, 3: 4. Each light receiving element outputs a value corresponding to the amount of received light. The image sensor 27 generates luminance distribution data having a plurality of received light amount values output from a plurality of light receiving element forces. The image sensor 27 generates luminance distribution data at a predetermined cycle.

[0036] The fish-eye lens 22 has a wide viewing angle of, for example, 180 degrees or more, and has a stereoscopic projection method. Compared with a general equidistant projection fisheye lens, the three-dimensional projection fisheye lens 22 causes less distortion in the peripheral portion of the image formed and increases the amount of information in the peripheral portion. As the projection method of the fish-eye lens 22, in addition to the stereoscopic projection method and the equidistant projection method, an equal stereoscopic projection method, an orthographic projection method, and the like can be adopted. However, when the 360-degree video camera device 1 is placed on the table in the conference room with the fisheye lens 22 facing upward, the amount of information such as a person or blackboard that appears in the surrounding area increases, so the stereoscopic projection method The one is the best.

The fisheye lens 22 is disposed above the light receiving surface 33 of the image sensor 27. This Thus, a circular image (hereinafter referred to as a circular image) by the fisheye lens 22 is formed on the light receiving surface 33 of the image sensor 27. The image sensor 27 periodically generates luminance distribution data and outputs it to the FP GA28. In the FPGA 28, a color conversion processing unit 36 is realized as a part of the imaging means.

[0038] The color conversion processing unit 36 replaces the data of each pixel of the luminance distribution data using a color conversion table (not shown). The color conversion processing unit 36 replaces, for example, pixel data in a circular image with predetermined color data using the pixel value and the peripheral pixel values in the luminance distribution data. Thereby, captured image data having appropriate color data is generated.

FIG. 4 is a diagram illustrating an example of a captured image that the color conversion processing unit 36 generates based on the luminance distribution data of the image sensor 27. As shown in FIG. 4, the captured image has a circular image at the center. An image of the subject is formed inside the circular image. The color conversion processing unit 36 outputs the generated captured image data to the DSP 29.

[0040] The microphone 26 generates a waveform signal corresponding to the sound. The waveform signal is converted into an audio signal by the audio IC 31 and supplied to the audio output connector 25. For example, a speaker unit or headphones can be connected to the audio output connector 25. Audio can be heard through the speaker unit or headphones connected to the audio output connector 25. In addition, a voice storage processing unit 37 is realized in the voice IC 31. The sound storage processing unit 37 samples the waveform signal supplied from the microphone 26 and stores the sound data 38 generated by the sampling in the external memory 30.

The DSP 29 has an EEPROM (Electrically Erasable Programmable Read-Only Memory) 41 as a memory. The EEPROM 41 stores inner circle boundary data 42, outer circle boundary data 43, an incident angle image height table 44, an image height correction table 45, a direction table 46, and the like.

FIG. 5 is an explanatory diagram showing an image in which a circular mark 47 indicating the position of the inner circle boundary and a circular mark 48 indicating the position of the outer circle boundary are superimposed on the captured image. The inner circle boundary and the outer circle boundary are boundaries that specify an image range to be expanded as a panoramic expansion image. The image portion between the inner circle boundary and the outer circle boundary is developed as a panoramic image. Inner circle boundary The data 42 is data indicating the position and size of the inner circle boundary in the captured image. The outer circle boundary data 43 is data indicating the position and size of the outer circle boundary in the captured image.

FIG. 6 is an image height (field angle difference) characteristic diagram of the fisheye lens 22 of the three-dimensional projection method. The horizontal axis is the relative field angle when the optical axis direction of the fish-eye lens 22 is 0 degrees, and the vertical axis is the image height (field angle difference). When the fisheye lens 22 of the three-dimensional projection method having the characteristics shown in FIG. 6 is used, for example, the image height (field angle difference) of the subject existing in the direction of the angle of view of the fisheye lens 22 is about 0.1. When the angle difference is reached, the image height (angle difference) of the subject that exists in the direction of 90 degrees is about 0.2. This means that a subject in the peripheral direction of the fish-eye lens 22 forms an image with a size approximately twice that of a subject in the central direction. It also means that when the subject is photographed from the peripheral part to the central part, the balance is lost. The incident angle image height table 44 is obtained by converting the image height (view angle difference) characteristics with respect to the angle of view of the fisheye lens 22 into data.

[0044] When a fisheye lens is used, an image of one subject is generally distorted as it goes to the periphery. There is a three-dimensional projection method as a method of reducing the degree of distortion and increasing the amount of surrounding information. When this stereoscopic projection type fisheye lens 22 is used, the subject is imaged more in the case of being imaged in the peripheral area than in the case of being imaged in the central area in terms of angle of view. The subject imaged in the central part forms an image with a smaller angle of view than the subject imaged in the peripheral part. This is because, as shown in FIG. 6, the four light receiving elements near the center of the image sensor 27 and the four light receiving elements near the periphery of the image sensor 27, that is, a portion near the angle of view of 0 degrees, that is, the zenith. This means that an image with a wider angle of view is formed on a nearby light receiving element than a portion near 90 degrees of view angle of the image sensor 27, that is, a light receiving element in the lateral direction.

In this way, the stereoscopic projection method has a larger amount of peripheral information than the conventional equi-distance projection fish-eye lens, so that the blind spots are reduced and the image is less distorted. However, the distortion is reduced, but not zero. On the other hand, it is desirable for the subject to be displayed with a balance of the original size in the no-rama development image. The image height correction table 45 stores correction data for obtaining a developed image in which the subject appears as a natural balance from the image formed on the image sensor 27 by the fisheye lens 22 of the three-dimensional projection method.

[0046] The DSP 29 has a CPU (not shown). This CPU executes the program The DSP 29 includes a still image storage processing unit 51, a developed still image generation unit 52 as a V Lama development image generation unit, a compressed still image generation unit 53, a streaming generation unit 54 as a display data generation unit, and a user update unit. The processing management unit 55, the communication processing unit 56, and the like are realized.

Further, an external memory 30 is connected to the DSP 29. External memory 30 is SRAM (Static

It consists of storage devices such as RAM) and DDR—SDRAM (Double Data Rate SDRAM), and is accessible to the DSP29 CPU.

[0048] The external memory 30 stores still image data 39, audio data 38, two compressed and expanded still image data 61 and 62, and the like. The external memory 30 has a first VRAM (VideoRAM) area 63 and a second VRAM area 64. One of the two compressed decompressed still image data 61 and 62 is stored in the first VRAM area 63, and the other is stored in the second VRAM area 64. In FIG. 2, the first VRAM area 63 stores the nth compressed decompression still image data 61, and the second VRAM area 64 stores the nth compressed decompression still image data 62.

[0049] When new captured image data is supplied from the color conversion processing unit 36, the still image storage processing unit 51 implemented in the DSP 29 stores the captured image data in the external memory 30 as still image data 39. save.

The developed still image generation unit 52 generates expanded still image data from the still image data 39 stored in the external memory 30. Expanded still image data is image data obtained by expanding a circular image in a captured image into a 360-degree V-Rama image. The expanded still image generating unit 52 generates and stores expanded still image data alternately in the first VRAM area 63 and the second VRAM area 64.

[0051] The compressed still image generation unit 53 compresses the expanded still image data stored in the first VRAM area 63 or the second VRAM area 64. For compressing the developed still image data, an image compression method such as a PEG (Joint Photographic Coding Experts Group) compression method or an MPEG (Moving Picture Coding Experts Group) compression method may be used. As a result, the compressed expanded still image data 61 is stored in the first VRAM area 63, and the compressed expanded still image data 62 is stored in the second VRAM area 64. [0052] The streaming generation unit 54 reads the compressed and expanded still image data 61 and 62 and the audio data 38 from the external memory 30, and generates streaming data having these content data. For streaming data, a streaming format such as MPEG or div-X (divix) may be adopted.

The communication processing unit 56 controls data communication using the USB connector 23.

The process management unit 55 manages the execution of the still image storage processing unit 51, the developed still image generation unit 52, the compressed still image generation unit 53, the streaming generation unit 54, the communication processing unit 56, and the like. The process management unit 55 instructs the still image storage processing unit 51, the developed still image generation unit 52, the compressed still image generation unit 53, the streaming generation unit 54, the communication processing unit 56, and the like to start or stop them.

[0055] The video encoder 32 reads the compressed decompressed still image data 61 and 62 from the external memory 30, and generates a video signal. Examples of video signals include NTSC (National TV Standards Committee) and PAL (Phase Alternating Line). The video encoder 32 outputs the generated video signal to the video output connector 24. A television receiver or the like can be connected to the video output connector 24. With a television receiver connected to the video output connector 24, the video signal can be reproduced and the video can be viewed.

Next, the operation of the panoramic unfolded image capturing system having the above configuration will be described.

[0057] When power is supplied to the 360-degree video camera device 1 from, for example, the USB cable 2, the DP S29 and the like operate, and the 360-degree video camera device 1 includes a color conversion processing unit 36, an audio storage processing unit 37, and a stationary state. An image storage processing unit 51, a developed still image generation unit 52, a compressed still image generation unit 53, a streaming generation unit 54, a process management unit 55, and the like are realized. The process management unit 55 instructs the still image storage processing unit 51, the developed still image generation unit 52, the compressed still image generation unit 53, and the streaming generation unit 54 to start them.

[0058] On the image sensor 27 of the 360-degree video camera device 1, an image formed by the light collected by the fisheye lens 22 is formed. The image sensor 27 generates luminance distribution data including the luminance distribution of the circular image. The color conversion processing unit 36 generates captured image data having a circular image as illustrated in FIG. 4 from the luminance distribution data using a color conversion table (not shown). The still image storage processing unit 51 converts the captured image data into the external memory 3 as still image data 39. Save to 0.

[0059] Further, the image sensor 27 periodically generates luminance distribution data. Therefore, the still image data 39 in the external memory 30 is updated to new captured image data every predetermined period.

When the still image data 39 in the external memory 30 is updated, the developed still image generating unit 52 generates expanded still image data from the updated still image data 39.

[0061] The expanded still image generation unit 52 first reads the inner circle boundary data 42 and the outer circle boundary data 43 from the EEPROM 41, and generates a panoramic expanded image in the captured image based on the updated still image data 39. Determine the range of the ring shape to do. In addition, the developed still image generation unit 52 determines two division ranges obtained by equally dividing the ring-shaped image range into two at 180 degrees. In FIG. 5, dotted lines 71 and 72 for horizontally dividing the ring-shaped image range into two are shown. The developed still image generation unit 52 divides the ring-shaped image between the outer circle boundary and the inner circle boundary into two parts every 180 degrees with these two dotted lines 71 and 72, for example, and two upper and lower divided images. For each of the ranges 73 and 74, the divided development image generation processing is executed.

FIG. 7 is a flowchart showing the flow of a divided expanded image generation process by the expanded still image generation unit 52 for one divided image range 73 (74). The developed still image generating unit 52 first confirms whether or not the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 have been updated after the last generated generated image (step ST1).

[0063] When the inner circle boundary data 42 and the outer circle boundary data 43 have been updated after the last developed image is generated (in the case of No in step ST), the developed still image generating unit 52 displays the divided developed image. The direction in each divided pixel range 73 (74) of each developed pixel column is calculated to generate the direction table 46 (step ST2).

FIG. 8 is a diagram illustrating an example of an array of a plurality of display pixels of the divided development image. In order to simplify the explanation, a plurality of display pixels are arranged in 3 rows x 4 columns in this divided expanded image. In this case, the developed still image generation unit 52 has a ring shape for each of four developed pixel columns (1, Z) 81, (2, Z) 8 2, (3, Z) 83, (4, Z) 84. The direction in the image (direction in the divided image range 73 (74)) is calculated.

[0065] FIG. 9 shows the direction of the developed pixel columns 81, 82, 83, 84 in FIG. FIG. 16 is an explanatory diagram illustrating a direction in a divided image range 73 (74). The developed still image generation unit 52 calculates the angle obtained by equally dividing 180 degrees (= 360 degrees ÷ 2) by the number of columns (= 4), assuming the intersection of the two axes in Fig. 9 as the center of the circular image. .

As shown in FIG. 9, if, for example, the first developed pixel row (1, Z) 81 in FIG. 8 is in the direction of the angle θ 1, the developed still image generating unit 52 The angle of the expanded pixel row (2, Z) 82 is calculated as “Θ 1 + uniform division angle”. The expanded still image generation unit 52 calculates the angle of the third expanded pixel column (3, Z) 83 as “Θ 1 + equal division angle X 2”. The expanded still image generating unit 52 calculates the angle of the fourth expanded pixel column (4, Z) 84 as “Θ 1 + equal division angle X 3”.

The developed still image generating unit 52 calculates the direction of each of the developed pixel columns 81, 82, 83, and 84 as described above. The developed still image generating unit 52 generates a direction table 46 in which each direction is associated with a plurality of developed pixel columns 81, 82, 83, and 84. The developed still image generation unit 52 stores the generated direction table 46 in the EEPROM 41. The direction table 46 may be stored in the external memory 30 or the like.

[0068] After generating the direction table 46, the expanded still image generating unit 52 specifies the first display pixel of the divided expanded image (step ST3). The expanded still image generation unit 52 specifies, for example, the upper left display pixel ((1, 1) in FIG. 8) of the divided expanded image. Even when the inner circle boundary data 42 and the outer circle boundary data 43 are not updated (Yes in step ST1), the developed still image generation unit 52 identifies the first display pixel of the divided developed image (step ST3).

[0069] Then, the developed still image generation unit 52 specifies a pixel in the captured image corresponding to the specified display pixel, acquires a pixel value of the pixel in the specified captured image, and acquires the acquired pixel value. Is stored as the pixel value of the specified display pixel.

[0070] Specifically, the developed still image generation unit 52 first refers to the image height correction table 45 and calculates the distance R from the center of the original image (circular image) for the specified display pixel. (Step ST4). For example, referring to the image height correction table 45, the developed still image generation unit 52 compares one developed pixel row of the divided developed image with a pixel array in the radial direction within the ring shape range of the captured image. The distance R from the center of the donut at the position of the original image (circular image) corresponding to the specific display pixel is calculated. Next, the developed still image generation unit 52 uses the direction of the specific display pixel stored in the direction table 46 and the calculated distance R to perform the horizontal display of the specific display pixel in the original image (circular image). Calculate the coordinates (step ST5). The developed still image generation unit 52 calculates the ordinate in the original image (circular image) of the specific display pixel from the direction of the specific display pixel stored in the direction table 46 and the calculated distance R (step) ST6).

FIG. 10 is an explanatory diagram of the coordinate position in the original image (circular image) of the specific display pixel. In Fig. 10, the center of the original image (circular image) (IOX, IOY) force is the center of the inner circle boundary and the center of the outer circle boundary. In other words, (IOX, IOY) is the center of the donut with the inner and outer circle boundaries. If the distance from the center of the specific display pixel is R and the direction is Θ, the coordinate position (IX, IY) of the specific display pixel in the original image (circular image) is Is calculated by The coordinate position (IX, IY) is based on the upper left of the original image (circular image).

[0073] IX = IOX + RX cos Θ · · · · Equation 1

IY = IOY-R X sin 0… Formula 2

[0074] When the coordinate position (IX, IY) of the specific display pixel in the original image (circular image) is calculated, the developed still image generating unit 52 generates the pixel 91 of the original image (circular image) at that coordinate. The value is acquired and stored as the pixel value of the specified display pixel. The developed still image generation unit 52 stores the pixel value of the specified display pixel in the first VRAM area 63 or the second VRAM area 64 of the external memory 30 (step ST7).

[0075] Through the processing from step ST4 to ST7, the developed still image generation unit 52 determines and stores the pixel value of one display pixel specified in step ST3. Thereafter, the developed still image generating unit 52 determines whether or not the display pixels for which the pixel value has not been determined remain among the plurality of display pixels of the divided expanded image, determines the pixel value, and determines the pixel value. Until the number of display pixels is exhausted, the process for determining the pixel value of the above display pixels is repeated.

Specifically, for example, the developed still image generating unit 52 first determines whether or not the determination of the pixel value for the display pixels for one column of the divided expanded image has been completed (step ST8). If one column has not been completed (No in step ST8), the developed still image generation unit 52 identifies the display pixel of the next row in the same column (step ST9), and Pixel Determine the value (steps ST4 to ST7).

[0077] When the determination of all the pixel values for the display pixels for one column is completed! /, (When YES in step ST8), the expanded still image generation unit 52 further determines all of the divided expanded images. It is determined whether or not the force has been determined for the pixel value for the column (step ST10). If the processing has not been completed for all columns (No in step ST10), the developed still image generation unit 52 identifies the display pixel at the upper end (top row) of the next column (step ST11). Then, the pixel value of the specific display pixel is determined (steps ST4 to ST7).

[0078] When the determination of the pixel values is completed for all the columns of the divided expanded image (Yes in step ST10), the expanded still image generation unit 52 performs the divided expanded image generation processing of FIG. finish. Thus, the divided expanded image generation process for one divided image range 73 (74) is completed.

[0079] Thereafter, the developed still image generation unit 52 executes a divided expanded image generation process for the remaining one divided image range 74 (73). As a result, two divided expanded images based on the still image data 39 are stored in the first VRAM area 63 or the second VRAM area 64 of the external memory 30.

[0080] When two divided expanded images are stored in the first VRAM area 63 or the second VRAM area 64 of the external memory 30, the compressed still image generating unit 53 compresses the two divided expanded images. As a result, the compressed and expanded still image data 61 and 62 are stored in the first VRAM area 63 or the second VRAM area 64.

[0081] The streaming generation unit 54 reads the compressed decompressed still image data 61 and 62 in order from the first VRAM area 63 and the second VRAM area 64, and also reads the audio data 38 from the external memory 30. Generate streaming data with content data. In the streaming data, the two divided expanded images are arranged one above the other.

The streaming generation unit 54 supplies the generated streaming data to the communication processing unit 56. The communication processing unit 56 of the 360-degree video camera apparatus 1 transmits streaming data to the communication processing unit 15 of the PC 3 via the USB connector 23, the USB cable 2, and the USB connector 11 of the PC 3.

[0083] The communication processing unit 15 of the PC 3 supplies the received streaming data to the reproduction processing unit 16. The The playback processing unit 16 of the PC 3 extracts compressed decompressed still image data from the streaming data and decodes it, and supplies the data of the two divided expanded images to the liquid crystal display device 12 as display data. The liquid crystal display device 12 displays two divided expanded images. In addition, the playback processing unit 16 of the PC 3 extracts the audio data with the streaming data power and supplies it to the speaker 13. The speaker 13 outputs a sound based on the extracted audio data.

FIG. 11 is a display example of a 360-degree panoramic developed image displayed on the liquid crystal display device 12 of the PC 3 by imaging. On the liquid crystal display device 12, a 360-degree image expanded by imaging is displayed by two divided expanded screens 101 and 102 arranged vertically. Since the two divided development screens 101 and 102 are arranged one above the other, the display screen of the liquid crystal display device 12 can be used more effectively than when these are one development image, for example. In addition, a 360-degree panoramic image can be displayed large on the liquid crystal display device 12.

[0085] It should be noted that the two split development screens 101 and 102 in FIG. 11 have two split developments every 180 degrees obtained by dividing the ring-shaped image range of FIG. 5 into two by dotted lines 71 and 72. An image is displayed. The two split developed images are continuous with each other at both ends, and a 360-degree panoramic image is displayed seamlessly as a whole. With this display, the user of the PC 3 can view the 360-degree video image developed by the 360-degree video camera device 1. Further, the user of the PC 3 can hear the sound acquired by the 360-degree video camera device 1 by the sound emitted from the speaker 13.

Note that, as described above, the image sensor 27 of the 360-degree video camera device 1 generates brightness distribution data for each period. The color conversion processing unit 36 generates captured image data from the luminance distribution data. The still image storage processing unit 51 updates the still image data 39 in the external memory 30 with the captured image data generated every period. The developed still image generation unit 52 reads still image data 39 from the external memory 30, and alternately generates two divided expanded image data based on the read still image data 39 in the first VRAM area 63 and the second VRAM area 64. Write. The streaming generation unit 54 reads the compressed data of the two divided expanded images written in the first VRAM area 63 or the second VRAM area 64, and generates streaming data. [0087] Therefore, the streaming generation unit 54 power generates a decompressed still image while reading the data of two compressed expanded images from one of the first VRAM area 63 and the second VRAM area 64. The unit 52 can generate two divided expanded images in the other VRAM area, and the compressed still image generating unit 53 can compress the two divided expanded images in the other VRAM area. The streaming generation unit 54 can assign the two divided development images based on the luminance distribution data generated by the image sensor 27 every period to the streaming data without causing image omission (frame omission). Accordingly, the moving image based on the luminance distribution data periodically generated by the image sensor 27 is displayed on the liquid crystal display device 12 of PC3.

[0088] Next, an operation for auto-panning the V-llama development image will be described.

The command generation unit 17 of the PC 3 generates an auto pan instruction command based on the input data input from the input device 14. The communication processing unit 15 of the PC 3 transmits the auto pan instruction command to the communication processing unit 56 of the 360-degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360-degree video camera device 1 supplies the received autopan command to the processing management unit 55.

When the auto pan instruction command is supplied, the process management unit 55 periodically updates the direction table 46 stored in the EEPROM 41. Specifically, the process management unit 55 periodically repeats the process of adding a predetermined constant angle to the data in a plurality of directions for each developed pixel column in the direction table 46.

[0091] Thereby, the developed still image generation unit 52 periodically increases the direction of the specific display pixel by a predetermined constant angle in the flowchart of FIG. For example, the angle 01 of the developed pixel row (1, Z) 81 in the first row in FIG. 9 periodically increases by a predetermined constant angle. The direction of the specific display pixel used in steps ST5 and ST6 in FIG. 7 also periodically increases by a predetermined constant angle.

As a result, the coordinate position of each specific display pixel in the original image (circular image) rotates by a certain angle, and the pixel value of each specific display pixel also changes according to the rotation. As a result, the subject in the original image (circular image) moves sideways in the divided development image. Auto-pan operation is realized in 360-degree images. Next, the operation for adjusting the height of the panoramic image will be described.

The command generation unit 17 of the PC 3 generates a change mode start instruction command based on the input data input from the input device 14. The communication processing unit 15 of the PC 3 transmits a change mode start instruction command to the communication processing unit 56 of the 360-degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360-degree video camera device 1 supplies the received change mode start instruction command to the processing management unit 55.

FIG. 12 is a flowchart showing the operation of the process management unit 55 in the change mode.

When the change mode start instruction command is supplied (step ST21), the process management unit 55 supplies a boundary display instruction to the stream generation unit 54 (step ST22). After that, the process management unit 55 waits to receive a boundary change instruction command and a change mode end instruction command (steps ST23 and ST24).

The streaming generation unit 54 to which the boundary display instruction is supplied reads the still image data 39 from the external memory 30. Every time the still image data 39 is read, the streaming generation unit 54 reads the inner circle boundary data 42 and the outer circle boundary data 43 from the EEPROM 41 following the reading. As shown in FIG. 5, the streaming generation unit 54 generates a display image in which the inner circle boundary mark 47 and the outer circle boundary mark 48 are superimposed on the captured image of the still image data 39. Note that the streaming generation unit 54 may read the inner circle boundary data 42 and the outer circle boundary data 43 each time the still image data 39 is read a plurality of times.

Further, the streaming generation unit 54 generates streaming data including the generated display image data, and supplies the streaming data to the communication processing unit 56. The communication processing unit 56 of the 360-degree video camera device 1 transmits the streaming data to the communication processing unit 15 of the PC 3 via the USB cable 2.

The communication processing unit 15 of the PC 3 supplies the received streaming data to the reproduction processing unit 16. The reproduction processing unit 16 of the PC 3 extracts display image data from the streaming data and supplies it to the liquid crystal display device 12 as display data. As shown in FIG. 5, the liquid crystal display device 12 displays a captured image in which a circular mark 47 on the inner circle boundary and a circular mark 48 on the outer circle boundary are superimposed. [0099] The command generation unit 17 of the PC 3 moves the positions of the inner circle boundary and the outer circle boundary, and enlarges / reduces the radii of the inner circle boundary and the outer circle boundary based on a user operation on the input device 14. A boundary change instruction command is generated. The communication processing unit 15 of the PC 3 transmits a boundary change instruction command to the communication processing unit 56 of the 360-degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360 degree video camera apparatus 1 supplies the received boundary change instruction command to the processing management unit 55.

[0100] When the boundary change instruction command is received (Yes in step ST23), the process management unit 55 first determines whether the instruction is appropriate (step ST25). Specifically, the process management unit 55 determines whether, for example, the radius of the inner circle boundary after the change or the radius of the outer circle boundary after the change is equal to or larger than a predetermined radius (for example, 144 pixels).

[0101] If it is determined that the instruction is appropriate (Yes in step ST25), the process management unit 55 determines that the inner circle boundary data 42 and the outer circle boundary stored in the EEPROM 41 are based on the change instruction. The boundary data 43 is updated (step ST26). Further, the process management unit 55 updates the image height correction table 45 stored in the EEPROM 41 based on the new combination of the inner circle boundary and the outer circle boundary (step ST27).

[0102] When the distance between the inner circle boundary and the outer circle boundary changes due to the change, the number of pixels between the inner circle boundary and the outer circle boundary in the captured image changes. As a result, the range of the space imaged by the plurality of pixels changes, and the division ratio of the imaging space by the plurality of pixels also changes.

[0103] When the fisheye lens 22 of the three-dimensional projection method is used, the subject forms an image with a different size depending on the angle of view. Therefore, as described in FIG. 6, even if the distance between the inner circle boundary and the outer circle boundary has not changed due to the update, the position of the inner circle boundary and the outer circle boundary only moves, and the imaging is performed. The division ratio of a plurality of pixels in the space changes.

The processing management unit 55 updates the image height correction table 45 in accordance with the change in the division ratio of the plurality of pixels accompanying the change between the inner circle boundary and the outer circle boundary. Thereafter, the process management unit 55 returns to the reception waiting state for the boundary change instruction command and the change mode end instruction command (steps ST23 and ST24).

[0105] If the indication is not appropriate (No in step ST25), for example, the radius of the inner circle boundary is If the radius is smaller than the predetermined radius, process management unit 55 ignores the change instruction and returns to a waiting state for receiving a boundary change instruction command and a change mode end instruction command (steps ST23 and ST24).

By the way, the streaming generation unit 54 to which the boundary display instruction is supplied reads the inner circle boundary data 42 and the outer circle boundary data 43 from the EEPROM 41 each time the still image data 39 is read from the external memory 30. Therefore, when the process management unit 55 updates the inner circle boundary data 42 and the outer circle boundary data 43, the streaming generation unit 54 also updates the position and size of the overlap of the circular marks 47 and 48 with respect to the captured image. The positions and sizes of the circular marks 47 and 48 displayed on the liquid crystal display device 12 so as to overlap the captured image are changed according to the boundary change instruction command. The user can know that the inner circle boundary data 42 and the outer circle boundary data 43 have been updated by the display on the liquid crystal display device 12.

The command generation unit 17 of the PC 3 generates a change mode end instruction command based on a user operation on the input device 14. The communication processing unit 15 of the PC 3 transmits a change mode end instruction command to the communication processing unit 56 of the 360-degree video camera device 1 via the USB cable 2. The communication processing unit 56 of the 360-degree video camera apparatus 1 supplies the received change mode end instruction command to the process management unit 55. When the change mode end instruction command is received (Yes in step ST24), the process management unit 55 ends the process of the change mode in FIG. After the processing in this change mode, the inner circle boundary data 42, the outer circle boundary data 43, and the image height correction table 45 are changed to new ones, and the developed still image generation unit 52 of the 360-degree video camera device 1 performs this change. Using this data, a 360-degree no-V llama expansion image is generated.

As described above, according to this embodiment, a circular image by the fisheye lens 22 is formed on the image sensor 27 of the 360-degree video camera device 1. The still image storage processing unit 51 generates captured image data having a circular image based on the luminance distribution data of the image sensor 27. This circular image includes the subject's head without being cut off.

[0109] Further, the developed still image generating unit 52 uses two divided developed images based on an image within the range of the ring shape sandwiched between the inner circle boundary and the outer circle boundary in the captured image. 360 A panorama unfolded image is generated. The position and size of the inner and outer circle boundaries are

The processing management unit 55 can freely set the user.

Therefore, by updating the position or size of the inner circle boundary or outer circle boundary by the process management unit 55, it is possible to adjust the subject in the circular image to be within the ring shape range. As a result, for example, even when the subject is close to the fisheye lens 22 and the physical distance of the fisheye lens 22 to the subject cannot be ensured, by adjusting the position of the inner circle boundary, etc. It can generate a 360-degree image with no head cut. In addition, for example, it is possible to generate a 360-degree expanded image that displays a person's head and body as a subject in a well-balanced manner.

[0111] In addition, two horizontally long divided developed images of every 180 degrees are displayed on the liquid crystal display device 12 of the PC 3 side by side. Therefore, the display screen of the liquid crystal display device 12 can be used more effectively than if, for example, a 360-degree panoramic development image is a single development image. The liquid crystal display device 12 can display a large 360-degree image of the V llama.

[0112] In this embodiment, the fisheye lens 22 is a stereoscopic projection system. Compared to the case of using a regular equidistant projection fisheye lens, the stereoscopic projection fisheye lens 22 has less distortion at the periphery in the image formed by the fisheye lens 22, and the amount of information at the periphery. Will increase. The EEPROM 41 also stores an image height correction table 45 that corrects the balance of the imaging size of the subject obtained based on the relationship between the incident angle and the image height of the fisheye lens 22 of this stereoscopic projection method! To do. The developed still image generation unit 52 refers to the image height correction table 45 and generates a 360-degree panoramic developed image in which the balance of the imaging size of the subject is adjusted.

[0113] Therefore, when the 360-degree video camera device 1 is placed on the table with the fisheye lens 22 facing upward, it is possible to obtain an image of a person or blackboard around the high-resolution image. Characters on the blackboard can be reproduced in a 360-degree image. It is also possible to obtain a well-balanced image of a person or blackboard around the fisheye lens 22.

In this embodiment, the image height correction table 45 stored in the EEPROM 41 is It is updated each time the inner circle boundary data 42 or the outer circle boundary data 43 is updated. When the distance between the inner circle boundary and the outer circle boundary changes due to the change, the division ratio of the image space between them changes by a plurality of pixels. Even if the positions of the inner circle boundary and the outer circle boundary only move, the division ratio of the plurality of pixels in the image space between them changes. Thus, by updating the image height correction table 45 each time the inner circle boundary data 42 or the outer circle boundary data 43 is updated, the image height correction table 45 is maintained to be compatible with these.

In this embodiment, the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 are updated when the changed size is a radius of 144 pixels or more. For example, if the radius of the inner circle boundary becomes 1 pixel, the same pixel in the captured image will be used many times to obtain the pixel values of the top display pixels of the 360 ° panorama development image. It's a little bit. In this case, the load of generating a 360-degree panoramic image increases without improving the image quality of the 360-degree image. By preventing the radius of the inner circle boundary or outer circle boundary from becoming smaller than 144 pixels as in this embodiment, it is possible to prevent such disadvantages from occurring.

In this embodiment, the developed still image generation unit 52 calculates the distance R and the direction from the center of the circular image of each display pixel of the 360-degree panoramic developed image, and calculates the calculated distance R and Using the direction, the coordinates (IX, IY) in the captured image of each display pixel are calculated, and the pixel value of the pixel 91 in the captured image located at each calculated coordinate (IX, IY) is calculated for each display pixel. Obtained as a pixel value.

Therefore, it is possible to generate a 360-degree no Vrama developed image using the pixel values of the pixels of the circular image in the captured image as they are. There is no need to calculate the pixel value of each display pixel in the 360-degree llama image. As in the case of obtaining the pixel value of each display pixel by calculation, image quality deterioration due to calculation does not occur. In addition, the generation time of one panorama development image can be shortened. As a result, a 360-degree panoramic image can be generated in a short time. In addition, by continuously repeating the panorama development image generation process, it is possible to obtain a moving image with a 360 degree normal image. [0118] As described above, this embodiment is an example of a preferred embodiment of the present invention.

1S The present invention is not limited thereto, and various modifications and changes can be made without departing from the scope of the invention.

[0119] For example, in the above-described embodiment, the fish-eye lens 22 uses a stereoscopic projection type. In addition to this, for example, a general equidistant projection method, an isometric projection method, an orthographic projection method, or the like may be used as the fisheye lens 22.

[0120] In the above-described embodiment, the developed still image generating unit 52 generates two divided expanded images every 180 degrees as a 360-degree panoramic expanded image. In addition to this, for example, the developed still image generating unit 52 may generate one 360-degree panoramic expanded image, or may be divided into three or four.

In the above embodiment, the process management unit 55 updates both the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 based on a user operation. In addition, for example, the process management unit 55 may update only one of the inner circle boundary data 42 and the outer circle boundary data 43 stored in the EEPROM 41 based on a user operation.

In the above embodiment, the PC 3 is connected to the 360 degree video camera apparatus 1 by the USB cable 2. In addition to this, for example, the 360 degree video camera apparatus 1 may be connected to an input board for outputting a command, for example, via a USB cable 2 or the like.

[0123] In the above embodiment, the PC 3 connected to the 360-degree video camera device 1 displays a moving image based on a 360-degree panoramic image on the liquid crystal display device 12. In addition to this, for example, PC3 should store a 360-degree video image in a storage device (not shown).

Industrial applicability

[0124] The present invention can be widely used as a panorama developed image capturing apparatus, a panorama expanded image capturing system, and the like for capturing an image of a meeting.

Claims

The scope of the claims

[1] A panoramic unfolding image capturing device that generates 360-degree panoramic unfolding images.

With a fisheye lens,

An imaging means for forming a circular image by the fisheye lens on a light receiving surface on which a plurality of light receiving elements are arranged, and generating data of a captured image having a circular image;

Based on the user's operation and memory that stores the position or size of the inner and outer circle boundaries that define the range of the ring shape used to generate the 360-degree V-Rama development image in the captured image. User update means for updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory, and stored in the memory in the captured image generated by the imaging means. Panorama unfolded image generating means for generating a 360-degree panorama unfolded image based on an image within a ring-shaped range sandwiched between the inner circle boundary and the outer circle boundary;

A panoramic developed image imaging device characterized by comprising:

[2] The fisheye lens is of a stereoscopic projection type,

The memory stores an image height correction table for correcting a balance of the imaging size of a subject obtained based on a relationship between an incident angle and an image height of the stereoscopic projection fisheye lens, and the panoramic developed image generation unit includes: Referencing the above image height correction table, generating a 360 degree no-llama development image that balances the image size of the subject,

The panoramic unfolded image capturing apparatus according to claim 1, wherein:

[3] The user update means updates the image height correction table stored in the memory after updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory. 3. The panorama development image according to claim 2, wherein the panorama development image is updated based on the updated inner circle boundary and outer circle boundary, and a relationship between an incident angle and an image height of the stereoscopic eye fisheye lens. Imaging device.

[4] The user updating means, when the size force after the change of the inner circle boundary or the outer circle boundary is a radius greater than a predetermined number of pixels, is based on an instruction to change the inner circle boundary or the outer circle boundary. The inner circle boundary stored in the memory is the outer circle boundary. 2. The panoramic unfolded image capturing apparatus according to claim 1, wherein

[5] The panoramic expanded image generation means calculates a distance and a direction of each display pixel of the 360-degree panoramic expanded image from the center of the circular image, and uses the calculated distance and direction to The coordinates of the pixel in the captured image are calculated, and the pixel value of the pixel of the captured image located at each calculated coordinate is obtained as the pixel value of each display pixel. The panoramic development image capturing apparatus described.

[6] The panoramic expanded image generation means divides a range between the inner circle boundary and the outer circle boundary in the captured image into two ranges every 180 degrees, and the range of the divided image Generate two horizontally wide V llama divided unfolded images for each

Having display data generating means for generating display data for displaying the above two horizontally elongated V-llama divided development images vertically;

The panoramic unfolded image capturing apparatus according to claim 1, wherein:

[7] A panoramic unfolded image pickup device that generates a 360-degree panoramic unfolded image by imaging and a computer device that displays or stores the panorama unfolded image generated by the panorama unfolded image pick-up device. An imaging system,

The panoramic expanded image imaging device

With a fisheye lens,

Based on the user's operation and the memory that stores the position or size of the inner circle boundary and outer circle boundary that define the range of the ring shape used to generate the 360-degree NORMAL development image in the captured image. User update means for updating the position or size of at least one of the inner circle boundary and the outer circle boundary stored in the memory, and stored in the memory in the captured image generated by the imaging means. Panorama unfolded image generating means for generating a 360-degree panoramic unfolded image based on an image within a ring-shaped range sandwiched between the inner circle boundary and the outer circle boundary;

A panoramic unfolded image capturing system characterized by comprising: