WO2016038886A1

WO2016038886A1 - Imaging device, image processing device, and imaging method

Info

Publication number: WO2016038886A1
Application number: PCT/JP2015/004566
Authority: WO
Inventors: Norikatsu Niinami
Original assignee: Ricoh Company, Limited
Priority date: 2014-09-08
Filing date: 2015-09-08
Publication date: 2016-03-17
Also published as: JP2016058840A

Abstract

　 An imaging device of an embodiment includes an imaging unit that captures an image, a segmentation unit that segments the image into a plurality of segmented images, a determination unit that determines a scene of each of the segmented images, and a processing unit that processes each of the segmented images by a correction parameter according to the scene.

Description

IMAGING DEVICE, IMAGE PROCESSING DEVICE, AND IMAGING METHOD

　　　　　 The present invention relates to an imaging device, an image processing device, and an imaging method.

　　　　　 A technique to image all directions of an imaging spot and form them into one image (hereinafter referred to as the “whole celestial sphere image”) is known.　 Patent Literature 1 discloses, for example, an invention of an image processing device that can highly accurately detect a backlight area from a whole celestial sphere image and appropriately correct backlight.

　　　　　 In the related art, however, it has been difficult to generate an optimum quality image when a plurality of scenes is included in an image.

　　　　　 In view of the above, there is a need to provide an imaging device, an image processing device, and an imaging method which can generate an optimum quality image even when a plurality of scenes is included in an image.

　　　　　 According to an embodiment of the present invention, there is provided an imaging device comprising: an imaging unit that captures an image; a segmentation unit that segments the image into a plurality of segmented images; a determination unit that determines a scene of each of the segmented images; and a processing unit that processes each of the segmented images by a correction parameter according to the scene.

　　　　　 The above and other objects, features, advantages and technical and industrial significance of this invention will be better understood by reading the following detailed description of presently preferred embodiments of the invention, when considered in connection with the accompanying drawings.

FIG. 1 is a pattern diagram illustrating an exemplary imaging device of an embodiment of the present invention. FIG. 2 is a diagram illustrating an exemplary configuration of the imaging device of the embodiment. FIG. 3 is a diagram illustrating an exemplary whole celestial sphere image of the embodiment. FIG. 4 is a diagram illustrating an exemplary segmented image of the embodiment. FIG. 5 is a diagram illustrating an exemplary correction parameter of the embodiment. FIG. 6 is a diagram illustrating an exemplary scene selected by the imaging device of the embodiment. FIG. 7 is a flowchart illustrating an exemplary method for operating the imaging device of the embodiment. FIG. 8 is a flowchart illustrating an exemplary indoor image/outdoor image determining method of the embodiment. FIG. 9 is a flowchart illustrating an exemplary boundary image determining method of the embodiment. FIG. 10 is a flowchart illustrating an exemplary image processing method of the embodiment. FIG. 11 is a diagram illustrating an exemplary hardware configuration of the imaging device of the embodiment.

　　　　　 An embodiment of an imaging device, an image processing device, and an imaging method will be described below with reference to the attached figures.　 In the present embodiment, the device for imaging a whole celestial sphere, using two imaging units, will be described.　 However, the present invention can be applied when a captured image includes a plurality of categories and scenes.　 That is, the present invention can also be applied to an imaging device such as an ordinary single-lens reflex camera and a smartphone, as long as an image can be segmented into a plurality of areas, and a category and a scene can be set for each segmented area.

　　　　　 FIG. 1 is a pattern diagram illustrating an imaging device 100 of the embodiment.　 The imaging device 100 of the embodiment includes an imaging element 101, a fish-eye lens 102, an imaging element 103, a fish-eye lens 104, a housing 121, and an imaging switch 122.　 The imaging element 101 converts light, having passed through the fish-eye lens 102 having an angle of view of 180 degrees or more in the front of the imaging device 100, into an electric signal.　 The imaging element 103 converts light, having passed through the fish-eye lens 104 having an angle of view of 180 degrees or more in the back of the imaging device 100, into an electric signal.　 The

imaging elements

101 and 103 include a complementary metal oxide semiconductor (CMOS) sensor, for example.　 The imaging switch 122 is provided on one side of the housing 121.　 Although not illustrated in FIG. 1, the imaging device 100 also includes various types of operation buttons, a power switch, a touch panel, and the like.

　　　　　 FIG. 2 is a diagram illustrating an exemplary configuration of the imaging device 100 of the embodiment.　 The imaging device 100 of the embodiment includes a first imaging unit 1, a second imaging unit 2, a storage unit 3, a combining unit 4, a first detection unit 5, a first correction unit 6, a segmentation unit 7, a determining unit 8, a determination unit 9, a processing unit 10, a second detection unit 11, a calculation unit 12, a second correction unit 13, a display unit 14, and a selection unit 15.

　　　　　 The first imaging unit 1 captures a first image.　 The first image is an image including an object in the front (0 to 180 degrees) of the imaging device 100.　 At this time, an angle of view photographed by the first imaging unit 1 is 180 degrees or more (preferably 180 degrees or more and 190 degrees or less).　 This first image photographed by the first imaging unit 1 has an area overlapping a second image in the back which will be described later.　 The first imaging unit 1 stores the first image in the storage unit 3.

　　　　　 The second imaging unit 2 captures the second image.　 The second image is an image including an object in the back (180 to 360 degrees) of the imaging device 100.　 At this time, an angle of view photographed by the second imaging unit 2 is 180 degrees or more (preferably 180 degrees or more and 190 degrees or less).　 This second image photographed by the second imaging unit 2 has an area overlapping the first image in the front described above.　 The second imaging unit 2 stores the second image in the storage unit 3.

　　　　　 The storage unit 3 stores the first and second images.　 The storage unit 3 also combines the first and second images and stores the generated whole celestial sphere image which will be described later.　 Furthermore, the storage unit 3 stores a category and a scene for each segmented image (image corresponding to a part of the areas of the whole celestial sphere image) which will be described later.　 The scene indicates the type of the segmented image, determined in accordance with an object included in the segmented image.　 The category is information on the segmented image classified in accordance with the scene.　 The category of the segmented image includes an outdoor image obtained by imaging outdoors, an indoor image obtained by imaging indoors, or a boundary image positioned at the boundary between the outdoor image and the indoor image.　 The storage unit 3 also stores the (type of) category and the (type of) scene included therein.　 In addition, the storage unit 3 stores a correction parameter used by the processing unit 10 which will be described later.　 The storage unit 3 stores the correction parameter for each scene.　 An example of the correction parameter stored for each scene will be described later with reference to FIG. 5.

　　　　　 The combining unit 4 generates the whole celestial sphere image showing images in all directions from the imaging spot by combining the first and second images by superimposing the overlapping areas.　 The combining unit 4 stores the whole celestial sphere image in the storage unit 3.

　　　　　 FIG. 3 is a diagram illustrating an example of the whole celestial sphere image 23 of the embodiment.　 FIG. 3 is an example of a case where the whole celestial sphere image 23 including the objects in all directions from the imaging spot is generated by combining the overlapping areas of the first image 21 and the second image 22 as a mark for each other.　 The whole celestial sphere image 23 is created by the equidistant cylindrical projection, Mercator projection, or the like.　 When this whole celestial sphere image 23 is pasted and displayed on a spherical object, the images in all directions will be displayed.　 The whole celestial sphere image 23 may be a moving image or a still image.

　　　　　 Referring back to FIG. 2, the first detection unit 5 detects a zenith direction of the imaging device 100 at the time of imaging (inclination of the imaging device 100 at the time of imaging).　 The first detection unit 5 detects the zenith direction of the imaging device 100 at the time of imaging by an acceleration sensor, for example.　 The first detection unit 5 inputs, into the first correction unit 6, information on zenith direction that indicates the zenith direction at the time of imaging.

　　　　　 The first correction unit 6 receives the information on zenith direction from the first detection unit 5.　 The first correction unit 6 reads the whole celestial sphere image from the storage unit 3.　 The first correction unit 6 corrects the whole celestial sphere image based on the information on zenith direction.　 Specifically, the first correction unit 6 corrects the zenith direction of the whole celestial sphere image by the rotation of the whole celestial sphere image or the movement of the position thereof.　 Accordingly, a deviation in the zenith direction of the whole celestial sphere image, attributable to the inclination of the imaging device 100 at the time of imaging, is corrected.　 The first correction unit 6 overwrites the whole celestial sphere image in the storage unit 3 with the whole celestial sphere image after correction.

　　　　　 The segmentation unit 7 reads, from the storage unit 3, the whole celestial sphere image with the deviation in the zenith direction corrected.　 The segmentation unit 7 segments the whole celestial sphere image into a plurality of segmented images.

　　　　　 FIG. 4 is a diagram illustrating an example of the segmented images of the embodiment.　 The segmentation unit 7 segments the whole celestial sphere image into 2N pieces of segmented images by segmenting the whole celestial sphere image 23 into N pieces (N is an integer equal to or greater than two) in a vertical direction, and two pieces in a horizontal direction.　 Specifically, the segmentation unit 7 first segments an area corresponding to the first image 21 into n pieces in strips (n = N/2), and by further segmenting the area segmented into strips into two parts, an upper half and a bottom half, obtains segmented images Fun and Fbn (1 ≦ n ≦ N/2).　 Similarly, the segmentation unit 7 segments an area corresponding to the second image 22 into n pieces in strips (n = N/2), and by further segmenting the area segmented into strips into two parts, an upper half and a bottom half, obtains segmented images Run and Rbn (1 ≦ n ≦ N/2).

　　　　　 Referring back to FIG. 2, the determining unit 8 reads, from the storage unit 3, the segmented images in the upper half of the whole celestial sphere image 23 (segmented images Fun (1 ≦ n ≦ N/2) and Run (1 ≦ n ≦ N/2)).　 The determining unit 8 determines a category of segmented image for each of the segmented images.　 Determining processing is processing of determining the category (outdoor, indoor, or boundary) of the segmented image, indicating any one of outdoor images obtained by imaging outdoors, indoor images obtained by imaging indoors, and boundary images positioned at the boundary between the outdoor images and the indoor images.　 Details of the determining processing will be described later with reference to FIGS. 8 and 9.　 The determining unit 8 determines, by the categories of the segmented images in the upper half, the category of each of the segmented images in the bottom half (segmented images Fbn (1 ≦ n ≦ N/2) and segmented images (Rbn (1 ≦ n ≦ N/2)), the position in the horizontal direction of which is the same as that of the upper half.　 The determining unit 8 associates the categories determined for the segmented images and stores them in the storage unit 3.

　　　　　 The determination unit 9 determines the scene of segmented image for each segmented image.　 Specifically, the determination unit 9 reads the segmented images and the categories thereof from the storage unit 3.　 The determination unit 9 determines the scene of the segmented image by recognizing the optimum scene among a plurality of scenes associated with the category.　 The scene is determined in order for the processing unit 10 described later to appropriately process each segmented image by using a correction parameter according to an object.　 Scenes in the indoor category include, for example, cooking, furniture, and the like.　 Scenes in the outdoor category include autumn leaf, snow, cloud, wood, and the like.　 In a case where the category of segmented image is boundary, the determination unit 9 does not determine the scene of the segmented image.　 The determination unit 9 stores the scene determined for each segmented image in the storage unit 3.　 Therefore, in the storage unit 3, the category and the scene are associated and stored for each segmented image.　 In a case where the category is boundary, the scene is stored as Null.

　　　　　 The processing unit 10 processes each segmented image by a correction parameter according to the scene.　 In a case where the category of the segmented image is boundary, the processing unit 10 processes the image by an intermediate-value correction parameter between a correction parameter used for the outdoor image adjacent to the boundary image and a correction parameter used for the indoor image adjacent to the boundary image.　 For example, in processing the boundary image between the outdoor image (autumn leaf) and the indoor image (cooking), an intermediate-value correction parameter between a correction parameter used in a case where the scene is autumn leaf, and a correction parameter used in a case where the scene is cooking, is used.　 As a result, it is possible to prevent inconsistencies in the area of the boundary image, caused by the effects of image processing using different correction parameters.　 The processing unit 10 overwrites the segmented image in the storage unit 3 (some area of the whole celestial sphere image) with the segmented image after image processing.

　　　　　 Here, an example of the correction parameter according to the scene will be described.

　　　　　 FIG. 5 is a diagram illustrating an example of the correction parameter of the embodiment.　 For example, in a case where the scene is cooking, the processing unit 10 processes the image by a correction parameter related to exposure setting to make the segmented image brighter and a correction parameter related to color matrix parameter (color reproduction correction) to increase the saturation of the segmented image.　 The actual values of the correction parameters may be set at appropriate values in accordance with, for example, photographing conditions.

　　　　　 Referring back to FIG. 2, the second detection unit 11 reads the segmented image after image processing from the storage unit 3.　 The second detection unit 11 detects the luminance of the segmented image.　 The second detection unit 11 inputs luminance information indicating the luminance of the segmented image into the calculation unit 12.

　　　　　 The calculation unit 12 receives luminance information of each of the segmented images from the second detection unit 11.　 The calculation unit 12 calculates, for each of the segmented images in the upper half and in the bottom half, a difference in luminance of the segmented images between the upper half and the bottom half, the positions in the horizontal direction of which are the same.　 The calculation unit 12 inputs, into the second correction unit 13, luminance-difference information indicating a difference in luminance of the segmented images between the upper half and the bottom half, the positions in the horizontal direction of which are the same.

　　　　　 The second correction unit 13 receives luminance-difference information from the calculation unit 12.　 The second correction unit 13 makes different corrections for gradation characteristics between the segmented images in the upper half and the segmented images in the bottom half, in a case where the number of sets of the segmented images in the upper half and the segmented images in the bottom half, the luminance difference between which is greater than a first threshold, is greater than a second threshold.　 As a result, a whole celestial sphere image with a wider D range of luminance can be obtained.　 The second correction unit 13 overwrites the whole celestial sphere image in the storage unit 3 with the whole celestial sphere image where different corrections have been made for gradation characteristics between the segmented images in the upper half and the segmented images in the bottom half.

　　　　　 The display unit 14 displays, for example, the conditions of the imaging device 100.　 The display unit 14 displays a monitoring screen which displays the conditions of all directions of the imaging spot at the time of imaging by the imaging device 100, for example.　 At this time, the processing in the first imaging unit 1, the second imaging unit 2, the storage unit 3, the combining unit 4, the first detection unit 5, the first correction unit 6, the segmentation unit 7, the determining unit 8, the determination unit 9, the processing unit 10, the second detection unit 11, the calculation unit 12, and the second correction unit 13 is executed in real time during monitoring.　 The whole celestial sphere image obtained as a result of the processing is displayed (reproduced) on the display unit 14.　 The display unit 14 also displays one or more types of scenes (autumn leaf, cooking, and the like) determined by the determination unit 9.

　　　　　 The selection unit 15 selects a scene from among one or more types of scenes displayed on the display unit 14 based on the operation by a user of the imaging device 100.　 At this time, the display unit 14 displays (reproduces), as one image, one or more segmented images recognized as the scenes.　 The storage unit 3 stores, as one image, one or more segmented images recognized as the scenes.　 As a result, only images including scenes the user of the imaging device 100 wishes to image can be displayed and stored.

　　　　　 FIG. 6 is a pattern diagram illustrating an exemplary scene selected by the imaging device 100.　 FIG. 6 illustrates a case where a dog is selected as a scene the user of the imaging device 100 wishes to image.　 At this time, the display unit 14 displays, as one image, one or more segmented images including the dog as an object (four images in the example of FIG. 6).　 The storage unit 3 stores, as one image, one or more segmented images including the dog as an object.

　　　　　 Next, a method for operating the imaging device 100 of the embodiment will be described.

　　　　　 FIG. 7 is a flowchart illustrating an exemplary method for operating the imaging device 100 of the embodiment.

　　　　　 First, the first imaging unit 1 captures the first image, and the second imaging unit 2 captures the second image (step S1).　 Next, the combining unit 4 generates the whole celestial sphere image showing images in all directions of the imaging spot by combining the first and second images (step S2).　 Next, the first correction unit 6 corrects the zenith direction of the whole celestial sphere image by the rotation of the whole celestial sphere image or the movement of the position thereof in accordance with the zenith direction (step S3).

　　　　　 Next, the segmentation unit 7 segments the whole celestial sphere image into a plurality of segmented images (step S4).　 Next, the determining unit 8 determines the category (indoor image/outdoor image) of segmented image for each of the segmented images (step S5).　 Details of the indoor image/outdoor image determining processing in step S4 will be described later with reference to FIG. 8.

　　　　　 Next, the determining unit 8 determines the category (boundary image) of segmented image for each of the segmented images (step S6).　 Details of the boundary image determining processing in step S6 will be described later with reference to FIG. 9.

　　　　　 Next, the determination unit 9 determines the scene of each segmented image by recognizing the optimum scene among a plurality of scenes associated with the category of the segmented image, and the processing unit 10 processes each segmented image by the correction parameter according to the scene (step S7).　 Details of the image processing in step S7 will be described later with reference to FIG. 10.

　　　　　 Next, the second correction unit 13 makes corrections for gradation characteristics of the whole celestial sphere image (step S8).　 Specifically, the second detection unit 11 first detects the luminance of the segmented image.　 Next, the calculation unit 12 calculates, for each of the segmented images in the upper half and in the bottom half, a difference in luminance of the segmented images between the upper half and the bottom half, the positions in the horizontal direction of which are the same.　 The second correction unit 13 makes different corrections for gradation characteristics between the segmented images in the upper half and the segmented images in the bottom half, in a case where the number of sets of the segmented images in the upper half and the segmented images in the bottom half, the luminance difference between which is greater than a first threshold, is greater than a second threshold.

　　　　　 Next, details of the indoor image/outdoor image determining method in step S5 will be described.

　　　　　 FIG. 8 is a flowchart illustrating an example of the indoor image/outdoor image determining method of the embodiment.　 In FIG. 8, the indoor image/outdoor image determining method of the segmented images Fun (1 ≦ n ≦ N/2) corresponding to the area in the upper half of the first image 21 and the segmented images Fbn (1 ≦ n ≦ N/2) corresponding to the area in the bottom half of the first image 21 will be described.

　　　　　 First, the determining unit 8 determines whether or not an image showing a blue sky is included in the segmented images Fun (1 ≦ n ≦ N/2) (step S21).　 The determining unit 8 compares, for example, the segmented images Fun with an image showing a blue sky stored in advance, and determines whether or not the image showing the blue sky is included in the segmented images Fun based on the similarity between the two images.　 If the image showing the blue sky is included (step S21, Yes), the determining unit 8 determines that the category of the segmented images Fun is outdoor (step S26).

　　　　　 If the image showing the blue sky is not included (step S21, No), the determining unit 8 determines whether or not an image showing a cloud is included in the segmented images Fun (step S22).　 If the image showing the cloud is included (step S22, Yes), the determining unit 8 determines that the category of the segmented images Fun is outdoor (step S26).

　　　　　 If the image showing the cloud is not included (step S22, No), the determining unit 8 determines whether or not an image showing a sunrise glow or a sunset glow is included in the segmented images Fun (step S23).　 If the image showing the sunrise glow or the sunset glow is included (step S23, Yes), the determining unit 8 determines that the category of the segmented images Fun is outdoor (step S26).

　　　　　 If the image showing the sunrise glow or the sunset glow is not included (step S23, No), the determining unit 8 determines whether an image showing a night sky is included in the segmented images Fun (step S24).　 If the image showing the night sky is included (step S24, Yes), the determining unit 8 determines that the category of the segmented images Fun is outdoor (step S26).

　　　　　 If the image showing the night sky is not included (step S24, No), the determining unit 8 determines that the category of the segmented images Fun is indoor (step S25).

　　　　　 The determining unit 8 determines the categories of the segmented images Fbn (1 ≦ n ≦ N/2) in the bottom half by using the segmented images Fun (1 ≦ n ≦ N/2) in the upper half, the position in the horizontal direction of which is the same as that of the bottom half.　 For example, if a segmented image Fu1 = outdoor, it is assumed that a segmented image Fb1 = outdoor, and if a segmented image Fu2 = indoor, it is assumed that a segmented image Fb2 = indoor.

　　　　　 The flowchart of FIG. 8 also applies to segmented images Run (1 ≦ n ≦ N/2) corresponding to the area in the upper half of the second image 22 and segmented images Rbn (1 ≦ n ≦ N/2) corresponding to the area in the bottom half of the second image 22.

　　　　　 Next, details of a boundary image determining method in step S6 will be described.

　　　　　 FIG. 9 is a flowchart illustrating an example of the boundary image determining method of the embodiment.　 In FIG. 9, a case will be described, where the boundary of the area in the upper half of the first image 21 (segmented images Fun (1 ≦ n ≦ N/2)) is determined.　 The flowchart of FIG. 8 also applies to the area in the bottom half of the first image 21 (segmented images Fbn (1 ≦ n ≦ N/2)), the area in the upper half of the second image 22 (segmented images Run (1 ≦ n ≦ N/2)), and the area in the bottom half of the second image 22 (segmented images Rbn (1 ≦ n ≦ N/2)).

　　　　　 First, the determining unit 8 assigns a value of 1 to a variable x (step S41).　 Next, the determining unit 8 determines whether or not Fux(S) = indoor and Fux+1(S) = outdoor (step S42).　 Here, Fux(S) indicates the category (outdoor or indoor) of the segmented image Fux, determined by the flowchart of FIG. 8.

　　　　　 If Fux(S) = indoor and Fux+1(S) = outdoor (step S42, Yes), the determining unit 8 determines that Fux(S) = boundary (step S44).　 Next, the processing proceeds to step S45.

　　　　　 If Fux(S) = indoor and Fux+1(S) = outdoor do not apply (step S42, No), the determining unit 8 determines whether or not Fux(S) = outdoor and Fux+1(S) = indoor (step S43).

　　　　　 If Fux(S) = outdoor and Fux+1(S) = indoor (step S43, Yes), the determining unit 8 determines that Fux(S) = boundary (step S44).　 Next, the processing proceeds to step S45.

　　　　　 If Fux(S) = outdoor, and Fux+1(S) = indoor do not apply (step S43, No), the processing proceeds to step S45.

　　　　　 Next, the determining unit 8 determines whether or not x + 1 = N/2 (step S45).　 If x + 1 = N/2 does not apply (step S45, No), the determining unit 8 assigns x + 1 to x (step S46) and the processing returns to step S42.　 If x + 1 = N/2 (step S45, Yes), the processing is finished.

　　　　　 Next, details of the image processing method in step S7 will be described.　 In FIG. 10, a case will be described, where the image of the area in the upper half of the first image 21 (segmented images Fun (1 ≦ n ≦ N/2)) is processed.　 The flowchart of FIG. 10 also applies to the area in the bottom half of the first image 21 (segmented images Fbn (1 ≦ n ≦ N/2)), the area in the upper half of the second image 22 (segmented images Run (1 ≦ n ≦ N/2)), and the area in the bottom half of the second image 22 (segmented images Rbn (1 ≦ n ≦ N/2)).

　　　　　 FIG. 10 is a flowchart illustrating an example of the image processing method of the embodiment.　 First, the determination unit 9 assigns a value of 1 to a variable x (step S61).　 Next, the determination unit 9 determines whether or not Fux(S) = outdoor (step S62).　 Here, Fux(S) indicates the category (outdoor, indoor, or boundary) of the segmented image Fux, determined by the flowcharts of FIGS. 8 and 9.

　　　　　 If Fux(S) = outdoor (step S62, Yes), the determination unit 9 recognizes a scene among the scenes in the outdoor category (step S63) and determines the scene of the segmented image Fux.　 Scenes in the outdoor category include, for example, autumn leaf, snow, cloud, wood, and the like.　 Next, the processing unit 10 subjects the segmented image Fux to image processing optimized for each scene (step S64).　 Specifically, the processing unit 10 processes the segmented image Fux by using a correction parameter according to the scene determined in step S62.　 Next, the processing proceeds to step S69.

　　　　　 If Fux(S) = outdoor does not apply (step S62, No), the determination unit 9 determines whether or not Fux(S) = boundary (step S65).

　　　　　 If Fux(S) = boundary does not apply (step S65, No), the determination unit 9 recognizes a scene among the scenes in the indoor category (step S66), and determines the scene of the segmented image Fux.　 Scenes in the indoor category include, for example, cooking, furniture, and the like.　 Next, the processing unit 10 subjects the segmented image Fux to image processing optimized for each scene (step S67).　 Specifically, the processing unit 10 processes the segmented image Fux by using a correction parameter according to the scene determined in step S66.　 Next, the processing proceeds to step S69.

　　　　　 If Fux(S) = boundary (step S65, Yes), the processing unit 10 processes the segmented image Fux by the intermediate-value correction parameter between the correction parameters used for processing the adjacent segmented images.　 The processing unit 10 processes the image, for example, by the intermediate-value correction parameter between the correction parameter used for the outdoor image Fux-1 adjacent to the boundary image Fux and the correction parameter used for the indoor image Fux+1 adjacent to the boundary image (step S68).　 Next, the processing proceeds to step S69.

　　　　　 Next, the determination unit 9 determines whether or not x + 1 = n (step S69).　 If x + 1 = n does not apply (step S69, No), the determination unit 9 assigns x + 1 to x (step S70) and the processing returns to step S62.　 If x + 1 = n (step S69, Yes), the processing is finished.

　　　　　 Finally, an exemplary hardware configuration of the imaging device 100 of the embodiment will be described.

　　　　　 FIG. 11 is a diagram illustrating an example of the hardware configuration of the imaging device 100 of the embodiment.　 The imaging device 100 of the embodiment includes the imaging element 101, the fish-eye lens 102, the imaging element 103, the fish-eye lens 104, an image processing device 105, a CPU 106, an acceleration sensor 107, a display device 108, an operation button 109, and a storage medium 111.　 The imaging device 100 can also communicate with a smart device 110.

　　　　　 The imaging element 101 forms an image with light entering through the fish-eye lens 102.　 A target image formed on the imaging element 101 is converted into an image signal (electric signal) showing the first image 21.　 The imaging element 101 inputs the image signal into the image processing device 105.　 The imaging element 101 and the fish-eye lens 102 correspond to the first imaging unit 1 (see FIG. 2).

　　　　　 The imaging element 103 forms an image with light entering through the fish-eye lens 104.　 A target image formed on the imaging element 103 is converted into an image signal (electric signal) showing the second image 22.　 The imaging element 103 inputs the image signal into the image processing device 105.　 The imaging element 103 and the fish-eye lens 104 correspond to the second imaging unit 2 (see FIG. 2).

　　　　　 The image processing device 105 and the CPU 106 correspond to the combining unit 4, the first correction unit 6, the segmentation unit 7, the determining unit 8, the determination unit 9, the processing unit 10, the second detection unit 11, the calculation unit 12, and the second correction unit 13 (see FIG. 2).　 A function block realized by hardware is realized by the image processing device 105.　 A function block realized by software (program) is realized by the CPU 106.　 Of the combining unit 4, the first correction unit 6, the segmentation unit 7, the determining unit 8, the determination unit 9, the processing unit 10, the second detection unit 11, the calculation unit 12, and the second correction unit 13, which function block is to be realized by hardware and which function block is to be realized by software can be determined arbitrarily.

　　　　　 The acceleration sensor 107 detects the zenith direction of the imaging device 100 at the time of imaging.　 Information detected by the acceleration sensor 107 is processed by the first correction unit 6 realized by the image processing device 105 or the CPU 106.

　　　　　 The display device 108, the operation button 109, and the smart device 110 are user interfaces for operating the imaging device 100.　 The display device 108, the operation button 109, and the smart device 110 correspond to the display unit 14 and the selection unit 15 (see FIG. 2).

　　　　　 The storage medium 111 stores, for example, the first image 21, the second image 22, and the whole celestial sphere image 23.　 The storage medium 111 corresponds to the storage unit 3 (see FIG. 2).

　　　　　 When a part or all of the combining unit 4, the first correction unit 6, the segmentation unit 7, the determining unit 8, the determination unit 9, the processing unit 10, the second detection unit 11, the calculation unit 12, and the second correction unit 13 are realized as a program, a modularly-configured program including a part or all thereof is housed in the storage medium 111.　 By executing the program of the storage medium 111 by the CPU 106, a part or all of the combining unit 4, the first correction unit 6, the segmentation unit 7, the determining unit 8, the determination unit 9, the processing unit 10, the second detection unit 11, the calculation unit 12, and the second correction unit 13 are realized on a RAM not illustrated in FIG. 11.

　　　　　 The program may be recorded and provided in a computer readable recording medium such as a CD-ROM, a flexible disc (FD), a CD-R, a digital versatile disk (DVD), and a universal serial bus (USB) in a file in an installable form or in an executable form.　 The program may be provided or distributed via a network such as the Internet.

　　　　　 As described above, in the imaging device of the embodiment, the segmentation unit 7 segments the whole celestial sphere image into a plurality of segmented images.　 The determination unit 9 determines the scene of each segmented image.　 Thus, the processing unit 10 can process each segmented image by using the correction parameter according to the scene.　 Therefore, even when a plurality of scenes is included in one whole celestial sphere image, an optimum quality whole celestial sphere image can be generated.

　　　　　 Even when the imaging device is applied to an imaging device such as a single-lens reflex camera and a smartphone, the segmentation unit 7 segments an image into a plurality of segmented images.　 The determination unit 9 determines the scene of each segmented image.　 Thus, the processing unit 10 can process each segmented image by using the correction parameter according to the scene.　 Therefore, even when a plurality of scenes is included in one image, an optimum quality image can be generated.

　　　　　 According to the embodiment of the present invention, an optimum quality image can be generated even when a plurality of scenes is included in an image.

　　　　　 Although the invention has been described with respect to specific embodiments for a complete and clear disclosure, the appended claims are not to be thus limited but are to be construed as embodying all modifications and alternative constructions that may occur to one skilled in the art that fairly fall within the basic teaching herein set forth.

1 　first imaging unit
2 　second imaging unit
3 　storage unit
4 　combining unit
5 　first detection unit
6 　first correction unit
7 　segmentation unit
8 　determining unit
9 　determination unit
10 　processing unit
11 　second detection unit
12 　calculation unit
13 　second correction unit
14 　display unit
15 　selection unit

Japanese Laid-open Patent Publication No. 2013-198070

Claims

　　　　　 An imaging device comprising:
　　　　　 an imaging unit that captures an image;
　　　　　 a segmentation unit that segments the image into a plurality of segmented images;
　　　　　 a determination unit that determines a scene of each of the segmented images; and
　　　　　 a processing unit that processes each of the segmented images by a correction parameter according to the scene.
　　　　　 The imaging device according to claim 1, further comprising a determining unit that determines a category of the segmented image showing any one of an outdoor image obtained by imaging outdoors, an indoor image obtained by imaging indoors, and a boundary image positioned at a boundary between the outdoor image and the indoor image, wherein
　　　　　 the determination unit determines the scene by recognizing an optimum scene among a plurality of scenes associated with the category.
　　　　　 The imaging device according to claim 1 or 2, wherein
　　　　　 the segmentation unit segments the image into 2N pieces of the segmented images by segmenting the image into N pieces (N is an integer equal to or greater than two) in a vertical direction and two pieces in a horizontal direction, and
　　　　　 the determining unit uses each of the segmented images in the upper half and determines a category of each of the segmented images in the bottom half, a position in the horizontal direction of which is the same.
　　　　　 The imaging device according to claim 2 or 3, wherein
　　　　　 the processing unit processes an image by an intermediate-value correction parameter between the correction parameter used for the outdoor image adjacent to the boundary image and the correction parameter used for the indoor image adjacent to the boundary image, when a category of the segmented image is the boundary image.
　　　　　 The imaging device according to claim 3 or 4, further comprising:
　　　　　 a second detection unit that detects luminance of the segmented image;
　　　　　 a calculation unit that calculates, for each of the segmented images in the upper half and in the bottom half, a difference in luminance between the segmented images in the upper half and those in the bottom half where positions in the horizontal direction are the same; and
　　　　　 a second correction unit that makes different corrections for gradation characteristics between the segmented images in the upper half and the segmented images in the bottom half, in a case where the number of sets of the segmented images in the upper half and the segmented images in the bottom half, the luminance difference between which is greater than a first threshold, is greater than a second threshold.
　　　　　 The imaging device according to any one of claims 2 to 5, further comprising:
　　　　　 a display unit that displays one or more types of the scenes included in the image;
　　　　　 a selection unit that selects the scene among the one or more types of the scenes; and
　　　　　 a storage unit that stores one or more of the segmented images of the selected scene.
　　　　　 The imaging device according to claim 6, wherein
　　　　　 the display unit displays one or more of the segmented images of the scene selected by the selection unit.
　　　　　 The imaging device according to any one of claims 1 to 7, wherein
　　　　　 the imaging unit comprises:
　　　　　 a first imaging unit that captures a first image; and
　　　　　 a second imaging unit that captures a second image,
　　　　　 and the imaging device further comprises a combining unit that generates a whole celestial sphere image showing images in all directions of an imaging spot by combining the first and second images.
　　　　　 The imaging device according to claim 8, further comprising:
　　　　　 a first detection unit that detects a zenith direction of the imaging device at the time of imaging; and
　　　　　 a first correction unit that corrects a deviation in a zenith direction of the whole celestial sphere image based on the zenith direction of the imaging device.
　　　　　 An image processing device comprising:
　　　　　 a segmentation unit that segments an image into a plurality of segmented images;
　　　　　 a determination unit that determines a scene of each of the segmented images; and
　　　　　 a processing unit that processes each of the segmented images by a correction parameter according to the scene.
　　　　　 An imaging method, comprising the steps of:
　　　　　 capturing an image by an imaging unit;
　　　　　 segmenting, by a segmentation unit, the image into a plurality of segmented images;
　　　　　 determining, by a determination unit, a scene of each of the segmented images; and
　　　　　 processing, by a processing unit, each of the segmented images by a correction parameter according to the scene.