JP2017118472A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a user with a fisheye image by clarifying a correspondence between the fisheye image and a prescribed subject reflected in the fisheye image.SOLUTION: A control part 21 of a body device 20 recognizes a prescribed subject (the face of a person) included in a wide-angle image (a fisheye image) photographed by an imaging device 10 to correct distortion of an image (a face image of the person) of its subject part, and performs processing for displaying, for example, a partial image of the wide-angle image and a corrected image of the subject part on a reproduction screen as processing for associating the partial image of the wide-angle image with the corrected image of the subject part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing a wide-angle image photographed at a wide angle.

  For example, a fisheye lens used in an imaging apparatus such as a digital still camera can take a wide range of images with an angle of view of approximately 180 °. However, since a projection method is adopted, an image taken with a fisheye lens (fish The eye image) becomes more distorted as it goes from the center to the end (peripheral). As a technique for displaying a fish-eye image captured using such a fish-eye lens, for example, an image in a predetermined area is cut out from the fish-eye image, and distortion is corrected for the image in the cut-out area. A technique has been disclosed in which processing is provided to a user without distortion (corrected image) (see Patent Document 1). In addition, when a distorted circular image (fisheye image) including the face of each participant in the conference is captured using a fisheye lens, each participant's face is recognized and each participant's utterance time is displayed along with each participant's face. Is disclosed (see Patent Document 2).

Utility Model Registration No. 3066594 JP, 2015-19162, A

  According to the technique of each patent document described above, even when a wide range of photographing is performed using a fisheye lens, it is possible to obtain an image without distortion with respect to an image in a region cut out from a part thereof, but the image is cut out. By simply displaying an image, it was difficult for the user to confirm the correspondence with the fisheye image, such as in what scene the person was photographed.

  An object of the present invention is to clarify the correspondence between a fish-eye image and a predetermined subject appearing in the fish-eye image so that it can be provided to the user.

In order to solve the above-described problems, the present invention
Detecting means for recognizing a predetermined subject included in a wide-angle image photographed at a wide angle and detecting an image of the subject portion;
Correction means for correcting distortion of the image of the subject portion detected by the detection means;
Processing means for performing processing for associating at least a part of the wide-angle image with the image of the subject portion corrected by the correction means;
An image processing apparatus comprising:

  According to the present invention, it is possible to clarify the correspondence between a fish-eye image and a predetermined subject in the fish-eye image and provide it to the user.

(1) is an external view showing a state in which the imaging device 10 and the main body device 20 constituting a digital camera applied as an image display device are integrally combined, and (2) is a diagram illustrating the imaging device 10 and the main body device 20. The external view showing the separated state. (1) is a block diagram showing a configuration of the imaging device 10, and (2) is a block diagram showing a configuration of the main device 20. (1), (2) is a figure for demonstrating the various attitude | positions of the imaging device 10 at the time of imaging | photography. (1), (2) is the figure which showed the example of a display of the fisheye image image | photographed using the fisheye lens 16B. (1)-(4) is a figure for demonstrating a mode in the case of reproducing a fish-eye image (saved image). The figure for demonstrating the image management table 23C by the side of the main body apparatus 20. FIG. The flowchart for demonstrating the operation | movement (operation | movement of the characteristic part of 1st Embodiment) by the side of the main body apparatus 20 in the case of 360 degree imaging | photography. The flowchart which showed the operation | movement (operation | movement of the characteristic part of 1st Embodiment) by the side of the main body apparatus 20 in the case of reproducing a fish-eye image (saved image). The flowchart which showed the operation | movement following FIG. The flowchart which showed the operation | movement (operation | movement of the characteristic part of 2nd Embodiment) by the side of the main body apparatus 20 in the case of reproducing | regenerating a fisheye image (saved image). (1)-(4) is a figure for demonstrating a mode in the case of reproducing a fish-eye image (saved image) in 2nd Embodiment. The flowchart which showed the operation | movement (operation | movement of the characteristic part of 3rd Embodiment) by the side of the main body apparatus 20 in the case of reproducing a fish-eye image (saved image). (1)-(3) is a figure for demonstrating a mode in the case of reproducing a fish-eye image (saved image) in 3rd Embodiment. (1)-(3) is a figure for demonstrating the other mode in the case of reproducing a fish-eye image (saved image) in 3rd Embodiment.

(First embodiment)
The first embodiment of the present invention will be described below with reference to FIGS.
The present embodiment exemplifies a case where the present invention is applied to a digital camera as an image processing device, and this digital camera can be separated into an imaging device 10 having an imaging unit to be described later and a main body device 20 having a display unit to be described later. This is a separate digital camera. FIG. 1A shows a state in which the imaging device 10 and the main body device 20 are integrally combined, and FIG. 1B shows a state in which the imaging device 10 and the main body device 20 are separated. . The imaging device 10 and the main body device 20 constituting the separate digital camera can be paired (wireless connection recognition) using wireless communication that can be used, and wireless communication includes wireless LAN (Wi-Fi). -Fi) or Bluetooth (registered trademark) is used. On the main device 20 side, an image captured on the imaging device 10 side is received and acquired, and this captured image (live view image) is displayed. In the present embodiment, the captured image is not limited to a saved image, but means an image displayed on the live view screen (live view image: image before saving).

FIG. 2A is a block diagram illustrating a configuration of the imaging device 10, and FIG. 2B is a block diagram illustrating a configuration of the main body device 20.
2 (1), the imaging device 10 includes a control unit 11, a power supply unit 12, a storage unit 13, a communication unit 14, an operation unit 15, an imaging unit 16, and an attitude detection unit 17. The control unit 11 operates by supplying power from the power supply unit (secondary battery) 12 and controls the overall operation of the imaging apparatus 10 according to various programs in the storage unit 13. Are provided with a CPU (Central Processing Unit) and a memory (not shown). The storage unit 13 includes, for example, a ROM, a flash memory, and the like, and stores a program and various applications for realizing the present embodiment. The communication unit 14 transmits a captured image to the main device 20 side, or receives an operation instruction signal or the like from the main device 20. The operation unit 15 includes basic operation keys (hardware keys) such as a power switch.

  The imaging unit 16 constitutes a camera unit capable of photographing a subject with high definition. The lens unit 16A of the imaging unit 16 is provided with a fisheye lens 16B, an imaging element 16C, and the like. In the camera of this embodiment, a normal imaging lens (not shown) and the fisheye lens 16B are interchangeable, and the illustrated example shows a state in which the fisheye lens 16B is mounted. The fish-eye lens 16B is a circumferential fish-eye lens (all-round fish-eye lens), which is composed of, for example, a three-lens lens system and can capture a wide range of angles of view of approximately 180 °, and a wide-angle image (taken by the fish-eye lens 16B) The whole (fish-eye image) is a circular image. In this case, since the projection method is adopted, the wide-angle image (fish-eye image) photographed by the fish-eye lens 16B is greatly distorted from the center toward the end.

  That is, since the fisheye lens 16B is a circumferential fisheye lens capable of taking a wide range of images with an angle of view of approximately 180 °, the entire fisheye image is a circular image, and extends from the center to the end (periphery). The image becomes more distorted as it goes further, and the periphery of the fisheye image is reduced compared to the center of the fisheye image, so even if the user tries to visually check the contents of the periphery in detail. Confirmation is extremely difficult. When a subject image (optical image) by such a fish-eye lens 16B is formed on an image sensor (for example, CMOS or CCD) 16C, an image signal (analog value signal) photoelectrically converted by the image sensor 16C is illustrated. The digital signal is converted into a digital signal by an omitted A / D converter, and after a predetermined image display process is performed, it is transmitted to the main device 20 side and displayed on a monitor. The posture detection unit 17 is a three-axis acceleration sensor that detects acceleration applied to the imaging device 10, and gives each of the acceleration components in the X, Y, and Z directions detected according to the posture of the imaging device 10 to the control unit 11.

  2 (2), the main device 20 has a reproduction function for reproducing an image photographed using a fisheye lens or the like, and includes a control unit 21, a power supply unit 22, a storage unit 23, a communication unit 24, and an operation unit 25. The touch display unit 26 and the posture detection unit 27 are provided. The control unit 21 operates by supplying power from the power source unit (secondary battery) 22 and controls the overall operation of the main body device 20 according to various programs in the storage unit 23. Are provided with a CPU (Central Processing Unit) and a memory (not shown). The storage unit 23 has a configuration including, for example, a ROM, a flash memory, and the like, and is necessary for operating the program memory 23A in which a program for realizing the present embodiment and various applications are stored, and the main body device 20 operate. A work memory 23B that temporarily stores various information (for example, flags) and an image management table 23C described later.

  The communication unit 24 transmits / receives various data to / from the imaging device 10. The operation unit 25 includes various buttons of a push button type, such as a power button, a release button button, and a setting button for setting shooting conditions such as exposure and shutter speed. The process according to the input operation signal is executed, or the input operation signal is transmitted to the imaging apparatus 10. The touch display unit 26 has a configuration in which a touch panel 26B is stacked on a display 26A such as a high-definition liquid crystal, and the display screen is a monitor screen (live view screen) that displays a captured image (fisheye image) in real time. Or a playback screen for playing back captured images. The posture detection unit 27 is a three-axis acceleration sensor that detects acceleration applied to the main body device 20, and gives the X, Y, and Z direction acceleration components detected according to the posture of the main body device 20 to the control unit 21.

  When the release button for instructing photographing is operated on the main device 20 side, the control unit 21 performs development processing on the fisheye image to generate a photographed image, and performs image compression processing to perform a standard file. After conversion into the format, the data is recorded and saved on a recording medium in the storage unit 23. In this case, the control unit 21 generates a corrected image by performing processing for correcting the distortion of the fisheye image, and records and saves the corrected image in association with the fisheye image. In this case, a technology is used to correct distortion of the fisheye image by converting a point on the virtual spherical model into a point on the flat screen using a plane that touches an arbitrary point on the virtual spherical model as a screen. Since the processing for correcting the distortion of the fisheye image (fisheye distortion correction processing) uses a well-known technique that is generally used in image processing, its specific description is omitted. And

FIG. 3 is a diagram for explaining the posture of the imaging apparatus 10.
The imaging device 10 has a configuration in which, for example, the whole is formed in a box shape, and a fisheye lens 16B is disposed at the front center portion thereof. FIG. 3 (1) shows a state where the optical axis direction of the fisheye lens 16B is substantially orthogonal to the direction of gravity, that is, when the camera is photographed so as to be substantially perpendicular to the ground (when vertical photographing is performed). In addition, the hemisphere indicated by the broken line in the drawing indicates the photographing range of the fisheye lens 16B having an angle of view of approximately 180 °. FIG. 3 (2) shows a state where the front direction of the camera (the optical axis direction of the fisheye lens 16B) is directed to the sky, that is, the optical axis direction of the fisheye lens 16B is substantially opposite to the direction of gravity. This shows the case (when horizontal shooting is performed).

FIG. 4 is a diagram exemplifying a wide-angle image (fish-eye image) photographed using the fish-eye lens 16B, and shows a state where the fish-eye image is displayed on the reproduction screen of the touch display unit 26 on the main device 20 side. .
The example shown in the figure shows a fish-eye image taken by the vertical shooting shown in FIG. 3 (1), and FIG. 4 (1) shows a fish-eye image when the front 180 ° is taken vertically as the shooting range. 2) is a fish-eye image in the case where 180 ° is taken vertically as a photographing range. In this way, when 360-degree shooting is performed, shooting is performed twice, divided into 180-degree shooting at the front and 180-degree shooting at the rear. In the example shown in the figure, a party is held outdoors surrounding a dining table. A pair of parents and children are shown in the fish-eye image of FIG. In addition, a case is shown in which a pair of parent and child is also reflected in the fish-eye image of FIG. Thus, each fisheye image photographed at 360 ° is stored and saved as a group of fisheye images.

  The control unit 21 of the main device 20 receives and obtains a fisheye image photographed by the imaging device 10 and corrects the distortion of the entire image, and selects a predetermined subject included in the corrected image. After recognizing and detecting an image of the subject portion for each subject, a process of associating a partial image of the corrected image with the detected image of the subject portion is performed. That is, the control unit 21 performs a process for correcting the distortion on each fisheye image photographed at 360 ° as described above to generate a corrected image, and corresponds the corrected image to each fisheye image. The record is saved. Further, for each corrected image, the image is analyzed to detect all predetermined subjects (human faces) included in the corrected image. Note that this face detection process is a technique commonly used in cameras, such as detecting face parts or detecting skin color, and in this embodiment, the well-known technique is used. Therefore, the specific description thereof will be omitted.

  As a result of the face detection process, when all faces of persons included in the corrected image are detected, the control unit 21 cuts out the subject part (face part) for each person along the outline of the person, An image is generated, and each face image is recorded and stored in association with the fisheye image. The face image is an image centered on the face so that a person can be specified, but may be an image including the upper body (head, neck, shoulders), and is not limited to a face. A feature point may be detected from the inside, and a feature region based on the feature point may be extracted. Further, the control unit 21 records and saves after performing image processing for aligning each face image with a predetermined shape and size.

FIG. 5 is a diagram for explaining a situation when a fisheye image (stored image) is reproduced.
FIG. 5A shows a list screen in which various fisheye images (stored images) are displayed as thumbnails on the touch display unit 26 on the main device 20 side, and the circles in the figure indicate individual fisheye images, It shows a state in which fisheye images are displayed as thumbnails in a 3 × 3 matrix. Here, when the position of an arbitrary thumbnail image is touched, the control unit 21 selects the touched thumbnail image (fisheye image) as a reproduction target. FIG. 5 (2) shows a fish-eye image selected as a reproduction target, and a fish-eye image (see FIG. 4 (1)) obtained by photographing the front 180 ° is selected from the 360 ° fish-eye images described above. Is the case.

  FIG. 5 (3) shows an image within a predetermined range cut out from the corrected image recorded and stored in association with the fisheye image selected as the reproduction target. In this case, an image within a predetermined range (for example, a rectangular cut frame) is cut out from the corrected image. This cutout frame (virtual frame) is similar to the corrected image, and is, for example, a rectangular frame having a size that is 1/4 of the corrected image, and at the initial point in time when the fisheye image is selected, Although it is set to a position (a central position), it can be moved to an arbitrary position on the corrected image by a user operation. In the illustrated example, a case where an image (an image including a parent and child) within a rectangular cut frame indicated by a broken line is cut out is illustrated.

  FIG. 5 (4) is a diagram showing a reproduction screen of a fisheye image selected as a reproduction target. On this playback screen, as described above, a part of the corrected image (image in the cutout frame) is enlarged and displayed, and a list of face images of each person associated with the fisheye image is displayed. . In this case, the image is not limited to the face image of each person included in the fish-eye image selected by the touch operation (for example, the fish-eye image in FIG. 4 (1)). The face images of each person included in the other fisheye images (for example, the fisheye image in FIG. 4B) are also displayed in a list. At that time, the face images of the persons are displayed in a list in accordance with the order of arrangement of the persons in the fisheye image. The face images of each person are displayed as a list in a straight line at the bottom of the playback screen as shown. In this case, the parent and child face images arranged on the left side in the figure are the face images of persons included in the fisheye image of FIG. 4 (1), and the parent and child face images arranged on the right side are shown in FIG. 4 (2). Is a face image of a person included in the fisheye image.

  Also, in the figure, a broken-line arrow attached to the periphery of the enlarged display image (image in the cutout frame) indicates the moving direction of the cutout frame that can be moved by the touch operation, and this cutout frame is displayed on the touch display unit 26. It is moved according to the slide direction and slide amount by touch operation (slide operation). As a result, the content of the image within the clipping frame (enlarged display image) displayed on the reproduction screen also changes according to the movement of the clipping frame. Note that the moving direction of the clipping frame is not limited to the four directions of up, down, left, and right, and may be moved in an oblique direction. Thus, on the playback screen, in addition to the above-described slide operation, two consecutive tap operations (double tap operations) are also effective touch operations, and when a double tap operation is performed on any face image The control unit 21 detects a position including the face image from the corrected image, and moves (jumps) the cutout frame to the detection position, whereby an enlarged display image (an image in the cutout frame) is displayed. ) Is included in the face image.

FIG. 6 is a diagram for explaining the image management table 23C on the main device 20 side.
The image management table 23C is a table for managing fish-eye images (stored images) and related images. The “fish-eye image ID”, “shooting type”, “link ID”, “corrected image ID”, Each item includes “face image ID”, “face image ID”,. The “fisheye image ID” is an item for storing information (serial number or the like) for identifying the fisheye image file (not shown) in order to manage the fisheye image. The case where “A0002”,... Is stored is shown. The “shooting type” is an item for storing a shooting type indicating whether it is a fisheye image shot at 180 ° or a fisheye image shot at 360 °. "A0002" and "A0003" indicate that each image group is a set of 360-degree fisheye images.

  “Link ID” is an item in which “fish-eye image ID” indicating another fish-eye image that is paired when “shooting type” is 360 ° photographing is stored. "A0003" is stored in the "link ID" of "A0002" as "fisheye image ID" indicating another fisheye image, and the other "fisheye image" is displayed in the "link ID" of "A0003". “A0002” is stored as the “fisheye image ID”. The “corrected image ID” is an item for storing information (serial number or the like) for identifying a corrected image file (not shown) in order to manage a corrected image whose distortion has been corrected corresponding to a fisheye image. In the illustrated example, “B0001”, “B0002”,... Are stored. “Face image ID” is information (serial number or the like) for identifying the face image file (not shown) in order to manage the face image of each person included in the fish-eye image corresponding to the fish-eye image. In the illustrated example, “C0001”, “C0002”,... Are stored.

  Next, the operation concept of the image processing apparatus (digital camera) in the first embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium. 7 to 9 are flowcharts showing an outline of the operation of the characteristic part of the first embodiment of the overall operation of the digital camera. When the flow of FIGS. Return to the main flow of operation (not shown).

FIG. 7 is a flowchart for explaining the operation on the main device 20 side (operation of the characteristic part of the present embodiment) when performing 360 ° imaging. Note that 360 ° shooting is a menu item arbitrarily selected by a user operation from a menu screen for selecting various shooting modes, and when this 360 ° shooting menu item is selected, the operation of FIG. 7 is performed. Is started.
First, the control unit 21 on the main body device 20 side develops the fisheye image received and acquired from the imaging device 10 when the release operation is performed (YES in step A1) in the waiting state for the release operation (step A1). Processing is performed to generate a captured image, and image compression processing is performed to convert the image into a standard file format (step A2), and then recording and saving is performed on a recording medium in the storage unit 23 (step A3). In this case, two fisheye images taken at 360 ° are recorded and stored as a set of images, and “A0002” is generated as a new “fisheye image ID”, for example, and the image management table 23C Register additional.

  Then, the control unit 21 registers “360 ° shooting” as the “shooting type” corresponding to the new “fisheye image ID” in the image management table 23C, and also registers the corresponding “link ID”. Although it is performed (step A4), the registration of the “link ID” is performed at the time of the second shooting of 360 ° shooting. Next, the control unit 21 performs processing for correcting the distortion on the fisheye image to generate the corrected image (step A5), and records and saves the corrected image on the recording medium of the storage unit 23. At this time, for example, “B0002” is generated as a new “corrected image ID”, and is registered in the image management table 23C in association with “A0002” of “fisheye image ID” (step A6).

  Further, the control unit 21 detects the face of the person by analyzing the entire corrected image (step A7). As a result, it is checked whether or not a person's face can be detected (step A8). If a person's face cannot be detected (NO in step A8), the process proceeds to step A11, and 360 ° shooting has been completed. Check out. Since it is a case where the first shooting of 360 ° shooting has been completed (NO in step A11), the process returns to the above-described step A1 and waits until the release that instructs the second shooting of 360 ° shooting is operated. It becomes.

  If a face of a person in the corrected image is detected (YES in step A8), the face image is generated by cutting out each person along the outline of the person (step A9). In this case, the face images are generated for all the persons included in the corrected image, but the face distortion is too large like the person located in the periphery of the fisheye image. Even if the person's face cannot be detected with high accuracy even if the distortion correction is performed, the face image may be generated if there is a possibility of the person. After each face image generated by cutting out for each person in this way is arranged in a predetermined shape and size, it is recorded and stored in the recording medium of the storage unit 23 in association with the fisheye image. For example, “C0004” and “C0005” are generated as “ID”, and are registered in the image management table 23C in association with “A0002” of “fisheye image ID” (step A10). Then, the above-described operation (steps A1 to A11) is repeated until 360 ° shooting is completed, but when 360 ° shooting is completed (YES in step A11), the flow of FIG.

FIG. 8 and FIG. 9 are flowcharts showing operations on the main device 20 side (operations of characteristic portions of the present embodiment) when a fisheye image (stored image) is reproduced.
First, when an instruction to reproduce a fisheye image (stored image) is given by a user operation, the control unit 21 reads a plurality of fisheye images (stored images) recorded and stored in the recording medium of the storage unit 23. (Step B1), converted into thumbnail images and displayed as a list (step B2). In this state, it is checked whether or not a touch operation for designating an arbitrary thumbnail image as a reproduction target is performed from the list screen (step B3). If the touch operation is not performed (NO in step B3), the operation unit 25 is checked. It is checked whether a key for instructing the end of reproduction has been operated as a predetermined key (step B4). If the end of reproduction is not instructed (NO in step B4), the process returns to the above-described step B3. However, if the end of reproduction is instructed (YES in step B4), the process exits the flow of FIGS. .

  Now, in a state where a plurality of fisheye images are displayed as a list of thumbnail images in a matrix, when the position of any thumbnail image is touched (YES in step B3), the control unit 21 controls the fish. An eye image is selected and designated as a reproduction target (step B5), and a correction image stored in correspondence with the designated image (fisheye image) is read (step B6), and a rectangular area (cutout frame) is read from the correction image. ) Are cut out (step B7). In this case, since the clipping frame is set to the initial position (center position) on the corrected image at the first time point when the fisheye image is selected, the image of the central portion of the corrected image is cut out. Then, the clipped image is enlarged to a predetermined size and displayed at the center of the reproduction screen (step B8).

  Next, each face image stored corresponding to the above-mentioned designated image (fisheye image) is read (step B9), and each face image is displayed as a list on the reproduction screen (step B10). In this case, as shown in FIG. 5 (4), the face images are displayed in a list in a horizontal row at the bottom of the playback screen. The order of arrangement is the order of arrangement of the persons in the fisheye image as described above. It has become. And then. It is checked whether another fisheye image forming a pair with the above-described designated image (fisheye image) is undesignated for other fisheye images of 360 ° (step B11). If it remains (YES in step B11), the undesignated fisheye image is designated as a reproduction target (step B12), and then the process returns to step B9 described above to correspond to this new designated image (fisheye image). Each face image stored is read out and displayed in a list on the reproduction screen (step B10). As a result, when all the 360-degree fisheye images have been specified (NO in step B11), the flow proceeds to FIG.

  When such a reproduction screen is displayed on the touch display unit 26, when a touch operation is performed on the reproduction screen (YES in step B13), the touch operation is performed on the face images displayed in a list. It is checked whether the operation is a double tap operation (step B14) or a slide operation in the vicinity of the enlarged corrected image (image in the cutout frame) (step B15). If a double-tap operation is performed on the face image (YES in step B14), the position including the face is specified from the corrected image, and the clipping frame is moved to the detected position (step B16). . For example, as shown in FIG. 5 (4), when a double-tap operation is performed on a girl's face image, the girl's face image is specified from the corrected image, and the face image is the center of the cutout frame. The cutout frame is moved so that the position is reached. Then, the image within the cutout frame is cut out and enlarged and displayed on the reproduction screen (step B17). Thereafter, the process returns to Step B13.

  If the touch operation on the playback screen is a slide operation near the corrected image (image in the cutout frame) displayed in an enlarged manner (YES in step B15), the slide direction and slide amount are detected ( In step B18), the cutout frame is moved on the corrected image in accordance with the slide direction and the slide amount (step B19). Then, the image within the cutout frame is cut out from the corrected image and enlarged and displayed on the reproduction screen (step B17), and then the process returns to step B13 described above.

  On the other hand, instead of a touch operation on the playback screen (NO in step B13), for example, when a key for instructing switching to another fisheye image of 360 ° shooting is operated as a predetermined key of the operation unit 25 (YES in step B20) After designating another fisheye image as a reproduction target from the two fisheye images shot at 360 ° (step B21), the image is saved corresponding to the designated image. The corrected image is read out (step B22), the image in the cutout frame is cut out from the corrected image (step B23), and the cutout image is enlarged and displayed on the reproduction screen (step B24). Thereafter, the process returns to Step B13. For example, when a key for instructing the end of reproduction is operated as a predetermined key of the operation unit 25 (YES in step B25), the process exits the flow of FIGS.

  As described above, in the first embodiment, the control unit 21 of the main device 20 recognizes a predetermined subject included in a wide-angle image (fisheye image) photographed by the imaging device 10 and recognizes the subject portion. Since the processing of correlating the image of the image and correlating the image of a part of the wide-angle image with the image of the corrected subject part is performed, the fish-eye image and a predetermined subject appearing in the fish-eye image are The correspondence can be clarified and provided to the user.

  The control unit 21 corrects distortion of the wide-angle image (fisheye image) photographed by the imaging device 10, recognizes a predetermined subject (person's face) included in the corrected image, and detects the subject portion. Image (face image) is detected for each subject (person), and a process of associating a part of the corrected image with the detected image (face image) of each subject part is performed. Can be converted into a form that makes it easy to see a predetermined subject. For example, when displaying an image, if a wide-angle image with a large amount of distortion is displayed as it is and an image with the distortion corrected and the face image of each person are displayed in association with each other, the wide-angle image is converted into a form that is easy to see. The user can easily confirm the person.

  Since the control unit 21 enlarges and displays the image within the predetermined range (cutout frame) of the corrected image and displays a list of the images (face images) of each detected subject portion, distortion occurs during image reproduction. If a part of the corrected image is displayed rather than displaying the entire large wide-angle image, the wide-angle image can be displayed in an easy-to-see form. In other words, the user can check the subject by focusing only on a part of the corrected image taken at a wide angle, and can include a person who is not displayed on a part of the corrected image. All persons included in can be confirmed.

  When displaying a list of images (face images) of each detected subject portion, the control unit 21 displays the images in a list according to the arrangement order of each subject (person) on the wide-angle image. Therefore, it is possible to easily confirm where the person was and the like just by looking at the face image list display.

  The control unit 21 arbitrarily moves the predetermined fixed range (cutout frame) by a user operation (slide operation), and enlarges and displays the clipped image according to the movement of the cutout frame. It is possible to easily confirm the details of the desired portion of the corrected image.

  The control unit 21 recognizes all the predetermined subjects (person's face) included in the wide-angle image group that includes a plurality of wide-angle images captured by changing the photographing direction, and images of the subject portions. Since the (face image) is detected for each subject, for example, even if 360 ° shooting is performed in two steps of 180 ° forward shooting and 180 ° backward shooting, the faces of all persons around the subject Can be detected.

  When a wide-angle image to be displayed is specified from a group of wide-angle images in which a plurality of wide-angle images captured by changing the imaging direction are specified, the control unit 21 includes a corrected image of the specified wide-angle image, Since all the images of the subject portion (person's face) detected from the wide-angle image group are displayed in association with each other, the user can change the display by switching between a plurality of images (corrected images). You can see multiple images.

  The control unit 21 designates, as a display target, a wide-angle image arbitrarily selected by a user operation from a group of wide-angle images captured by changing the shooting direction. Therefore, the user changes the shooting direction. It is possible to arbitrarily switch to a desired image from a plurality of captured images (corrected images) and view an enlarged display of the image.

  When a wide-angle image (fisheye image) is displayed as a list of thumbnail images and any wide-angle image is selected from the thumbnail images by a user operation, the control unit 21 selects the selected wide-angle image. The correction image is enlarged and displayed, and the images (face images) of each detected subject portion are displayed in a list, so that the user can arbitrarily select a desired fisheye image from many fisheye images. Thus, the image can be confirmed in an easy-to-see form.

  Since the control unit 21 records and saves the corrected image and the image (face image) of each subject portion in association with each other, the control unit 21 performs distortion correction of the wide-angle image every time image reproduction is performed, The process of cutting out is no longer necessary.

(Modification 1)
In the first embodiment described above, a fisheye image arbitrarily selected by a user operation from a set of fisheye images captured by changing the shooting direction is designated as a display target. Each fisheye image may be sequentially selected one by one in accordance with a predetermined order at each timing, and the corrected images may be switched and displayed. As a result, the user can view each fisheye image (corrected image) in a predetermined order. In this case, in 360-degree shooting, the face image of a person included in a fisheye image (corrected image) being displayed and the face image of a person included in a non-displayed fisheye image (corrected image) May be identified and displayed in a list. As a result, the correspondence between the fisheye image (corrected image) and the person's face image can be easily confirmed.

(Modification 2)
In the first embodiment described above, the face images of all the persons included in the set of fisheye images taken by changing the shooting direction are displayed as a list. You may make it identify and display. For example, a person included in a part of the corrected image being enlarged may be specified, and only the face image of the person may be identified and displayed (for example, color-coded display or enlarged display). As a result, it is possible to identify and display a person included in a part of the corrected image being enlarged and a person included in a non-display part in the corrected image. Here, when the person included in the non-display portion is confirmed by the enlarged display, the cutout frame may be moved so that the person is cut out.

(Modification 3)
In 1st Embodiment mentioned above, although the circumference fisheye lens (all the circumference fisheye lens) was illustrated as the fisheye lens 16B, you may use a diagonal fisheye lens. In the above-described embodiment, the single fisheye lens 16B is used to perform 360 ° photographing in two steps of 180 ° forward photographing and 180 ° backward photographing. However, the two fisheye lenses 16B are provided. Thus, for example, if a fisheye lens is provided on each of the front and back surfaces of the camera housing, it is possible to perform forward 180 ° photographing and backward 180 ° photographing simultaneously in accordance with a 360 ° photographing instruction. Further, 360 ° photographing may be performed by performing photographing a plurality of times using an ultra-wide-angle lens.

(Modification 4)
In the first embodiment described above, the control unit 21 of the main body device 20 receives and acquires the fisheye image captured by the imaging device 10 and performs a process of correcting the distortion of the entire image, and within the corrected image. After recognizing a predetermined subject included and detecting an image of the subject portion for each subject, a process of associating a partial image of the corrected image with the detected image of the subject portion is performed. However, the present invention is not limited thereto, and when the control unit 21 of the main body device 20 receives and acquires the fisheye image photographed by the imaging device 10, it is included in the fisheye image without correcting the distortion of the fisheye image. After a predetermined subject is recognized and an image of the subject portion is detected for each subject, a process of associating a part of the fisheye image with the detected image of the subject portion may be performed. There.

  In this case, since the detected image of each subject portion remains distorted, the subject portion is cut out at a predetermined timing, for example, at the same time as or before and after the association process described above, or for each person. The distortion correction may be performed on the image of each subject portion at the same time or before and after the timing. As a process of associating the image of a part of the fisheye image with the distortion left in this way and the image of the corrected subject part, for example, the image of the part of the fisheye image is displayed on the reproduction screen, and the corrected subject part is displayed. If the process of displaying a list of images and a list for playback is performed, the correspondence between the fisheye image and the predetermined subject in the fisheye image is clarified and Can be provided.

(Modification 5)
In the first embodiment described above, an image within a predetermined range cut out from the corrected image recorded and stored in association with the fisheye image selected as the reproduction target is displayed. You may display the fish-eye image itself selected as object. That is, a fisheye image in which distortion remains and a list of detected images (face images) of each subject portion may be displayed. Also in this case, as in the above-described embodiment, the correspondence between the fisheye image and a predetermined subject appearing in the fisheye image can be clarified and provided to the user.

(Modification 6)
In the above-described first embodiment, when a wide-angle image to be displayed is designated by a user operation from a wide-angle image group including a plurality of wide-angle images photographed by changing the photographing direction, the designation is made. The corrected image of the wide-angle image and the images of all the subject parts (person's face) detected from the wide-angle image group are displayed in association with each other. However, the wide-angle image to be displayed is selected from the wide-angle image group. When an image is designated by a user operation, the designated wide-angle image (an image in which distortion remains), and images of all subject parts (a person's face) detected from the wide-angle image group, You may make it display correspondingly. As a result, the user can view the plurality of wide-angle images by switching display of the plurality of wide-angle images.

  In the first embodiment described above, the image within the predetermined range (cutout frame) of the corrected image is enlarged and displayed, and the detected images (face images) of each subject portion are displayed as a list. Not limited to this, the entire corrected image recorded and stored in association with the fisheye image selected as the reproduction target and the detected images (face images) of each subject portion may be displayed in a list.

  In the first embodiment described above, the fisheye image, the correction image, and the face image of each person are recorded and stored in association with each other. However, the fisheye image, the correction image, and the face image of each person are stored in, for example, SD. Output to the outside via a removable portable memory (recording medium) such as a card or USB memory, or communication means, and display the corrected image and each person's face image in association with each other on an external device. Also good.

  In the first embodiment described above, a part of the corrected image obtained by correcting the distortion of the fisheye image (the image in the clipping frame) is enlarged and displayed on the playback screen. However, the entire corrected image is displayed on the playback screen. You may make it do.

  In the above-described first embodiment, 360 ° photographing is performed in two steps of 180 ° front photographing and 180 ° rear photographing, and the plurality of fisheye images are grouped, but the fisheye lens 16B is used. The shooting range is not limited to 180 ° shooting and is arbitrary. Needless to say, the present invention can be applied not only to 360 ° shooting but also to a shot image shot in the range of 180 °.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described with reference to FIGS.
In the first embodiment described above, when a fish-eye image (stored image) is reproduced, a plurality of fish-eye images are displayed on the thumbnail list screen. When the display position is touched, a part of the corrected image (image in the cutout frame) associated with the fisheye image is switched and displayed. In this second embodiment, the fish When an eye image (thumbnail image) is touched, the face image list screen of each person associated with the fisheye image is switched and displayed.

  Furthermore, in the first embodiment, when a face image displayed together with the correction image is touched, a position including the face image is detected from the correction image, and the clipping frame is moved to the detected position. In the second embodiment, when an arbitrary face image is touched from the face image list screen, the face image is controlled so as to include the face image in the image within the cutout frame. Is switched to a corrected image associated with a fish-eye image including the image (the fish-eye image selected from the thumbnail list screen). Here, in both the embodiments, the same or the same names are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the second embodiment. .

  The operation at the time of shooting in the second embodiment is the same as that of the first embodiment described above. When the shot fisheye image is developed and recorded in the recording medium of the storage unit 23, The fisheye image is subjected to a process for correcting distortion to generate a corrected image, and the corrected image is recorded and stored in association with the fisheye image. Further, the face of each person included in the corrected image is detected, the face part is cut out for each person to generate a face image (corrected image), and recorded and stored in association with the fisheye image. Like to do.

FIG. 10 is a flowchart showing the operation (operation of the characteristic part of the second embodiment) on the main device 20 side when a fisheye image (stored image) is reproduced.
First, when an instruction to reproduce a fisheye image (stored image) is given by a user operation, the control unit 21 reads a plurality of fisheye images (stored images) recorded and stored in the recording medium of the storage unit 23. (Step C1), this fisheye image is converted into a thumbnail image and displayed as a list (Step C2). FIG. 11 (1) shows a thumbnail list screen of fisheye images. In the figure, circles arranged in a 3 × 3 matrix form show thumbnail images of fisheye images. FIG. 11B is an enlarged view of a part of the thumbnail list screen.

  In a state where such a thumbnail list screen is displayed, it is checked whether a predetermined touch operation (for example, a double tap operation) has been performed on the display position of an arbitrary thumbnail image (step C5). If a double-tap operation is not performed for two consecutive hits on the thumbnail image (NO in step C3), the process proceeds to step C9. If a double-tap operation is performed on the thumbnail image (YES in step C3), Each person's face image (corrected image) recorded and saved in association with the double-tapped fisheye image is read (step C4), and each face image is enlarged and displayed on the playback screen (step S4). C5). FIG. 11 (3) shows a screen displaying a list of face images of each person. For example, the face images clipped into rectangles are arranged and displayed in the same order as in the first embodiment described above, for example.

  In the state where the face image list screen is displayed as described above, it is checked whether a predetermined touch operation (for example, a double tap operation) is performed on the display position of an arbitrary face image (step C6). If the double tap operation is not performed on the face image (NO in step C6), the process proceeds to step C9. If the double tap operation is performed (YES in step C6), the face image subjected to the double tap operation is included. The correction image recorded and saved in association with the fisheye image is read (step C7), and the correction image is switched and displayed (entire display) (step C8).

  In this case, the fisheye image selected on the thumbnail list screen is specified as the fisheye image including the face image designated by touch, and the correction image associated with the fisheye image is read out and displayed for switching. At that time, an image cut out from the corrected image with the face image as the center is enlarged and switched and displayed (entire display) so that the face image designated by touch is located at the center. FIG. 11 (4) shows an example of a correction image associated with a fish-eye image that includes a face image of the girl when the face image of the girl is double-tapped as shown in FIG. 11 (3). It is the figure which illustrated the case where a part was cut out and enlarged and displayed, and is a case where the enlarged image centering on a girl's face is switched and displayed.

  FIG. 11 (5) is a diagram for explaining another display example different from the display example of FIG. 11 (4). In the case of FIG. 11 (4), the fisheye image selected on the thumbnail list screen is specified as the fisheye image including the touch-designated face image. This is a case where another fisheye image including the face image other than the fisheye image is specified, and a part of the image is cut out from the correction image associated with the fisheye image and enlarged and displayed. Also in this case, an image cut out from the corrected image with the face image as the center is enlarged and switched and displayed (entire display) so that the touch-designated face image is located at the center.

  It should be noted that the fisheye image selected on the thumbnail list screen is preferentially specified as the fisheye image including the face image designated by touch, but the fisheye image or other than the fisheye image is selected. Whether to preferentially specify any of the other fisheye images including the face image designated for touch may be set in advance by a user operation. Further, as a fish-eye image including a face image designated for touch, the fish-eye image and another fish-eye image may be switched by a user operation, or may be automatically switched after a predetermined time. Further, if a plurality of other fisheye images are stored, the plurality of fisheye images are sequentially identified by a switching operation, and a correction image associated with the identified fisheye image is read and switched for display. You may make it make it.

  In this state where the corrected image is switched and displayed, it is checked whether a return instruction operation has been performed (step C9). If the return instruction operation is not performed (NO in step C9), other instruction operations are performed. (Step C10). If another instruction operation (for example, an operation for instructing image editing) is performed (YES in step C10), the process proceeds to a process corresponding to the other instruction, but a return instruction operation is performed. (YES in step C9), the process returns to step C3 described above.

  As described above, in the second embodiment, when a wide-angle image is displayed as a list of thumbnail images and an arbitrary wide-angle image is selected by a user operation, the wide-angle image is included in the wide-angle image. Since a list of images of a predetermined subject portion is displayed, even if a wide-angle image in which the entire image is distorted is displayed as a thumbnail as it is, the user can easily view the subject. . In other words, because the wide-angle image (fish-eye image) is largely distorted, it is difficult to accurately determine who is in the thumbnail image, and any thumbnail image can be manipulated by the user. It is time-consuming and burdensome for the user to check the faces that appear in the image one by one, while enlarging the image while specifying it, but any thumbnail image (fisheye image) is selected. The face images of each person included in the fisheye image are displayed in a list, so that the user can display images with a sense of reality while grasping the entire shooting area using the fisheye images displayed in thumbnails. In addition to being able to see, it is possible to easily visually recognize who is in the person's face image.

  Furthermore, when any face image is selected by a user operation in a state where the above-described face images are displayed in a list, a corrected image in which distortion of the wide-angle image included in the face image is corrected is displayed. As a result, the user can see a wide range of images without distortion.

  In the second embodiment described above, a correction image is generated by performing processing for correcting the distortion of the captured fish-eye image, and a face image of each person cut out from the correction image is generated. The correction image and the face image of each person are recorded and stored in association with the image, and the face images of each person associated with the fisheye image are read and displayed as a list when each fisheye image is played back. The corrected image and the face image of each person may be generated when the fisheye image is reproduced. That is, when displaying a thumbnail of a fisheye image, a correction image is generated by correcting the distortion of an arbitrarily selected fisheye image, and then a face image of each person cut out from the corrected image is generated. You may make it do. Moreover, the face image may be generated by cutting out the face image of each person from the fisheye image and correcting the distortion, without being limited to cutting out the face portion of each person after generating the corrected image. .

  In the second embodiment described above, a case where a fisheye image is displayed as a thumbnail has been shown, but a corrected image obtained by correcting the distortion of the fisheye image may be displayed as a thumbnail. In this case as well, when a thumbnail image (corrected image) is selected by a user operation, the screen is switched to a screen displaying a list of face images of each person included in the image, as in the second embodiment. .

  In the second embodiment described above, 360 ° photographing may be performed in two steps of 180 ° front photographing and 180 ° rear photographing, and the plurality of fisheye images may be grouped. Needless to say, the present invention is not limited to photographing, and may be applied to a photographed image photographed in a 180 ° range (180 ° photographing).

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described with reference to FIGS.
In the third embodiment, when a plurality of fisheye images are converted into thumbnail images and displayed as a list, the face images of each person stored corresponding to the fisheye images are displayed in the vicinity of each thumbnail image. The list is displayed. That is, the face image of each person related to the image is also displayed around the thumbnail image. Here, in both the embodiments, the same or the same names are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the third embodiment. . In the third embodiment, for example, as shown in FIG. 4 (1), a fisheye image obtained by vertically shooting 180 ° forward, and 180 ° backward as shown in FIG. 4 (2). The fisheye images in the case of vertical shooting, that is, two fisheye images shot twice (360 ° shooting) divided into 180 ° forward shooting and 180 ° backward shooting are recorded and saved as a set of images. ing.

FIG. 12 is a flowchart showing an operation (operation of the characteristic part of the second embodiment) on the main device 20 side when a fisheye image (stored image) is reproduced.
First, when a reproduction of a fisheye image is instructed by a user operation, the control unit 21 reads a plurality of recorded and stored fisheye images (step D1), converts the fisheye images into thumbnail images, and displays a list. Display (step D2). Then, the face image of each person recorded and stored in association with each fisheye image is read for each fisheye image (step D3), and each corresponding to the vicinity (periphery) of the fisheye image (thumbnail image). A face image is arranged and displayed (step D4).

  FIG. 13 (1) is a diagram showing an example of a thumbnail list screen of fisheye images. In the figure, circles arranged in a 3 × 3 matrix form thumbnail images of fisheye images. A vertically long small circle (ellipse) arranged around the (thumbnail image) indicates a face image of each person corresponding to the fisheye image. FIG. 13B is an enlarged view of a part of the thumbnail list screen. In the example shown in the drawing, a fish-eye image is displayed as a thumbnail when 360 ° shooting is not performed, that is, when 180 ° shooting is performed. A face image of each person included in the fisheye image is displayed in a list around the thumbnail image.

  FIG. 14 (1) is a diagram showing another example of a thumbnail list screen of fisheye images. In the figure, fisheye image A was taken vertically 180 ° forward as shown in FIG. 4 (1). The fisheye image B shows the fisheye image in the case where the rear 180 ° is vertically photographed as shown in FIG. In this way, in the case of a set of fisheye images that are shot twice (360 ° shooting) divided into 180 ° forward shooting and 180 ° backward shooting, the periphery of each thumbnail image is included in the own fisheye image. In addition to the face image of the person, the face image of the person included in another fisheye image forming the set is also displayed.

  FIG. 14B is an enlarged view of a part of the thumbnail list screen and shows a thumbnail display of the fisheye image A. In this case, in the periphery of the thumbnail image, in addition to the face image of the parent and child shown in the fisheye image A, the parent and child shown in another fisheye image B that forms a pair with the fisheye image A A list of facial images is also displayed. The same applies when the fisheye image B is displayed as a thumbnail. Although not shown, the fisheye image B is displayed around the thumbnail image in addition to the parent-child face image shown in the fisheye image B. A list of the face images of the parent and child shown in another fisheye image A that forms a pair with the image B is also displayed.

  When face images are arranged and displayed around thumbnail images, the size of each face image is made uniform and arranged in a well-balanced manner as shown in FIGS. 13 (2) and 14 (2). ing. In a state where the thumbnail list screen is displayed, it is checked whether the display position of the face image in the list screen is touched (double tapped) (step D5). If the display position of the face image is not double-tapped (NO in step D5), the process proceeds to step D8, but if double-tapped (YES in step D5), the corresponding fisheye image from the double-tapped face image. The correction image recorded and stored in association with this fisheye image is read (step D6), and the correction image is switched to the thumbnail list screen and displayed (step D7). Also in this case, as in the second embodiment described above, the fisheye image selected on the thumbnail list screen or another fisheye image including a face image that is touch-designated other than the fisheye image is specified, The correction image associated with the specified fisheye image is read out and displayed for switching.

  FIG. 13 (3) shows a case where the corrected image including the face image is enlarged and displayed when the child's face image is double-tapped as shown in FIG. 13 (2). Similarly, FIG. 14 (3) shows a case where the corrected image including the face image of the double-tapped child is enlarged, but in the case of FIG. 13 (3), the image displayed as a thumbnail is displayed. In the case of FIG. 14 (3), the corrected image of the image B that forms a pair with the image A displayed as a thumbnail is displayed.

  In this state where the corrected image is switched and displayed, it is checked whether a return instruction operation has been performed (step D8). If no return instruction operation is performed (NO in step D8), other instruction operations are performed. (Step D9). If another instruction operation (for example, an operation for instructing image editing) is performed (YES in step D9), the process proceeds to a process corresponding to the other instruction, but a return instruction operation is performed. (YES in step D8), the process returns to step D2 described above to display the thumbnail list screen.

  As described above, in the third embodiment, the wide-angle image is displayed as a list of thumbnail images, and the image of the subject portion cut out from the wide-angle image is displayed in association with the thumbnail image. Even if a wide-angle image that is distorted as a whole is displayed as a thumbnail as it is, the user can easily visually recognize the subject.

  In addition, the control unit 1 includes any wide-angle image in the wide-angle image group that includes a plurality of wide-angle images photographed by changing the photographing direction as thumbnails, and is included in the wide-angle image group. Since the images of all the subject portions are displayed in association with the thumbnail images, the user can check the thumbnail display of any wide-angle image in the wide-angle image group, and the wide-angle image group It is possible to confirm images of all the subject portions included in the. For example, when 360 ° shooting is performed in two steps of 180 ° forward shooting and 180 ° backward shooting, the thumbnail image (fisheye image) of the 180 ° forward shooting is included in the fisheye image. A face image of a person included in a fisheye image taken 180 ° behind is also displayed together with the face image of the person being photographed, so that without checking the thumbnail display of the fisheye image taken 180 ° behind The face image of the person included in the fisheye image can be confirmed.

  In the third embodiment described above, a correction image is generated by performing processing for correcting distortion of a captured fish-eye image, and a face image of each person cut out from the correction image is generated. The correction image and the face image of each person are recorded and stored in association with the image, and when displaying the thumbnails of each fisheye image, the face images of each person associated with the fisheye image are read out and displayed in a list. When an arbitrary face image is specified by the user from the display, a correction image including the face image is read and displayed. However, the correction image and each person's face image are displayed as thumbnails of fisheye images. Sometimes it may be generated. That is, a correction image may be generated by performing a process of correcting the distortion of the fisheye image when the thumbnail of the fisheye image is displayed, and a face image of each person cut out from the correction image may be generated. Further, not only when a face image of each person is cut out from the corrected image after the correction image is generated, but the face image is generated by cutting out the face image of each person from the fisheye image and correcting the distortion. It may be.

  In the above-described third embodiment, the single fisheye lens 16B is used to perform 360 ° photography in two steps of 180 ° forward photography and 180 ° backward photography. However, the two fisheye lenses 16B are provided. Thus, for example, if a fisheye lens is provided on each of the front and back surfaces of the camera housing, it is possible to perform forward 180 ° photographing and backward 180 ° photographing simultaneously in accordance with a 360 ° photographing instruction. Further, 360 ° photographing may be performed by performing photographing a plurality of times using an ultra-wide-angle lens.

  In each of the above-described embodiments, the case where the present invention is applied to a digital camera as an image processing apparatus has been described. However, the present invention is not limited thereto, and for example, a personal computer, a PDA (personal portable information communication device), a tablet terminal device, a smartphone, etc. Mobile phones, electronic games, music players, and the like.

  Further, the “apparatus” and “unit” shown in each of the above-described embodiments may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

  In each of the above-described embodiments, a fisheye image or a wide-angle image is displayed as a thumbnail image. For example, a fisheye image or an image obtained by correcting distortion in a predetermined range of the wide-angle image may be used. Images associated with images and wide-angle images may be displayed as thumbnail images.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
Detecting means for recognizing a predetermined subject included in a wide-angle image photographed at a wide angle and detecting an image of the subject portion;
Correction means for correcting distortion of the image of the subject portion detected by the detection means;
Processing means for performing processing for associating at least a part of the wide-angle image with the image of the subject portion corrected by the correction means;
An image processing apparatus comprising:
(Claim 2)
The invention according to claim 2 is the image processing apparatus according to claim 1,
The correction means further corrects distortion of the wide-angle image,
The processing means performs a process of associating at least a part of an image of the wide-angle image whose distortion has been corrected by the correction means with the image of the subject portion corrected by the correction means.
It is characterized by that.
(Claim 3)
The invention according to claim 3 is the image processing apparatus according to claim 1,
The processing means performs a process of associating an image of at least a part of the wide-angle image having distortion with an image of the subject portion corrected by the correction means.
It is characterized by that.
(Claim 4)
The invention according to claim 4 is the image processing apparatus according to any one of claims 1 to 3,
A display means,
The processing unit displays the image of at least a part of the wide-angle image on the display unit, and performs the association process by displaying the image of the subject portion corrected by the correction unit on the display unit.
It is characterized by that.
(Claim 5)
The invention according to claim 5 is the image processing apparatus according to claim 4,
The processing means performs the associating process by simultaneously displaying an image of at least a part of the wide-angle image and an image of the subject portion corrected by the correction means on the display means.
It is characterized by that.
(Claim 6)
The invention according to claim 6 is the image processing apparatus according to claim 4 or 5, wherein
An acquisition means for acquiring an image within a predetermined range as a part of the wide-angle image;
The processing unit displays the image within a predetermined range acquired by the acquisition unit on the display unit, and displays the image of the subject portion detected by the detection unit on the display unit, thereby performing the association process. Do,
It is characterized by that.
(Claim 7)
The invention according to claim 7 is the image processing apparatus according to any one of claims 4 to 6,
The detection means recognizes a predetermined subject included in a wide-angle image photographed at a wide angle, detects an image of the subject portion,
The processing means displays at least a part of the wide-angle image on the display means, and performs the association processing by causing the display means to display a list of images of the subject portions detected by the detection means. Features.
(Claim 8)
The invention according to claim 8 is the image processing apparatus according to claim 7,
A second selecting means for selecting an image of an arbitrary subject portion from the subject portions in accordance with a user operation in a state in which the images of the subject portions are displayed in a list on the display means;
The processing means, when an image of an arbitrary subject portion is selected by the second selection means, corresponds to at least a part of the wide-angle image corresponding to the subject portion selected by the second selection means. Performing the association process by displaying an image on the display means;
It is characterized by that.
(Claim 9)
The invention according to claim 9 is the image processing apparatus according to claim 7,
The correction means further corrects distortion in a predetermined range of the wide-angle image including the image of the subject portion selected by the second selection means,
The processing means displays a corrected image of a wide-angle image within a predetermined range corrected by the correcting means on the display means when an image of an arbitrary subject portion is selected by the second selecting means. Perform the association process,
It is characterized by that.
(Claim 10)
According to a tenth aspect of the present invention, in the image processing apparatus according to any one of the seventh to ninth aspects,
The processing unit switches the display of the display unit so that at least a part of the wide-angle image is displayed from a state in which the image of the subject portion is displayed as a list, and the association process is performed. I do,
It is characterized by that.
(Claim 11)
The invention according to claim 11 is the image processing apparatus according to claim 8 or 9, wherein
The processing means includes at least a part of the wide-angle image corresponding to the subject portion selected by the second selection means from a state in which the images of the subject portion are displayed as a list on the display means. Performing the associating process by switching the display of the display means so as to be in the state displayed on the display means;
It is characterized by that.
(Claim 12)
The invention according to claim 12 is the image processing apparatus according to any one of claims 7 to 11,
The detection means recognizes a plurality of predetermined subjects included in a wide-angle image photographed at a wide angle, detects an image of the subject portion for each subject,
The processing means, when displaying a list of images of each subject portion detected by the detection means on the display means, displays the list in order according to the arrangement order of the subjects on the wide-angle image,
It is characterized by that.
(Claim 13)
The invention according to claim 13 is the image processing apparatus according to claim 6 or 9, wherein
A movement instruction means for arbitrarily moving the predetermined range by a user operation;
The acquisition unit acquires an image within a predetermined range moved by the movement instruction unit;
It is characterized by that.
(Claim 14)
The invention according to claim 14 is the image processing apparatus according to any one of claims 4 to 13,
A first wide-angle image or a corresponding image is selected from the thumbnail images in response to a user operation in a state where the wide-angle image or the corresponding image corresponding to the wide-angle image is listed as a thumbnail image on the display means. Further comprising a selection means,
The detection means recognizes a predetermined subject included in the wide-angle image selected by the first selection means or the corresponding image, and detects an image of the subject portion;
The processing means converts an image of a subject portion detected by the detection means and corrected by the correction means when an arbitrary wide-angle image or corresponding image is selected by the first selection means. Display a list on the display means,
It is characterized by that.
(Claim 15)
The invention described in claim is the image processing apparatus according to any one of claims 4 to 14,
First selection means for selecting an arbitrary wide-angle image or corresponding image from the thumbnail images in response to a user operation in a state where the wide-angle image or the corresponding image corresponding to the wide-angle image is displayed as a list of thumbnail images. In addition,
The processing means, when an arbitrary wide-angle image or corresponding image is selected by the first selection means, displays the wide-angle image or corresponding wide-angle image and the image of the subject portion corrected by the correction means. Performing the association process by simultaneously displaying on the display means;
It is characterized by that.
(Claim 16)
The invention described in claim is the image processing apparatus according to any one of claims 4 to 15,
The detection means recognizes all predetermined subjects included in a wide-angle image group including a plurality of wide-angle images photographed by changing the photographing direction, and images of the subject portion for each subject. Detect
The processing means includes images of all the subject portions included in the wide-angle image group detected by the detection means when any wide-angle image in the wide-angle image group is displayed as a thumbnail. The image corrected by the correction means is displayed in association with the wide-angle image displayed as a thumbnail,
It is characterized by that.
(Claim 17)
The invention described in claim is the image processing apparatus according to any one of claims 1 to 16,
The detection means recognizes all predetermined subjects included in a wide-angle image group including a plurality of wide-angle images photographed by changing the photographing direction, and images of the subject portion for each subject. To detect,
It is characterized by that.
(Claim 18)
The invention described in claim is the image processing apparatus according to claim 17,
Further comprising designation means for designating one wide-angle image from a group of wide-angle images in which a plurality of wide-angle images photographed by changing the photographing direction are set;
The processing means performs a process of associating the wide-angle image designated by the designation means with the images of all the subject portions detected from the wide-angle image group by the detection means.
It is characterized by that.
(Claim 19)
The invention according to claim 19 is the image processing device according to any one of claims 1 to 18,
The processing means performs processing to record and save the wide-angle image and the image of the subject portion detected by the detection means in association with each other.
It is characterized by that.
(Claim 20)
The invention according to claim 20 is the image processing apparatus according to any one of claims 1 to 19,
The wide-angle image is a fish-eye image taken using a fish-eye lens.
It is characterized by that.
(Claim 21)
The invention according to claim 21
An image processing method in an image processing apparatus,
A process of recognizing a predetermined subject included in a wide-angle image taken at a wide angle and detecting an image of the subject portion;
Processing for correcting distortion of the image of the detected subject portion;
Associating at least a portion of the wide-angle image with the corrected image of the subject portion;
It is characterized by including.
(Claim 22)
The invention described in claim 22
For the computer of the image processing device,
A function of recognizing a predetermined subject included in a wide-angle image taken at a wide angle and detecting an image of the subject portion;
A function of correcting image distortion of the detected subject portion;
A function of associating at least a portion of the wide-angle image with the corrected image of the subject portion;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 10 Imaging apparatus 20 Main body apparatus 11, 21 Control part 13, 23 Storage part 23A Program memory 23C Image management table 14, 24 Communication part 16 Imaging part 16B Fisheye lens 16C Imaging element 25 Operation part 26 Touch display part

Claims (22)

Detecting means for recognizing a predetermined subject included in a wide-angle image photographed at a wide angle and detecting an image of the subject portion;
Correction means for correcting distortion of the image of the subject portion detected by the detection means;
Processing means for performing processing for associating at least a part of the wide-angle image with the image of the subject portion corrected by the correction means;
An image processing apparatus comprising: The correction means further corrects distortion of the wide-angle image,
The processing means performs a process of associating at least a part of an image of the wide-angle image whose distortion has been corrected by the correction means with the image of the subject portion corrected by the correction means.
The image processing apparatus according to claim 1. The processing means performs a process of associating an image of at least a part of the wide-angle image having distortion with an image of the subject portion corrected by the correction means.
The image processing apparatus according to claim 1. A display means,
The processing unit displays the image of at least a part of the wide-angle image on the display unit, and performs the association process by displaying the image of the subject portion corrected by the correction unit on the display unit.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The processing means performs the associating process by simultaneously displaying an image of at least a part of the wide-angle image and an image of the subject portion corrected by the correction means on the display means.
The image processing apparatus according to claim 4. An acquisition means for acquiring an image within a predetermined range as a part of the wide-angle image;
The processing unit displays the image within a predetermined range acquired by the acquisition unit on the display unit, and displays the image of the subject portion detected by the detection unit on the display unit, thereby performing the association process. Do,
The image processing apparatus according to claim 4, wherein the image processing apparatus is an image processing apparatus. The detection means recognizes a predetermined subject included in a wide-angle image photographed at a wide angle, detects an image of the subject portion,
The processing means displays at least a part of the wide-angle image on the display means, and performs the association processing by causing the display means to display a list of images of the subject portions detected by the detection means. The image processing apparatus according to claim 4, wherein the image processing apparatus is characterized. A second selecting means for selecting an image of an arbitrary subject portion from the subject portions in accordance with a user operation in a state in which the images of the subject portions are displayed in a list on the display means;
The processing means, when an image of an arbitrary subject portion is selected by the second selection means, corresponds to at least a part of the wide-angle image corresponding to the subject portion selected by the second selection means. Performing the association process by displaying an image on the display means;
The image processing apparatus according to claim 7. The correction means further corrects distortion in a predetermined range of the wide-angle image including the image of the subject portion selected by the second selection means,
The processing means displays a corrected image of a wide-angle image within a predetermined range corrected by the correcting means on the display means when an image of an arbitrary subject portion is selected by the second selecting means. Perform the association process,
The image processing apparatus according to claim 7. The processing unit switches the display of the display unit so that at least a part of the wide-angle image is displayed from a state in which the image of the subject portion is displayed as a list, and the association process is performed. I do,
The image processing apparatus according to claim 7, wherein the image processing apparatus is an image processing apparatus. The processing means includes at least a part of the wide-angle image corresponding to the subject portion selected by the second selection means from a state in which the images of the subject portion are displayed as a list on the display means. Performing the associating process by switching the display of the display means so as to be in the state displayed on the display means;
The image processing apparatus according to claim 8, wherein the image processing apparatus is an image processing apparatus. The detection means recognizes a plurality of predetermined subjects included in a wide-angle image photographed at a wide angle, detects an image of the subject portion for each subject,
The processing means, when displaying a list of images of each subject portion detected by the detection means on the display means, displays the list in order according to the arrangement order of the subjects on the wide-angle image,
The image processing apparatus according to claim 7, wherein the image processing apparatus is an image processing apparatus. A movement instruction means for arbitrarily moving the predetermined range by a user operation;
The acquisition unit acquires an image within a predetermined range moved by the movement instruction unit;
The image processing apparatus according to claim 6, wherein the image processing apparatus is an image processing apparatus. A first wide-angle image or a corresponding image is selected from the thumbnail images in response to a user operation in a state where the wide-angle image or the corresponding image corresponding to the wide-angle image is listed as a thumbnail image on the display means. Further comprising a selection means,
The detection means recognizes a predetermined subject included in the wide-angle image selected by the first selection means or the corresponding image, and detects an image of the subject portion;
The processing means converts an image of a subject portion detected by the detection means and corrected by the correction means when an arbitrary wide-angle image or corresponding image is selected by the first selection means. Display a list on the display means,
The image processing apparatus according to claim 4, wherein the image processing apparatus is an image processing apparatus. First selection means for selecting an arbitrary wide-angle image or corresponding image from the thumbnail images in response to a user operation in a state where the wide-angle image or the corresponding image corresponding to the wide-angle image is displayed as a list of thumbnail images. In addition,
The processing means, when an arbitrary wide-angle image or corresponding image is selected by the first selection means, displays the wide-angle image or corresponding wide-angle image and the image of the subject portion corrected by the correction means. Performing the association process by simultaneously displaying on the display means;
The image processing apparatus according to claim 4, wherein the image processing apparatus is an image processing apparatus. The detection means recognizes all predetermined subjects included in a wide-angle image group including a plurality of wide-angle images photographed by changing the photographing direction, and images of the subject portion for each subject. Detect
The processing means includes images of all the subject portions included in the wide-angle image group detected by the detection means when any wide-angle image in the wide-angle image group is displayed as a thumbnail. The image corrected by the correction means is displayed in association with the wide-angle image displayed as a thumbnail,
The image processing apparatus according to claim 4, wherein the image processing apparatus is an image processing apparatus. The detection means recognizes all predetermined subjects included in a wide-angle image group including a plurality of wide-angle images photographed by changing the photographing direction, and images of the subject portion for each subject. To detect,
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Further comprising designation means for designating one wide-angle image from a group of wide-angle images in which a plurality of wide-angle images photographed by changing the photographing direction are set;
The processing means performs a process of associating the wide-angle image designated by the designation means with the images of all the subject portions detected from the wide-angle image group by the detection means.
The image processing apparatus according to claim 17. The processing means performs processing to record and save the wide-angle image and the image of the subject portion detected by the detection means in association with each other.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The wide-angle image is a fish-eye image taken using a fish-eye lens.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. An image processing method in an image processing apparatus,
A process of recognizing a predetermined subject included in a wide-angle image taken at a wide angle and detecting an image of the subject portion;
Processing for correcting distortion of the image of the detected subject portion;
Associating at least a portion of the wide-angle image with the corrected image of the subject portion;
An image processing method comprising: For the computer of the image processing device,
A function of recognizing a predetermined subject included in a wide-angle image taken at a wide angle and detecting an image of the subject portion;
A function of correcting image distortion of the detected subject portion;
A function of associating at least a portion of the wide-angle image with the corrected image of the subject portion;
A program to realize

Images