JP2017058801A - Image display apparatus, image display method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display apparatus, an image display method and a program capable of allowing a user to easily confirm what part (position) of an image (fisheye image) taken via a fisheye lens is displayed.SOLUTION: When displaying a part of image (fisheye image) taken via a fisheye lens 5B after subjecting the image to a distortion correction, a control unit 1 identifies the position of a pick-up area in the fisheye image, and displays a mark at the position on a touch display unit 6 to notify the position of the pick-up area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image display device, an image display method, and a program for correcting and displaying distortion of a fisheye image captured using a fisheye lens.

  For example, a fisheye lens used in an imaging apparatus such as a digital still camera can capture a wide range of angles of view of approximately 180 °. However, since a projection method is adopted, an image captured by a fisheye lens (fish The eye image) has a characteristic that it becomes more distorted as it goes from the center to the end (peripheral). As a technique for displaying a fish-eye image captured using such a fish-eye lens, for example, an image in a predetermined area is cut out from the fish-eye image, and distortion is corrected for the image in the cut-out area. An image processing apparatus is disclosed in which a distortion-free image (corrected image) is provided to a user by performing processing (see Patent Document 1).

Utility Model Registration No. 3066594

  According to the technology of the above-mentioned patent document, even when a wide range of imaging is performed using a fisheye lens, it is possible to obtain an image without distortion for an image in an area cut out from a part of the image. In that case, it is not possible to confirm which part (position) of the image (fisheye image) captured by the fisheye lens is just by looking at the corrected image (corrected image).

  An object of the present invention is to enable a user to easily confirm which part (region) of an image in a fisheye image has been corrected for distortion even if the image is captured with a fisheye lens (fisheye image). Is to do.

In order to solve the above-described problems, the present invention
A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit An image display device comprising display means for displaying the corrected image.
Position specifying means for specifying the position of the cutout region on the fisheye image;
Informing means for informing the position of the cutout area specified by the position specifying means;
It is characterized by providing.

  According to the present invention, even if an image (fisheye image) is captured with a fisheye lens, the user can easily confirm which part (region) of the fisheye image has been corrected for distortion. Can do.

The block diagram which showed the basic component of the digital camera applied as an image display apparatus. (1) to (3) are diagrams for explaining various postures of the camera, (4) is a fish-eye image captured in the posture of the camera (camera orientation) shown in (3), and The figure which showed the relationship with the ground direction. (1) is a diagram illustrating an image (fisheye image) captured by the fisheye lens 5B, and (2) is a desired image on the fisheye image in a state where the fisheye image is displayed on the touch display unit 6. The figure which showed the case where the arbitrary positions to touch were touched. (1) is a diagram illustrating a corrected image after distortion correction is performed on an image in the cutout region, and (2) is an image in the cutout region (image after distortion correction) on a fisheye image. The figure which illustrated the case where it superimposed and displayed on the touch position of. The flowchart which showed the operation | movement started when it switches to reproduction | regeneration mode. 6 is a flowchart for explaining in detail processing (step A7 in FIG. 5) for specifying a cutout region on a fisheye image based on a touch position. 6 is a flowchart for explaining in detail a process (step A8 in FIG. 5) of recognizing a camera direction at the time of imaging and cutting out an image in a cutout region. 6 is a flowchart for explaining in detail the notification process (step A11 in FIG. 5). (1), (2) is a figure for demonstrating concretely the process which specifies a cut-out area | region. The figure for demonstrating the alerting | reporting format table 3C. (1) is a display example according to notification format A, (2) is a display example according to notification format B, and (3) is a diagram illustrating a display example according to notification format C.

Hereinafter, embodiments of the present invention will be described with reference to FIGS.
This embodiment exemplifies a case where the present invention is applied to a digital camera as an image display device, and FIG. 1 is a block diagram showing basic components of the digital camera.
As a basic function, a digital camera has an imaging function capable of imaging using a fisheye lens or other normal imaging lens, a playback function for reproducing an image captured using a fisheye lens, or the like. . The control unit 1 operates by supplying power from the power source unit (secondary battery) 2 and controls the overall operation of the digital camera in accordance with various programs in the storage unit 3. A CPU (Central Processing Unit) and a memory (not shown) are provided.

  The storage unit 3 includes, for example, a ROM, a flash memory, and the like, and stores programs and various applications for realizing the present embodiment in accordance with operation procedures shown in FIGS. A program memory 3A, a work memory 3B for temporarily storing various information (for example, flags) necessary for the operation of the camera, a notification format table 3C described later, and the like. The storage unit 3 may be configured to include a removable portable memory (recording medium) such as an SD card or an IC card that records and saves fisheye images or the like. In the state connected to the network via the network, a storage area on a predetermined server device side may be included. The operation unit 4 includes various push button type keys such as a power button, a release button, and a setting button for setting shooting conditions such as exposure and shutter speed. Processing according to the input operation signal is performed.

  The imaging unit 5 constitutes a camera unit capable of photographing a subject with high definition. The lens unit 5A of the imaging unit 5 is provided with a fisheye lens 5B, an imaging element 5C, and the like. In the camera of this embodiment, a normal imaging lens (not shown) and a fisheye lens 5B are interchangeable. In the example shown in the figure, a fisheye lens 5B is attached instead of a normal imaging lens (not shown). Is shown. The fish-eye lens 5B is a circumferential fish-eye lens (all-around fish-eye lens) that has a three-lens lens system and can capture a wide range of angles of view of approximately 180 °. The entire image captured by the fish-eye lens 5B Becomes a circular image. In this case, since the projection method is adopted, the image (fisheye image) picked up by the fisheye lens 5B has a feature that the image is distorted as it is farther from the center (toward the end).

  When a subject image (optical image) from the fisheye lens 5B is formed on an image sensor (for example, CMOS or CCD) 5C, an image signal (analog signal) photoelectrically converted by the image sensor 5C is not shown. After being converted into a digital signal (image signal) by the A / D conversion unit and subjected to predetermined image display processing, it is displayed on the touch display unit 6 in real time as a live view image. The control unit 1 performs development processing on the captured image in accordance with the operation of a release button that instructs shooting, generates a captured image, converts the image into a standard file format by performing image compression processing, and stores the image. The data is recorded and saved on the recording medium of section 3.

  The touch display unit 6 has a configuration in which a touch panel 6B is stacked on a display 6A such as a high-definition liquid crystal, and various software keys (touch keys) are allocated and displayed to display the function names or a touch operation with a finger or the like. And an operation signal corresponding to the touch position is input. The display screen of the touch display unit 6 becomes a monitor screen (live view screen) that displays an image (fisheye image) or the like captured by the fisheye lens 5B in real time, or reproduces a captured image (fisheye image) or the like. The playback screen may appear. The posture detection unit 7 is a three-axis acceleration sensor such as a piezoresistive type or a capacitance detection type that detects acceleration applied to the camera, and detects gravitational acceleration in three axis directions (X, Y, and Z directions) orthogonal to each other. Then, each acceleration component in the X, Y, and Z directions detected in accordance with the posture of the camera is given to the control unit 1. The control unit 1 detects various postures (described later) of the camera based on a signal indicating an acceleration component output from the posture detection unit 7. The posture detection unit 7 detects the detected gravity direction or a direction within a predetermined angle range with respect to the gravity direction as a top-and-bottom direction. In this case, for example, the gravity direction is set to the minus direction, that is, the minus direction is the ground direction. Direction. In addition, the above-mentioned predetermined angle considers the inclination of the ground, and may be arbitrarily changed by a user operation.

FIG. 2 is a diagram for explaining the posture of the digital camera.
The digital camera, for example, has a rectangular (rectangular) box shape as a whole, and has a configuration in which a fisheye lens 5B is arranged at the front center. FIG. 2A shows a state in which the front direction of the digital camera (the optical axis direction of the fisheye lens 5B) is directed to the sky, that is, the optical axis direction of the fisheye lens 5B is substantially opposite to the gravity direction. (2) shows a state in which the optical axis direction of the fisheye lens 5B is substantially perpendicular to the direction of gravity, that is, the camera is substantially perpendicular to the ground. A case where an image is taken (when a vertical shooting is performed) is shown, and a hemisphere indicated by a broken line in the drawing indicates an imaging range of the fish-eye lens 5B having an angle of view of approximately 180 °.

  FIG. 2 (3) shows the orientation of the camera when vertical shooting is performed. In the example shown in the drawing, the orientation of the camera when the lower surface a is in the ground direction, and the right side b is in the ground direction. The camera orientation when the left side c is in the ground direction and the camera orientation when the upper surface d is in the ground direction are shown. FIG. 2 (4) is a diagram showing the relationship between the fish-eye image captured with the camera posture (camera orientation) shown in FIG. 3 (3) and the ground direction at the time of shooting. In the figure, each arrow direction indicates a ground direction corresponding to each camera direction. For example, if the lower surface of the camera in vertical shooting is a and the ground direction is (when the ground direction is a in the camera direction of FIG. 2 (3)), in the fish-eye image of FIG. 2 (4) The direction a is the ground direction. At the time of imaging, the control unit 1 sets the camera posture based on the detection result of the posture detection unit 7 as the horizontal posture at the time of horizontal shooting in FIG. 2 (1), the vertical posture at the time of vertical shooting in FIG. 2 (3) camera orientations a to d are detected. This camera posture information is used when recording and saving a captured image, that is, when performing a predetermined image processing (development processing, etc.) on the captured image, generating a captured image, and recording and saving it. In addition, the posture information of the camera at the time of imaging is added to the captured image and recorded and saved.

FIG. 3A is a diagram illustrating an image (fisheye image) captured by the fisheye lens 5 </ b> B, and shows a state displayed on the reproduction screen of the touch display unit 6.
This fish-eye image shows an image taken by horizontal shooting as shown in FIG. 2 (1), and the example shown shows a case where the group of buildings is imaged toward the sky in a city where buildings are lined up. . In this case, since the fisheye lens 5B employs a circumferential fisheye lens that can capture a wide range of angles of view of approximately 180 °, the entire fisheye image (fisheye image having characteristic distortion) It becomes a circular image.

3 (2) shows an arbitrary position desired by the user (an arbitrary position on the fisheye image) in a state where the fisheye image of FIG. 3 (1) is displayed on the reproduction screen of the touch display unit 6. FIG. It is the figure which showed the case where it touches with a finger etc.
As shown in FIG. 3 (1), the peripheral part (end part) away from the central part of the fisheye image is imaged over a wide range and is greatly distorted, and the peripheral part is compared with the central part of the fisheye image. Since the image becomes a reduced image, even if the user tries to visually confirm the contents of the peripheral portion in detail, the confirmation is extremely difficult.

  In such a case, when the user touches an arbitrary position on the fish-eye image (a position where details are to be confirmed) with a finger or the like, for example, as shown in FIG. When an arbitrary position on the peripheral part away from the touch is touched with a finger or the like, the control unit 1 specifies an area including the touch position (for example, a substantially trapezoid) as a cutout area, and the image in the cutout area is Distortion correction is performed. Note that the “cutout area” indicates a range (readout area or read area) in which a part of the image is read from the fisheye image as a display target, but the image in the area is extracted from the fisheye image. The range may be (extracted) (the same applies hereinafter).

  FIG. 4A is a diagram illustrating a corrected image after distortion correction is performed on an image cut out from a fisheye image captured by horizontal shooting as illustrated in FIG. This is a case where the entire fisheye image shown in (1) is switched to the enlarged display of the corrected image. In the illustrated example, the enlarged display of the corrected image (corrected image) enables clear visual confirmation of a building name (for example, AB) that was difficult to visually confirm in the entire fisheye image display. The case where it became possible is shown.

  In this embodiment, as processing for correcting distortion of a fisheye image, for example, a plane that touches an arbitrary point on the virtual spherical model is used as a screen, and a point on the virtual spherical model is coordinate-converted to a point on the flat screen. However, the processing for correcting the distortion of the fisheye image (fisheye distortion correction processing) is a well-known technique generally used in image processing. Since it is used, its specific description will be omitted. Such a fish-eye distortion correction process is not limited to the technique described above, but is arbitrary.

FIG. 4 (2) shows an image (image after distortion correction: corrected image) in the cutout area in the reproduction screen on which the entire fisheye image is displayed as shown in FIG. 3 (1). It is the figure which illustrated the case where it superimposed and displayed on the touch position on an image.
That is, FIG. 4B is a diagram showing a state in which the position of the cutout area on the fisheye image is identified and displayed and the position is notified. The control unit 1 specifies the position of the cutout area on the fisheye image, and superimposes the image (corrected image) in the cutout area on the position of the cutout area in order to notify the position of the cutout area together with the correction image. To display. In this case, for example, the corrected image is changed to a size of about 1/8 or 1/12 of the whole fisheye image, the center of the corrected image is aligned with the touch position, and the ground direction of the corrected image is set. The fisheye image is superimposed and displayed so as to be in the outer direction of the fisheye image (the direction away from the center of the fisheye image on the line connecting the center of the fisheye image to the center of the cut-out region). At that time, a ground mark (▽) indicating the ground direction is additionally displayed in the corrected image.

  Next, the operation concept of the digital camera in the present embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium. FIG. 5 is a flowchart showing an outline of the operation of the characteristic part of the present embodiment in the overall operation of the digital camera. When the flow of FIG. 5 is exited, the main flow of the overall operation (not shown) Return to).

FIG. 5 is a flowchart showing an operation that is started when the playback mode is switched.
First, the control unit 1 displays a list of various images (stored images) captured and stored by the fish-eye lens 5B and the like, and displays a desired image (for example, fish) from the list display (menu display). When the (eye image) is selected as a display target by a user operation (step A1), the selected image (fisheye image) is displayed on the reproduction screen of the touch display unit 6 (step A2). In this state, the control unit 1 checks whether or not the displayed selection image (fisheye image) is touched (step A3), whether playback is instructed (whether an operation for canceling the playback mode has been performed) ) (Step A4) or whether a switching operation for instructing switching of the selected image (fisheye image) has been performed (step A5).

  If the end of reproduction is instructed (YES in step A4), the flow of FIG. 5 is exited, but if a fisheye image switching operation is performed (YES in step A5), the first step Return to A1. If none of the touch operation, the operation for instructing the reproduction end, or the operation for instructing the switching of the fisheye image is performed (NO in steps A3 to A5), the process returns to the above-described step A3 and enters the operation waiting state. . Now, when an arbitrary position on the fisheye image is touched to arbitrarily cut out a part of the image from the fisheye image (YES in step A3), the touch position on the touch panel 6B is specified ( Step A6), based on the identified touch position, a process for identifying a cutout region on the fisheye image is performed (step A7).

FIG. 6 is a flowchart for explaining in detail a process (step A7 in FIG. 5) for specifying a cutout region on the fisheye image based on the touch position. FIG. 9A is a drawing 3 for specifying a cut-out area from the captured fish-eye image when horizontal shooting is performed using the fish-eye lens 5B as shown in FIG. The types of virtual lines are shown, and FIG. 9B is a diagram showing the cut-out areas specified from the three types of virtual lines.
First, the control unit 1 draws a line segment connecting the touch position and the center point of the fisheye image as a first virtual line (straight line) based on the touch position (step B1). Next, a pair of second virtual lines (straight lines) is drawn on the fisheye image (step B2). That is, a pair of sides that bisect the center point of the fisheye image with the apex of the triangle and the apex angle (θ) with the first virtual line as the middle line are drawn as the second virtual line. Further, a pair of third virtual lines (curves that slightly bulge outward) are drawn on the fisheye image (step B3). That is, a pair of line segments that are separated from each other by a predetermined distance (1 / 2H) in the center direction of the fisheye image and in the opposite direction from the touch position are drawn as the third virtual line. Then, a substantially trapezoidal region connecting the intersections A, B, C, and D between the pair of second virtual lines and the third virtual line is specified as a cutout region (step B4).

  When the process for specifying the cutout area (step A7 in FIG. 5) is completed in this way, the process proceeds to the next step A8, where the posture information of the camera at the time of imaging is recognized and the image in the cutout area is cut out (described later). 7), a trapezoidal distortion correction process (projection conversion process) is performed on the image in the cutout area (step A9). This trapezoidal distortion correction process is performed not only when the cutout area is a trapezoid but also when it is a rectangle close to a trapezoid. Note that, as the trapezoidal distortion correction process, a well-known technique generally used in image processing is used, and therefore a specific description thereof will be omitted. Further, a process for correcting the distortion of the fisheye image is performed (step A10). In this fish-eye distortion correction process, as described above, coordinate conversion is performed using a plane touching an arbitrary point on the virtual spherical model as a screen. Then, the process proceeds to a process (process shown in FIG. 8 described later) for notifying the position of the cutout area on the fisheye image together with the image (corrected image) in the cutout area (step A11).

FIG. 7 is a flowchart for explaining in detail the process (step A8 in FIG. 5) of recognizing the camera direction at the time of imaging and cutting out an image in the cutout area.
First, in a state where the selected image (fisheye image) is displayed on the playback screen of the touch display unit 6, the control unit 1 reads camera posture information at the time of imaging that is added to the fisheye image in advance ( Step C1) Based on the posture information, it is checked whether horizontal shooting or vertical shooting is detected (Step C2). When a fisheye image is shot vertically (vertical in Step C1) The image is cut out from the specified cutout area (substantially trapezoidal area) (step C3), and the camera orientations a to d are specified as the ground direction (step C4).

  On the other hand, when it is detected that the fisheye image is taken horizontally based on the camera posture information (taken horizontally in step C1), the image is cut out from the specified cutout region (substantially trapezoidal region) (step C5). ) As the ground direction of the cut-out image, the direction from the center of the fish-eye image toward its end is specified (step C6). That is, the direction from the center of the fisheye image toward the edge thereof is specified as the ground direction of the cut-out image regardless of the ground direction at the time of capturing the fisheye image. And the process which rotates to the ground direction which specified this cut-out image is performed (step C7). When detecting whether vertical shooting or horizontal shooting, the posture information (acceleration sensor measurement value, etc.) added to the fisheye image has exceeded a certain threshold value, and whether vertical shooting or horizontal shooting is taken. It is only necessary to determine whether or not it is possible to detect by other methods.

FIG. 8 is a flowchart for explaining the notification process (step A11 in FIG. 5) in detail.
First, the control unit 1 refers to the notification format table 3C to determine the notification format of the image (corrected image: corrected image) in the cutout region (step D1). FIG. 10 is a diagram for explaining the notification format table 3C. When displaying a corrected image after cutting out a part of the image in the fisheye image and performing distortion correction, which part in the fisheye image ( It is configured to store a plurality of notification formats for notifying whether the (position) has been cut out, and has items of “notification format”, “content”, and “selection flag”.

  “Notification format” indicates the type of notification format, and the example shown in the figure is a case where “A”, “B”, and “C” are stored as three types of notification format. “Content” indicates the content of the corresponding notification format. The notification format “A” indicates a cut-out area while switching between a fisheye image and a correction image, and the notification format “B” indicates a correction. The notification format “C” is a format in which the position of the cutout area is notified while displaying the fisheye image superimposed on a part of the image, while the correction image is displayed while being superimposed on the position of the cutout area on the fisheye image. A format for informing the position of the cutout area is shown.

  “Selection flag” indicates a notification method arbitrarily selected and designated in advance by the user from among three types of notification formats. In the figure, “1 (flag on”) is assigned corresponding to the notification format “B”. ) "Indicates that the notification method is selected by a user operation, and" 0 (flag off) "indicates that the notification method is not selected. In this case, the control unit 1 touches (selects) any notification format from the menu screen in a state where a plurality of predetermined notification formats are displayed in a list (menu display) on the touch display unit 6. ), The notification format table 3C is accessed, and the “selection flag” corresponding to the selected notification format is turned on (“1”).

  If the notification format “A” is selected by the user (selection flag is turned on) (step D1), a position mark (◯ mark) indicating the position of the cutout area is added on the fisheye image (step D2). At the same time, a ground mark (▽ mark) indicating the ground direction is added to the corrected image (step D3). Each time a switching operation is performed in this state, the control unit 1 switches the fish-eye image with the position mark (◯ mark) and the corrected image with the ground mark (▽ mark) to the playback screen of the touch display unit 6. Display (step D4). In this case, the fish-eye image or the correction image is switched and displayed by matching the ground direction of the fish-eye image or the correction image with the lower direction of the reproduction screen. FIG. 11 (1) is a diagram showing an example of switching display in the notification format A. In this case, the image (corrected image) in the cut-out area is enlarged and displayed on the entire reproduction screen, so that the user can check even the detailed portion and the ground mark (▽ mark). It becomes possible to confirm the direction of the ground by adding, and further, it is possible to confirm which part of the fish-eye image has been cut out by adding a position mark (◯ mark).

  When the notification format “B” is selected by the user (selection flag is on) (step D1), a position mark (◯ mark) indicating the position of the cut-out area is added on the fisheye image (step D5). A ground mark (▽ mark) indicating the ground direction is added on the corrected image (step D6). Then, in a state where the corrected image with the ground mark (▽ mark) is enlarged and displayed on the entire playback screen, the control unit 1 reduces the fisheye image to a predetermined size at the corner of the playback screen (corrected image). The superimposed images are superimposed and displayed simultaneously (step D7).

  In this case, the correction image is displayed on the reproduction screen by matching the ground direction of the correction image with the ground direction corresponding to the camera direction (direction of the reproduction screen) detected by the posture detection unit 7. FIG. 11 (2) is a diagram exemplifying the state of simultaneous display in the notification format B. The user can confirm the corrected image up to its detailed portion, and the ground mark (▽ mark). It becomes possible to confirm the direction of the ground by adding the, and further, which position is indicated by the position mark (○ mark) in the fish-eye image displayed at the corner (the lower right corner in the illustrated example) while viewing the corrected image It becomes possible to confirm at the same time whether the portion has been cut out.

  When the notification format “C” is selected by the user (selection flag is on) (step D1), the process proceeds to the next step D8, and a ground mark (示 す mark) indicating the ground direction is added on the corrected image. Then, the correction image is superimposed and displayed at the position of the cutout area on the fisheye image (step D9). That is, the control unit 1 aligns the center of the corrected image with the touch position on the fisheye image, and sets the ground direction of the corrected image in the outward direction of the fisheye image (on the line connecting the center of the cut-out area from the center of the fisheye image). In the direction away from the center of the fisheye image). FIG. 11 (3) is a diagram illustrating the state of simultaneous display in the notification format C. For the user, which part (position) on the fisheye image is displayed by the corrected image superimposed on the fisheye image. It is possible to confirm whether the image has been cut out, and it is possible to confirm the corrected image at the same time, and also to confirm the direction of the ground simultaneously by adding a ground mark (（).

  When the notification process (step A11 in FIG. 5) is completed in this way, it is checked whether the end of reproduction has been instructed (whether an operation to cancel the reproduction mode has been performed) (step A12), or a selected image (fisheye image). It is checked whether or not a switching operation for instructing switching is performed (step A13). If the end of reproduction is instructed (YES in step A12), the flow of FIG. 5 is exited, but if an image switching operation is performed (YES in step A13), the first step A1 is performed. Thereafter, the above-described operation is repeated.

  As described above, in the present embodiment, the control unit 1 cuts out a partial image as a display target from an image (fisheye image) captured using the fisheye lens 5B, and applies the image in the cutout region to the image in the cutout region. When displaying with distortion correction, the position of the cutout area on the fisheye image is specified and notified, so even if it is a fisheye image with characteristic distortion, It is possible to easily confirm which part (region) in the fisheye image is cut out and displayed.

  Since the control unit 1 identifies and displays the position of the cutout area on the fisheye image while the fisheye image is displayed, the correspondence between the fisheye image and the cutout image (corrected image) is determined. It will be clear.

  Since the control unit 1 is configured to identify and display the position by superimposing a predetermined mark on the position of the cut-out area in the state where the fisheye image is displayed, the identification display (notification) is intuitive and easy to understand. Is possible.

  Since the control unit 1 switches between the fisheye image in which the position of the cutout area is identified and displayed and the correction image obtained by correcting the cutout image, the correction image can be displayed on the entire screen. In this case, it is possible to confirm even the detailed portion of the corrected image, and it is possible to confirm which portion is cut out by the identification display in the fisheye image that has been switched and displayed.

  In the state where the corrected image cut out from the fisheye image and corrected is displayed, the control unit 1 simultaneously displays the corrected image and the fisheye image in a state where the position of the cutout region is identified and displayed. Therefore, it is possible for the user to check which part of the fisheye image has been cut out while comparing the corrected image with the fisheye image.

  In a state where the corrected image cut out from the fisheye image and corrected is displayed, the control unit 1 identifies and displays the position of the cutout region in a predetermined region (for example, a corner) of the corrected image. Since the eye image is superimposed and displayed, it is possible for the user to check the corrected image up to its detailed part, and at the same time which part is cut out by the identification display in the fish eye image. It becomes possible to confirm.

  In the state where the fisheye image is displayed, the control unit 1 displays the position by identifying the position by superimposing the correction image on the position of the cutout region. It is possible to easily confirm which part (position) on the fisheye image has been cut out by the correction image superimposed and displayed on the fisheye image, and to simultaneously confirm the correction image. .

  The control unit 1 refers to the notification format table 3C and is selected in advance when notifying the position of the clipped image on the fisheye image in a state where any one of the notification formats is selected. Since the position is notified according to the notification format, it is possible to notify the user's desire.

  When an arbitrary position on the fisheye image displayed on the touch display unit 6 is touched, an image in a predetermined area including the touch position is cut out from the fisheye image. Can arbitrarily specify a desired portion from the fish-eye image, and can confirm an image without distortion of the portion.

  In a state where the fisheye image is displayed on the touch display unit 6, the control unit 1 sets a line segment connecting the touch position on the touch panel 6 </ b> B and the center point of the fisheye image as a first virtual line. The center point of the eye image is a triangle vertex, and the apex angle is a pair of sides that bisect the first imaginary line as a middle line. A pair of line segments separated by a predetermined distance in the direction and the opposite direction is drawn as a third virtual line, and a trapezoid connecting each intersection of the pair of second virtual lines and the pair of third virtual lines is cut out. Since the region is specified as a region, a trapezoidal cutout region can be easily specified on a fisheye image, and a rectangular shape (for example, asbestos ratio 4: 3) can be obtained by performing trapezoidal distortion correction on the cutout region. ) Can be easily converted to images It can become.

  In addition, since the fish-eye image has a feature such that the shooting range becomes wider as it moves away from the center (towards the end), the cut-out area is trapezoidal or substantially trapezoidal, and the upper base is fish-eye. When the image is converted to a rectangular image by correcting trapezoidal distortion by directing the lower base toward the edge of the fisheye image toward the center of the image, there will be insufficient image components in the image portion corresponding to the lower base of the trapezoid. Can be prevented.

  In the above-described embodiment, as the notification format A, the fisheye image and the corrected image cut out from the image and corrected for distortion are displayed for each switching operation. Alternatively, the fisheye image and the corrected image may be displayed in parallel. In addition, when the notification format is set to B or C, a state in which the correction image is superimposed on the fisheye image and a state in which the fisheye image is superimposed on the correction image for each switching operation. You may make it the structure which switches.

  In the embodiment described above, the position mark (◯ mark) indicating the position of the cutout region is added to the fisheye image so as to be identifiable. However, the present invention is not limited to the case where the position mark (◯ mark) is added. The position of the cutout area is notified by blinking the entire cutout area, the position of the cutout area is notified by coloring a specific color in the cutout area, or the cutout area is displayed by displaying the cutout area with a thick frame You may make it alert | report the position of. Further, not limited to display, for example, the upper right corner, lower left corner, lower center, etc. may be output as voice messages to notify the position of the cutout region, and the notification method is arbitrary. is there. In the above-described embodiment, the ground direction can be confirmed by adding a ground mark (▽ mark) on the corrected image. However, if it is not necessary to confirm the ground direction, the ground mark may be omitted. In addition, the ground mark may be omitted and the correction image may be displayed so that the ground side is always in the predetermined direction of the display screen.

  In the above-described embodiment, when specifying the cutout region, the cutout region is not limited to a trapezoid, but is arbitrary. For example, a fisheye image has a shooting range as it moves away from the center (towards the end). Since it has features such as widening, the size of the cutout area increases, such as increasing the cutout area like a trapezoid in order to increase the image components as it moves away (towards the end) in consideration of the feature. These may be different depending on the distance from the center of the fisheye image.

  In the above-described embodiment, when the cutout area is specified according to the touch position, the touch position is specified so as to be the center of the cutout area. In other words, when a fisheye image is touched, an area to which the touch position belongs is specified as a cut-out area. In addition, when the whole fisheye image is divided into a plurality of regions in advance, the size of each divided region may be different depending on the distance from the center of the fisheye image in consideration of the characteristics of the fisheye lens 5B. .

  In the embodiment described above, in the case of the notification format “C”, as shown in FIG. 11 (3), the center of the corrected image is aligned with the touch position on the fisheye image, and the ground direction of the corrected image is the fisheye. The superimposed image is displayed so that it is in the outer direction of the image (the direction away from the center). On the contrary, the corrected image is superimposed in the ground direction of the fisheye image (the center direction). You may make it make it. Further, in the display example of FIG. 10 (3), the case where one correction image is superimposed and displayed on the fisheye image is illustrated. However, when a plurality of positions are touched on the fisheye image, each touch position is displayed. Alternatively, an image cut out at that position and corrected for distortion may be superimposed and displayed.

  In the above-described embodiment, the case where the fisheye lens 5B is attached to the imaging unit 5 instead of the normal imaging lens has been described. However, a digital camera in which the fisheye lens 5B is fixedly attached may be used. Moreover, although the case where it applied to the compact camera which integrated the imaging part 5 and the touch display part 6 was illustrated, the main body provided with the imaging device (housing | housing on the imaging side) provided with the imaging part 5, and the touch display part 6 The camera may be a separate type camera that is separated into a device (a housing on the main body side) and separated. In this case, what is necessary is just to comprise so that transmission / reception is possible via near field communication between an imaging device and a main body apparatus.

  In the above-described embodiments, the case where the present invention is applied to a digital camera as an image display device has been shown. However, the present invention is not limited to a camera, but a personal computer having a display function, a personal digital assistant (PDA), a tablet terminal device, It may be a mobile phone such as a smartphone, an electronic game, an electronic clock, a music player, or the like.

  Further, the “apparatus” and “unit” shown in the above-described embodiments may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit An image display device comprising display means for displaying the corrected image.
Position specifying means for specifying the position of the cutout region on the fisheye image;
Informing means for informing the position of the cutout area specified by the position specifying means;
It is characterized by providing.
(Claim 2)
The invention according to claim 2 is the image display device according to claim 1,
The notification means identifies and displays the position of the cutout area specified by the position specifying means on a fisheye image;
It is characterized by that.
(Claim 3)
The invention according to claim 3 is the image display device according to claim 2,
The notification means identifies and displays the position by superimposing a predetermined mark on the position of the cutout area specified by the position specifying means in a state where the fisheye image is displayed.
It is characterized by that.
(Claim 4)
The invention according to claim 4 is the image display device according to claim 2 or 3,
The display unit switches between the fisheye image in which the position of the cutout area is identified and displayed by the notification unit and the correction image cut out by the cutout unit and corrected by the correction unit. ,
It is characterized by that.
(Claim 5)
The invention according to claim 5 is the image display device according to claim 2 or 3,
The display means identifies and displays the corrected image and the position of the cutout area specified by the position specifying means in a state where the correction image cut out by the cutout means and corrected by the correction means is displayed. Simultaneously displaying the fisheye image in a state of being
It is characterized by that.
(Claim 6)
The invention according to claim 6 is the image display device according to claim 2 or 3,
In the state where the fisheye image is displayed, the display unit superimposes the correction image cut out by the cutout unit and corrected by the correction unit at the position of the cutout region specified by the position specifying unit. The position is identified and displayed by displaying
It is characterized by that.
(Claim 7)
The invention according to claim 7 is the image display device according to any one of claims 1 to 6,
A selection means for selecting an arbitrary notification format from a plurality of predetermined notification formats;
The notification means notifies the position of the cutout area specified by the position specifying means according to a notification format selected by the selection means,
It is characterized by that.
(Claim 8)
The invention according to claim 8 is the image display device according to any one of claims 1 to 7,
The display means includes a touch panel,
The cutout means cuts out an image in a predetermined area including the touch position from the fisheye image when an arbitrary position on the fisheye image displayed on the display means is touched.
It is characterized by that.
(Claim 9)
The invention according to claim 9 is the image display device according to claim 8,
When the fisheye image is displayed on the display unit and the arbitrary position on the touch panel is touch-designated, the clipping unit calculates the touch position and the center point of the fisheye image. A connecting line segment is a first virtual line, a center point of the fish-eye image is a vertex of a triangle, and a pair of sides that bisect its apex angle with the first virtual line as a middle line is a second virtual line And a pair of line segments separated by a predetermined distance in the center direction of the fisheye image and the opposite direction from the touch position as a third virtual line, and the pair of second virtual lines and the pair of pairs Cut out the image in that area using the trapezoid connecting each intersection with the third virtual line as the cutout area,
It is characterized by that.
(Claim 10)
The invention according to claim 10 is:
A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit An image display method in an image display device comprising display means for displaying a corrected image,
Processing for specifying the position of the cutout region on the fisheye image;
A process of notifying the position of the specified cutout area;
It is characterized by including.
(Claim 11)
The invention according to claim 11
A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit For a computer of an image display device comprising display means for displaying the corrected image,
A function of specifying the position of the cutout region on the fisheye image;
A function of informing the position of the identified cutout region;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 1 Control part 3 Memory | storage part 3A Program memory 3C Notification format table 4 Operation part 5 Imaging part 5B Fisheye lens 5C Imaging element 6 Touch display part 6B Touch panel 7 Posture detection part A, B, C Notification format

Claims (11)

A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit An image display device comprising display means for displaying the corrected image.
Position specifying means for specifying the position of the cutout region on the fisheye image;
Informing means for informing the position of the cutout area specified by the position specifying means;
An image display device comprising: The notification means identifies and displays the position of the cutout area specified by the position specifying means on a fisheye image;
The image display apparatus according to claim 1. The notification means identifies and displays the position by superimposing a predetermined mark on the position of the cutout area specified by the position specifying means in a state where the fisheye image is displayed.
The image display device according to claim 2. The display unit switches between the fisheye image in which the position of the cutout area is identified and displayed by the notification unit and the correction image cut out by the cutout unit and corrected by the correction unit. ,
The image display device according to claim 2, wherein the image display device is an image display device. The display means identifies and displays the corrected image and the position of the cutout area specified by the position specifying means in a state where the correction image cut out by the cutout means and corrected by the correction means is displayed. Simultaneously displaying the fisheye image in a state of being
The image display device according to claim 2, wherein the image display device is an image display device. In the state where the fisheye image is displayed, the display unit superimposes the correction image cut out by the cutout unit and corrected by the correction unit at the position of the cutout region specified by the position specifying unit. The position is identified and displayed by displaying
The image display device according to claim 2, wherein the image display device is an image display device. A selection means for selecting an arbitrary notification format from a plurality of predetermined notification formats;
The notification means notifies the position of the cutout area specified by the position specifying means according to a notification format selected by the selection means,
The image display device according to claim 1, wherein the image display device is an image display device. The display means includes a touch panel,
The cutout means cuts out an image in a predetermined area including the touch position from the fisheye image when an arbitrary position on the fisheye image displayed on the display means is touched.
The image display device according to claim 1, wherein the image display device is an image display device. When the fisheye image is displayed on the display unit and the arbitrary position on the touch panel is touch-designated, the clipping unit calculates the touch position and the center point of the fisheye image. A connecting line segment is a first virtual line, a center point of the fish-eye image is a vertex of a triangle, and a pair of sides that bisect its apex angle with the first virtual line as a middle line is a second virtual line And a pair of line segments separated by a predetermined distance in the center direction of the fisheye image and the opposite direction from the touch position as a third virtual line, and the pair of second virtual lines and the pair of pairs Cut out the image in that area using the trapezoid connecting each intersection with the third virtual line as the cutout area,
The image display apparatus according to claim 8. A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit An image display method in an image display device comprising display means for displaying a corrected image,
Processing for specifying the position of the cutout region on the fisheye image;
A process of notifying the position of the specified cutout area;
An image processing method comprising: A cutting unit that cuts out a part of a fisheye image captured using a fisheye lens as a cutout region, a correcting unit that corrects distortion of the image cut out by the cutting unit, and the correction unit For a computer of an image display device comprising display means for displaying the corrected image,
A function of specifying the position of the cutout region on the fisheye image;
A function of informing the position of the identified cutout region;
A program to realize

Images