JP7353821B2 - Image processing device, its control method, program - Google Patents

Description

The present invention relates to an image processing device, and particularly to a technique for processing images over a wide viewing angle.

2. Description of the Related Art In recent years, it has become common practice to view captured images, such as omnidirectional images, captured at a wider angle of view than that captured by a conventional camera using an HMD (Head Mounted Display). An omnidirectional image is an image captured by a camera using an omnidirectional camera, a camera using a fisheye lens, or the like, and which covers the entire range of 360 degrees in the horizontal and vertical directions within the angle of view.

For example, in Patent Document 1, a virtual space is generated by pasting an omnidirectional panoramic image taken with an omnidirectional camera on the inner wall of a virtual three-dimensional sphere, and a desired location in the generated virtual space is displayed on an HMD. A technology for visual recognition has been disclosed.

Furthermore, Patent Document 1 also discloses setting which direction of the photographed image is displayed as the initial direction (front direction) when the user visually confirms the image using the HMD. It is said that it is common to set this as the front direction when viewing the HMD.

Japanese Patent Application Publication No. 2017-208809

However, for example, when viewing a video shot while moving the camera on an HMD or the like, if the front direction for the user at the time of viewing is misaligned with the direction in which the camera is moving, the user is likely to feel uncomfortable.

One of the image processing devices according to the present invention includes a video acquisition unit that acquires a video with a wide viewing angle, and a front direction that specifies a front direction that is a reference direction when playing the video acquired by the video acquisition unit. identifying means; a motion information acquisition means for acquiring motion information of the video acquired by the video acquisition means; and a traveling direction identification for determining the traveling direction of the video based on the motion information acquired by the motion information acquisition means. means, a calculating means for calculating information indicating a degree of discomfort based on a difference between the front direction specified by the front direction specifying means and the traveling direction specified by the direction specifying means, and information indicating the degree of discomfort. is larger than a predetermined threshold , output means outputs predetermined warning information , and the information indicating the degree of discomfort includes a front direction specified by the front direction specifying means and a forward direction specified by the traveling direction specifying means. The larger the difference in the direction of travel, the larger the difference .

Advantageous Effects of Invention According to the present invention, it is possible to reduce the discomfort of a user who views images with a wide viewing angle.

FIG. 1 is a block diagram showing the configuration of a digital camera according to the present invention. FIG. 2 is a block diagram showing the configuration of an image processing section in the present invention. FIG. 3 is a block diagram showing motion information in the present invention. FIG. 3 is a diagram showing a warning control flow during photographing according to the present invention. It is a figure showing the characteristic for discomfort level calculation in the present invention. It is a figure which shows the example of a warning at the time of photography in this invention. FIG. 3 is a diagram showing an image selection control flow during playback in the present invention. FIG. 6 is a diagram showing an example of image selection during playback in the present invention. FIG. 6 is a diagram showing a control flow when editing in the front direction according to the present invention. FIG. 7 is a diagram showing an example of an output image when editing in the front direction according to the present invention. FIG. 6 is a diagram showing control characteristics during front direction editing in the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail based on the accompanying drawings. In this embodiment, an example will be described in which a digital camera, which is a type of imaging device, is used as an example of an image processing device. Of course, the present invention is not limited to digital cameras, and can be used in mobile terminals such as so-called smartphones and tablet devices, surveillance cameras, medical equipment, and various measuring instruments. Further, the operations during image capture, playback, and editing, which will be described later, do not necessarily need to be performed by the same device, and various devices having means for handling images can be used. In other words, a first device performs the shooting operation, a second device that captures the captured image performs the playback operation, and a second device that is different from the second device or the second device that captures the captured image performs the shooting operation. The editing operation may be performed using the device No. 3.

<First embodiment>
A digital camera according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 11.

FIG. 1 is a block diagram showing an example of the configuration of a digital camera according to an embodiment of the present invention.

In FIG. 1, 100 is the entire digital camera, and 101 and 103 are optical systems including a zoom lens and a focus lens. Reference numerals 102 and 104 designate image pickup units that include an image pickup device such as a CCD or CMOS device that converts an optical image into an electrical signal, and an A/D converter that converts an analog signal into a digital signal. An image processing unit 105 performs various image processing such as white balance processing, γ processing, edge enhancement, and color correction processing on the image data output from the imaging units 102 and 104. 106 is an image memory, 107 is a display unit such as an LCD, and 108 is a codec unit that compresses and encodes and decodes image data.

109 is an interface I/F with the recording medium 110; 110 is a recording medium such as a memory card or hard disk; and 50 is a system control unit that controls the entire system of the digital camera 100.

Further, 121 is a nonvolatile memory such as an EEPROM that stores programs, parameters, and the like. A system memory 122 stores constants, variables, and programs read from the nonvolatile memory 124 for the operation of the system control unit 50. Reference numeral 123 denotes an operation unit that accepts input operations from the user.

Figure 6(A) shows the external appearance of this digital camera. The two optical systems 101 and 103 are arranged 180 degrees in opposite directions as shown in FIG. 6(A), and the two optical systems and the imaging section can capture images of the entire 360 degrees.

Next, the basic operation when photographing a subject in the digital camera 100 configured as described above will be explained. The imaging units 102 and 104 photoelectrically convert the light that has entered through the optical systems 101 and 103, and output it to the image processing unit 105 as an input image signal.

The image processing unit 105 performs color conversion processing such as white balance, γ processing, contour enhancement processing, etc. on the image data output from the imaging units 102 and 104. Further, the image processing unit 105 performs a predetermined evaluation value calculation process (not shown) using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained evaluation value result. . As a result, TTL (through-the-lens) type AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, etc. are performed.

The image data output from the image processing unit 105 is written into the image memory 106. The image memory 106 stores image data output from the imaging section and image data to be displayed on the display section 107.

The display unit 107 displays images, operation instruction information for the user (described later), etc. on a display such as an LCD.

The codec unit 106 compresses and encodes the image data recorded in the image memory 106 based on standards such as JPEG and MPEG. The system control unit 50 associates the encoded image data and stores it in the recording medium 110 via the recording interface 109.

The basic operations when photographing a subject have been explained above.

In addition to the basic operations described above, the system control unit 50 executes the programs recorded in the nonvolatile memory 124 described above to realize various processes of the present embodiment described later. The program here refers to a program for executing various flowcharts described later in this embodiment. At this time, constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 121, and the like are loaded into the system memory 122.

Next, details of the image processing unit 105 will be explained using FIG. 2. FIG. 2 is a block diagram showing the configuration of the image processing unit 105.

In FIG. 2, 200(a) and 200(b) are synchronization processing units, 201(a) and 201(b) are WB amplification units, and 202(a) and 202(b) are brightness/color signal generation units. (a) and (b) indicate that two systems of the same block are provided. 203 is a composition processing unit, and 204 is a motion detection unit.

Next, processing in the image processing unit 105 will be explained. The image signal output from the imaging unit 102 in FIG. 1 is input to the synchronization processing unit 200(a), and the image signal output from the imaging unit 104 is input to the synchronization processing unit 200(b).

The synchronization processing unit 200 performs synchronization processing on the inputted Bayer RGB image data to generate color signals R, G, and B. The WB amplification unit 201 applies a gain to the RGB color signals based on the white balance gain value calculated by the system control unit 50 to adjust the white balance. The RGB signals output from the WB amplification section 201 are input to the luminance/color signal generation section 202. The luminance/color signal generation unit 202 generates a luminance signal Y from the RGB signals, performs various processing on the luminance signal Y such as edge enhancement processing and luminance gamma correction, and outputs the resultant signal. Further, a matrix calculation or the like is performed on the color signals RGB to convert them to a desired color balance, and after performing gamma correction, a color difference signal UV is generated.

The synthesis processing unit 203 performs panoramic synthesis of the images respectively output from the luminance/color signal generation units (a) and (b) to generate one omnidirectional image. The images taken by the imaging units 102 and 104 cover a range of 180 degrees horizontally and vertically, and by combining them, it is possible to generate a 360-degree all-around image. FIG. 3A shows an example of an image developed by developing the combined 360-degree all-around image using the Mercator projection.

The 360-degree all-around image synthesized by the synthesis processing section 203 is output to the image memory 106 and also to the motion detection section 204. The motion detection unit calculates a representative value (median value or average value) of motion vectors in block units as shown in FIG. 3(B). An example of the calculated motion vector is shown in FIG. 3(C). By calculating the representative value of the motion vector for each block, the direction of movement of the entire image can be detected without detecting the movement of a local moving object. The motion detection unit 204 is an example of motion information acquisition means.

The image processing unit 105 outputs the combined omnidirectional YUV image and motion vector information.

The YUV image signal output from the image processing section 105 to the image memory 106 is compressed and encoded by the codec section 110 and recorded on the recording medium 200. The calculated motion vector information is used in various controls described later.

As described above, the image processing unit 105 functions as a moving image acquisition unit and processes images captured by the imaging units 102 and 104, thereby making it possible to acquire a 360-degree all-around image. In this case, the imaging units 102 and 104 and the image processing unit 105 are an example of working together as a moving image acquisition unit.

(Warning control during shooting)
Next, a processing flow in which the system control unit 50 displays a warning based on the photographing information when photographing an image will be described using FIG. 4. The system control unit 50 determines whether or not to issue a warning to the user according to the flowchart in FIG. 4 at every fixed period of shooting (at the time of shooting in the case of a still image), and if it is determined that a warning should be issued, the system control unit 50 displays the The unit 107 is controlled to issue a warning display. In other words, the system control unit 50 can execute the process according to the flowchart of FIG. 4 while capturing a moving image.

Note that, prior to the processing in the flowchart of FIG. 4, the digital camera 100 receives a menu operation (not shown) by the user, and sets the front direction (reference direction) to be displayed on the HMD. The front direction displayed on the HMD can be selected from, for example, the front direction of the camera, the 45-degree right direction of the camera, the 45-degree left direction of the camera, and the like. When determining the front direction of the camera in this way, the system control unit 50 functions as a front direction specifying means. In this embodiment, a case will be described in which the front direction of the camera is set as the front direction displayed on the HMD.

FIG. 4 is a flowchart for performing warning control during photographing.

First, in step S401, the system control unit 50 determines whether the current shooting mode is a video shooting mode or a still image shooting mode. If it is determined that the mode is still image shooting mode, the direction of movement will not cause discomfort, and therefore the process proceeds to step S409, where warning information "no warning" is recorded as metadata. On the other hand, if it is determined that the mode is video shooting mode, the process advances to step S402.

In step S402, the system control unit 50 acquires motion vector information detected by the motion detection unit 204. An example of the detected motion vector is shown in FIG. 3(C).

In step S403, the system control unit 50 estimates the direction in which the camera body is moving during shooting, based on the acquired motion vector information. The motion vector shown in FIG. 3(C) is used to estimate the traveling direction. Specifically, among vertically adjacent blocks, the upper block has an upward vector, and the lower block has a downward vector. In the case of the example in FIG. 3C, adjacent blocks 301 and 302 correspond. When a corresponding adjacent block is found, the position of this adjacent block is determined to be the traveling direction. When a plurality of adjacent blocks are detected as shown in FIG. 3(C), the average value of the detected horizontal positions of the plurality of blocks is taken as the traveling direction. In the example of FIG. 3C, the positions of blocks 301 and 302 are averaged, and the direction of 303 is determined as the traveling direction. In this step, the system control unit 50 functions as a traveling direction specifying means.

In step S404, the system control unit 50 calculates the moving speed of the camera. To estimate the moving speed of the camera, refer to the motion vectors of the blocks (301 and 302 in the case of FIG. 3C) near the advancing direction detected in step S403, and move the average value of the length of each motion vector. Let it be the speed.

In step S405, the system control unit 50 calculates the degree of discomfort based on the information on the traveling direction and moving speed calculated in steps S403 and S404. Hereinafter, calculation of the degree of discomfort will be explained using FIG. 5.

First, the system control unit 50 calculates the difference between the direction of movement of the camera and the front direction displayed on the HMD in order to calculate the degree of discomfort. In this embodiment, as described above, the front direction displayed on the HMD is set in advance to be the same as the front direction of the camera. In this case, the relationship between the front direction displayed on the HMD (=the front direction of the camera) and the direction of movement of the camera is shown in FIG. 5(A). FIG. 5(A) is a view looking down on the camera from directly above, where 100 is the camera body, 501 is the front direction displayed on the HMD (=the front direction of the camera), and 502 is the direction of movement of the camera. ing. Here, the angular difference between the front direction displayed on the HMD (=the front direction of the camera) and the direction of movement of the camera as shown in 503 is calculated.

Next, the degree of angular discomfort is calculated based on the angular difference between the front direction displayed on the HMD (=the front direction of the camera) and the direction of movement of the camera. The degree of angular discomfort can be calculated from the graph characteristics shown in FIG. 5(B). As shown in FIG. 5B, the degree of angular discomfort increases as the angle difference between the front direction displayed on the HMD (=the front direction of the camera) and the direction of movement of the camera increases.

Furthermore, the system control unit 50 calculates the degree of speed discomfort based on the moving speed. The degree of speed discomfort can be calculated from the graph characteristics shown in FIG. 5(C). The speed discomfort level is such that the higher the moving speed of the camera, the higher the discomfort level becomes.

After calculating the degree of angular discomfort and the degree of speed discomfort as described above, the system control unit 50 adds the weights to these and calculates the total degree of discomfort. The overall discomfort level can be calculated as follows.
Total discomfort level = α x Angle discomfort level + β x Speed discomfort level Here, α and β are weighting coefficients of 0 or more and 1 or less.

Returning to FIG. 4, in step S406, the system control unit 50 determines whether the overall discomfort level is equal to or higher than a certain threshold value for a predetermined period of time or more. Calculation of the overall discomfort level in step S405 is performed at regular time intervals during imaging, and if it is determined that the overall discomfort level has exceeded a predetermined threshold value for a period longer than the predetermined threshold value, step S405 is performed. Proceed to S407.

In step S407, the system control unit 50 generates warning information that prompts the user to perform an operation to reduce the level of discomfort. Specifically, the main factors that increase the degree of discomfort are determined, and warning information for reducing the degree of discomfort is generated. In the case of this embodiment, the weighted angular discomfort level and speed discomfort level are compared, and the larger one is determined to be the main factor increasing the discomfort level. Then, warning information is generated that prompts the user to perform an operation that reduces the main factor of discomfort. Specifically, if angle discomfort is determined to be the main factor, "information that instructs which direction the camera should face" is generated, and if speed discomfort is determined to be the main factor, "movement" is generated. Generates "information instructing the driver to slow down."

In step S408, the system control unit 50 displays the generated warning information on the display unit 107. FIG. 6 shows an example of warning information display. In FIG. 6, (A) shows the camera body, and 107 is a display section. FIG. 6B shows an example of a display when it is determined that the degree of angular discomfort is the main factor, and is an example of guidance that prompts the user to change the imaging direction. In FIG. 6(B), text information 601 and an arrow 602 are used to indicate which direction the camera should face. FIG. 6C shows a display example when the speed discomfort level is the main factor, and is an example of guidance that prompts the user to suppress the movement of the video. In FIG. 6C, text information is used to warn the user to reduce his or her movement speed. Note that in the present embodiment, warning information that prompts the user to perform an operation that reduces the main factor of discomfort level is generated, but the present invention is not necessarily limited to this form, and various types of notification information may be output to the user. For example, a warning such as ``There is a high possibility that the image will be highly unpleasant'' may be used, or a warning such as ``The direction of travel and the direction of photography are very different'' or ``The speed of movement is too fast'' may be used. Furthermore, the notification may be output to the user by displaying not only text information but also a predetermined icon or the like.

In step S409, the system control unit 50 records the warning information as metadata in the image data (warning information metadata). Information to be recorded includes information on the presence or absence of a warning, traveling direction information, moving speed information, various discomfort level information, and the set front direction of the HMD.

As described above, the digital camera 100 according to the present embodiment can output a warning display according to the shooting state during shooting. By displaying the warning as described above, it is possible to reduce the situation where the user unintentionally takes a picture under conditions that cause discomfort during viewing, such as when the front direction of the HMD and the direction of movement are misaligned. I can do it.

Note that in this embodiment, the moving direction of the camera is estimated based on the motion vector in the image, but other configurations may be used as long as the moving direction of the camera can be estimated. For example, a configuration may be adopted in which the traveling direction is estimated using GPS time series information, a direction sensor, or the like.

Furthermore, in this embodiment, the case where the front direction displayed on the HMD is set to be the same as the front direction of the camera has been described, but the front direction displayed on the HMD is not limited to this, and the user It shall be possible to freely set the settings.

Further, in this embodiment, a case has been described in which warning information is output to the display unit 107, but any method may be used as long as the warning is conveyed to the photographer. For example, it is possible to equip the camera with an LED and issue a warning based on the position, color, lighting pattern, etc. of the LED, or to send warning information to another device such as a smartphone.

Further, in this embodiment, a case has been described in which a 360-degree all-around image is handled, but the present invention is applicable to other wide-viewing-angle images, not necessarily a 360-degree all-around image. For example, it may also be applied to a 180 degree wide-angle image.

(Control during playback)
Next, a processing flow in which the system control unit 50 performs playback control based on warning information will be described using FIG. 7. The system control unit 50 selects an image to be reproduced from among the images recorded on the recording medium 110 according to the flow shown in FIG. 7, and outputs the selected image to the display unit 107.

FIG. 7 is a flowchart of playback control by the system control unit 50 based on warning information.

First, in step S701, the system control unit 50 determines whether the playback mode is the "sickness reduction mode." In this embodiment, the image reproduction mode includes a "normal reproduction mode" and a "sickness reduction reproduction mode." The system control unit 50 can switch between the two playback modes in response to a user's operation on a menu screen (not shown). The "sickness reduction mode" is a mode in which images with a high degree of discomfort and which are likely to cause motion sickness are excluded from playback targets and reproduced. If the "sickness reduction mode" is selected, the process advances to step S702, and if the "normal playback mode" is selected, the process advances to step S704.

In step S702, the system control unit 50 extracts the warning information metadata recorded at the time of photography from the image data recorded on the recording medium 110.

In step S703, the system control unit 50 extracts only images without a warning according to the extracted warning information, and generates a reproducible image list.

On the other hand, in step S704, the system control unit 50 generates all images recorded on the recording medium 110 for the "normal playback mode" as a playable image list.

In step S705, the system control unit 50 displays images recorded on the recording medium 110 as thumbnails. At this time, images that are not included in the above-mentioned reproducible image list are grayed out or hidden so that they cannot be reproduced. FIG. 8(A) shows an example of thumbnail display. In FIG. 8A, thumbnails of each image recorded on the recording medium 110 are displayed, but images that are not included in the reproducible image list are displayed in grayout as shown at 801, so that they cannot be selected.

In step S706, the system control unit 50 accepts image selection by user operation, and controls the display unit 107 to display the selected image.

The processing flow in which the system control unit 50 performs playback control based on warning information has been described above. As described above, when the "sickness reduction mode" is selected, images for which a warning of discomfort level was determined at the time of shooting are not played back. This makes it possible to reduce the possibility of playing back images that tend to cause discomfort.

In addition, in the above embodiment, an image for which a warning has been determined is not played in image file units, but if a warning is displayed only in a part of the video, only the range where the warning is displayed is excluded. It is also possible to adopt a configuration in which the data is played back. For example, when selecting a playlist and sequentially playing back the images included in the playlist, in the case of the "sickness reduction mode", it is also possible to exclude only the range where the warning is displayed and playback. An example of this is shown in FIG. 8(B). In FIG. 8(B), a timeline of moving images consisting of moving images 1 to 6 is shown. Warning information shall be recorded as metadata in each frame of a moving image. It is assumed that video 4 includes a warning determination in the area 802, and video 5 includes a warning determination throughout (803). In this case, in the case of "sickness reduction mode", the area where the warning is displayed is excluded from playback.

Furthermore, in the embodiment described above, control is performed to perform playback or not depending on the presence or absence of a warning, but it is also possible to perform control other than playback or non-playback. For example, if an image that causes discomfort is included, in the case of the "sickness reduction mode", it is possible to reduce the discomfort by using a different reproduction method from normal reproduction. As a specific example, angle discomfort degree information and speed discomfort degree information are each recorded as metadata. When the degree of angular discomfort is high, the front direction of the HMD is changed and played back so that the degree of angular discomfort is reduced. Furthermore, when the degree of speed discomfort is high, control is performed to reduce the reproduction speed to reduce the degree of discomfort. By controlling the reproduction method in this way, it is possible to reduce discomfort.

Further, in the above embodiment, a case has been described in which playback control is performed based on metadata of warning information generated at the time of shooting, but it is also possible to adopt a configuration in which warning information and discomfort level are calculated at the time of playback.

(Warning control during editing)
Next, a warning processing flow when the system control unit 50 edits the front direction displayed on the HMD will be described using FIG. 9. The system control unit 50 can receive an operation from the user and later edit the front direction to be displayed on the HMD, which is set in the image recorded on the recording medium 110.

FIG. 9 shows a warning processing flow when the front direction displayed on the HMD is edited later.

First, in step S901, the system control unit 50 selects, by user operation, an image file whose front direction to be displayed on the HMD is desired to be edited. FIG. 10 is a diagram showing an editing screen in the front direction displayed on the display unit 107. The selected image is developed and displayed in the Mercator projection as shown in 1001 in FIG. In FIG. 10, 1004 indicates the currently set front direction of the HMD. The user can freely change the front direction 1004 of the HMD on the editing screen of FIG. 10.

In step S902, the system control unit 50 acquires traveling direction information and moving speed information from the metadata of the image file.

In step S903, the system control unit 50 calculates an angle range that is permissible as the front direction of the HMD display based on the moving direction information of the camera and the moving speed information acquired from the metadata. Calculation of the allowable angle range for the front direction is determined based on the characteristics of the graph shown in FIG. Specifically, the faster the moving speed of the camera during photographing, the narrower the permissible range in the front direction. FIG. 10 shows the allowable angle range centered on the direction of movement of the camera. In FIG. 10, 1002 indicates the direction of movement of the camera, and a calculated tolerance range 1003 is set around the direction of movement of the camera.

In step S904, the system control unit 50 determines whether to issue a warning based on the camera traveling direction, the currently set front direction of the HMD, and the front tolerance range calculated in step S903. Specifically, it is determined whether the currently set front direction of the HMD is within the permissible front direction. In the example of FIG. 10, the front direction of the HMD is 1004, which is not included in the allowable range 1003, so it is determined that a warning should be issued.

In step S905, the system control unit 50 displays a warning if it is determined in step S904 that a warning should be issued. In the example of FIG. 10, 1005 is an example of a warning. Further, in FIG. 10, 1006 indicates the timeline of the video data currently being edited, and indicates that there is an area in which a warning has occurred in a portion 1007.

The processing flow when the system control unit 50 edits the front direction displayed on the HMD has been described above. In this way, when editing the front direction displayed on the HMD, the user can visually check the warning information while controlling the front direction displayed on the HMD, making it difficult to generate images that cause discomfort.

<Other embodiments>
Further, the present invention can also be realized by executing the following processing. That is, the software (program) that realizes the functions of the embodiments described above is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

Claims (7)

a video acquisition means for acquiring a wide viewing angle video;
a front direction specifying means for specifying a front direction that is a reference direction when playing the video acquired by the video acquisition means;
motion information acquisition means for acquiring motion information of the video acquired by the video acquisition means;
a traveling direction identifying means for identifying a traveling direction of the moving image based on the motion information acquired by the motion information acquiring means;
Calculating means for calculating information indicating a degree of discomfort based on the difference between the front direction specified by the front direction specifying means and the traveling direction specified by the traveling direction specifying means;
output means for outputting predetermined warning information when the information indicating the degree of discomfort is greater than a predetermined threshold ;
The image processing device is characterized in that the information indicating the degree of discomfort increases as the difference between the front direction specified by the front direction specifying means and the traveling direction specified by the traveling direction specifying means increases. The image processing apparatus according to claim 1, wherein the warning information is recorded in association with the moving image acquired by the moving image acquiring means. The calculation means further calculates information indicating the degree of discomfort based on the moving speed of the imaging device estimated based on the movement information acquired by the movement information acquisition means,
3. The image processing apparatus according to claim 1 , wherein the information indicating the degree of discomfort increases as the moving speed increases . The video acquisition means includes an imaging means,
The image processing apparatus according to any one of claims 1 to 3 , wherein the output means is capable of outputting a notification prompting a change in the imaging direction as the warning information during imaging of the moving image. The video acquisition means includes an imaging means,
The image processing according to any one of claims 1 to 4 , wherein the output means is capable of outputting a notification urging suppression of movement of the moving image as the warning information during imaging of the moving image. Device. A method for controlling an image processing device, the method comprising:
a video acquisition step of acquiring a wide viewing angle video;
a front direction identification step of identifying a front direction that is a reference direction when playing the video acquired in the video acquisition step;
a motion information acquisition step of acquiring motion information of the video acquired in the video acquisition step;
a traveling direction identifying step of identifying the traveling direction of the video based on the motion information acquired in the motion information acquiring step;
a calculating step of calculating information indicating a degree of discomfort based on the difference between the front direction specified in the front direction specifying step and the traveling direction specified in the traveling direction specifying step;
an output step of outputting predetermined warning information when the information indicating the degree of discomfort is greater than a predetermined threshold ;
A method for controlling an image processing device having the following. A program executable by a computer that causes the computer to function as each means of the image processing apparatus according to any one of claims 1 to 5 .

Images