JP7214475B2 - Electronic device and its control method - Google Patents

Description

TECHNICAL FIELD The present invention relates to an electronic device, a control method, a program, and a storage medium for the electronic device, and more particularly to a control method for displaying an image having a wide range of video.

2. Description of the Related Art In recent years, imaging apparatuses capable of capturing an image having a wider range than the human viewing angle, such as an omnidirectional image and an omnidirectional image, have become widespread. In addition, by displaying a part of the image with such a wide range of images on the display and changing the range of the image displayed on the display (display range) according to the change in the posture of the device, it is possible to create a sense of immersion and There is also known a method of displaying a highly realistic image (VR view).

In taking an omnidirectional image, there are many cases in which an unnecessary subject such as the photographer himself is captured. Therefore, it is useful for electronic devices capable of reproducing an omnidirectional image to provide a function that allows the user to display an image by extracting only a necessary portion or excluding an unnecessary portion.

Patent Documents 1 and 2 disclose techniques for specifying an arbitrary position in an omnidirectional image by a user and extracting a range including the arbitrary position from the omnidirectional image.

JP 2013-127739 A Japanese Unexamined Patent Application Publication No. 2012-29180

Patent Documents 1 and 2 disclose a method of extracting a normal image that is not VR content from an omnidirectional image (VR content). No consideration is given to clipping. When extracting a narrower range of VR content from VR content, although the range of the video of the VR content after clipping is narrower than before clipping, the range is larger than the display range that can be displayed at once in a normal magnification VR view. is also wide. Therefore, in the process of cutting out a narrower range of VR content from VR content, it is difficult to confirm which range of the VR content before cutting is included in the video range of the VR content after cutting.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a technique that makes it possible to more easily confirm the range of the clipped VR content when generating a narrower range of VR content from the VR content.

In order to solve the above problems, the electronic device of the present invention includes:
In the first screen, control is performed so that a partial range of the VR content having the first image range is displayed on the display means as a display range, and in response to a change in posture of the electronic device or a user's operation to change the display range display control means for changing the position of the display range;
receiving means for receiving a cutting instruction from a user;
generating means for generating edited VR content having a second video range narrower than the first video range of the VR content;
A second display in which a range of the VR content, which is wider than the partial range and includes the entire second video range, is displayed in one screen so that the extent of the second video range can be identified. and control means for controlling to display a screen.

According to the present invention, when generating VR content with a narrower range from VR content, it is possible to more easily check the range of the clipped VR content video.

1A and 1B are an external view and a block diagram of a digital camera; FIG. They are an external view, a block diagram, etc. of an electronic device. 6 is a flowchart of VR view display; 10 is a flowchart of cut-out editing processing; 10 is a flowchart of clipping preview processing. It is an example of a VR view display, cutout edit screen, and preview screen. It is an example of the whole hemisphere display. 4 is a flowchart of dual-screen display processing; This is an example of two-screen display. 9 is a flowchart of end fine adjustment processing; It is an example of an end fine adjustment screen. 8 is a flowchart of unnecessary part designation processing; It is an example of an unnecessary part specification screen. FIG. 10 is a schematic diagram showing an example of a flow until a selection invalid range is set; It is an example of a cutout editing screen and a preview screen. FIG. 10 is a schematic diagram showing an example of a difference in clipping depending on whether the elevation/depression angle is enabled/disabled; It is an example of the guidance display of the extraction range. It is an example of various displays of VR images.

Preferred embodiments of the present invention will be described below with reference to the drawings. FIG. 1A is a front perspective view (external view) of a digital camera 100 (imaging device). FIG. 1B is a rear perspective view (outside view) of the digital camera 100. FIG. The digital camera 100 is an omnidirectional camera (omnidirectional camera).

The barrier 102a is a protective window for the front camera section whose photographing range is the front of the digital camera 100. As shown in FIG. The front camera section is, for example, a wide-angle camera section that takes a wide range of 180 degrees or more on the front side of the digital camera 100 up, down, left, and right. The barrier 102b is a protective window for the rear camera section whose photographing range is the rear of the digital camera 100. As shown in FIG. The rear camera section is, for example, a wide-angle camera section that covers a wide range of 180 degrees or more in the vertical and horizontal directions on the rear side of the digital camera 100 .

The display unit 28 displays various information. A shutter button 61 is an operation unit (operation member) for instructing photographing. A mode changeover switch 60 is an operation unit for switching between various modes. The connection I/F 25 is a connector for connecting a connection cable to the digital camera 100, and an external device such as a smart phone, personal computer, or television is connected to the digital camera 100 using the connection cable. The operation unit 70 includes various switches, buttons, dials, touch sensors, etc. for receiving various operations from the user. The power switch 72 is a push button for switching on/off the power.

The light-emitting unit 21 is a light-emitting member such as a light-emitting diode (LED), and notifies the user of various states of the digital camera 100 using light-emitting patterns and light-emitting colors. The fixing part 40 is, for example, a tripod screw hole, and is used to fix and install the digital camera 100 with a fixing device such as a tripod.

FIG. 1C is a block diagram showing a configuration example of the digital camera 100. As shown in FIG.

The barrier 102a prevents the imaging system from being soiled or damaged by covering the imaging system of the front camera section (the imaging lens 103a, the shutter 101a, the imaging section 22a, etc.). The photographing lens 103a is a lens group including a zoom lens and a focus lens, and is a wide-angle lens. The shutter 101a is a shutter having a diaphragm function that adjusts the amount of subject light incident on the imaging unit 22a. The imaging unit 22a is an imaging device (imaging sensor) configured by a CCD, a CMOS device, or the like that converts an optical image into an electrical signal. The A/D converter 23a converts the analog signal output from the imaging section 22a into a digital signal. Alternatively, without providing the barrier 102a, the outer surface of the photographing lens 103a may be exposed, and the photographing lens 103a may prevent other imaging systems (the shutter 101a and the imaging section 22a) from being soiled or damaged.

The barrier 102b covers the imaging system of the rear camera section (the imaging lens 103b, the shutter 101b, the imaging section 22b, etc.), thereby preventing the imaging system from being soiled or damaged. The photographing lens 103b is a lens group including a zoom lens and a focus lens, and is a wide-angle lens. The shutter 101b is a shutter that has a diaphragm function that adjusts the amount of subject light incident on the imaging section 22b. The imaging unit 22b is an imaging element configured by a CCD, CMOS element, or the like that converts an optical image into an electric signal. The A/D converter 23b converts the analog signal output from the imaging section 22b into a digital signal. Alternatively, without providing the barrier 102b, the outer surface of the imaging lens 103b may be exposed, and the imaging lens 103b may prevent other imaging systems (shutter 101b and imaging unit 22b) from being soiled or damaged.

A VR (Virtual Reality) image is captured by the imaging unit 22a and the imaging unit 22b. A VR image is an image that can be displayed in VR (displayed in the display mode “VR view”). VR images include an omnidirectional image (omnidirectional image) captured by an omnidirectional camera (omnidirectional camera), or a panoramic image with a video range (effective video range) wider than the display range that can be displayed at once on the display unit. etc. shall be included. VR images include not only still images but also moving images and live-view images (images obtained from cameras in almost real time). The VR image has a maximum visual range of 360 degrees in the vertical direction (vertical angle, angle from the zenith, elevation angle, depression angle, altitude angle, pitch angle) and 360 degrees in the horizontal direction (horizontal angle, azimuth angle, yaw angle). (Effective video range).

In addition, even if the VR image is less than 360 degrees vertically and 360 degrees horizontally, it can be displayed at a wide angle of view (viewing range) that is wider than the angle of view that can be taken with a normal camera, or on the display unit at once. An image having a video range (effective video range) wider than the display range is also included. For example, an image captured by an omnidirectional camera capable of capturing a subject with a field of view (angle of view) of 360 degrees in the horizontal direction (horizontal angle, azimuth angle) and a vertical angle of 210 degrees centered on the zenith is It is a kind of VR image. Also, for example, an image captured by a camera capable of capturing a subject with a field of view (angle of view) of 180 degrees in the horizontal direction (horizontal angle, azimuth angle) and 180 degrees in the vertical angle centered on the horizontal direction is a VR image. It is one kind. That is, an image that has a video range of 160 degrees (±80 degrees) or more in the vertical direction and the horizontal direction and has a video range that is wider than the range that can be visually recognized by humans at once is a VR image. is a kind of

When this VR image is displayed in VR (displayed in the display mode "VR view"), by changing the posture of the display device (display device that displays the VR image) in the left and right rotation direction, the left and right direction (horizontal rotation direction) You can watch a seamless omnidirectional video. In the vertical direction (vertical rotation direction), seamless omnidirectional images can be viewed within a range of ±105 degrees from directly above (zenith), but no images exist within a range exceeding 105 degrees from directly above. It becomes a blank area. A VR image can also be said to be "an image whose video range is at least part of a virtual space (VR space)".

VR display (VR view) is a display method (display mode) that allows the display range to be changed by displaying an image within a viewing range corresponding to the posture of the display device. When the user wears a head-mounted display (HMD) as a display device to view the video, the video is displayed within the visual range corresponding to the orientation of the user's face. For example, in the VR image, at a certain point in time, the viewing angle (angle of view) is centered at 0 degrees in the horizontal direction (a specific direction, such as north) and 90 degrees in the vertical direction (90 degrees from the zenith, that is, horizontal). shall be displayed. From this state, if the posture of the display device is reversed (for example, if the display surface is changed from facing south to facing north), the same VR image can be displayed by 180 degrees in the left and right direction (opposite orientation, for example, south), and up and down. The display range is changed to an image with a viewing angle centered at 90 degrees (horizontal) in the direction. In the case where the user is viewing the HMD, if the user turns his face from north to south (that is, turns his back), the image displayed on the HMD will also change from the north image to the south image. be. With such VR display, it is possible to provide the user with a sense (immersive feeling) as if he or she were visually present in the VR image (inside the VR space). A smartphone attached to VR goggles (head mount adapter) can be said to be a kind of HMD.

Note that the display method of the VR image is not limited to the above. The display range may be moved (scrolled) according to the user's operation on the touch panel, direction buttons, or the like, instead of changing the posture. When displaying VR (in the display mode "VR view"), in addition to changing the display range by changing the posture, the display range can be changed according to the touch and move on the touch panel, the drag operation on the mouse, the pressing of the direction button, etc. You may do so.

The image processing unit 24 performs resizing processing such as predetermined pixel interpolation and reduction, and color conversion processing on the data from the A/D converters 23a and 23b or the data from the memory control unit 15. FIG. Further, the image processing unit 24 performs predetermined arithmetic processing using the captured image data. The system control unit 50 performs exposure control and distance measurement control based on the calculation results obtained by the image processing unit 24 . As a result, TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, EF (flash pre-emission) processing, and the like are performed. The image processing unit 24 further performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (Auto White Balance) processing based on the obtained arithmetic result. Further, the image processing unit 24 performs basic image processing on two images (two fisheye images; two wide-angle images) obtained from the A/D converters 23a and 23b. A single VR image is generated by performing stitching image processing for synthesizing two images that have undergone such image processing. In addition, the image processing unit 24 performs image clipping processing, enlargement processing, distortion correction, etc. for VR display of VR images during VR display in live view or during playback, and draws the processing results in the VRAM of the memory 32. Rendering is performed.

In the stitching image processing, the image processing unit 24 uses one of the two images as a reference image and the other as a comparison image, and calculates the amount of deviation between the reference image and the comparison image for each area by pattern matching processing. Based on the shift amount, a joint position where the two images are joined is detected. The image processing unit 24 corrects the distortion of each image by geometric transformation in consideration of the detected joint position and the lens characteristics of each optical system, and converts each image into an omnidirectional format (omnidirectional image format). Convert to image. Then, the image processing unit 24 generates one omnidirectional image (VR image) by synthesizing (blending) the two omnidirectional images. The generated omnidirectional image is, for example, an image using equirectangular projection, and the position of each pixel in the omnidirectional image can be associated with the surface coordinates of the sphere (VR space).

Output data from the A/D converters 23a and 23b are written to the memory 32 via the image processing section 24 and the memory control section 15, or via the memory control section 15 without the image processing section 24. FIG. The memory 32 stores image data obtained by the imaging units 22a and 22b and converted into digital data by the A/D converters 23a and 23b, and image data to be output from the connection I/F 25 to an external display. The memory 32 has a storage capacity sufficient to store a predetermined number of still images, moving images for a predetermined period of time, and audio.

The memory 32 also serves as an image display memory (video memory). Image display data stored in the memory 32 can be output from the connection I/F 25 to an external display. Functions as an electronic viewfinder by sequentially transferring the VR images captured by the imaging units 22a and 22b and generated by the image processing unit 24 and stored in the memory 32 to an external display for display. can be realized, and live view display (LV display) can be performed. An image displayed in live view display is hereinafter referred to as a live view image (LV image). Live view display (remote LV display) can also be performed by transferring VR images accumulated in the memory 32 to an external device (smartphone, etc.) wirelessly connected via the communication unit 54 and displaying them on the external device side. .

The nonvolatile memory 56 is an electrically erasable/recordable recording medium, such as an EEPROM. Constants for the operation of the system control unit 50, programs, and the like are recorded in the nonvolatile memory 56. FIG. The program here is a computer program for executing various processes.

The system control section 50 is a control section having at least one processor or circuit, and controls the digital camera 100 as a whole. The system control unit 50 implements each process by executing the program recorded in the non-volatile memory 56 described above. The system memory 52 is, for example, a RAM, and in the system memory 52, constants and variables for operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like are developed. The system control unit 50 also performs display control by controlling the memory 32, the image processing unit 24, the memory control unit 15, and the like. The system timer 53 is a timer that measures the time used for various controls and the time of the built-in clock.

The mode switch 60 , shutter button 61 , operation unit 70 , and power switch 72 are used to input various operation instructions to the system control unit 50 .

The mode changeover switch 60 switches the operation mode of the system control unit 50 between a still image recording mode, a moving image shooting mode, a reproduction mode, a communication connection mode, and the like. Modes included in the still image recording mode include an auto shooting mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode. In addition, there are various scene modes, custom modes, and the like, which are shooting settings for each shooting scene. A mode selector switch 60 allows the user to switch directly to any of these modes. Alternatively, after once switching to the shooting mode list screen with the mode switching switch 60, any one of the plurality of modes displayed on the display unit 28 may be selectively switched using another operation member. Similarly, the movie shooting mode may also include multiple modes.

The shutter button 61 has a first shutter switch 62 and a second shutter switch 64 . The first shutter switch 62 is turned on when the shutter button 61 is being operated, that is, when the shutter button 61 is half-pressed (imaging preparation instruction), and generates a first shutter switch signal SW1. The system control unit 50 starts shooting preparation operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (flash pre-emission) processing in response to the first shutter switch signal SW1. do. The second shutter switch 64 is turned ON when the operation of the shutter button 61 is completed, that is, when the shutter button 61 is fully pressed (imaging instruction), and generates a second shutter switch signal SW2. The system control unit 50 starts a series of photographing processing operations from reading out signals from the imaging units 22a and 22b to writing image data in the recording medium 90 in response to the second shutter switch signal SW2.

It should be noted that the shutter button 61 is not limited to an operation member that can be operated in two steps of full-press and half-press, and may be an operation member that can be pressed in only one step. In that case, the photographing preparation operation and the photographing process are continuously performed by pressing the button in one step. This is the same operation as when the shutter button, which can be half-pressed and fully-pressed, is fully pressed (when the first shutter switch signal SW1 and the second shutter switch signal SW2 are generated substantially at the same time).

The operation unit 70 is appropriately assigned functions for each scene by selecting and operating various function icons and options displayed on the display unit 28, and functions as various function buttons. The function buttons include, for example, an end button, a return button, an image forward button, a jump button, a refinement button, an attribute change button, and the like. For example, when the menu button is pressed, a menu screen on which various settings can be made is displayed on the display unit 28 . The user can intuitively perform various settings by operating the operation unit 70 while viewing the menu screen displayed on the display unit 28 .

The power switch 72 is a push button for switching on/off the power. The power supply control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit for switching blocks to be energized, and the like, and detects whether or not a battery is installed, the type of battery, the remaining battery level, and the like. Also, the power supply control unit 80 controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies necessary voltage to each unit including the recording medium 90 for a necessary period. The power supply unit 30 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

A recording medium I/F 18 is an interface with a recording medium 90 such as a memory card or hard disk. A recording medium 90 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, an optical disk, a magnetic disk, or the like. The recording medium 90 may be an exchangeable recording medium detachable from the digital camera 100 or a recording medium built into the digital camera 100 .

The communication unit 54 transmits and receives video signals, audio signals, and the like to and from an external device connected wirelessly or by a wired cable. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can transmit images (including LV images) captured by the imaging units 22a and 22b and images recorded in the recording medium 90, and can receive images and other various information from external devices. .

The orientation detection unit 55 detects the orientation of the digital camera 100 with respect to the direction of gravity. Based on the posture detected by the posture detection unit 55, the image captured by the imaging units 22a and 22b is an image captured with the digital camera 100 held horizontally or an image captured with the digital camera 100 held vertically. It is possible to determine whether there is In addition, it is possible to determine whether the images captured by the imaging units 22a and 22b are images captured by tilting the digital camera 100 in the three axial directions (rotational directions) of the yaw direction, the pitch direction, and the roll direction. be. The system control unit 50 adds orientation information corresponding to the posture detected by the posture detection unit 55 to the image file of the VR image captured by the imaging units 22a and 22b, and rotates the image (tilt correction (zenith correction)). It is possible to adjust the orientation of the image so that it is recorded). One sensor or a combination of a plurality of sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, an orientation sensor, an altitude sensor, etc. can be used as the attitude detection unit 55 . It is also possible to detect the movement of the digital camera 100 (pan, tilt, lift, whether it is stationary, etc.) using an acceleration sensor, a gyro sensor, an orientation sensor, etc. that configure the attitude detection unit 55 .

The microphone 20 is a microphone that collects sounds around the digital camera 100 that are recorded as sounds of VR images (VR moving images) that are moving images. The connection I/F 25 is a connection plug to which an HDMI (registered trademark) cable, a USB cable, or the like is connected for connecting to an external device and transmitting/receiving video.

FIG. 2A is an external view of an electronic device 200, which is a type of display control device. The electronic device 200 is, for example, a display device such as a smartphone. A display 205 is a display unit that displays images and various types of information. The display 205 is configured integrally with a touch panel 206a so that a touch operation on the display surface of the display 205 can be detected. Electronic device 200 is capable of VR-displaying a VR image (VR content) on display 205 . The operation unit 206b is a power button that accepts an operation for switching the power of the electronic device 200 between on and off. The operation unit 206c and the operation unit 206d are volume buttons for increasing or decreasing the volume of audio output from the speaker 212b, an earphone connected to the audio output terminal 212a, an external speaker, or the like. The operation unit 206e is a home button for displaying a home screen on the display 205. FIG. The audio output terminal 212a is an earphone jack, and is a terminal for outputting an audio signal to an earphone, an external speaker, or the like. The speaker 212b is a built-in speaker for outputting sound.

FIG. 2B is a block diagram showing a configuration example of the electronic device 200. As shown in FIG. A CPU 201, a memory 202, a nonvolatile memory 203, an image processing unit 204, a display 205, an operation unit 206, a recording medium I/F 207, an external I/F 209, and a communication I/F 210 are connected to the internal bus 250. . An audio output unit 212 and a posture detection unit 213 are also connected to the internal bus 250 . Each unit connected to the internal bus 250 is capable of exchanging data with each other via the internal bus 250 .

The CPU 201 is a control unit that controls the entire electronic device 200, and is composed of at least one processor or circuit. The memory 202 is, for example, a RAM (a volatile memory using a semiconductor element, etc.). The CPU 201 uses the memory 202 as a work memory to control each part of the electronic device 200 according to a program stored in the nonvolatile memory 203, for example. The nonvolatile memory 203 stores image data, audio data, other data, various programs for the CPU 201 to operate, and the like. The nonvolatile memory 203 is composed of, for example, flash memory or ROM.

Under the control of the CPU 201, the image processing unit 204 processes images stored in the nonvolatile memory 203 and the recording medium 208, video signals obtained via the external I/F 209, and images obtained via the communication I/F 210. etc. are subjected to various image processing. The image processing performed by the image processing unit 204 includes A/D conversion processing, D/A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement/reduction processing (resize), noise reduction processing, color conversion, and so on. processing, etc. It also performs various image processing such as panorama development, mapping processing, and conversion of an omnidirectional image or a VR image that is a wide-range image having a wide-range image, even if it is not omnidirectional. The image processing unit 204 may be composed of a dedicated circuit block for performing specific image processing. Further, depending on the type of image processing, the CPU 201 can perform image processing according to a program without using the image processing unit 204 .

The display 205 displays images, GUI screens that form a GUI (Graphical User Interface), and the like, under the control of the CPU 201 . CPU 201 controls each unit of electronic device 200 to generate a display control signal according to a program, generate a video signal to be displayed on display 205 , and output the video signal to display 205 . A display 205 displays an image based on the generated and output image signal. Note that the electronic device 200 itself includes an interface for outputting a video signal to be displayed on the display 205, and the display 205 may be configured by an external monitor (television, HMD, etc.).

An operation unit 206 is an input device for accepting user operations, including a character information input device such as a keyboard, a pointing device such as a mouse and a touch panel, buttons, dials, joysticks, touch sensors, and touch pads. In this embodiment, the operation unit 206 includes a touch panel 206a and operation units 206b, 206c, 206d, and 206e.

A recording medium 208 such as a memory card, CD, or DVD can be detachably attached to the recording medium I/F 207 . A recording medium I/F 207 reads data from the attached recording medium 208 and writes data to the recording medium 208 under the control of the CPU 201 . A recording medium 208 stores data such as images to be displayed on the display 205 . The external I/F 209 is an interface for connecting to an external device via a wired cable (such as a USB cable) or wirelessly, and performing input/output (data communication) of video signals and audio signals. The communication I/F 210 is an interface for communicating (wireless communication) with an external device, the Internet 211, etc., and transmitting/receiving various data such as files and commands (data communication).

The audio output unit 212 outputs audio of moving images and music data reproduced by the electronic device 200, operation sounds, ringtones, various notification sounds, and the like. The audio output unit 212 includes an audio output terminal 212a for connecting an earphone or the like and a speaker 212b, but the audio output unit 212 may output audio data to an external speaker through wireless communication or the like.

Posture detection unit 213 detects the posture (inclination) of electronic device 200 with respect to the direction of gravity and the posture of electronic device 200 with respect to each axis of the yaw direction, pitch direction, and roll direction, and notifies CPU 201 of the posture information. Based on the posture detected by the posture detection unit 213, whether the electronic device 200 is held horizontally, held vertically, directed upward, downward, oblique, etc. can be determined. In addition, it is possible to determine whether or not the electronic device 200 is tilted in the rotational direction such as the yaw direction, the pitch direction, and the roll direction, and whether the electronic device 200 is rotated in the rotational direction. One sensor or a combination of multiple sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, an orientation sensor, an altitude sensor, etc. can be used as the attitude detection unit 213 .

As described above, the operation unit 206 includes the touch panel 206a. The touch panel 206a is an input device that is superimposed on the display 205 and has a two-dimensional configuration, and outputs coordinate information according to the touched position. The CPU 201 can detect the following operations or states on the touch panel 206a.
- The touch panel 206a is newly touched by a finger or pen that has not touched the touch panel 206a, that is, the start of touch (hereinafter referred to as Touch-Down).
A state in which a finger or pen is touching the touch panel 206a (hereinafter referred to as Touch-On)
・The finger or pen is moving while touching the touch panel 206a (hereinafter referred to as touch-move)
- The finger or pen touching the touch panel 206a leaves the touch panel 206a, that is, the end of the touch (hereinafter referred to as Touch-Up)
- A state in which nothing is touched on the touch panel 206a (hereinafter referred to as Touch-Off)

When touchdown is detected, touchon is also detected at the same time. After touchdown, touchon continues to be detected unless touchup is detected. Touch-on is detected at the same time when touch-move is detected. Even if touch-on is detected, touch-move is not detected if the touch position does not move. Touch-off is detected when it is detected that all the fingers and pens that were in touch have touched up.

The CPU 201 is notified of these operations/states and the coordinates of the position where the finger or pen touches the touch panel 206a through the internal bus. ) has been performed. As for the touch move, the moving direction of the finger or pen moving on the touch panel 206a can also be determined for each vertical component/horizontal component on the touch panel 206a based on the change in the position coordinates. If it is detected that the touch-move has been performed for a predetermined distance or more, it is determined that the slide operation has been performed. A touch move is a movement operation performed by the user on the touch panel 206a. Various processes in response to touch-move, which will be described later, can also be performed in response to a drag operation with a mouse, which is also a movement operation.

An operation of touching the touch panel 206a with a finger and quickly moving it by a certain distance and then releasing it is called a flick. A flick is, in other words, an operation of quickly tracing the touch panel 206a as if flicking it with a finger. It can be determined that a flick has been performed when a touch-move of a predetermined distance or more at a predetermined speed or more is detected, and a touch-up is detected as it is (it can be determined that a flick has occurred following a slide operation).

Further, a touch operation of simultaneously touching a plurality of points (for example, two points) to bring the touch positions closer to each other is called pinch-in, and a touch operation of moving the touch positions away from each other is called pinch-out. Pinch-out and pinch-in are collectively called pinch operation (or simply pinch). The touch panel 206a may be any of various types of touch panels such as a resistive film type, a capacitance type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. good. There is a method of detecting that there is a touch when there is contact with the touch panel, and a method of detecting that there is a touch when there is an approach of a finger or pen to the touch panel.

FIG. 2C is an external view of VR goggles (head mount adapter) 230 to which the electronic device 200 can be attached. Electronic device 200 can also be used as a head-mounted display by wearing VR goggles 230 . Insertion opening 231 is an insertion opening for inserting electronic device 200 . The entire electronic device 200 can be inserted into the VR goggles 230 with the display surface of the display 205 facing the headband 232 side (that is, the user side) for fixing the VR goggles 230 to the user's head. The user can visually recognize the display 205 without holding the electronic device 200 by hand while wearing the VR goggles 230 on which the electronic device 200 is mounted. In this case, when the user moves the head or the whole body, the posture of electronic device 200 also changes. The attitude detection unit 213 detects the attitude change of the electronic device 200 at this time, and the CPU 201 performs processing for VR display based on this attitude change. In this case, detecting the orientation of electronic device 200 by orientation detection unit 213 is equivalent to detecting the orientation of the user's head (the direction in which the user's line of sight is directed). Note that the electronic device 200 itself may be an HMD that can reach the head without VR goggles.

In this embodiment, a VR image having a visual range (effective video range) of 360 degrees vertically and horizontally at maximum is converted to a VR image having a visual range (effective video range) of 180 degrees vertically and horizontally. The extraction operation and processing will be described. In the following, a VR image having a video range (effective video range) corresponding to a field of view of 360 degrees vertically and horizontally at maximum is referred to as a 360-degree VR image. Also, a VR image having a video range (effective video range) corresponding to a field of view of 180 degrees vertically and horizontally is referred to as a 180-degree VR image. Conceptually, the process of cutting out a 180-degree VR image from a 360-degree VR image is a process of cutting out a hemisphere from a virtual sphere on which a 360-degree video is entirely mapped. The clipping described in this embodiment is not a process of clipping a normal rectangular image (planar image), but a process of clipping an image that can be viewed as a VR image even after clipping (mapped onto a sphere and viewable in a VR view). is. Since the clipped image has an effective video range of 180 degrees vertically and horizontally, when mapped on a sphere, a hemispherical effective video (photographed image, etc.) is mapped. The rest of the hemisphere becomes an invalid image range, and images that are filled with a single color or a predetermined pattern or that are complemented in some way are mapped. When viewing such a cut-out VR image in a VR view, the user can visually recognize the image within a range of 180 degrees in width in front (a range of 90 degrees to the left, right, top and bottom from the center). The reasons for performing such clipping processing are as follows.

First, the amount of image data can be reduced. The clipped 180-degree VR image has a smaller data amount than the 360-degree VR image. Therefore, it is possible to suppress the capacity pressure of the recording medium at the time of saving. In addition, it is possible to reduce the amount of communication data to be transmitted and received, and to reduce the processing load to display, thereby improving processing speed and response speed, and reducing power consumption.

Second, unnecessary subjects can be deleted. For example, in most cases of capturing a VR image having an effective video range of 360 degrees, it is inevitable that the photographer himself will appear in the picture. However, if what the photographer wants to photograph is not the photographer himself but a landscape or the like, the photographer himself is an unnecessary subject and is unintended. If an unintended image is captured, it hinders the viewer from concentrating on the image such as the landscape that the photographer considered to be the main subject. In addition, information that should be kept confidential, such as the face of a passerby or the license plate of a passing vehicle, is often captured, which is undesirable from the viewpoint of privacy and security. These problems can be avoided by clipping only the range intended by the photographer and discarding other unnecessary ranges of the image.

Thirdly, it is possible to reduce the physical burden on the user watching in the VR view. When viewing a 360-degree VR image in a VR view, the image exists even behind the viewing user. Viewers can see a 180-degree image in front of them by simply turning their head, but if they want to see behind them, they have to twist their upper body or change the direction in which they stand. However, the physical burden is rather high. Such a physical movement to look behind is not suitable for viewing from a seat, unless it is a swivel chair. On the other hand, with a 180-degree VR image, the viewer can see the entire scene simply by changing the direction of his/her head, so the physical burden of viewing the image is relatively small. If the viewer recognizes that the VR image is 180 degrees through a guide display or a display that indicates that the range beyond 180 degrees is an invalid area, the viewer may try to look behind. Absent. Therefore, it is possible to prevent the viewer from performing physical actions such as looking behind.

Fourthly, a 360-degree VR image can be displayed in correspondence with a recording format for a 180-degree VR image. If the recording format for 360-degree VR images and the recording format for 180-degree VR images are different, a playback device (playback application software) that only supports viewing of 180-degree VR images will not be able to record 360-degree VR images. Images cannot be viewed. However, if a 180-degree VR image is generated by clipping from a 360-degree VR image, it becomes possible to view even a playback device that can only view 180-degree VR images. Also, even if both 180-degree VR image viewing and 360-degree VR image viewing are supported, it is necessary to switch between the 180-degree VR image viewing mode and the 360-degree VR image viewing mode. In some cases, the switching operation is troublesome. However, if a 180-degree VR image is generated by extracting a 360-degree VR image, a plurality of VR images can be displayed by switching one after another without switching the display mode for the 180-degree VR image. In addition, two 180-degree VR images for the right eye and left eye, captured by two camera units installed at a distance equal to the parallax, facing the subject in the same direction, are displayed so that they can be viewed stereoscopically. A display mode (dino VR view) is assumed. If a 180-degree VR image for the right eye and a 180-degree VR image for the left eye are cut out from a 360-degree VR image and recorded, it is possible to view even such a twin-lens VR view.

In the present embodiment, an example in which a 180-degree VR image is generated by cutting out a 360-degree VR image will be described. The processing can also be applied to VR images with other viewing angles. That is, the VR image before clipping is not limited to a 360-degree VR image, and the VR image after clipping is not limited to a 180-degree VR image.

In each process to be described later, an example will be described in which various touch buttons are displayed, and a touch operation to the various touch buttons is accepted as a user operation to proceed with the process. A touch operation for receiving instructions for various touch buttons may specifically be a touch-up from a touch-responsive area corresponding to a touch button, or a touch-down from a touch-responsive area corresponding to a touch button. Further, the present invention is not limited to this, and each instruction explained as being received by operating a touch button to be described later may be received by another user's operation. For example, it may be an operation on each physical button, an operation of selecting a display item using a direction key and pressing an enter button, an operation of selecting a display item using a mouse, a voice command, or the like.

FIG. 3 is a flowchart of VR view display (processing) in electronic device 200 . This processing is realized by expanding the program recorded in the nonvolatile memory 203 into the memory 202 and executing it by the CPU 201 . With the electronic device 200 turned on, when application software capable of displaying a 360-degree VR image in a VR view and having a function of extracting a 180-degree VR image is activated, the CPU 201 initializes flags, control variables, and the like. After that, the process of FIG. 3 is started.

In S<b>301 , the CPU 201 reads (acquires) a VR image to be displayed from the recording medium 208 or the communication destination via the communication I/F 210 . Also, information indicating the video range (effective video range) attached as attribute information of the VR image is acquired. The information indicating the effective image range is information indicating which angular range of effective image the VR image has in the vertical direction and the horizontal direction. The information indicating the effective video range may be information that can specify the effective video range of the VR image, and may be information such as angle of view, viewing angle, azimuth angle, elevation angle, number of left and right pixels, and coordinates. The information indicating the effective image range may be model information of the camera that captured the VR image (specifying the model makes it possible to specify the shootable range), zoom information at the time of shooting, or the like. When acquiring the information indicating the effective video range, the CPU 201 also identifies (calculates) the invalid video range (non-video range) based on the difference between 360 degrees vertically and horizontally. Conversely, information indicating the invalid video range may be obtained from the attribute information of the VR image, and the effective video range may be calculated from the information. It may be acquired from the attribute information of the VR image.

In S302, the CPU 201 performs display processing in the VR view of the VR image acquired in S301. The original data (image data) of the VR image is, for example, a distorted image using equirectangular projection, and is an image in a format in which the position of each pixel can be associated with the coordinates of the surface of a sphere. The original image of this VR image is mapped on a sphere, and a part of it is cut out and displayed. That is, the image displayed in S302 is an image obtained by cutting out and enlarging a part of the VR image, and is an image from which the distortion of the original image has been removed (or reduced).

In S303, the CPU 201 determines whether or not the VR image to be displayed in S302 is a VR image from which a 180-degree VR image can be extracted based on the information acquired in S301. If the image is at least one of an image whose number of recorded pixels is less than the threshold, an image whose effective video range is narrower (smaller) than the threshold, an image that has already been cut out as described later, and a 180-degree VR image, It is determined that it is not a VR image that can be extracted, and the process proceeds to S305. If it is determined that the VR image can be extracted, the process proceeds to S304.

In S304, the CPU 201 displays a clip edit button. FIG. 6A shows a display example of the VR image displayed in S304 and the cutout edit button. A VR image 602 is the VR image displayed in S302. A clipping edit button 601 is a touch icon for instructing clipping of a VR image. Note that if it is determined in S303 that the VR image is not a clippable VR image, the clipping edit button 601 is not displayed, but is clipped in a display form such as a grayout that makes it recognizable that the image is in a non-selectable state. An edit button 601 may be displayed.

In S305, the CPU 201 determines whether or not there has been a touch move (display range change operation) to the area where the VR image 602 is displayed, based on information notified from the touch panel 206a. If there is a touch move, the process proceeds to S306; otherwise, the process proceeds to S307. In S306, the CPU 201 scrolls the displayed VR image 602 according to the direction and amount of touch movement, and changes the display range of the VR image. Specifically, the display range rotates around the vertical axis (zenith axis, gravity axis) of the virtual sphere on which the VR image is mapped, based on the left-right (horizontal) component of the movement component of the touch move. Scroll the VR image to the right or left so that This is a display range change that changes the orientation of the line of sight (the direction of the display range) viewed from the viewer placed at the center of the virtual sphere. Also, based on the up-down (vertical) component of the movement component of the touch move, the virtual sphere on which the VR image is mapped is displayed centered on the horizontal axis (the axis perpendicular to the zenith axis, the axis perpendicular to gravity). Scroll the VR image up or down so that the range rotates. In other words, this is a display range change that changes the elevation angle or depression angle (hereinafter referred to as the elevation/depression angle) of the line of sight (direction of the display range) from the viewer placed at the center of the virtual sphere.

In S<b>307 , the CPU 201 determines whether or not the posture detection unit 213 has detected a change in posture that should change the display range. If a change in posture for which the display range should be changed is detected, the process proceeds to S308; otherwise, the process proceeds to S309. In S<b>308 , the CPU 201 changes (moves) the display range of the VR image 602 according to the posture change detected by the posture detection unit 213 .

In S309, the CPU 201 determines whether or not there has been a pinch-in or pinch-out (that is, magnification change operation) with respect to the area where the VR image 602 is displayed, based on information notified from the touch panel 206a. If there is pinch-in or pinch-out, the process proceeds to S310; otherwise, the process proceeds to S311. In S310, the CPU 201 changes the display magnification of the VR image (continuously or stepwise in a plurality of steps of 3 or more) according to the pinch operation. In the case of pinch-in (reduction operation), the display magnification of the VR image is lowered. Along with this, the display range of the VR image is widened. That is, although the subject becomes smaller, a wider range is displayed. In the case of pinch out (enlargement operation), the display magnification of the VR image is increased. Accordingly, the display range of the VR image is narrowed. That is, the subject becomes larger, but a narrower range is displayed.

In S<b>311 , the CPU 201 determines whether or not there is a touch operation to designate the cutout edit button 601 based on information notified from the touch panel 206 a. If the cutout edit button 601 has been touched, the process proceeds to S312; otherwise, the process proceeds to S313. In S312, the CPU 201 performs cutout editing processing. The cutout editing process will be described later with reference to FIG.

In S313, the CPU 201 determines whether or not an image switching operation has been performed. If there is an image switching operation, the process proceeds to S314; otherwise, the process proceeds to S315. The image switching operation includes, for example, operating an image switching button included in the operation unit 206, touching an image switching icon (image forward icon, image backward icon) displayed on the display 205, or touching two points at the same time. There is also a double-drag that drags in the same direction. In S314, the CPU 201 identifies, as the next display target, a VR image of an image file different from the currently displayed VR image in accordance with the image switching operation. Subsequently, the process returns to S301 to acquire and display the next VR image to be displayed.

In S315, the CPU 201 determines whether or not there is another operation. If there is any other operation, the process proceeds to S316; otherwise, the process proceeds to S317. In S316, the CPU 201 performs processing according to other operations. For example, processing according to other operations includes display processing of VR images in twin-lens VR view, file deletion of displayed VR images, sharing (posting to SNS, sending e-mail, usage instructions in other applications) etc.), and editing processes such as color adjustment.

In S<b>317 , the CPU 201 determines whether or not an end operation has been performed on the operation unit 206 . If there is no end operation, the process returns to S305 and repeats the process. If there is a termination operation, the application software that is the VR image display application is closed, and the processing in FIG. 3 is terminated.

FIG. 4 shows a flowchart of the cutout editing process described in S312 of FIG. This processing is realized by expanding the program recorded in the nonvolatile memory 203 into the memory 202 and executing it by the CPU 201 . The editing process of FIG. 4 is basically a process of specifying the center of the range to be cut out as a 180-degree VR image from the 360-degree VR image displayed in the VR view as it is. Although it is possible to finely adjust the specified range, the range to be extracted can be easily specified by simply determining the center roughly without fine adjustment. In the VR view, there are many cases where the entire VR image is not displayed, and the boundary of the range to be cut out is not a straight boundary like in the case of a plane image. Therefore, if a trimming frame is displayed and the cropping range is determined by adjusting the displayed trimming frame, it would be complicated and difficult for the user to understand. On the other hand, according to the cut-out editing process of the present embodiment, the cut-out range can be specified simply by specifying the center of the cut-out range as the 180-degree VR image from the 360-degree VR image as it is in the VR view. . For this reason, the user can perform the clipping with an intuitive, easy-to-understand, and simple operational feeling.

In S401, the CPU 201 causes the display 205 to display a cutout editing screen, which is a screen for designating the cutout range. FIG. 6B shows a display example of the cutout editing screen. A VR image 602 is the same VR image as the image displayed in the VR view display before transitioning to the cutout editing screen. Although the display range can be enlarged or reduced by a pinch operation, for example, the VR image 602 is displayed in a display range narrower than the range of 180 degrees immediately after the cutout edit screen is displayed (default state). A pointer 603 is an indicator for designating the center of the extraction range (the reference position of the extraction range). A cutout button 604 is an instruction icon for designating a cutout range based on the current display range (reference position). On the cutout edit screen, the pointer 603 is displayed at a fixed position in the center of the screen, and even if the display range of the VR image 602 changes, the position of the pointer 603 on the display 205 does not change. The user manipulates the display range of the VR image 602 displayed in the VR view in the normal VR view so that the position of the desired subject to be the center of the extracted VR image is the position indicated by the pointer 603. change by almost the same operation as By touching the cutout button 604, a range based on the current display range (reference position) can be specified (or tentatively specified) as a range to be cut out. More specifically, it is possible to specify the range after clipping such that the position of the VR image corresponding to the position where the pointer 603 is displayed is the center after clipping or the azimuth of the center.

Of the circular display direction guides, a fan-shaped display range 605 indicates the azimuth range (angular range around the zenith axis) of the current display range in all directions of the circular VR image. . A non-display range 606 indicates a range of azimuths (angular range around the zenith axis) that is not the current display range in all directions of the VR image indicated by a circle. The reference direction 607 is the direction (hereinafter referred to as the display start direction) that is the center of the display range that is displayed first when the display of the VR image 602 is started (that is, before the operation to change the display range and the attitude change). showing.

The angle information 608 (angle scale) indicates the current angle (azimuth angle) of the pointer 603 with respect to the display start direction, and the display is updated as the display range is changed. An entire hemisphere display button 609 is an instruction icon for switching the display magnification of the VR image 602 to a magnification that includes the entire range specified as the range after cropping with one touch. A two-screen confirmation button 610 is used to switch to a display mode in which both the range designated as the cutout range and the deletion range that is not designated as the cutout range and is discarded in the cutout VR image are displayed at the same time. It is an instruction icon. The edge fine adjustment button 611 is an instruction icon for switching to a display mode for displaying the edge (boundary portion) of the extraction range in order to finely adjust the extraction range. An unnecessary portion designation button 612 is an instruction icon for switching to a mode for designating an unnecessary portion of a subject that is not desired to be included in the cropping range. When specifying the unnecessary part, when specifying the center of the extraction range so as not to include the unnecessary part, the specifiable range and the non-specifiable range are displayed so as to be identifiable.

An elevation/depression angle enable/disable button 613 is used to enable or disable the elevation/depression angle corresponding to the current display range (the elevation/depression angle at the position indicated by the pointer 603 in the entire VR image mapped onto the sphere) as the center of the extraction range. This is an instruction icon for switching between . The elevation/depression angle enable/disable button 613 also indicates the current setting state. is displayed as an icon in the form of display indicating . When it is enabled, a range of 90 degrees around the azimuth and elevation/depression angle indicated by the pointer 603 and a total of 180 degrees (hemispherical range) is set as the extraction range. When disabled, the elevation/depression angle is not based on the elevation/depression angle indicated by the pointer 603, and the elevation/depression angle is 0 degrees, that is, a range of 90 degrees around the center of the position indicated by the pointer 603 on the horizontal plane, and a total of 180 degrees (hemispherical range). is the extraction range. It is assumed that the elevation/depression angle valid/invalid setting is invalid in the initial setting before the user performs the designation operation.

A posture detection ON/OFF button 614 is an instruction icon for switching between validating and invalidating changes in the display range of the VR image 602 due to posture changes. The posture detection ON/OFF button 614 also indicates the current setting state, and if the current setting state is valid, it is displayed as an icon in a display form indicating valid, and if the current setting state is invalid, it is invalid. is displayed as an icon in the form of display indicating . When enabled, the center of the extraction range can be determined simply by changing the posture of electronic device 200, which is simple and intuitive. It may be difficult to adjust the display range. By disabling it, it is possible to prevent the display range from changing due to posture changes, preventing unintended changes in the display range, and by touch-move operations (slide operations), the display range can be more as intended by the user. can be adjusted to The orientation detection ON/OFF setting may be disabled (OFF) in the initial setting before the user performs the designation operation.

In S402 and S403, the CPU 201 changes the display range in accordance with the touch move to the area where the VR image 602 is displayed. The details of the processing are the same as those of S305 and S306 in FIG. 3 described above. As described above, even if the display range changes, the display position of the pointer 603 on the display 205 does not change. If the display range of the VR image changes, the relative positional relationship of the pointer 603 with respect to the VR image will change. In addition, this processing changes the position of the cutout range in the VR image before the cutout without changing the width (180 degrees) of the cutout range in the cutout processing described later in S502 and S503.

FIG. 6(c) shows a display example when the display range is changed by touch-moving to the left from the state of FIG. 6(b). The display range of the VR image 602 has changed, and accordingly, the angle information 608 and the range indicated by the display range 605 and non-display range 606 of the display direction guide have also been changed. The VR image 602 has been scrolled to the left as compared to FIG. 6B, and the area on the right side of the VR image has become a new display range compared to the state of FIG. 6B. . That is, the display range has moved to the right. As can be seen from the angle information 608 and the display direction guide in the example of FIG. 6(c), the display range has been changed to the right by approximately 90 degrees compared to the display range of FIG. 6(b). In this display range, a passing car with a license plate visible is shown as a subject. For example, the user thinks that the license plate of a passing car is not information that should be disclosed, and that he/she would like to prevent it from being included in the extracted image.

FIG. 6(d) shows a display example when the display range is changed by further touch-moving from the state of FIG. 6(c) to the left. The display range of the VR image 602 has changed, and accordingly, the angle information 608 and the range indicated by the display range 605 and non-display range 606 of the display direction guide have also been changed. The VR image 602 has been scrolled to move to the left compared to FIG. 6C, and the area on the right side of the VR image has become a new display range compared to the state of FIG. 6C. . That is, the display range has moved to the right. As can be seen from the angle information 608 and the display direction guide in the example of FIG. 6(d), the display range is changed to a direction facing almost directly behind (180 degrees to the right) compared to the display range of FIG. 6(b). ing. In this display range, the photographer himself appears as a subject. For example, the user thinks that such a photographer himself is not the main subject that he wanted to photograph, and that he would like to avoid being included in the cropped image.

In S<b>404 , the CPU 201 refers to the setting information held in the memory 202 and determines whether or not the function for changing the display range in accordance with posture detection is enabled (turned on). Enabling (on)/disabling (off) of the display range changing function according to orientation detection is a setting item that is set by a touch operation on the orientation detection ON/OFF button 614 described above. If the change function of the display range according to posture detection is enabled (ON), the process proceeds to S405; otherwise, the process proceeds to S407. In S<b>405 and S<b>406 , the CPU 201 changes the display range of the VR image 602 according to the change in posture detected by the posture detection unit 213 . Since this processing is the same processing as S307 and S308 in FIG. 3 described above, detailed description thereof will be omitted. Note that this process changes the position of the cutout range in the VR image before cutting without changing the width (180 degrees) of the cutout range in the cutout process described later in S502 and S503. If it is determined in S404 that the function for changing the display range according to orientation detection is disabled (OFF), the processing proceeds to S407 without proceeding to S405 and S406. The display range is not changed according to the change in

In S407 and S408, the CPU 201 changes the display magnification of the VR image 602 according to the pinch operation (magnification change instruction). Since this processing is the same processing as S309 and S310 in FIG. 3 described above, detailed description thereof will be omitted.

In S409, the CPU 201 determines whether or not there is a touch operation (entire hemisphere instruction; magnification change instruction) on the whole hemisphere display button 609 based on the information notified from the touch panel 206a. If there is a touch operation designating the whole hemisphere display button 609, the process proceeds to S410; otherwise, the process proceeds to S413. In S410, the CPU 201 determines whether or not the current display magnification is already the magnification of the hemisphere whole display (hemisphere magnification). The current display magnification may be referred to for determination, or the current display mode may be referred to for determination. If the hemispherical magnification has already been set, the process proceeds to S412 to restore the original display magnification. If the hemispherical magnification is not displayed, the flow advances to S411 to switch to the display with the hemispherical magnification. That is, each time the hemisphere display button 609 is touched, the hemisphere display (hemisphere magnification display) is switched on and off.

In S411, the CPU 201 records the current display magnification before displaying the whole hemisphere in the memory 202, and then displays the whole hemisphere. The entire hemisphere display is a display at a magnification such that the entire cutout range when cut out centering on the position currently indicated by the pointer 603 is within the display range on the display 205 . In this embodiment, since the range of 180 degrees is set as the cutout range, the display magnification including the hemispherical range is set as the hemispherical magnification. The entire display range is displayed at a magnification that fits within the display range.

FIG. 7(a) shows a display example of the whole hemisphere display on the cutout editing screen. From the normal display magnification in the VR view as shown in FIG. display magnification (lower magnification than the initial magnification) with one touch (without stopping at another magnification). A VR image 602 in FIG. 7A is the same image as the VR image 602 displayed in FIG. Since a wide range of 180 degrees or more, which is originally mapped on a sphere, is contained in a plane display range, a VR image 602 with its periphery distorted as if photographed with a fisheye lens is displayed. A shaded area 702 is superimposed on the VR image 602 and displayed. A shaded area 702 indicates a range outside the clipping range (outside the range of 180 degrees from the position indicated by the pointer 603) in the displayed VR image 602, and the range to be deleted in the clipped image ( exclusion range). The user can see the non-slanted central portion 701 of the VR image 602 to confirm the portion included in the extraction range, and can also see the shaded portion 702 to confirm the exclusion range. Therefore, it is possible to specify the extraction range while confirming to what extent the range is included in the extraction range and from where it is not included in the extraction range. This makes it easy to set the cropping range so that the river, which is the main subject, is within the cropping range while the passing cars and people are excluded.

Note that the shaded area 702 only needs to be known to be the exclusion range, so the display form is not limited to the diagonal lines, and the exclusion range may be indicated by other display forms such as a translucent mask, hatching, and monochrome display. . Further, even if the exclusion range is not indicated, only the boundary between the extraction range and the exclusion range may be displayed by a circular line or the like. Furthermore, the exclusion range may not be displayed, and the entire cutout range may be displayed within the display range. Even while the entire hemisphere is being displayed, processing according to various operations described in the processing of FIG. 4 is possible. That is, it is possible to change the display range by posture change or touch move so that the desired range can be adjusted to be the extraction range. In addition, it is possible to determine the cutout range by processing corresponding to a touch operation on the cutout button 604 described later in S425 to S431. The shaded area 702 indicating the exclusion range may be displayed when the whole hemisphere is displayed in response to the touch operation on the whole hemisphere display button 609, and may not be displayed when the display magnification is reduced from the normal magnification by pinch-in. Also, the shaded area 702 indicating the exclusion range may be displayed when the whole hemisphere is displayed in response to the touch operation on the whole hemisphere display button 609, and may also be displayed when the display magnification is reduced from the normal magnification by pinching in. good.

In S412, the CPU 201 refers to the original display magnification before switching held in the memory 202 from the state where the entire hemisphere was being displayed, and restores the display state at the original display magnification. That is, for example, the display state shown in FIG. 7A is switched to the display state shown in FIG. 6B. As a result, the display state in which the peripheral portion is distorted and displayed small can be switched to a display state in which the central portion included in the clipping range is displayed large with no or little distortion.

In S413, the CPU 201 determines whether or not the double screen confirmation button 610 has been touched based on the information notified from the touch panel 206a. If there is a touch operation designating the double screen confirmation button 610, the process proceeds to S414; otherwise, the process proceeds to S415. In S414, the CPU 201 performs dual screen display processing. The two-screen display process will be described later with reference to FIGS. 8 and 9. FIG.

In S415, the CPU 201 determines whether or not there is a touch operation (a fine adjustment instruction for starting fine adjustment of the display range) on the edge fine adjustment button 611, based on information notified from the touch panel 206a. If there is a touch operation designating the edge fine adjustment button 611, the process proceeds to S416; otherwise, the process proceeds to S417. In S416, the CPU 201 performs edge fine adjustment processing. The end fine adjustment processing will be described later with reference to FIGS. 10 and 11. FIG.

In S417, the CPU 201 determines whether or not there is a touch operation on the unnecessary part designation button 612 based on the information notified from the touch panel 206a. If there is a touch operation to designate the unnecessary part designation button 612, the process proceeds to S418; otherwise, the process proceeds to S419. In S418, the CPU 201 performs unnecessary part designation processing. The unnecessary part designation processing will be described later with reference to FIGS. 12 to 14. FIG.

In S419, the CPU 201 determines whether or not there has been a touch operation (switching instruction for switching the elevation/depression angle enable/disable) on the elevation/depression angle enable/disable button 613 based on the information notified from the touch panel 206a. If there is a touch operation designating the elevation/depression angle valid/invalid button 613, the process proceeds to S420; otherwise, the process proceeds to S423. In S420, the CPU 201 refers to the setting information held in the memory 202, and determines whether or not the current setting before receiving the touch operation on the elevation/depression angle enable/disable button 613 is valid for the elevation/depression angle. If it is determined that the elevation/depression angle is valid, the process proceeds to S421, and if it is invalid, the process proceeds to S422. In S<b>421 , the CPU 201 changes the elevation/depression angle valid/invalid setting from valid to invalid, and records the setting state in the memory 202 . In S<b>422 , the CPU 201 changes the elevation/depression angle valid/invalid setting from invalid to valid, and records the setting state in the memory 202 . When the elevation/depression angle enable/disable setting is changed, the display form of the elevation/depression angle enable/disable button 613 is also updated so as to indicate the setting state after the change. Note that if the elevation/depression angle enable/disable setting is disabled in S421, the direction in which the display range of the VR image 602 can be changed is limited to only the azimuth direction in S403 and S406, and the display range is changed in the elevation/depression angle direction. You can make it impossible. In other words, depending on the touch move or posture change, left/right scrolling (change of display range in azimuth direction) is performed, but up/down scrolling (change of display range in elevation/depression direction) is restricted. good.

In S423, the CPU 201 determines whether or not the posture detection ON/OFF button 614 has been touched based on information notified from the touch panel 206a. If there is a touch operation designating the posture detection ON/OFF button 614, the process proceeds to S424; otherwise, the process proceeds to S425. In S424, the CPU 201 changes the setting state of the display range changing function according to the orientation detection. If it was valid (on) before the change, it is changed to invalid (off), and setting information indicating the setting after the change is recorded in the memory 202 . If it was disabled (off) before the change, it is changed to enabled (on), and setting information indicating the setting after the change is recorded in the memory 202 . When the setting state of the change function of the display range according to posture detection is changed, the display mode of the posture detection ON/OFF button 614 is also updated so as to show the setting state after the change.

In S425, the CPU 201 determines whether or not there has been a touch operation (cutout instruction, reference position designation operation) on the cutout button 604 based on information notified from the touch panel 206a. If there is a touch operation designating the cutout button 604 (if a cutout instruction is received), the process proceeds to S426; otherwise, the process proceeds to S432.

In S426, the CPU 201 determines whether or not an unnecessary portion is registered by unnecessary portion designation processing, which will be described later. If the unnecessary part is registered, the process proceeds to S428, and if the unnecessary part is not registered, the process proceeds to S427. In S427, the CPU 201 performs clipping preview processing. The clipping preview process will be described later with reference to FIG. Note that if it is determined in S425 that there has been a touch operation on the extraction button 604, the process may proceed to S427 without performing the processing of S426. Furthermore, if it is determined in S425 that there has been a touch operation on the extraction button 604 without performing the processing of S426 and S427, the process proceeds to S535 of FIG. good too. The processing of S428 to S431 is processing when an unnecessary portion is registered, and is the same processing as S1215 to S1218 in FIG. 12 described later, so the details are omitted.

In S<b>432 , the CPU 201 determines whether or not an operation has been performed to end the cutout editing process and return to the VR view display process of FIG. 3 . If the return operation has not been performed, the process returns to S402 to repeat the process, and if the return operation has been performed, the process in FIG. 4 is terminated, the original VR view is displayed, and the process proceeds to S313 in FIG.

FIG. 5 shows a flowchart of the extraction preview process described in S427 of FIG. This processing is realized by expanding the program recorded in the nonvolatile memory 203 into the memory 202 and executing it by the CPU 201 . The extraction preview process of FIG. 5 is a process for confirming whether or not the extraction range specified by the user in the extraction editing process of FIG. 4 can be confirmed.

In S501, the CPU 201 refers to the setting information held in the memory 202, and determines whether or not the current setting is valid for the elevation/depression angle. It should be noted that what is referred to here is a setting item whose setting can be changed by a touch operation on the elevation/depression angle enable/disable button 613 . If it is determined that the elevation/depression angle is valid, the process proceeds to S502, and if it is invalid, the process proceeds to S503.

In S502, the CPU 201 calculates the azimuth angle and the elevation/depression angle of the current display range (the display range immediately before the cutout button 604 is touched on the cutout edit screen of the process described in FIG. 4 or a preview screen described later). to determine the extraction range, and perform extraction processing. More specifically, from the 360-degree VR image, a 180-degree range centered on the center of the display range (the position where the pointer 603 is displayed) (a range of 90 degrees in each direction from the center) is cut out, Generate a 180 degree VR image (edited VR content). This process will be described later with reference to FIGS. 16(a) and 16(c).

In S503, the CPU 201 displays the azimuth angle and horizontal direction (elevation) of the current display range (the display range immediately before the cutout button 604 is touched on the cutout editing screen of the processing described with reference to FIG. 4 or a preview screen described later). The extraction range is determined based on the depression angle of 0 degrees, and the extraction process is performed. At this time, regardless of the elevation/depression angle of the display range before extraction, the extraction range is determined based on the horizontal direction (the direction perpendicular to the zenith axis). More specifically, a 180-degree range centered on a point determined by an azimuth angle corresponding to the center of the display range in the horizontal direction (the position where the pointer 603 is displayed) is extracted from the 360-degree VR image. , to generate a 180-degree VR image (edited VR content). This process will be described later with reference to FIGS. 16(a) and 16(b).

In S504, the CPU 201 displays a preview screen on the display 205 so that the clipping result in S502 or S503 can be checked. 6(e), 6(f), and 6(g) show display examples of the preview screen.

FIG. 6(e) shows that the clipping button 604 is touched in the state of FIG. 6(b) (the display range of the VR image 602 is the range of FIG. 6(b)), and the clipping preview process is started. It is a display example of the preview screen when the transition is made. In addition to the fan-shaped display range 605 and non-display range 606, the circular display direction guide in the lower right part of the screen includes an exclusion range 622 (black painted part) is displayed. This allows the user to recognize that there is no image in the range corresponding to the exclusion range 622 in all 360 degrees of the VR image 602 . Also, the display form of the pointer 603 is changed and displayed as a pointer 621 . On the preview screen, the pointer 621 is displayed as an indicator that indicates the center of the designated extraction range. Therefore, the pointer 621 is not fixed at the center of the display 205, but displayed at a position indicating the center of the clipping range (effective video range) of the VR image 602. FIG. Therefore, as the display range is changed, the display position of the pointer 621 is also moved in conjunction with the movement of the display position of the center of the extraction range (effective video range) of the VR image 602 . Other display items with the same reference numerals as those in FIG. 6B are the same display items as those on the cutout editing screen. The double screen confirmation button 610 and the unnecessary part designation button 612 are hidden on the preview screen. A playback start direction designation button 623, a save button 624, and a cancel button 625 are displayed as display items that are not displayed on the cutout editing screen but are displayed on the preview screen. The reproduction start direction designation button 623 is an instruction icon for designating a portion to be a display range when the extracted VR image is first displayed next time. The center of the display range at the time when the touch operation (direction designation instruction for designating the reproduction start direction) is performed on the reproduction start direction designation button 623 is the display range when the cropped VR image is first displayed next time. It is the center, that is, the aforementioned display start direction in the VR image after clipping. The save button 624 is an instruction icon for instructing to fix the cutout range and record the image file of the cutout VR image. A cancel button 625 is an instruction icon for returning to the cutout edit screen without saving the VR image in the current cutout range.

In S505 and S506, the CPU 201 changes the display range in accordance with the touch move to the area where the VR image 602 is displayed. The details of the processing are the same as those of S305 and S306 in FIG. 3 described above. As described above, when the display range changes, the display position of the pointer 621 on the display 205 also changes. Even if the display range of the VR image changes, the relative positional relationship of the pointer 621 with respect to the VR image does not change.

FIG. 6(f) shows a display example when the display range is changed by touch-moving to the left from the state of FIG. 6(e). The display range of the VR image 602 has changed, and accordingly, the angle information 608 and the range indicated by the display range 605 and non-display range 606 of the display direction guide have also been changed. Of the display direction guide, the portion of the exclusion range included in the display range is displayed in a pattern 626 so as to be identifiable. The VR image 602 is scrolled to the left compared to FIG. 6(e), and the area on the right side of the VR image has become the new display range compared to the state of FIG. 6(e). . That is, the display range has moved to the right. As can be seen from the angle information 608 and the display direction guide in the example of FIG. 6(f), the display range has been changed to the right by approximately 90 degrees compared to the display range of FIG. 6(e). Since the cutout is performed in the range of 180 degrees around the position indicated by the pointer 621, the exclusion range 630 begins to appear within the display range. The excluded range 630 may be identified as a range in which no valid image exists, and may be displayed in a single color, a predetermined pattern, or a gradation, for example. In FIG. 6(c), the car passing with the license plate visible, which was described as an example that the user wants to exclude from the image after clipping, is excluded as an exclusion range 630 in FIG. 6(f). It can be confirmed that it is not included in the VR image of.

FIG. 6(g) shows a display example when the display range is changed by further touch-moving from the state of FIG. 6(f) to the left. The display range of the VR image 602 has changed, and accordingly, the angle information 608 and the range indicated by the display direction guide have also changed. As can be seen from the angle information 608 and display direction guide in the example of FIG. ing. The VR image 602 is not included within the display range, and the exclusion range 630 occupies the entire display range. In FIG. 6(d), the photographer himself explained as an example that the user wants to exclude from the cropped image is excluded as an exclusion range 630 in FIG. 6(g), and is not included in the cropped VR image. It can be confirmed that the

The processing of S507 to S515 is processing for changing the display range according to the posture change, processing for changing the display magnification according to the pinch operation, and display processing at the hemisphere magnification according to the entire hemisphere instruction. Since the processing is the same as that of S412, description thereof will be omitted. Note that in the hemisphere whole display in S514, unlike the hemisphere whole display in S411, the exclusion range (the hatched portion 702 in FIG. 7A) is not displayed. FIG. 7B shows a display example of the whole hemisphere display on the preview screen. Since only the effective video range of the cut-out VR image excluding the excluded range is displayed, the shaded portion 702 in FIG. 7A is not displayed. That is, the VR image before clipping is not displayed outside the effective image range after clipping. With this display, the user can confirm what kind of subject is located in the central part of the VR image after clipping, and what subject is included in the clipping range within one screen. When the extraction button 604 is touched in the display state shown in FIG. 7A, the display state is changed to that shown in FIG. 7B. In the display states of FIGS. 6(e) to 6(g), when there is a touch operation on the whole hemisphere display button 609, the display state transitions to the display state of FIG. 7(b).

The processes of S516 and S517 are processes in response to the touch operation on the edge fine adjustment button 611, and are the same as the above-described processes of S415 and S416, so description thereof will be omitted.

S518 to S521 are processes in response to a touch operation on the elevation/depression angle enable/disable button 613, and are the same as the above-described processes of S419 to S422, so description thereof will be omitted. Note that if the elevation/depression angle is switched from valid to invalid in S520, the process proceeds to S503 to change the extraction range. Note that when proceeding from S520 to S503, the extraction range is not set based on the display range at that time, but the extraction button 604 that was the factor in determining the extraction range at S520 is set. It is performed based on the display range when there is an operation. That is, the azimuth of the center of the cutout range at the time of S520 remains the same, and the elevation/depression angle at the center of the cutout range becomes horizontal, so that the 360-degree VR image (global range) to the 180-degree range ( Hemispheric extent) segmentation is performed. Also, if the elevation/depression angle is switched from invalid to valid in S521, the process advances to S502 to change the extraction range. Note that when proceeding from S521 to S502, the extraction range is not set based on the display range at that time, but the extraction button 604 that was the factor in determining the extraction range at the time of S521 is displayed. It is performed based on the display range when there is an operation. That is, the azimuth at the center of the cutout range at the time of S521 remains unchanged, and the elevation/depression angle at the center of the cutout range is the determining factor of the cutout range at the time of S521. A VR image of 180 degrees is cut out from a VR image of 360 degrees so that the elevation/depression angle is within the display range.

The processes of S522 and S523 are processes in response to a touch operation on the orientation detection ON/OFF button 614, and are the same as the above-described processes of S423 and S424, so description thereof will be omitted.

In S524, the CPU 201 determines whether or not the reproduction start direction specifying button 623 has been touched based on the information notified from the touch panel 206a. If the playback start direction specifying button 623 has been touched, the process proceeds to S525; otherwise, the process proceeds to S526. In S525, the CPU 201 determines the display start direction of the clipped VR image to be the center of the current display range, and records in the memory 202 position information indicating the center of the current display range of the VR image. This position information is recorded as information indicating the display start direction in the attribute information of the image file of the clipped VR image in S535, which will be described later.

In S526, the CPU 201 determines whether or not the cut button 604 has been touched based on information notified from the touch panel 206a. If there is a touch operation designating the cutout button 604, the process proceeds to S527; otherwise, the process proceeds to S532.

In S527, the CPU 201 determines whether or not an unnecessary portion is registered by unnecessary portion specifying processing, which will be described later. If the unnecessary part is registered, the process proceeds to S528, and if the unnecessary part is not registered, the process proceeds to S501. Subsequently, the process proceeds from S501 to S502 or S503, and the cutout range is changed based on the display range at the time when it is determined that the cutout button 604 has been touched in S526. As described above, in the present embodiment, when the user wishes to change the cutout range on the preview screen, by performing a touch operation on the cutout button 604, the display range at that time, more specifically, the display at that time can be changed. The segmentation range can be modified based on the location of the center of the range. If the clipping is performed again in S502 or S503, the preview screen is updated in S504 so as to show the corrected clipping range. It should be noted that the clipping range may not be corrected on the preview screen. In this case, the cutout button 604 is hidden on the preview screen, or displayed in a display form such as grayed out indicating that the operation is impossible, and the determination in S526 is not performed. Even so, if the user wants to change the cutout range after viewing the preview screen, the user can touch the cancel button 625 to return to the cutout edit screen and then specify the cutout range again. You can change the extraction range by doing. The processing of S528 to S531 is processing when an unnecessary portion is registered, and is the same processing as S1215 to S1218 in FIG. 12, which will be described later.

In S532, the CPU 201 determines whether or not the cancel button 625 has been touched based on the information notified from the touch panel 206a. If there is a touch operation on the cancel button 625, the process proceeds to S533, and returns to the cutout edit processing (cutout edit screen) described above with reference to FIG. If there is no touch operation on the cancel button 625, the process proceeds to S534.

In S534, the CPU 201 determines whether or not the save button 624 has been touched based on the information notified from the touch panel 206a. If the save button 624 has been touched, the process proceeds to S535; otherwise, the process proceeds to S505 and repeats the process.

In S<b>535 , the CPU 201 generates an image file that can be displayed in the VR view from the VR image that has been clipped in the clipping range confirmed on the preview screen, and records it in the recording medium 208 . Here, it is assumed that the 360-degree VR image before clipping is clipped in S502 or S503, and the image held in the memory 202 or recording medium 208 is recorded as an image file. However, at the time of S535, the image processing unit 204 may be controlled from the 360-degree VR image before cropping, and a 180-degree VR image may be cropped and recorded. Also, the image file of the clipped VR image is recorded as a new save file. However, the image file of the 360-degree VR image before clipping may be overwritten and saved. The image file of the clipped VR image is recorded in a format conforming to the VR180 standard. When the image file of the clipped VR image is recorded, the process of FIG. 5 is terminated, and the process returns to S301 of FIG. 3 to perform the VR view display of the clipped original 360-degree VR image. It should be noted that after the processing of FIG. 5 is completed, the VR view display of the newly saved cropped 180-degree VR image may be performed.

The clipped VR image (image file that can be displayed in the VR view) recorded in S535 will be described. If the clipped VR image is a still image, the image within the clipped range is deformed into a rectangle using the equirectangular projection method, and multiple images are stored in one file in a multi-picture format. to record. When the 360-degree VR image (original image) before cropping is displayed as a normal planar image instead of a VR view, it is an image drawn in an equirectangular projection within a rectangle as shown in FIG. 18(a). Become. When a 180-degree range is cut out from this original image by the above-described processing, when displayed as a normal planar image instead of a VR view, it is drawn in an equirectangular projection within a substantially square as shown in FIG. 18(b). The captured image is recorded. At this time, the same image obtained by copying the image within the clipping range is recorded in the same file as the image for the right eye and the image for the left eye, respectively, even if there is no parallax. Two 180-degree ranges are extracted from one 360-degree VR image before extraction so as to have a pseudo parallax (that is, the ranges of the image for the left eye and the image for the right eye are slightly different). may Also in this case, the extraction range is determined based on the display range when the extraction button 604 is touch-operated. The file name of the extracted still image VR image is, for example, "123456.vr.jpg", and a three-letter string ".vr" is written before the extension ".jpg". When an image saved as a still image file is reproduced and displayed in a VR view, the image in FIG. 18(b) is mapped to a hemisphere and displayed. FIG. 18D shows an example of reproducing and displaying a clipped still image VR image file in a twin-lens VR view (an example of display on the display 205 when the VR goggles are not worn).

If the clipped VR image is a video, the video (video) in which the images within the clipped range are mapped in a circle or ellipse instead of the equirectangular projection is arranged side by side in the video of one video. generate and record a video file At this time, the same image obtained by copying the image within the clipping range is recorded side by side as an image for the right eye and an image for the left eye, respectively, even if there is no parallax. When such a moving image is displayed not as a VR view but as a normal planar image, it is displayed as shown in FIG. 18(c). Two 180-degree ranges are extracted from one 360-degree VR image before extraction so as to have a pseudo parallax (that is, the ranges of the image for the left eye and the image for the right eye are slightly different). may Also in this case, the extraction range is determined based on the display range when the extraction button 604 is touch-operated. The file name of the VR image of the clipped moving image is, for example, "123456.vr.mp4", and the three-character string ".vr" is written before the extension ".mp4". FIG. 18D shows a display example of a frame (display example on the display 205 when the VR goggles are not worn) when a VR image file of a cut-out moving image is reproduced and displayed in a twin-lens VR view. Similar to the example shown.

FIG. 8 shows a flowchart of the two-screen display process of S414 in FIG. 4 described above. This processing is realized by expanding the program recorded in the nonvolatile memory 203 into the memory 202 and executing it by the CPU 201 .

In S<b>801 , the CPU 201 performs initial display of two-screen display on the display 205 . FIG. 9A shows a display example of the initial display of the two-screen display on the display 205. FIG. The display screen of the two-screen display is divided into an upper area 901 that displays the cutout range 903 of the VR image 602 and a lower area 902 that displays the exclusion range 904 of the same VR image 602 . In the initial display, a distorted circular image is displayed in the circular display area of the upper area 901 so that the entire cropped range is within the display range. This display range is the same display range as the entire hemisphere display described with reference to FIG. 7(b). Furthermore, in the initial display, a distorted circular image is displayed in the circular display area of the lower area 902 so that the entire exclusion range is within the display range. This display range corresponds to the hemisphere range that is discarded after clipping. Other display items with the same reference numerals as above are the same as above. By viewing the screen in which the entire cutout range and the entire exclusion range are displayed at the same time in this way, the user can determine what subject is included in the cutout range included in the 360-degree VR image before cropping. , you can easily check what subjects are included in the exclusion range. In the VR view of the normal cutout edit screen, at a display magnification larger than the whole hemisphere display, only a part of the cutout range is displayed, and the display range is always the center of the cutout range, so the cutout range and exclusion Range boundaries are not displayed. Therefore, the boundary cannot be confirmed. In addition, at a display magnification higher than that for displaying the whole hemisphere on the preview screen, only part of the cutout range is displayed, so only one portion of the boundary between the cutout range and the exclusion range can be confirmed. If you adjust the extraction range by looking at only one part of the boundary, even if you fit the desired part inside the extraction range on the displayed boundary, it will fit in the opposite direction near the non-displayed boundary. It may happen that the desired subject protrudes from the cropping range. It is necessary to change the display range a plurality of times or significantly to check whether or not the expected portions are contained in the vicinity of all boundaries. On the other hand, in the two-screen display as shown in FIG. 9A, the user can check the entire extraction range and the exclusion range without changing the display range. Furthermore, the user can adjust the extraction range and the exclusion range by touch movement while performing such confirmation. Therefore, it is possible to prevent mistakes such as ignoring the presence of a subject that should be included in the clipping range and fitting it into the exclusion range, or unknowingly fitting a subject that should be excluded from the clipping range into the clipping range. can do. Note that the exclusion range displayed in the lower region 902 is a semi-transmissive mask of a predetermined color, color conversion such as monochrome or sepia, and shading so that it is easy to understand that it is a portion that will be discarded after extraction. etc. may be applied.

In S<b>802 , the CPU 201 determines whether or not there has been a touch move (image range changing operation) in the upper area 901 or the lower area 902 . If it is determined that there has been a touch move, the process proceeds to S803; otherwise, the process proceeds to S804. In S<b>803 , the CPU 201 changes the portion of the VR image 602 that will be the extraction range and the portion that will be the exclusion range. Specifically, the CPU 201 changes the display range of the VR image 602 displayed in one of the upper area 901 and the lower area 902 where the touch move is performed, following the movement of the touch position. At this time, if the elevation/depression angle is set to be valid, the display range is changed following both the horizontal movement component and the vertical movement component of the touch move. If the elevation/depression angle is disabled, the display range is moved according to the horizontal movement component of the touch move, but the display range is not moved according to the vertical movement component. ), and the display range does not move up or down. In addition, the display range of the VR image 602 displayed in the other area is changed in conjunction with the change of the display range in the area where the touch-move occurred. The hemisphere on the opposite side, which is not within the display range. As a result, the relative position of the pointer 603 (center position of the upper area) displayed at a fixed position on the display 205 (predetermined position on the screen of the display 205) with respect to the VR image 602 changes. The CPU 201 sets a hemispherical range centered on the position corresponding to the pointer 603 in the VR image 602 as the extraction range.

In S<b>804 , the CPU 201 determines whether or not a pinch operation has been performed in the upper area 901 . If it is determined that a pinch operation has been performed in the upper area 901, the process proceeds to S805; otherwise, the process proceeds to S807. In S805, if the image of the exclusion range in the lower area 902 is enlarged or reduced in S809 to be described later than the initial display in S801, the CPU 201 changes the display magnification of the image of the exclusion range so that the entire exclusion range is displayed. reset to the original display magnification. In S<b>806 , the CPU 201 changes the display magnification of the image of the extraction range portion of the VR image 602 displayed in the upper area 901 according to the pinch operation performed in the upper area 901 , and enlarges the image. Or zoom out and display. By enlarging and displaying the image of the cutout range, the user can check the details of the subject included in the cutout range.

In S<b>807 , the CPU 201 determines whether or not a pinch operation has been performed in the lower area 902 . If it is determined that there has been a pinch operation in the lower area 902, the process proceeds to S808; otherwise, the process proceeds to S810. In S808, if the image of the cutout range in the upper region 901 has been enlarged or reduced in the above-described S806 from the initial display in S801, the CPU 201 changes the display magnification of the image of the cutout range to the entire cutout range. is displayed, that is, reset to the initial display magnification. In S<b>809 , the CPU 201 changes the display magnification of the image of the exclusion range portion of the VR image 602 displayed in the lower area 902 according to the pinch operation performed in the lower area 902 , and enlarges or expands the image. Zoom out and display. By enlarging and displaying the image of the exclusion range, the user can check the details of the subject included in the exclusion range.

FIG. 9B shows a display example when the image displayed in the lower area 902 is enlarged in response to pinching out in the lower area 902 . The range displayed in the lower area 902 is not the entire exclusion range but a part of it, but since it is enlarged from the initial display in FIG. 9A, it is easy to check the state of the subject. When a pinch operation is performed in the upper area 901 from this state, the display magnification is reset by the process of S805, and the display magnification in the lower area 902 returns to the display magnification of the initial display shown in FIG. 9A. In this way, either the cutout range or the excluded range is always displayed at a display magnification that fits the entire area, so that the boundary between the cutout range and the excluded range can always be confirmed in one of the distorted circular images. do.

It should be noted that in the change of the magnification in accordance with the pinch operation in the two-screen display processing in S804 to S809, only the display magnification is changed, and the clipping range is not changed. That is, the relative position of the pointer 603 with respect to the VR image 602 remains unchanged.

In S<b>810 , the CPU 201 determines whether or not the cut button 604 has been touched. If there is a touch operation designating the clipping button 604, the process advances to step S811, and the CPU 201 performs the clipping preview process of FIG. In this case, in the first step S502 or S503 of the clipping preview process, the CPU 201 displays the center of the display range displayed in the upper area 901 (the position indicated by the pointer 603) when the clipping button 604 is touched. is the center of the extraction range. It should be noted that if the unnecessary portion is registered, the same processing as S426 to S431 in FIG. 4 described above may be performed.

In S<b>812 , the CPU 201 determines whether or not the save button 624 has been touched. If there is a touch operation designating the save button 624, the process proceeds to S813; otherwise, the process proceeds to S814. In S813, the CPU 201 performs extraction so that the center of the display range displayed in the upper area 901 (the position indicated by the pointer 603) is the center of the extraction range when the save button 624 is touched. More specifically, the CPU 201 cuts out a 180-degree range around the center of the display range in the upper region 901 (90-degree range in all directions from the center) from the 360-degree VR image. Then, the CPU 201 records the cropped VR image in the recording medium 208 as an image file that can be displayed in the VR view. The recording process after clipping is the same process as S535 described above. When the process of S813 is finished, the dual screen display process is finished and the process returns to S301 of FIG.

In S<b>814 , the CPU 201 determines whether or not the cancel button 625 has been touched. If there is a touch operation designating the cancel button 625, the dual-screen display process is terminated, and the process returns to S415 to display the cutout editing screen. At this time, the display range (the relative positional relationship of the pointer 603 with respect to the VR image 602) adjusted in the two-screen display process is handed over to the cutout editing screen. If there is no touch operation on the cancel button 625, the process returns to S802 and repeats the process.

Note that the two-screen display processing has been described as being displayed when the user performs a touch operation on the two-screen confirmation button 610 on the cutout edit screen. , two-screen display may be performed as a preview display of the clipping range. For example, in response to the touch operation of the clipping button 604 in S425 of FIG. 4, the two-screen display processing of FIG. 8 may be performed instead of the clipping preview processing of FIG.

FIG. 10 shows a flow chart of the edge fine adjustment processing of S416 in FIG. 4 and S517 in FIG. This processing is realized by expanding the program recorded in the nonvolatile memory 203 into the memory 202 and executing it by the CPU 201 . The edge fine adjustment process in FIG. 10 is a process for enlarging and displaying the vicinity of the boundary between the clipping range and the exclusion range of the VR image and allowing the user to finely adjust the clipping range.

In S1001, the CPU 201 initially displays an edge fine adjustment screen for cutting out an image from the VR view.

FIG. 11A shows a display example of the edge fine adjustment screen displayed on the display 205 immediately after transitioning to the edge fine adjustment process. FIG. 11A shows an example of the horizontal display of the two types of edge fine adjustment screens, horizontal display and vertical display. The main area 1101 displays the central portion of the clipping range, which is a portion of the same VR image 602 that was displayed on the clipping edit screen or preview screen. In the main area 1101, since the VR image 602 is displayed at a display magnification that does not distort the peripheral portion, the edges of the clipping range (boundary between the clipping range and the exclusion range) are not displayed. A pointer 603 is displayed at a fixed position in the main area 1101 (center of the main area 1101). The display position of the background VR image is determined so that the pointer 603 is at the same position as the position displayed on the cut-out editing screen with respect to the VR image. By looking at the main area 1101, the user can confirm what is captured in the center of the cutout range. Below the main area 1101, a left area 1102 and a right area 1103, which are edge display areas, are displayed side by side. A left area 1102 and a right area 1103 display a portion of the extraction range that includes the boundary between the extraction range and the exclusion range.

A left area 1102 displays a partial boundary portion (partial range including the edge of the extraction range) of the extraction range. Immediately after initial display and resetting, the left boundary portion at the vertical position indicated by the pointer 603 (position of elevation/depression angle indicated by the pointer 603) is displayed. In the left area 1102, an image area 1102a indicating the inside of the cutout range and an exclusion area 1102b indicating the outside of the cutout range (within the exclusion range) are divided left and right and displayed adjacent to each other. You can check what is going on. The positional relationship between the image area 1102a and the exclusion area 1102b in the left area 1102 is fixed, and does not change even if there is a touch move, which will be described later.

In the right area 1103, there is a part of the cutout range that is 180 degrees opposite to the range displayed in the left area 1102 (a boundary that is discontinuous with the range displayed in the left area 1102). Is displayed. Immediately after initial display and resetting, the right boundary portion at the vertical position indicated by the pointer 603 (position of elevation/depression angle indicated by the pointer 603) is displayed. In the right area 1103, an image area 1103a indicating the inside of the cutout range and an exclusion area 1103b indicating the outside of the cutout range (within the exclusion range) are divided left and right and displayed adjacent to each other. You can check what is going on. The positional relationship between the image area 1103a and the exclusion area 1103b in the right area 1103 is fixed, and does not change even if there is a touch move, which will be described later. That is, even if the position of the cropping range in the VR image before cropping is changed according to the user's operation, the positions on the display 205 corresponding to the ends of the cropping range are not changed. The position on the display 205 corresponding to the edge of the cutout range is the boundary position between the image area 1103a and the exclusion area 1103b and the boundary position between the image area 1102a and the exclusion area 1102b.

When the user sees such a display, for example, when the roadway portion of the VR image 602 is excluded and the sidewalk portion is to be included in the clipping range, the desired portion is included in the clipping range. It is possible to easily confirm whether or not there is, and preferably perform fine adjustment of the extraction range. In the VR view of the normal cutout edit screen, at a display magnification larger than the whole hemisphere display, only a part of the cutout range is displayed, and the display range is always the center of the cutout range, so the cutout range and exclusion Range boundaries are not displayed. Therefore, the boundary cannot be confirmed. In addition, when the preview screen is displayed at a display magnification higher than that for displaying the whole hemisphere, only a part of the clipping range is displayed, so only one boundary between the clipping range and the exclusion range can be confirmed. If you adjust the cropping range by looking at only one boundary, even if you fit the desired part inside the cropping range on the displayed boundary, the object you want to fit around the other boundary is the opposite. may protrude from the clipping range. It is necessary to change the display range a plurality of times or significantly to check whether or not the desired portions are contained near the boundaries of both sides. On the other hand, on the edge fine adjustment screen as shown in FIG. 11( a ), the user can confirm the boundary between the extraction range and the exclusion range without changing the display range, and find the desired image within the extraction range. You can check if the part fits. Furthermore, on the edge fine-tuning screen as shown in FIG. 11(a), the user can check a plurality of boundary portions (boundary portions at both ends) at the same time, and check whether or not the desired portion is within the cropping range. can be confirmed in a short time or with a small number of operations.

A horizontal/vertical switching button 1104, which is a touch icon, is displayed in the upper left part of the left area 1102 (above the exclusion area 1102b). Displaying the horizontal/vertical switching button 1104 at this position does not interfere with the visibility of the image within the clipping range displayed in the image area 1102a. Also, even when the horizontal display and the vertical display are switched, the horizontal/vertical switching button 1104 does not change the display position, and is displayed at a position that does not interfere with the visibility of the image within the cutout range after switching. do. Note that the horizontal/vertical switching button 1104 may be displayed in the lower right portion of the right area 1103 (under the exclusion area 1103b). Even at this position, the image within the clipping range can be displayed in both the horizontal direction and the vertical direction without obstructing the visibility. A reset button 1105, which is a touch icon, is displayed in the lower right portion of the right area 1103 (under the exclusion area 1103b). At this position, the reset button 1105 can be displayed without interfering with the visibility of the image within the clipping range in both horizontal display and vertical display. Note that the reset button 1105 may be displayed in the upper left portion of the left area 1102 (above the exclusion area 1102b) for the same reason.

FIG. 11(c) shows a display example of the vertical display of the two types of edge fine adjustment screens, horizontal display and vertical display. The display contents of the main area 1101 are the same as those of the horizontal display. Below the main area 1101, a left area 1112 and a right area 1113, which are end display areas, are displayed side by side. A left region 1112 and a right region 1113 display a portion of the extraction range that includes the boundary between the extraction range and the exclusion range.

A left area 1112 displays a part of the boundary portion of the extraction range. Immediately after initial display and resetting, the upper boundary portion at the horizontal position indicated by the pointer 603 (position of the azimuth angle indicated by the pointer 603) is displayed. In the left area 1112, an image area 1112a indicating the inside of the cutout range and an exclusion area 1112b indicating the outside of the cutout range (within the exclusion range) are divided vertically and displayed adjacent to each other. You can check what is going on. The positional relationship between the image area 1112a and the exclusion area 1112b in the left area 1112 is fixed, and does not change even if there is a touch move, which will be described later.

A right area 1113 displays a part of the boundary portion of the cutout range that is 180 degrees opposite to the range displayed in the left area 1112 . Immediately after initial display and resetting, the lower boundary portion at the horizontal position indicated by the pointer 603 (position of the azimuth angle indicated by the pointer 603) is displayed. In the right area 1113, an image area 1103a indicating the inside of the cutout range and an exclusion area 1103b indicating the outside of the cutout range (within the exclusion range) are divided vertically and displayed adjacent to each other. You can check what is going on. The positional relationship between the image area 1113a and the exclusion area 1113b in the right side area 1113 is fixed, and does not change even if there is a touch move, which will be described later.

A horizontal/vertical switching button 1104, which is a touch icon, is displayed in the upper left portion of the left area 1112 (above the exclusion area 1112b). This display position is the same as in FIG. 11(a). This display position is an area where the exclusion area 1102b in FIG. 11A and the exclusion area 1112b in FIG. 11C overlap. Also, a reset button 1105 is displayed in the lower right portion of the right area 1113 (below the exclusion area 1113b). This display position is the same as in FIG. 11(a). This display position is an area where the excluded area 1103b in FIG. 11A and the excluded area 1113b in FIG. 11C overlap.

In S1002, the CPU 201 determines whether or not the reset button 1105 has been touched based on information notified from the touch panel 206a. If there is a touch operation designating the reset button 1105, the process advances to S1001, and the display position changed in S1014 or S1020, which will be described later, is returned to the original display position. As a result of this resetting, the left and right boundaries of the vertical center of the extraction range are displayed in the horizontal display, and the upper and lower boundaries of the horizontal center of the extraction range are displayed in the vertical display. If the reset button 1105 is not touched, the process advances to step S1103.

In S1003, the CPU 201 determines whether or not the horizontal/vertical switching button 1104 has been touched based on information notified from the touch panel 206a. If there is a touch operation designating the horizontal/vertical switching button 1104, the process proceeds to S1004; otherwise, the process proceeds to S1005.

In S1004, the CPU 201 switches to vertical display if the edge fine adjustment screen (edge display area) is displayed in the horizontal direction, and switches to horizontal display if it is displayed in the vertical direction. For example, when the horizontal/vertical switching button 1104 is touched while the display is as shown in FIG. 11A, the display is switched to that shown in FIG. 11C. When the horizontal/vertical switching button 1104 is touched while the display is as shown in FIG. 11C, the display is switched to that shown in FIG. 11A.

In S1005, the CPU 201 determines whether a touch move has been performed in the main area 1101 based on information notified from the touch panel 206a. More specifically, it is determined whether or not the main area 1101 has been touched down and a touch move has been performed. If the touch-move is performed starting from the main area 1101, the process proceeds to S1006; otherwise, the process proceeds to S1008.

In S1006, the CPU 201 changes the display range of the VR image displayed in the main area 1101 in accordance with the slide operation by touch move. This process is the same as the process of S306 in FIG. 3 described above.

In S1007, the CPU 201, in response to the slide operation by touch move, displays the display range of the VR image in the left area 1102 and the right area 1103 in the case of horizontal display (that is, the portion displayed in the image area 1102a and the image area 1103a). to change In the case of vertical display, the display range of the VR image in the left side area 1112 and the right side area 1113 (that is, the portion displayed in the image area 1112a and the image area 1113a) is changed. This is because the cutout range is changed according to the touch and move in the main area 1101, and the positional relationship between the cutout range and the exclusion range in the VR image changes.

In S1008, the CPU 201 determines, based on the information notified from the touch panel 206a, whether or not any of the left area 1102, the right area 1103, the left area 1112, and the right area 1113 excluding the touch button display area has been touched down. judge. If any of the left area 1102, right area 1103, left area 1112, and right area 1113 excluding the touch button display area is touched down, the process proceeds to S1009; otherwise, the process proceeds to S1023.

In S<b>1009 , the CPU 201 clears (turns off) a move flag indicating a vertical touch move or a horizontal touch move held in the memory 202 and initializes it.

In S1010, the CPU 201 determines whether a touch move has been performed in the vertical direction of the display 205 based on information notified from the touch panel 206a. For example, whenever there is a touch move of a unit movement amount, if the vertical movement component is larger than the horizontal movement component, it is determined that the vertical touch move has been performed. If the touch move is performed in the vertical direction, the process advances to S1011; otherwise, the process advances to S1016.

In S1011, the CPU 201 determines whether or not the left/right move flag, which is held in the memory 202 and indicates that the touch move in the left/right direction is in progress, is turned on. If it is determined that the left/right move flag is ON, the vertical movement component is ignored, so the process proceeds to S1016. If it is determined that the left/right move flag is off, the process proceeds to S1012.

In S<b>1012 , the CPU 201 sets on (records in the memory 202 as on) a vertical move flag indicating that the vertical touch move is in progress, and proceeds to S<b>1013 .

In S1013, the CPU 201 determines whether or not the current display mode is the horizontal display mode, out of the horizontal display mode and the vertical display mode of the edge fine adjustment screen (edge display area). If the display is in the horizontal direction, the process proceeds to S1014. Otherwise, if the display is in the vertical direction, the process proceeds to S1015.

In S1014, the CPU 201 updates the display positions of the VR images in the left area 1102 and the right area 1103 along the boundaries of the extraction range. This means that if a hemisphere is specified as the cutout range of the entire sphere on which the 360-degree VR image is mapped, the display range moves along the cutout boundary of the circular hemisphere. Therefore, the positional relationship between the clipping range and the exclusion area in the VR image does not change, and the range of the VR image displayed in the image area 1102a and the range of the VR image displayed in the image area 1103a change. The image area 1102a and the image area 1103a maintain the relationship of displaying a portion with a cut-out boundary that differs by 180 degrees from each other even if the display range changes. Therefore, when the user performs a touch move up and down by 180 degrees, it is possible to check the entire range of the boundary of the cutout range currently set. Since the extraction range is not changed in the process of S1014, the display content of the main area 1101 is not changed.

In FIG. 11B, a downward touch move is performed on the left side area 1102 from the state of FIG. A display example when scrolling is shown. The portion displayed in the left area 1102 moves along the boundary of the extraction range from the state of FIG. In (b), the boundary portion in the direction of the sky is displayed in the left area 1102 . The portion displayed in the right area 1103 is an image of the ground direction that is 180 degrees opposite to the sky direction displayed in the left area 1102 in the boundary of the extraction range.

In S1015, the CPU 201 changes the display range of the VR image displayed in the image area 1112a and the image area 1113a according to the up and down touch move. This means that the position of the boundary of the cropping range for the VR image before cropping moves according to the vertical touch movement. Since the angle of the extraction range is maintained at 180 degrees, the display range of both the image area 1112a and the image area 1113a changes regardless of whether the vertical touch move is performed from either the left area 1112 or the right area 1113. . For example, when the display range of the image area 1112a changes due to movement of the boundary in the direction of narrowing the cutout range, movement of the boundary in the direction of widening the cutout range such that the angle of the cutout range is maintained at 180 degrees The display range of the image area 1113a also changes. As a result, the extraction range is changed, and the range displayed in the main area 1101 is also changed. The user performs a touch move up and down while confirming the extent of the cutout range and the extent of the exclusion range near the boundaries (image area 1112a and image area 1113a) to precisely adjust the cutout range. be able to. In this process, the width (180 degrees) of the cropping range remains unchanged, and the position of the cropping range in the VR image before cropping is changed.

In S1016, the CPU 201 determines whether or not a touch move has been performed in the horizontal direction of the display 205, based on information notified from the touch panel 206a. For example, every time there is a touch move of a unit movement amount, if the horizontal movement component is larger than the vertical movement component, it is determined that the touch move has been performed in the horizontal direction. If the touch move is performed in the horizontal direction, the process proceeds to S1017; otherwise, the process proceeds to S1022.

In S1017, the CPU 201 determines whether or not the vertical move flag, which is held in the memory 202 and indicates that the vertical touch move is in progress, is turned on. If it is determined that the vertical move flag is ON, the horizontal movement component is ignored, so the process proceeds to S1022. If it is determined that the vertical move flag is off, the process proceeds to S1018.

In S<b>1018 , the CPU 201 sets on (records in the memory 202 as on) a left/right move flag indicating that the touch move in the left/right direction is in progress, and proceeds to S<b>1019 .

In S1019, the CPU 201 determines whether or not the current display mode is the vertical display mode, out of the horizontal display mode and vertical display mode of the edge fine adjustment screen (edge display area). If the display is in the vertical direction, the process proceeds to S1020; otherwise, in the case that the display is in the horizontal direction, the process proceeds to S1021.

In S1020, the CPU 201 updates the display positions of the VR images in the left area 1112 and the right area 1113 along the boundaries of the extraction range. This is the same processing as S1014 described above. However, the display orientation of the VR image in the left area 1112 and the right area 1113 is different from S1014 by 90 degrees. Since the extraction range is not changed in the process of S1020, the display content of the main area 1101 is not changed.

In FIG. 11(d), from the state of FIG. 11(c), a left touch move is performed on the right side area 1113 to move the image portion displayed in the image area 1113a to the left (the display range is moved to the right in the direction of the image area 1113a). ) shows an example of display when scrolling to move to The portion displayed in the right area 1113 moves along the boundary of the extraction range from the state of FIG. In FIG. 11D, a boundary portion in the horizontal direction (horizontal direction) is displayed in the right area 1113 . The portion displayed in the left area 1112 is an image in the horizontal direction (horizon direction) 180 degrees opposite to the direction displayed in the right area 1113 in the boundary of the extraction range.

In S1021, the CPU 201 changes the display range of the VR image displayed in the image area 1102a and the image area 1103a according to the left and right touch-move. This means that the position of the boundary of the cropping range for the VR image before cropping moves according to left and right touch movements. Since the angle of the extraction range is maintained at 180 degrees, the display range of both the image area 1102a and the image area 1103a changes regardless of which position of the left area 1102 or the right area 1103 is touch-moved to the left or right. . For example, when the display range of the image region 1102a changes due to movement of the boundary in the direction of narrowing the cutout range, movement of the boundary in the direction of widening the cutout range such that the angle of the cutout range is maintained at 180 degrees The display range of the image area 1103a also changes. As a result, the extraction range is changed, and the range displayed in the main area 1101 is also changed. The user performs touch movements to the left and right while confirming the extent of the cutout range and the exclusion range near the boundaries (image areas 1102a and 1103a) to precisely adjust the cutout range. be able to. In this process, the width (180 degrees) of the cropping range remains unchanged, and the position of the cropping range in the VR image before cropping is changed.

In S1022, the CPU 201 determines whether or not touch-up has been performed based on information notified from the touch panel 206a. If touch-up has been performed, the process proceeds to S1023; otherwise, the process returns to S1010 and repeats the process.

In S1023, the CPU 201 determines whether or not the return button 1106 has been touched based on information notified from the touch panel 206a. If there is no touch operation to designate the return button 1106, the process returns to S1002 and the processing is repeated.

When the edge fine adjustment process ends, the cutout range adjusted in the edge fine adjustment process is inherited even in the process preceding the end fine adjustment process. Therefore, for example, when fine adjustment of the cutout range is performed in the edge fine adjustment process, and the end fine adjustment process is finished and the screen returns to the cutout edit process (cutout edit screen) of FIG. 4, the display range is not changed thereafter. If the cutout button 604 is touched at the end, the cutout range adjusted in the edge fine adjustment processing can be cut out. It should be noted that the edge fine adjustment screen may also receive an instruction to cut out, and when the instruction to cut out is received on the edge fine adjustment screen, the process may proceed directly to S426 or S427. Alternatively, the edge fine adjustment screen may also receive an instruction to cut and save, and when the instruction to cut and save is received on the edge fine adjustment screen, the process may proceed directly to S535.

FIG. 12 shows a flowchart of the unnecessary part designation processing of S418 of FIG. 4 described above. This processing is realized by expanding the program recorded in the nonvolatile memory 203 into the memory 202 and executing it by the CPU 201 .

In S<b>1201 , the CPU 201 displays an unnecessary part designation screen on the display 205 . FIG. 13(a) shows a display example of the unnecessary part designation screen. The display items with the same reference numerals as described above are the same as those described above (display items on the cutout edit screen and preview screen).

A pointer 1301 is fixedly displayed in the center of the screen for specifying an unnecessary part, and is an indicator for specifying an unnecessary part (the position of an unnecessary subject, the position of a subject to be excluded) in the VR image 602 . In order to avoid confusion with the pointers 603 and 621, the pointers 603 and 621 are displayed in a different display form (color, shape, pattern, etc. are different). However, the pointer 1301 functions as a pointer indicating the center of the extraction range when a save button 624 displayed on an unnecessary part designation screen (to be described later) is touched. An unnecessary part registration button 1302 is a touch button for designating and registering an unnecessary part. When a touch operation for designating the unnecessary part registration button 1302 is performed, the position information of the position indicated by the pointer 1301 at that point in the VR image 602 is registered as an unnecessary part (recorded in the memory 202). An unnecessary part information deletion button 1303 is a touch button for deleting registered unnecessary part information. When a touch operation for designating an unnecessary part information deletion button 1303 is performed, one piece of registered unnecessary part information is deleted (cleared), and registration as an unnecessary part is cancelled. A margin setting button 1304 is a touch button for setting the margin of the selection invalid range starting from the unnecessary part. An end button 1305 is a touch button for accepting an instruction to end the unnecessary part designation processing and return to the cutout editing processing.

S1202 and S1203 are processes for changing the display range according to the touch move, and since they are the same processes as S305 and S306 in FIG. 3, detailed description thereof will be omitted. Note that even if the display range of the VR image 602 is changed by touch-move, the display range of the pointer 1301 is not changed. Also, if an unnecessary portion is registered and the invalid selection range is included in the changed display range, the invalid selection range is displayed in a recognizable (emphasized) manner in the displayed VR image. This will be described later with reference to FIG. 13(b).

S1204 and S1205 are processing for changing the display magnification according to the pinch operation, and are the same processing as S309 and S310 in FIG. 3, so detailed description thereof will be omitted. If an unnecessary portion is registered and the display range after the display magnification is changed includes the invalid selection range, the invalid selection range is displayed in the displayed VR image so that the selection invalid range can be identified.

In S<b>1206 , the CPU 201 determines whether or not the unnecessary part registration button 1302 has been touched. If the unnecessary part registration button 1302 is touched, the process advances to S1207; otherwise, the process advances to S1208. In S<b>1207 , the CPU 201 registers position information (angle information of elevation/depression angle and angle information of azimuth angle) of the position indicated by the pointer 1301 in the VR image 602 as an unnecessary part and registers corresponding unnecessary part information. As unnecessary part information, the position information and the registration number (starting from 1 and incremented by 1 each time it is registered) are recorded in the memory 202 . Note that if the position indicated by the pointer 1301 has already been registered as an unnecessary part, the process of S1207 is skipped.

When the unnecessary part is newly registered, the CPU 201 sets an area of an angle obtained by combining the cutout angle and the margin angle around the position information of the newly registered unnecessary part as a selection invalid range. If you set the reference position of the extraction range in the invalid selection range, the registered unnecessary part will be included in the extraction range. It can also be said. Even if the reference position of the extraction range is set in the range other than the selection invalid range (selection valid range), the registered unnecessary part will not be included in the extraction range, so the range other than the selection invalid range It can also be said that it is an appropriate range suitable for setting the reference position of the extraction range. The cutting angle is 180 degrees in this embodiment. The margin angle can be arbitrarily changed by the user in S1210 and S1211, which will be described later, but the initial value is 10 degrees. If the cutout angle is 180 degrees and the margin angle is 10 degrees, the range of 100 degrees around the newly registered unnecessary part (90 degrees + 10 degrees, which is half of the cutout range 180), for a total of 200 degrees It becomes a selection invalid range. If there is already another registered unnecessary part and an invalid selection range has already been set, the invalid selection range set based on the newly registered unnecessary part is merged with the already set invalid selection range. The selected range is set as the selection invalid range. If any position within the selection invalid range is set as the center of the cutout range, there is a possibility that the subject (image) at the position designated as the unnecessary part will be included in the cutout range. That is, each position within the selection invalid range is a position that should not be set as the center of the extraction range. Then, the CPU 201 performs identification display on the VR image 602 displayed on the display 205 so that the set invalid selection range can be identified.

FIG. 13(b) shows a display example of the identification display of the selection invalid range. In the displayed VR image 602, a selection invalid range 1307 is displayed so as to be identifiable by diagonal lines. The display form of the identification display is not limited to this, and may be display using a translucent mask, display by changing colors such as monochrome or sepia, or the like. Also, in this embodiment, an example of highlighting and displaying the selection invalid range will be described. Also good. Further, it is also possible to hide the image of the selection invalid range in the VR image 602 and display the image other than the selection invalid range.

It should be noted that in the registration processing of the unnecessary part in S1207, if the entire range of the VR image before cropping becomes the selection invalid range due to the newly registered unnecessary part, An error is displayed indicating that additional registration is not possible. An error indicating that additional registration is not possible without registering new unnecessary parts even if the width of the range other than the invalid selection range (valid selection range) in the VR image before cropping is less than the threshold. display.

The flow until the selection invalid range is set will be described with reference to the image diagrams of FIGS. 14(a) to 14(g). A range 1450 in FIG. 14A indicates an area (display range) of the VR image displayed on the display 205, and a pointer direction 1451 indicates the direction of the center of the display range as viewed from the center of the virtual sphere. , correspond to the position indicated by the pointer 1301 . An unnecessary portion candidate 1452 is a portion that the user considers to be an unnecessary portion for clipping (a portion to be excluded) in the VR image. If there is an unnecessary portion for clipping, the user aligns a pointer direction 1451 corresponding to the center of the display area of the display 205 with an unnecessary portion 1452 for clipping (subject considered unnecessary for clipping). Then, an unnecessary part registration button 1302 is touched.

FIG. 14B is an image diagram immediately after one unnecessary place is registered by touching. A shaded selection invalid range 1453 represents an area corresponding to a cutout angle 1454 (180°) from the pointer direction 1451, and indicates the selection invalid range when the margin is 0 degrees. As shown in FIG. 14(c), if a position 1455 within the selection invalid range is specified as the center of extraction, the extraction range will be the range of the white portion 1456 in FIG. put away. In addition, in FIG.14(d), the black-painted part is an exclusion range.

When the margin angle is set, the invalid selection range is set to a range wider than 180 degrees (hemisphere) centering on the unnecessary part, and becomes the invalid selection range 1457 in FIG. 14(e). In addition, when a plurality of unnecessary parts are registered as shown in FIG. 14(f) (an example of registering two unnecessary parts 1452 and 1458), the selection invalid range is calculated centering on each, and all selection invalid The area where the range is combined becomes the selection invalid range.

Among the invalid selection range set (determined) in this way, the area displayed on the display 205 is displayed in an identifiable manner like the invalid selection range 1307 in FIG. 13(b). The identification display such as the selection invalid range 1307 is displayed not only on the screen for specifying an unnecessary portion but also on the cutout editing screen of the cutout editing process described with reference to FIG. Note that the identification display of the invalid selection range may be switched on/off (display/hide) by user operation. Also, a boundary line between the selection invalid range and the non-selectable range may be displayed. Furthermore, when the pointer 603 is in the selection invalid range on the cutout edit screen, the position of the pointer 603 can be changed as the center designation point of the cutout range by changing the display form of the pointer 603, displaying a warning icon or a warning message, etc. The user may be notified that it is unsuitable.

In S<b>1208 , the CPU 201 determines whether or not the unnecessary part information deletion button 1303 has been touched based on the information notified from the touch panel 206 a. If there is a touch operation designating the unnecessary part information deletion button 1303, the process proceeds to S1209; otherwise, the process proceeds to S1210. In S<b>1209 , the CPU 201 acquires unnecessary part information from the memory 202 . If no unnecessary part information is registered, the process is interrupted and the process proceeds to S1210. If it is registered, the display range of the VR image 602 is moved so that the angle of the position information with the largest registration number (unnecessary part registered last) becomes the center of the display, and the unnecessary part information is displayed. Delete from memory 202 . Then, the invalid selection range based on the deleted unnecessary portion is cleared (released). In other words, the invalid selection range based only on the deleted unnecessary part is removed from the merged invalid selection range, and the identification indication of the removed part is hidden (displayed as the valid selection range). If only one piece of unnecessary part information is registered and the unnecessary part information is deleted, the shaded portion such as the selection invalid range 1307 shown in FIG. 13B disappears.

In S1210, the CPU 201 determines whether or not the margin setting button 1304 has been touched based on information notified from the touch panel 206a. If there is a touch operation designating the margin setting button 1304, the process advances to S1211; otherwise, the process advances to S1211. In S1211, the CPU 201 reads margin setting information from the memory 202, displays a margin setting dialog, and accepts a margin setting operation. FIG. 13C shows a display example of the margin setting dialog 1306. As shown in FIG. The background area other than the margin setting dialog 1306 may be displayed dark. The CPU 201 changes the set numerical value of the margin according to the touch operation on the up and down arrow marks of the margin setting dialog 1306, the up and down touch move operation, and the numerical input operation performed by displaying the numeric keypad. The changed numerical value is stored in memory 202 . When the set numerical value of the margin is changed, accordingly, the width of the selection invalid range based on the registered unnecessary part information is also changed by the amount of the change of the margin. The setting of the margin is a setting of how much the width of the selection invalid range set based on one unnecessary portion should be wider than the width of the extraction range (180 degrees in this embodiment). The numerical range of the margin setting information is 0 to (360-cutting angle), and if the range is exceeded, the numerical value is not changed. When a portion other than the margin setting dialog 1306 is touched, the margin setting dialog 1306 is closed (hidden).

In S1212, the CPU 201 determines whether or not the save button 624 has been touched based on information notified from the touch panel 206a. If there is a touch operation designating the save button 624, the process proceeds to S1213; otherwise, the process proceeds to S1219.

In S<b>1213 , the CPU 201 determines whether or not unnecessary parts are registered in the memory 202 . If one or more unnecessary parts are registered, the process advances to S1215, and if none is registered, the process advances to S1214.

In step S1214, the CPU 201 cuts out based on the azimuth angle and the elevation/depression angle of the center of the current display range, that is, the center of the cutout range (the position indicated by the pointer 1301 in the case of the screen for specifying an unnecessary part). Determine the range and perform extraction processing. More specifically, a 180-degree range around the center of the display range (a 90-degree range in each direction from the center) is cut out from the 360-degree VR image. Then, the clipped VR image is recorded on the recording medium 208 as an image file that can be displayed in the VR view. The recording process after clipping is the same process as S535 described above. When the process of S1214 is finished, the unnecessary part designation process is finished and the process returns to S301 of FIG.

In S1215, the CPU 201 determines whether the center of the current display range, that is, the center of the extraction range (the position indicated by the pointer 1301 in the case of the screen for specifying an unnecessary portion) is within the selection invalid range. determine whether If it is within the selection invalid range, the process proceeds to S1216; otherwise, the process proceeds to S1214.

In S<b>1216 , the CPU 201 displays on the display 205 a warning notifying that the registered unnecessary part will fall within the extraction range based on the display range when the save button 624 is touched. FIG. 13D shows a display example of the warning displayed in S1216. A dialog 1308 displays an OK button 1309 and a Cancel button 1310 together with the message "Are you sure you want to include unnecessary parts?" By viewing this display, the user can recognize that the registered unnecessary portion will fall within the cutout range based on the display range at the time when the save button 624 is touched.

In S<b>1217 , the CPU 201 determines whether or not the OK button 1309 has been touched. If the OK button 1309 has been touched, the process advances to step S1214. That is, since the user is aware that the unnecessary portion will fit within the cut-out range, the cut-out range including the unnecessary portion is cut out, and the cut-out VR image is saved. If the OK button 1309 is not touched, the process advances to step S1218.

In S<b>1218 , the CPU 201 determines whether or not the cancel button 1310 has been touched. If there is no touch operation on the cancel button 1310, the process returns to S1217. When the cancel button 1310 is touched, the dialog 1308 is hidden without clipping and saving in the clipping range based on the display range when the save button 624 is touched, and the process returns to step S1202. repeat.

The processing of S1215 to S1218 can prevent the user from carelessly specifying the center of the extraction range within the selection invalid range. If the pointer is within the selection invalid range, the cut-out and save instruction may be ignored (invalidated) without displaying the warning (that is, without performing the processing of S1216 to S1218). Alternatively, only a warning may be displayed, and the selection of OK or Cancel may not be accepted and the extraction and storage may not be performed.

Note that a jump button may be displayed in addition to the various display items described with reference to FIG. 13(a). Then, when the pointer 1301 is within the selection invalid range, the VR image 602 is displayed so that the pointer 1301 is positioned outside the selection invalid range according to the touch operation (touch operation without moving the touch position) on the jump button. You can change the range.

Also, instead of or in addition to the unnecessary part registration button 1302 described with reference to FIG. 13A, a necessary part registration button may be displayed. In response to the touch operation on the necessary place registration button, the position of the VR image 602 indicated by the pointer 1301 at that time is recorded (registered) in the memory 202 as position information of the necessary place. Then, an area other than the area of the angle obtained by subtracting the margin angle from the clipping angle is calculated as the selection invalid range, centering on the position information of the required portion. As shown in the image diagram, if the necessary part 1459 is specified in FIG. Set as a range. If there are a plurality of unnecessary or necessary locations, the invalid selection range is set by combining the invalid selection ranges calculated based on all the designated positions (registered positions).

In S<b>1219 , the CPU 201 determines whether or not the end button 1305 has been touched. If there is no touch operation to designate the end button 1305, the process returns to S1202 and repeats the process.

Using FIGS. 15(a) to 15(d) and FIGS. 16(a) to 16(c), extraction processing when the elevation/depression angle is valid as described in S502, and when the elevation/depression angle is invalid as described in S503. Extraction processing in this case will be described.

FIG. 15(a) shows a display example of the cutout edit screen on the display 205 (that is, a display example before designation of the cutout range). The reference direction (display start direction) is assumed to be an azimuth angle of 0 degrees (for example, north) and an elevation/depression angle of 0 degrees (a direction perpendicular to the zenith axis and gravity direction axis, horizontal direction). FIG. 15A is a display example of a VR image 602 when the azimuth angle is 0 degrees and the elevation/depression angle is about 60 degrees on the cutout edit screen. That is, the pointer 603 indicating the center of the display range is displayed in a direction pointing upward (upward) from the horizontal direction. FIG. 15A shows an example of display when the above-described setting of the elevation/depression angle is disabled. It's becoming

FIG. 16(a) is a diagram schematically showing the display range of FIG. 15(a) on a virtual sphere onto which a 360-degree VR image is mapped. With the viewing position 1601 of the viewer as the center, the azimuth of 360 degrees around the zenith axis 1602 and the elevation/depression angle of 360 degrees around the horizontal axis perpendicular to the zenith axis 1602 can specify the position on the sphere. The 360-degree VR image before clipping is mapped on the entire sphere, and the entire sphere becomes the effective image range 1603 . FIG. 16A shows an example in which the line-of-sight direction 1605 is directed toward an object 1604 in the sky above the viewer in the north (that is, at an azimuth angle of 0 degrees). That is, the display range 1607 is set at a position where the center of the display range 1607 intersects with the line-of-sight direction 1605 . The display range of FIGS. 16(a) to 16(c) is deformed and narrower than the display range of FIGS. 15(a) to 15(d). However, the center of the display range in FIGS. 16(a) to 16(c) is shown on the same basis as the center of the display range shown in FIGS. 15(a) to 15(d). Therefore, the center of the display range 1607 is the position indicated by the pointer 603 in FIG.

When the angle of elevation/depression is invalid and the display range is set to the range shown in FIGS. is performed, and the display range shown in FIG. 15(b) is obtained. FIG. 15(b) is a display example of the preview screen immediately after the clipping range is set by touching the clipping button 604 from the state of FIG. 15(a). The elevation/depression angle of the display range is different from 60 degrees before clipping, and is in the horizontal direction (0 degree elevation/depression angle), and the pointer 621 is also displayed at the position of 0 degree elevation/depression angle. The azimuth angle of the display range in FIG. 15(b) is 0 degrees (north), which is the same as in FIG. 15(a) before cropping. Since the azimuth angle is the same as before extraction, if the display range is set to an azimuth angle of 90 degrees (east) before extraction, the display range of the immediately following preview screen and the center of the extraction range will also have an azimuth angle of 90 degrees (east). east).

FIG. 16(b) is a diagram schematically showing the cutout range in the state of FIG. 15(b) on a virtual sphere on which a 360-degree VR image is mapped. The white hemisphere on the right side of the sphere is the clipping range 1611 (effective video range) that becomes the 180-degree VR image after clipping. The opposite hemisphere is an exclusion range 1612 (invalid video range) that is excluded from the cropped VR image and discarded. A cutout range 1611 is a hemispherical range centered at a central position 1613 . A central position 1613 is a position with an azimuth of 0 degrees indicated by the specified clipping position 1606 and an elevation/depression angle of 0 degrees (horizontal) regardless of the specified clipping position 1606 . The center position 1613 of the cutout range 1611 does not match the specified cutout position 1606 based on the line-of-sight direction 1605 and the display range 1607 immediately before the cutout range designation operation is performed, and the elevation/depression angles are different. This is because the elevation/depression angle is disabled. A cutout plane 1610 of the cutout range 1611 is parallel to the zenith axis 1602 .

When the angle of elevation/depression is valid and the display range is at the positions shown in FIGS. is performed, and the display range shown in FIG. 15(d) is obtained. In addition, FIG.15(c) shall become the same display range as Fig.15 (a). The only difference from FIG. 15A is the setting of elevation/depression angle valid/invalid, and in FIG. It has become. FIG. 15(d) is a display example of the preview screen immediately after the clipping range is set by touching the clipping button 604 from the state of FIG. 15(c). The elevation/depression angle of the display range is the same as 60 degrees before clipping, and the pointer 621 is also displayed at the elevation/depression angle of 60 degrees. The azimuth angle of the display range in FIG. 15(d) is 0 degree (north), which is the same as in FIG. 15(c) before cropping.

FIG. 16(c) is a diagram schematically showing the display range and cutout range in the state of FIG. 15(d) on a virtual sphere on which a 360-degree VR image is mapped. A white hemisphere on the upper right side of the sphere is a clipping range 1611 (effective video range) that becomes a 180-degree VR image after clipping. The opposite hemisphere is an exclusion range 1612 (invalid video range) that is excluded from the cropped VR image and discarded. A central position 1613 of the extraction range 1611 coincides with the specified extraction position 1606 both in azimuth and elevation/depression angle. This is because the elevation/depression angle is set to valid.

In the extraction of the VR image according to the present embodiment, the initial setting of valid/invalid of the elevation/depression angle is invalid, and the extraction is performed in the extraction range as shown in FIG. 16(b). With this extraction range, the effective image range is from directly above (90 degrees elevation angle) to directly below (90 degrees depression angle), so viewers viewing with an HMD can simply change the direction of their face (just shake their head). ), the entire VR image after clipping can be easily viewed. If, as shown in FIG. 16(c), the effective image range extends from directly above to the rear side, it is necessary to turn the direction of the face beyond directly above to view the rear side beyond directly above. You have to point it backwards. In order to do so, it is necessary not only to shake the head, but also to perform movements that impose a relatively large physical burden, such as bending the upper body, twisting the upper body, and changing the standing direction to face the back. That is, it is relatively physically difficult to look over the cropped VR image by changing the display range according to the change in posture. On the other hand, regardless of the elevation/depression angle of the display range when specifying the extraction range, such a physical burden can be reduced by extracting a range centered on the horizontal direction as shown in FIG. 16(b). It is possible to generate a VR image in which it is easy to see the whole. In addition, since the human eye is at a position where it can see the front in the horizontal direction when facing the front, setting the elevation/depression angle of the center of the field of view (that is, the cropping range) to the horizontal direction is less uncomfortable in terms of the structure of the human body. .

Also, in the extraction of the VR image in this embodiment, if the elevation/depression angle valid/invalid setting is set to valid, the extraction is performed in the extraction range as shown in FIG. 16(c). This cropping range is suitable depending on the content of the subject, such as being able to overlook the entire starry sky and excluding unnecessary ground parts in the VR image after cropping, or being able to overlook a wide range of the room as seen from the ceiling.

By changing the setting of elevation/depression angle valid/invalid, whether the extraction is performed within the range 1611 shown in FIG. 16(b) or within the range 1611 shown in FIG. , is difficult for users to imagine and understand. Therefore, it is preferable to display guidance using a schematic diagram so that the user can easily understand what kind of difference in extraction range is caused by the difference in the setting of the elevation/depression angle enabled/disabled.

FIG. 17A shows a display example of the guidance display of the extraction range on the display 205. As shown in FIG. A guidance dialog 1700 in FIG. 17A is displayed when, for example, the elevation/depression angle enable/disable button 613 is touched to switch the elevation/depression angle setting from disabled to enabled (S422 in FIG. 4, S521 in FIG. 5). Time (eg 5 seconds) is displayed. A dialog 1700 displays a graphic 1701 showing a schematic diagram before extraction, a graphic 1702 showing a schematic diagram after extraction, and a message 1704 . A figure 1701 is a guide using a schematic diagram of a sphere, and shows a schematic diagram when a position above the elevation angle of 0 degrees is specified as a cut-out designated position (position indicated by an arrow) in the sphere before cut-out. A figure 1702 is a guide using a schematic diagram of a sphere, and indicates the extraction range when the elevation/depression angle setting is valid. is shown. A message 1704 indicates that the setting of the elevation/depression angle of the extraction range has been validated. Instead of the graphic 1701, a graphic 1703 shown in FIG. 17B may be displayed that shows the extraction range when the elevation/depression angle setting is valid.

FIG. 17(b) shows a display example of the guidance display of the extraction range on the display 205. As shown in FIG. A guidance dialog 1700 in FIG. 17B is displayed when, for example, the elevation/depression angle enable/disable button 613 is touched to switch the elevation/depression angle setting from enabled to disabled (S421 in FIG. 4, S520 in FIG. 5). Time (eg 5 seconds) is displayed. A dialog 1700 displays a graphic 1701 showing a schematic diagram before extraction, a graphic 1703 showing a schematic diagram after extraction, and a message 1705 . A figure 1701 is a guide using a schematic diagram of a sphere, and shows a schematic diagram when a position above an elevation/depression angle of 0 degree is designated as a designated cut-out position (position indicated by an arrow) before cut-out. It is the same as figure 1701 of 17(a). A figure 1703 is a guide using a schematic diagram of a sphere, and shows the extraction range when the elevation/depression angle setting is disabled. It shows that the hemisphere (the hemisphere whose center is horizontal) is cut out. A message 1705 indicates that the setting of the elevation/depression angle of the extraction range has been invalidated. Instead of the graphic 1701, a graphic 1702 shown in FIG. 17A may be displayed that shows the extraction range when the elevation/depression angle setting is disabled.

The guides shown in FIGS. 17(a) and 17(b) may be displayed not only when the elevation/depression angle setting is switched, but also when an explanation regarding the elevation/depression angle setting is displayed in an electronic manual or the like. . Information indicating the setting state, for example, guides as shown in FIGS. Also, instead of displaying the figure 1701 before extraction, a figure showing after extraction (a figure 1702 when the elevation/depression angle is invalid and a figure 1703 when the elevation/depression angle is valid) may be displayed.

In the above-described embodiment, an example of generating a narrower VR image from a VR image captured by an imaging device has been described, but the original VR image is not limited to the VR image captured by the imaging device. It is also applicable when clipping and generating narrower VR content from VR content created by computer graphics (CG) or the like.

Further, the processes of the various flowcharts described above do not necessarily need to be performed in combination, and each can be performed separately, and it is also possible to employ a part of the flowcharts described above in combination with other embodiments. be. For example, the whole hemisphere display, the two-screen display process, the edge fine adjustment process, etc. can be performed without being combined with the cutout editing process of FIG. It can be implemented alone as a process for specifying the extraction range. That is, for example, when it is determined in S311 of FIG. 3 that there is a touch operation on the cutout edit button, the cutout range is changed by directly shifting to the entire hemisphere display described with reference to FIGS. 7A and 7B. You may allow it to be specified. 8 and 9, or the edge fine adjustment process described with reference to FIGS.

According to the present invention described above, it is possible to generate, with good operability, VR content of a narrower range based on the user's manipulation of specifying a cut-out range from VR content.

Note that the various controls described above that are performed by the CPU 201 may be performed by one piece of hardware, or a plurality of pieces of hardware (for example, a plurality of processors or circuits) may share the processing to control the entire apparatus. may be performed.

Moreover, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention can be applied to the present invention. included. Furthermore, each embodiment described above merely shows one embodiment of the present invention, and it is also possible to combine each embodiment as appropriate.

Further, in the above-described embodiment, the case where the present invention is applied to the electronic device 200 (smartphone, etc.) has been described as an example, but this is not limited to this example. The present invention is applicable to any electronic device that can be controlled to display an operation screen for extracting narrower VR content from VR content. For example, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, printers, digital photo frames, music players, game machines, electronic book readers, video players, and the like. In addition, the present invention can also be applied to digital cameras, television devices, projection devices, tablet terminals, AI speakers, home appliances, vehicle-mounted devices, medical devices, and the like.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Claims (18)

an electronic device,
In the first screen, control is performed so that a partial range of the VR content having the first image range is displayed on the display means as a display range, and in response to a change in posture of the electronic device or a user's operation to change the display range display control means for changing the position of the display range;
receiving means for receiving a cutting instruction from a user;
generating means for generating edited VR content having a second video range narrower than the first video range of the VR content;
A second display in which a range of the VR content that is wider than the partial range and includes the entire second video range is displayed in one screen so that the extent of the second video range can be identified. and control means for controlling to display a screen. The control means displays, on the second screen, a range that includes the entire second video range and is wider than the second video range of the VR content before accepting the clipping instruction. 2. The electronic device according to claim 1, wherein control is performed so as to perform identification display that enables identification of the extent of the second image range. 3. The electronic device according to claim 2, wherein the identification display is at least one of a display indicating a boundary of the second image range and a display allowing identification of a range outside the second image range. machine. 3. The controlling means performs control so as not to display the outside of the second image range of the VR content on the second screen after receiving the clipping instruction. 4. The electronic device according to 3. 2. The controller controls to change the position of the second image range in the VR content according to a user's operation while the second screen is being displayed. 5. The electronic device according to any one of items 1 to 4. 6. The second screen according to any one of claims 1 to 5, wherein the VR content is displayed at a predetermined display magnification lower than that of the first screen. electronics. The control means is
While the VR image is being displayed on the first screen at a first display magnification, the display magnification of the VR content is reduced stepwise in a plurality of steps of 3 or more according to a first user operation. control and
Control to switch to the second screen in response to a second user operation different from the first user operation while the VR image is being displayed on the first screen at a first display magnification. 7. The electronic device according to claim 6, wherein: The control means switches to the first screen in response to the second user operation while the second screen is being displayed, and displays a display magnification before switching to the second screen. 8. The electronic device according to claim 7, wherein the control is performed so that the VR content is displayed in. 9. The electronic device according to claim 7, wherein the first display magnification is a display magnification when starting display of the first screen. 10. The electronic device according to any one of claims 1 to 9, further comprising recording means for generating and recording a file containing the edited VR content generated by said generating means. 11. The electronic device according to claim 10, wherein said recording means generates one still image file storing a plurality of images based on said second video range. 12. The electronic device according to claim 11, wherein the file name of the still image file includes a character string ".vr" before the extension. 13. The electronic device according to any one of claims 10 to 12, wherein said recording means generates one moving image file in which a plurality of moving images based on said second image range are arranged. 14. The electronic device according to claim 13, wherein the file name of the moving image file includes a character string ".vr" before the extension. In the second screen, regardless of the position of the second video range in the VR content, the control means displays an item indicating the center position of the second video range at a predetermined position on the screen. 15. The electronic device according to any one of claims 1 to 14, wherein the electronic device is controlled to display . A control method for an electronic device,
controlling the first screen to display a partial range of the VR content having the first image range on the display means as a display range, and in response to a change in posture of the electronic device or a user's operation to change the display range; a display control step of changing the position of the display range;
a reception step for receiving a cutting instruction from a user;
a generating step of generating edited VR content having a second video range narrower than the first video range of the VR content;
A second display in which a range of the VR content, which is wider than the partial range and includes the entire second video range, is displayed in one screen so that the extent of the second video range can be identified. A control method for an electronic device, comprising: a control step of controlling to display a screen. A program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 15 . A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 15 .

Images