JP6086125B2 - Imaging system, editing apparatus, editing method, and program - Google Patents

Description

  The present invention relates to panoramic video technology. In particular, the present invention relates to a technique for easily editing panoramic images.

  In recent years, content services that provide images according to the field of view of users who freely move in space using panoramic images of 360 ° (omnidirectional) continuously captured while moving in space are becoming widespread ( For example, non-patent documents 1 to 3).

“Google Street View”, [online], Google, Inc, [October 17, 2012 search], Internet <http://maps.google.co.jp/intl/en/help/maps/streetview/> “Motion VR of the Month 2006 calendar” using Quick Time VR, [online], Apple Computer, Inc, [October 17, 2012 search], Internet <http: //www.worldinmotionvr. com / motionvr_month / calendar.html> “360VR (Web content collection)”, [online] 360, [October 17, 2012 search], Internet <http://unimoto.sakura.ne.jp/360vr/>

  The panoramic image described above can be obtained, for example, by placing an omnidirectional camera on a photographing vehicle, a traveling platform or the like (hereinafter simply referred to as “cart”) and performing continuous photographing while the cart is traveling. Please refer to FIG. For example, consider a case where video content is created from a panoramic image taken between the reference position 51 and the reference position 52 on the shooting path 71 in FIG. In this case, at the shooting site, the running and shooting of the cart with the omnidirectional camera is usually started from the shooting start position 61 before the reference position 51. Then, panoramic images are continuously shot every moment while traveling on the shooting route 71. Finally, the cart is stopped at the photographing end position 62 ahead of the reference position 52, and at the same time, the photographing is finished. That is, at the shooting site, shooting is usually performed in excess of the section necessary for the content (section between the reference position 51 and the reference position 52).

  The main reason for extra shooting is the run-up section (the section from the shooting start position 61 to the reference position 51 in FIG. 8) until the cart reaches a certain traveling speed, and the deceleration is stopped from the certain traveling speed. 8 is provided (a section from the reference position 52 to the photographing end position 62 in FIG. 8), the cart traveling speed is kept as constant as possible in the section from the reference position 51 to the reference position 52, and the photographing intervals are uniform. This is to obtain an image as follows. In addition, by taking a picture including an extra section, there is also a purpose of avoiding a situation in which a part of a section necessary for content is not photographed (in this case, photographing is performed again). . From this point of view, it is usual to take a panoramic image including a section that is more than the section necessary for the content.

  By the way, since the panoramic image taken excessively becomes an unnecessary frame in the video content, it is necessary to exclude it at the time of video editing. For this reason, the video editor or the like performs a work of extracting panoramic image frames taken in the section from the reference position 51 to the reference position 52 using video editing software and the like, and excluding other unnecessary frames. However, the necessity / unnecessity of the frame largely depends on the subjectivity of the content editor, and there is a problem that the editing accuracy of the video content varies among the editors. Also, the task of identifying the necessary / unnecessary panoramic images was a burden on the editor.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to edit a panoramic image while maintaining editing accuracy at a certain level or more without depending on the subjective judgment of the editor. It is to provide a photographing system that reduces the burden on the occasion.

In order to achieve the above-described object, the first invention provides two or more markers attached at the same altitude, and a panoramic image including the markers from a position higher than the marker while traveling on a route on the marker. A photographing device for photographing and an editing device;
The editing device includes an acquisition unit that acquires vertical position information of each of the markers in the panoramic image, and an identification unit that specifies the panoramic image that has the closest vertical position information of each of the acquired markers. And a photographing system characterized by comprising:
According to the first invention, among the panoramic images photographed by the photographing device, only the panoramic image photographed immediately above the position (reference position) on the photographing route equidistant from each marker is included in each panoramic image. The position coordinates of the marker in the vertical direction (elevation direction) match. Therefore, a video editor or the like can easily specify a panoramic image taken near the reference position by confirming the display position of each marker of each panoramic image. Based on the identified panoramic image, it is possible to easily determine whether each frame of the captured panoramic image is necessary or unnecessary. That is, according to the first invention, video editing can be performed based on objective information (position information of each marker in the panoramic image) that does not depend on the subjectivity of the video editor or the like, and thus the editing accuracy varies. , And the burden of video editing can be reduced.

  The editing apparatus includes means for acquiring horizontal position information of markers in a plurality of panoramic images having different shooting directions specified by the specifying means, and horizontal position information of the markers in the plurality of panoramic images. Preferably, the apparatus further comprises means for determining a slide amount for aligning the orientations of the plurality of panoramic images. Thereby, it becomes possible to perform orientation alignment between panoramic images based on the horizontal position information of the markers in the panoramic images specified by the specifying means.

In order to achieve the above-described object, the second invention stores a panoramic image including the marker taken from a position higher than the marker while traveling on a route on two or more markers attached at the same altitude. Storage means; acquisition means for acquiring vertical position information of each of the markers in the panoramic image; and specifying means for specifying the panoramic image having the closest vertical position information of each of the acquired markers. ,
An editing apparatus comprising:

In order to achieve the above-described object, the third invention stores a panoramic image including the marker taken from a position higher than the marker while traveling on a route on two or more markers attached at the same altitude. A storage step; an acquisition step of acquiring vertical position information of each of the markers in the panoramic image; and a specifying step of specifying the panoramic image having the closest vertical position information of each of the acquired markers. The editing method is characterized by including.

In order to achieve the above-mentioned object, a fourth invention is a panoramic image including the marker photographed from a position higher than the marker while running on a route on two or more markers attached at the same altitude. Storing means for storing, acquiring means for acquiring vertical position information of each marker in the panoramic image, specifying means for specifying the panoramic image having the closest vertical position information of each acquired marker , A program characterized by functioning as.

  With the photographing system of the present invention, it is possible to reduce the burden of editing a panoramic image while maintaining the editing accuracy at a certain level or more without depending on the subjective judgment of the editor.

.... Conceptual diagram showing the photographing system 1 in the present embodiment. .... An example of the hardware configuration of the editing apparatus 10 .... Flowchart 1 for explaining the operation of the photographing system 1 .... Display of panoramic images taken at each shooting position .... Conceptual diagram showing how shooting is performed on two shooting paths .... Flowchart 2 for explaining the operation of the photographing system 1 .... A diagram representing a reference panoramic image specified on two shooting paths .... Conceptual diagram showing general panoramic image shooting

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Shooting system 1>
FIG. 1 is a schematic diagram of a photographing system 1 according to the present embodiment. The imaging system 1 mainly includes a plurality of markers 3 (markers 3A and 3B), an imaging device 4 that captures a panoramic image while traveling on the imaging path 7, and an editing device 10 that edits the captured panoramic image. Is composed of.
Each configuration will be described below.

(Marker 3)
As shown in FIG. 1, a plurality of markers 3 (markers 3 </ b> A and 3 </ b> B) are attached at positions that are substantially equidistant from the reference position 5 on the imaging path 7. In the case of FIG. 1, the marker 3 </ b> A is attached at a distance d south from the reference position 5, and the marker 3 </ b> B is attached at a distance d north from the reference position 5. It is assumed that the marker 3A and the marker 3B exist on the same altitude. In addition, it is desirable that the marker has an aspect that can be easily identified by a person or a computer, for example, a predetermined color or a predetermined shape.

(Photographing device 4)
The photographing device 4 includes an omnidirectional camera capable of photographing an omnidirectional panoramic image, a control device that performs photographing control of the omnidirectional camera, and a cart that travels with the omnidirectional camera and the control device mounted thereon. Composed. The photographing device 4 continuously photographs panoramic images while traveling on the photographing route 7 (in the case of FIG. 1, traveling from “south” to “north”). The time-series panoramic images captured and acquired are stored in the storage unit of the control device and also stored in the storage unit 12 of the editing device 10 to be described later.

  Unlike the present embodiment, the control device that performs shooting control of the omnidirectional camera or the like may be fixedly arranged at another position instead of being placed on the cart. In this case, the control device performs shooting control and the like of the traveling omnidirectional camera using wireless communication means. Further, unlike the present embodiment, the control device and the omnidirectional camera may be configured integrally. Furthermore, unlike the present embodiment, a person may take a picture while holding and moving the omnidirectional camera without using a cart.

(Editing device 10)
The editing device 10 is a general-purpose computer for editing a panoramic image photographed by the photographing device 4. The editing apparatus 10 provides various means for easily editing a panoramic image video to a video editor or the like.

  FIG. 2 is a hardware configuration diagram of the editing apparatus 10. Note that the hardware configuration in FIG. 2 is an example, and various configurations can be adopted depending on the application and purpose, and any of a desktop PC (Personal Computer), a notebook PC, a tablet PC, a smartphone, etc. Also good.

  As shown in FIG. 2, the editing apparatus 10 includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, and the like. 18 is connected.

  The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls and executes a program stored in the storage unit 12, ROM, storage medium, etc., in the work memory area on the RAM, drives and controls each device connected via the bus 18, and the image display device 1 The process to be described later is realized. The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily holds a loaded program, data, and the like, and includes a work area used by the control unit 11 to perform each process.

  The storage unit 12 is an HDD (Hard Disk Drive) or the like, and stores a program executed by the control unit 11, data necessary for program execution, an OS (Operating System), and the like. These program codes are read by the control unit 11 as necessary, transferred to the RAM, and read and executed by the CPU.

  In the present embodiment, the storage unit 12 stores time-series panoramic images captured and acquired by the imaging device 4. The storage unit 12 may hold marker identification information. For example, the identification information includes marker color information (red, blue, green, etc.), marker shape information (circle, triangle, square, etc.), marker position information (arrangement direction (north, south, etc.)), etc. It is. The control unit 11 can automatically identify the marker in the panoramic image based on the marker identification information.

The media input / output unit 13 is a media input / output device such as a CD drive, a DVD drive, an MO drive, and a floppy (registered trademark) disk drive, and inputs / outputs data such as images.
The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other devices via the network.

  The input unit 15 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 15.

The display unit 16 includes a display device such as a liquid crystal panel, and a logic circuit or the like (video adapter or the like) for realizing a video function of the computer in cooperation with the display device. The input unit 15 and the display unit 16 may be integrated like a touch panel display.
In the present embodiment, the panorama image stored in the storage unit 12 is displayed on the display unit 16.

  The peripheral device I / F (Interface) unit 17 is a port or an antenna for transmitting and receiving data between the computer and the peripheral device. The computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 17. I do. The connection form with the peripheral device may be wired (for example, USB (Universal Serial Bus) or the like) or wireless (for example, Bluetooth (registered trademark) or the like). The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

<Operation of the photographing system 1>
Next, the operation of the imaging system 1 will be described with reference to the flowchart of FIG.
First, the imaging device 4 continuously captures panoramic images while traveling on the imaging path 7 on the markers 3A and 3B (step S101). The time-series panoramic images acquired and acquired are stored in the storage unit 12 of the editing apparatus 10 (step S102).

  Next, the control unit 11 of the editing apparatus 10 reads the panorama image from the storage unit 12 and displays the read panorama image on the display unit 16 (step S103). Here, the control unit 11 can display an arbitrary panoramic image on the display unit 16 by receiving an operation of a video editor or the like via the input unit 15.

  FIG. 4 is a diagram illustrating an example of each panoramic image displayed on the display unit 16. Each of the panoramic images 9-1 to 9-5 in FIGS. 4A to 4E is a panoramic image captured at each of the shooting positions P1 to P5 on the shooting path 7 in the lower right diagram of FIG. Here, the marker 3a and the marker 3b displayed in the panoramic image correspond to the marker 3A and the marker 3B existing in the real space.

  For example, FIG. 4A shows a panoramic image 9-1 photographed at the photographing position P1. At the photographing position P1, the marker 3A and the marker 3B are located in the same direction as viewed from the photographing position P1 (see the lower right diagram in FIG. 4). At this time, in the panorama image 9-1, both the marker 3a and the marker 3b are displayed in the same horizontal direction (θ1 in FIG. 4). However, since the distance from the photographing position P1 to the marker 3A and the distance from the photographing position P1 to the marker 3B are different, the elevation angle at which each marker is desired differs from the photographing position P1. For this reason, the position coordinate in the vertical direction (elevation angle direction) of the marker 3a in the panoramic image is different from the position coordinate in the vertical direction (elevation angle direction) of the marker 3b.

  FIG. 4B shows a panoramic image 9-2 photographed at the photographing position P2. At the photographing position P2, the marker 3A and the marker 3B are located in different directions as viewed from the photographing position P2. At this time, in the panoramic image 9-2, the marker 3a and the marker 3b are displayed in different horizontal directions (θ1 and θ2 in FIG. 4), respectively. Further, since the distance from the photographing position P2 to the marker 3A and the distance from the photographing position P2 to the marker 3B are different, the elevation angle at which each marker is desired from the photographing position P2 is different. For this reason, the position coordinates in the vertical direction (elevation direction) of the marker 3a are different from the position coordinates in the vertical direction (elevation direction) of the marker 3b.

  FIG. 4C shows a panoramic image 9-3 photographed at the photographing position P3 (just above the reference position 5). At the photographing position P3, the marker 3A and the marker 3B are located in different orientations when viewed from the photographing position P3, as in the case of the photographing position P2. At this time, in the panorama image 9-3, similarly to the panorama image 9-2, the marker 3a and the marker 3b are displayed in different horizontal directions (θ1 and θ2 in FIG. 4). However, since the distance between the shooting position P3 and the marker 3A and the distance between the shooting position P3 and the marker 3B match, the elevation angle at which each marker is desired from the shooting position P3 matches. Therefore, the position coordinates (φ in FIG. 4) in the vertical direction (elevation direction) of the marker 3a and the marker 3b in the panoramic image match.

  FIG. 4D shows a panoramic image 9-4 photographed at the photographing position P4. At the photographing position P4, the marker 3A and the marker 3B are located in different directions as viewed from the photographing position P4, as in the case of the photographing positions P2 and P3. At this time, in the panorama image 9-4, similarly to the panorama images 9-2 and 9-3, the marker 3a and the marker 3b are displayed in different horizontal directions (θ1 and θ2 in FIG. 4), respectively. Similarly to the case of the photographing position P2, since the distance between the photographing position P4 and the marker 3A and the distance between the photographing position P4 and the marker 3B are different, the elevation angle at which each marker is desired from the photographing position P4 is different. For this reason, the position coordinates in the vertical direction (elevation direction) of the marker 3a in the panoramic image are also different from the position coordinates in the vertical direction (elevation direction) of the marker 3b.

  FIG. 4E shows a panoramic image 9-5 photographed at the photographing position P5. At the photographing position P5, the marker 3A and the marker 3B are located in the same direction as viewed from the photographing position P5. At this time, in the captured panoramic image 9-5, both the marker 3a and the marker 3b are displayed in the same horizontal direction (θ2 in FIG. 4).

  As described above, only in the panoramic image 9-3 (see FIG. 4C) photographed immediately above the reference position 5 (the photographing position P3 in FIG. 4 and the lower right diagram), the marker 3a and the marker in the panoramic image are displayed. It can be seen that the position coordinates in the vertical direction (elevation direction) of 3b coincide. Therefore, the video editor or the like was photographed near the reference position 5 by checking the display positions of the markers 3a and 3b in the vertical direction (elevation direction) for each panoramic image displayed on the display unit 16. A panoramic image can be easily specified. Then, based on the identified panoramic image, it is possible to easily determine whether each frame of the panoramic image is necessary or unnecessary.

  Returning to the flowchart of FIG. Subsequent processing relates to processing in which the editing apparatus 10 automatically specifies a reference panoramic image. Here, the reference panoramic image refers to a panoramic image taken closest to the reference position 5.

  The control unit 11 of the editing apparatus 10 identifies the marker 3a and the marker 3b in the panoramic image for each panoramic image, and acquires position information in the vertical direction (elevation direction) of the identified marker 3a and marker 3b (step S104). ). And the control part 11 specifies the panoramic image with the closest position information of the marker 3a and the marker 3b of the perpendicular direction (elevation angle direction) as a reference | standard panorama image (step S105). Thereby, a video editor or the like can easily determine whether each frame of the panoramic image is necessary or unnecessary based on the automatically specified reference panoramic image.

  Note that the control unit 11 uses the marker identification information (color information, shape information, position information, etc.) held in the storage unit 12 to identify the marker, and the marker 3a and the marker in the panoramic image. You may make it identify 3b.

  Further, the control unit 11 of the editing apparatus 10 may automatically extract necessary frames or remove unnecessary frames from the time-series panoramic images based on the reference panoramic image specified in step S103. Good (step S106). For example, the control unit 11 can also perform automatic editing processing by automatically extracting or automatically deleting a panorama image that is later in time series order or a previous panorama image in time series order with the reference panorama image as a boundary.

  Also, as shown in FIG. 8, when two reference positions (for example, the start position and the end position of video content) are defined on the shooting path, the control unit 11 of the editing apparatus 10 sets the time series of shooting. Two reference panorama images are specified from the panorama image, and based on the two specified reference panorama images, the panorama image between the two reference positions is automatically extracted as a necessary frame on the content, or other than the panorama image on the content Automatic editing can be performed by automatically deleting unnecessary frames.

  The operation of the imaging system 1 has been described above. According to this embodiment, the imaging device 4 performs continuous imaging of panoramic images while traveling on the imaging path 7 on a plurality of markers (markers 3A and 3B) attached at substantially equal distances from the reference position 5. Of the captured panoramic images, only in the panoramic image captured immediately above the reference position 5, the position coordinates in the vertical direction (elevation direction) of each marker in the panoramic image are the same. Therefore, a video editor or the like can easily specify a panoramic image taken near the reference position by confirming the display position of each marker of each panoramic image. Then, based on the identified panoramic image, it is possible to easily determine whether each frame of the panoramic image is necessary or unnecessary. That is, according to the present embodiment, video editing can be performed based on objective information (position information of each marker in the panoramic image) that does not depend on the subjectivity of the video editor or the like. And the burden of video editing can be reduced.

  The editing apparatus 10 also provides means for automatically specifying a reference panoramic image (a panoramic image taken closest to the reference position) based on the position information of each marker of the taken panoramic image. As a result, video editors can easily determine the necessity / unnecessity of panoramic images based on the automatically specified reference panoramic image, and the editing accuracy varies and the editing burden of video editing. Can be further reduced.

  Note that, unlike the present embodiment, the photographing apparatus 4 automatically performs editing based on the specified reference panorama image, means for automatically displaying the panorama image, means for automatically specifying the reference panorama image (automatic processing of necessary frames). It may be configured to include an extraction (or automatic removal of unnecessary frames) means. That is, a part or all of the functions of the control unit 11 of the editing apparatus 10 may be provided on the photographing apparatus 4 side.

  Further, as a post-processing of editing the panoramic image, the control unit 11 may perform a process of removing the marker from the panoramic image by assimilating the marker displayed in the panoramic image with the background. Accordingly, it is possible to prevent the video editing marker from being displayed on the final video content.

(Shooting panoramic images on multiple shooting paths)
By the way, it is rare that there is a single shooting path as shown in FIG. 1, and usually a panoramic image is shot on a plurality of shooting paths, and a series of video contents are connected by connecting panoramic images shot on each shooting path. Create The imaging system 1 according to the present embodiment can also be applied when imaging is performed on a plurality of imaging paths in this way.

  FIG. 5 is a conceptual diagram illustrating an example of the photographing system 1 when panoramic images are photographed on two photographing routes 7-1 and 7-2 in which the traveling direction of the photographing device 4 is different. Here, for the sake of simplicity, video content is created from the panoramic images included in the “content section W” shown in FIG. 5 among the panoramic images shot on the two shooting paths 7-1 and 7-2. Assume a case.

  Here, in order to acquire a panoramic image included in the content section W, a reference position 5 is defined at a position where the shooting path 7-1 and the shooting path 7-2 intersect. Then, each marker (marker 3A, 3B, 3C, 3D) is attached on the same altitude at a position that is substantially equidistant from the reference position 5.

  Here, the marker 3 </ b> A and the marker 3 </ b> B were photographed near the reference position 5 from a time-series panoramic image photographed while the photographing device 4 was traveling on the photographing route 7-1 from “south” to “north”. Used to specify a panoramic image.

  On the other hand, the marker 3C and the marker 3D are panoramas taken near the reference position 5 from time-series panoramic images taken while the imaging device 4 is traveling from “west” to “east” on the imaging path 7-2. Used to specify an image.

  By attaching the marker as described above, it is possible to specify the panoramic image included in the “content section W” from the time-series panoramic images photographed on the photographing route 7-1 and the photographing route 7-2. It becomes.

With reference to the flowchart of FIG. 6, the operation of the imaging system 1 when there are a plurality of imaging paths will be described.
First, the imaging device 4 continuously captures panoramic images while traveling from “South” to “North” on the imaging path 7-1 on the markers 3A and 3B (step S201). Hereinafter, the captured time-series panoramic images are referred to as “panoramic image group GX”.

Similarly, panoramic images are continuously shot while traveling from “west” to “east” on the shooting route 7-2 on the markers 3A and 3B (step S202). Hereinafter, the captured time-series panoramic images are referred to as “panoramic image group GY”.
The captured “panoramic image group GX” and “panoramic image GY” are stored in the storage unit 12 of the editing apparatus 10 (step S203).

  Next, the control unit 11 of the editing apparatus 10 reads the panorama image group GX and the panorama image group GY from the storage unit 12 and displays them on the display unit 16 (step S204). Here, the control unit 11 can display an arbitrary panoramic image on the display unit 16 by receiving an operation of a video editor or the like via the input unit 15.

  As described above, only in the panoramic image taken immediately above the reference position 5, the position coordinates in the vertical direction (elevation direction) of each marker in the panoramic image coincide with each other. The panorama image group GX and the panorama image group GY are captured near the reference position 5 in the panorama image group GX by confirming the vertical display position (elevation direction) of each marker of each frame of the panorama image group GY. Panoramic images (referred to as “panoramic images X”) and panoramic images (referred to as “panoramic images Y”) taken near the reference position 5 in the panoramic image group GY can be specified. Then, based on the specified panorama image X and panorama image Y, it is possible to easily determine whether the frames of the panorama image group GX and the panorama image group GY are necessary or unnecessary. That is, the panoramic image included in the content section W can be easily identified from the panoramic images taken by a plurality of routes, and desired video content can be easily created.

  Further, as will be described later, a panoramic image included in the content section W may be automatically specified.

  In this case, the control unit 11 of the editing apparatus 10 first automatically specifies the reference panorama image X and the reference panorama image Y on each shooting path (7-1, 7-2) (step S205). Here, the reference panoramic image X is a panoramic image taken closest to the reference position 5 in the panoramic image group GX. The reference panoramic image Y is a panoramic image taken closest to the reference position 5 in the panoramic image group GY.

  Specifically, the control unit 11 identifies the marker 3a and the marker 3b displayed in each frame of the panoramic image group GX, and acquires position information in the vertical direction (elevation direction) of the identified marker 3a and the marker 3b. The panoramic image having the closest position information in the vertical direction (elevation direction) between the acquired marker 3a and marker 3b is specified as the reference panoramic image X.

  Similarly, the marker 3c and the marker 3d displayed in each frame of the panoramic image group GY are identified, position information in the vertical direction (elevation direction) of the identified marker 3c and the marker 3d is obtained, and the obtained marker 3c and the marker are obtained. The panorama image with the closest position information in the vertical direction (elevation angle direction) of 3d is specified as the reference panorama image Y.

  The controller 11 may use marker identification information (color information, shape information, position information, etc.) held in the storage unit 12 when identifying the markers 3a to 3d.

  Then, the control unit 11 of the editing apparatus 10 automatically specifies the panorama image included in the “content section W” based on the specified reference panorama image X and the reference panorama image Y (step S206).

  Specifically, the control unit 11 extracts a necessary frame included in the “content section W” from the panorama image group GX based on the reference panorama image X specified in step S205 (or deletes an unnecessary frame). Similarly, based on the reference panoramic image Y specified in step S205, necessary frames included in the “content section W” are extracted from the panoramic image group GY (or unnecessary frames are deleted).

  Thereby, the video editor or the like connects the frames of the panorama image group GX and the frames of the panorama image group GY included in the “content section W” automatically identified in step S206 to the “content section W”. A series of video contents composed of included panoramic images can be easily created.

  Here, when the video content is created by connecting the panorama image group GX and the panorama image group GY, it is desirable to perform image correction that matches the orientation of each panorama image group. In particular, as shown in FIG. 5, if panoramic images having different shooting directions are connected as they are, panoramic images having different shooting directions (different display coordinate systems) are mixed in the video content. When panoramic images with different shooting directions are mixed in the video content, the panoramic image display control processing and the like become complicated. Therefore, it is important to align the azimuths between panoramic images photographed in each photographing path in advance (align the display coordinate system).

  Therefore, the control unit 11 of the editing apparatus 10 may perform image correction for aligning the orientations between the panoramic images based on the reference panoramic image X and the reference panoramic image Y specified in step S205 (step S207).

  FIG. 7A is a diagram showing the specified reference panorama image X, and FIG. 7B is a diagram showing the specified reference panorama image Y. As shown in FIG. 7, since both the reference panorama image X and the reference panorama image Y are panorama images taken immediately above the reference position 5, the position coordinates in the vertical direction (elevation direction) of each marker in the panorama image. Match.

  On the other hand, the traveling direction D of the photographing device 4 on the photographing route 7-1 and the traveling direction E of the photographing device 4 on the photographing route 7-2 are different in azimuth direction by 90 °. The panoramic image Y is an image that is slid (rotated) by 90 ° in the horizontal direction (azimuth direction). For this reason, as shown in FIG. 7, the display positions of the markers displayed in the reference panorama image X and the reference panorama image Y are also displayed by being shifted by 90 ° in the horizontal direction (azimuth direction).

  Here, in step S207, the control unit 11 causes the reference panorama image X and the reference panorama image Y to face the same orientation (so that the reference panorama image X and the reference panorama image Y become the same display image). A slide amount in the horizontal direction (azimuth direction) of the reference panorama image X and / or the reference panorama image Y is determined, and the panorama image group GX and / or the panorama image group GY are slid (rotated) based on the slide amount. Image correction.

  Thereby, it is possible to align the display orientations between the panoramic images of the panoramic image group GX and the panoramic image group GY photographed by the plurality of photographing paths 7-1 and 7-2. Note that the slide amount may be automatically calculated by the control unit 11 or may be manually set by a video editor or the like.

  Further, when determining the slide amount of the panorama image, the control unit 11 may use markers displayed on the reference panorama image X and the reference panorama image Y as landmarks. In this case, the control unit 11 acquires the horizontal position information of the markers displayed in the reference panorama image X and the reference panorama image Y, and the reference panorama image so that the acquired horizontal position information of the markers matches. The slide amount in the horizontal direction (azimuth direction) of X and / or the reference panoramic image Y is determined. Then, based on the determined slide amount, the control unit 11 performs image correction by sliding (rotating) the panorama image group GX and / or the panorama image group GY.

  In this case, it is desirable to attach at least the marker used as the alignment mark in the real space in a manner that makes it easy to distinguish from other markers (the marker displayed in the reference panorama image X is any marker in the reference panorama image Y). Because it ’s necessary to know what to do). For example, when the slide amount is determined using the marker 3A of FIG. 5 as a mark, the marker 3A is preferably formed with a color, shape, or the like different from the other markers (markers 3B, 3C, 3D). The orientation alignment marker may be provided separately from markers (markers 3A to 3D in FIG. 5) for specifying the reference panoramic image. In this case as well, it is desirable to attach the orientation alignment marker on the real space in a manner that makes it easy to distinguish from other markers.

  The example in which the present invention is applied when shooting panoramic images on a plurality of shooting paths has been described above.

  The preferred embodiments of the imaging system and the like according to the present invention have been described with reference to the accompanying drawings, but the present invention is not limited to such examples. In addition, it is obvious that those skilled in the art can arrive at various changes or modifications within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. It is understood.

DESCRIPTION OF SYMBOLS 1 ..... Imaging system 3, 3A-3D .... Marker 3a-3d in real space .... Marker displayed on a panoramic image 4 .... Shooting device 5 .... Reference position 7 ..... Shooting Routes 9-1 to 9-5 ......... Panorama images P1 to P5 ......... Shooting position 10 ......... Editing device 11 ......... Control unit 12 ......... Storage unit 13 ......... Media input / output unit 14 ... ... Communication control unit 15 ... Input unit 16 ... Display unit 17 ... Peripheral device I / F unit 18 ......... Bus

Claims (5)

Two or more markers mounted at the same altitude;
An imaging device that captures a panoramic image including the marker from a position higher than the marker while traveling on a route on the marker, and an editing device,
The editing device
Obtaining means for obtaining vertical position information of each of the markers in the panoramic image;
Identifying means for identifying the panoramic image with the closest vertical position information of each of the acquired markers;
Imaging system comprising: a. The editing device
Means for acquiring horizontal position information of markers in a plurality of panoramic images having different shooting directions specified by the specifying means;
Means for determining a slide amount for aligning the orientations of the plurality of panoramic images based on position information in a horizontal direction of the markers in the plurality of panoramic images;
The imaging system according to claim 1 , further comprising: Storage means for storing a panoramic image including the marker photographed from a position higher than the marker while traveling on a route on two or more markers attached at the same altitude;
Obtaining means for obtaining vertical position information of each of the markers in the panoramic image;
Specifying means for specifying the panoramic image with the closest vertical position information of each acquired marker;
An editing apparatus comprising: Storing a panoramic image including the marker taken from a position higher than the marker while traveling on a route on two or more markers attached at the same altitude;
An obtaining step of obtaining vertical position information of each of the markers in the panoramic image;
A specifying step of specifying the panoramic image with the closest vertical position information of each acquired marker;
The editing method characterized by including. Computer
Storage means for storing a panoramic image including the marker taken from a position higher than the marker while traveling on a route on two or more markers attached at the same altitude,
Acquisition means for acquiring vertical position information of each of the markers in the panoramic image;
Specifying means for specifying the panoramic image with the closest vertical position information of each of the acquired markers;
A program characterized by functioning as

Images