KR20070097273A - Device and method for forming panorama image, and computer readable recording medium recording the panorama image forming program - Google Patents

Abstract

An apparatus and a method for writing a panorama image and a recording medium capable of being read by a computer which records an image writing program are provided to use computer resources effectively by using operation data generated in the moving picture preparation. A panorama writing unit(S520) includes a device(a1) for calculating a moving vector for two or more images, a device(a2) for calculating space frequencies of one or more images, a still picture index writing device for writing an index on the basis of position data of an image, a memorizing device, and an initial panorama still picture writing device for synthesizing object images on the basis of the index and generating an initial panorama image. A moving vector monitoring unit(S540) detects that the calculated moving vector is higher than a threshold value. A space frequency monitoring unit(S542) detects that the calculated space frequency is lower than the threshold value. Differential still picture writing units(S550,S552) acquire the object images on the basis of the detected results of the moving vector monitoring unit and the space frequency monitoring unit, and write a different still picture.

Description

DEVICE AND METHOD FOR FORMING PANORAMA IMAGE, AND COMPUTER READABLE RECORDING MEDIUM RECORDING THE PANORAMA IMAGE FORMING PROGRAM}

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing an outline of a panoramic image creation system according to an embodiment of the present invention.

2 is a flowchart showing an operation of displaying a video of the system according to the embodiment of the present invention;

3 is a flowchart showing an operation during camera operation of the system according to the embodiment of the present invention.

4 is a flow chart showing operation when the system is not in use, of the system according to the embodiment of the present invention.

5 is a flowchart and a configuration diagram for creating a panoramic image from a moving image for explaining the details of the panoramic image creation according to the embodiment of the present invention.

FIG. 6 is a first schematic diagram showing processing performed when creating a moving image. FIG.

Fig. 7 is a second schematic diagram showing processing to be performed at the time of creating a moving image.

8 is a schematic diagram of a hardware configuration for executing a program according to the present invention.

9 is a schematic diagram showing a panorama image creation method according to the prior art.

<Explanation of symbols for main parts of the drawings>

100: imaging means 110: moving picture creation center portion

111: moving picture coding unit 112: panorama authoring unit

113: index creation unit 114: differential still image creation unit

115: imaging range distribution feedback means

116: Image drop detection and supplement means 120: Client terminal

122: Panoramic image displayed on the display means of the client terminal

124: moving picture displayed on the display means of the client terminal

500: imaging means 510: moving picture coding unit

520: Panorama creating unit 800: CPU

802: memory 804: input device

806: output device 808: external storage device

810: medium drive device 812: portable recording medium

814: network connection device 816: bus

900: moving image shooting camera 910: moving image creation device

920: client terminal 922: display means of the client terminal

924: Display of panoramic image 926: Display of moving image

930: Camera for still image shooting 932: Panoramic image

940: web server

The present invention relates to a panoramic image generating apparatus, a method and a program.

In a system for photographing a moving image, a wider range can be taken than in the case where the shooting area is changed and the camera is stopped and photographed by causing the camera for shooting to perform operations such as pan / tilt / zoom (PTZ) control. As described above, when capturing a moving image, it is sometimes necessary to obtain a still image (hereinafter referred to as a panoramic image) of the entire shooting range.

For example, when controlling a camera from a remote terminal in a system having a large video delay such as live video distribution using a WMV format, when controlling the camera in real time, the picture information of the control result reaches the terminal (operator). Since a time delay occurs, it is not immediately known how much the camera has moved, and camera control becomes difficult. Therefore, it is generally made to pan or zoom the camera on the indicated portion by preparing the panoramic image in advance and presenting it to the terminal and instructing the operator of the portion of the panoramic image. .

9 is a schematic diagram showing an example of a panorama image creation method according to the prior art. The panoramic image generating system includes a general moving image photographing camera 900, a moving image generating device 910, a client terminal 920, a general still image photographing camera 930, and a web server 940. When the moving image photographing camera 900 operates and photographs, digital moving images (eg, JPEG and MPEG) are created by the moving image encoding process in the moving image generating device 910. Here, with respect to the range that the moving image photographing camera 900 captures, a still image photograph is taken (for example, manually) by using a separate still image photographing camera 930 in advance and synthesized by software on a personal computer, for example. It is necessary to generate the panoramic image 932 by processing. The panoramic image 932 is uploaded to the web server 940 so that the client terminal 920 can download it.

The moving picture is sent from the moving picture creation device 910 to the client terminal 920 and displayed on the display means 922 of the client terminal 920. On the display means 922, the downloaded panoramic image 924 and the moving image 926 are displayed. On the panoramic image 924, the area currently illuminated by the moving image 926 is represented by, for example, a rectangular frame. Thus, the viewer viewing the moving image through the client terminal 920 can know the shooting position of the moving image camera.

There is also a method of creating a panoramic image directly from a moving image. In such a case, the moving amount of the camera is appropriately adjusted to record (record), and a portion overlapping with part or all of the frame image of the moving image is extracted. Appropriate calculation is performed on the overlapping portion, and a process of combining the images with each other as appropriate is performed. By executing this method, for example, in software on a computer, a panoramic image can be obtained.

As a method of producing a panoramic image from the above-mentioned moving picture, the method described in patent document 1 and patent document 2 is known, for example.

<Patent Document 1>

Japanese Patent Laid-Open No. 2000-244814

<Patent Document 2>

Japanese Patent Laid-Open No. 11-088811

However, since the conventional operation of photographing a panoramic image with a separate camera takes time, there is a problem that updating of the panoramic image tends to be neglected (especially when manually performed).

In addition, even in the method of creating a panoramic image from a moving image as described in the above-mentioned prior art document, there arises a problem that the generation timing of the panoramic image cannot be adjusted.

For example, like a surveillance camera application, a panoramic image may be created from the moving image while shooting a moving image for a certain time. In such a case, since the panoramic image also indicates an overview of the shooting range of the moving image, it is preferable that the panoramic image is the same image (weather, time, etc.) as the moving image (the image at the present time). For this reason, it is preferable that a panoramic image is updated at appropriate intervals.

However, in general, the process of creating a panoramic image from a moving image is a very expensive process. Therefore, if a continuous panoramic image is continuously created while shooting a moving image ("real-time panoramic image creation"), it is enormous. It consumes resources.

In addition, in the above-described conventional method, the update of the panoramic image is performed by adding new additional image data in order, thereby causing a problem that the data transfer amount becomes large.

In view of these problems, in particular, in the panorama image creation according to the distribution of moving images through a network, the conventional method of inputting such a huge resource only to the panorama image creation and transmitting new data each time is a problem of cost and the like. It's hard to say realistic.

Therefore, in view of the above problems, a new method has been required in which a panoramic image can be created from a moving image at an appropriate timing, and effective utilization of computer resources can be made.

In the embodiment of the present invention, the video encoding means for moving picture encoding the image signal obtained from the imaging means, the motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, Spatial frequency calculating means for calculating a spatial frequency for at least one frame image of the plurality of encoded frame images, still image index creation means for creating an index based on position data of the encoded plurality of frame images, and A panoramic image producing apparatus comprising: storage means for storing an index, and initial panoramic still image generating means for generating and storing an initial panoramic image in the storage means by arranging and combining the target images based on the index;

Motion vector monitoring means for detecting that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value;

Spatial frequency monitoring means for detecting that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value;

Based on the detection result of the said motion vector monitoring means or the detection result of the said spatial frequency monitoring means, the differential still image of the target frame image is created, and the said differential still image produced based on the said index is stored in the said storage means. A panoramic image producing apparatus is provided, comprising differential still image generating means for storing.

In addition, in the embodiment of the present invention, image encoding means for moving picture encoding the image signal obtained from the imaging means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images; A spatial frequency calculating means for calculating a spatial frequency for at least one frame image of the plurality of encoded frame images, a still image index creating means for creating an index based on position data of the encoded plurality of frame images; And storage means for storing the index, and initial panoramic still image creating means for generating and storing an initial panoramic image in the storage means by arranging and combining the target image based on the index. Create panoramic image As a method,

Detecting, by the motion vector monitoring means, that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value;

Detecting, by the spatial frequency monitoring means, that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value;

The difference still image generating means creates a difference still image of the frame image as a target based on a detection result of the motion vector monitoring means or a detection result of the spatial frequency monitoring means, and creates the difference based on the index. There is also provided a panorama image creation method comprising the step of storing a still image in said storage means.

In addition, in the embodiment of the present invention, image encoding means for moving picture encoding the image signal obtained from the imaging means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images; A spatial frequency calculating means for calculating a spatial frequency for at least one frame image of the plurality of encoded frame images, a still image index creating means for creating an index based on position data of the encoded plurality of frame images; And storage means for storing the index, and initial panoramic still image creating means for generating and storing an initial panoramic image in the storage means by arranging and combining the target image based on the index. Full panorama image making As a program,

Detecting, by the motion vector monitoring means, that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value;

Detecting, by the spatial frequency monitoring means, that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value;

The difference still image generating means creates a difference still image of the frame image as a target based on a detection result of the motion vector monitoring means or a detection result of the spatial frequency monitoring means, and creates the difference based on the index. There is also provided a panoramic image creation program comprising the step of storing a still image in said storage means.

EMBODIMENT OF THE INVENTION Hereinafter, although embodiment of this invention is described in detail based on drawing, this does not limit this invention at all.

[summary]

1 is a diagram showing an outline of a panoramic image creation system according to an embodiment of the present invention. The host (center side) of this panoramic image creation system has a moving picture creation center unit 110 to which the imaging means 100 is connected. For example, the client terminal 120 has a client side connected to the center side via a network or the like. . In addition, although the case where there is only one client terminal 120 is demonstrated here for simplicity, this invention is not limited to this and can be implemented using the some client terminal 120. FIG. In addition, although the case where there is only one imaging means 100 is demonstrated here for simplicity, this invention is not limited to this, It can also implement using several imaging means 100. FIG.

The imaging means 100 is not particularly limited as long as it can capture a moving image. For example, a general NTSC camera can be used.

The moving picture creation center unit 110 includes a moving picture coding unit 111 as a moving picture coding unit, a panorama producing unit 112, an index creating unit 113 as a still image index creating unit, and a differential still image creating unit as a differential still image creating unit. 114, an image capturing range distribution feedback means 115, an image drop detection / compensation means 116, and a storage means (not shown) are included.

An analog image signal such as NTSC photographed by the imaging means 100 is sent to the moving image coding unit 111 of the moving image creation center unit 110, and for example, a moving vector of a moving vector is used for image compression processing such as MPEG. Processing, discrete cosine transform (DCT), and the like are performed to create a digital moving image. The created moving image has a predetermined format such as MPEG, and usually consists of a plurality of frame images. The panorama creation unit 112 performs the process of joining the frame images together. The still image index creation means described later included in the panorama creation unit 112 is used to determine the position data (for example, coordinates in the entire range photographed by the image pickup means 100) included in these frame images, or at the point of time at which image capture starts. Indexes are created and stored in the storage means. Based on this index, the initial panoramic still image creation means described later generates an initial panoramic image, stores it in the storage means, and transmits it to the client terminal 120. On the other hand, the underlying moving picture is also sent to the client terminal 120. However, the image range covered by the moving image is only a part of the image range covered by the panoramic image.

The client terminal 120 receiving the initial panoramic image displays the moving image 124 and the panoramic image 122 on the display means. It is preferable that the panoramic image 122 has an indicator (for example, a frame in the panoramic image 122) indicating a range currently illuminated by the moving image 124. As time passes, the moving image 124 is updated. On the other hand, the timing at which the panoramic image 122 is updated is made in accordance with the judgment regarding the threshold value described later in detail. When it is determined that the panoramic image 122 has been updated, the differential still image generating unit 114 receives the frame image (for example, I frame of MPEG) from the moving image encoding unit 111 to create a differential still image. To the client terminal 120. The client terminal 120 which has received the differential still image can update the panoramic image by, for example, overwriting the differential still image to the initial panoramic image 122.

The client terminal 120 can also have imaging range adjustment means (not shown) for operating and controlling the imaging means 100. The imaging range adjustment means is connected with the imaging means 100 via a network, for example. From the client terminal 120, the imaging range 100 can be controlled by the imaging range adjustment means, and the imaging range can be changed. In another aspect of the present invention, the imaging range adjusting means may have only the center side. In still another aspect of the present invention, the imaging range adjusting means is provided by both the center side and the client side, and the imaging means 100 can be operated by appropriate priority control. The distribution of the imaging range controlled by the imaging range adjustment means can be collected by the imaging range distribution feedback means 115. The image capturing range distribution feedback means 115 extracts the image capturing range which has been taken for the longest time or the image capturing range which has been requested the most from the client terminal 120, and then "views it frequently" (angle with many requests). Can be stored in the storage means. Alternatively, the image capturing range distribution feedback means 115 may collect a distribution of a path traveled by the image capturing means 100 in response to a request, and may extract a “visible path”. With regard to the distribution of the imaging range thus obtained, it is also possible to feed this back to the imaging means 100 to optimize the operation of the imaging means. Such optimization includes, for example, adjusting the initial position of the image pickup means or the angle position of the image pickup means to the "view angle".

Preferably, the moving image creation center unit 110 includes image drop detection and supplementing means 116. Since the imaging means 100 does not necessarily perform the operation | movement which photographs all the ranges in order correctly normally, "missing" of a panoramic image may occur. If such "outs" remain, the poomsae becomes worse as a panoramic image (for example, the presence of "outs" is undesirable for surveillance camera applications). Therefore, it is preferable that the operation of "filling" such "extraction" into an appropriate image is performed. In some embodiments of the present invention, there is an image drop detection / compensation means 116. The image omission detection / compensation means 116 includes a shape and a size of the panoramic image 122 displayed on the display means of the client terminal 120 and a total of the imaging ranges (all ranges that the imaging means 100 can capture). By comparing the shape and the size, a process of compensating for &quot; missing &quot; of the panoramic image as described above (for example, the black beta portion of the panoramic image 122 of FIG. Preferably, the image omission detecting and supplementing means 116 can supplement the processing at the vacant time of the moving image creation center unit 110 to effectively utilize the resource.

2 is a flowchart showing an operation of displaying a video of the system according to the embodiment of the present invention. First, when shooting of a moving image is started by the image pickup means (step S200), the video data is sent to the moving image encoding means and created as a moving image (step S210). When creating this moving image, data of an operation (for example, a motion vector calculation or a DCT operation) and data of a frame image performed by the moving image encoding means are useful, and an initial panoramic image is created by the initial panoramic still image generating means. (Step S220), creation of the differential still image by the differential still image creation means (step S230), and creation of the index by the still image index creation means (step S240).

In step S220, in this embodiment, the initial panoramic image is sent to the client terminal only once from the start of the operation of the system. In another embodiment, an initial panoramic image is created and sent to a client terminal periodically at predetermined time intervals.

In step S230, the difference still image is created at an appropriate timing based on the result of the determination process described later. In this embodiment, the created differential still image is stocked in a storage means (not shown), and is periodically sent to the client terminal at a predetermined time interval. In another embodiment, the differential still image is sent to the client terminal as soon as it is created.

In step S240, an index is created from data obtained from the frame image of the moving image (for example, the position data as described above, the total shooting time, the number of shots, and the like). In this embodiment, an index is created using the frame image of a moving picture in order and stored in the storage means. In another embodiment, an index is created at the time of creation of an initial panoramic image, and is updated according to creation of subsequent difference still images. The index may be maintained only by the moving picture creation center unit, and the client terminal and the moving picture creation center unit may share and use the index.

The encoded moving picture data is directly received by the client terminal (step S250).

3 is a flowchart showing an operation during camera operation of the system according to the embodiment of the present invention. In this embodiment, the client terminal has an imaging range adjusting means. First, the client terminal sends an operation command via the imaging range adjusting means (step S300). In some embodiments, the index creation means in the moving picture creation center unit creates the index in response to this operation instruction (step S310). In another embodiment, the client terminal directly operates the imaging means so that the index is created and maintained in the client terminal or in the imaging means.

According to the contents of this index, the imaging means can be operated, for example, in the same manner as the PZT control (step S320), so that the operator of the client terminal can be directed to the imaging means at an angle desired to be viewed.

4 is a flowchart illustrating the operation when the system is not used, in the system according to the embodiment of the present invention. In this case, since the command for operating the imaging means is not really issued, the system is in the stopped state (step S400). In some embodiments, the moving picture creation center unit judges that it has entered the paused state, and refers to the index stored in the storage means (step S410) to perform the complementary processing of the empty area (step S420). In this complementary process, imaging is performed by operating the imaging means at an angle not captured so far to acquire an image for supplementation.

When the necessary complementary processing is finished, the imaging means is preferably set again by, for example, "an angle frequently viewed" (angle with the most index) or a predetermined initial position (step S430). This process can be performed automatically based on the index.

[The details of panorama image creation processing]

5 is a flowchart and a configuration diagram for creating a panoramic image from a moving image for explaining the details of the panoramic image creation according to the embodiment of the present invention. Here, the detail of the moving picture and the panoramic image creation center connected to the general imaging means 500 is shown. The moving picture and panorama image creation centers are largely classified into the moving picture coding unit 510 and the panorama producing unit 520. In addition, the path which transmits a moving image to a client terminal is abbreviate | omitted and not shown here.

The moving picture coding unit 510 obtains an analog picture signal from the imaging unit 500, for example, NTSC or the like, performs an encoding process to obtain a compressed digital moving picture such as MPEG. In this embodiment, the encoding process includes motion vector calculation (step S530), DCT operation (step S532), and I-Frame (Intra Frame) creation (step S534), and arithmetic coding as another process (step S536). , Inter-frame prediction, and intra prediction. On the other hand, in the following description, for convenience, the frame image can be independently restored as described. However, in the embodiment of the present invention, a video in any compressed format or an uncompressed video is handled. Note that it is possible to use a frame image that is actually compressed and cannot be restored alone.

In step S530, the motion vector calculation means performs calculation of the motion vector at the time of creating the moving image (a1). Using the calculation result of (a1), the motion vector monitoring means determines whether the motion vector is larger than a predetermined threshold as the determination process A (step S540). Here, for any two sets of frame images included in the original moving image, it is determined whether the motion vector is larger than a predetermined threshold. Here, the set of frame images to be targeted can be arbitrarily set. As a result of the determination process A, a frame image whose motion vector has exceeded the threshold (that is, the object reflected in the moving picture is largely moved) is extracted and sent to the panorama preparation unit 520 for the processing described later.

In step S532, the spatial frequency calculation means calculates the spatial frequency at the time of creating the moving image (a2). Using the calculation result of (a2), the spatial frequency monitoring means judges whether or not the spatial frequency is smaller than a predetermined threshold as the determination process B (step S542). In this embodiment, a discrete cosine transform (DCT) operation is used as the spatial frequency calculation. Here, it is determined whether or not the spatial frequency is smaller than a predetermined threshold value among frame images included in the original moving image. The frame image of interest here can be set arbitrarily. As a result of the determination process B, a frame image whose spatial frequency is below the threshold value (i.e., there is little change in color over the entire frame image) is extracted and sent to the panorama preparation unit 520 for processing to be described later. It becomes.

In step S534 according to this embodiment, I-Frame, which is a frame image that can be independently restored, is created (a3), and this is also sent to the panorama creating unit 520.

When each of the frame images obtained in the above (a1), (a2) and (a3) is sent to the panorama preparation unit 520, an index is created based on these frame images (step S550). In this embodiment, when either of the above-described determination processing A or B becomes true, a differential still image is created (step S552) (c). In another embodiment which concerns on this invention, it is possible to comprise so that the result of determination process A and B may be combined suitably, and it may be comprehensively determined whether creation of a differential still image is possible.

In this embodiment, the initial panoramic image is generated only once in the system operation (step S554) and stored in the storage means (step S556) (b). The initial panoramic image of (b) is transmitted only once to the client terminal (d). In another embodiment according to the present invention, the initial panoramic image is periodically created at predetermined time intervals and periodically transmitted to the client terminal.

In this embodiment, the differential still image is stored in the storage means included in the moving image / panorama image creation center and is periodically transmitted to the client terminal (e). In another embodiment according to the present invention, it is also possible to cause the differential still image to be transmitted to the client terminal every time it is created.

In this embodiment, when the free time is empty, the panoramic image creating unit 520 performs an automatic complement process using an index for effective utilization of system resources (step S558), and creates a differential still image for supplement, Send to the client terminal at an appropriate timing.

Fig. 6 is a schematic diagram showing a process performed when creating a moving image. Here, the above-described determination processing A will be described. In the following description, the motion vector calculation is performed for each part obtained by dividing the frame image into 6x6 parts, but the present invention is not limited to this, and the motion vector can be calculated by any method. In addition, notation is abbreviate | omitted about the frame image used as the reference which computes a motion vector.

(1) in FIG. 6 is an example when the motion vector is small. As a result of combining the motion vectors of the respective parts of the frame image, the result of the determination process A is false because the predetermined threshold is not exceeded. That is, in this frame image, there is little movement from the frame image as a reference, therefore, it is determined that creation of the panoramic image (differential still image) is unnecessary.

(2) in FIG. 6 is an example when the motion vector is large. As a result of combining the motion vectors of the respective parts of the frame image, since the predetermined threshold value has been exceeded, the result of the determination process A becomes true. That is, in this frame image, the movement from the frame image as a reference is large, and therefore, it is determined that creation of the panoramic image (differential still image) is necessary.

FIG. 7 is a schematic diagram showing processing performed at the time of creating a moving image for explaining the above-described determination processing B. FIG.

Above (3) of FIG. 7 is an example when the motion vector is small but there is a high frequency component in the spatial frequency. Since this frame image is said to be "rich in change", it is determined that a panorama image (differential still image) needs to be created even if the motion vector is small.

(4) in FIG. 7 is an example when the motion vector is large but there is no high frequency component in the spatial frequency. Since this frame image is said to reflect "lack of change", it is judged that creation of a panorama image (differential still image) is unnecessary even if the motion vector is large.

8 is a schematic diagram of a hardware configuration for executing a program according to the present invention, i.e., a host configuration, which includes a CPU 800, a memory 802, an input device 804, an output device 806, an external storage device 808, The medium drive device 810, the portable recording medium 812, and the network connection device 814 are connected by the bus 816. The program according to the present invention can be stored in the portable storage medium 812 driven by the external storage device 808 or the medium drive device 810. In addition, the hardware can be connected to the network through the network connection device 814, it is possible to input and output the program according to the present invention through the input device 804 and the output device 806.

By using the panoramic image generating apparatus, method and program according to the present invention, calculation data made at the time of creating a moving image can be utilized, and a panoramic image can be created from the moving image at an appropriate timing, thereby effectively utilizing computer resources and increasing data transfer amount. It becomes possible to suppress it.

(Book 1)

Moving picture encoding means for moving picture encoding the image signal obtained from the imaging means, moving vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, and at least one of the encoded plurality of frame images. Spatial frequency calculating means for calculating a spatial frequency with respect to one frame image, still image index creating means for creating an index based on position data of the encoded plurality of frame images, storage means for storing the index, A panoramic image producing apparatus comprising initial panoramic still image generating means for generating and storing an initial panoramic image in said storage means by arranging and combining said target image based on said index,

Motion vector monitoring means for detecting that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value;

Spatial frequency monitoring means for detecting that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value;

On the basis of the detection result of the motion vector monitoring means or the detection result of the spatial frequency monitoring means, a differential still image of a target frame image is created, and the difference still image generated based on the index is stored in the storage means. And a differential still image creating means for storing in the photograph.

(Supplementary Note 2)

Comparing the shape and size of a predetermined panoramic image frame with the shape and size of the initial panoramic image or the panorama image updated by the synthesis of the initial panoramic image and the differential still image, The apparatus according to Appendix 1, further comprising image drop detection and supplementing means for detecting and supplementing a missing portion in the panoramic image frame.

(Supplementary Note 3)

The apparatus according to Appendix 1, wherein the spatial frequency calculating means includes discrete cosine transform means.

(Appendix 4)

The apparatus according to any one of notes 1 to 3, further comprising network transmitting means for distributing the initial panoramic image and the difference still image to a client terminal via a network.

(Appendix 5)

The initial panoramic still image creating means distributes the initial panoramic still image to the client terminal through the network transmitting means, and the differential still image creating means delivers the differential still image to the client terminal through the network transmitting means. And the client terminal updates the panoramic image by overwriting the differential still image on the initial panoramic image.

(Supplementary Note 6)

Said client terminal having display means and an imaging range adjusting means for operating said imaging means to set an image capturing range arbitrarily displayed by said display means within a range of said predetermined panoramic image frame, said image capturing range adjusting means The apparatus according to Appendix 5, wherein the display means obtains a distribution of the selected imaging range displayed by the display means, feeds back to the control of the imaging means, and optimizes the operation of the imaging means.

(Appendix 7)

Image encoding means for moving picture encoding the image signal obtained from the image pickup means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, and at least one of the encoded plurality of frame images. Spatial frequency calculating means for calculating a spatial frequency with respect to one frame image, still image index creating means for creating an index based on position data of the encoded plurality of frame images, storage means for storing the index, A panorama image creation method executed in a computer system comprising an initial panoramic still image creation means for generating and storing an initial panoramic image in said storage means by arranging and combining said target image based on said index.

Detecting, by the motion vector monitoring means, that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value;

Detecting, by the spatial frequency monitoring means, that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value;

The difference still image generating means creates a difference still image of the frame image as a target based on a detection result of the motion vector monitoring means or a detection result of the spatial frequency monitoring means, and creates the difference based on the index. Storing a still image in said storage means.

(Appendix 8)

Image encoding means for moving picture encoding the image signal obtained from the image pickup means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, and at least one of the encoded plurality of frame images. Spatial frequency calculating means for calculating a spatial frequency with respect to one frame image, still image index creating means for creating an index based on position data of the encoded plurality of frame images, storage means for storing the index, A panoramic image creation program to be executed in a computer system having an initial panoramic still image creating means for arranging and combining the target images based on the index and generating and storing an initial panoramic image in the storage means.

Detecting, by the motion vector monitoring means, that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value;

Detecting, by the spatial frequency monitoring means, that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value;

The difference still image generating means creates a difference still image of the frame image as a target based on a detection result of the motion vector monitoring means or a detection result of the spatial frequency monitoring means, and creates the difference based on the index. And storing the still image in the storage means.

By using the panoramic image generating apparatus, method and program according to the present invention, the calculation data made during the creation of a moving image can be utilized, so that a panoramic image can be created from the moving image at an appropriate timing, thereby enabling effective utilization of computer resources. .

Claims (5)

Moving picture encoding means for moving picture encoding the image signal obtained from the image pickup means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, and at least one of the encoded plurality of frame images. Spatial frequency calculating means for calculating a spatial frequency with respect to one frame image, still image index creating means for creating an index based on position data of the encoded plurality of frame images, storage means for storing the index, A panoramic image producing apparatus comprising initial panoramic still image generating means for generating and storing an initial panoramic image in said storage means by arranging and combining said target image based on said index, Motion vector monitoring means for detecting that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value; Spatial frequency monitoring means for detecting that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value; Based on the detection result of the said motion vector monitoring means or the detection result of the said spatial frequency monitoring means, the differential still image of the target frame image is created, and the said differential still image produced based on the said index is stored in the said storage means. Differential still image creation means to save Panoramic image generating device comprising a. 2. The shape and size of a panoramic image frame according to claim 1, wherein the shape and size of the panoramic image updated by the composition and size of the initial panoramic image or by combining the initial panoramic image and the differential still image are determined. And a missing image detection / compensation means for detecting and supplementing a missing portion in said predetermined panoramic image frame. The panoramic image producing apparatus according to claim 1 or 2, further comprising network transmitting means for distributing the initial panoramic image and the differential still image to a client terminal via a network. Image encoding means for moving picture encoding the image signal obtained from the image pickup means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, and at least one of the encoded plurality of frame images. Spatial frequency calculating means for calculating a spatial frequency with respect to one frame image, still image index creating means for creating an index based on position data of the encoded plurality of frame images, storage means for storing the index, A panorama image creation method executed in a computer system comprising an initial panoramic still image creation means for generating and storing an initial panoramic image in said storage means by arranging and combining said target image based on said index. Detecting, by the motion vector monitoring means, that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value; Detecting, by the spatial frequency monitoring means, that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value; The difference still image generating means creates a difference still image of the frame image as a target based on a detection result of the motion vector monitoring means or a detection result of the spatial frequency monitoring means, and creates the difference based on the index. Storing a still image in the storage means Panorama image creation method comprising a. Image encoding means for moving picture encoding the image signal obtained from the image pickup means, motion vector calculating means for calculating a motion vector for at least two frame images of the plurality of encoded frame images, and at least one of the encoded plurality of frame images. Spatial frequency calculating means for calculating a spatial frequency with respect to one frame image, still image index creating means for creating an index based on position data of the encoded plurality of frame images, storage means for storing the index, Computer-readable recording of a panoramic image creation program to be executed in a computer system having an initial panoramic still image creating means for generating and storing an initial panoramic image in said storage means by arranging and combining said target image based on said index.As recording media, Detecting, by the motion vector monitoring means, that the motion vector calculated by the motion vector calculating means is larger than a predetermined threshold value; Detecting, by the spatial frequency monitoring means, that the spatial frequency calculated by the spatial frequency calculating means is smaller than a predetermined threshold value; The difference still image generating means creates a difference still image of the frame image as a target based on a detection result of the motion vector monitoring means or a detection result of the spatial frequency monitoring means, and creates the difference based on the index. Storing a still image in the storage means And a computer-readable recording medium having recorded thereon a panoramic image creating program.

Images