JP2004241834A - Moving picture generating apparatus and method, moving picture transmission system, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system capable of easily acquiring a moving picture of a desired object from a photographed image. <P>SOLUTION: A moving picture generating apparatus sequentially generates an entire image comprising a plurality of unit images by changing a photographing direction to image a photographing range, records the generated entire image to a server 53, segments the unit image in the same photographing direction as that of the unit image designated by a user by each of the entire images recorded in the server 53 or the unit image on a photographing direction identified in response to a compared luminance level, and generates a moving picture on the basis of the segmented unit image. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a moving image generating apparatus and method, which cuts out one unit image for each captured panoramic image and generates a moving image based on the unit image, a moving image transmitting system, a program, and a recording medium.
[0002]
[Prior art]
Conventionally, a so-called video camera combining a CCD camera and a video tape recorder has been widely used for monitoring. In this video camera, a video signal received by a CCD camera at the time of imaging is recorded on a recording medium such as a magnetic tape. Then, by displaying the recorded video signal on a built-in electronic viewfinder or an external monitor device, a desired observation by the user can be realized.
[0003]
In such a video camera, when imaging a desired subject continuously, it is necessary to always make the shooting direction of the video camera follow a change in the position of the subject. Normally, when the photographing direction is manually controlled, vibration of the video camera due to so-called camera shake or the like appears in an image. For this reason, a video camera capable of suppressing the vibration of an image due to the camera shake and reliably automatically tracking a desired subject has been conventionally proposed (for example, see Patent Document 1).
[0004]
The video camera 200 has an image sensor 210 that forms an imaging signal representing a moving subject 202 in an image area 204, for example, as shown in FIG. The imaging signal formed by the image sensor 210 is supplied to the image motion vector detector 214. The image motion vector detector 214 detects, at the output 216, an image motion vector representing the motion of the subject 202 in the image area 204 based on the supplied imaging signal. An image motion vector as an error signal is supplied from the output terminal 216 to the camera position servo circuit 220, and the position of the image sensor 210 is adjusted so as to track the subject 202 according to the image motion vector.
[0005]
Further, in the related art, there has been proposed an image information distribution system or the like that can accurately realize such motion vector detection even in a situation where an image shows a specific motion (for example, see Patent Document 2). ).
[0006]
[Patent Document 1]
JP-A-6-217188
[Patent Document 2]
JP-A-11-331817
[0007]
[Problems to be solved by the invention]
However, in recent years, a system in which not only a single user but also a plurality of users can visually recognize a captured panoramic image via a network has been desired. In particular, for a wide range of situations such as a merchandise store such as a department store or a restricted area, multiple users need to manage them at the same time. In some cases, you may want to make sure.
[0008]
However, the above-described conventional video camera 200 and the like are suitable for the case where the user independently captures an automatically tracked subject and observes it. Therefore, simultaneous acquisition of moving images by a plurality of users, It was difficult to confirm the contents of the shooting after the fact.
[0009]
It is also necessary to reduce the user's effort to adjust the shooting direction according to the subject to be shot.Furthermore, by making it possible to extract moving images in a general image format, it is possible to check the shooting contents afterwards. It also needed to be easy to implement.
[0010]
Therefore, the present invention has been devised in view of the above-described problems, and a purpose thereof is to easily obtain a moving image of a desired subject from a captured image, It is an object of the present invention to provide a moving image generation apparatus and method, a moving image transmission system, a program, and a recording medium that can simultaneously acquire a moving image generated by the method.
[0011]
[Means for Solving the Problems]
In order to solve the above-described problems, a moving image generation device to which the present invention is applied includes an imaging unit configured to sequentially generate a panoramic image including a plurality of unit images by imaging a shooting range by changing a shooting direction, A recording unit for recording the generated panoramic image to the server, and a moving image generating unit for cutting out one unit image for each panoramic image recorded on the server and generating a moving image based on the cut out unit image.
[0012]
In addition, in order to solve the above-described problem, the moving image generation method to which the present invention is applied is configured to sequentially generate a panoramic image including a plurality of unit images by imaging a shooting range while changing a shooting direction. The obtained panoramic image is recorded in the server, one unit image is cut out for each panoramic image recorded in the server, and a moving image is generated based on the cut out unit image.
[0013]
Further, in order to solve the above-described problem, a moving image transmission system to which the present invention is applied is an imaging device that sequentially generates a panoramic image including a plurality of unit images by imaging a shooting range while changing a shooting direction. And a server for recording the generated panoramic image, an electronic device that cuts out one unit image for each panoramic image recorded on the server, and generates a moving image based on the cut out unit image, and an electronic device. A terminal device for reproducing a moving image received via the communication network.
[0014]
Further, in order to solve the above-described problem, the program to which the present invention is applied changes a photographing direction and images a photographing range, thereby sequentially generating a panoramic image including a plurality of unit images, and generating the panoramic image. Is recorded on the server, one unit image is cut out for each panoramic image recorded on the server, and a computer is caused to generate a moving image based on the cut out unit image.
[0015]
Further, in order to solve the above-described problem, the recording medium to which the present invention is applied changes the photographing direction and images a photographing range, thereby sequentially generating a panoramic image including a plurality of unit images, and generating the panoramic image. An image is recorded on a server, a unit image is cut out for each panoramic image recorded on the server, and a program for causing a computer to generate a moving image based on the cut out unit image is recorded.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. A moving image generation system 1 to which the present invention is applied, for example, as shown in FIG. 1, a camera unit 2 that captures a subject to generate an image signal, a microphone 12 that receives an audio signal, and a video signal transmitted from the camera unit 2. An image input / output unit 13 that applies predetermined processing to the received image signal, an audio input / output unit 14 that applies predetermined processing to the audio signal transmitted from the microphone 12, and at least the image input / output unit 13 and the audio input / output unit 14. An arithmetic processing unit 15 connected to generate a moving image based on the image signal; a server 53 connected to the arithmetic processing unit 15 for recording the image signal; and an operation unit 16 for a user to control the system 1 And a display 6 connected to the arithmetic processing unit 15 for displaying moving images and various information generated by the user, and a terminal device for allowing other users to execute applications. 9, includes a terminal display 10 which is connected to the terminal device 9, and a network 8 to transmit and receive various information to the terminal device 9.
[0017]
The camera unit 2 includes a pan tilter unit 3 and a camera unit 4 integrally formed. The pan-tilt unit 3 is configured as a turntable for freely changing a shooting direction for, for example, two axes of pan and tilt.
[0018]
The camera unit 4 is disposed on a turntable that constitutes the pan tilter unit 3, and captures an image of a subject while adjusting a shooting direction in a horizontal direction or a vertical direction according to control by the arithmetic processing unit 15 and the like. Further, the camera unit 4 sequentially changes the photographing angle of view in accordance with the control of the arithmetic processing unit 15 and the like, thereby enlarging or reducing the photographing magnification, and photographing the subject.
[0019]
The image input / output unit 13, the audio input / output unit 14, the arithmetic processing unit 15, the server and the operation unit 16 may be integrated with an electronic device 100 such as a personal computer (PC). According to the electronic device 100, the image signal transmitted from the camera unit 2 is recorded, and the recorded image signal is displayed to the user via the display 6. Further, when a desired image area or image position is designated by the user, the electronic device 100 controls so as to select an optimum one from the recorded image signals and display the selected signal on the display 6. The electronic device 100 also plays a role as a so-called central control device for controlling the entire network 8, and transmits an image and a sound in response to a request from another terminal device 9.
[0020]
The network 8 includes, for example, an Internet network connected to the electronic device 100 via a telephone line, and information such as ISDN (Integrated Services Digital Network) / B (broadband) -ISDN connected to a TA / modem. This is a public communication network that enables bidirectional transmission and reception. Incidentally, when the moving image generation system 1 is operated in a fixed small area, the network 8 may be configured by a LAN (Local Area Network). Further, the network 8 may be configured to transmit a moving image in addition to a still image. In such a case, based on the Internet Protocol (IP), a moving image including, for example, MPEG (Moving Picture Experts Group) data is continuously transmitted from one channel, and the still image data is transmitted from another channel for a certain period of time. It will be transmitted every time. In addition, a network server (not shown) may be connected to the network 8. The network server (not shown) manages, for example, Internet information, and transmits predetermined information stored therein in response to a request from the terminal device 9.
[0021]
The terminal device 9 is a PC that allows a user waiting at each home or company to acquire an image from the electronic device 100 via the network 8 and execute a desired process. By connecting the plurality of terminal devices 9 to the network 8, it is possible to simultaneously provide the application of the moving image generation system 1 to a plurality of users. The terminal device 9 displays an image acquired from the electronic device 100 on the terminal display 10. In addition, the terminal device 9 generates a request signal in accordance with a designation operation by the user, and transmits the request signal to the electronic device 100. Note that the block configuration of the terminal device 9 will be referred to the configuration of the electronic device 100 described later, and description thereof will be omitted.
[0022]
Next, the configurations of the camera unit 2 and the electronic device 100 in the moving image generation system 1 to which the present invention is applied will be described in detail.
[0023]
FIG. 2 is a configuration diagram of the camera unit 2 and the electronic device 100. In FIG. 2, each component of the camera unit 2 and the electronic device 100 is connected to a common controller bus 21.
[0024]
The pan / tilt unit 3 included in the camera unit 2 includes a tilt unit 3a and a pan unit 3b that control a turntable for changing an imaging direction. The camera unit 4 constituting the camera unit 2 includes a lens control unit 23 for mainly changing the angle of view of the lens unit 22 and an imaging unit 24 disposed at a position orthogonal to the optical axis of the lens unit 22. , An IEEE (Institute of Electrical and Electronics Engineers) 1394 interface 25 for transmitting an image signal generated by the imaging unit 24 to the image input / output unit 13, and a GPS (Global Positioning) for detecting the current position of the camera unit 2. (System) receiving unit 28 and a metadata generating unit 29 attached to the GPS receiving unit 28. Incidentally, the IEEE 1394 interface 25 may be replaced by Ethernet (registered trademark).
[0025]
The image input / output unit 13 includes a buffer memory 51 connected to the IEEE 1394 interface 25, and an encoder 52 connected to the buffer memory 51.
[0026]
The arithmetic processing unit 15 includes an image compression unit 54 for compressing an image read from the server 53, a graphic controller 55 connected to the server 53 and the image compression unit 54, and generating an image to be displayed on the display 6. , A CPU 56 for controlling each unit via the controller bus 21, a memory card 61 and a clock 62 respectively connected to the I / O port 58.
[0027]
The operation unit 16 includes a keyboard 59 and a mouse 60 for the user to specify a desired image area and an image position from an image displayed on the display 6.
[0028]
The Tilt unit 3a and the Pan unit 3b rotate a stepping motor configured as a drive source of the turntable based on a drive signal from the CPU 56. Thereby, the shooting direction of the camera unit 4 mounted on the turntable can be changed in the horizontal direction or the vertical direction.
[0029]
The lens control unit 23 performs an automatic aperture control operation and an automatic focus control operation on the lens unit 22 based on a drive signal from the CPU 56. The lens control unit 23 changes the angle of view of the subject with respect to the subject based on the drive signal. Thus, the camera unit 4 can also sequentially adjust the photographing magnification and image the subject.
[0030]
The imaging unit 24 is configured by a solid-state imaging device such as a CCD (Charge Coupled Device), and forms an image of a subject incident through the lens unit 22 on an imaging surface, generates an image signal by photoelectric conversion, This is transmitted to the IEEE 1394 interface 25.
[0031]
The GPS receiving unit 28 detects an installation location and a shooting direction of the camera unit 2 based on a signal transmitted by the GPS system. By providing the GPS receiving unit 28, particularly when a plurality of camera units 2 are installed, it is possible to control both shooting directions in conjunction with each other. The output signal from the GPS receiving unit 28 is supplied to a metadata generating unit 29, which generates position information such as latitude, longitude, azimuth, altitude, and metadata including time, various parameters, and the like based on the positioning result by GPS. Is done. The metadata generation unit 29 supplies the generated position information and metadata to the encoder 52. In the present invention, the configurations of the GPS receiving unit 28 and the metadata generating unit 29 may be omitted.
[0032]
The buffer memory 51 temporarily stores an image signal supplied from the IEEE 1394 interface 25 based on a control signal from the CPU 56. The image signal temporarily stored in the buffer memory 51 is supplied to an encoder 52, and compression-encoded based on a standard such as JPEG (Joint Photographic Experts Group). Incidentally, the encoder 52 may add position information and metadata supplied from the metadata generation unit 29 to the image signal to be compression-encoded. The encoder 52 outputs the compression-encoded image signal to the server 53 or the image compression unit 54. When the supplied image signal is not subjected to the compression encoding, the processing in the encoder 52 is omitted.
[0033]
The server 53 sequentially records the image signal output from the encoder 52 in association with position information and metadata. Incidentally, the server 53 may be replaced with, for example, a hard disk or a removable disk-shaped recording medium. The image signal recorded in the server 53 is read out to the image compression unit 54 and the graphic controller 55 under the control of the CPU 56. By controlling the image signal recorded on the server 53 to be recorded on the memory card 61, the user can transfer the captured image to another PC. Further, by controlling the image signal recorded in the server 53 to be recorded in the above-described network server (not shown), the server 53 can be replaced with a network server (not shown).
[0034]
The image compression unit 54 generates a compressed image or a thumbnail image for each of the JPEG image signals read from the server 53. The image compression unit 54 reads an image recorded in the server 53 and generates a moving image under the control of the CPU 56. Incidentally, as a compression method at the time of generating the moving image, it may be executed based on, for example, MPEG, Motion-JPEG, Motion-JPEG2000, or the like. The graphic controller 55 executes a painting process on the display 6 based on the image signal read from the server 53 or the image signal output from the image compression unit 54. The graphic controller 55 controls the contrast and brightness of the display 6 based on the control by the CPU 56.
[0035]
The CPU 56 controls a drive signal for driving the pan tilter unit 3 and the lens control unit 23 and controls each unit in the electronic device 100 when an image area and an image position are designated by the user via the keyboard 59 and the mouse 60. Is transmitted via the controller bus 21. Further, the CPU 56 receives a predetermined request signal from the terminal device 9, selects an optimal still image, a moving image, or various information recorded in the server 53, and transmits it to the terminal device 9. To control.
[0036]
The audio input / output unit 12 encodes an audio signal input from the microphone 12. The audio signal encoded by the audio input / output unit 12 is also supplied to the server 53 and recorded in the same manner as the image signal as described above.
[0037]
Next, an imaging operation in the moving image generation system 1 to which the present invention is applied will be described.
[0038]
FIG. 3 shows a case where the camera unit 2 captures an image within the shooting range indicated by the black frame at the shooting angle of view u. In order to image the entire imaging range at the imaging angle of view u, it is necessary to sequentially shift the imaging direction in the horizontal direction or the vertical direction. If the size of the photographing range can be represented by i × j times the size of a frame (hereinafter, referred to as a unit image) obtained by photographing at an arbitrary photographing angle of view u, at least i × j photographing directions are set. There is a need to. By bonding the i × j unit images captured at the shooting angle of view u, an entire image representing the entire shooting range can be synthesized.
[0039]
Here, the coordinates (M, N) of each unit image in the shooting range are 1, 2,... M, i in the horizontal direction from the left end, and 1, 2,. .., J, the CPU 56 transmits a drive signal to the Tilt unit 3a and the Pan unit 3b, so that the shooting direction of the camera unit 4 is first adjusted to the coordinates (1, 1) located at the upper left. Perform imaging. An image signal based on a unit image generated by imaging the coordinates (1, 1) is temporarily stored in the buffer memory 51, and is compression-coded by the encoder 52 based on the JPEG standard. The image signal is recorded in the server 53 at the same time as position information and metadata indicating the shooting direction and the like transmitted from the GPS 28 are added.
[0040]
Similarly, the CPU 56 transmits a drive signal to the Tilt unit 3a and the Pan unit 3b to shift the shooting direction of the camera unit 4 to the right by one image frame and to pick up an image in accordance with the coordinates (2, 1). Execute An image signal generated by imaging the coordinates (2, 1) is also recorded in the server 53. Under the control of the CPU 56, the camera unit 4 changes the shooting direction to the coordinates (3,1), (4,1)... (I, 1) in the horizontal direction, and executes the shooting.
[0041]
After terminating the imaging of the first column, the camera unit 4 adjusts the imaging direction to the coordinates (1, 2) of the second column and executes imaging under the control of the CPU 56, and then sequentially in the horizontal direction. The imaging is performed while shifting. When this operation is repeated to end the imaging up to the coordinates (i, j), the server 53 enters a state in which an image signal based on i × j unit images captured for each coordinate is recorded. FIG. 4 shows an example of an entire image sequentially recorded in the server 53 by executing the above-described imaging operation at predetermined intervals. That is, the server 53 sequentially records the whole image in which such a slight change in the situation in the photographing range is displayed at each time (t1, t2, t3, t4,...).
[0042]
Incidentally, the image signal based on each unit image recorded in the server 53 is sequentially read out by the image compressing section 54 and reduced so as to be adapted to the size of the display screen of the display 6. Each of the reduced unit images is displayed on the display 6 via the graphic controller 15. By displaying all of the i × j unit images recorded on the server 53 on the display 6, one whole image is synthesized. By executing the above-described imaging operation at regular intervals, it is possible to obtain an entire image indicating the latest situation of the imaging range.
[0043]
FIG. 5 shows an example in which the entire image synthesized by pasting the captured i × j unit images is displayed on the entire image display unit 70 of the display 6. The electronic device 100 may cause the whole image display unit 70 to display a boundary between the unit images constituting the whole image, or may display only a seamless whole image. In addition, the electronic device 100 causes the whole image display unit 70 to display one whole image photographed at a photographing angle of view capable of capturing the entire photographing range as an alternative to the panoramic whole image (panoramic image). Is also good.
[0044]
Incidentally, the display screen 45 is further provided with an enlarged image display section 71 for displaying an enlarged image obtained by enlarging the unit image. The enlarged image display unit 71 may enlarge and display one unit image specified by the user among the unit images constituting the entire image displayed on the entire image display unit 70. The moving images may be displayed sequentially for the unit image shooting directions. Thereby, the user can also check the situation of the shooting direction in the specified unit image in real time. Further, the enlarged image display section 71 may display a moving image generated by a method described later.
[0045]
Also, on the display screen 45, a WIDE button 72 for reducing the magnification and displaying the unit image displayed on the enlarged image display unit 71 and a ZOOM button 73 for enlarging and displaying the magnification are displayed. Is done. The display screen 45 includes a shooting direction control unit 75 for adjusting the shooting direction of the camera unit 4 in the horizontal and vertical directions, setting of various modes, and image signals based on unit images at desired addresses to the server. A setting button 76 or the like for recording the data is also displayed.
[0046]
Further, the user can designate a desired image area and image position on the whole image display unit 70 and the enlarged image display unit 71 using the keyboard 59 and the mouse 60. Note that the display units 70 and 71 may further display a sight line and a pointer for executing the above-described designation operation in conjunction with the movement of the mouse 60 or the like.
[0047]
Next, a method of generating a moving image by the moving image generation system 1 to which the present invention is applied will be described.
[0048]
FIG. 6 is a flowchart illustrating a procedure for generating a moving image based on unit images in the same shooting direction.
[0049]
First, in step S11, the user specifies one unit image from the whole image displayed on the whole image display unit 70 using the mouse 60 or the like. Next, the process proceeds to step S12, where the CPU 56 identifies the coordinates (M, N) of the specified unit image, that is, the shooting direction of the specified unit image.
[0050]
Next, the process proceeds to step S13, where the CPU 56 reads, from the server 53, a unit image in the identified shooting direction from the entire image taken at a certain time t1. The CPU 56 temporarily stores the read unit image in a buffer (not shown) or the like.
[0051]
Next, the process proceeds to step S14, where the CPU 56 focuses on the entire image at the next time. If the entire image is recorded in the server 53 in the order of the times t1, t2,... As shown in FIG. 4, the CPU 56 will pay attention to the entire image at the time t2.
[0052]
If the time t of the unit image read in the previous step S13 is the last time, the process proceeds to step S15, otherwise, the process proceeds to step S12. The last time is the time of a unit image constituting the last frame of the generated moving image.
[0053]
When the process returns to step S12, the unit images in the same shooting direction as the identified shooting direction are read out in step S13 and sequentially stored in a buffer (not shown). By repeatedly executing this, a buffer (not shown) is in a state where unit images in the same shooting direction are sequentially stored.
[0054]
FIG. 7 shows unit images read from the server 53 in the same shooting direction from time t1 to time t4. In FIG. 7, the photographing direction specified by the user in step S11 is “near the entrance of the church” displayed in the entire image, and the unit images are sequentially read out until time t4 in this photographing direction. This is shown in each case. It can be seen that as the time elapses, the moving subject (dove) approaches the camera unit 4 and is gradually displayed larger.
[0055]
When the process proceeds to step S15, the CPU 56 sequentially extracts the unit images stored in the buffer (not shown) from the earlier time, and creates a moving image in which the unit images are successively extracted. Incidentally, in the example shown in FIG. 7, a moving image in which the moving subject gradually approaches is created.
[0056]
Next, the process proceeds to step S16, and the created moving image is stored in, for example, the server 53 or the like. By storing the created moving image as a separate file from the still image such as the whole image or the unit image, only the moving image file can be transmitted to another terminal device 9 via the network 8 and stored in a recording medium. This can be distributed to a large number of customers.
[0057]
Also, by installing a playback tool in advance in the electronic device 100, the moving image file stored in step S16 may be played back by the electronic device 100.
[0058]
Next, a procedure for generating a moving image following a moving object in the entire image will be described with reference to FIG.
[0059]
First, in step S21, the user selects a moving subject for which a moving image is desired to be created from the whole image displayed on the whole image display unit 70. For example, as shown in FIG. 9, when selecting a moving subject moving from left to right, an image area in which the moving subject is displayed is designated using the mouse 60 or the like. Hereinafter, the image area specified in step S21 is referred to as a difference detection frame. Incidentally, in this step S21, not only the user but also the CPU 56 may select the moving subject.
[0060]
Next, the process proceeds to step S22, where the CPU 56 identifies the coordinates of the unit image including the specified image region, that is, the shooting direction of the unit image including the specified image region.
[0061]
Next, the process proceeds to step S23, where the CPU 56 reads, from the server 53, a unit image in the identified shooting direction from the entire image taken at a certain time t1. The CPU 56 temporarily stores the read unit image in a buffer (not shown) or the like.
[0062]
Next, the process proceeds to step S24, and the CPU 56 focuses on the entire image at the next time. If the entire image is recorded in the server 53 in the order of the times t1, t2,... As shown in FIG. 4, the CPU 56 will pay attention to the entire image at the time t2.
[0063]
Next, the process proceeds to step S25, where the CPU 56 compares the luminance level of the pixel in the difference detection frame at the previous time t1 with the luminance level of the entire image at the next time t2. At this time, as an alternative to the brightness level of the pixel in the difference detection frame, the brightness level of the pixel in the unit image including the difference detection frame may be set as the comparison target. As a result, when the luminance level of the entire image at the next time t2 changes based on the luminance level in the difference detection frame, the process proceeds to step S26, and otherwise, the process proceeds to step S28. Incidentally, the threshold value for determining whether or not the luminance level has changed may be freely set by the user or may be automatically set by the CPU 56 according to the situation of the subject or the photographing range.
[0064]
FIG. 9 is a diagram illustrating a case where a change occurs in the luminance level of the entire image at the next time t2. As shown in FIG. 9, when the moving subject displayed at the coordinates (4, 2) at time t1 moves to the coordinates (5, 2) at the time t2, the coordinate (4, 2) The luminance level changes for the unit image and the unit image at the coordinates (5, 2).
[0065]
When the process proceeds to step S26, the CPU 56 identifies, from the entire image at the next time t2, the unit image in which the difference is detected in the comparison of the brightness levels in step S25. Incidentally, since the difference in the brightness level is based on the brightness level in the difference detection frame, in other words, based on the brightness level corresponding to the image of the moving subject, the unit image in which such a difference is detected at time t2 This is an image in which the luminance level has changed as a result of the moving object being newly displayed. For this reason, in the example shown in FIG. 9, the CPU 56 identifies the unit image at the coordinates (5, 2), and obtains the unit image to obtain an image in which the moving subject is always displayed. It becomes possible.
[0066]
Next, in step S27, the CPU 56 identifies the shooting direction of the unit image identified in step S26 from the entire image at time t2.
[0067]
In step S28, the CPU 56 reads, from the server 53, a unit image in the identified shooting direction from the entire image taken at time t2, and stores the read unit image in a buffer (not shown). Incidentally, when the process directly proceeds from step S25 to step S28 without going through steps S26 and S27, the photographing direction of the unit image identified by the CPU 56 is the same as the photographing direction identified in step S22. It is. Therefore, at time t2, a unit image located in the same shooting direction as at time t1 is read.
[0068]
Note that if the time t of the unit image read in the previous step S23 is the last time, the process proceeds to step S29; otherwise, the process proceeds to step S24.
[0069]
When the process proceeds to step S24 again, the photographing direction of the unit image is identified by comparing the brightness level with the whole image at the next time in the same manner. Then, in step S28, the unit images in the same shooting direction as the identified shooting direction are read out and sequentially stored in a buffer (not shown). By repeatedly executing this, the buffer (not shown) is in a state in which unit images displaying the moving subject are sequentially stored.
[0070]
FIG. 10 shows a unit image read from the server 53 from time t1 to time t4. As the time elapses, a unit image can be acquired following a moving subject that moves from left to right. That is, by detecting the difference in the luminance level, the moving subject can be automatically tracked and the unit image can be obtained, so that it is as if the user manually operated a handy camera or the like to take an image. You can get a picture.
[0071]
When the process proceeds to step S29, the CPU 56 sequentially extracts the unit images stored in the buffer (not shown) from the earlier time, and creates a moving image in which the unit images are successively extracted. By the way, in the example shown in FIG. 10, a moving image focusing on the movement of the moving subject is generated.
[0072]
Next, the process proceeds to step S30, and the created moving image is stored in, for example, the server 53 or the like. By saving the created moving image as a separate file from a still image such as an entire image or a unit image, only the moving image file can be transmitted to another terminal device 9 via the network 8 or stored in a disk-shaped recording medium. It can then be distributed to many customers. In addition, the stored moving image file may be reproduced by installing a reproduction tool in the electronic device 100 in advance.
[0073]
In the above-described flow, the example in which the CPU 56 stores the unit image read out from the server 53 in a buffer (not shown) has been described. However, the present invention is not limited to such a case. For example, instead of reading the unit image from the server 53, Alternatively, the coordinates of the unit image to be read or the shooting direction may be stored, and the unit image may be read from the stored coordinates or the like when a moving image is created. Further, the server 53 may be used as a substitute for a buffer not shown.
[0074]
In addition, any other method of comparing the luminance levels may be applied. For example, the luminance levels may be compared for each block using block matching for the difference detection frame. The present invention is not limited to the case where the luminance levels are compared, and the respective primary color components of R, G, and B may be compared.
[0075]
As described above in detail, in the moving image generation system 1 to which the present invention is applied, the entire image including a plurality of unit images is sequentially generated by changing the shooting direction to capture the shooting range, and the generated entire image is generated. The image is recorded in the server 53, and a unit image in the same photographing direction as the unit image designated by the user for each whole image recorded in the server 53, or in the photographing direction identified according to the compared luminance level. A certain unit image is cut out, and a moving image is generated based on the cut out unit image.
[0076]
As a result, in the present invention, a moving image of a desired subject can be automatically acquired from a captured image, and it is easy for a user to perform post-analysis of the captured content.
[0077]
Note that the present invention is not limited to the above embodiment. For example, you may comprise as a program which makes a computer perform the above-mentioned operation | movement. Further, the program may be configured as a computer-readable recording medium on which such a program is recorded. This recording medium is a DVD-R (DVD-Recordable) capable of recording data or a DVD-RW (DVD) capable of rewriting recorded data in a DVD (Digital Versatile Disc) which is one of large-capacity optical disks. -Rewritable, DVD-RAM (DVD-Random Access Memory), etc., or CD (Compact Disk), CD-R (CD-Recordable), CD-RW (CD-Rewritable), etc., and even removable memory etc. May be. Incidentally, the recording medium may record not only the above-described program but also an entire image or a unit image recorded in the server 53 and / or a moving image file generated in the present system 1. Other users who have obtained this recording medium can execute the above program using their own PC or the like, and can also view recorded moving images.
[0078]
The moving image generation system 1 to which the present invention is applied is a moving image transmission system that transmits a moving image to the terminal device 9 by connecting a terminal device 9 having the same configuration as the electronic device 100 to a network 8. May be applied. Thereby, the user who operates the terminal device 9 can also acquire the moving image. Since the entire image is recorded once in the server 53, by connecting a plurality of terminal devices 9 to the network 8, a plurality of users can execute the above-described processing ex post facto. In such a case, by sequentially recording each captured whole image in a network server (not shown) connected to the network 8, the terminal device 9 itself accesses the network server (not shown) to read out each unit image, and A moving image may be generated. Thus, the burden on the electronic device 100 that has been simultaneously accessed from the plurality of terminal devices 9 can be reduced.
[0079]
【The invention's effect】
As described in detail above, in the present invention, by changing the shooting direction and shooting the shooting range, an entire image including a plurality of unit images is sequentially generated, the generated entire image is recorded in the server, and Cut out a unit image in the same shooting direction as the unit image specified by the user or a unit image in the shooting direction identified according to the compared brightness level for each whole image recorded in the server, and cut out the unit image A moving image is generated based on.
[0080]
Thus, according to the present invention, a moving image of a desired subject can be automatically acquired from a photographed image, so that the burden on the user can be reduced, and post-analysis of photographed contents can be easily performed. Can be. In addition, by connecting a plurality of terminal devices to the communication network, a plurality of users can simultaneously acquire moving images.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a moving image generation system to which the present invention has been applied.
FIG. 2 is a block diagram of a camera unit and an electronic device.
FIG. 3 is a diagram for describing a case where a camera unit captures an image within a shooting range indicated by a black frame at a shooting angle of view u.
FIG. 4 is a diagram illustrating an example of an entire image sequentially recorded in a server by executing an imaging operation at predetermined intervals.
FIG. 5 is a diagram illustrating a configuration example of a display screen on a display.
FIG. 6 is a flowchart illustrating a procedure for generating a moving image based on unit images in the same shooting direction.
FIG. 7 is a diagram showing unit images read from the server 53 in the same shooting direction from time t1 to time t4.
FIG. 8 is a flowchart illustrating a procedure for generating a moving image following a moving object in the entire image.
FIG. 9 is a diagram for describing a specific example of difference detection.
FIG. 10 is a diagram illustrating an example of generating a moving image by focusing on the movement of a subject.
FIG. 11 is a diagram for describing a configuration of a conventional video camera.
[Explanation of symbols]
Reference Signs List 1 moving image generation system, 2 camera unit, 3 pan tilter section, 4 camera section, 6 display, 8 network, 9 user terminal, 10 terminal display, 12 microphone, 13 image input / output section, 14 audio input / output section, 15 arithmetic processing Unit, 16 operation unit, 21 controller bus, 22 lens unit, 23 lens control unit, 24 imaging unit, 25 IEEE1394 interface, 28 GPS receiving unit, 29 metadata generation unit, 51 buffer memory, 52 encoder, 53 server, 54 image Compression unit, 55 graphic controller, 56 CPU, 58 I / O port, 59 keyboard, 60 mouse, 61 memory card, 62 clock

Claims (26)

An imaging unit configured to sequentially generate a panoramic image including a plurality of unit images by imaging a shooting range by changing a shooting direction;
Recording means for recording the generated panoramic image on a server,
A moving image generating apparatus comprising: a moving image generating unit that cuts out one unit image for each panoramic image recorded in the server and generates a moving image based on the cut out unit image. The moving image generating apparatus according to claim 1, further comprising a reproducing unit that reproduces the moving image generated by the moving image generating unit. Further comprising a designation unit for designating a unit image to be cut out from the panoramic image,
2. The moving image generating apparatus according to claim 1, wherein the moving image generating unit cuts out, from each panoramic image recorded in the server, a unit image located in the same shooting direction as the designated unit image. . The moving image generating means compares a luminance level between a panoramic image obtained by cutting out the unit image and another panoramic image captured after the panoramic image, and, based on a result of the comparison, calculates a luminance level from the other panoramic image. The moving image generating apparatus according to claim 1, wherein a unit image is cut out. The moving image generating means calculates a difference value of a luminance level between the panoramic image obtained by cutting out the unit image and the another panoramic image, and, based on the calculated difference value, calculates a unit image from the other panoramic image. The moving image generating apparatus according to claim 4, wherein the moving image is cut out. 5. The moving image generating apparatus according to claim 4, further comprising a designating unit for designating a unit image cut out from the panoramic image. An audio signal receiving unit that receives an audio signal in accordance with the panorama image imaging by the imaging unit,
The recording means records the audio signal received by the audio signal receiving means in the server,
The moving image generating apparatus according to claim 1, wherein the moving image generating means further reads an audio signal from the server according to the panoramic image obtained by cutting out the unit image, and adds the audio signal to a moving image to be generated. By changing the shooting direction and imaging the shooting range, a panoramic image composed of a plurality of unit images is sequentially generated,
Record the generated panoramic image on the server,
A moving image generating method, wherein one unit image is cut out for each panoramic image recorded in the server, and a moving image is generated based on the cut out unit images. Specify the unit image to be cut out from the above panoramic image,
9. The moving image generation method according to claim 8, wherein a unit image in the same shooting direction as the designated unit image is cut out from each panoramic image recorded in the server. Comparing the brightness level between the panoramic image obtained by cutting out the unit image and another panoramic image captured after the panoramic image, and cutting out the unit image from the other panoramic image according to the comparison result. 9. The moving image generating method according to claim 8, wherein A difference value of a luminance level is calculated between the panoramic image obtained by cutting out the unit image and the other panoramic image, and the unit image is cut out from the other panoramic image based on the calculated difference value. The moving image generation method according to claim 10. 11. The moving image generation method according to claim 10, wherein a unit image cut out from the panoramic image is specified. Receiving an audio signal in accordance with the imaging of the panoramic image,
Recording the received audio signal in the server,
9. The moving image generation method according to claim 8, further comprising reading an audio signal from the server according to the panoramic image obtained by cutting out the unit image, and adding the audio signal to a moving image to be generated. An imaging device that sequentially generates a panoramic image including a plurality of unit images by imaging a shooting range by changing a shooting direction;
A server for recording the generated panoramic image,
An electronic device that cuts out one unit image for each panoramic image recorded in the server and generates a moving image based on the cut out unit image,
A moving image transmission system, comprising: a terminal device for reproducing the moving image received from the electronic device via a communication network. The moving image transmission system according to claim 14, wherein a plurality of the terminal devices are provided. By changing the shooting direction and imaging the shooting range, a panoramic image composed of a plurality of unit images is sequentially generated,
Record the generated panoramic image on the server,
A program for causing a computer to cut out one unit image for each panoramic image recorded in the server and generate a moving image based on the cut out unit image. Specify the unit image to be cut out from the above panoramic image,
17. The program according to claim 16, wherein the program causes the computer to cut out, from each panoramic image recorded in the server, a unit image in the same shooting direction as the designated unit image. Comparing the brightness level between the panoramic image obtained by cutting out the unit image and another panoramic image captured after the panoramic image, and cutting out the unit image from the other panoramic image according to the comparison result. 17. The program according to claim 16, which is executed by a computer. A computer calculates a difference value of a luminance level between the panoramic image obtained by cutting out the unit image and the other panoramic image, and cuts out the unit image from the other panoramic image based on the calculated difference value. The program according to claim 18, which causes the program to execute. Receiving an audio signal in accordance with the imaging of the panoramic image,
Recording the received audio signal in the server,
17. The program according to claim 16, wherein the program causes the computer to read out an audio signal from the server in accordance with the panoramic image obtained by cutting out the unit image and to add the audio signal to a moving image to be generated. By changing the shooting direction and imaging the shooting range, a panoramic image composed of a plurality of unit images is sequentially generated,
Record the generated panoramic image on the server,
A computer-readable recording medium storing a program for causing a computer to cut out one unit image for each panoramic image recorded in the server and generate a moving image based on the cut-out unit image. Specify the unit image to be cut out from the above panoramic image,
22. The recording medium according to claim 21, which stores a program for causing a computer to cut out, from each panoramic image recorded in the server, a unit image located in the same shooting direction as the designated unit image. . Comparing the brightness level between the panoramic image obtained by cutting out the unit image and another panoramic image captured after the panoramic image, and cutting out the unit image from the other panoramic image according to the comparison result. 22. The recording medium according to claim 21, wherein the recording medium stores a program to be executed by a computer. A computer calculates a difference value of a luminance level between the panoramic image obtained by cutting out the unit image and the other panoramic image, and cuts out the unit image from the other panoramic image based on the calculated difference value. 24. The recording medium according to claim 23, wherein the recording medium records a program to be read. Receiving an audio signal in accordance with the imaging of the panoramic image,
Recording the received audio signal in the server,
22. The recording medium according to claim 21, further comprising: a computer-readable recording medium storing a program for causing a computer to read out an audio signal from the server according to a panoramic image obtained by cutting out the unit image and to add the audio signal to a moving image to be generated. 22. The recording medium according to claim 21, further comprising a computer readable recording the generated panoramic image and / or moving image.

Images