JP2008252915A - Control apparatus, control method and monitor system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a panorama image without the sense of incongruity, with respect to variations in the image photographic condition, or the like. <P>SOLUTION: A video signal from an imaging apparatus 10 as an imaging means is supplied to a video capture board 31 of a computer unit 30. At a lens block section 11, a magnification lens is driven by a driving instruction from a camera controller 41. The imaging apparatus 10 is installed on a turning universal head 50 and driven in respective directions in accordance by a driving instruction from a pan/tilt controller 43. A mode controller 42 controls the imaging apparatus 10 and the rotating universal head 50 in response to an absolute position driving request from the computer unit 30. The computer unit 30 comprises a storage device 32, an image processing device 33 and a control device 34, video signals for forming a panorama image are processed and controlled by these devices; and when the video signals are stored in the storage device 32, various imaging conditions, such as imaging time, exposure, white balance are stored additionally. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging, storage, processing, display, reproduction, transmission device, and recording medium suitable for use in, for example, remote viewing using a computer network.

  For the purpose of observing, monitoring, and guiding events, it is considered to view images from a remote location using, for example, a computer network (Internet). When viewing from such a remote place, a so-called panoramic image with a wide angle of view in the horizontal direction is suitable for viewing.

  Therefore, as a method for obtaining such a panoramic image, for example, a method using a wide-angle lens is known, but in general, a wide-angle lens is expensive, and a normal wide-angle lens is perpendicular. Since the angle of view is also widened, there is a possibility that the number of scanning lines in the panoramic image range is reduced and the resolution is lowered.

  On the other hand, a method has been implemented in which a panoramic image is captured in advance and stored in a storage device such as a hard disk, and the newly captured image is inserted and synthesized therein to form a panoramic image. However, in this method, only a pseudo panoramic image is formed, and a sense of incongruity occurs between a new image that has been fitted and synthesized and a stored image with respect to changes in imaging time, imaging conditions, and the like. There was a problem such as.

  This application has been made in view of the above points, and the problem to be solved is that the resolution of conventional devices may be reduced even if an expensive wide-angle lens is used. In addition, in the method of forming a panoramic image by fitting and synthesizing with a stored image, there is a problem that, for example, a sense of incongruity occurs with respect to changes in imaging time and imaging conditions.

  For this reason, in the present invention, the imaging range of the imaging means is changed, the video signals from such imaging means are combined to form a video signal having a wide angle of view, and the imaging time and / or imaging at that time The conditions are stored, and according to this, it is possible to obtain a panoramic image with a high resolution with a simple configuration that does not cause a sense of incongruity with respect to changes in imaging time, imaging conditions, and the like. .

  Further, in the present invention, the video signal of the other partial area is corrected according to the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Therefore, even when only a part of the panoramic image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in the imaging time, imaging conditions, and the like.

  According to the first aspect of the present invention, the imaging range of the imaging means is changed, and the video signals from such imaging means are combined to form a video signal having a wide angle of view. Alternatively, by storing the imaging conditions, it is possible to obtain a panoramic image with a high resolution with a simple configuration and without causing a sense of incongruity with respect to changes in imaging time, imaging conditions, and the like.

  As a result, with conventional devices, the resolution may be lowered even when an expensive wide-angle lens is used, and in the method of forming a panoramic image by fitting into a stored image, for example, the imaging time and the imaging conditions According to the present invention, these problems can be solved in the case where there is a problem such as an uncomfortable feeling with respect to the change of the above.

  According to the second to fifth aspects of the present invention, the panoramic image is obtained by correcting the video signal of the other partial area in accordance with the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  According to the invention of claim 6, by changing the imaging range of the imaging means, and recording the video signal from such imaging means, the imaging time and imaging conditions at the time of imaging, with a simple configuration, A panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like can be recorded with high resolution.

  According to the seventh to tenth aspects of the present invention, the panoramic image is obtained by correcting the video signal of the other partial area in accordance with the imaging condition of the video signal of the partial area designated on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  According to the invention of claim 11, by changing the imaging range of the imaging means, and recording the video signal from such imaging means and the imaging time and imaging conditions at the time of imaging, with a simple configuration, A panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like can be formed with high resolution.

  According to the invention of claims 12 to 15, a panoramic image is obtained by correcting the video signal of the other partial area according to the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  According to the invention of claim 16, the imaging range of the imaging means is changed, and the video signals from such imaging means are synthesized to form a video signal having a wide angle of view, and this video signal is used as the display means. By displaying, a panoramic image that does not cause a sense of incongruity with changes in imaging time, imaging conditions, and the like can be viewed with high resolution with a simple configuration.

  According to the invention of claims 17 to 20, the panoramic image is obtained by correcting the video signal of the other partial area in accordance with the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  According to the invention of claim 21, the video signal in which the imaging range is sequentially changed, the imaging time and imaging conditions at the time of imaging are reproduced, and the reproduced video signal is synthesized to form a video signal having a wide angle of view. By doing so, a panoramic image that does not cause a sense of incongruity with changes in imaging time, imaging conditions, etc. can be reproduced with high resolution with a simple configuration.

  According to the invention of claims 22 to 25, the panoramic image is obtained by correcting the video signal of the other partial area in accordance with the imaging condition of the video signal of the partial area designated on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  According to the invention of claim 26, by synthesizing the video signals picked up by the image pickup means installed at a remote place to form a video signal having a wide angle of view, the weather, time, etc. can be obtained with a simple configuration. A panoramic image that does not cause a sense of incongruity with changes can be viewed with high resolution.

  According to the invention of claims 27 to 30, the panoramic image is obtained by correcting the video signal of the other partial area according to the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  According to the invention of claim 31, the recording range of the image pickup means is changed, and the recording medium is reproduced by recording the video signal from the image pickup means and the image pickup time and the image pickup condition at the time of the image pickup. By connecting them, it is possible to obtain a panoramic image with high resolution with a simple configuration and without causing a sense of incongruity with respect to changes in imaging time, imaging conditions, and the like.

  According to the invention of claims 32 to 35, the panoramic image is obtained by correcting the video signal of the other partial area according to the imaging condition of the video signal of the partial area designated on the display means for displaying the formed video signal. Even when only a part of the image is changed, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  Furthermore, according to the present invention, it is possible to quickly and easily search for a video signal imaged using information such as stored imaging time and imaging conditions.

  That is, the present invention relates to an imaging unit that images a subject and generates a video signal, a control unit that controls the operation of the imaging unit, a storage unit that stores the captured video signal, and an imaged and stored video signal. And processing means for forming a video signal having an angle of view wider than one imaging range of the imaging means, and storing the imaging time and / or imaging conditions when imaging with the imaging means together with the video signal in the storage means It is something to remember.

  The present invention also provides an image pickup means for picking up an image of a subject and generating a video signal, a control means for controlling the operation of the image pickup means, a storage means for storing the picked-up video signal, and an image pickup and stored video signal. And processing means for forming a video signal having a wider angle of view than one imaging range of the imaging means, and storing the imaging time and / or imaging conditions when imaging with the imaging means together with the video signal It is.

  The present invention also provides an image pickup means for picking up a subject to generate a video signal, a control means for controlling the operation of the image pickup means, an image pickup time and / or an image pickup time when the image pickup means is picked up together with the picked up video signal. Storage means for storing the conditions, and connecting the captured and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means.

  The present invention also provides an image pickup means for picking up an image of a subject and generating a video signal, a control means for controlling the operation of the image pickup means, a storage means for storing the picked-up video signal, and an image pickup and stored video signal. An imaging time when imaging is performed by the imaging unit, which includes a processing unit that forms a video signal having a wider angle of view than one imaging range of the imaging unit and a display unit that displays the formed video signal In addition, the imaging condition is stored in the storage unit together with the video signal, and the video signal captured by the imaging unit is connected to the partial area of the video signal instructed on the display unit to form the video signal.

  The present invention also provides a reproducing unit for reproducing a recording medium on which an imaging time and / or an imaging condition at which an imaging unit has been recorded together with a video signal captured by controlling the operation of the imaging unit, and a reproducing unit. Are combined to form a video signal having a wider angle of view than one imaging range of the imaging means.

  The present invention also provides an image pickup means for picking up a subject to generate a video signal, a control means for controlling the operation of the image pickup means, an image pickup time and / or an image pickup time when the image pickup means is picked up together with the picked up video signal. Storage means for storing conditions, processing means for connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means, and a video signal formed by the processing means It comprises transmission means for transmitting.

  The present invention also provides an image pickup means for picking up an image of a subject to generate a video signal, a control means for controlling the operation of the image pickup means, and a video signal that is picked up and stored to be wider than one image pickup range of the image pickup means. Using the processing means for forming the video signal having the angle of view and the imaging means for generating the video signal, the imaging time and / or the imaging conditions when the imaging means performs imaging together with the video signal captured by the imaging means are recorded. It has been made.

  Furthermore, according to the present invention, the imaging conditions include one or more information of exposure, white balance, focus, magnification, size, and position coordinates, and imaging is performed by connecting video signals that are close to the imaging time and / or imaging conditions. A video signal having an angle of view wider than one imaging range of the means is formed.

  Further, according to the present invention, the imaging condition includes one or more information of exposure, white balance, focus, magnification, size, and position coordinates, and the imaged and stored video signals are connected to one of the imaging means. When a video signal having an angle of view wider than the imaging range is formed, the video signal is corrected according to the imaging conditions.

  Further, according to the present invention, the imaging conditions include one or more pieces of information of exposure, white balance, focus, magnification, size, and position coordinates, and an image specified on a display means for displaying the formed video signal The video signal picked up by the image pickup means is connected to a partial area of the signal to form a video signal.

  Further, according to the present invention, the imaging condition includes one or more information of exposure, white balance, focus, magnification, size, and position coordinates, and the imaged and stored video signals are connected to one of the imaging means. Correcting video signals of other partial areas in accordance with the imaging conditions of the video signal of the partial area indicated on the display means for displaying the video signal formed when forming the video signal having a wider angle of view than the imaging range It is.

  Hereinafter, the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an example of an apparatus to which the present invention is applied.

  In FIG. 1, an image pickup apparatus 10 as an image pickup means shows a video camera as a whole, and image pickup light reaching from a subject (not shown) passes through a lens, a diaphragm, etc. of a lens block unit 11 to a solid-state image pickup device, for example, a CCD 12. Imaged. An arbitrary visual field image is converted into a video signal by the CCD 12, and is then supplied to a signal separation (sample hold: SH) / automatic gain control (AGC) device 13.

  In the signal separation / automatic gain control device 13, the video output signal from the CCD 12 is sampled and held according to a predetermined pixel clock, for example, and automatic gain control is performed so as to have a predetermined gain by an auto iris (AE) control signal, for example. Done. The video output signal thus obtained is supplied to the signal processing device 15 via the A / D conversion device 14.

  In the signal processing device 15, the input signal is converted into each signal such as luminance, color difference, and video signal, and is output as a video signal to a video capture board 31 of a computer device 30, which will be described later, such as a television monitor 20 or a personal computer. Supplied. As a result, the video signals supplied to the video capture board 31 are sequentially taken into the computer device 30.

  The lens block unit 11 of the imaging device 10 is a zoom lens that can change the angle of view for imaging by driving a zoom lens. A motor, for example, a zooming stepping motor 16 is rotated by a drive command from a camera controller 41, which will be described later, and the above-described zoom lens is driven.

  The camera controller 41 always performs lens control (focus, zoom), exposure control (aperture, gain, electronic shutter speed), white balance control, image quality control, and the like of the imaging apparatus 10 and also interfaces with the mode controller 42. . For example, in response to a zoom drive request, a control signal for driving the zoom lens to the required position is output to a drive circuit (not shown) of the motor 16, and the zoom lens position information is always in the mode. Communicated to the controller 42.

  Further, the imaging device 10 is installed on a device having a degree of freedom in two-axis rotational directions, such as pan (left and right) and tilt (up and down), for example, as a driving unit, for example, a rotating pan head 50. In accordance with a drive command from a pantilter controller 43 described later, a motor, for example, a panning stepping motor 51 and a tilting stepping motor 52 are rotated, and the rotary head 50 is driven in each direction.

  In the pantilter controller 43, an interface with the mode controller 42 is performed. For example, in response to a drive request in each direction of pan and tilt, a control signal for driving the rotary head 50 in the requested direction is output to a drive circuit (not shown) of the motors 51 and 52 and rotated. The position information of the pan head 50 in, for example, pan and tilt directions is always communicated to the mode controller 42.

  Further, the mode controller 42 controls the entire system in accordance with the internal state of the imaging device 10 and the rotary pan head 50 and interface information from outside the device. The mode controller 42 is connected to the computer apparatus 30 by, for example, RS232C, and controls the absolute position drive request from the computer apparatus 30.

  That is, the mode controller 42 drives to drive the motor 16 of the lens block unit 11 and the motors 51 and 52 of the rotary head 50 in order to image the position by the imaging device 10 in response to the absolute position drive request from the computer device 30. The command is distributed to the camera controller 41 and the pantilter controller 43.

  At the same time, various shooting conditions such as position information, lens control information, exposure control information, white balance control information, and image quality control information from these controllers 41 and 43 are transmitted to the computer apparatus 30 through the mode controller 42.

  Further, the computer device 30 is provided with a storage device 32, an image processing device 33, and a control device 34. With these devices, the computer device 30 can control, for example, video signal processing means for forming a panoramic image, and control. Used as a means. A video signal formed by the computer device 30 is supplied to the computer monitor 60 to serve as a user interface, and conditions for internal processing and the like are determined using an image graphically displayed on the tube surface.

  Further, in the above-described video capture board 31, the captured video signal is displayed on the monitor 60 with an arbitrary quality, and an arbitrary image format (for example, a bitmap format, a JPEG still image, a moving image, etc.) is arbitrarily determined by the capture signal. Can be captured and stored in a storage device 32 such as a hard disk.

  At this time, the imaging time when the above-described video signal is captured and the above-described various shooting conditions provided from the mode controller 42 are stored in the storage device 32 together with the captured video signal.

  Further, a control algorithm executed by the computer device 30 will be described with reference to FIG.

  When the program is started in FIG. 2, first, in step [1], a process of setting the display areas of the captured video and the panoramic image on the monitor 60 is performed. Thus, on the monitor 60, for example, as shown in FIG. 1, a captured image display area 61 and a panoramic image display area 62 are set. Then, for example, a video signal input from the imaging device 10 is displayed in the display area 61.

  Furthermore, in step [2], for example, a timer for periodically communicating with the mode controller 42 is set. When these initial settings are completed, in step [3], a transition is made to a waiting state for various events to be generated. The generated events are a timer event set in step [2] described above (details will be described later) and a panorama image creation request event described later.

  Details of the timer event will be described with reference to FIG. Note that this timer event is an event that is generated in order to periodically communicate with the mode controller 42, for example.

  When this event occurs, it is determined in step [11] whether or not the communication port setting has been completed. Since the communication port setting has not been completed only for the first time, the communication port establishment process is performed in step [12]. Is done. In the above-described device, for example, an RS232C port on the computer device 30 is opened as a communication port.

  Further, in the second and subsequent timer events, the presence / absence of received data is determined in step [13], and when there is received data, the received data is analyzed in step [14]. That is, in this step [14], for example, the position information of the zoom lens of the lens block unit 11 and the pan and tilt position information of the rotary head 50 are acquired by communication with the mode controller 42, for example. From these pieces of information, zoom magnification information and pan / tilt angle information are extracted.

  Next, in step [15], it is confirmed whether or not there is a data transfer request (Flag so). When there is a transfer request (True), since the data has already been stored in the transmission buffer, the transfer processing is immediately performed using the data in this transmission buffer in Step [16], and there is no transfer request ( Flag so = False).

  If there is no transfer request (False) in step [15], the value of the internal counter (req cnt) is determined in steps [17] [18] [19]. Here, when (req cnt = 0), the process proceeds from step [17] to step [20] to transfer the communication data of the absolute position drive request for driving the motors 51 and 52 of the rotary head 50. Is done.

  When (req cnt = 1), the process proceeds from step [18] to step [21], and communication data transfer processing for absolute position drive request for driving the motor 16 of the lens block unit 11 is performed. . When (req cnt = 2), the process proceeds from step [19] to step [22], and a panoramic image creation process to be described later is performed.

  After steps [20] and [21] are executed, a process of counting up (req cnt) is performed in step [23]. After step [22] is executed, a process of returning (req cnt) to 0 is performed in step [24]. As a result, the processes of these steps [20], [21] and [22] are executed in a circulating manner for each event process.

  Further, the above panoramic image creation process is performed as described below. That is, this panorama image creation process is executed in step [22] of the above-described timer event by setting a panorama creation request (Flag pa) in step [30] shown in FIG. 3, for example. .

  Details of the panorama image creation process will be described with reference to FIG. In this example, the processing procedure is performed according to an internal counter (Pano cnt).

  Therefore, first, in steps [31] and [32], the value of the internal counter (Pano cnt) is determined. Here, when (Pano cnt = 1), the process proceeds from step [31] to step [33], and the motor 16 of the lens block unit 11 is driven so as to set the angle of view of the imaging device 10 to the widest angle. Along with the process, a process of counting up (Pano cnt) is performed. In addition, by setting the angle of view to the widest angle, a panoramic image is formed with a smaller number of image acquisitions.

  When (Pano cnt = 2), the process proceeds from step [32] to step [34], and the imaging range of the imaging device 10 is set to a position where the first image acquisition is performed in the panorama image creation process. In addition, processing for driving the motors 51 and 52 of the rotary head 50 is performed, and processing for counting up (Pano cnt) is performed.

  Further, when (Pano cnt) is 3 or more, it is confirmed in steps [35] and [36] that the zoom position and the imaging range set in steps [33] and [34] are moved. When these set positions are reached, the first image acquisition is performed in step [37]. Further, in step [38], processing for driving the motors 51 and 52 of the rotary head 50 to the next image acquisition position is performed.

  In step [39], the previously acquired image is compressed in the horizontal and vertical directions, and the panorama image creation status is displayed in the display area 62 of the monitor 60 in step [40]. Further, in step [41], a process of counting up (Pano cnt) is performed. In step [42], it is determined whether or not the panoramic image is completed. When the panorama image is not completed, the above operation is repeated.

  When the panorama image is completed in step [42], the panorama creation request (Flag pa) is not issued (Flag pa = False) in step [43], and the panorama image completed in step [44] is displayed. Saved and the operation is finished.

  In this manner, for example, images acquired while the imaging range of the imaging device 10 is sequentially moved by the rotating pan head 50 are connected while being compressed, and a panoramic image is formed. Here, the rotary head 50 is operated so as to always enter the commanded target position from the same direction. As a result, for example, it is possible to correct the shift of the linked image due to play of the drive system mechanism (gear backlash, etc.) that may occur when the images are linked.

  In this apparatus, since the original video signals connected in the above-described panoramic image formation are not constant in imaging conditions, if they are connected as they are, the exposure, white balance, etc. will be uneven depending on the part, and the panoramic image with uniform image quality Can't get. Therefore, in the above-described apparatus, the image processing apparatus 33 of the computer apparatus 30 performs image processing to obtain a panoramic image with uniform image quality.

  That is, FIG. 6 shows an example of image processing for that purpose. In FIG. 6, each imaging signal forming a panoramic image is stored with its imaging time, imaging information such as position information lens control information, exposure control information, white balance control information, and image quality control information. ing.

  Therefore, for example, average luminance information is extracted from each connected video signal as shown in FIG. 6A, weighted according to each imaging condition, and then compared with a preset reference luminance. The pixel correction amount is determined (B in FIG. 6). By executing this correction, the overall exposure of the panoramic image is made uniform.

  After that, for example, the integration of red (R), green (G), and blue (B) of pixels having a certain luminance or higher is obtained, the amount of deviation from white is calculated, and white balance adjustment ( Perform C) of FIG. Thereby, for example, a panoramic image with uniform image quality as shown in FIG. 6D can be obtained.

  Therefore, in this apparatus, the imaging range of the imaging means is changed, and the video signals from such imaging means are combined to form a video signal having a wide angle of view, and the imaging time and / or imaging conditions at that time are set. By storing, a panoramic image that does not cause a sense of incongruity with changes in imaging time, imaging conditions, and the like can be obtained with high resolution with a simple configuration.

  As a result, with conventional devices, the resolution may be lowered even when an expensive wide-angle lens is used, and in the method of forming a panoramic image by fitting into a stored image, for example, the imaging time and the imaging conditions According to the present invention, these problems can be solved in the case where there is a problem such as a sense of incongruity with respect to the change of.

  Further, in the above-described apparatus, the created panorama image is displayed in the panorama image display area 62 of the computer monitor 60 described above. In this case, an operation button 63 as shown in FIG. 7, for example, is displayed on the monitor 60 together with the display areas 61 and 62 described above.

  On such a computer monitor 60, when the user designates an arbitrary pixel in the display area 62 by an input means such as a pointing device (mouse), the designated pixel is set with the upper left corner of the display area 62 as an origin, for example. Are obtained, and the obtained coordinates of the designated point are transferred to the computer device 30.

  Therefore, the computer device 30 determines an imaging area including the pixel at the designated point from the transferred coordinates, and transmits the coordinates of the imaging area and a movement command to the mode controller 42. As a result, the mode controller 42 analyzes the data received in step [14] of the timer event described above, drives the rotary head 50 in accordance with the analyzed data, and captures the target including the instructed pixel. Control is performed so as to image the region.

  That is, for example, while confirming a panoramic image on the computer monitor 60, the user uses an input means such as a pointing device (mouse) to instruct a place to be observed in detail. In response to this instruction, the imaging apparatus 10 is directed in the direction instructed by the rotary head 50, and the current video of the location is displayed in the display area 61 of the captured video.

  Then, the user observes the image displayed in the display area 61 of the picked-up video and presses the recording start button of the operation button 63 on the monitor 60 with a pointing device (mouse) or the like when the video to be recorded is obtained. Necessary images can be stored (saved) in the storage device 32 such as a hard disk.

  The stored image may be a still image at the moment when the user presses the recording start button, or a moving image having a desired length can be stored by operating any operation button 63. Or when there is no room in the capacity of the storage device 32, an intermittent video using a still image may be stored. Also, when storing (saving) these still images, moving images, and intermittent moving images, the data of the above-described imaging time and imaging conditions are attached and stored.

  Therefore, a panoramic image is created using the video stored (saved) in this manner as follows.

  That is, in the display on the computer monitor 60 as shown in FIG. 7 described above, when the user designates an arbitrary pixel in the display area 62 by an input means such as a pointing device (mouse), for example, the upper left corner of the display area 62 The coordinates of the designated pixel are obtained with the origin as the origin, and the obtained coordinates of the designated point are transferred to the computer device 30.

  Therefore, the computer device 30 determines an imaging area including the pixel of the designated point from the transferred coordinates, and when a partial image including the imaging area is stored, the area is colored on the panoramic image, for example. The image is displayed by surrounding it with a boundary line and the image stored there is displayed. As a result, a stored image in a desired area can be viewed on the panoramic image.

  In this case, since the image is compressed and displayed on the panoramic image, the image can be enlarged and displayed outside the display area 62 when the user wants to see details. In this case, for example, the captured image can be displayed on the display area 61 or can be enlarged and displayed on the entire screen of the monitor 60.

  If a plurality of images are stored in the same designated area, a desired image can be searched by operating the operation button 63 on the monitor 60, for example. In this case, the image search can be performed by designating the imaging time, the imaging conditions, etc. stored together with the above video signal.

  Further, when viewing a stored image on the above-described panoramic image, if an image of a specified area is simply displayed superimposed on the panoramic image, the image becomes unnatural when there is a difference in shooting conditions. In such a case, a panoramic image with uniform image quality can be obtained by performing the same processing as shown in FIG.

  Therefore, in this apparatus, only a part of the panoramic image is changed by correcting the video signal of the other partial area according to the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Even in this case, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  As a result, with conventional devices, there is a risk that the resolution will be lowered even if an expensive wide-angle lens is used, and for example, a method of forming a panoramic image by inserting and synthesizing the video signal of the designated partial area. For example, according to the present invention, these problems can be solved when there is a problem such as an uncomfortable feeling with respect to changes in imaging time or imaging conditions.

  In the above-described apparatus, the panorama creation request (Flag pa) in step [30] in FIG. 3 is set by, for example, a user operating a button shown on the monitor 60 or by so-called remote control. However, the setting can also be automatically performed by setting by specifying the photographing time, image recognition (processing), detection of various sensors, and the like.

  That is, for example, when setting is performed by specifying the shooting time, the panorama shooting time is set by the input device 35 such as a keyboard or a pointing device such as a mouse attached to the computer device 30 of FIG. The time is stored in the storage device 32.

  Then, for example, in the event determination step [3] of FIG. 2, a clock (not shown) built in the computer device 30 is compared with the stored set time, and when they match, a panoramic image creation request is issued. Commanded and taken. Note that continuous shooting can be automatically performed by setting a plurality of times or by instructing a certain time interval.

  Alternatively, the video information obtained from the imaging device 10 can be compared between the present and the past, and the difference can be detected to automatically issue a panoramic image creation request. The video information obtained here is image brightness, color difference, contour, color balance, and the like.

  Therefore, for example, in a method using luminance information, the average luminance of the portion of the image currently displayed from the imaging device 10 is compared with the average luminance of a portion of the panoramic image that has been captured and stored in the storage device 32. . When the difference is larger than a certain level, a panoramic image creation request is instructed and photographing is performed. The created panoramic image is immediately displayed on the monitor 60 or the like.

  In this method, when shooting the scenery or the like, when the ambient brightness changes with time, the panoramic image is automatically captured and updated, so that the current imaging device 10 always captures the image. It is possible to provide a panoramic image that is close to the existing video.

  For example, as shown in FIG. 8, a representative point is designated for the subject imaged by the imaging device 10, and the luminance and color difference information are combined and modeled, and stored in the storage device 32 together with the position information. The same range is always imaged by the imaging device 10, and when a difference occurs in the position coordinates of the subject model, a panorama image creation request is instructed and imaging can be performed.

  That is, in A of FIG. 8, when the position coordinates (X0, Y0) of the original subject model become the position coordinates (X1, Y1) as shown in B of FIG. 8, X0 ≠ X1 or Y0 ≠ Y1 is set. As a result of the determination, it can be recognized that the subject has moved as shown in FIG. 8C, a panoramic image creation request can be instructed, and shooting can be started. By this method, the above-described apparatus can be used as a dynamic monitoring system.

  In addition, it is also possible to provide a means for obtaining audio information such as a microphone, and start imaging by instructing a panoramic image creation request in accordance with detection of audio. In this case, for example, as shown in FIG. 9, for example, a plurality of directional microphones are installed in a range where the imaging apparatus 10 is moved by a driving unit such as a rotary head 50 and can be imaged. Audio information collected by these directional microphones is input to the computer apparatus 30 shown in FIG.

  In the computer device 30, when specific audio information is detected in the audio information collected by the microphone, a panoramic image creation request is issued, and the directionality of the microphone from which the audio information is detected is determined in which direction. It is possible to store the panorama image by taking a picture from that direction if necessary.

  In this way, it is possible to recognize a change in the subject by detecting specific audio information, instruct a panoramic image creation request, and start shooting. In this method, similarly, various sensors such as an optical sensor and a pyroelectric sensor can be used to perform detection and update of a panoramic image as necessary.

  Further, in the apparatus shown in FIG. 1 described above, the storage apparatus 32 as the recording means is provided, so that the apparatus can be implemented as a recording apparatus. In this case, the storage device 32 records the image signal from the image capturing device 10 and the image capturing time and image capturing conditions at the time of image capturing.

  Therefore, in this apparatus, by changing the imaging range of the imaging means and recording the video signal from such imaging means and the imaging time and imaging conditions at the time of imaging, the imaging time and imaging can be made with a simple configuration. A panoramic image that does not give a sense of incongruity to changes in conditions or the like can be recorded with high resolution.

  Further, in the apparatus of FIG. 1 described above, the image processing apparatus 33 is provided as a video signal processing means, so that the apparatus can be implemented as a processing apparatus. In this case, the image processing device 33 connects the video signals from the imaging device 10 to form a panoramic image, and corrects the connected video signals in response to changes in imaging time, imaging conditions, and the like. is there.

  Therefore, in this apparatus, by changing the imaging range of the imaging means and recording the video signal from such imaging means and the imaging time and imaging conditions at the time of imaging, the imaging time and imaging can be made with a simple configuration. A panoramic image that does not give a sense of incongruity to changes in conditions or the like can be formed with high resolution.

  Further, in the apparatus of FIG. 1 described above, by providing a computer monitor 60 as a display means, this apparatus can be implemented as a display apparatus.

  Therefore, in this apparatus, by changing the imaging range of the imaging means, combining the video signals from such imaging means to form a video signal having a wide angle of view, and displaying this video signal on the display means With a simple configuration, it is possible to view a panoramic image with high resolution that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  Further, in the apparatus of FIG. 1 described above, by providing the storage device 32 as reproducing means, this apparatus can be implemented as a reproducing apparatus. In this case, the storage device 32 records and reproduces the video signal from the imaging device 10 and the imaging time and imaging conditions at the time of imaging.

  Therefore, in this apparatus, by reproducing the video signal in which the imaging range is sequentially changed, the imaging time and imaging conditions at the time of imaging, and synthesizing the reproduced video signal to form a video signal having a wide angle of view, With a simple configuration, a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, etc. can be reproduced with high resolution.

  Further, in the apparatus of FIG. 1 described above, a signal transmitted from the computer apparatus 30 to the computer monitor 60 is transmitted to a computer network such as the Internet via a modem (not shown), so that the apparatus becomes a transmission apparatus. Can be implemented.

  That is, in such a computer network, with respect to the configuration of the imaging device 10 to the computer device 30 to the rotary head 50 installed in a remote place, a network from an arbitrary computer device, a display device, and a user side device having a modem. Access through. As a result, a captured image at that time and a panoramic image stored in the storage device 32 of the computer device 30, for example, can be obtained on the display device on the user side with the same screen configuration as that of the monitor 60 of FIG.

  Therefore, in this apparatus, by synthesizing the video signals picked up by the image pickup means installed at a remote place to form a video signal having a wide angle of view, it is possible to cope with changes in weather, time, etc. with a simple configuration. A panoramic image that does not cause a sense of incongruity can be viewed with high resolution.

  Further, in the apparatus of FIG. 1 described above, the recording medium of the storage device 32 is removable, and this apparatus can be implemented as a recording medium by taking out the recording medium.

  Therefore, in this apparatus, by changing the imaging range of the imaging means and recording the video signal from such imaging means and the imaging time and imaging conditions at the time of imaging, this recording medium is reproduced and connected. Thus, it is possible to obtain a panoramic image with high resolution with a simple configuration and without causing a sense of incongruity with respect to changes in imaging time, imaging conditions, and the like.

  In these devices, only a part of the panoramic image is changed by correcting the video signal of the other partial area according to the imaging condition of the video signal of the partial area instructed on the display means for displaying the formed video signal. Even in this case, it is possible to obtain a panoramic image that does not give a sense of incongruity to changes in imaging time, imaging conditions, and the like.

  Furthermore, in the above-described apparatus, when storing information such as the imaging time and / or imaging conditions at the time of imaging together with the captured video signal, for example, as shown in FIG. This information is added to and stored. Alternatively, as shown in FIG. 7B, separately from each video file, only information such as the imaging time and / or imaging conditions can be stored together and associated with each video file.

  In the above-described apparatus, for example, one imaging apparatus 10 is used to change the imaging range by the rotary head 50 and the panoramic image is captured. However, this is not limited to such means. . That is, as means for capturing such a panoramic image, for example, a panoramic image is created based on a series of images captured using a digital still image camera or based on a moving image captured by a video camera. be able to.

  Further, instead of using the rotary head 50, for example, a plurality of image pickup devices are arranged in each direction for forming a panoramic image, and the video signals from these image pickup devices are sequentially switched as drive means. The video signal can be acquired.

  Further, in the above-described apparatus, it is possible to quickly and easily search for a video signal captured using information such as stored imaging time and imaging conditions. Thus, for example, when a desired image is searched for by operating the operation button 63 on the monitor 60 described above, the search can be performed very smoothly.

It is a block diagram of the whole system of an example of the imaging and display apparatus to which this invention is applied. It is a flowchart figure for description of the operation. It is a flowchart figure for description of the operation. It is a flowchart figure for description of the operation. It is a flowchart figure for description of the operation. It is a figure for the description. It is a figure for the description. It is a figure for the description. It is a figure for the description. It is a figure for the description.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Lens block part, 12 ... CCD, 13 ... Signal separation / automatic gain control device, 14 ... A / D conversion device, 15 ... Signal processing device, 16 ... Stepping motor for zoom, 20 ... Television Monitor 30. Computer 31 Capture video board 32 Storage device 33 Image processing device 34 Control device 35 Input device 41 Camera controller 42 Mode controller 43 Pantilter controller 50 Rotating head, 51 ... Stepping motor for panning, 52 ... Stepping motor for tilting, 60 ... Computer monitor, 61 ... Display area for captured image, 62 ... Display area for panoramic image

Claims (35)

Imaging means for imaging a subject and generating a video signal;
Control means for controlling the operation of the imaging means;
Storage means for storing the imaged video signal;
Processing means for connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means;
An image pickup apparatus, wherein an image pickup time and / or an image pickup condition when image pickup is performed by the image pickup unit is stored in the storage unit together with the video signal. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
2. The image pickup apparatus according to claim 1, wherein the image signals having a wider angle of view than one image pickup range of the image pickup means are formed by connecting the image signals close to the image pickup time and / or image pickup conditions. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. The imaging apparatus according to 1. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
2. The imaging apparatus according to claim 1, wherein the video signal is formed by connecting the video signal captured by the imaging unit to a partial area of the video signal instructed on the display unit. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
The imaging condition of the video signal in the partial area designated on the display means when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals The image pickup apparatus according to claim 1, wherein the video signal of the other partial area is corrected according to: Imaging means for imaging a subject and generating a video signal;
Control means for controlling the operation of the imaging means;
Storage means for storing the imaged video signal;
Processing means for connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means;
A storage device characterized by storing an imaging time and / or imaging conditions when imaging is performed by the imaging means together with the video signal. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
7. The storage device according to claim 6, wherein the video signals having the imaging time and / or the imaging conditions close to each other are connected to form a video signal having a wider angle of view than one imaging range of the imaging means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. 6. The storage device according to 6. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
7. The storage device according to claim 6, wherein the video signal is formed by connecting the video signal picked up by the image pickup means to a partial region of the video signal instructed on the display means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
The imaging condition of the video signal of the partial area designated on the display means when the video signal having a wider angle of view than one imaging range of the imaging means is formed by connecting the imaged and stored video signals 7. The storage device according to claim 6, wherein the video signal of the other partial area is corrected according to the above. Imaging means for imaging a subject and generating a video signal;
Control means for controlling the operation of the imaging means;
Storage means for storing an imaging time and / or imaging conditions when the imaging means performs imaging together with the imaged video signal,
A processing apparatus comprising: connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
12. The processing apparatus according to claim 11, wherein the video signals having a wider angle of view than one imaging range of the imaging means are formed by connecting the video signals having the imaging time and / or imaging conditions close to each other. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. 11. The processing apparatus according to 11. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
12. The processing apparatus according to claim 11, wherein the video signal is formed by connecting the video signal picked up by the image pickup means to a partial area of the video signal instructed on the display means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
The imaging condition of the video signal of the partial area designated on the display means when the video signal having a wider angle of view than one imaging range of the imaging means is formed by connecting the imaged and stored video signals 12. The processing apparatus according to claim 11, wherein the video signal of the other partial area is corrected according to the above. Imaging means for imaging a subject and generating a video signal;
Control means for controlling the operation of the imaging means;
Storage means for storing the imaged video signal;
Processing means for connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means; and display means for displaying the formed video signal. ,
In addition to storing the imaging time and / or imaging conditions at the time of imaging with the imaging means together with the video signal in the storage means,
A display device, wherein the video signal is formed by connecting the video signal picked up by the image pickup means to a partial region of the video signal instructed on the display means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
17. The display device according to claim 16, wherein the video signals having a wider angle of view than one imaging range of the imaging means are formed by connecting the video signals having similar imaging times and / or imaging conditions. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a wider angle of view than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. 16. The display device according to 16. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
17. The display device according to claim 16, wherein the video signal is formed by connecting the video signal picked up by the image pickup means to a partial region of the video signal instructed on the display means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
When the imaged and stored video signals are connected to form a video signal having a wider angle of view than one imaging range of the imaging means, the other part according to the imaging condition of the video signal in the indicated partial area The display device according to claim 16, wherein the video signal in the region is corrected. Playback means for playing back a recording medium on which the imaging time and / or imaging conditions at which the imaging means was recorded together with a video signal captured by controlling the operation of the imaging means;
A video signal having a field angle wider than one imaging range of the imaging means is formed by connecting video signals reproduced by the reproduction means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The reproducing apparatus according to claim 21, wherein the video signals having the imaging time and / or the imaging conditions close to each other are connected to form a video signal having a wider angle of view than one imaging range of the imaging means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. 21. A reproducing apparatus according to item 21. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
The reproducing apparatus according to claim 21, wherein the video signal is formed by connecting the video signal picked up by the image pickup means to a partial area of the video signal instructed on the display means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
When the imaged and stored video signals are connected to form a video signal having a wider angle of view than one imaging range of the imaging means, the other part according to the imaging condition of the video signal in the indicated partial area The reproduction apparatus according to claim 21, wherein the video signal in the area is corrected. Imaging means for imaging a subject and generating a video signal;
Control means for controlling the operation of the imaging means;
Storage means for storing an imaging time and / or imaging conditions when imaging is performed by the imaging means together with the imaged video signal;
Processing means for connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means;
And a transmission means for transmitting the video signal formed by the processing means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
27. The transmission apparatus according to claim 26, wherein the video signals having the imaging time and / or the imaging conditions close to each other are connected to form a video signal having a wider angle of view than one imaging range of the imaging means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. 26. The transmission apparatus according to 26. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
27. The transmission apparatus according to claim 26, wherein the video signal is formed by connecting the video signal picked up by the image pickup means to a partial area of the video signal instructed on the display means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
Display means for displaying the formed video signal,
The imaging condition of the video signal of the partial area designated on the display means when the video signal having a wider angle of view than one imaging range of the imaging means is formed by connecting the imaged and stored video signals 27. The transmission apparatus according to claim 26, wherein the video signal of the other partial area is corrected according to: Imaging means for imaging a subject and generating a video signal;
Control means for controlling the operation of the imaging means;
Using processing means for connecting the imaged and stored video signals to form a video signal having a wider angle of view than one imaging range of the imaging means, and imaging means for generating the video signals,
A recording medium in which an imaging time and / or an imaging condition when imaging is performed by the imaging unit is recorded together with a video signal captured by the imaging unit. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
32. The recording medium according to claim 31, wherein the video signals having the imaging time and / or the imaging conditions close to each other are connected to form a video signal having a wider angle of view than one imaging range of the imaging means. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
The video signal is corrected according to the imaging condition when the video signal having a field angle wider than one imaging range of the imaging means is formed by connecting the imaged and stored video signals. 31. The recording medium according to 31. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
32. The video signal is formed by connecting the video signal picked up by the image pickup means to a partial area of the video signal indicated on the display means for displaying the formed video signal. The recording medium described. The imaging conditions include one or more information of exposure, white balance, focus, magnification, size, position coordinates,
A portion designated on the display means for displaying the formed video signal when the image signal having the angle of view wider than one imaging range of the image pickup means is formed by connecting the image pickup and stored video signals. 32. The recording medium according to claim 31, wherein the video signal in another partial area is corrected in accordance with an imaging condition of the video signal in the area.

Images