JP2007281961A - Imaging apparatus, imaging apparatus control method, and imaging control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve matching accuracy of motion detection even for an object of great movement with the same search amount of processing for motion vector search as the conventional art. <P>SOLUTION: An imaging apparatus includes an imaging means capable of varying an imaging rate and a means capable of compositing a motion vector converted into a reference frame rate even when the imaging rate is changed. When a moving amount is great, photographing is performed at the quickened imaging rate, thereby reducing the relative moving amount of the object between photographed frame image data, and providing an effect that motion detection matching accuracy can be improved even for an object of great movement with the same search amount of processing for motion vector search as the conventional art. Furthermore, by changing the photographing rate in accordance with the moving amount, motion vector detection can be performed at the optimal rate for both a quickly moving object and a slowly moving object, thereby improving matching accuracy in comparison with a case where the rate is fixed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for digitizing an imaging signal and processing the digital signal, and more particularly to a technique for processing using a motion vector.

  Conventionally, a method of performing processing using a motion vector when processing a moving image has been widely used. A typical example of the processing is MPEG image compression.

  Various methods have been proposed for motion vector detection and accurate and efficient search. For example, there is an apparatus using acceleration information of an imaging device for limiting a motion vector search range by a telescopic search (see, for example, Patent Document 1).

  On the other hand, a technique for extracting a motion vector from an image having a high frame rate has been proposed in order to track the motion vector with high accuracy (see, for example, Patent Document 2).

JP-A-6-261309 (6th page, FIG. 1, FIG. 2) JP 2004-289709 A (page 11, FIG. 2, page 12, FIG. 3)

  In order to reliably search for a motion vector, it is necessary to make the search range of corresponding points as wide as possible. However, as the search range is expanded, the amount of calculation required for the search increases, and it is necessary to effectively limit the search range for the search in real time.

  In the invention disclosed in Patent Document 1, the acceleration information of the imaging device is used to limit the motion vector search range by the telescopic search. However, the subject to be imaged is within the search range within a certain time independently of the motion of the imaging device itself. If it moves at a speed that deviates from, the motion vector cannot be extracted.

  On the other hand, in order to perform tracking with high accuracy of motion vectors, there is a method of obtaining the motion of each image of a moving image by shooting using a high frame rate. However, in the case of obtaining a motion vector only at a high frame rate, if a moving object that is relatively slow with respect to the high frame rate is a subject, the motion amount is less than the amount of motion that can be recognized between the frame images. Thus, there is a limit to the detection of motion vectors.

  Accordingly, an object of the present invention is to improve the matching accuracy of motion detection even for an object having a large motion with the same amount of search processing for motion vector search as that of the prior art.

An imaging apparatus according to the present invention includes an imaging unit that converts a subject image into an electrical signal, a unit that changes an imaging rate of the imaging unit, a developing unit that processes the electrical signal from the imaging unit and outputs an image signal. Storing means for storing the image signal for a plurality of screens as frame image data for each screen; and motion vector detecting means for detecting a motion vector in units of blocks among the plurality of frame image data stored in the storing means; And a motion vector synthesis means for synthesizing a plurality of the detected motion vectors, and imaging of the imaging means according to the detected motion vector or the magnitude of the motion vector synthesized by the motion vector synthesis means The rate is changed to a frame rate different from the reference frame rate, and if changed, the different frame rate Characterized by combining a motion vector in terms of the reference frame rate using the detected motion vector from the shadows plurality of frame image data.
An imaging apparatus control method according to the present invention includes an imaging unit that converts a subject image into an electrical signal, a unit that changes an imaging rate of the imaging unit, and a development that processes an electrical signal from the imaging unit and outputs an image signal. Means, a storage means for storing the image signal for a plurality of screens as frame image data for each screen, and a motion vector detection for detecting a motion vector in units of blocks between the plurality of frame image data stored in the storage means Means for synthesizing a plurality of detected motion vectors, and the imaging rate of the imaging means can be changed to a frame rate different from a reference frame rate in accordance with the detected motion vectors, depending on the different frame rates. When captured, converted to the reference frame rate using motion vectors detected from multiple frame image data A representative obtained from a plurality of the detected motion vectors existing in one screen or a motion vector synthesized so as to be converted into the reference frame rate by a predetermined method. An imaging apparatus control method for instructing an imaging apparatus having a representative motion vector setting means for determining a motion vector to instruct a setting related to the imaging rate from a display apparatus displaying an image output from the imaging apparatus. And a means for designating a specific method from a plurality of setting methods, wherein the representative motion vector is generated based on the specific method set by the setting means.
The imaging control method of the present invention includes a step of changing an imaging rate when converting a subject image into an electrical signal, a developing step of processing an electrical signal from the imaging means and outputting an image signal, and the image signal An accumulation step of accumulating a plurality of frames as frame image data for each screen; a motion vector detection step of detecting a motion vector in units of blocks among the plural frame image data accumulated in the accumulation means; A motion vector synthesis step of synthesizing a plurality of motion vectors, and the imaging rate is different from a reference frame rate according to the detected motion vector or the magnitude of the motion vector synthesized by the motion vector synthesis step. In addition to changing to a rate, if changed, a plurality of frame images taken at the different frame rates Characterized by combining a motion vector in terms of the reference frame rate using the detected motion vector from the chromatography data.
A program according to the present invention causes a computer to execute the imaging control method.

  According to the present invention, the imaging means having a variable imaging rate and the means capable of synthesizing a motion vector converted to the reference frame rate even when the imaging rate is changed, when the amount of motion is large. By shooting at a high imaging rate, it is possible to reduce the relative amount of motion of the object between the captured frame image data. This has the effect of making it possible to improve the matching accuracy of motion detection even for large objects.

  In addition, according to the present invention, by changing the shooting rate according to the amount of motion, it is possible to detect a motion vector at an optimal rate for both fast and slow motion, compared to a fixed rate. There is an effect that the matching accuracy can be improved.

  According to the present invention, when detecting a motion vector from a plurality of frame image data shot at different frame rates based on the magnitude of the representative motion vector, the motion vector converted to the immediately preceding reference frame rate is detected. By making it possible to select a reference image to be used in motion vector detection from the plurality of frame image data based on the size, for example, a frame with an increased imaging rate for a fast-moving block It is possible to perform motion detection between image data, and it is possible to perform motion detection between frame image data at intervals with respect to blocks with small motion.

  As a result, it is possible to improve the motion vector detection system for the entire image without unnecessarily imposing a processing load on blocks with small motion.

  Further, according to the present invention, blocks for which a motion vector is obtained are grouped based on the magnitude of the motion vector converted to the immediately preceding reference frame rate and its positional relationship, and used to create a motion vector for each group. By selecting the reference frame to be used at an appropriate interval based on the size of the immediately preceding reference rate motion vector, etc., for example, the imaging rate for a fast-moving block is reduced while suppressing the amount of image data read from the frame memory. It is possible to perform motion detection between the frame image data that have risen, and to perform motion detection between the frame image data having a large interval for blocks with small motion.

  As a result, it is possible to reduce the processing time overhead required for memory reading, so it is possible to lengthen the time required for the search process, and ultimately improve the accuracy of motion vector detection in the entire image. It becomes possible to improve.

  Further, according to the present invention, the means for synthesizing one frame image from the plurality of frame image data accumulated by the accumulation means, the means for selecting one frame image, or the means for synthesizing and selecting By having both of the means, even when the imaging rate is changed from the reference frame rate, the image user is not affected and the necessary image data is supplied to the image user with high accuracy. It is possible to perform motion vector detection.

  According to the present invention, in the case where the imaging apparatus includes an image compression unit that uses a motion vector, the frame image data of the generated reference frame rate can be obtained even when the image is captured at a different frame rate. By performing image compression based on the reference frame rate using the motion vector converted into the reference frame rate, the frame rate requested by the user of the compressed image is not affected and the accuracy is high. By using the motion vector detection result for compression encoding, the amount of codes can be reduced.

  In addition, according to the present invention, when the imaging apparatus includes the moving object detection unit, the moving object is detected using the motion vector converted into the reference frame rate, so that an object having a larger movement than the conventional one can be used. It becomes possible to accurately follow, and the accuracy of moving object detection can be improved.

  Furthermore, as a method for instructing such an imaging apparatus to set an imaging rate setting from a display device displaying an image output from the imaging apparatus, means for designating a specific system from a plurality of setting systems is provided. The user can specify how to change the imaging frame rate according to the purpose of use by allowing the user to change the different frame rate based on the specific method set by the specifying means. Thus, it becomes possible to perform motion detection according to the purpose of use of the user.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

-First embodiment-
First, a first embodiment of the present invention will be described.
FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to the first embodiment of the present invention. An imaging apparatus 1 is connected to the network 2. In the imaging device 1, light is collected by the imaging lens 100 included in the imaging unit 10, and light controlled based on an appropriate shutter speed or the like is photoelectrically converted in the imaging element 101. The photoelectrically converted signal is supplied to the development processing unit 102, where it is output to the frame control unit 11 as an image signal.

  The imaging rate of the imaging element is controlled by the imaging rate control unit 104 inside the imaging control unit 103. The imaging control unit 103 also controls the shutter speed of the image sensor 101 and the like.

  The image data sent to the frame control unit 11 is processed by the frame control unit 11 as necessary, and then stored in the frame memory 12 as frame image data for each screen.

  The frame image data stored in the frame memory 12 is stored in a local region (hereinafter, referred to as “frame region data”) in the frame vector data by using the past frame image data stored in the frame memory 12 as reference frame image data. Called each block). A motion vector is generated for each block.

  Each generated motion vector is combined with the motion vector for the frame image data before and after that by the motion vector combining unit 14 as necessary, and used as a new motion vector.

  Thereafter, the frame image data stored in the frame memory 12 is compressed by the compression processing unit 15. In the present embodiment, it is assumed that compression processing is performed using a motion vector as in the MPEG format.

  The compression processing unit 15 obtains an appropriate motion vector from the motion vector detection unit 13 and the motion vector synthesis unit 14, and performs compression processing based on the motion vector information.

  The compressed image data is sent from the network control unit 16 to the network 2 via the network interface 17 and displayed on the receiving terminal.

  In addition, the moving object detection unit 18 obtains an appropriate motion vector from the motion vector detection unit 13 and the motion vector synthesis unit 14, and performs a moving object detection process based on the motion vector information.

  The result of the moving body detection is transmitted from the network control unit 16 to the network 2 via the network interface 17, for example, and used to sound an alarm on the receiving side terminal when a moving object is detected. .

  Further, the video output control unit 20 reads out the frame image data from the frame memory 12 according to a predetermined output rate, converts the frame image data into an appropriate format according to the output, and then converts the video signal through the video output interface 21. This is supplied to the external monitor 22.

  Here, the monitor 22 is an external monitor, but it may be a liquid crystal display device attached to the imaging device, and the configuration is not limited in this embodiment.

FIG. 2 is a diagram illustrating a configuration example of a system using the imaging apparatus 1 according to the present embodiment.
Data obtained by compressing an image captured by the imaging device 1 is transmitted from the imaging device 1 to the display terminal 3 via the network 2.

  The display terminal 3 receives the compressed image data, expands it, and displays the image on the viewer 30.

  In an imaging apparatus having such a configuration, how to change the imaging rate of an imaging device using a motion vector and how to extract a motion vector after changing the imaging rate and create an image requested by the user at the same time This will be described with reference to FIGS. 1 and 2 and FIGS.

FIG. 3 shows a motion vector detection method in the motion vector detection unit.
FIG. 4 is a diagram relating to subdivision of the motion vector search area for explaining a mechanism for determining the magnitude of the detected motion vector and a mechanism for instructing the change of the imaging rate as necessary.
FIG. 5 is a diagram for explaining motion vector detection and synthesis when the imaging rate is changed from the reference frame rate.
FIG. 6 is a diagram for explaining how the frame control unit 11 generates an image at the reference frame rate from the captured frame data.

Hereinafter, detection of a motion vector in the motion vector detection unit 13 in this embodiment will be described with reference to FIG.
In FIG. 3, it is assumed that an image captured by the imaging unit 10 at time t = a is stored in the frame memory 12 as frame image data 60-a for each screen. Here, it is assumed that the current imaging rate of the image sensor 101 is set to the reference frame rate, and the reference frame rate is 30 frames / second.

  It is assumed that there are moving objects 61 and 62 on the frame image data 60-a, where 61 is at the position (61-a, 62-a) indicated by ● and 62.

  At the time t = b after 1/30 second, which is the imaging interval of the reference frame rate, in the captured frame image data 60-b, the moving object 61 is at the position indicated by ◯ and the moving object 62 is at the position indicated by Δ (61 -B, 62-b).

  Here, the motion vector detection unit 13 reads the frame image data 60-b from the frame memory 12 and detects the motion vectors of the moving objects 61 and 62 at time t = b. As a reference frame image for detecting the motion vector, 60-a which is an image one frame before is used.

  First, motion vector detection related to the moving object 61 will be described. The motion vector detection unit 13 sets the position information corresponding to 61-b as 61-b ′ on the reference frame image 60-a (corresponding to “◯” on the image data 60-a). A motion vector search area 51 (area surrounded by a thick line in the figure) is set as a rectangular area centered at.

  Then, an image block that most closely matches the content of the block located at the position 61-b in the image data 60-b in the motion vector search area 51 is searched.

  This pattern matching is performed by obtaining a difference absolute value, a difference square value, and the like and comparing the calculation results.

  Here, since the moving object 61 has moved from the position 61-a to the position 61-b, the block existing at the position 61-a indicated by ● in the image 60-a is the image data. It most closely matches the contents of the block existing at position 61-b in 60-b.

  Therefore, the motion vector corresponding to the position 61-b at time t = b is the motion vector indicated by the arrow 64.

  Similarly, for the motion vector related to the moving object 62, the position information corresponding to 62-b is set on the 60-a image as the reference frame image data (Δ62-b on the image data 60-a). As a result of setting and searching for a motion vector search region 52 of a rectangular region centered on the region, it is detected as a motion vector indicated by an arrow 66.

  Hereinafter, a mechanism for determining the magnitude of the motion vector and a mechanism for instructing the change of the imaging rate as necessary will be described in more detail with reference to FIG. 1 and FIG.

  FIG. 4 shows a search area for searching for a motion vector in the case of a certain first imaging rate.

  In the figure, reference numeral 43 denotes a place where a block serving as a search origin is located, and it is assumed that a motion vector for a block existing at this position is to be extracted. A line 44 of the outer frame of the hatched area 40 indicates a search area at the first imaging rate.

  A search for a motion vector is performed within the range of this region. The search area at the first imaging rate includes three areas of a high speed area 40 (shaded area), a medium speed area 41 (white area), and a low speed area 42 (halftone area) according to the positional relationship from the search origin 43. Divided into two.

  In the present embodiment, when the frame length of the search area 44 is 1, the outer frame length of the medium speed area 41 is 0.7, and the outer frame length of the low speed area 42 is 0.4. The size of each region is set at such a ratio, but this ratio is not limited to the ratio shown here, and can be arbitrarily set as necessary.

  When a motion vector is detected, the motion vector detection unit 13 checks where the start point position of the motion vector belongs to among the high speed area 40, the medium speed area 41, and the low speed area 42 described above.

  Then, when the start point of the motion vector belongs to the high speed region 40 as in the point C in FIG. 4, it is determined that the motion of the block being noticed is large, and the imaging rate is set so as to prepare for the higher speed motion. To the imaging rate control unit 104 of the imaging control unit 103.

  On the other hand, when the start point of the motion vector belongs to the low speed region 42 as indicated by the point A in FIG. 4, it is determined that the motion of the focused block is small.

  If the observation continues for a certain period and the state where the motion of the block is small continues, the imaging rate control unit 104 of the imaging control unit 103 is instructed to lower the imaging rate.

  On the other hand, when the start point of the motion vector belongs to the medium speed region 41 as in the point B in FIG.

  In FIG. 1, the imaging rate control unit 104 in the imaging control unit 103 that has received an instruction from the motion vector detection unit 13 indicates the imaging rate currently instructed to the imaging element 101 and the instruction from the motion vector detection unit 13. Are combined to make a determination for changing the imaging rate.

  Basically, the imaging rate is increased or decreased in accordance with an instruction from the motion vector detection unit 13, but if the current imaging rate is set to the reference frame rate, the imaging rate is not further decreased. .

  In FIG. 4, a mechanism for determining the size of a general motion vector and a mechanism for instructing to change the imaging rate as necessary are described. However, this series of operations is performed when a representative motion vector is set. It is also possible to perform only for the representative motion vector.

  Note that the representative motion vector is set by the representative motion vector setting unit 19 in accordance with an instruction from the display device 3 connected to the end of the network 2, and details thereof will be described later.

  In addition, it is possible to instruct to change the imaging frame rate using only the determination result based on the magnitude of the representative motion vector from among all the motion vectors only for the size determination. It is not limited to the content.

  The above contents will be described using the case of FIG. In FIG. 3, the motion vector search areas are set to 51 and 52 for the moving objects 61 and 62.

  The motion vector search areas 51 and 52 are shown as high speed areas 63 and 65 (shaded areas) and medium speed areas 67 and 68 (white areas) according to the positional relationship from the search origins 61-b and 62-b. ing. A low speed region may also exist but is not shown here.

  In the present embodiment, the motion vector 66 of the moving object 62 has a start point in the medium speed region 68, but the motion vector 64 of the moving object 61 has a start point in the high speed region 63. At this time, in this embodiment, it is assumed that the motion vector 64 is designated as the representative motion vector.

  Then, the motion vector detection unit 13 notifies the imaging rate control unit 104 of the imaging control unit 103 to increase the imaging rate because the start point of the representative motion vector belongs to the high speed region.

Upon receiving the notification, the imaging rate control unit 104 changes the imaging rate based on a predetermined method.
Since an instruction is given to increase the imaging rate, for example, the imaging rate is changed to 60 frames / second, which is doubled.

  Here, the change in the imaging rate is doubled, but the magnitude of the change is not limited to double, and an appropriate value can be taken as necessary.

  FIG. 5 shows the state of the image, the state of motion vector detection, and the composition of the detected motion vector after the imaging rate is changed from 30 frames / second, which is the reference frame rate, to 60 frames / second, which is doubled. FIG.

  In FIG. 5, a part of 60-b in FIG. 3 is cut out to 60-b ′, and attention is paid to the moving object 61.

  In FIG. 5, it is assumed that the moving object 61 moves from the state of FIG. 3 at a constant speed in the same direction. In that case, in the photographed image 6-c ′ at time t = c 1/60 seconds after 60-b ′, the photographed image 60− at position 61-c and further at time t = d 1/60 seconds later. In d ′, the moving object 61 has moved to the position 61-d. At this time, a motion vector is detected between the respective frame images.

  Here, it is assumed that the motion vector detection unit 13 reads the frame image data 60-c ′ from the frame memory 12 and detects the motion vector of the moving object 61 at time t = c. As a reference frame image for detecting the motion vector, 60-b ′ ′ which is an image one frame before is used.

  As in the case of FIG. 3, the motion vector detection unit 13 sets the position information corresponding to 61-c as 61-c ′ on the frame image data 60-b ′, and a rectangular area centered there The motion vector search region 51-b is set.

  Here, in the motion vector search area, the imaging frame rate is doubled from 30 frames / second to 60 frames / second, so that the time required for motion vector search does not change as a whole. On the contrary, the search area is set to be approximately ½.

  That is, the motion vector search area 51-b, c, d in FIG. 5 is approximately ½ the area of the motion vector search area 51 in FIG. It is the same size as the white area.

  Then, an image block that most closely matches the content of the block located at the position 61-c in the image data 60-c ′ is searched for in the motion vector search region.

  Now, since the moving object 61 has moved from 61-b to 61-c, the block existing at the position 61-b indicated by a circle in the image 60-b ′ is the image data 60-. It most closely matches the contents of the block located at position 61-c in c ′. Therefore, the motion vector corresponding to the position 61-c at time t = c is the motion vector indicated by the arrow 64-b.

  Similarly, in the frame image 60-d ′ at time t = d, the motion vector corresponding to the position 61-d is the case where the reference frame image is 60-c ′ which is a frame 1/60 second before. , 64-c.

  Here, a motion vector is obtained for each successive frame image after the imaging rate has increased, but when using a motion vector, a motion vector that matches the form to be used is required.

  For example, when a video compressed at a reference frame rate is distributed to a network, a motion vector (hereinafter referred to as a reference rate motion vector) for an image at the reference frame rate is required.

  In the present embodiment, the synthesis of the reference rate motion vector is performed by the motion vector synthesis unit 14.

  In FIG. 1, the motion vector detection unit 13 transmits information of the detected motion vector together with the frame number to the motion vector synthesis unit 14 when an image is captured at an imaging rate other than the reference frame rate. The motion vector synthesizing unit obtains information on the imaging rate from the imaging control unit 104 and information on the frame image from the frame control unit 11, and based on the information, the motion vector information transmitted from the motion vector detection unit 13 is obtained. Judge how to process.

  For example, in FIG. 5, if the reference rate motion vector is obtained for the frame image 60-d ′, the motion vector synthesis unit 14 obtains the two pieces of motion vector information of 64-b and 64-c obtained previously. The reference rate motion vector 69 is synthesized from the above.

  The synthesized motion vector 69 is used as motion vector information at the reference frame rate interval by the compression processing unit 15 and the moving object detection unit 18.

  Next, when the imaging rate is changed from the reference frame rate of 30 frames / second, how is the output frame data at the reference frame rate from the imaging frame data captured at a higher imaging rate than the reference frame rate? With reference to FIG. 1, it shows using FIG. 6 regarding whether it produces.

  In FIG. 1, the frame control unit 11 obtains an imaging rate and shutter speed information at the time of imaging from the imaging control unit 103. Based on the acquired information, the frame control unit 11 processes the output image data in the form shown in FIG. 6 and stores it in the frame memory 12 as frame image data for each screen.

  FIG. 6 shows an example of processing the imaging frame data in the frame control unit 11.

  FIG. 6A) schematically shows the state of the imaging frame data when the imaging element 101 is imaging at the reference frame rate (30 frames / second). At this time, as the requested output frame data, the imaging frame data shown here is used as it is.

  Therefore, no particular processing is performed on the image, and the imaging frame data [0], [1], and [2] are stored in the frame memory 12 as they are.

  FIG. 6B) schematically shows the state of the imaging frame data when the imaging device 101 is imaging at a high-speed imaging rate higher than the reference frame rate (for example, four times the imaging rate (120 frames / second)). Is.

  In each period used for capturing the frame data [0], [1], and [2] when shooting at the reference frame rate, four images are captured, and [1A], [1B], [1] Images such as [1C] and [1D] are output as frame data at 1/120 second intervals.

  At this time, since image data having a reference frame rate is required as output, it is necessary to create output frame data in accordance with the reference frame rate from the captured frame data captured at these high-speed imaging rates.

  As the creation method, considering the reference shutter speed applied when image data is captured based on the reference frame rate, the following two methods are used according to the reference shutter speed.

  First, when the reference shutter speed is sufficiently high, that is, when the reference shutter speed is equal to or less than the imaging interval of the high-speed frame rate, each imaging using the reference shutter speed is performed even when shooting is performed at the high-speed frame rate. Frame data is captured.

  In that case, the output frame data is created by thinning out pictures at intervals according to the reference frame rate from the captured frame data captured at a high frame rate.

  FIG. 6c) corresponds to this. Here, since the shutter speed is 1/120 seconds or more, the shutter speed is shorter than the quadruple frame rate imaging interval (1/120 seconds).

  Therefore, the output frame data is not particularly processed with respect to the imaging frame data, and is extracted in a form such as [0D], [1D], [2D] at a necessary interval, and is stored in the frame memory 12 as it is.

  Here, in the case of FIGS. 6a) and 6c), it is written that the image is not processed and is stored in the frame memory 12 as it is. However, the captured frame data is already stored in the frame memory for motion vector detection. Since it is stored, it is possible to use the frame data for motion vector detection as output frame data. The output frame data is stored on the frame memory separately from the image data for motion vector detection. It is not essential to accumulate.

  Next, when the reference shutter speed is not fast, that is, when the reference shutter speed is longer than the high-speed frame rate imaging interval, when shooting at the high-speed frame rate, the high-speed frame rate imaging interval is set as the shutter speed. Each imaging frame data is captured.

  In that case, the image data based on the high-speed frame rate is insufficient in light quantity as compared with the case of imaging at the reference frame rate.

  Therefore, in order to compensate for the insufficient light quantity, output frame data is obtained by combining the imaging frame data shot at the high frame rate according to the ratio of the reference shutter speed and the imaging interval of the high frame rate. The

  In FIG. 6, d) and e) correspond to this. In FIG. 6d), the standard shutter speed is 1/60 seconds, which is twice the frame rate imaging interval (1/120 seconds) that is four times. In this case, the frame control unit 11 synthesizes two pieces of the captured frame data [1C] [1D] and accumulates them on the frame memory 12 in order to create image data based on the reference frame rate.

  In FIG. 6e), the reference shutter speed is 1/45 seconds, which is 8/3 times the quadruple frame rate imaging interval (1/120 seconds). In this case, in order to create image data based on the reference frame rate, the frame control unit 11 multiplies the image information component of [1B] by 2/3 in addition to the two pieces of imaging frame data [1C] [1D]. The data is synthesized and stored on the frame memory 12.

  Here, in the case of FIGS. 6d) and 6e), it is expressed that the images are synthesized. However, the synthesis is performed by using the frame control unit to capture the imaging frame data based on the high-speed frame rate once placed on the frame memory 12. Read out the required number of images and synthesize them at a predetermined ratio in the inside, and the imaging frame data input from the imaging processing unit 102 is written out to the frame memory 12 and at the same time, if necessary inside the frame control unit 11 There is a method of creating a frame composition memory by storing it in a frame composition memory (not shown) and adding sequentially inputted imaged frame data on the frame composition memory as required.

  Further, only the frame composition method is stored as supplementary data of the frame image on the frame memory 12, and the video output control unit or the like uses the supplementary data to read and synthesize the captured frame data from the frame memory as necessary. Several methods are conceivable.

  Next, a user interface for performing settings relating to the imaging frame rate in the present embodiment will be described with reference to FIGS.

FIG. 7 illustrates the viewer 30 of the display terminal 3 in the present embodiment.
The display terminal 3 can be configured, for example, by using a PC and its user interface device. Here, the viewer 30 is expressed as a window displayed on the display of the display terminal 3 configured by the PC.

  Also, it is assumed that each icon on the viewer 30 can be selected by a user interface device such as a mouse or a keyboard connected to the PC and instructed to operate.

FIG. 7A shows the state of the viewer 30 in the normal state.
Reference numeral 31 denotes a display window on which an image taken by the imaging device 1 is displayed. Reference numeral 32 denotes a menu, which can be used to display pull-down menu contents by selecting with the mouse cursor 33. The selection is performed by moving the mouse cursor to the desired location and clicking the mouse button.

  Reference numeral 34 denotes a connected camera name display unit, in which the imaging apparatus 1 is indicated by the name “camera 1”.

  In FIG. 7A, when the user selects the menu 32-a, various setting menus 32-b are displayed as shown in FIG. 7B.

  Next, when the user selects the imaging rate setting from the various setting menus 32-b, an imaging rate setting menu 32-c is displayed as shown in FIG. 7C.

  For example, it is assumed that the user has selected the priority area designation method from the imaging rate setting menu 32-c. In that case, as shown in FIG. 7D, the priority area designation frame 36 is displayed in the display window 31.

  The user moves the emphasis area designation frame 36 in the display window 31 using the mouse cursor 33, surrounds the desired emphasis area with the designation frame, and determines the position by clicking the mouse button.

  The determined priority area information is transmitted from the display terminal 3 to the imaging device 1 via the network 2.

  In the imaging apparatus 1, after the network interface 17 receives an instruction from the display terminal 3 in the form of a packet or the like, the information is transmitted to the network control unit 16.

  The network control unit 16 extracts important area information from the received information and transmits it to the representative motion vector setting unit 19.

  When the representative motion vector setting unit 19 receives the priority area information, the representative motion vector setting unit 19 sets, as the representative motion vector, a motion vector of a block located in the area designated by the priority area designation frame in the imaging angle of view.

  If there are multiple blocks in the priority area specification frame and multiple motion vectors can be candidates for the representative motion vector, select the representative motion vector candidate with the largest amount of motion. The representative motion vector is used as a representative motion vector, or the average of the higher number vectors having a large motion amount is calculated as the representative motion vector size and used for controlling the imaging rate.

  Note that the position in the priority area designation frame is fixed, and the imaging apparatus 1 extracts a representative motion vector from within this area unless a different designation is made.

  This designation method is particularly effective when the user wants to use a motion vector for motion detection by paying attention to the motion of a specific area for monitoring purposes.

  On the other hand, it is assumed here that the user has selected an object designation method from the imaging rate setting menu 32-c.

  In that case, an object designation frame 37 is displayed in the display window 31 as shown in FIG.

  The user moves the object designation frame 37-a in the display window 31 using the mouse cursor 33, surrounds the desired object with the designation frame, and confirms the object by clicking a mouse button.

  The confirmed object information is transmitted from the display terminal 3 to the imaging device 1 via the network 2 as in the case of the prior important area information. In the imaging apparatus 1, the object information is finally transmitted to the representative motion vector setting unit 19.

  When the representative motion vector setting unit 19 receives the object information, the representative motion vector setting unit 19 sets, as the representative motion vector, the motion vector of the block located in the area designated by the object designation frame 37 in the imaging angle of view.

  When there are a plurality of blocks in the object designation frame 37 and, as a result, a plurality of motion vectors can be candidates for the representative motion vector, the one with the largest amount of motion from among the plurality of representative motion vector candidates or the object designation Pick a motion vector of a block located at a position closer to the center of the frame 37 as a representative motion vector, or calculate the average of the higher number vectors having a large amount of motion as the size of the representative motion vector Used for rate control.

  Since the object designation frame 37 is for designating a target object for detecting a motion vector of interest, when the target object moves, the object designation frame 37 also moves along with the movement.

  The object designation frame 37 that was at the position 37-a in FIG. 7E has moved to the position 37-b in FIG. 7F. This movement of the frame is performed based on a motion vector as the object designation frame 37 which is obtained by averaging the motion vectors detected in the object designation frame 37 in the form of center weighting averaging.

  This calculation is performed in the representative motion vector setting unit 19, and the result is fed back to the display device 3 via the network control unit 16, the network interface 17, and further the network 2.

  In FIG. 7F, the object designation frame 37 has moved to the position 37-b as a result of reflecting this feedback.

  If the object designation method is selected, the selected object may move out of the screen. In such a case, the designation of the object is canceled and the high-speed operation priority method is automatically selected.

  In the high-speed operation priority method, the distribution of the magnitudes of motion vectors detected at all angles of view captured by the imaging apparatus 1 is determined, and the imaging rate is changed based on the distribution.

  When the number of motion vectors greater than or equal to a predetermined value exceeds a certain number, control is performed to increase the imaging rate, and conversely, the maximum motion vector size is less than or equal to the predetermined value. When the operation continues for a while, control is performed so as to lower the imaging rate.

  The determination of the motion vector distribution is also performed in the representative motion vector 19. As described above, the imaging unit has a variable imaging rate, a motion vector detection unit, and a motion vector synthesis unit that synthesizes a motion vector converted to the reference frame rate regardless of the imaging rate. By making the shooting faster, it becomes possible to reduce the relative movement amount of the object between the shot frame image data.

  As a result, it is possible to improve the matching accuracy of motion detection even for an object having a large motion with the same amount of search processing for motion vector search as in the prior art.

  In addition, by changing the shooting rate according to the amount of motion, motion vectors can be detected at the optimal rate for both fast and slow motions, and matching accuracy is improved compared to fixed rates. It becomes possible to make it.

  In addition, even when the imaging rate is changed from the reference frame rate, the system can provide images based on the reference frame rate, so that necessary image data can be displayed without affecting the user of the image. It is possible to perform highly accurate motion vector detection while supplying the user.

  In addition, when the imaging apparatus includes an image compression unit that uses a motion vector as in the present embodiment, even when the imaging rate is changed from the reference frame rate, the frame image data at the reference frame rate and the reference frame By performing image compression using motion vectors converted to rates, accurate motion vector detection results are used for compression coding without affecting the frame rate required by users of compressed images. And the amount of code can be reduced.

Further, when the imaging apparatus includes a moving body detection unit that uses a motion vector as in the present embodiment, even when the imaging rate is changed from the reference frame rate, the frame image data of the reference frame rate and the reference frame rate are changed. By performing image compression using the converted motion vector, it is possible to use accurate motion vector detection results for moving object detection without affecting the frame rate requested by the user of the compressed image. Become.
As a result, it is possible to improve the accuracy of moving object detection.

  Further, with respect to the imaging rate control method, a specific method can be specified from a plurality of setting methods, and a motion vector to be noticed can be changed based on the specified method. However, it is possible to specify a method for changing the imaging frame rate according to the purpose of use, and to perform motion detection according to the purpose of use of the user.

-Second Embodiment-
Next, a second embodiment of the present invention will be described.
In the first embodiment, the reference frame image for obtaining the motion vector is a frame image one frame before the target frame regardless of the imaging frame rate. However, if both the large and small portions of motion exist in one frame, the search for finding the motion vector is performed for the small portions of the motion by appropriately selecting the reference frame image. Can be omitted.
Note that the motion vector detection unit 13 in this embodiment includes a CPU, and its operation is controlled by software.

  Hereinafter, the flow of control in this example will be described with reference to the flowchart of FIG. 8, FIG. 9, and FIG. 1 of the first embodiment.

FIG. 8 is a flowchart showing the flow of processing of the motion vector detection unit 13.
FIG. 9 is a diagram specifically showing how the reference frame image is selected based on the processing flow of the motion vector detection unit 13 described in the flowchart of FIG.

  When the processing related to motion vector detection is started (step S801), the motion vector detection unit 13 first moves the motion of each block with respect to the current frame image to be processed based on the size of the past motion vector and the state of the change. The amount is estimated (step S802).

  The past motion vector here is a motion vector derived between frame images based on the reference frame rate (reference rate motion vector). When the imaging rate is different from the reference frame rate, motion vector synthesis is performed. This means a motion vector converted into a reference frame rate by the unit 14.

  When the motion amount for each block is estimated, the size of the motion vector search region is determined from the motion amount distribution state and the current imaging rate (step S803).

  As will be described later, for a block with a small amount of motion, it is possible to set the reference frame image used for motion vector detection in a thinned-out form rather than as a continuous frame.

  Therefore, if the size of the search area is constant, if there are a lot of thinned frames, there will be a surplus in processing time for search.

  Therefore, the margin time is estimated here in advance, and the search area used for one motion vector is expanded at this stage as necessary. This makes it possible to further improve the motion vector detection accuracy.

  When the size of the search area is set, selection of a processing target block (step S804) is started, and work for sequentially obtaining a motion vector is started.

  First, a reference frame image used for motion vector detection is determined from the assumed motion amount for the selected processing target block and the imaging rate currently used for imaging (step S805).

  At this time, when an image is captured as a reference frame at an imaging rate different from the standard frame rate, the detection process is repeated using all of the continuous captured frame images, or it is thinned out. It is determined whether to use in the case of, or what to do with the thinning interval when thinning out.

  Next, the motion vector detection unit 13 reads the reference frame determined from the frame memory 12 and performs a motion detection process (step S806).

  As a result, if the motion vector has been detected (step S807 / YES), it is confirmed whether the motion vector information sufficient to obtain the reference rate motion vector is available (step S808).

  As described above, when it is assumed that the detection processing is repeatedly performed using all the images of the continuous imaging frames, it may be assumed that not all of them are prepared (step S808 / NO).

  In such a case, the image currently used as the reference frame so that the next motion vector can be detected is set again as a detection target frame for detecting the next motion vector (step S814), and the motion is again detected. The vector detection process is repeated (step S806).

  As a result, when motion vector information sufficient to obtain the reference rate motion vector is obtained (step S808 / YES), the motion vector detection unit 13 transmits the motion vector information to the motion vector synthesis unit 14 (step S809). The motion vector synthesizer 14 that has received the motion vector information generates a reference rate motion vector.

  When motion vectors relating to a certain block are generated in this way, the motion vector detection unit 13 checks whether or not the motion vectors of all the blocks of the target image have been detected (step S810).

  If the motion vectors of all blocks have been detected (step S810 / YES), the motion vector detection result is transmitted to the imaging control unit 103 (step S815).

  The imaging control unit 103 that has obtained the detection result of the motion vector makes a determination regarding the change of the imaging rate from the result, and issues an instruction to the imaging rate control unit 104.

  As a result, imaging rate control according to the motion vector is performed, and the motion vector detection process ends (step S816). If the detection of the motion vectors of all the blocks has not been completed (step S810 / NO), the process returns to the selection of the target block (step S804) again, and the operation for obtaining the motion vectors regarding the unprocessed blocks is repeated. .

  Note that if motion vector detection fails when detecting a motion vector (step S807 / NO), a case where the setting of the reference image is inappropriate may be considered.

  For example, there is a case where the actual movement is larger than the movement amount estimated in step S802. In such a case (step S811 / YES), the motion vector can be detected by resetting the estimated motion amount (step S812) and changing the way of selecting the reference frame (step S805). .

  If the motion vector cannot be detected even if the assumed motion amount that can be set is maximized and the reference frame selection interval is minimized, the content displayed as the motion vector detection target block is newly added. There may be a case in which information does not exist in the previous frame due to factors such as occurrence.

  In such a case (step S811 / NO), the motion vector detection of the block is abandoned (step S813), and the process is changed to confirm whether there are still unprocessed blocks (step S810). Become.

The content described in FIG. 8 will be described with reference to FIG.
FIG. 9 is a diagram showing how a reference frame image used for motion vector detection is selected. In FIG. 9, (a) represents the frame image 70-a at a certain time t = T.

  Assume that the imaging interval based on the reference frame rate is N, and the current imaging rate is four times the reference frame rate. At that time, images taken at a frame rate of 4 times are represented as 70-a, b, c, d, and e in FIG. 9C, respectively.

  Moving objects 71, 72, and 73 exist on the frame image 70. In the frame image 70-a at time t = T, they are assumed to be located at 71-a (displayed by ▲), 72-a (displayed by ●), and 73-a (displayed by ■), respectively.

  These moving objects are 71-e (displayed by Δ), 72-e (displayed by ○), and 73-e (displayed by ○) in the frame image 70-e of t = T + N after the imaging interval based on the reference frame rate. (Displayed with □).

  FIG. 9B shows the position information of the search area and the past moving object for detecting each motion vector over the frame image 70-e of t = T + N after the imaging interval.

  Now, it is assumed that moving object 71-e, 72-e, 73-e reference rate motion vector in frame image 70-e is obtained at time t = T + N. At this time, since the reference rate motion vectors of the moving objects 71-a, 72-a, 73-a in the frame image 70-a at the time t = T have already been obtained, each moving object is based on this. Is estimated (corresponding to step S802 in FIG. 8).

  Specifically, in this embodiment, it is assumed that each moving object is moving at a constant speed, and the positions 71-a, 72-a, 73-a of the moving object in the frame image 70-a at time t = T. Then, using the reference rate motion vector of each moving object obtained at time t = T, the position of each moving object on frame image 70-e at time t = T + N is estimated.

  These positions are in the vicinity of 71-e, 72-e, and 73-e, respectively, in this embodiment. Then, when obtaining the motion vector of the block corresponding to the estimated position, the size of the reference rate motion vector of each moving object obtained at time t = T is reversed to obtain the reference frame image to be used. decide.

  For example, assume that the block at the position 72-e is selected as the target of the motion vector detection processing block in the frame image 70-e at time t = T + N (corresponding to step S804 in FIG. 8).

  At this time, the block at the position 72-e may have been in the vicinity of 72-a in the frame image 70-a at time t = T, which is the previous frame period by the method described above. Can be estimated.

  However, when a motion vector search region is considered with reference to the position 72-e, the search region in FIG. 9B is a range 74-d, and the position 72-a is outside the search area range. Become.

  Therefore, for the block including the point 72-e where the moving object 72 exists, from the relationship between the assumed amount of motion, the search area, and the imaging rate, first, one block before the first imaging frame before time N / 4. After selecting an image, a motion vector is detected, and by using the result, images of N / 4 shooting intervals are sequentially traced to obtain a motion vector between successive imaging frames. A policy of combining the results is determined.

  As a result, first, a region 74-d on the frame image 70-d at time t = T + (3/4) * N is selected as a reference image for obtaining a motion vector for the block at the position 72-e. (Corresponding to step S805 in FIG. 8).

  The motion vector detection unit 13 obtains, from the frame memory 12, information on the motion vector search region 74-d centered on the point 72-e ′ on the frame image 70-d corresponding to the point 72-e on the frame image 70-e. The motion vector is searched by reading and comparing with the information of the point 72-e on the frame image 70-e (corresponding to step S806 in FIG. 8).

  This is shown in FIG. The top row in FIG. 9D represents the setting of the motion vector search area for the moving object 72 and the state of motion vector detection.

  On 74-d, it is determined that the block located at 72-d has the highest association with the block at 72-e, and the motion vector 77-d is extracted (corresponding to step S807 / YES in FIG. 8). .

  When motion vectors are extracted in this way, the motion vector detection unit 13 then determines whether or not there is enough motion vector information to synthesize the reference rate motion vector for the block 72-e (step S808 in FIG. 8). Corresponding).

  Here, since all the information has not been collected yet (corresponding to step S808 / NO in FIG. 8), this time, the block located at the point 72-d on the frame image 70-D previously used as the reference frame image Is set as a block for obtaining a motion vector (corresponding to step S814 in FIG. 8).

  In addition, since it is already determined that the frame image 70-c immediately before 70-d is selected as the reference frame, the motion vector detection unit 13 sets the point 72- on the frame image 70-d. The information of the motion vector search area 74-c centered on the point 72-d 'on the frame image 70-c corresponding to d is read from the frame memory 12, and the information on the point 72-d on the frame image 70-d By comparing, a motion vector is searched (again corresponding to step S806 in FIG. 8). As a result, a motion vector 77-c is extracted.

  Thereafter, the same operation is repeated, and when the motion vector is detected by using the 70-a frame separated from the 70-e by the time N as the reference frame image, the motion of the reference rate is performed for the block 72-e. The motion vector information that can synthesize the vectors is collected as 77-a, b, c, and d (corresponding to step S808 / YES in FIG. 8).

  The obtained motion vector information is transferred to the motion vector synthesis unit 14 (corresponding to step S809 in FIG. 8), and the reference rate motion vector corresponding to the block at the position 72-e in the frame image 70-e at time t = T + N. As a result, the motion vector 77 shown in FIG. 9B is generated.

  When the reference rate motion vector for 72 is obtained, the motion vector for another block is then obtained (corresponding to step S810 / NO in FIG. 8).

  FIG. 9D shows the reference rate movement not only for the block at the position 72-e in the frame image 70-e at the previous time t = T + N but also for the blocks at the positions 71-e and 73-e. The state of vector extraction is also illustrated.

  The basic operation flow is the same as that in the block located at 72-e, but the estimated motion amount is smaller than that in the case of 72-e.

  As a result, two frames 70-c and 70-a are selected as reference frames in the case of 73-e, and a motion vector is obtained as a result of searching by setting 75-a and 75-c as motion vector search areas. As a result, 78-c and 78-a are obtained.

  Then, a reference rate motion vector 78 is obtained as a result of combining the two. Further, only 70-a is selected as the reference frame for 71-e, and the motion vector 79-a is obtained as a result of searching by setting 76-a as the motion vector search region.

  The motion vector 79-a is adopted as the reference rate motion vector 79 as it is. The search time required for extracting the reference rate motion vector for the blocks at positions 71-e and 73-e is reduced to about ½ and ¼ compared to the block located at 72-e.

  In this way, by selecting a reference frame used for creating a motion vector at an appropriate interval based on the size of the immediately preceding reference rate motion vector, for example, the imaging rate is increased for a fast motion block. It is possible to perform motion detection between frame image data, and it is possible to detect motion between frame image data having a large interval for blocks with small motion.

  As a result, no processing load is applied to blocks with small motion, so the entire search area can be increased accordingly, and the motion vector detection system for the entire image can be improved. become.

-Third embodiment-
Next, a third embodiment of the present invention will be described.
In the second embodiment, the image previously used as the reference frame in step S814 in FIG. 8 is set again as a detection target frame for detecting the next motion vector, and the motion vector detection process is repeated again. It was like that.

  However, in this method, since the same frame is used for the reference data in the near block, the processing is performed for each block, so that the frame data that can be used simultaneously is detected many times by detecting a plurality of motion vectors. Overhead may occur.

  In the third embodiment, this is improved and the concept of a processing target block group is introduced. The processing target block group is a group of adjacent blocks among blocks having the same imaging frame interval used as a reference frame because the assumed amount of motion is close.

  Hereinafter, the flow of processing in the motion vector 13 will be described with reference to the flowchart of FIG. Note that the basic processing flow is almost the same as that of the flowchart of FIG. 8 in the second embodiment, and therefore only the parts having characteristic differences will be described.

  In FIG. 10, when the size of the search area is set (step S1003), the selection of the processing target block group (step S1004) is started, and the operation for obtaining the motion vector is started sequentially.

  Here, as the processing target block group, since the magnitude of the assumed motion amount obtained in step S1002 is close, the adjacent blocks among the blocks having the same imaging frame interval used as the reference frame are collected. Is.

  Then, a reference frame image used for motion vector detection is determined for the processing target block group from the assumed motion amount common to the processing target block group and the imaging rate currently used for imaging (step S1005).

  Next, the motion vector detection unit 13 reads the reference frame determined from the frame memory 12, and performs a motion detection process for one block included in the processing target block group (step S1006). As a result, when the motion vector has been detected (step S1007 / YES), it is determined whether or not all the blocks in the processing target block group have been processed. If the processing has not been completed (step S1017 / NO), the process moves to the next block, and the motion vector is detected using the same reference frame image (step S1006).

  At this time, the search range of the reference frame image to be used is also changed due to the movement of the motion detection target block, but since the image used in the previous search has already been read, this time The image can be searched by simply reading from the frame memory 12 image data that covers only the difference required for the search.

  When all the processes are completed for the blocks included in the processing target block group (step S1017 / YES), the motion vector detection unit 13 confirms whether the motion vector information sufficient for obtaining the reference rate motion vector is available (step S1008). ), The process proceeds below, and after detecting the motion vectors of all blocks of the target image (step S1010 / YES), the motion vector detection result is transmitted to the imaging control unit 103 (step S1015). In response to the result, the imaging control unit 103 instructs the imaging rate control unit 104 to change the imaging frame rate as necessary.

  Note that if motion vector detection fails when detecting a motion vector (step S1007 / NO), it is possible that the setting of the reference image is inappropriate. For example, there is a case where the actual movement is larger than the movement amount estimated in step S1002.

  In such a case (step S1011 / YES), the estimated motion amount is reset (step S1012). However, by resetting the estimated motion amount, the selection of the processing target block group is performed again (step S1012). S1004). Since the contents other than these are the same as the contents of FIG. 8 in the second embodiment, the description thereof will be omitted.

  In this way, blocks for which motion vectors are obtained are grouped based on the size of the immediately preceding reference rate motion vector and its positional relationship, and the reference frame used to create a motion vector for each group is used as the immediately preceding reference rate. By selecting an appropriate interval based on the size of the motion vector, etc., while reducing the amount of image data read from the memory, for example, for a fast-moving block, motion detection between frame image data with an increased imaging rate Thus, it is possible to detect motion between frame image data with a large interval for blocks with small motion.

  As a result, it is possible to reduce the processing time overhead required for memory reading, so it is possible to lengthen the time required for the search process, and ultimately improve the accuracy of motion vector detection in the entire image. It becomes possible to improve.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and a computer such as the system reads and executes the program codes from the storage medium. Is also achieved.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  In addition, the case where the functions of the above-described embodiment are realized by performing part or all of the actual processing by an OS or the like running on the computer based on the instruction of the program code read by the computer. It is.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion unit connected to the computer, the CPU or the like performs actual processing based on the instruction of the program code, and the above-described processing is performed. The case where the functions of the embodiment are realized is also included.

It is a figure which shows the structural example of the imaging device in the 1st Embodiment of this invention. It is a figure which shows the system using the imaging device in the 1st Embodiment of this invention. It is a figure which shows the mode of the detection of the motion vector in the motion vector detection part in the 1st Embodiment of this invention. It is a figure which shows the detail of the search area | region of the motion vector in the 1st Embodiment of this invention. It is a figure which shows the mode of the motion vector detection after changing the imaging frame rate in the 1st Embodiment of this invention. It is a figure which shows the detail of a process of the imaging frame data in the 1st Embodiment of this invention. It is a figure which shows the detail of the user interface for performing the setting regarding the imaging frame rate in the 1st Embodiment of this invention. It is a flowchart which shows the flow of a process in the motion vector detection part in the 2nd Embodiment of this invention. It is a figure which shows the mode of selection of the frame image for a reference used for the motion vector detection in the 2nd Embodiment of this invention. It is a flowchart which shows the flow of a process in the motion vector detection part in the 3rd Embodiment of this invention.

Explanation of symbols

1: Image pickup device 2: Network network 3: Display terminal 10: Image pickup unit 11: Frame control unit 12: Frame memory 13: Motion vector detection unit 14: Motion vector synthesis unit 15: Compression processing unit 16: Network control unit 17: Network Interface 18: Motion detection unit 19: Representative motion vector setting unit 20: Video output control unit 21: Video output interface 22: Monitor 30: Viewer 31: Display window 32: Menu 33: Mouse cursor 34: Connected camera name display unit 36: Priority area designation frame 37: Object designation frame 44, 51, 52, 74, 75, 76: Motion vector search area 61, 62, 71, 72, 73: Moving object 64, 66, 69, 77, 78, 79: Movement vector

Claims (13)

Imaging means for converting a subject image into an electrical signal;
Means for changing the imaging rate of the imaging means;
Developing means for processing an electrical signal from the imaging means and outputting an image signal;
Storage means for storing the image signal for a plurality of screens as frame image data for each screen;
A motion vector detection means for detecting a motion vector in units of blocks between a plurality of frame image data stored in the storage means;
Motion vector synthesizing means for synthesizing a plurality of the detected motion vectors;
The imaging rate of the imaging unit is changed to a frame rate different from a reference frame rate according to the detected motion vector or the magnitude of the motion vector synthesized by the motion vector synthesis unit, and if changed, the An imaging apparatus characterized in that a motion vector converted to the reference frame rate is synthesized using motion vectors detected from a plurality of frame image data photographed at different frame rates. A representative motion vector for determining a representative motion vector obtained by a predetermined method from a plurality of the detected motion vectors existing in one screen or a motion vector synthesized so as to be converted into the reference frame rate. It further has setting means,
The imaging apparatus according to claim 1, wherein the value of the different frame rate is changed based on the magnitude of the representative motion vector.   When detecting a motion vector from a plurality of frame image data photographed at the different frame rate, based on the magnitude of the motion vector converted to the immediately preceding reference frame rate, for reference used for motion vector detection The imaging apparatus according to claim 1, further comprising means for selecting an image from the plurality of frame image data.   When detecting a motion vector from a plurality of frame image data shot at the different frame rates, based on the size of the motion vector converted to the immediately preceding reference frame rate and the position of the local region where the motion vector should be detected 3. The apparatus according to claim 1, further comprising means for grouping a plurality of local regions and selecting a reference image to be used for motion vector detection in the group unit from the plurality of frame image data. The imaging device described in 1. And further comprising at least one of means for synthesizing one frame image and means for selecting one frame image from the plurality of frame image data accumulated by the storage means,
5. The image pickup apparatus according to claim 1, wherein frame image data having a reference frame rate is generated even when the image is shot at the different frame rate. 6. An image compression means using a motion vector;
Even when the image compression unit is photographed at the different frame rate, the image compression unit uses the generated frame image data of the reference frame rate and the motion vector converted to the reference frame rate, based on the reference frame rate. The image pickup apparatus according to claim 1, wherein the image compression is performed. It further has a moving object detection means,
The imaging apparatus according to claim 1, wherein the moving object detection is performed using a motion vector converted into the reference frame rate. Imaging means for converting a subject image into an electrical signal, means for changing the imaging rate of the imaging means, developing means for processing an electrical signal from the imaging means and outputting an image signal, and the image signal on one screen Storage means for storing a plurality of frames as frame image data for each frame; motion vector detection means for detecting a motion vector in units of blocks among a plurality of frame image data stored in the storage means; and the detected motion vectors And a plurality of frames when captured at the different frame rates, the imaging rate of the imaging means can be changed to a frame rate different from a reference frame rate according to the detected motion vector. A method of synthesizing a motion vector converted to the reference frame rate using a motion vector detected from image data. And a representative motion for determining a representative motion vector obtained by a method designated in advance from a plurality of the detected motion vectors existing in one screen or a motion vector synthesized so as to be converted into the reference frame rate An image pickup apparatus control method for instructing an image pickup apparatus having a vector setting means to perform a setting related to the image pickup rate from a display device displaying an image output from the image pickup apparatus,
An imaging apparatus control method comprising: means for designating a specific method from a plurality of setting methods, and generating the representative motion vector based on the specific method set by the setting unit.   One of the plurality of setting methods is a high-speed operation priority method that automatically determines and sets the different frame rate from the magnitude of the motion vector with respect to a motion vector converted into a plurality of reference frame rates. The imaging apparatus control method according to claim 8, further comprising:   In one of the plurality of setting methods, a user sets a region for an output image displayed on the display device, and the difference is made based on one or a plurality of motion vectors existing in the region. The imaging apparatus control method according to claim 8, further comprising a priority area designation method for automatically determining and setting the frame rate.   In one of the plurality of setting methods, a user designates an object for an output image displayed on the display device, and the different frame rate is automatically set based on a motion vector of the designated object. 9. The imaging apparatus control method according to claim 8, further comprising an object designation method that is determined and set. Changing the imaging rate when converting the subject image into an electrical signal;
A development step of processing an electrical signal from the imaging means and outputting an image signal;
A storage step of storing the image signal for a plurality of screens as frame image data for each screen;
A motion vector detection step of detecting a motion vector in units of blocks between a plurality of frame image data stored in the storage means;
A motion vector synthesis step of synthesizing a plurality of the detected motion vectors,
The imaging rate is changed to a frame rate different from a reference frame rate in accordance with the detected motion vector or the magnitude of the motion vector synthesized by the motion vector synthesis step, and if changed, the different frame rate An imaging control method comprising: combining a motion vector converted into the reference frame rate using motion vectors detected from a plurality of frame image data photographed by the method.   A program for causing a computer to execute the imaging control method according to claim 12.

Images