JP2006279568A - Image-pickup device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image-pickup device in which attribute information on a mobile object is provided automatically to an image region occupied by the mobile object by detecting the mobile object caught within a visual field of a camera whose position and posture are fixed with respect to the image-pickup device for taking an image of the mobile object. <P>SOLUTION: The image-pickup device includes an image-pickup device 5 for generating image information, an RF tag 1 that is mounted to the mobile object to hold unique information on the mobile object, a directive antenna for enabling a position of the RF tag 1 to be recognized by performing radio communication, an RF tag reader/writer 3 for performing reading-in or the like of the RF tag 1 and unique information on the mobile object via the antenna 2, a mobile object detecting means 6 for detecting a mobile-object region from the image information, a clipped region/scaling ratio calculating means 7 for extracting a clipped portion containing the mobile-object region and then calculating the scaling ratio depending on the outline size of a contour of the clipped region to make the scaling ratio calculated correspond to the clipped region, and attribute information providing means 4, 8 each generating attribute information based on the unique information output from the RF tag reader/writer 3 to provide the attribute information made to correspond to the clipped portion and outputting video information attached with the attribute provided with the attribute information and scaling ratio information. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that performs moving body imaging, and more particularly to an imaging apparatus that simplifies video editing work by automatically generating attribute information of each scene of the video during shooting.

  In recent years, there has been a demand for an apparatus capable of processing and editing a large amount of video material in a short time as digital contents can be easily distributed and broadcast by using an Internet line or the like. For example, it corresponds to a device for generating a replay video by cutting out a highlight scene etc. at the same time as performing a live broadcast on a sports video or the like.

  Conventionally, there is a nonlinear editing apparatus as a typical apparatus for video editing. In this device, the video captured by the video camera and the video recorded on the tape are converted into digital data, which is taken into a hard disk of a personal computer and processed and edited. For example, to cut out a soccer goal scene, play the video stored on the hard disk, set the start and end positions of the cut manually, and save the section as a separate data file or record it on videotape. To create replay video. However, in the non-linear editing apparatus, it is necessary to manually perform editing operations such as specifying a cut-out section and storing a replay video, which requires a great deal of labor and cost.

  In view of this, several methods have been proposed in which attribute information is given to each scene in a video and the extracted section is automatically specified by searching the attribute information.

  For example, as one of them, there is a method of recognizing an ID (Identification) signal emitted from an infrared LED (Light Emitting Diode) attached to an object to be photographed and indexing the accumulated video data in real time (for example, non-patent) Reference 1). However, since the LED is used as a signal source, there is a problem that the ID information cannot be received when there is no light receiver in the light emitting direction or when there is an obstacle between the LED and the light receiver.

  As another method, when recording a camera image, there is a method for preventing falsification of an image by simultaneously reading ID information such as an RF (Radio Frequency) tag attached to a subject and storing it together with image data ( For example, Patent Document 1). In the invention, a signal is received only from an RF tag to which the camera is directed by a directional antenna. However, there is no specific means regarding a method for associating the directional antenna with the camera field of view. That is, there is a problem that even if ID information of the RF tag is recorded, it is not guaranteed whether or not the corresponding photographing target is shown.

  As another method, there is a method of making it easy to find a desired article by detecting the direction of the RF tag using a directional antenna and irradiating light beams such as LEDs in the same direction (for example, patents). Reference 2). However, although a directional antenna is used, there is no means for associating the antenna with the camera or associating the RF tag ID information with the moving object.

Furthermore, there is a method of tracking where a moving object exists in an image using an image processing method such as a local correlation method (for example, Non-Patent Document 2). However, there is a problem that it is impossible to automatically give in real time what kind of attribute each moving object has.
JP 2000-261551 A JP 2002-271229 A Tsuno, Ito, Matsuguchi, SIDney Fels, Utsumi, Suzuki, Nakahara, Iwasawa, Kogure, Mase, Tomita, “Collaborative Interaction Recording with Multiple Sensors”, Interaction 2003 Proceedings Information Processing Society of Japan, pp. 255-262. Morita, “Motion Detection and Tracking by Local Correlation”, IEICE Transactions D-II, Vol. J84-D-II, No. 2, pp. 299-309, 2001.

  With the above-described conventional technique or a simple combination of them, a moving object with an RF tag in the camera field of view cannot be detected. Therefore, attribute information can be assigned to an image area occupied by the target moving object, There is a problem that zooming and tracking shooting, extraction of a video section where a target moving object exists cannot be performed.

  Therefore, the present invention provides an imaging device that detects a moving object reflected in the field of view of a camera whose position and orientation are fixed, and automatically assigns attribute information of the moving object to an image area occupied by the moving object. For the purpose.

  An imaging apparatus according to a first aspect of the present invention is an imaging apparatus for outputting video information, comprising an imaging means for generating image information by imaging a fixed desired field of view, and a moving object that is an imaging target An RF tag that is attached to the wireless tag and that can communicate with the RF tag, and that is capable of recognizing the position of the RF tag, and the directional antenna. An RF tag reader / writer that reads or writes the RF tag and the specific information via a moving object, a moving object detection unit that detects a moving object region from the image information from the imaging unit, and at least the movement from the image information A cutout area that extracts a cutout area including an object area, calculates a scaling ratio to a desired size according to the outline size of the cutout area, and associates the cutout area with the cutout area Attribute-added video to which attribute information is generated based on the unique information from the RF tag reader / writer and attached in association with the cut-out area, and the attribute information and the enlargement / reduction ratio information are assigned. An attribute information adding unit that outputs information is included.

  According to the first aspect of the invention, the cutout area including the moving object area and the attribute information from the RF tag attached to the moving object are associated with each other, and further output as attributed video information associated with the scaling ratio. Therefore, by using the assigned attribute information, it is possible to obtain attribute-added video information that can be easily edited, and to obtain a zoom video of a desired moving object that has been tracked.

  According to a second aspect of the present invention, there is provided the photographing apparatus according to the first aspect, wherein the moving object detecting means includes a background difference method detecting means for detecting a first moving object region by a predetermined background difference method; The detection target range is limited based on one moving object region, and a local correlation method detecting unit is used to detect the second moving object region by a predetermined local correlation method.

  According to the second aspect of the present invention, the moving object region is roughly determined by the background subtraction method, then the detection target range of the moving object is limited based on the determined moving object region, and the local correlation method Since the moving object area is determined finely, it is resistant to noise such as fluctuations in brightness, and image information with attributes can be obtained in a shorter processing time.

  According to a third aspect of the present invention, there is provided the photographing apparatus according to the first aspect, wherein the cut-out area / enlargement / reduction ratio calculating means includes the RF tag whose position is recognized by the directional antenna within the outline of the moving object area. And when the moving speed and moving direction of the specific position in the moving object area coincide with the moving speed and moving direction of the RF tag, the moving object area and the unique information are associated with each other. A cutout area is extracted.

  According to the third aspect, when the moving speed and moving direction of the specific position in the moving object area match the moving speed and moving direction of the RF tag, the attribute information from the RF tag is added to the cut-out area. Thus, it is possible to obtain video information with attributes in which the moving object and the RF tag can reliably correspond to each other.

  According to a fourth aspect of the present invention, there is provided the photographing apparatus according to the first or second aspect, wherein the moving object detecting means is configured to detect the moving object with respect to the image information based on the pre-existing position information of the RF tag. The region detection target range is limited.

  According to the fourth aspect, since the detection target range of the moving object region is limited, the moving object region detection process can be performed at high speed.

  According to a fifth aspect of the present invention, there is provided the photographing apparatus according to the first aspect, wherein the attribute information providing means includes means for storing the attribute-added video information, and the attribute-related information related to the attribute information. Based on the setting, only the video section associated with the attribute information is extracted and output from the stored video information with attribute.

  According to the fifth aspect, since only the video section associated with the attribute information can be extracted from the stored video information with attributes, the attribute information given when selecting or extracting the desired shooting target or video scene Editing can be easily performed by using.

  According to the present invention, a moving object reflected in the field of view of a camera can be detected, and attribute information of the moving object can be automatically given to an image area occupied by the moving object. When selecting or extracting video scenes, the editing operation can be automated by using the assigned attribute information.

  In addition, it is possible to automatically perform follow-up shooting while zooming up on a subject such as a specific person wearing a tag.

Example 1
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of a photographing apparatus according to an embodiment of the present invention.

In FIG. 1, an imaging apparatus includes an RF tag 1, a directional antenna 2, an RF tag reader / writer 3, an attribute information generation unit 4, an imaging unit 5, a moving object detection unit 6, a cutout area / scale ratio. The calculation unit 7 and the attributed video generation unit 8 are included.
[RF tag 1]
The RF tag 1 includes at least an IC chip with a built-in memory (or an external memory may be used) and an antenna (such as a coil antenna), and can communicate with the RF tag reader / writer 3 in a non-contact manner by using wireless communication. It can be a device. As the RF tag 1, an RF tag (passive tag) of a type that generates driving power from electromagnetic waves transmitted from the RF reader / writer 3 without incorporating a power source such as a battery, or an RF tag of a battery built-in type ( Any of active tags) may be used. The place where the RF tag 1 is attached to the moving object is preferably a place where the movement of the moving object is as small as possible. For example, in the case of a person, the RF tag 1 is preferably attached to the head or chest where the movement is relatively stable.

Further, in the memory in the RF tag 1, for example, the name, characteristics (color, shape, ID number, etc.), specifications, or the like of the target are described according to the subject to which the RF tag 1 is attached. Unique information such as a network address is stored in advance. Further, when a writable RF tag is used, it is possible to rewrite the data contents stored in the memory in the RF tag 1 as necessary during shooting with the camera. .
[Directional antenna 2]
The directional antenna 2 is an antenna that can wirelessly communicate with the RF tag 1 and variably control the directivity so that the position of the RF tag 1 can be recognized. As the directional antenna 2, for example, an adaptive array antenna having a variable directivity pattern is used. As shown in FIG. 2, in the adaptive array antenna, the directivity pattern can be controlled by weighting signals received by a plurality of antenna elements and appropriately changing the weights.

  When the direction of arrival of radio waves from the RF tag 1 with respect to the directional antenna is θ and the antenna elements are arranged in a straight line at equal intervals D, the propagation to the i-th antenna when compared with the reference antenna Delay time τi (θ) is

で表わされる。なお、ｃは、伝播速度である。変調信号をm(n)、搬送波の角周波数をω、第i番目のアンテナ出力をx_i(n)、ｎを時間とすると、

It is represented by Note that c is the propagation speed. If the modulation signal is m (n), the angular frequency of the carrier wave is ω, the i-th antenna output is x _i (n), and n is time,

となる。

It becomes.

  When the desired signal waveform is known and can be used as the reference signal d (n), the weighting factor can be determined by minimizing the mean square error with the array output signal y (n). Let u (n) be an error component such as interference or noise.

各アンテナの入力信号x_i(n)、重み係数ｗ_i(θ)をベクトルで表現し、各エレメントの合成出力をy(n)として、上式に代入する。

The input signal x _i (n) and the weighting coefficient w _i (θ) of each antenna are expressed by vectors, and the combined output of each element is substituted by y (n) into the above equation.

なお、添え字Tは転置、Hは複素共役転置を表わす。このとき、誤差成分の２乗の期待値Jは次式で表わされる。

Note that the subscript T represents transposition, and H represents complex conjugate transposition. At this time, the expected value J of the square of the error component is expressed by the following equation.

各アンテナの入力信号の自己相関行列をR_xx=E{x(n)x(n)^H}、参照信号と各アンテナの入力信号との相互相関行列をr_xd=E{x(n)d*(n)}と定義すると、最適な重みは、期待値Jを重みベクトルWで偏微分することにより、

The autocorrelation matrix of the input signal of each antenna is R _xx = E {x (n) x (n) ^H }, and the cross correlation matrix of the reference signal and the input signal of each antenna is r _xd = E {x (n) d When defined as * (n)}, the optimal weight is obtained by partial differentiation of the expected value J with the weight vector W,

で与えられる。

Given in.

As described above, the radio wave arrival direction θ can be associated with the weighting factor W (θ). Therefore, when changing the directivity of the antenna in the specified range θ ₁ <θ <θ ₂ , by gradually changing the corresponding weighting factor within the range of W (θ ₁ ) to W (θ ₂ ), The direction of the antenna can be scanned. For example, when the direction of the antenna is swung with a step size Δθ,

のように算出しておき、刻み値k・Δθに応じて、重み係数Wを変化させることで、アンテナの指向性パターンをコントロールすることができる。

The antenna directivity pattern can be controlled by changing the weighting coefficient W according to the step value k · Δθ.

Further, the directional antenna may be configured by a mechanism that mechanically rotates the antenna using an antenna having a fixed directional pattern such as a Yagi antenna. In the case of this configuration, processing equivalent to the above-described adaptive array antenna can be realized by changing the rotation angle θ of the antenna in the specified range θ ₁ <θ <θ ₂ .

Furthermore, the positional relationship between the directional antenna and the camera will be described. When detecting a moving object to be detected, a process of associating the direction of radio wave arrival θ from the RF tag 1 obtained by the above-described method with a gaze area in a camera image described later is performed. At this time, the moving object, the antenna, and the camera need to be in the same straight line. Since the position of the moving object is unspecified, it is desirable that the center position of the antenna and the focal position of the camera lens coincide on the vertical axis in order to satisfy the condition of the same straight line.
[RF tag reader / writer 3]
When reading data from the RF tag 1, the RF tag reader / writer 3 demodulates the signal modulated by the data of the RF tag 1 from the signal y (n) output from the directional antenna, and further performs AD conversion. By applying, unique digital data based on the RF tag 1 is obtained. The obtained digital data is original data representing attributes relating to video, and is output to the attribute information generation unit 4. Further, in the case of data writing to the RF tag 1, digital data is written to the memory in the RF tag 1 by performing a process reverse to that at the time of data reading.
[Attribute information generation unit 4]
The attribute information generation unit 4 applies predetermined rules to the digital data output from the RF tag reader / writer 3 and converts the digital data into attribute values, thereby generating attribute data. An example of conversion to an attribute value will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of conversion from unique data of RF tags to attribute data.

  For example, when the imaging device is applied to a marathon competition, the memory of the RF tag 1 stores numbers and character string data for a racer's unique data such as a bib number, a competition date, a start time, and an event. After these numerical values and character string data are read by the RF reader / writer 3, they are converted into specific attribute values as shown in FIG. Needless to say, when the RF tag 1 has a large memory capacity and can be stored in the form of the attribute value itself (for example, coded information), the conversion is unnecessary.

Note that attribute data is generated only from digital data corresponding to a valid RF tag 1. In this case, the validity or invalidity of the RF tag 1 is determined as follows. The upper and lower limits of the viewing angle of the camera currently being photographed are θmax and θmin, and the angle of the directional antenna when the reading operation is performed on the RF tag is θ _R (= θ ₁ + k · Δθ). θ _R is

を満たすときに(図４に示すケース)、そのＲＦタグ１を有効と判定する。
〔撮像部５〕
撮像部５は、固定された所望の撮影視野を撮影して画像データを生成するカメラであり、例えばＣＣＤカメラなどのテレビカメラが使用される。画像データは、一定のサンプリング時間間隔 (例えば、３３ｍｓ）で取得され、動画像が生成される。

When the condition is satisfied (case shown in FIG. 4), the RF tag 1 is determined to be valid.
[Imaging unit 5]
The imaging unit 5 is a camera that captures a fixed desired field of view and generates image data. For example, a television camera such as a CCD camera is used. The image data is acquired at a constant sampling time interval (for example, 33 ms), and a moving image is generated.

One pixel of image data is a gradation value of a red component (R), a green component (G), and a blue component (B) in the case of a color image, while in the case of a monochrome image, the luminance gradation value. Given in. For example, the gradation values of the red component (R), green component (G), blue component (B), and luminance (I) of the pixel at coordinates (x, y) indicated by integers x and y are digital values R, respectively. (X, y), G (x, y), B (x, y), and I (x, y). For convenience of explanation, luminance (monochrome gray value) is used as a pixel value used in the subsequent calculations.
[Moving object detector 6]
The moving object detection unit 6 detects and extracts a moving object region that is a region of a moving object included in the image data from the image data output from the imaging unit 5. As illustrated in FIG. 5, the moving object detection unit 6 combines a background difference method detection unit using a predetermined background difference method and a local correlation method detection unit using a predetermined local correlation method.

  The background difference method is a method in which an image that does not include a moving object is prepared in advance as a background image, and then a difference from an image that includes a moving object is obtained, and a portion having a large difference is used as a moving object. However, this method can be processed in a shorter processing time, but has a drawback that it is vulnerable to noise such as brightness fluctuations.

  In the other local correlation method, an image is first divided into grid-like blocks (for example, 8 × 8 pixels). Next, a correlation value between neighboring blocks is obtained between a frame at a certain sampling time and a frame acquired at the next sampling time, and it is determined whether or not the block has moved according to the magnitude of the value. . However, although this method has a large amount of calculation, the image size cannot be increased, and the distance between blocks for which correlation calculation is performed is limited. It is more resistant to noise than the method.

Therefore, in the present invention, first, a roughly moving object region is obtained by the background subtraction method, and then the detection target range of the moving object is limited based on the obtained moving object region, and more detailed movement is performed by the local correlation method. A procedure for determining the object area is used. Next, processing by these methods will be described.
[Background difference method]
First, a moving object area that is a moving area is detected from the acquired image by the background subtraction method. FIG. 6 is a flowchart showing processing by the background subtraction method, and processing for detecting a moving object region by the background subtraction method will be described with reference to FIG. The following parenthesized symbols correspond to the parenthesized symbols in the figure.

  (a) First, image data obtained by the imaging unit and converted into a bitmap format is acquired at a constant period (every sampling time).

  (b) Next, the time-series data of each pixel is passed through a low-pass filter. By performing this processing, for example, when a low-pass filter having a low cut-off frequency is used ((b1) in the figure), an image of an object having no motion, that is, a background image can be obtained. Further, when the cutoff frequency is made higher than the above ((b2) in the figure), it is possible to acquire an image excluding the influence of the flickering of the fluorescent lamp while leaving a moving object. Therefore, in the subsequent processing, an image to which a low cut-off frequency is applied is used as a background image, while an image to which a high cut-off frequency is applied is used as the current image for current sampling.

  (c) The current image and the background image are each divided by sub-partitions (squares of n × n pixels), and the sum T of the differences between pixels at the same position in each sub-partition is calculated for each sub-partition. To calculate. The calculation formula is as follows.

なお、Ｉ^Ｃ(x、y)は現画像における画素位置(x、y)の白黒濃淡値を、Ｉ^Ｂ(x、y)は
背景画像における画素位置(x、y)の白黒濃淡値を、R_subはサブ区画を表わす。

Note that I ^C (x, y) is the black and white gray value of the pixel position (x, y) in the current image, and I ^B (x, y) is the black and white gray value of the pixel position (x, y) in the background image, R _sub represents a sub-partition.

When the calculated value T is equal to or greater than the set value (threshold value) Tth, it indicates that an object different from the background image exists, and it is determined and extracted as a sub-section with motion. By comparing the sub-partitions with respect to all regions of the image data, it is possible to detect all regions with motion.
[Local correlation method]
In the local correlation method, the sub-compartment in which the sub-part determined to have motion by the background subtraction method has moved from which sub-part of the previous frame image is searched by correlation calculation between blocks. FIG. 7 is a diagram showing sub-partitions, sub-partition templates, and tracking blocks. In the figure, the acquired image is image data acquired at a constant cycle by the background subtraction method, the sub-partition is composed of a square of vertical n pixels × horizontal n pixels, and the sub-partition template is a sub-partition of the same object. Of moving objects (all sub-sections detected by the background subtraction method).

  The tracking block is an area set so as to surround the sub-partition template (arranged so that the boundary line of the tracking block is outside the m pixels), and the search by the local correlation method is performed on the area surrounded by the tracking block. Executed in.

  FIG. 8 is a flowchart showing processing by the local correlation method, and processing for detecting a sub-partition having motion by the local correlation method will be described with reference to FIG. The following parenthesized symbols correspond to the parenthesized symbols in the figure.

  (a) In the local correlation method, the sub-partition with motion detected by the background subtraction method is targeted. By performing such two-stage extraction processing, it is possible to remove sub-partitions erroneously detected due to the influence of noise or the like.

(b) Next, the correlation calculation is performed to determine which sub-section S _q (x ′, y ′) of the previous frame image has moved from the sub-section S _p (x, y) of interest in the current frame image. Search using. The correlation value C between the sub-partitions S _p and S _q is calculated by the following equation.

上式において、Ｉ^ｐ(x、y)、Ｉ^ｑ(x’、y’)は各サブ区画S_p、S_q内の画素の白黒濃淡値を、また、R_p、R_qはサブ区画領域内であることを表わす。

In the above ^{formula, I p (x, y)} , I q (x ', y') is the sub-zone S _p, the black-and-white grayscale value of the pixels in S _q, also, R _p, R _q is sub divided areas Indicates that it is within.

  (c) It is determined whether or not the candidate obtained in (b) is a moving sub-partition by a predetermined zero comparison method (described in Non-Patent Document 2). In the zero comparison method, first, the correlation value C0 between the sub-part of the previous frame at the same position as the sub-part of interest and the minimum value C1 of the correlation value obtained from the correlation calculation with all the sub-parts within the scanning range Find the difference Q from.

次に、移動を判定するための設定値（閾値）Ｍthと比較し、

Next, compared with a set value (threshold value) Mth for determining movement,

であれば、注目しているサブ区画S_p(x、y)が、移動サブ区画であると判定する。

If so, it is determined that the focused sub-partition S _p (x, y) is a moving sub-partition.

  (d) The section determined as the moving sub section is set as an effective area.

  (e) The speed and direction of movement are calculated from the positional relationship between the sub-part of the previous frame and the sub-part of the current frame of interest. The calculated speed and direction are compared with the adjacent upper, lower, left and right sub-sections. At this time, it is examined whether or not the difference in speed and direction is a certain value or less.

  (f) If it is less than a certain value, it is determined that the vehicle is moving at the same speed, and is combined with adjacent sub-partitions and assigned one tracking block.

  (g) If the value is above a certain value, it is determined that each sub-partition is a region of a different object, and a new tracking block in which the target sub-partition becomes the center position is determined for each sub-partition. create.

  (h) One tracking block is defined for a set of adjacent sub-partitions (hereinafter referred to as a sub-partition template). The range of the tracking block is set so as to be a constant size (m pixels) from the top, bottom, left, and right ends of the sub-partition template.

    (h-1) A tracking block setting method will be described. The upper, lower, left and right ends of the sub partition template are set as xsub_min, xsub_max, ysub_min, and ysub_max. At this time, the range of the tracking block is given as follows using the pixel value m (> 0) predetermined by the operator.

また、追跡ブロックの中心位置を次式で与える。

The center position of the tracking block is given by the following equation.

(h-2) １つ前のフレームk-1で求めた追跡ブロックの中心位置の移動速さと移動方向ベクトルを、

(h-2) The moving speed and moving direction vector of the center position of the tracking block obtained in the previous frame k-1 are

とおき、現フレームkの追跡ブロックの中心位置の移動速さと移動方向ベクトルを、

And the moving speed and moving direction vector of the center position of the tracking block of the current frame k,

とおく。

far.

        (h-3) When the difference in the moving speed is equal to or greater than the threshold value or the inner product of the moving direction vectors is equal to or less than the threshold value, it is regarded as another tracking block and is excluded from the tracking target. Tracking target condition is

で与えられる。

Given in.

(h-4) When the tracking block satisfies the tracking target condition during a continuous f _ok frame, it is determined that the region is a moving object. For example, if fok = 3, it is assumed that the object is a moving object if the above tracking target condition is satisfied for three consecutive frames.

  (i) If the tracking target condition is satisfied, the sub-partition template is regarded as a moving object.

By following the above procedure, a moving object region can be extracted from the acquired image.
[Cutout area / scale rate calculator 7]
The cut-out area / enlargement / reduction ratio calculation unit 7 extracts a cut-out area including the moving object area from the moving object detection unit 6 from the image data, and calculates the enlargement / reduction ratio to a desired size according to the outline size of the cut-out area. To associate with the cutout area. First, a method for associating the RF tag 1 detected by the directional antenna 2 with the moving object (runner wearing the RF tag) region extracted by the above-described method will be described below.

FIG. 9 is a diagram for explaining the association between the RF tag and the moving object region. The diagram showing the relationship between the image size and the antenna direction in FIG. 9A shows a camera, an image acquired by the camera, and an RF tag. FIG. 2 shows a schematic diagram overlooking the antenna detection angle θ _{R of} the vehicle, the runner, and the like. Further, the diagram showing the relationship between the RF tag position and the center position of the moving object in FIG. 10B is an enlarged view of the image portion in FIG. In the following description, for convenience of explanation, the RF tag and the moving object are associated with each other in the horizontal direction. However, since they can be associated with each other in the vertical direction, the description in the vertical direction is omitted.

  Since the antenna directions θmax and θmin coincide with the upper and lower limits of the camera viewing angle in the horizontal direction, they are associated with the left and right ends xmax and xmin of the image, respectively. Using the left end xmin of the image as a reference, the center position in the x direction of the moving object (shaded portion) is xg, and the center position xg matches the center position of the sub-partition template or tracking block described in (h-1). .

移動物体が装着しているＲＦタグの画像領域における位置との関係から、次式が成立すると考えられる。

From the relationship with the position in the image area of the RF tag attached to the moving object, it is considered that the following equation is established.

Δｘは、アンテナの検出誤差や移動物体の検出誤差などに依存する定数である。同様に、画像領域における、移動物体の速度ｖgと、ＲＦタグの移動速度ｖ_θは、

Δx is a constant that depends on the detection error of the antenna, the detection error of the moving object, and the like. Similarly, the velocity vg of the moving object and the velocity v _θ of the RF tag in the image area are

のような関係式を満たす。そのことから、移動物体の中心位置ｘgでの速度ｖgと、ＲＦタグの検出方向θ_Rでの移動速度ｖ_θとが上記の式を満たせば、そのＲＦタグは、検出された移動物体に装着されているものと見做すことができる。

The relational expression like Since the, the velocity vg at the center position xg of a moving object, if the moving speed v _theta in the detection direction theta _R of RF tag satisfies the above equation, the RF tag is attached to the detected moving object It can be regarded as being done.

  As described above, when the RF tag and the moving object are associated with each other, the extraction region (ROI: Region of Interest) is determined as follows. When the upper, lower, left and right ends of the sub-partition template representing the moving object region as shown by the hatched portion in FIG. 9B are placed as xsub_min, xsub_max, ysub_min, ysub_max, It can be represented by a rectangle connecting (xsub_min, ysub_max), (xsub_max, ysub_min), (xsub_max, ysub_max) (for example, a thick line frame in FIG. 9B).

  The zoom ratio z when zooming the moving object of interest is calculated using the display sizes xdisp_max and xdisp_min of the display image,

のように求まる。

It is obtained like this.

  Further, when the image to be displayed is divided, the display sizes xdisp_max and xdisp_min are changed according to the number of divisions. For example, the display size when dividing into four is xdisp_max / 2 and xdisp_min / 2 for one division.

  In FIG. 9, there is only one runner, but even when there are a plurality of runners, the RF tag and the sub-partition template in the image can be associated with each other by the same method.

By executing the above-described series of processing, the moving object of interest can be tracked and photographed. (A) of FIG. 10 is a diagram illustrating an example in which a moving object is detected for each frame by applying the background difference method + local correlation method. FIG. 10B is a diagram showing an example in which the extracted region / scale rate calculation unit is applied to FIG. 10A and the tracked moving object is displayed on the monitor. Since the camera that is the imaging unit 5 is fixed, the moving object moves on the screen as shown in FIG. 5A, and the moving object detection unit detects the sub-partition template of the moving object, When the area is enlarged, each frame is displayed as shown in FIG. As described above, tracking shooting can be realized even when the camera is fixed.
[Attribute-added image generation unit 8]
The attribute-added video generation unit 8 associates the image data from the imaging unit 5 with the attribute data of the RF tag output from the attribute information generation unit 4, and outputs the attributed video data. Note that the output destination of the video data is, for example, a television monitor, streaming relay using the Internet, and a recording / reproducing apparatus (including a personal computer) using a DVD, video tape, hard disk, CD-ROM, or the like as a recording medium. Can be considered.

  FIG. 11 is a diagram illustrating an example of association between video data and attribute data. Usually, the image pickup unit 5 outputs image data for one frame, which is one screen image, at an appropriate sampling period (for example, 33 ms). Therefore, as shown in FIG. 11, for example, the frame number (frame data column) and each attribute data (tag 1 column and tag 2 column) based on the valid RF tag 1 output from the attribute information generation unit 4 are obtained. Express in the associated format.

  In FIG. 11, the attribute value (attribute data) D1 of the tag 1 is added to the video data of the third and fourth frames, and the attribute value D2 of the tag 2 is added to the video data of the fourth and fifth frames. Yes. When the attribute value is added to the video data, the video data with the attribute may be generated at an appropriate interval (for example, every 10 frames) instead of being associated with all the frames. Further, instead of adding the attribute value directly to the video data, a method of embedding it in the video data by a method such as digital watermarking may be used.

  Furthermore, the attributed video data may be generated by using a predetermined standard such as MPEG7 and describing the attribute value of the RF tag as MPEG7 metadata. The video data and the attribute value may be stored as separate files on a recording medium such as a hard disk.

[Description of processing]
Next, the processing of the present invention will be described. FIG. 12 is a flowchart showing processing of the photographing apparatus of the present invention. A processing operation of the photographing apparatus will be described with reference to FIG. The following parenthesized symbols correspond to the parenthesized symbols in the figure.
(a) Set the following items in advance.

    (1) The weighting factor of the directional antenna 2 (adaptive array antenna) is associated with the angle of the antenna using a reference table (table) prepared in advance.

    (2) Set a rule for converting unique data in the RF tag 1 into attribute values.

    (3) Write unique data corresponding to the attribute value to the RF tag 1.

(4) Values such as threshold Tth (background difference method) and threshold Mth (local correlation method) used for detection of a moving object region, and determination amount Δx used for associating the RF tag 1 with a moving object.
(b) Attach the RF tag 1 to an appropriate position of the subject to be photographed (for example, a runner) and start photographing.
(c) The weighting coefficient of the directional antenna 2 is changed according to the step angle, and the antenna direction is scanned.
(d) It is determined whether or not the RF tag 1 is present in the antenna direction.
(e) A radio signal from the RF tag 1 received by the directional antenna 2 is converted into digital data (specific data) by the RF tag reader / writer 3.
(f) If the angle θR of the antenna when the RF tag 1 is detected is within the viewing angle of the camera, it is determined that the RF tag 1 is effective.
(g) If it is determined to be valid, the digital data is converted into attribute data according to a preset rule. If it is determined as invalid, the digital data is invalidated and the process returns to (c).
(h) A moving object region is detected and a candidate is extracted by the background subtraction method in the moving object detection unit 6.
(i) Next, a local correlation method is applied to candidate regions extracted by the background subtraction method to determine a moving object region.
(j) The cutout region / scale factor calculation unit 7 determines that the RF tag 1 is a moving object in the moving object region based on the relationship between the antenna detection angle θR of the RF tag 1 and the determined center position Xg of the moving object region. Check if it is installed.
(k) It is determined whether or not the RF tag 1 is attached to a corresponding moving object (existing in the moving object region).
(l) When it is determined that it is attached (Yes determination), the cutout area is determined based on the moving object area, the enlargement / reduction ratio is calculated, and the two are associated with each other. In addition, when it determines with not mounting | wearing (No determination), they are invalidated and it returns to said (c).
(m) Extract the corresponding cut-out area and zoom the image using the calculated scaling ratio.
(n) Append the attribute value to the image data acquired by the camera, and output the video data with the attribute.

  Note that, for example, when the camera viewing angle changes to θmax ′ and θmin ′ by a zoom operation by the camera that is the photographing unit 5, the attribute information generation unit 4 determines whether the RF tag is valid or invalid. Range,

とすることで、ズーム操作にも追従させることが可能となる。

By doing so, it is possible to follow the zoom operation.

  With the configuration and processing described above, a moving object reflected in the camera field of view is detected, and attribute data of the moving object can be automatically assigned to the image area occupied by the moving object. When selecting or extracting a shooting target or a video scene, the editing operation can be automated by using the assigned attribute information. In addition, it is possible to automatically perform tracking shooting while zooming up on a shooting target equipped with an RF tag.

  Further, based on the angle (existing position) of the RF tag 1 detected by the directional antenna 2 in the moving object detection unit 6, the scanning range (detection target range) for searching the moving object region in the background difference method and the local correlation method. By limiting this, it is possible to speed up the detection process of the moving object region. The case will be described below.

  (1) In the background subtraction method, since the scanning range of the search is the entire screen (entire screen) in the above description, the scanning range is narrowed only around the detected RF tag, so that the speed can be increased.

  (2) The local correlation method can be speeded up by reducing the number of correlation calculations by reducing the size of the tracking block.

  A specific processing flow will be described with reference to the drawings. FIG. 13 is a diagram for explaining the limitation of the detection target range of the moving object region. In the figure, xmax, xmin, θmax and θmin are known values, and xmax and xmin are the maximum and minimum values at the left and right edges of the image corresponding to the camera field of view, and θmax and θmin are the upper and lower limits of the antenna direction. Indicates the value. At this time, the position xtag on the image of the moving object is

で表わされる。

It is represented by

  At this time, the scanning range is a hatched portion in the figure with the xscan having a predetermined width in consideration of the speed of the moving object and the zoom magnification of the camera with the xtag as the center. The limited scanning range (xtag−xscan, xtag + xscan) is replaced with the entire region of the image data shown in the description of the background difference method (d) or the region overlapping the tracking block set by the local correlation method. By doing this, it is possible to delete the correlation calculation for a region not related to the RF tag.

  In addition, as described above, the imaging apparatus of the present invention can generate video data in which image data and RF tag attribute data are simultaneously recorded at an appropriate sampling period. When the video data is stored in a hard disk such as a personal computer, for example, it is possible to easily extract only the video section in which the desired shooting target is shown from all the video data by using the functions of the personal computer. Is possible.

  FIG. 14 is a diagram for explaining part extraction of video data, and shows an operation of extracting a desired video section from video data with attributes. As shown in the figure, a video extraction unit is provided in the attributed video generation unit 8 described above, and video data with attributes and an attribute value related to a desired shooting target are input to the video extraction unit. The video extraction unit compares the attribute value simultaneously recorded in the attributed video data with the input attribute value, and only the frame in the section where the attribute value matches (A portion in the figure) is used as the extracted video data. Output.

  Therefore, only a desired video section can be extracted. The extracted extracted video data is sent to the next stage, for example, a distribution processing unit and distributed. Since the extracted video data still contains the attribute data, further extraction can be performed by inputting the extracted video data again into the video extraction unit or equivalent functional means, and inputting and extracting other attribute values. Needless to say, the extracted video data obtained can be obtained.

The figure which shows the structure of the imaging device of the Example of this invention. The figure explaining the directional antenna of this invention The figure which shows the example of conversion from the specific data of RF tag to attribute data Diagram showing the relationship between camera viewing angle and antenna direction Diagram showing the configuration of the moving object detector Flow chart showing processing in background subtraction method Diagram showing subpartitions, subpartition templates and tracking blocks Flow chart showing processing in local correlation method The figure explaining the correlation with RF tag and a moving object area | region Diagram explaining moving object area detection and tracking shooting The figure which shows an example of correlation with video data and attribute data The flowchart which shows the process of the imaging device of this invention The figure explaining limitation of the detection target range of a moving object area Diagram explaining partial extraction of video data

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 RF tag 2 Directional antenna 3 RF tag reader / writer 4 Attribute information generation part 5 Imaging part 6 Moving object detection part 7 Cutout area / scale rate calculation part 8 Image generation part with attribute

Claims (5)

An imaging apparatus for outputting video information, comprising imaging means for imaging a fixed desired field of view and generating image information,
An RF tag attached to a moving object to be imaged and capable of wireless communication holding unique information of the moving object;
A directional antenna capable of wirelessly communicating with the RF tag and recognizing the position of the RF tag;
An RF tag reader / writer that reads or writes the RF tag and the unique information via the directional antenna;
Moving object detection means for detecting a moving object region from the image information from the imaging means;
A cut-out area / scale-ratio calculating means for extracting a cut-out area including at least the moving object area from the image information, calculating an enlargement / reduction ratio to a desired size according to a contour size of the cut-out area, and associating with the cut-out area ,
Attribute information that generates attribute information based on the unique information from the RF tag reader / writer, assigns the attribute information in association with the cut-out area, and outputs attributed video information to which the attribute information and the enlargement / reduction ratio information are added Granting means;
A photographing apparatus comprising: The imaging device according to claim 1,
The moving object detection means includes
A background difference method detecting means for detecting the first moving object region by a predetermined background difference method;
Local correlation method detection means for limiting the detection target range based on the first moving object region and detecting the second moving object region by a predetermined local correlation method;
A photographing apparatus comprising: The imaging device according to claim 1,
The cut-out area / scale rate calculation means is:
The RF tag whose position is recognized by the directional antenna exists within the outline of the moving object area, and the moving speed and moving direction of the specific position in the moving object area are the moving speed and moving direction of the RF tag. In the case where the moving object region and the unique information are associated with each other, the cutout region is extracted. In the imaging device according to claim 1 or 2,
The moving object detection means includes
An imaging apparatus, wherein a detection target range of the moving object region with respect to the image information is limited based on presence position information of the RF tag acquired in advance. The imaging device according to claim 1,
The attribute information giving means is
Means for storing the attributed video information;
An imaging apparatus, wherein based on the setting of attribute-related information related to the attribute information, only the video section associated with the attribute information is extracted from the stored video information with attribute and output.

Images