JPH0983855A - Image system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain desired zoom effect processing selectively for image data with different in-focus point positions for each display terminal by sending plural image data picked up at different in-focus positions via a communication line. SOLUTION: Different in-focus point positions are set to two cameras 11, 12, which pick up an image of the same object and generate two kinds of digital image data corresponding to the respective in-focus point positions such as an image focused to a near scenery in the wide angle mode and an image focused to a remote scenery in the telescopic mode in parallel. Display terminals 41, 42 are respectively provided with input selection sections 21, 22 and receive selectively two kinds of digital image data via a communication line and display the data. Image processing sections 31, 32 apply image processing such as partial magnification to the digital image data according to a command from control signal generators 51, 52 to obtain desired panning or zooming effects.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image system and an image processing method, and more particularly to a technique effectively applied to an image system which electronically performs camera operations such as zooming, panning and focusing by image processing. .

[0002]

2. Description of the Related Art In recent years, with the development of digital image processing technology in image systems, it has become possible to easily edit images captured by a camera.

In these recent image edits, not only the conventional work such as image composition and cutting, but also the digital image processing technique is utilized, and the lens and camera parameters such as zoom and pan, which are conventionally mechanical operations, can be changed. It can be done.

FIG. 8 is a block diagram showing a schematic configuration of a conventional image system in which processing such as zooming and panning is performed by using a digital image processing technique.

In FIG. 8, reference numeral 1 is a camera, and 3 is an image processing unit. In the image system shown in FIG. 8, the camera 1 photographs an object and the captured image is A / D converter (not shown). The image processing unit 3 converts the image data into
Digital image processing is performed on the image data to perform processing such as pan and zoom.

Regarding the conventional image system, refer to the following document (A).

(A) Nikkei New Media Separate Volume "Scenario for Communication / Broadcast Convergence", Nikkei BP, 1994

[0008]

Normally, in a captured image of a subject at a certain moment taken by the camera 1, the focusing distance is determined by the focal position of the lens of the camera 1 at that moment.

Therefore, for example, when a subject close to the camera is focused, a distant captured image serving as a background becomes an image with a defocus.

Therefore, in the conventional image system, even if the zoom function is realized by the image processing unit 3, the camera 1
Other than the subject at the distance where the lens is in focus, it can be generated only as an out-of-focus image.

Of course, the zoom function can be achieved by adjusting the focus of the lens of the camera 1 as in the conventional case.

However, since the focus adjustment of the image is performed by the lens operation of the camera 1, the adjustment can be performed only by the real-time image data from the camera 1, and the focus position of the stored image data can be changed. There was a problem that changes could not be made.

Further, in an image system in which a plurality of display terminal devices are connected to the image processing 3 via a communication line and the plurality of display terminal devices output the picked-up images by the camera 1, a certain display terminal is provided. When the device requests a captured image focused on a near view and another display terminal device requests a captured image focused on a distant view, both requirements cannot be satisfied.

In this case, in order to focus on the screen irrespective of the distance to the subject, it is sufficient to reduce the lens diaphragm, but there is a problem that the image becomes dark.

In addition, there is a problem that this method cannot be realized when the subject other than the in-focus position is intentionally made into an out-of-focus image.

The present invention has been made to solve the above-mentioned problems of the prior art, and an object of the present invention is to display image data from an image pickup unit on a plurality of display terminal devices. In, in each display terminal device, freely select image data with different in-focus position,
It is to provide a technology capable of realizing a zoom function by image processing.

Another object of the present invention is to provide an image processing method for an image system in which image data from an image pickup section is displayed on a plurality of display terminal devices, and the focus position of each display terminal device is changed. It is an object of the present invention to provide a technique for interpolating the most suitable in-focus position image data when the zoom function is realized by image processing by freely selecting different image data.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[0019]

SUMMARY OF THE INVENTION Among the inventions disclosed in the present application, the outline of a representative one will be briefly described.
It is as follows.

(1) An image pickup unit for generating image data of a subject, an image processing unit for processing the image data from the image pickup unit, and a plurality of display terminal devices for the image data processed by the image processing unit. In the image system configured to display the image, the imaging unit is configured to include an imaging unit that captures captured images of the same subject with different in-focus positions, and a plurality of captured images that have different in-focus positions captured by the imaging unit. The image data generating means for parallelly generating a plurality of image data corresponding to each in-focus position and the plurality of image data with different in-focus positions generated in parallel can be independently selected by each of the plurality of display terminal devices. And an image selection unit.

(2) In the means of (1) above, the image pickup means shifts the focus position every predetermined time, and time-divisionally takes the imaged images of the same subject having different focus points. It is characterized by being.

(3) In the means of (2), the image pickup means shifts the focal position every n frames (n is an integer of 1 or more) or every n fields.

(4) In the image processing method according to the means (2) or (3), a plurality of image data corresponding to each in-focus position from a plurality of picked-up images having different in-focus positions taken at predetermined time intervals. In parallel generation, from the image data at the present time and the image data before the current time, a static region in which the luminance change does not occur temporally, a moving region in which the luminance change occurs, and a motion vector of the moving object are detected, and the detection is performed. The image data of the same in-focus position at the present time is generated from the image data of the same in-focus position before the current time or the image data of the same in-focus position after the current time based on the static region, the moving region, and the motion vector Is characterized by.

According to the above means (1) to (4),
Since the image pickup unit is configured to generate image data corresponding to different in-focus positions, for example, a plurality of image data in which a near view is focused in a wide angle and image data in which a distant view is focused in a telephoto are displayed. It can be freely selected by the terminal device, and the zoom function can be realized by image processing.

[0025]

DETAILED DESCRIPTION OF THE INVENTION The present invention will be described in detail below with reference to the drawings.

In all of the drawings for explaining the embodiments (examples) of the invention, those having the same function are designated by the same reference numerals, and the repeated description thereof will be omitted.

[Embodiment 1] FIG. 1 is a block diagram showing a schematic configuration of an image system which is an embodiment (Embodiment 1) of the present invention.

In FIG. 1, 11 and 12 are cameras and 2
Reference numerals 1 and 22 denote input selection units, 31 and 32 denote image processing units, 4
1, 42 are display terminal devices, and 51, 52 are control signal generators.

The image system of the first embodiment includes a camera (1
1, 12), an input selection unit (21, 22), and an image processing unit (31, 32) are arranged in the center, and the display terminal device (41, 42) and the control signal generator ( 51, 52) is an image system connected to the image processing unit (31, 32).

Here, the cameras (11, 12) and the input selection sections (21, 22) constitute an image pickup section.

In the image system of the first embodiment, two cameras (11, 12) photograph the same subject, and the photographed image is converted into image data by an A / D converter (not shown). Convert.

At that time, each camera (11, 12)
By changing the in-focus position of the lens, the moving image data corresponding to each in-focus position is generated in parallel.

Therefore, the image pickup means for picking up the picked-up images of the same subject having different focal points and the image data generating means for parallelly generating a plurality of image data corresponding to the respective focal points are the two cameras. It is realized by parallel imaging by.

The image data from the two cameras (11, 12) is input to the input selection section (21, 22) installed for each display terminal device (41, 42), and the display terminal device (4
The image data required by the display side is selected by the control signal from the (1, 42) side.

Next, the image processing section (31, 32) performs digital image processing on the image data, for example, partial enlargement of the input image, and processing such as pan and zoom.

The image data after these processes are displayed on the display terminal device (41, 42).

As described above, according to this embodiment,
In addition to zooming, panning, etc., focus position changes are also performed electronically, and all camera operations that were conventionally performed mechanically are independently performed electronically on multiple display terminal devices (41, 42). Is possible.

[Embodiment 2] FIG. 2 is a block diagram showing a schematic configuration of an image system which is another embodiment (Embodiment 2) of the present invention.

In FIG. 2, 1 is a camera, 6 is a signal distributor, 21 and 22 are input selectors, 31 and 32 are image processors, 41 and 42 are display terminal devices, 51 and 52 are control signal generators, Reference numerals 71 and 72 are frame interpolation units.

The image system of the second embodiment includes a camera 1,
The signal distribution unit 6, the frame interpolation unit (71, 72), the input selection unit (21, 22), and the image processing unit (31, 3)
2) is an image system which is arranged in the center, and a display terminal device (41, 42) and a control signal generator (51, 52) are connected to an image processing unit (31, 32) via a communication line. .

Here, the camera 1, the signal distribution unit 6, the frame interpolation unit (71, 72), and the input selection unit (21,
22) constitutes an image pickup section.

In the image system of the first embodiment, one camera 1 photographs the same subject, and the photographed image thus photographed is converted into image data by an A / D converter (not shown).

At this time, the camera 1 shoots the same subject while changing the in-focus position for each frame and time-divisionally generates moving image data corresponding to each in-focus position.

The image data from one camera 1 is processed by the signal distributor 6 into the respective frame interpolators (71, 7).
2) is input to the frame interpolation unit (71, 72),
The image data from the camera 1 and the frame-interpolated image data are input to the input selection units (21, 22) installed for each display terminal device (41, 42).

Then, the image data required by the display side is selected by a control signal from the display terminal device (41, 42) side.

Next, the image processing section (31, 32) performs digital image processing on the input image data, for example, partially enlarging the input image and performing processing such as pan and zoom.

The image data after these processes are displayed on the display terminal device (41, 42).

FIG. 3 is a block diagram showing a schematic configuration of the camera 1 of the second embodiment.

In FIG. 3, 101 is a lens, 102 is an image sensor, 103 is a motor, and 104 is a rotating plate.

In the image system of the second embodiment, since the single camera 1 shoots the same subject by changing the in-focus position for each frame, the lens 101 and the image sensor 102 are used.
The rotating plate 104 is rotated between and.

FIG. 4 is a plan view of the rotary plate 104 shown in FIG. 3. In FIG. 4, 105 is a transparent plate.

This rotating plate 104, as shown in FIG.
The transparent plate 105 having a shape of a quarter circle is arranged in a point object, and there is no object at a position where the transparent plate 105 is not present.

When the rotating plate 104 is rotated by 90 ° for each frame, an image of light transmitted through the transparent plate 105 and an image of light not transmitted are captured every other frame.

Here, the focal length f of the lens is 25 mm,
Assuming that the thickness of the transparent plate 105 is 0.8 mm and the refractive index is 1.4, the optical path length between the lens and the image pickup element is different by 0.32 mm when the transparent plate 105 is transmitted and when it is not transmitted. As a result, images with different focus positions are captured every other frame.

In this case, when the image is focused at infinity when light does not pass through the transparent plate 105, the transparent plate 105
When the light passes through, the image is focused at a position about 2 m from the lens.

The image data from the camera 1 is distributed by the signal distribution unit 6 depending on the difference in the focus position.

In this case, since the distributed signals are signals every other frame, the frame interpolator (71,
In 72), the image of the missing frame is generated.

FIG. 5 shows the frame interpolator (7
It is a figure explaining the processing procedure of (1, 72).

In FIG. 5, I (j) is input image data, and In (j) and If (j) are output image data generated as near-focus image data and far-focus image data, respectively.

Here, the frame number is shown in parentheses.

FIG. 5 shows a case in which n, n + 2, ... Frames are focused far, and n-1, n + 1 ,.

In this case, n-1, n of the near focus image data
+1, ... For the frame image data, the same frame image data of the input image data is directly used as the image data of the frame.

On the other hand, n, n + 2, ... Frames are generated from the frame image data in the input image data and the image data one frame later.

However, in the image data generation processing method of this embodiment, since motion adaptation processing is performed, motion information is obtained using the current frame image data.

FIG. 6 shows a frame interpolating unit (7) shown in FIG.
It is a figure for explaining the image data generation processing method of (1, 72).

In FIG. 6, m is a moving body region, S is a stationary region, and D is a moving region.

In order to detect the moving area, the difference between the input image data I (n) and the input image data I (n-1) is calculated.

However, before the difference is obtained, a low-pass filter is applied in advance so that both image data have the same blurring state on the entire screen.

Near-focus image data In to be generated
In (n), the area other than the moving area D obtained by the above method, that is, the still area S is generated by copying the image data of the same area of the input image data I (n-1) of the previous frame.

Next, by the template matching method,
The moving body region m and the moving body motion vector are obtained, and the moving body region m of the input image data I (n-1) is copied to the position determined by this moving vector in the image data In (n).

In the template matching method, the image data is divided into blocks of 8 × 8 pixels and the like, and the correlation is obtained while moving each block on the next frame image data. This is a method for determining that the block has moved, and the moving object area can be obtained by this method. Difference between the still area S and the moving area in the current frame (nth frame) determined by the above method. A region (region indicated by a dotted line In (n)) is a still region S newly appearing in the nth frame, and in the input image data I (n-1) of the previous frame, the still image data data of this region is not exist.

Therefore, this area is generated by copying the data of the same area of the input image data I (n + 1) after one frame.

Since the frame image data In (n) thus generated is generated from the near-focus image data input in the preceding and succeeding frames, the image data in the same focus state as the preceding and following frame image data. Becomes

In the above description, the method of generating the image data of the missing frame in the near focus image data has been described, but the image data of the missing frame in the far focus image data can also be generated by the same method. Is.

FIG. 7 shows a frame interpolator (7) shown in FIG.
1, 72) is a block diagram showing a schematic configuration of an example.

In FIG. 7, 711 and 712 are frame memories, 713 is an adder, 714 is a difference unit, 715 is template means, and 716 is a coefficient generator.

Frame interpolator (71, 72) shown in FIG.
In, two frame memories (711, 712)
Of the input image data I (n-
1), input image data I (n), input image data I (n
+1) three image data are generated.

Further, the input image data I
(N-1) and the input image data I (n) are taken to detect the stationary area S, and the template means 715 is further used.
Then, the moving object region m and the motion vector are obtained by the template matching method.

The output signals from the difference unit 714 and the template means 715 are input to the coefficient generator 716, and the coefficient generator 716 performs a logical operation. For example, when the output of the difference unit 714 is 0, C1 = 1, C2 = 0, C3 = 0, the coefficients (C1 to C3) are obtained, and the input image data I (n-1) and the input image data I are calculated according to the coefficients.
(N), a desired output signal is obtained while switching the input image data I (n + 1).

In the second embodiment, the focus position is shifted every frame, but the focus position may be shifted every arbitrary frame, or it may be shifted field by field instead of frame. .

Two image data having different focus positions generated by the above method are set in each input terminal unit (41, 42) in the same manner as in the first embodiment.
22) and the image data requested by the display side is selected by a control signal from the display terminal device (41, 42) side.

In the second embodiment, the focus position is set to 2
Although the shift is performed by using the rotary plate 104 including the one transparent plate 105, the image pickup element may be mechanically moved back and forth using a piezoelectric element, a step motor, or the like.

Further, in each of the above-described embodiments, the image pickup unit generates image data corresponding to two in-focus positions, but it is also possible to generate image data corresponding to three or more in-focus positions. It is possible.

As described above, the invention made by the present inventor is:
Although the present invention has been specifically described based on the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments, and it goes without saying that various modifications can be made without departing from the scope of the invention.

[0085]

The effects obtained by the typical ones of the inventions disclosed in the present application will be briefly described as follows.

(1) According to the present invention, in the image system in which the image data from the image pickup unit is displayed on a plurality of display terminal devices, the image data corresponding to different in-focus positions is output from the image pickup unit, Since the image data corresponding to different focus positions can be freely selected by the plurality of display terminal devices, the zoom function can be realized by the image processing.

As a result, in addition to zooming, panning, etc., focus position change is also executed by digital image processing, and all camera operations conventionally performed mechanically can be independently electronically processed by a plurality of display terminal devices. It becomes possible to perform the editing work of the image data easily.

(2) According to the present invention, since a plurality of image data corresponding to different in-focus positions are generated, it is possible to change the focus position even with respect to already stored image data. Become.

(3) According to the present invention, with one camera,
Since the image data having different in-focus positions are generated in a time division manner, the system configuration becomes simple.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an image system that is an embodiment (Embodiment 1) of the present invention.

FIG. 2 is a block diagram showing a schematic configuration of an image system which is another embodiment (second embodiment) of the present invention.

FIG. 3 is a block diagram showing a schematic configuration of a camera 1 of a second embodiment.

FIG. 4 is a plan view of a rotary plate 104 shown in FIG.

5 is a diagram illustrating a processing procedure of a frame interpolation unit (71, 72) shown in FIG.

FIG. 6 is a diagram for explaining an image data generation processing method of the frame interpolation unit (71, 72) shown in FIG.

7 is a block diagram showing a schematic configuration of an example of a frame interpolation unit (71, 72) shown in FIG.

FIG. 8 is a block diagram showing a schematic configuration of a conventional image system in which processing such as zooming and panning is performed using a digital image processing technique.

[Explanation of symbols]

1, 11, 12 ... Camera, 3, 31, 32 ... Image processing unit, 6 ... Signal distribution unit, 21, 22 ... Input selection unit, 41,
42 ... Display terminal device, 51, 52 ... Control signal generator, 7
1, 72 ... Frame interpolation unit, 101 ... Lens, 102 ...
Image sensor, 103 ... Motor, 104 ... Rotating plate, 105 ...
Transparent plate, 711, 712 ... Frame memory, 713 ... Adder, 714 ... Difference calculator, 715 ... Template means, 7
16 ... Coefficient generator.

Claims (4)

[Claims] 1. An image capturing unit for generating image data of a subject, an image processing unit for processing image data from the image capturing unit, and image data processed by the image processing unit in a plurality of display terminal devices. In the image system configured to display, the image pickup unit combines an image pickup unit that picks up picked-up images of the same subject having different focus positions and a plurality of picked-up images that have different focus positions taken by the image pickup unit. An image data generation unit that parallelly generates a plurality of image data corresponding to each focus position, and an image in which the plurality of display terminal devices can independently select the plurality of image data with different in-focus positions generated in parallel An image system comprising: a selection unit. 2. The image pickup unit is an image pickup unit that shifts a focus position at predetermined time intervals and time-divisionally takes a picked-up image of the same subject having different in-focus positions. Imaging system described in. 3. The image system according to claim 2, wherein the image pickup means shifts the focal position every n frames (n is an integer of 1 or more) or every n fields. 4. The image processing method in the image system according to claim 2 or 3, wherein a plurality of captured images having different in-focus positions taken at predetermined time intervals are associated with each of the in-focus positions. When the image data is generated in parallel, from the image data at the current time point and the image data at the current time point, a static area in which the brightness does not change temporally, a moving area in which the brightness change occurs, and a motion vector of the moving object are detected, Based on the detected still area, moving area, and motion vector, image data at the same in-focus position at the current time is generated from image data at the same in-focus position before the current time or image data at the same in-focus position after the current time. An image processing method comprising: