JPH01250177A - Image input method - Google Patents

Abstract

PURPOSE:To obtain the image input method which can be used for high-level image processing by performing respective detection processings of the distance to an object to be examined, the size, and the direction of the existence of the object plural times while stepwise varying the focal length of a photographic lens. CONSTITUTION:The focusing state on the image surface of the photographic lens is two-dimensionally observed, and the distance to the object, the size, and the direction of the existence of the object are detected by the focusing state on the image surface, the size and the position of the focused image on the image surface, and the focal length and the field angle of the photographic lens respectively. This processing is performed by the focusing operation of the photographic lens where they are detected plural times while stepwise varying the focal length of the photographic lens. Thus, a number of degrees of freedom is given to positional relations between the object and a TV camera to not only provide a mere image taking-in function but also detect the distance to the object, the size, and the direction of the existence, and this method can be used for high-level image processing.

Description

DETAILED DESCRIPTION OF THE INVENTION TECHNICAL FIELD The present invention relates to manual image methods such as pattern recognition and object recognition.

BACKGROUND OF THE INVENTION Pattern recognition systems currently used in the FA (factory automation) field often use a TV camera as an image input device. However, the positional relationship between the test object and the TV camera is fixed, and the human-powered device only has the function of capturing images. This also applies to the method using two TV cameras as disclosed in, for example, Japanese Unexamined Patent Publication No. 58-114172, and there is a limit to the position of the test object with respect to the two cameras.

In addition, there are various methods for providing 3D visual functions for robots, including ``Journal of the Institute of Image Electronics Engineers, Vol. 14, No. 3 (19
85), each has its own problems, as shown in the technical explanation "3-dimensional visual function of robots". For example, binocular stereoscopic vision sensors use techniques such as the correlation method, point feature matching method, and edge feature matching method, which can indirectly obtain 3D information and can be applied over long distances. However, there are problems in that it is difficult to search for corresponding points, distances can only be determined from locations where corresponding points are found, and processing time is required. This is also the case with the binocular stereoscopic vision disclosed in Japanese Unexamined Patent Publication No. 60-27085, and it is also difficult to search for corresponding points for both eyes. For this reason, both visual systems are activated to help search for corresponding points.

In addition, visual sensors, which are distance sensors that use light, are based on reflected light processing methods (for example, spot, slit, morea, pattern light projection) and direct light processing methods, and are short-range sensors that actively project light onto an object. Although it is suitable for various applications and has the feature of being used by attaching a light emitting element to a robot device, it has the problem that it may not be able to capture reflected light. Furthermore, distance images have problems with resolution and processing time, and also have the problem of requiring an element to be attached to the object.

Furthermore, the monocular visual system is based on the photometric stereo method, which has the characteristic of determining the normal direction and three-dimensional information of the object surface by changing the knowledge of the object and the environment. There is a problem in that the environment needs to be controlled.

Purpose The present invention has been made in view of the above points, and has the following objects: While providing flexibility in the positional relationship between the test object and the TV camera,
It is an object of the present invention to provide an image input method that is capable of detecting the distance to the test object, the direction in which the test object exists, and the size of the test object and can be used for advanced image processing.

Structure In order to achieve the above object, the present invention two-dimensionally observes the in-focus state on the image plane of the taking lens of a TV camera, and determines the in-focus state on the image plane and the size of the focused image on the image plane. The process of detecting the distance to the object, the size, and the direction in which the object exists from the position of the object and the focusing distance and angle of view of the taking lens is performed by varying the focusing distance of the taking lens in stages. It is characterized by being performed multiple times.

Hereinafter, one embodiment of the present invention will be described based on the drawings.

First, the concept of the image information processing system handled by the present invention will be explained in the second section.
This will be explained with reference to the figures. This image information processing system is based on human visual information processing, and basically includes an image input section 1, an information processing section 2, and an output section 3.

Here, the image input unit 1 is for capturing general images using a TV camera or the like, and at that time, the location of the object of interest,
Detect size and color. These pieces of information are sent to the information processing section 2 together with the image information.

Further, the information processing unit 2 receives designations of objects of interest, such as "pay attention to a red object", "pay attention to a nearby object", and "pay attention to an object in the center of the visual field". Further, the information processing unit 2 realizes processing in the human cerebrum. That is, it processes information from the image input unit 1, learns and stores it, and makes decisions using functions such as recognition of incomplete images, associative memory, learning, and associative recall. Here, comparison with past learning/memory content 4 is performed. In addition, an active processing function operates during the processing process, and designates the object of interest to the image input unit 1. Further, the output unit 3 receives the processing results or judgments from the information processing unit 2, displays the CRT, generates an audio signal,
Performs manipulator operations and outputs such as printouts.

Therefore, in this embodiment, the image input unit 1 is concerned with detecting the distance of the test object, detecting the position of the test object, and detecting the size of the test object. This is done by focusing the photographic lens in the TV camera. These distance detection methods, presence direction detection methods, and size detection methods will be explained.

First, object distance detection does not involve focusing on a specific object, but rather detecting which part of the image plane is in focus when the photographic lens is at a specific focusing distance. This method involves finding the in-focus area at each focusing distance by varying the focusing distance of the photographic lens in stages. That is, the image plane position at which the image is in focus is sequentially determined for each focusing distance of the photographic lens.

FIG. 1 shows an optical system in the image input section 1 for finding such a focused area. First, a photographing lens 5 is provided, and a beam splitter 6 is provided which divides the amplitude of the optical information passing through the photographing lens 5 into four equal parts. Four two-dimensional light receiving elements 7a, 7b, 7c, and 7d are provided at predetermined positions on the exit side of the beam splitter 6 to receive the optical information divided into four parts by the beam splitter 6.

The photographic lens 5 is movable in the direction of the optical axis shown by arrow A, and is configured so that the focusing distance can be arbitrarily varied.

22, the two-dimensional light receiving elements 7a, 7b, and 7c.

The relationship between 7d and the optical path length at each installation position is shown in FIG. 3 and an enlarged FIG. 4. In the figure, a, b.

c and d are two-dimensional light receiving elements 7a, 7b, and 7c, respectively.

7d indicates the light receiving position, and F indicates the focal plane for the photographic lens 5. Specifically, the light receiving position a is located at a distance of ΔD from the focusing position F toward the photographing lens 5. The light receiving position d is located at a distance of ΔD from the focusing position F in the opposite direction to the light receiving position a.

The light receiving position S is located at a distance of Δd from the focusing position F toward the photographic lens 5, and the light receiving position C is located at a distance of Δd from the focusing position F in the opposite direction to the light receiving position.

Here, the two-dimensional coordinates (i.

Let the light receiving output at j) be 5alJ, and similarly, the two-dimensional coordinates (
The received light output at i, j) is Sb1,1゜5CIJI
It is assumed to be 5dIJ. Then, if it is assumed that the coordinates (i, j) are in focus, the (i, j) coordinates of the focusing plane F are in focus. Then, if the values of ΔD and Δd are determined appropriately, S alJ and S diJ and S bIJ and S
dlJ are blurred to the same degree.

Here, F = h-la (SblJ-5CIJ) + β(S
At, + -5dlJ) Consider the focal plane F defined by 1. Here, F is a numerical value representing the focused state, h is a numerical value that determines the allowable amount of the focused state, and α and β are positive constants.

Here, in the above formula, S btJ and S CIJ, S
Since alJ and S dIJ each have the same degree of blur,
lα(Sbt, +5CtJ)+β(Sat
, + -5alJ The value of the month term is a small value. However, once out of focus rest, I a (Sb1.+ −S
et, +) + β (Sat, + 5dlJ) The value of the term increases. Then, the value of h is determined appropriately, and when the value of F is positive, the object is in focus, and when it is negative, the object is out of focus. Thereby, it is determined from the value of F whether each coordinate is a focused portion.

However, these 5aIJ + 5btJ, 5c1
J, 5dLJ can be matched at the coordinate (t+
J) is located near the optical axis. Coordinates (t, J
) as the distance from the optical axis increases, it is necessary to add a correction value to the coordinates of each of the two-dimensional light receiving elements 7a to 7d when calculating the above equation. The amount of correction depends on the coordinates (i, j), ΔD, Δd, the angle of view of the photographing lens 5, the density of the light receiving element, and the lateral magnification of imaging.

FIGS. 5 and 6 show how the coordinate shift occurs.

First, FIG. 5 shows the direction of object point 0 and the two-dimensional light receiving elements 7a to 7.
Figure 5(a), which shows the relationship between the incident position of d and the light receiving position a-d, shows when object point 0 exists near the optical axis, and as shown in the figure, the chief ray from object point ○ is The light receiving positions a, b, and c pass near the optical axis of the optical system.

The incident coordinates to d are also approximately the same near the optical axis.

On the other hand, in FIG. 5(b), as is clear from the state shown in FIG. The incident coordinates of the light rays are different from each other. FIG. 6 shows an enlarged view of the state shown in FIG. 5(b).

That is, when in focus at the i-coordinate of the focusing plane F in the {circle around (2)} direction, the incident positions of the principal ray of the object point O relative to the light receiving positions a-d are a', b', c', and d', respectively. Then, there is a difference between these positions 87-42. As mentioned above, such a difference in coordinates is determined by the coordinates (i
, j), ΔD, Δd depends on the angle of view of the photographic lens 5, the density of the light receiving elements, and the lateral magnification of imaging.

Next, a method for detecting the direction in which a test object exists will be explained.

The direction in which the object exists is calculated from the image plane coordinates and the distance between the principal point of the photographic lens 5 and the image plane.

Note that, as shown in FIG. 7, the distance X between the principal point of the photographic lens 5 and the image plane depends on the angle of view θ of the photographic lens 5 and the distance X between the subject 8.

Furthermore, the size W of the test object 8 is calculated from the size W on the image plane and the vertical magnification of the image, as shown in FIG. Here, the vertical magnification is the ratio of the distance X to the test object 8 and the distance X between the principal point and image plane of the photographic lens 5 x / X (= w
/W).

In this way, according to the method of this embodiment, the photographic lens 5
By focusing, the distance, direction and size of the object can be detected three-dimensionally.

In other words, it is possible to have a degree of freedom in the positional relationship between the test object and the TV camera, and it is possible to play a role as a bridge to advanced image processing, such as processing functions possessed by humans.

Effects As described above, the present invention two-dimensionally observes the in-focus state on the image plane of the photographing lens, and determines the in-focus state on the image plane, the size and position of the focused image on the image plane, and the position of the photographic lens. A photographing lens that performs processing multiple times to detect the distance to the object, its size, and the direction in which the object exists from the focusing distance and angle of view, while varying the focusing distance of the photographing lens stepwise. Since this is done by the focusing operation of the subject, it is possible to have a degree of freedom in the positional relationship between the subject and the TV camera, and it is not only a simple image capture function, but also a detection of the distance, size, and direction of existence of the subject. , which can be used for advanced image processing.

[Brief explanation of the drawing]

The drawings show one embodiment of the present invention, in which Fig. 1 is a block diagram of the optical system of the image input section, Fig. 2 is a schematic block diagram of the image information processing system, and Fig. 3 is the relationship between the light receiving element position and the optical path length. FIG. 4 is an explanatory diagram showing the state of FIG. 3 in an enlarged manner. FIG. FIG. 7 is an explanatory diagram showing an enlarged view of the state shown in FIG. 7(b), and FIG. FIG. 8 is an explanatory diagram illustrating detection of the size of a test object. 5...Photographing lens, 8...Test object figure 33 h, 75 magazine figure

Claims (1)

[Claims] The in-focus state on the image plane of the photographing lens of the TV camera is observed two-dimensionally, and the in-focus state on the image plane, the size and position of the focused image on the image plane, and the focal distance and image of the photographing lens are determined. An image input method characterized in that the process of detecting the distance from the corner to the test object, its size, and the direction in which the test object exists is performed multiple times while varying the focusing distance of the photographing lens stepwise. .