KR101230346B1 - A method and an appratus for photographing an image in a stereo-vision system with adjustable intensity of radiation - Google Patents

Abstract

PURPOSE: Image photographing method and apparatus for a stereo vision system capable of adjusting the intensity of radiation are provided to accurately obtain the physical properties of a subject in the lighting difference by adjusting the light quantity based on the distance between the apparatus and the subject. CONSTITUTION: An image photographing unit(405) photographs a subject. A motion sensing unit(465) determines the stopped condition of the subject based on the obtained image information. A mode converting unit(460) changes a photographing mode of the image photographing unit to a second mode when the subject is stopped. A distance information acquisition unit(445) obtains the distance between the image photographing unit and the subject when the subject is stopped. A lighting control unit(430) adjusts the intensity of radiation of a lighting unit based on the obtained distance information. [Reference numerals] (410) Image analysis unit; (430) Lighting control unit; (435) Image processing unit; (440) Camera control unit; (445) Distance information acquisition unit; (450) Illuminance acquisition unit; (455) Image estimating unit; (460) Mode converting unit; (465) Motion sensing unit; (470) Stroke sensing unit; (AA,CC) Camera; (BB) Lighting part

Description

Image capturing method and apparatus for stereo vision system with adjustable light quantity {A METHOD AND AN APPRATUS FOR PHOTOGRAPHING AN IMAGE IN A STEREO-VISION SYSTEM WITH ADJUSTABLE INTENSITY OF RADIATION}

The present invention relates to an image capturing method and apparatus of a stereo vision system capable of adjusting the amount of light, and more particularly, to a camera having low performance by grabbing an intersection image using a plurality of cameras in a stereo vision system. The present invention relates to an image capturing method and a device of a stereo vision system capable of adjusting the amount of light to obtain an effect of a high performance camera.

In addition, the present invention, in order to obtain information about the physical characteristics of the subject in the stereo vision system, by obtaining the predictive image information based on the image information obtained by grabbing the intersection image using a plurality of cameras, The present invention relates to an image capturing method and a device of a stereo vision system capable of controlling an amount of light capable of capturing an image efficiently using a camera.

In addition, the present invention by adjusting the amount of light according to the distance from the image photographing apparatus to the subject in the stereo vision system, even if a difference in illumination occurs according to the distance to compensate for this to obtain accurate information about the physical characteristics of the subject The present invention relates to an image capturing method and a device of a stereo vision system capable of adjusting the amount of light.

In addition, the present invention in the stereo vision system by grabbing the intersection image using a plurality of cameras, by using a low-performance camera to shoot a moving image of a stereo vision system capable of adjusting the amount of light that can be taken in close proximity to a fast moving subject And to the apparatus.

Background Art A number of image photographing apparatuses that acquire physical information (eg, speed, direction, rotation, etc. of a subject) of a subject by using image information obtained by photographing a still or moving subject have been introduced.

In particular, recently, imaging devices including digital cameras such as DSLR (Digital Single Lens Reflex) have been widely used by the majority of consumers, and due to the development of related technologies, low cost imaging devices have been widely used. .

One of the objectives of the image capturing apparatus is to find or recognize a subject from the acquired image information, which cannot be implemented using a single camera, but requires at least two cameras and is used to implement the technique. This is a stereo vision system (SVS).

Stereo vision system is an application field of computer vision for realizing human visual system, and it means an imaging system obtained by two cameras composed of left and right, which is used for auto parking and unmanned driving system of automobiles in industry. In addition, in athletic games, the movement and speed of a player may be checked, or in a game such as baseball, golf, and soccer, which uses a ball, the movement speed and direction of a ball, which is a subject, may be checked based on the acquired image information. Can be used.

On the other hand, if the subject moves at a very high speed, or if the image is taken very close to the subject, the camera speed, which indicates the speed at which the film is shot through the camera's exposure device, must be very high, but the camera speed must be high. In this case, it is very expensive to implement a stereo vision system using such a camera. Therefore, there is a demand for a method capable of implementing the same stereo vision system as a high performance camera by using a low performance camera.

In addition, when the image is taken in close proximity to the ball, which is the subject, the physical characteristics of the ball may not be accurately extracted even with a slight difference in illumination according to the distance between the image photographing apparatus and the subject. Therefore, there is a demand for a method for compensating for the difference in illumination according to the distance between the image photographing apparatus and the subject.

Also, when shooting a moving subject that is stationary, for example, when a golf player hits a stationary ball on the field, a pitcher throws a baseball in a glove to a catcher in a baseball game. If you take a video at a high camera speed continuously, you can increase the load on the system because unnecessary images are analyzed and stored. Therefore, there is a demand for a method of converting a photographing mode of an image photographing apparatus according to a state of a subject.

In addition, in order to acquire information about the physical characteristics of the subject using a stereo vision system, since a plurality of cameras must grab the image of the subject in time synchronization, information about the physical characteristics of the subject by taking a close-up shot of a subject moving at a high speed. It is very difficult to obtain. Accordingly, there is a demand for a method for acquiring information on physical characteristics of a subject even when a plurality of cameras constituting the stereo vision system do not grab an image of the subject in time synchronization.

The present invention was devised to solve the above problems, and the two cameras used in the stereo vision system are not time-synchronized, and the image is obtained by grabbing an image of a subject at an intersection with a predetermined time difference, thereby obtaining a price. It is an object of the present invention to provide a method and apparatus for imaging a stereo vision system capable of adjusting the amount of light to implement a stereo vision system similar to a high performance camera using a camera having a relatively low camera speed.

Still another object of the present invention is to adjust the amount of light according to the distance between the image photographing apparatus and the subject, so that the amount of light capable of obtaining accurate information about the physical characteristics of the subject despite the difference in illumination that may occur according to the distance. An object of the present invention is to provide a method and apparatus for photographing an adjustable stereo vision system.

Still another object of the present invention is to obtain predictive image information based on image information obtained from a plurality of cameras included in the stereo vision system, through which the image capturing method of the stereo vision system that can determine the physical characteristics of the subject And provision of the device.

The problems of the present invention are not limited to those mentioned above, and other problems not mentioned above may be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the image photographing apparatus of the stereo vision system according to the present invention includes a first camera and a second camera for capturing image information by photographing a subject in a first mode, and an illumination unit for generating a flash when photographing. An image capturing unit; A motion detector determining whether the subject is stationary based on the acquired image information; A distance information obtaining unit obtaining distance information indicating a distance from the image capturing unit to the subject based on the obtained image information when it is determined that the subject is stationary; And an illumination controller for adjusting the amount of light of the illumination unit based on the distance information.

The present invention may further include a mode converting unit for converting the photographing mode of the image capturing unit to the second mode when it is determined that the subject is stationary.

According to the present invention, the first mode and the second camera is time-synchronized to grab the image of the subject to obtain image information, the second mode is the first camera and the second camera is a predetermined time The image information may be obtained by grabbing an image of an intersection subject with a difference.

The present invention is characterized in that the first mode to grab the image at a low speed and the second mode to grab the image at a high speed.

The present invention may further include a camera controller, wherein the first camera and the second camera acquire image information in a second mode according to a control signal from the camera controller.

The present invention may further include a hit detection unit for detecting whether the subject is hit by the user based on the image information acquired in the second mode.

The present invention may further include a database storing image information obtained from the first camera and the second camera.

According to an embodiment of the present invention, when a subject is detected as being hit by a user, an image is obtained by checking a hitting point, extracting image information before and after the hitting point from the database, and obtaining predicted image information based on the image information. Characterized in that it further comprises an estimator.

The image estimating unit obtains first predictive image information by using first image information and second image information obtained from the first camera, and third image information obtained from the second camera. It is done.

The first image information and the second image information indicate an image of a continuous frame obtained from the first camera, and the third image information includes the first image information and the second image information from a second camera. It characterized in that the frame obtained in between.

The present invention may further include an image processor which acquires physical characteristics of the subject based on the image information before and after the hitting time and the predicted image information.

The present invention may further include an illuminance obtaining unit which obtains illuminance information indicating intensity of illuminance of the space in which the image photographing unit is located.

The illumination control unit is characterized in that for adjusting the amount of light of the illumination unit based on the distance information and illuminance information.

In order to achieve the above object, an image capturing method of a stereovision system according to the present invention includes determining whether a subject is stopped based on the obtained image information; If it is determined that the subject is stationary, obtaining distance information indicating a distance from the image capturing unit to the subject based on the acquired image information; And adjusting an amount of light of the lighting unit based on the distance information.

The present invention may further include converting the shooting mode to the second mode when it is determined that the subject is stationary.

The present invention may further include detecting whether the subject is hit by the user based on the image information acquired in the second mode to acquire the image information in the second mode according to a control signal from the camera controller. Characterized in that.

According to an embodiment of the present disclosure, when the subject is detected as being hit by a user, the method may include checking a hitting point, extracting image information before and after the hitting point from a database, and obtaining predicted image information based on the image information. It further comprises.

The present invention provides the following advantages and advantages.

First of all, the two cameras used in the stereo vision system are not time-synchronized and acquire image information by grabbing an image of a subject at an intersection with a predetermined time difference. The same stereo vision system as a camera can be implemented.

Second, by changing the shooting mode according to the state of the subject, it is possible to implement a stereo vision system that can reduce the load on the system and efficiently take images.

Third, by adjusting the amount of light according to the distance between the image photographing apparatus and the subject, it is possible to implement a stereo vision system that can accurately obtain the physical characteristics of the subject despite the difference in illumination.

Fourth, the prediction image information is obtained based on the image information obtained from the plurality of cameras included in the stereo vision system, and through this, the physical characteristics of the subject may be checked, thereby increasing the accuracy of the prediction image information.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a view for explaining a stereo vision system.
2 is a diagram illustrating a method of installing a camera in a stereo vision system.
3 is a diagram illustrating a sequence of grabbing an image by a camera in a general stereo vision system.
4 is a block diagram showing the configuration of a stereo vision imaging apparatus according to an embodiment of the present invention.
FIG. 5 is a block diagram illustrating a specific configuration, additional configuration, and function of each component of the image photographing apparatus shown in FIG. 4.
6 is a diagram illustrating a method of measuring a distance from an image photographing apparatus to a subject according to an embodiment of the present invention.
7 is a diagram illustrating a process of adjusting an amount of light according to a distance from an image photographing apparatus to a subject by an image photographing apparatus according to an embodiment of the present invention.
8 is a diagram illustrating a sequence of grabbing an image by a camera in an image photographing apparatus according to an exemplary embodiment of the present invention.
9 is a diagram illustrating an image photographed in a second mode using an image photographing apparatus according to an exemplary embodiment of the present invention.
10 is a diagram illustrating a method of estimating an image using an image photographed in a second mode using an image photographing apparatus according to an exemplary embodiment of the present invention.
FIG. 11 is a flowchart schematically illustrating a process of adjusting a light amount of an illumination unit by using an image photographing apparatus according to an exemplary embodiment of the present invention.
12 is a diagram illustrating a sensing device for a virtual golf simulation including an image photographing apparatus according to an embodiment of the present invention.
FIG. 13 is a block diagram illustrating a configuration of a sensing device for a virtual golf simulation shown in FIG. 12.
14 is a diagram illustrating a process of detecting whether a user hits a ball, which is a subject, by a hit detection unit of a sensing device according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms or words used in the specification and claims should not be construed as having a conventional or dictionary meaning, and the inventors should properly explain the concept of terms in order to best explain their own invention. Based on the principle that can be defined, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention. Therefore, the embodiments described in the specification and the drawings shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent all of the technical idea of the present invention, various modifications that can be replaced at the time of the present application It should be understood that there may be equivalents and variations.

In particular, in the present specification, information is a term that includes values, parameters, coefficients, elements, and the like, and in some cases, the meaning may be interpreted differently. The invention is not limited to this.

1 is a view for explaining a stereo vision system.

Referring to FIG. 1A, a person may recognize an object in three dimensions because the right eye and the left eye recognize different images. Images of the retina obtained by the binocular are obtained by synthesizing each other in the brain 100, which is obtained by the minute difference of the images obtained by the binocular. The image obtained by such a minute difference is transmitted to the processing part of the brain through the optic nerve of the retina, and the transmitted electrical signal finds characteristics through various processing processes of the brain and compares it with the information stored in the brain. Will be recognized.

Referring to FIG. 1B, it is a stereo vision system that introduces a process of recognizing a target object through both eyes to computer vision. Therefore, the stereo vision system includes a camera 110 corresponding to the human eye and a computer 120 corresponding to the human visual system.

2 is a diagram illustrating a method of installing a camera in a stereo vision system.

Referring to FIG. 2, a method of installing a camera in a stereo vision system may be divided into sequential and simultaneous methods. The sequential method is a method of inputting images in order using one camera without installing cameras at the same time.

On the other hand, the simultaneous method is to install both cameras to acquire images at the same time is a method commonly used in current stereo vision system. Synchronous type is divided into (a) Parallel Camera Setup and (b) Crossing Camera Setup depending on the camera configuration, which is classified according to how the optical axis of the camera is configured. Parallel expressions are parallel to the vertical or horizontal, and cross expressions are close to the human eye structure by making the camera's optical axis coincide.

The parallel installation method has the advantage of easy distance measurement, but it has the disadvantage that it is impossible to make a high quality stereoscopic image because there is no parallax adjustment function of the stereoscopic image. It has the advantage of being possible.

3 is a diagram illustrating a sequence of grabbing an image by a camera in a general stereo vision system.

As shown in FIG. 3, grabbing of an image in a typical stereo vision system is based on an intersection of time-synchronized images taken by two different cameras based on four frames.

For example, as shown in FIG. 3, when capturing an image with two cameras having a shooting speed of 450 fps (frame per second), an image of 450 frames per second may be uniformly acquired through each camera. The distance and direction of the photographed subject may be checked using the acquired time-synchronized image.

However, if you want to shoot a close-up subject very fast, you will need a camera with higher performance. For example, if you need to shoot a subject using two cameras with 900 fps performance, the price of the camera is very high. It is expensive and it is difficult to construct a stereo vision system.

Accordingly, there is a need for a method capable of constructing a stereo vision system using a low cost camera without using an expensive camera in order to photograph a subject having high speed in close proximity.

In addition, when obtaining information on the physical characteristics of the subject based on image information obtained by photographing a subject having a high speed of movement, it is necessary to photograph as close as possible to obtain more accurate information. However, when taking a close-up image of a subject having a high speed of movement, accurate information cannot be obtained even with a minute illumination difference generated according to the distance from the image photographing apparatus to the subject.

Therefore, there is a need for a method that can compensate for the difference in illumination generated according to the distance from the image photographing apparatus to the subject.

4 is a block diagram showing a configuration of a stereo vision imaging apparatus according to an embodiment of the present invention, Figure 5 is a detailed configuration and additional configuration and the function of each component of the image recording apparatus shown in FIG. The block diagram shown.

As shown in FIG. 4, the stereo non-dedicated image photographing apparatus 400 according to the present invention includes an image capturing unit 405, an image analyzing unit 410, a controller 425, and an image processor 435. do. In addition, the image capturing unit 405 includes a plurality of cameras 415 and an illuminating unit 420 including at least one light that causes glare during capturing to configure a stereo vision system.

As described above with reference to FIG. 3, the general stereo vision imaging apparatus is configured to continuously capture an image of a subject in time synchronization with one or more frame units at a constant shooting speed regardless of the state of the subject. The image capturing apparatus 400 according to the present invention converts the shooting mode from the first mode to the second mode according to the state of the subject. In particular, in the second mode, each camera captures an image of the subject at the intersection, and acquires the second mode. And predicting image information based on the received image information.

Here, the first mode and the second mode are classified into a photographing speed, a grab sequence, and the like. That is, in the first mode, a plurality of cameras 415 constituting a stereo vision system having a low shooting speed and time-synchronizing grab an image of a subject, and in the second mode, the shooting speed is high and the plurality of cameras 415 are predetermined. Grab the image of the subject at the intersection with the time difference of.

If you take a close-up shot of a moving subject while the subject is still moving, you need to shoot the video at a high shooting speed from the time when the subject starts to move (hereinafter referred to as the “play section”). Up to a point in time before the subject starts to move (ie, a point in which it is stationary) (hereinafter, referred to as a “ready section”), the image may be taken at a low shooting speed.

Thus, for example, when photographing the movement of a subject with an image photographing device composed of two cameras having a speed of 450 fps, in a ready section, the image is captured in a first mode (eg, grabbing the image of the subject by synchronizing time at a low speed such as 30 fps). In the play section, the image is grabbed using the second camera (e.g., grab the image using the second camera (Camera # 2) after 1/900 seconds). If you are shooting a video, you can use the two cameras with a speed of 450fps in the play section where you need to record a video at high speed. Even if you are ready to shoot a lot of unnecessary images in the ready section can reduce the load on the system.

 The image analyzing unit 410 determines whether the subject is stationary based on the image acquired from the image capturing unit 405, and when it is determined that the subject is stationary, the distance from the image capturing unit 405 to the subject. Obtain distance information indicating.

The control unit 425, more specifically, the lighting control unit 430 adjusts the amount of light of the lighting unit 420 based on the distance information.

In addition, when it is determined that the subject is stationary as a result of the determination by the image analyzer 410, the controller 425 causes the image capturing unit 405 to grab an image in a second mode, that is, the camera 415. Controls to grab the image of the subject at the intersection with a predetermined time difference at a high speed, and controls the illumination unit 420 to operate when the camera 415 grabs the image.

In the case of capturing the image in the second mode, since the subject is photographed at different positions, in order to extract the 3D information from the photographed image, the process of estimating the intermediate empty image by extracting the front and rear images of each camera is performed. Additionally required.

A detailed configuration of the image capturing apparatus 400 illustrated in FIG. 4 and a function of each component will be described with reference to FIG. 5.

As shown in FIGS. 4 and 5, in the image capturing apparatus 400 according to the present invention, the information flow is performed in the first mode in the image capturing unit 405-> image analyzing unit 410-> mode converting unit ( 460)-> the controller 425, in the second mode, in the order of the image capturing unit 405-> image analyzing unit 410-> image estimating unit 455-> image processing unit 435 Is done.

The controller 440 includes the camera controller 440 and the lighting controller 430 to control the plurality of cameras 415 and the lighting unit 420.

The image capturing unit 405 grabs an image in the first mode or the second mode according to the signal of the camera controller 440, and the lighting unit 420 is configured to control the plurality of cameras 415 according to the signal of the lighting controller 430. Provide lighting at the speed of grabbing the image.

The image analyzer 410 determines that the motion detector 465 determines whether the subject is stationary based on the image information acquired by the image capturer 405 in the first mode, and the image capturer 405 moves to the second mode. A hit detection unit 470 detects whether or not the subject is hit by the user based on the acquired image information.

As a result of the determination by the motion detector 465, when it is determined that the subject is stationary, the mode converter 460 converts the photographing mode of the image capturing unit 405 from the first mode to the second mode, and controls the camera controller ( The control unit 440 controls the image capturing unit 405 to capture an image in the second mode according to the signal of the mode converter 460.

In addition, when it is determined that the subject is stationary as a result of the determination by the motion detector 465, the distance information acquirer 445 acquires distance information indicating the distance from the image photographing unit 405 to the subject, and acquires illuminance. The unit 450 obtains illuminance information indicating the intensity of illuminance of the space where the image capturing unit 405 is located, and the lighting controller 430 adjusts the illuminance of the illuminator 420 based on the distance information or the illuminance information. do.

When the shooting mode is converted to the second mode, the camera controller 440 has two cameras having a speed of 450 fps so that the camera controller 440 grabs the image using the first camera (Camera # 1) 1/900 seconds later. The grab of the image using the second camera (Camera # 2) is controlled, and the lighting controller 430 may control the lighting unit to provide an optical environment in which the camera can capture the image at a speed of 900 fps.

In addition, the image capturing unit 405 acquires image information in a second mode according to a signal from the camera controller 440. That is, the first camera (eg, the camera located on the left side) and the second camera (eg, the camera located on the right side) of the image capturing unit 405 acquire image information by capturing an intersection image at a predetermined time difference. However, as described above, the image information acquired in the second mode is not information about an image captured by time synchronization (that is, the intersection of image information acquired through the first camera and the second camera does not occur). Information regarding physical characteristics of the subject, such as 3D information, cannot be obtained based on the corresponding image information.

In order to capture a scene in which a stationary subject is moving and to obtain physical information such as the moving speed, direction, and rotation of the subject, when the subject starts to move in a stationary state (for example, when the subject is moved by a hit by the user) It can be seen as "the time of hitting"). It is necessary to acquire image information before and after.

The hit detection unit 470 detects whether the subject has been hit by the user based on the image information acquired in the second mode, and the image estimator 455, when the subject is hit by the user, By checking the hitting point, image information before and after the hitting point is obtained from the image capturing unit 405 or a database (not shown) to obtain predictive image information based on the image information.

In detail, the image estimator 455 obtains first prediction image information by using first image information and second image information obtained from the first camera, and third image information obtained from the second camera. .

Meanwhile, the first image information and the second image information represent an image of a continuous frame obtained from the first camera, and the third image information is provided between the first image information and the second image information from a second camera. The image of the acquired frame is shown. A detailed description of the process of obtaining the predictive image information by the image estimating unit 455 will be described later with reference to FIG. 10.

The image capturing apparatus according to the present invention includes a grabber (not shown) for digitizing an electric image signal captured by the image capturing unit 405 and image information obtained from the image capturing unit 405 and the image estimating unit 455. It may further include a database (not shown) for storing the obtained prediction image information.

The image processor 435 acquires information on a physical characteristic of the subject using image information obtained from the image capturing unit 405, predictive image information obtained from the image estimating unit 455, or information stored in a database. For example, when a moving subject is photographed using the image capturing unit 405, the image processing unit 435 may acquire information about a distance to the subject, a speed and a direction of the subject, and the like using the information.

6 is a diagram illustrating a method of measuring a distance from an image photographing apparatus to a subject according to an embodiment of the present invention, and FIG. 7 is a view illustrating an image photographing apparatus according to an embodiment of the present invention. It is a figure which shows the process of adjusting the quantity of light according to distance.

Referring to FIG. 6A, the distance H from the image photographing apparatus to the subject in the parallel stereo camera can be obtained from Equation 1.

Referring to FIG. 6B, the distance H from the image photographing apparatus to the subject in the crossover stereo camera may be obtained from Equation 2.

Referring to FIG. 7, it is shown that the amount of light of the illumination may be adjusted according to the distances d1 and d2 from the image photographing apparatus 400 (specifically, the image capturing unit) to the subject. The distances d1 and d2 and the amount of light of the illumination may be proportionally increased or may be configured to set an optimized amount of light according to the distances d1 and d2.

As shown in FIG. 3, when the first camera (camera # 1) and the second camera (camera # 2) take time images at a constant high speed in time synchronization, unnecessary images are increased. Increase the load on the system.

The present invention proposes a method for efficiently capturing an image by changing the photographing mode of the image capturing unit according to whether or not the subject is stationary. Hereinafter, the operation of the image photographing apparatus according to the embodiment of the present invention will be described with reference to FIG. 8.

8 is a diagram illustrating a sequence of grabbing an image by a camera in an image photographing apparatus according to an exemplary embodiment of the present invention.

As shown in FIG. 8, the image capturing apparatus 400 according to an embodiment of the present invention divides the ready section and the play section based on whether the subject is stationary, and captures the image in the first mode in the ready section, and the play section. In this case, the image is captured in the second mode.

That is, in the ready section, the video is grabbed at a low speed (e.g., 30 fps recording speed) and the video is synchronized at a high speed (e.g., 450 fps recording speed) in the play section. Grab to obtain image information.

When shooting images using two cameras having the performance of N fps (N> 0, where N means camera speed), the ready section grabs images at a low speed. In the play section, the second camera (camera # 2) is configured to operate 1 / 2N seconds after the first camera (camera # 1) is activated. In this case, an effect of photographing an image using a camera having a performance of 2N fps can be obtained.

In addition, as illustrated in FIG. 8, when taking a picture by dividing the recording mode into the first mode and the second mode in the ready period and the play period, it is possible to prevent or reduce the load on the system by taking unnecessary images. .

9 is a diagram illustrating an image photographed in a second mode using an image photographing apparatus according to an exemplary embodiment of the present invention.

As shown in FIG. 9, an image photographed in the above-described second mode using the image capturing apparatus 400 according to an exemplary embodiment of the present invention is a first camera camera # 1 and a second camera camera # 2. ) Obtains the intersection image.

In this case, since the intersection of the images of the two cameras does not occur, it is impossible to obtain 3D information or the like of the subject. Therefore, a process of acquiring an image between the acquired images of each camera by using the acquired images at the intersection is necessary.

10 is a diagram illustrating a method of estimating an image using an image photographed in a second mode using an image photographing apparatus according to an exemplary embodiment of the present invention.

Images captured in the second mode using the image capturing apparatus 400 according to an exemplary embodiment of the present invention cannot directly obtain time-synchronized images because they grab an intersection image using two cameras.

Accordingly, as shown in FIG. 10, the first predicted image is obtained by using the image (frame # 1_1) acquired in the first frame of the first camera (camera # 1) and the image (frame # 1_2) acquired in the second frame. (Predicted) The image (P_frame # 1_1) can be obtained, the same image obtained in the second frame of the first camera (camera # 1) (frame # 1_2) and the image obtained in the third frame (frame # 1_3), a second predicted image P_frame # 1_2 may be obtained.

On the other hand, when estimating an image of a subject moving at high speed (for example, the movement of a golf ball in a golf game, the movement of a baseball in a baseball game, etc.) using the above-described method, the speed, direction, It is very difficult to predict the rotation and so on, and its accuracy is very low.

Therefore, in order to increase the accuracy of the predicted image, the image capturing apparatus 400 according to an exemplary embodiment of the present invention acquires the image (frame # 1_1) and the second frame obtained from the first frame of the first camera (camera # 1). The first predicted image (P_frame # 1_1) using three images including not only the image (frame # 1_2) acquired from the image but also the image (frame # 2_1) obtained from the first frame of the second camera (camera # 2). ).

That is, in the case of the parallel stereo camera of FIG. 2A, an image photographed using the first camera (camera # 1) and an image photographed using the second camera (camera # 2) are displayed on the x, y, and z coordinates. Coordinate values are the same, and in the case of the crossover stereo camera of FIG. 2B, one coordinate value is the same. Therefore, as described above, using three images (frame # 1_1, frame # 1_2 and frame # 2_1) is more accurate than using two images (frame # 1_1 and frame # 1_2). Can increase.

Prediction of an image using three images in the image capturing apparatus 400 according to an embodiment of the present invention may be implemented using various methods. For example, a method of estimating an image by using a difference between a previous image and a current image or dividing a single image into blocks may be used. One of ordinary skill in the art may understand the image capturing apparatus of the present invention that estimates an image in addition to the above-described method.

FIG. 11 is a flowchart schematically illustrating a process of adjusting a light amount of an illumination unit by using an image photographing apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 11, first, a subject is photographed in a first mode to acquire image information (S1100), and it is determined whether a subject is stopped based on the image information (S1110).

As a result of determination, when it is determined that the subject is stationary (S1120), based on the image information, distance information indicating a distance from the image photographing apparatus to the subject is obtained (S1130), and when the subject is not determined to be stationary ( In operation S1110, the determination of whether the subject is stopped is performed based on the image information. Subsequently, the amount of light of the lighting unit is adjusted based on the distance information (S1140).

As described above, the image photographing apparatus according to the present invention can be used in various industrial fields and sports fields. Hereinafter, an embodiment of the present invention for implementing a sensing device for a virtual golf simulation using the image photographing apparatus according to the present invention.

12 illustrates a sensing device for a virtual golf simulation including an image photographing apparatus according to an embodiment of the present invention, and FIG. 13 is a block diagram illustrating a configuration of a sensing device for a virtual golf simulation shown in FIG. 12.

In general, for a virtual golf simulation, a sensing device 1200 for sensing a user hitting a ball and an image of a virtual golf course are implemented, and a virtual golf course is generated according to a sensing result of the sensing device 1200. In order to provide a simulation image for the trajectory of the ball is configured to include a simulator (not shown) that proceeds the virtual golf simulation.

As shown in FIG. 12, in the virtual golf simulation apparatus, a batter 1220 for allowing a user to swing a golf is provided at a bottom of a golf booth that provides a predetermined size of space, and at one side of the batter 1220. The batting mat 1225 is provided so that the user can hit the ball placed on the batting mat 1225 at the plate 1220 by the golf swing, and the screen is provided at the front to receive the image information from the simulator. And to project the image onto the screen.

12 and 13, the sensing device 1200 according to an exemplary embodiment of the present invention includes an image capturing unit 1230 including a plurality of cameras 1205 and 1210 and an illumination unit 1215, and the image. Image of the virtual golf course by receiving information and image information according to the sensing result from the sensing processor 1235 and the sensing processor 1235 to extract the physical characteristics of the moving ball by processing the image taken from the photographing unit 1230 Or image processing unit 1285 for performing a predetermined image processing for realizing a simulation image of a trajectory of a ball unfolding in a virtual golf course, and outputting image information received from the image processing unit 1285 on a screen. Database 1295 for storing image information acquired from the image capturing unit 1230, the image processing unit 1285, the image output unit 1290 and the data for viewing. The image controller 1280 controls an operation of the database 1295 or the like.

The image capturing unit 1230 may constitute a stereo vision system by including two cameras which are time-synchronized, but by using a low-cost camera to shoot a golf ball moving at high speed in close proximity to the physical characteristics of the ball (eg, the speed of the ball). In order to extract the moving direction of the ball, the rotation of the ball, etc.), as described above, it may be configured to snap the image by using two cameras at the intersection.

The sensing processor 1235 receives the image obtained through the image capturing unit 1230 in a frame unit and collects the grabber 1255 and the image transmitted from the grabber 1255 to determine whether the subject is still. The motion detecting unit 1300 for determining, when it is determined that the subject is stationary, a distance information obtaining unit 1320 for obtaining distance information indicating a distance from the image capturing unit 1230 to the subject, based on the distance information The lighting controller 1250 to adjust the amount of light of the lighting unit 1215, and when it is determined that the subject is stationary, the mode converting unit 1310 to convert the shooting mode of the image capturing unit 1230 to the second mode. By receiving and processing the image transmitted from the grabber (1255) by the blow detection unit 1260, the blow detection unit 1260 to detect whether the blow is ready or whether the blow was made by the user When a hit is detected, the image estimator 1265 extracts image information before and after the hitting point from the database 1295 by acquiring the hitting time point, and obtains predicted image information based on the image information. Acquired from the ball image processor 1270 and the image information and image estimator 1265 obtained from the ball image processor 1270 and the image capturing unit 1230, which extract physical properties of a ball, which is a subject, based on before and after image information and the predicted image information. By matching the predicted image information with each other, the two-dimensional information of each camera in the image capturing unit 1230 is converted into three-dimensional information, or on the contrary, the three-dimensional information obtained by matching and extracting information about a plurality of cameras with each other is two-dimensional information. And a transform unit 1275 for inversely transforming the circuit.

The image processor 1285 performs a predetermined image processing for implementing an image of a virtual golf course or a simulation image of a trajectory of a ball unfolding in a virtual golf course, and the image output unit 1290 performs the image. The image information received from the processor 1285 is output on the screen to be visually recognized by the user, and the database 1295 stores all data related to the sensing device 1200.

14 is a diagram illustrating a process of detecting whether a user hits a ball, which is a subject, by a hit detection unit of a sensing device according to an embodiment of the present invention.

As shown in FIG. 14A, the hit detection unit 1260 sets up and separates a hitting area from a source image of several frames, and then finds a ball therefrom. In general, an area corresponding to the hitting mat 1225 is used as a hitting area. Set it. In this way, by separating the hitting area and finding the ball only in it, it is much faster and more accurate to find the ball than finding the ball throughout the source image.

The process of finding the ball by the hit detection unit 1260 may be performed by various methods. For example, the ball template preset as a reference image of the ball and a predetermined object stored in the hit area may be analyzed by analyzing the degree of similarity. If the level is similar, the specific object may be determined to be a ball.

As shown in FIG. 14B, a sensing area 1400 having a predetermined size can be set such that one ball is freely included around the sensed ball, and whether the ball exists in the sensing area 1400. It can be determined whether or not the user hit.

That is, in the state shown in FIG. 14B, as shown in FIG. 14C, it may be determined whether the user is hit by detecting that the ball does not exist in the sensing area 1400. When the hit detection unit 1260 detects that the ball, which is the subject, is hit by the user, the hit detection unit 1260 examines an image of a plurality of frames received before the detected time point, and the exact time point at which the ball starts to move. That is, the impact time point is determined, and image information of a plurality of frames before and after the impact time point is stored in the database 1295, and the image information is transmitted to the image estimator 1265, and the image estimator 1265 is based on the image information. Acquire predictive image information.

The ball image processor 1270 may adjust the speed, direction, and rotation of the ball moving based on the image information before and after the impact point received from the hit detection unit 1260 and the predicted image information received from the image estimator 1265. The physical property information of the containing ball is calculated.

The physical property information of the ball obtained through the ball image processor 1270 is transferred to a simulator (not shown), and the simulator outputs a simulation image of moving the ball in a virtual golf course based on the image output unit 1290. By outputting through the virtual golf simulation proceeds.

As described above, in the present invention, two cameras used in a stereo vision system are not time-synchronized, and grab an image of a subject at an intersection with a predetermined time difference to obtain image information, so that a camera having a relatively low cost and a low speed camera The present invention can provide a method and apparatus for imaging a stereovision system capable of implementing a stereo vision system similar to a camera having good performance.

In addition, the present invention can provide an image capturing method and apparatus for a stereo vision system capable of capturing an image efficiently by reducing the load on the system by switching the photographing mode according to the state of the subject.

In addition, the present invention can provide an image capturing method and apparatus for a stereo vision system that can accurately obtain the physical characteristics of the subject despite the difference in illumination by adjusting the amount of light according to the distance between the image capturing apparatus and the subject.

The present invention also provides a method and apparatus for photographing a stereo vision system that can obtain predictive image information based on image information obtained from a plurality of cameras included in a stereovision system, and thereby confirm physical characteristics of a subject. Can provide.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be understood that various modifications and changes may be made without departing from the scope of the appended claims.

Claims (17)

An image capturing unit including a first camera and a second camera capturing a subject in a first mode of capturing a subject at the same time and an illumination unit for generating a flash when photographing the subject;
A motion detector determining whether the subject is stationary based on the acquired image information;
A mode converter configured to convert the photographing mode of the image photographing unit into a second mode in which the first camera and the second camera photograph the subject at the intersection when it is determined that the subject is stationary;
A distance information obtaining unit obtaining distance information indicating a distance from the image capturing unit to the subject based on the obtained image information when it is determined that the subject is stationary; And
And an illumination control unit for adjusting the amount of light of the lighting unit based on the distance information.
delete The method of claim 1,
In the first mode, the first camera and the second camera time-synchronize the image of the subject to grab image information, and in the second mode, the first camera and the second camera cross each other at a predetermined time difference. An image photographing apparatus of a stereo vision system capable of adjusting the amount of light, characterized by obtaining image information by grabbing an image of a subject.
The method of claim 1,
And the first mode grabs an image at a low speed, and the second mode grabs an image at a high speed.
The method of claim 1,
Further comprising a camera controller,
And the first camera and the second camera acquire image information in a second mode according to a control signal from the camera controller.
The method of claim 5, wherein
And a blow detection unit for detecting whether a subject is hit by a user based on the image information acquired in the second mode.
The method according to claim 6,
And a database for storing image information acquired from the first and second cameras.
The method of claim 7, wherein
If it is detected that the subject is hit by the user, and checks the time of hitting, extracting the image information before and after the hitting point from the database further comprises an image estimating unit for obtaining predictive image information based on the image information Imaging device of the stereo vision system that can adjust the amount of light, characterized in that.
The method of claim 8,
The image estimator adjusts light quantity by using the first image information, the second image information acquired from the first camera, and the third image information obtained from the second camera. Stereo imaging system capable of imaging.
The method of claim 9,
The first image information and the second image information represent an image of a continuous frame obtained from the first camera, and the third image information is acquired between the first image information and the second image information from a second camera. Imaging device of a stereo vision system capable of adjusting the amount of light, characterized in that the frame.
The method of claim 8,
And an image processor for acquiring physical characteristics of the subject based on the image information before and after the hitting time point and the predicted image information.
The method of claim 1,
And an illuminance acquiring unit for acquiring illuminance information indicating the intensity of illuminance of the space in which the image capturing unit is located.
13. The method of claim 12,
The lighting controller is an image photographing apparatus of a stereo vision system capable of adjusting the amount of light, characterized in that for adjusting the amount of light of the lighting unit based on the distance information and illuminance information.
Obtaining image information by photographing a subject in a first mode in which the first camera and the second camera simultaneously photograph the subject;
Determining whether the subject is stationary based on the acquired image information;
If it is determined that the subject is stationary, changing the shooting mode to a second mode in which the first camera and the second camera photograph the subject at the intersection.
If it is determined that the subject is stationary, obtaining distance information indicating a distance from the image capturing unit to the subject based on the acquired image information; And
And adjusting the light amount of the lighting unit based on the distance information.
delete 15. The method of claim 14,
Acquiring image information in a second mode according to a control signal from a camera controller; And
And detecting whether the subject has been hit by the user based on the image information acquired in the second mode.
17. The method of claim 16,
If it is detected that the subject is hit by the user, checking the hitting point, extracting image information before and after the hitting point from the database, and obtaining predictive image information based on the image information. The image capturing method of the stereo vision system with adjustable light intensity.

Images