KR101850534B1 - System and method for picture taking using IR camera and maker and application therefor - Google Patents

Abstract

In order to capture an image of a subject, the image capturing system collects marker images of the infrared marker, calculates a control value for controlling the movement of the motor mounted on the image capturing apparatus based on the marker position in the marker image, The value is transmitted to the motor, and the image capturing device moves by the control of the motor and captures the image of the subject.

Description

Technical Field [0001] The present invention relates to an image capturing system, an image capturing method, and an application using an infrared camera and a marker,

The present invention relates to an imaging system, method and application using infrared cameras and markers.

Due to the development of smart devices and social networks, the culture of uploading and sharing UCC (User Created Contents) which are one-person lectures or performances such as YouTube (Youtube) and blogs is spreading rapidly. In this case, when users take a picture of the image contents alone, there is a limit to photographing a desired image without confirming the position of the subject and using an assistant force to photograph the image.

To this end, various techniques for automatically tracking the position of an object are being studied. One of them is a technology for tracking the position of a subject using a camera and an infrared LED marker, analyzing the image of the infrared marker, locating the subject, and locating the subject based on the position data.

In order to determine the position of the subject through this technique, image processing is mainly performed on an infrared LED marker through a single microcontroller unit (MCU) or a micro processing unit (MPU). However, since a high-capacity memory and a high-performance MCU or MPU are required in order to store and process high-capacity image data, there is a disadvantage in that it is difficult to miniaturize the product because of a problem of cost increase.

When the position of the subject is detected by the infrared LED marker and the camera, if there is an interference object such as an object or a reflector which emits light of a similar wavelength to the surrounding environment due to the characteristic of infrared rays, the camera distinguishes the marker and the interference object I can not. Therefore, there is a disadvantage that it is difficult to distinguish the subject automatically and accurately track the position of the subject.

Accordingly, the present invention provides an image capturing system, a method, and an application using an infrared camera and an infrared marker that can accurately capture the position of a subject while taking a dynamic image of the subject alone.

According to another aspect of the present invention, there is provided a method of capturing an image of a subject,

Collecting a marker image of the infrared marker; Calculating a control value for controlling the movement of the motor mounted on the image pickup apparatus based on the marker position in the marker image; And transmitting the control value to the motor, wherein the infrared marker is attached to the subject, and the image photographing apparatus photographs the subject.

The step of collecting the marker image includes the steps of: transmitting the infrared light emitted by the infrared marker and collecting the marker image; Converting the marker image into a black and white marker image; And compressing the converted black and white marker image to generate a compressed marker image.

Transmitting the compressed marker image to the image capturing apparatus; And receiving the marker position from the imaging device, wherein the marker position can be calculated based on the compressed marker image.

Wherein the position of the infrared marker further includes the first motor control value and the second motor control value calculated by the image photographing apparatus after the step of receiving from the image photographing apparatus, And controlling the motor based on the motor control value.

Wherein the step of calculating the control value includes: calculating a first motor control value for moving the image sensing apparatus in a first direction based on the marker position; Calculating a second motor control value for moving the imaging device in a second direction; And controlling the motor to move the imaging device in at least one of the first direction or the second direction based on the calculated first motor control value and the second motor control value .

According to another aspect of the present invention, there is provided an application program executed in combination with hardware including a processor, a memory, and a camera of the terminal,

Activating the camera to capture an image; Receiving a marker image of an infrared marker from the infrared marker recognition device; Calculating a marker position in the marker image; And transmitting instructions to the infrared marker recognition device, wherein the infrared marker is attached to a subject and the terminal is controlled by the infrared marker recognition device, And the infrared marker recognition device controls the motor based on the marker position.

The step of calculating the marker position may include analyzing pixel data of the marker image and extracting a pixel data value of the infrared marker; And calculating the position of the infrared marker based on the extracted pixel data value.

Wherein the step of calculating the position of the infrared marker includes the steps of: collecting a face image when two or more pixel data values including a pixel data value corresponding to the infrared marker are extracted; Confirming the number of facial faces of all the subjects included in the collected facial images, and assigning facial identification numbers to the facial images of the subject; And confirming presence or absence of the infrared marker on each of the face images of the subject.

The step of confirming presence or absence of the infrared marker may include checking the size and position of the facial area of the selected facial face if the facial identification number given to the subject facial surface is smaller than the number of facial facial numbers for all the subjects ; Designating a search area to detect image data corresponding to the infrared marker based on the position of the determined subject's face; Dividing a designated search area by a preset number to generate a plurality of divided screens; Confirming whether any one of the plurality of divided screens includes image data for the infrared marker; And analyzing the position of the infrared marker if image data for the infrared marker is included in any one of the divided screens.

Increasing the facial identification number given to the subject's face if the image data for the infrared marker is not included in the plurality of divided images; Expanding the search area if one of the facial identification numbers is larger than the number of facial facial numbers; And confirming that there is no infrared marker in the search area if the size of the extended search area is equal to the size of the preset maximum search area.

The step of transmitting to the infrared marker recognition unit may transmit the motor control value calculated based on the marker position to the infrared marker recognition unit.

According to another aspect of the present invention, there is provided a system for capturing an image of a subject having an infrared marker,

A main body for mounting the image photographing apparatus; At least one motor for controlling movement of the main body; An infrared camera recognizing an infrared marker; A memory for storing a program; And a processor for executing the program in cooperation with the motor, the infrared camera, and the memory, wherein the processor is configured to collect a marker image photographed by the infrared camera, to extract the marker position extracted based on the marker image And calculates a motor control value for controlling the motor based on the information.

Wherein the infrared marker comprises: an LED unit that is activated to emit infrared light based on a pulse signal to which a plurality of infrared LEDs are input, or is switched from an active state to an inactive state; A timer for generating and supplying a pulse signal having a preset interval to a plurality of infrared LEDs included in the LED unit; And a power supply unit for supplying power to the timer.

According to another aspect of the present invention, there is provided a system for capturing an image of a subject having an infrared marker,

A video camera for capturing an image; A main body including an infrared camera, the infrared camera being mounted on the camera and moving based on a control signal; A first processor for controlling the image capturing of the image camera; And a second processor for generating the control signal, wherein the second processor generates a marker image from the infrared light transmitted through the infrared camera, and transmits the marker image to the first processor, Wherein the first processor generates a control signal for controlling the motion of the main body on the basis of the marker position information, and the first processor receives the marker image extracted from the marker image, To calculate the marker position information.

According to the present invention, by capturing an image using an infrared camera and an infrared marker, it is possible to easily take an image of a subject alone by using the position of the subject and the facial recognition of the subject, which are subjects to generate image contents without auxiliary shooting force.

In addition, it is possible to shorten the processing time of the image image data analysis for obtaining the position coordinates of the subject by processing the tracking images of the subject in parallel without adding a high-capacity memory and a high-performance CPU.

In addition, it is possible to improve the speed of image analysis for detecting the position detection recognition rate of the subject and automatic tracking of the infrared marker by solving the problem of the object recognition error due to the infrared marker and the interference object through the face recognition of the subject.

1 is an illustration of an image capturing system according to an embodiment of the present invention.
2 is a structural diagram of a marker recognition apparatus according to an embodiment of the present invention.
3 is a structural view of a video photographing apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating an image processing method according to the first embodiment of the present invention.
5 is an exemplary diagram for comparing processing time according to parallel processing and single processing according to an embodiment of the present invention.
6 is a flowchart illustrating a method of distinguishing an infrared marker and an interference object using facial recognition according to an embodiment of the present invention.
FIG. 7 is an exemplary view of a subject position search using facial recognition according to an embodiment of the present invention.
8 is a flowchart illustrating an image processing method according to a second embodiment of the present invention.
9 is an exemplary view of a marker image according to an embodiment of the present invention.
10 is an exemplary diagram of a marker recognition apparatus according to an embodiment of the present invention.
FIG. 11 is an exemplary view of an additional installation of the image capturing system according to the embodiment of the present invention.
12 is a structural view of an infrared marker according to an embodiment of the present invention.
13 is an illustration of an infrared marker according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

Hereinafter, an image capturing system and method using an infrared camera and an infrared marker according to an embodiment of the present invention will be described with reference to the drawings.

1 is an illustration of an image capturing system according to an embodiment of the present invention.

As shown in FIG. 1, the image capturing system 10 recognizes the face of a subject together with the infrared marker 300 held by the subject, thereby tracking the position of the subject and collecting images. The image capturing system 10 includes a marker recognition apparatus 100 and a photographing apparatus 200. The marker recognition apparatus 100 and the image capturing apparatus 200 are implemented in a form in which separate components are combined.

In the embodiment of the present invention, a smart phone or a small camera (for example, GoPro) is used as the image capturing device 200 for recognizing the face of the subject and analyzing the position of the infrared marker, But are not limited to.

The marker recognition apparatus 100 is implemented in a cradle shape for mounting the image capturing apparatus 200 and includes a motor for moving the image capturing apparatus 200 up, down, left, and right. Although the marker recognition apparatus 100 is provided with an infrared camera capable of recognizing the infrared marker 300 held by the subject, the infrared camera is physically separated from the marker recognition apparatus 100 And the present invention is not necessarily limited thereto.

Each of the marker recognition apparatus 100 and the image capturing apparatus 200 includes at least one processor, at least one memory, and a communication interface. The image capturing apparatus 200 further includes a camera. At least one of the marker recognition apparatus 100 and the image capturing apparatus 200 includes an infrared camera (infrared marker recognition camera). The marker recognition apparatus 100 and the image capturing apparatus 200 exchange data through a communication interface. The marker recognition apparatus 100 and the image capturing apparatus 200 can process images in parallel through respective processors.

In addition, the marker recognition apparatus 100 can recognize the position of the infrared marker and control the motor provided at one side of the marker recognition apparatus 100 to control the image capturing apparatus 200 to move. Or the marker image of the infrared marker recognized by the marker recognition apparatus 100 is processed by the image capturing apparatus 200 to recognize the position of the marker and the image capturing apparatus 200 is provided on one side of the marker recognition apparatus 100 It is also possible to transmit control commands to the marker recognition apparatus 100 after all the control values for controlling the motor having the predetermined value are determined.

The application or software of the present invention includes instructions for performing the operations described with reference to the drawings in the present invention. The memory stores instructions for executing the application of the present invention, or temporarily stores instructions by loading the instructions from the storage device. The processor executes the instructions of the present invention by executing instructions stored in memory or loaded.

In the image capturing system 10 having such a structure, the structure of the marker recognition apparatus 100 and the structure of the image capturing apparatus 200 will be described with reference to FIGS. 2 and 3. FIG.

FIG. 2 is a structural view of a marker recognition apparatus according to an embodiment of the present invention, and FIG. 3 is a structural view of a photographing apparatus according to an embodiment of the present invention.

2, the marker recognition apparatus 100 includes an infrared camera 110, a processor 120, a communication interface 130, a memory 140, and a motor 150. [

The infrared camera 110 transmits the infrared light emitted from the infrared marker 300 held by the subject to acquire the marker image. For this purpose, the infrared camera 110 is implemented by a camera module having an infrared bandpass filter (IR bandpass filter) for transmitting infrared light, and the present invention is not limited thereto. Here, the size of the marker image is 4 bits.

In the embodiment of the present invention, in order to solve the infrared interference problem according to the surrounding environment of the subject, the infrared marker 300 transmits a pulse signal with a predetermined interval, and the infrared camera 110 interrupts the infrared marker according to the pulse signal And the position of the infrared marker is identified by distinguishing it from the light. Since the infrared camera 110 transmits the infrared light emitted from the infrared marker 300 and collects it as a marker image, it can be collected by various methods. Therefore, the present invention is not limited to any one method .

The processor 120 transmits the 4-bit marker image obtained by the infrared camera 110 directly to the image capturing apparatus 200 according to the embodiment of the present invention, so that the position of the infrared marker and the motor control value in the image capturing apparatus 200 . Alternatively, the marker image may be generated as a compressed marker image and transmitted to the image capturing apparatus 200, and the image processing may be processed in parallel by the infrared camera 110 and the image capturing apparatus 200. [ Alternatively, the processor 120 may be used to confirm the position of the infrared marker by receiving the photographed image photographed by the image photographing apparatus 200 without transmitting the marker image or the compressed marker image to the image photographing apparatus 200 .

In order for the processor 120 to generate a compressed marker image for parallel processing and transmit it to the image capturing apparatus 200, the processor 120 first converts the 4-bit marker image into a 2-bit black and white marker image. A method of converting a 4-bit marker image into a 2-bit black / white marker image can be performed through various methods, and thus a detailed description thereof will be omitted in the embodiment of the present invention.

In addition, the processor 120 compresses the converted 2-bit black and white marker image to generate a compressed marker image. The generated compressed marker image is transmitted to the image capturing apparatus 200 through the communication interface 130. Here, since the processor 120 can compress the black and white marker image through various methods, it is not limited to any one compression method in the embodiment of the present invention.

The processor recognizes that the communication interface 130 included in each of the marker recognition apparatus 100 and the image capturing apparatus 200 is connected via USB in order for the processor 120 to transmit the compressed marker image to the image capturing apparatus 200 . However, it is not necessarily limited thereto.

In addition, the processor 120 receives the infrared marker position from the image capturing apparatus 200. [ Here, the infrared marker position is received from the image capturing apparatus 200 via the communication interface 130 to the marker position calculated by analyzing the marker image by the image capturing apparatus 200. The location of the infrared marker includes the location of the x, y, and z coordinates of the infrared marker. The processor 120 calculates the control value of the motor for rotating the image sensing apparatus 200 based on the received infrared marker position. Here, in the embodiment of the present invention, the processor 120 may calculate the motor control value, and the image capturing apparatus 200 may calculate the motor control value and transmit it to the marker recognition apparatus 100. [

The image capturing apparatus 200 is connected to the marker recognizing apparatus 100 through a USB and is mounted on the marker recognizing apparatus 100. The image capturing apparatus 200 is mounted on the marker recognizing apparatus 100, A motor is provided on one side of the marker recognition apparatus 100 so that the marker recognition apparatus 100 can move.

The processor 120 calculates a first motor control value for moving the imaging device 200 up and down and a second motor control value for moving the imaging device 200 to the left and right. The method of calculating the motor control value based on the infrared marker position can be calculated in various ways. However, in the embodiment of the present invention, for example, motor control values are calculated in the following manner.

First, it is assumed that the minimum value and the maximum value of each value of x and y, which are the coordinates of the infrared marker 300, are predetermined. For example, it is assumed that the x coordinate is 0? X? 80 and the y coordinate is 0? Y? 60, which is determined differently depending on the size of the image acquired from the infrared camera 110.

When the image capturing apparatus 200 looks at the infrared camera 110 so that the subject is positioned at the center of the image by motor control, the x-coordinate and y-coordinate become (40, 30). The method of determining whether the infrared camera 110 is located at the center can be confirmed by various methods.

If the infrared camera 110 deviates greatly from the center position to the left or right direction, the deviation from the center can be calculated through the equation D = abs (w). Here, w = 40-x is calculated. If w is positive, it can be determined that the subject has moved to the left, and if w is negative, it can be determined that the subject has moved to the right.

Therefore, the direction and velocity values of the motor 150 can be determined and controlled through the PID calculation based on the D value and the w value. When the image capturing apparatus 200 moves in the subject movement direction under the control of a motor that moves the image capturing apparatus 200 moving left and right, the infrared marker 300 is located in the center of the infrared camera 110 again. Thus, the w value is calculated to be close to zero, and the D value is also zero.

When the subject is located at the center of the infrared camera 110, which is the image capturing apparatus 200, the motor stops operating. The motor control value for controlling the second motor moving up and down can be calculated in the same manner. Here, the method of changing the movement speed of the motor according to the calculated values is already known, and a detailed description thereof will be omitted in the embodiment of the present invention.

For example, if the central coordinate is (40, 30), it is assumed that coordinates (20, 30) for the subject are input. Then, the w value is calculated as 40-20 and 20, which indicates that the subject has moved to the left since the w value is positive. The method of calculating the abs 20 is already known, and controls the motor 150 to move the image pickup device 200 to the left until the D value 20 becomes zero.

The motor 150 moves the image capturing apparatus 200 by a control signal based on a motor control value calculated by the processor 120 according to a PID (Proportional Integral Derivative) control technique.

Here, the motor 150 may be a step motor (not shown) so as to move the image photographing apparatus 200 up and down and left and right by rotating the image photographing apparatus 200 together with the drive shaft in the motor at a preset angle or step As an example. A tilt step motor (hereinafter referred to as a "first motor") capable of moving the image capturing apparatus 200 in the first direction (up or down or vertical) (Hereinafter referred to as " second motor ") capable of moving in a horizontal direction.

Based on the first motor control value and the second motor control value calculated by the processor 120 or the first motor control value and the second motor control value calculated in the image pickup apparatus 200, Controls the first motor to move the apparatus 200 up and down, or controls the second motor to move the image photographing apparatus 200 to the left and right.

In the present invention, it is assumed that the two motors (first motor and second motor) are separately implemented in the marker recognition device 100, but they are replaced with one motor capable of performing both functions of the two motors It is possible.

3, the image capturing apparatus 200 includes an image camera 210, a processor 220, a communication interface 230, and a memory 240.

The image camera 210 collects the photographed image at the current position of the image photographing apparatus 200 and transmits the collected photographed image to the processor 220. In this case, the image camera 210 is a camera provided in a smart phone or a small camera.

Also, the image camera 210 collects the photographed image of the subject at the changed position of the image photographing apparatus 200 under the control of the marker recognizing apparatus 100. The photographed image of the photographed subject can be stored in the memory 240.

The processor 220 decodes the compressed marker image transmitted from the infrared camera 110 through the communication interface 230, and generates a 2-bit black and white marker image. Since the processor 220 can decode the compressed marker image through various methods, detailed description thereof will be omitted in the embodiment of the present invention.

The processor 220 analyzes the pixel data of the monochrome marker image. Then, pixel data values determined to be located in the infrared marker 300 are extracted from the analyzed pixel data. Here, the analysis of the pixel data value in the black and white marker image can be analyzed through various methods, so that the detailed description will be omitted in the embodiment of the present invention.

At this time, the processor 220 may extract a plurality of pixel data values in the process of extracting the pixel data values including the infrared marker 300 from the monochrome marker image through analysis of the pixel data of the monochrome marker image. When there is an object (hereinafter, referred to as an 'interfering object') that emits or reflects light having the same wavelength as that of the infrared marker of the infrared marker, the infrared marker The pixel data of the interference object is also extracted.

The processor 220 calculates the position of the infrared marker 300 held by the subject from at least one or more pixel data values. Since the position of the infrared marker includes the values of the x and y axes, the method of calculating the position of the infrared marker can be calculated through various methods, and thus the detailed description thereof will be omitted in the embodiment of the present invention. At this time, the width value of the marker image which is the z value can be included in the position of the infrared marker. Then, the calculated infrared marker position is transmitted to the marker recognition apparatus 100.

At this time, when there is one pixel data value checked by the processor 220, the position of the infrared marker is calculated by confirming that the infrared marker is located in the corresponding pixel data. However, when there are a plurality of received pixel data values, it is difficult to accurately determine to which pixel data the infrared marker is located.

Accordingly, the processor 220 calculates the position of the infrared marker of the infrared marker 300 having the actual subject based on the facial image in the captured image collected by the image camera 210. [ To this end, the processor 220 assigns a facial identification number to the face of the subjects in the received facial image. Then, the size and position of the face with the identification number are confirmed, and the search area is designated based on the confirmed face position.

In addition, the processor 220 generates a divided screen by dividing the designated search area into a predetermined number and confirms that the infrared marker 300 exists on any one of the generated divided screens, the infrared marker 300 ) Of the marker image. Then, the position of the infrared marker is calculated based on the confirmed marker image. Hereinafter, a method of calculating the position of the infrared marker through facial recognition will be described in detail later.

Further, the processor 220 may calculate the first motor control value and the second motor control value for controlling the motor in the marker recognition apparatus 100 based on the calculated marker position after calculating the marker position. The first and second motor control values thus calculated are transmitted to the marker recognition apparatus 100 through the communication interface 230 so that the marker recognition apparatus 100 can control the motor. To this end, the processor 220 calculates a first motor control value and a second motor control value to control the motor based on the currently calculated marker position.

The memory 240 stores and manages the 2-bit monochrome marker image generated by the image decoding unit 220. In the embodiment of the present invention, the black-and-white marker image is stored in the memory 240 for a predetermined period of time in consideration of the memory size of the smart phone or the small camera, and then deleted. However, the present invention is not limited thereto.

In addition, the memory 240 receives the photographed image from the image camera 210 and stores the photographed image. In addition, the memory 240 stores instructions for executing the application of the present invention, or temporarily stores instructions by loading the instructions from the storage device.

A method of capturing and processing an image after detecting the position of a subject in the environment in which the image capturing system 10 described above is applied will be described with reference to FIG.

4 is a flowchart illustrating an image processing method according to the first embodiment of the present invention.

In the first embodiment, the marker recognition apparatus 100 and the image capturing apparatus 200 process images by parallel processing, respectively, or the marker recognition apparatus 100 receives the position information of the infrared marker from the image capturing apparatus 200 And calculates the motor control value based on the calculated value to control the motor.

4, the infrared camera 110 of the marker recognition apparatus 100 transmits infrared light emitted by an infrared marker 300 possessed by a subject and collects marker images of the infrared marker 300 (S100). The marker image has a size of 4 bits as an example, but it is not necessarily limited thereto.

The processor 120 converts the 4-bit marker image collected by the infrared camera 110 into a 2-bit black and white marker image in step S100 (S200), and compresses the converted black and white marker image (S300). The method of converting a marker image into a black / white marker image or compressing a black / white marker image can be performed through various methods, and therefore the present invention is not limited to any one method.

The compressed marker image generated in step S300 is transmitted to the image capturing apparatus 200 (S400). For this, the marker recognition apparatus 100 and the image capturing apparatus 200 are explained as being connected through the USB, which is the communication interface 130 or 230, but it is not necessarily limited thereto.

The image capturing apparatus 200 receiving the compressed marker image through step S400 decodes the compressed marker image to extract the black and white marker image, and temporarily stores the extracted black and white marker image (S500). Here, the method of decoding the compressed marker image can be performed by various methods. Therefore, the present invention is not limited to any one method.

The image capturing apparatus 200 analyzes the image pixel data from the monochrome marker image temporarily stored in step S500, and calculates the position of the infrared marker from the analyzed image pixel data (S600). Here, in addition to the infrared marker 300, when an interference object that emits or reflects light having the same wavelength as that of the infrared marker is located around the infrared marker, a plurality of image pixels analyzed as the infrared marker among the image pixel data are extracted.

Accordingly, in addition to the marker images collected by the marker recognition apparatus 100, the image capturing apparatus 200 recognizes the face of the subject based on the facial image in the captured image that is captured by the image capturing apparatus 200 itself, Thereby distinguishing the interference object. This will be described first with reference to FIG.

6 is a flowchart illustrating a method of distinguishing an infrared marker and an interference object using facial recognition according to an embodiment of the present invention.

As shown in FIG. 6, the image camera 210 recognizes all facial faces exposed from the current position toward the image camera 210 and acquires facial images (S601).

In step S601, the processor 220 checks the number of facial images N for all the subjects in the facial image acquired in step S601, and assigns the facial identification number n to the facial images included in the facial image. Since the processor 220 can check the face of the subject in the collected facial image and the facial identification number when the plurality of facial faces are in the facial image, It is not limited to one method.

The processor 220 designates a search area for confirming whether there is a marker image for the infrared marker on each of the face images of the subject based on the face image of the face to which the face identification number is assigned. Then, the designated search area is divided into J pieces to generate J divided pictures, and a divided picture identification number (j) is assigned to each divided picture.

It is assumed that the size of the search area is determined according to the distance between the image camera 210 and the subject, and the size of the search area to be initially designated is predetermined based on the face size of the subject. It is also assumed that the maximum size of the search area obtained based on the size of the subject's face is also determined. Here, the search area is specified to be below the face position with respect to the face of the subject. This is because the infrared marker 300 is usually always located under the face such as the torso or the arm of the subject.

Further, it is assumed that the search area is expanded according to a predetermined size every time the search area is expanded. A method for dividing a search area or a method for assigning a divided screen identification number to a divided search area can also be performed by various methods. Therefore, the present invention is not limited to any one method.

The processor 220 identifies the face of the subject to which the face identification number n is assigned among the N subject faces. Here, the processor 220 identifies sequentially from the face of the subject having the facial identification number 1 assigned to the N facial face.

Then, the processor 220 determines whether the face identification number is smaller than the face face number (S602). If the facial identification number for the currently determined facial image is smaller than the number for all the facial facial features appearing in the facial image, the size of the facial region with respect to the recognized facial image and the position of the facial region are confirmed (S603) . Here, the method of confirming the size and the face position of the facial region by the processor 220 can be performed through various methods, so that the detailed description will be omitted in the embodiment of the present invention.

Then, the processor 220 identifies the search area based on the facial position of the recognized n-th object. After the search area is confirmed, the search area is divided by a preset number, and the infrared marker image data is checked in the divided screen starting from the first divided screen among the plurality of divided screens (S604, S605).

In the embodiment of the present invention, a divided screen is generated by dividing a predetermined search area on the basis of the face position of the subject into J pieces, and each divided screen is divided into a plurality of divided screen regions, However, the present invention is not limited thereto. A method of checking the presence or absence of the marker image data in the divided screen can also be performed through various methods, so that detailed description will be omitted in the embodiment of the present invention.

If it is confirmed that the infrared marker image data is present in the divided screen to which the arbitrary divided screen identification number is assigned, the processor 220 analyzes the position of the detected infrared marker (S606). However, if it is confirmed that there is no infrared marker image data in the divided screen, the processor 220 checks whether all the divided areas, that is, all the search areas are searched based on the subject's face (S607).

If it is confirmed that there are divided screens which are not searched out among all the search regions divided into J segments, the processor 220 searches the neighboring divided screens of the current divided screen for the presence of the infrared marker image data, And the procedure after step S604 is performed on the divided screen having the (j + 1) identification number (S609).

However, if it is confirmed in step S607 that all the search areas having the facial identification number n have been found, the processor 220 performs the procedure after step S602 on the facial area having the facial identification number n + 1. To this end, the processor 220 adds 1 to the facial identification number n where the current facial image is located (S608).

On the other hand, if it is confirmed in step S602 that the analysis of all facial areas is completed, but there is no detected infrared marker image data, it means that there is no infrared marker image data in the search area for the facial face. Accordingly, the processor 220 expands the search area (S610). That is, after the search area is expanded to a predetermined size, the search area is re-determined.

In this case, since the maximum expansion size of the search area is predetermined in the embodiment of the present invention, it is checked in step S610 whether the size of the expanded search area corresponds to the maximum value (S611). If the size of the extended search area exceeds the maximum value, it is confirmed that there is no infrared image marker in the face region of the subject (S612). That is, it is confirmed that the subject does not have an infrared marker.

However, if it is confirmed that the size of the extended search area does not exceed the maximum value, the procedure after step S620 is performed. The above procedure will be described first with reference to Fig.

FIG. 7 is an exemplary view of a subject position search using facial recognition according to an embodiment of the present invention.

7 (a) shows a marker image generated by the processor 120 through the infrared camera 110 of the marker recognition apparatus 100, and FIG. 7 (b) shows a marker image generated by the image photographing apparatus 200 ), Respectively.

As shown in FIG. 7 (a), the infrared camera 110 transmits infrared light and recognizes that two infrared markers (1, 2) exist. The portion indicated by the first infrared marker (1) corresponds to the light emitted from the infrared marker 300 having the actual object, and the portion indicated by the second infrared marker (2) corresponds to the light caused by the interference object.

Accordingly, the image capturing apparatus 200 confirms the number of face images of the subject through the captured image. The search area (3) for the first object located close to the image photographing apparatus (200) is set to be larger than the search area (4) for the second object located far away from the image photographing apparatus (200). The search area is set based on the size of the face of the subject.

In the embodiment of the present invention, it has been shown that the search area is divided into fifteen segments to generate a split screen. Therefore, in order to check whether the first subject carries the infrared marker from the divided screen displayed at the number 1, it is checked whether the infrared marker image data is found. In FIG. 7 (b), since the infrared marker is not displayed on the divided screen displayed at No. 1, the divided screen displayed at No. 2 is confirmed. If you check the split screen in this way, you will see that there is an infrared marker on the 3 split screens.

On the other hand, since the face of the subject is not recognized around the portion indicated by the second infrared marker (2), it can be confirmed that the second infrared marker corresponds to the light caused by the interference object.

4, the infrared marker position is calculated in step S600, and the calculated infrared marker position is transmitted to the marker recognition apparatus 100 (S700). The processor 120 receiving the infrared marker position including the x coordinate and the y coordinate calculates the control value of the motor to rotate the image photographing apparatus 200 based on the infrared marker position.

The control value of the motor may be divided into a first motor control value for moving the image sensing apparatus 200 up and down and a second motor control value for moving the image sensing apparatus 200 to the left and right. When the processor 120 calculates the first motor control value and the second motor control value, the control signal generated based on the calculated motor control value is transmitted to the motor 150, ) Is moved up or down or left and right so as to capture an image of the subject (S800). Then, the image capturing apparatus 200 captures an image of the subject at a changed position under the control of the marker recognition apparatus 100 (S900).

As described above, when the position of the subject is confirmed by processing the images in parallel using the marker recognition apparatus 100 and the image capturing apparatus 200, the waiting time for image processing can be minimized. This will be described with reference to FIG.

5 is an exemplary diagram for comparing processing time according to parallel processing and single processing according to an embodiment of the present invention.

FIG. 5A is a diagram illustrating an overall processing time by a general single processing, and FIG. 5B is an example of total processing time by parallel processing according to an embodiment of the present invention.

As shown in (a), when the image frame is analyzed and the motor is controlled by a general single process, after the frame analysis and the motor control procedure for the first image frame are completed, the frame analysis for the second image frame, A motor control procedure is performed.

Meanwhile, as shown in (b), in the embodiment of the present invention, the marker recognition apparatus 100 and the image capturing apparatus 200 analyze image frames at the same time and process the motors according to the analyzed image frames in parallel. The time required for image processing is shortened.

That is, while the marker recognition apparatus 100 processes the A and B tasks to analyze the second image frame, the image capturing apparatus 200 simultaneously executes the D and E tasks for analyzing the first image frame at the same time . Therefore, it is possible to control the motor in real time through the image analysis of the position of the infrared marker of the subject.

On the other hand, the marker recognizing apparatus 100 recognizes the position of the subject according to the second embodiment in which only the infrared marker recognition and the motor control are performed and the image photographing apparatus 200 processes the image, The method will be described with reference to Fig.

8 is a flowchart illustrating an image processing method according to a second embodiment of the present invention.

In the second embodiment, the image capturing apparatus 200 calculates the motor control value, and the marker recognizing apparatus 100 recognizes only the infrared marker so that the image capturing apparatus 200 can process all the processing.

As shown in FIG. 8, the infrared camera 110 of the marker recognition apparatus 100 transmits infrared light emitted by an infrared marker 300 possessed by a subject and collects marker images of the infrared marker 300 (S100 '). The collected marker image is as shown in Fig. 9, and this will be described first.

9 is an exemplary view of a marker image according to an embodiment of the present invention.

As shown in FIG. 9, the marker recognition apparatus 100 transmits infrared light through a filter and collects marker images for the infrared marker 300. At this time, based on the marker image generated by the infrared camera 110, in the first embodiment, the image capturing apparatus 200 can check the coordinate information of the infrared marker, or the marker recognition apparatus 100 can check the coordinate information of the marker It is possible.

8, the processor 120 transmits the marker image collected by the infrared camera 110 to the image capturing apparatus 200 (S200 ') in step S100' And the image capturing apparatus 200 are connected to each other via the USB, which is the communication interface 130 or 230, but the present invention is not limited thereto.

The image collecting apparatus 200 receiving the marker image through step S200 'temporarily stores the received marker image (S300'). The image capturing apparatus 200 analyzes the image pixel data from the marker image temporarily stored in step S300 ', and calculates the position of the infrared marker from the analyzed image pixel data (S400'). Here, in addition to the infrared marker 300, when an interference object that emits or reflects light of the same wavelength as the infrared marker is located in the vicinity of the infrared marker, a plurality of image pixels analyzed as having the infrared marker among the image pixel data are extracted.

Accordingly, in addition to the marker images collected by the marker recognition apparatus 100, the image capturing apparatus 200 recognizes the face of the subject based on the facial image in the captured image that is captured by the image capturing apparatus 200 itself, Thereby distinguishing the interference object. This is as described in FIG.

When the position of the infrared marker is calculated through step S400 ', the calculated infrared marker position is compared with the previously calculated position information of the same infrared marker to calculate the control value of the motor to rotate the image photographing apparatus 200 ( S500 '). To this end, the image capturing apparatus 200 temporarily stores all the position information on the calculated infrared marker.

The control value of the motor may be divided into a first motor control value for moving the image sensing apparatus 200 up and down and a second motor control value for moving the image sensing apparatus 200 to the left and right. When the processor 220 calculates the first motor control value and the second motor control value, the calculated first motor control value and the second motor control value are transmitted to the marker recognition apparatus 100 (S600 ').

The motor 150 drives the image capturing apparatus 200 up and down based on the control signal generated based on the first motor control value and the second motor control value received from the image capturing apparatus 200 by the processor 120 through S700 ' (S700 '), and the image capturing apparatus 200 captures an image of the subject at the moved position (S800').

On the other hand, an embodiment of the marker recognition apparatus 100 for recognizing a marker when a subject captures an image through the image capturing system 10 and an additional fixture for automatically tracking a subject to help capture various images 10 and Fig. 11.

10 is an exemplary diagram of a marker recognition apparatus according to an embodiment of the present invention.

10 (a) to 10 (c), the marker recognition apparatus 100 may be implemented in various forms. As shown in (a), the marker recognition apparatus 100 is implemented in a cradle form so that the image capturing apparatus 200 can be mounted on the upper end of the marker recognition apparatus 100.

FIG. 11 is an exemplary view of an additional installation of the image capturing system according to the embodiment of the present invention.

11, the image capturing system 10 is coupled to the supplementary fixtures 400 according to the embodiment of the present invention to automatically track a subject together with an operation preset by the subject, You can shoot the subject alone, or you can help focus the subject.

The supplementary installation 400 may be implemented in various forms such as a sliding mechanism or a jig mechanism (e.g., circular, C-shaped, vertical, or horizontal) so that the image capturing system 10 can move left or right or up and down . The marker recognition device 100 of the imaging system 10 coupled to the supplementary fixture 400 calculates the movement of the infrared marker based on the infrared marker position. Then, the subject is automatically tracked based on the calculated movement of the infrared marker, so that the user can photograph the subject by moving left and right or up and down on the supplementary fixture 400.

Next, the structure of the infrared marker 300 for allowing the image capturing system 10 to confirm the position of the subject will be described with reference to FIG. 12, and an example of the infrared marker 300 will be described with reference to FIG. 13 Explain.

FIG. 12 is a structural view of an infrared marker according to an embodiment of the present invention, and FIG. 13 is an exemplary view of an infrared marker according to an embodiment of the present invention.

12, the infrared marker 300 includes an LED unit 330 including a power supply unit 310, a timer 320, and a plurality of infrared LEDs 330-1 to 330-4.

The power supply unit 310 supplies power to the timer 320. In the embodiment of the present invention, a power supply unit is implemented in various ways such as a 2-cell 3.7-volt battery, USB, or external charging.

The timer 320 is connected to a plurality of infrared LEDs 330-1 to 330-4 so as to generate and provide a pulse signal having a preset interval so that the infrared LEDs are activated / deactivated.

The LED unit 330 includes a plurality of infrared LEDs 330-1 to 330-4. In the embodiment of the present invention, four infrared LEDs 330-1 to 330-4 However, the present invention is not necessarily limited thereto. Then, the LED unit 330 is activated to emit infrared light based on the pulse signal generated by the timer 320, or the infrared LEDs 330-1 to 330-4 are switched to the inactive state It is. The infrared LEDs 330-1 to 330-4 may be individually switched according to the pulse signal, or all the infrared LEDs 330-1 to 330-4 may be simultaneously switched.

The infrared marker 300 may be embodied in various forms such as a band shape or a necklace shape as shown in FIG. 13 (a) so that the infrared marker 300 can be worn at various body positions of a subject. The infrared marker 300 can be worn at various body positions such as the ankles and arms of the subject as shown in FIGS. 13 (b) and 13 (c), and as shown in (d) Light is emitted.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (19)

A method of capturing an image of a subject by a capturing system,
Collecting a marker image of the infrared marker;
The infrared marker and the interference object are distinguished from each other based on the position of the infrared marker collected from the image pixel data corresponding to the marker image and the position information of the infrared marker confirmed in the marker image, Calculating a control value for controlling movement of the motor on which the image photographing apparatus is mounted based on the control value; And
Transmitting the control value to the motor
Lt; / RTI >
Wherein the infrared marker is attached to the subject, and the image photographing apparatus photographs the subject. The method according to claim 1,
Wherein the collecting of the marker image comprises:
Transmitting the infrared light emitted by the infrared marker to collect the marker image;
Converting the marker image into a black and white marker image; And
A step of generating a compressed marker image by compressing the converted black and white marker image
And an image capturing unit for capturing an image of the subject. 3. The method of claim 2,
Transmitting the compressed marker image to the image capturing apparatus; And
Receiving the marker position from the image photographing apparatus
Further comprising:
Wherein the marker position is calculated based on the compressed marker image. The method of claim 3,
After the step of receiving from the image photographing apparatus,
Wherein the position of the infrared marker further includes a first motor control value and a second motor control value calculated by the image pickup device,
Controlling the motor based on the first motor control value and the second motor control value
And an image capturing unit for capturing an image of the subject. The method according to claim 1,
Wherein the step of calculating the control value comprises:
Calculating a first motor control value for moving the imaging device in a first direction based on the marker position;
Calculating a second motor control value for moving the imaging device in a second direction; And
Controlling the motor to move the imaging device in at least one of the first direction or the second direction based on the calculated first motor control value and the second motor control value
And an image capturing unit for capturing an image of the subject. An application that is loaded on a terminal and executes a programmed operation in combination with hardware including a processor, a memory, and a camera of the terminal,
Activating the camera to capture an image;
Receiving a marker image of an infrared marker from the infrared marker recognition device;
Calculating a marker position in a plurality of search regions for identifying the position of the infrared marker based on the number of subject faces for all subjects included in the marker image and the position of the face of the subject; And
Transmitting the marker position to the infrared marker recognizing device
Instructions that are programmed to perform the steps of:
Wherein the infrared marker is attached to a subject and the terminal is mounted on a motor whose movement is controlled by the infrared marker recognition device and the infrared marker recognition device controls the motor based on the marker position, An application that verifies that a marker exists in one of the plurality of search areas when the position of the marker calculated in the search area coincides with the position of the marker collected in the image. The method according to claim 6,
Wherein the step of calculating the marker position comprises:
Analyzing pixel data of the marker image and extracting a pixel data value for the infrared marker; And
Calculating a position of the infrared marker based on the extracted pixel data value
. 8. The method of claim 7,
Wherein calculating the position of the infrared marker comprises:
Collecting a face image when two or more pixel data values including a pixel data value corresponding to the infrared marker are extracted;
Confirming the number of facial faces of all the subjects included in the collected facial images, and assigning facial identification numbers to the facial images of the subject; And
Confirming presence or absence of an infrared marker on each of the face images of the subject
. 9. The method of claim 8,
Wherein the step of checking presence or absence of the infrared marker comprises:
Confirming the size and position of the face region of the selected subject face if the facial identification number given to the subject face is smaller than the number of the face faces for all the subjects;
Designating a search area to detect image data corresponding to the infrared marker based on the position of the determined subject's face;
Dividing a designated search area by a preset number to generate a plurality of divided screens;
Confirming whether any one of the plurality of divided screens includes image data for the infrared marker; And
Analyzing the position of the infrared marker if the image data for the infrared marker is included in one of the divided screens
. 10. The method of claim 9,
If no image data for the infrared marker is included in the plurality of divided screens,
Increasing one of the face identification numbers assigned to the subject's face;
Expanding the search area if one of the facial identification numbers is larger than the number of facial facial numbers; And
If the size of the extended search area is equal to the size of the preset maximum search area, confirming that there is no infrared marker in the search area
. The method according to claim 6,
Wherein the step of transmitting to the infrared marker recognition step comprises:
And transmits the motor control value calculated based on the marker position to the infrared marker recognizing device. A system for capturing an image of a subject having an infrared marker, the system comprising:
A main body for mounting the image photographing apparatus;
At least one motor for controlling movement of the main body;
An infrared camera recognizing an infrared marker;
A memory for storing a program; And
A processor for executing the program in cooperation with the motor, the infrared camera, and the memory,
Lt; / RTI >
The processor comprising:
And a controller for checking whether the object recognized by the infrared camera is the infrared marker or the interference object based on the marker position information extracted based on the marker image, And calculates a motor control value for controlling the motor based on the position information. 13. The method of claim 12,
The processor comprising:
Converting the marker image into a first image, and compressing the converted first image to generate a compressed image. 13. The method of claim 12,
The motor includes:
And moving the main body in at least one direction based on a motor control value received from the processor. 13. The method of claim 12,
The processor comprising:
And receives the marker position information from the image capturing apparatus. 13. The method of claim 12,
Wherein the infrared marker comprises:
An LED unit that is activated to emit infrared light based on a pulse signal to which a plurality of infrared LEDs are input, or is switched from an active state to an inactive state;
A timer for generating and supplying a pulse signal having a preset interval to a plurality of infrared LEDs included in the LED unit; And
A power supply for supplying power to the timer;
And an image capturing system. A system for capturing an image of a subject having an infrared marker, the system comprising:
A video camera for capturing an image;
A main body including an infrared camera, the infrared camera being mounted on the camera and moving based on a control signal;
A first processor for controlling the image capturing of the image camera; And
And a second processor
Lt; / RTI >
The second processor generates a marker image from the infrared light transmitted through the infrared camera, transmits the marker image to the first processor, receives marker position information extracted from the marker image from the first processor, Generating a control signal for controlling the movement of the main body based on the marker position information,
The first processor extracts at least one or more pixel data values determined to be positioned by the infrared marker based on the marker image received from the second processor and generates position information of the infrared marker based on the extracted pixel data value And an infrared marker and an interference object based on the at least one pixel data value and a facial image captured by the photographing camera to calculate a marker position of the infrared marker.
delete 18. The method of claim 17,
Wherein the first processor comprises:
A search area for each of all the objects is specified based on the size and position of the face of all the subjects included in the face image, and the image for identifying the position information of the infrared marker from the divided screen generated by dividing the designated search area Shooting system.

Images